all writing cloud networking

What actually happens when you peer two VPCs

The console drew me a clean little line between two VPCs and called it a peering connection. One hop, problem solved. Except the packet capture told a different story — and the packet capture doesn’t lie, even when you want it to.

Here’s the thing nobody tells you when you click “create peering connection”: you haven’t built a link. You’ve published a set of routes into a table, and a forwarding decision now happens at a place you can’t see from the console.

The mental model that fixes 80% of cloud networking bugs: there are no wires, only routing tables. Everything else is an abstraction over that one fact.

Tracing the actual path

So I traced it. Source instance, destination instance, and every routing decision in between. The gateway you provisioned is doing far more bookkeeping than the topology diagram suggests:

$ traceroute 10.1.4.7
 1  vpc-a-rt        0.41 ms   # local route table
 2  tgw-attach-a    1.10 ms   # handoff to transit gw
 3  tgw-route-tbl   1.12 ms   # longest-prefix match
 4  tgw-attach-b    1.94 ms   # handoff to vpc-b
 5  10.1.4.7        2.31 ms   # arrived

Five hops, not one. And the latency you care about is almost entirely in steps 2 through 4 — the part the diagram drew as a single straight line.

What this means in practice

Once you internalize that the gateway is a routing table with attachments, a few things stop being mysterious:

I’ll go deeper on the encapsulation tax in the next piece. For now: stop thinking in wires. Start thinking in tables, and the cloud network gets a lot more predictable.

Written by the 2× CCIE

Enterprise → cloud → AI networking. I write the breakdowns I wish I’d had. New field notes roughly twice a month.

keep reading

More writing