splithorizon
cloud networking

NICs, next-hop routing & secondary IPs

Every cloud lets you attach more than one NIC to a VM, route traffic to a NIC instead of a gateway, and stack extra addresses on top of all that. The consoles make all three look like the same checkbox. They aren't — and the gaps show up exactly when a design gets copied from the wrong cloud. This reference walks topology first, then routing, then address density, then the guest-OS mechanics nobody puts in the architecture diagram.

01topology

Multiple NICs on one VM

All three clouds support attaching more than one NIC to a VM. The part that actually matters splits into two separate questions — can those NICs share a network, and can they live in different networks — and each cloud only answers "yes" to one of them, never both the way AWS does.

AWS
both directions
ENIs can share a VPC, even the same subnet — instance-type-dependent up to 15+. They can also span two completely different VPCs: multi-VPC ENI attachment has been GA since late 2023, gated only by same-AZ and same-account. This is the documented pattern for centralized appliances and databases that straddle otherwise-segregated VPCs.
Azure
same VNet only
VM-size-dependent up to 8 NICs, different subnets fine — but Microsoft's own guidance is explicit: every NIC on a VM must live in the same VNet. There's no cross-VNet NIC attachment at all. Need a VM logically present in two networks? That's a peering-and-routing problem, not a NIC-count problem.
GCP
different VPC only
Up to 8 NICs, machine-type-dependent — but each one must sit in a distinct VPC network, not just a different subnet. Same-VPC multi-NIC isn't a smaller version of the feature, it's simply not supported. The native use case is an NVA dual-homed across an "untrust" and a "trust" VPC.
aws VM VPC A VPC B same VPC: yes diff VPC: yes (same AZ) azure VM VNet (subnet A) other VNet ✕ same VNet: yes diff VNet: no gcp VM same VPC ✕ VPC B same VPC: no diff VPC: yes (only mode)

AWS is the only cloud that does both. Azure locks you to one VNet per VM. GCP locks you to one VPC per NIC. A topology that works as drawn on AWS may simply not be buildable on either of the other two — and the fix depends on which constraint you hit. On GCP, a same-VPC redundancy design becomes either alias IP ranges (more addresses, same NIC) or a genuine multi-VPC topology (real isolation). On Azure, a design that wants one VM physically dual-homed across two VNets becomes peering plus routing instead.

The gotcha all three share regardless of topology: attaching a second NIC doesn't solve in-guest routing for you. The guest OS still has exactly one default route unless policy-based routing is configured — ip rule plus a per-table ip route on Linux. Traffic from a secondary NIC's subnet will try to egress via the primary's gateway and either get dropped or come back asymmetric. This is the single most common multi-NIC failure mode on every cloud here, AWS's cross-VPC case included.

02routing

Next-hop routing to a NIC

This is the standard NVA/firewall-insertion pattern: a route table entry points at a NIC instead of an internet gateway, NAT, or peering target. Same-VPC appliance insertion works everywhere, but the mechanics — and what happens once you try to cross a VPC boundary — diverge sharply.

capabilityawsazuregcp
same-vpc next-hopyesyesyes
forwarding togglesrc/dst check (per ENI)IP forwarding (per NIC)--can-ip-forward (immutable)
native ecmp to nvasnonoyes
direct cross-vpc next-hopnoyes (peered)no
ha lb-fronted nva patterngwlbstandard lb + ha portsilb as next hop
dynamic (bgp) route injectiontgw connectroute servercloud router

On AWS, the implicit local VPC route always wins over a more general route to an appliance, so you need more-specific prefixes to actually pull traffic toward it — and there's no native ECMP across multiple ENIs in one route entry, which is why production NVA insertion almost always means Gateway Load Balancer rather than a raw ENI next-hop. GCP flips that limitation: multiple static routes with the same prefix and priority, pointed at different next-hop instances, get genuine load distribution without AWS's single-target ceiling.

Azure is the asymmetric one here. A UDR's next-hop-IP can reference an NVA's private address sitting in a genuinely different, peered VNet — directly, given the right peering settings and no overlapping address space. That's exactly how hub-spoke NVA insertion commonly works: spokes peer to a hub, and UDRs on each spoke point straight at the firewall's IP in that hub. AWS and GCP both refuse this outright — a route table target must live in the same VPC, full stop — and force a dedicated transit construct instead: Transit Gateway plus GWLB endpoints on AWS, Network Connectivity Center on GCP. The catch on Azure's side: peering is non-transitive by default, so spoke-to-spoke traffic through the hub NVA still needs explicit UDRs on every spoke, or Route Server with branch-to-branch enabled to propagate routes dynamically instead.

03density

Secondary IPs on a single NIC

Once topology and routing are settled, the next layer is how many addresses can live on one NIC — and here the three clouds aren't just running different limits, they're built on genuinely different primitives. AWS and Azure both assign individual addresses; GCP assigns routable CIDR blocks from day one.

AWS
/28
prefix delegation block
Base limit is instance-type-dependent secondary addresses, one by one. Prefix delegation adds a /28 IPv4 or /80 IPv6 block option so you claim a contiguous range instead of consuming address slots individually — the same mechanism the VPC CNI uses to scale pod density on EKS, where each prefix counts as a single "slot" against the per-instance ceiling.
Azure
256
ip configs per NIC
A flat ceiling, expressed officially as 256 × NIC count per VM. Every IP — primary or secondary — is wrapped in its own "IP configuration" object with independent NSG inheritance and an optional attached public IP. There's no native block-allocation mechanism; it's individually-addressed the whole way down.
GCP
150
alias ranges per NIC
CIDR blocks, not addresses — the count is independent of netmask size, so a single /24 and a single /23 both count as one against the limit. Sourced from the subnet's primary range or an explicit secondary range. This is the exact mechanism behind VPC-native GKE: pod IPs come from a subnet secondary range via alias IPs, one range per node.

Worth the gut check before reaching for any of these: does the IP need to move independently of the instance, or attach to a workload rather than a host? If yes, it's a legitimate secondary or alias IP case — container density, NVA multi-tenancy, floating failover IPs. If the real need is just more capacity or higher availability, that's a load balancer or autoscaling group problem, and stacking secondary IPs on one instance only centralizes the risk.

04the part runbooks skip

What the API call doesn't do

The cloud control plane assigning a secondary IP is only half the job. Bringing the address up inside the guest and getting an application to actually bind to it are entirely separate steps that no cloud automates end-to-end — this is where most "I assigned the IP but it doesn't work" tickets actually come from.

1

cloud api assigns

A control-plane-only operation. The address now exists on the interface as far as AWS, Azure, or GCP is concerned — but it hasn't touched the instance's operating system in any way.

2

guest os configures

Manual fallback is ip addr add (or ip route add for a GCP alias range, since it's a block not an address). How automatic this is depends entirely on the image: Amazon Linux ships ec2-net-utils, Azure relies on WALinuxAgent or cloud-init, GCP needs google-guest-agent running and the alias set via the API.

3

app binds inbound

Wildcard bind (0.0.0.0) picks up every IP on the host for free — most simple services land here automatically. Explicit bind, like nginx's per-IP listen directive for multi-tenant SSL certs, needs deliberate config.

4

egress sources correctly

Inbound is the easy direction. Outbound traffic defaults to the primary IP unless something forces the source — policy routing (ip rule plus a per-table ip route) is the general-purpose fix when the app itself doesn't expose a bind-before-connect option.

identical across
all 3 clouds
most common failure
step 2 or 4
solved by
guest OS, not API

The mental model worth keeping: a cloud API call means "this IP is now permitted to exist on this interface." It does nothing to the guest OS and nothing to the application. The OS seeing the IP, the app listening on it, and outbound traffic sourcing from it are three separate failure points stacked on top of the cloud-side assignment — and unlike the topology and routing differences above, this part is exactly the same headache on every cloud.

throughline

Where the consoles lie to you

Four layers, four different shapes of divergence — none of them visible until a design built for one cloud gets ported to another.

nics
aws does both · azure same-net · gcp cross-net
routing
azure crosses peering · aws/gcp need transit
density
addresses (aws/azure) vs. blocks (gcp)