AI / HPC Fabric deep-dive

RoCEv2 vs InfiniBand

Both carry RDMA semantics. The delta is everything else — transport guarantees, congestion model, fabric scope, and operational posture.

RoCEv2
RDMA over Converged Ethernet v2 — RFC 7910
Transport
IB Transport → UDP/IP — GRH replaced by outer IP/UDP header
Reliability
Unreliable Datagram (UD) or Reliable Connected (RC) via QP state machines — no native fabric-layer ACK
Congestion control
PFC (802.1Qbb) + DCQCN — ECN-based rate reduction at sender. PFC pause frames create head-of-line blocking risk
Latency (typical)
1–3 µs one-way (NIC-to-NIC, same rack)
Bandwidth
100/200/400 GbE, roadmap to 800G
Addressing
GID (IPv6-mapped) — routable across L3 boundaries. Subnet-agnostic
Ethernet IP routable ECMP PFC required
InfiniBand
Native lossless fabric — IBTA spec
Transport
Native IB stack — link, network, transport are a single vertically integrated protocol. No IP/UDP encap overhead
Reliability
Link-layer credit-based flow control — losslessness is a fabric guarantee, not a QoS workaround. Fabric never drops
Congestion control
IB Congestion Control (IBCC) — CNPs injected at receiver, FECN marking in data path. No PFC, no head-of-line risk
Latency (typical)
<600 ns one-way (HDR/NDR, HCA-to-HCA)
Bandwidth
NDR 400 Gb/s per port; XDR 800 Gb/s in roadmap (NVIDIA)
Addressing
LID (local) + GID (global) — LIDs are subnet-scoped, 16-bit flat space, managed by SM
Native fabric Credit-based Subnet Manager Lossless native
Protocol stack comparison
RoCEv2 stack
Verbs / uverbslibibverbs, ibv_post_send()
RDMA CM / IWARPrdma_cm — connection setup
IB Transport (RC/UD)BTH, RETH, AETH opcodes
UDP port 4791demux + stateless routing
IPv4 / IPv6GID = IPv6 mapped EUI-64
Ethernet (802.1Q)PFC pause, DSCP-to-PCP marking
25/100/400 GbE PHYPAM4 SerDes, FEC
InfiniBand stack
Verbs / uverbslibibverbs, ibv_post_send()
RDMA CMrdma_cm — connection setup
IB TransportBTH, RETH, AETH opcodes
IB NetworkLID routing, GRH for inter-subnet
IB Link (data link)Credit-based flow control, VLs
IB PhysicalHDR 200G / NDR 400G per port
no UDP/IP encapsulation
Detailed comparison matrix
Attribute RoCEv2 InfiniBand
Losslessness Achieved via PFC (802.1Qbb) + DCQCN. Requires careful DSCP → CoS mapping and priority group configuration. One misconfigured switch breaks it Native — link-layer credit flow control guarantees zero drops. No configuration required beyond subnet bring-up
Congestion signalling ECN (DSCP CE bit) → CNP at receiver → DCQCN rate limiter at sender. Reaction loop adds ~1–2 RTT lag before rate reduction FECN in data packets → CNP from receiver → rate reduced at sender. Similar mechanism, but without PFC dependency
QP types supported RC, UD, UC, XRC — same Verbs model as IB. SRQ supported RC, UD, UC, XRC, DC (Dynamic Connected — NVIDIA ext). SRQ, DCT
Multipath / load balancing ECMP via 5-tuple hash (src IP:port, dst IP:port, proto). UDP src port entropy seeded by QP number — ECMP-friendly by design LID-based routing — per-flow path assigned by Subnet Manager. MLAG-style Adaptive Routing (AR) with NVIDIA SHArP and modern SMs
Fabric management Standard SDN / network OS (Cumulus, SONiC, EOS). No dedicated fabric manager. OpenSM not required Subnet Manager (SM) mandatory — typically UFM (NVIDIA) or OpenSM. SM owns LID assignment, routing tables, and fabric health
MTU 4096 B IB payload + IP/UDP overhead. Jumbo frames (9000 B Ethernet) reduce per-packet overhead 4096 B IB MTU per VL. No Ethernet overhead — wire efficiency ~3% higher than RoCEv2 per packet
Collective ops (MPI / NCCL) NCCL over RoCEv2 — software collectives only (ring-allreduce). No in-network compute acceleration natively NVIDIA SHArP (Scalable Hierarchical Aggregation and Reduction Protocol) — in-network allreduce on switch ASICs. Significant speedup for large-scale collective ops
Inter-fabric routing Native IP routing — routers, BGP, any L3 device. Scales to arbitrary topology. Multi-site trivial IB Router required at subnet boundaries. Inter-subnet latency penalty. GRH adds header overhead. Typically one SM per subnet (<48K end-nodes per LID space)
Tooling / observability perftest, rdma-core, standard Ethernet counters (ethtool -S), DPDK telemetry, OpenTelemetry exporters perftest, ibstat, ibping, ibdiagnet, UFM telemetry, per-port credit counters, fabric-level heat maps
Hardware ecosystem Broad — NVIDIA (CX-6/7), Intel E810, Marvell, Broadcom (co-packaged optics), any merchant silicon switching NVIDIA near-monopoly — ConnectX HCAs, Quantum switches. Limited non-NVIDIA IB at HDR/NDR
Deployment complexity High — PFC storm risk, DCQCN tuning (rp_dce_tcp_g, rp_min_rate), DSCP marking end-to-end, ECN thresholds per queue Medium — SM required, but fabric self-heals. No PFC. Operational complexity shifts to subnet management not per-switch QoS
Cost Merchant silicon switches (Spectrum/Tomahawk) — lower capex. Ethernet NIC cost parity. Optics reuse IB switches (Quantum-2) and HCAs at premium. NDR port cost 2–3× equivalent Ethernet. NVIDIA licensing for UFM/SHArP
Performance characteristics
latency (one-way µs) — lower is better
IB NDR 400G
< 0.6 µs
IB HDR 200G
~0.7 µs
RoCEv2 400G
~1.5 µs
RoCEv2 100G
2–3 µs
message rate (Mrps, 8B) — higher is better
IB NDR CX-7
~200 Mrps
IB HDR CX-6
~150 Mrps
RoCEv2 CX-7
~130 Mrps
RoCEv2 CX-6
~100 Mrps
Use-case fit

RoCEv2 — choose when

  • Ethernet ops model is non-negotiable
  • Multi-tenant cloud fabric — L3 isolation per VLAN/VRF
  • GPU clusters with commodity switching (SONiC / EOS)
  • Existing optics reuse (DAC / AOC / CWDM4)
  • Latency budget > 1 µs acceptable
  • ECMP-based fat-tree — no SM overhead
  • Mixed workload fabric (storage + RDMA + general IP)

InfiniBand — choose when

  • Sub-µs latency is a hard requirement
  • Large-scale MPI / NCCL allreduce (SHArP in-switch)
  • Purpose-built HPC / AI training fabric
  • DGX SuperPOD or H100/H200 dense clusters
  • Collective ops bound workloads (LLM training)
  • NVIDIA full-stack (NVLink + IB + UFM)
  • Budget for premium NIC + switch ASICs
Note: Figures derived from NVIDIA ConnectX-6/7 published datasheets, Mellanox/NVIDIA perftest benchmarks, and IEEE 802.1Qbb/IETF RFC 7910. Real-world latencies depend on QP type (RC vs UD), message size, and network congestion state. SHArP acceleration requires NVIDIA Quantum switches with SHArP-enabled SM configuration.