SONiC in Practice — hands-on lab

Building a SONiC leaf-spine fabric from scratch

A step-by-step lab guide for engineers who know Cisco IOS and want to map their mental model onto disaggregated networking. Four nodes, real configs, no fluff.

2 leaf · 2 spine IS-IS underlay BGP overlay GNS3 / containerlab SONiC 202305+
lab-topology — 2-leaf / 2-spine
SPINE TIER LEAF TIER Eth1/1 Eth1/2 Eth1/1 Eth1/2 spine-1 lo 10.0.0.1/32 spine-2 lo 10.0.0.2/32 leaf-1 lo 10.0.0.3/32 leaf-2 lo 10.0.0.4/32 172.16.1.0/31 172.16.4.0/31 IS-IS p2p link BGP loopback reachability p2p /31 · AS 65000

address plan

Lab IP addressing
NodeLoopbackEth1/1 (→ Spine-1)Eth1/2 (→ Spine-2)BGP AS
spine-110.0.0.1/32172.16.1.0/31172.16.2.0/3165000
spine-210.0.0.2/32172.16.3.0/31172.16.4.0/3165000
leaf-110.0.0.3/32172.16.1.1/31172.16.3.1/3165001
leaf-210.0.0.4/32172.16.2.1/31172.16.4.1/3165002
1
Step 01

Basic device configuration

hostname, SSH, local user, management IP, login hardening

⚡

Key difference. SONiC stores global config in /etc/sonic/config_db.json — the source of truth loaded at boot. The CLI tool is config for writes and show for reads. There is no configure terminal. Changes take effect immediately but must be saved explicitly with config save.

concept map — cisco → sonic

ActionCisco IOSSONiC
Hostnamehostname spine-1sudo config hostname spine-1
Local userusername admin priv 15 secret Xsudo useradd -m admin sudo usermod -aG sudo admin sudo passwd admin
SSH daemonip ssh version 2 crypto key gen rsa mod 2048SSH enabled by default. sudo systemctl enable ssh sudo systemctl start ssh
Management IPint GigE0/0 ip addr 192.168.1.1 255.255.255.0sudo config interface ip add \ eth0 192.168.1.1/24
Save configwrite memory / copy run startsudo config save -y
Reloadreloadsudo reboot

config — apply to all nodes (adjust hostname per device)

spine-1 basic setup

bash — SONiCspine-1
# Set hostname
sudo config hostname spine-1

# Create admin user and add to sudo group
sudo useradd -m -s /bin/bash netadmin
sudo usermod -aG sudo netadmin
sudo passwd netadmin

# Set management IP on eth0 (OOB port)
sudo config interface ip add eth0 192.168.100.1/24

# Default route via management gateway
sudo ip route add default via 192.168.100.254 dev eth0

# Persist the config
sudo config save -y
ℹ

SSH hardening. SONiC ships with sshd enabled. Edit /etc/ssh/sshd_config to disable root login (PermitRootLogin no) and restrict to key-based auth. Equivalent to Cisco's ip ssh authentication-retries + aaa authentication login.

verification

✓ verify — basic config
# Hostname
show version                   # shows hostname, SONiC version, asic
hostname                        # quick check

# Interface / IP
show interface status           # equiv: show ip interface brief
show management_interface address

# SSH reachability (from jump host)
ssh netadmin@192.168.100.1
Cisco verificationSONiC equivalent
show versionshow version
show ip interface briefshow interface status
show running-configshow runningconfiguration all
2
Step 02

Interface configuration

fabric links, IP addressing, MTU, port-up verification

⚡

Key difference. SONiC names front-panel ports as Ethernet0, Ethernet4, etc. (index based on breakout lanes). There is no interface range command — use a loop or sonic-cfggen for bulk config. Interface config goes through config interface; L3 mode is automatic once you assign an IP — no no switchport required.

concept map — cisco → sonic

ActionCisco IOSSONiC
Assign IPinterface Eth1/1 ip address 172.16.1.0 255.255.255.254config interface ip add \ Ethernet0 172.16.1.0/31
No shutdownno shutdownconfig interface startup Ethernet0
Shutdownshutdownconfig interface shutdown Ethernet0
Set MTUip mtu 9100config interface mtu Ethernet0 9100
Descriptiondescription "to leaf-1 Eth1/1"config interface description \ Ethernet0 "to leaf-1 Eth1/1"
Loopbackinterface Loopback0 ip addr 10.0.0.1 255.255.255.255config interface ip add \ Loopback0 10.0.0.1/32

spine-1 interfaces

bash — SONiCspine-1
# Loopback
sudo config interface ip add \
  Loopback0 10.0.0.1/32

# Ethernet0 → leaf-1 Eth1/1
sudo config interface ip add \
  Ethernet0 172.16.1.0/31
sudo config interface mtu \
  Ethernet0 9100
sudo config interface description \
  Ethernet0 "to leaf-1"
sudo config interface startup Ethernet0

# Ethernet4 → leaf-2 Eth1/1
sudo config interface ip add \
  Ethernet4 172.16.2.0/31
sudo config interface mtu \
  Ethernet4 9100
sudo config interface description \
  Ethernet4 "to leaf-2"
sudo config interface startup Ethernet4

sudo config save -y

leaf-1 interfaces

bash — SONiCleaf-1
# Loopback
sudo config interface ip add \
  Loopback0 10.0.0.3/32

# Ethernet0 → spine-1 Eth0
sudo config interface ip add \
  Ethernet0 172.16.1.1/31
sudo config interface mtu \
  Ethernet0 9100
sudo config interface description \
  Ethernet0 "to spine-1"
sudo config interface startup Ethernet0

# Ethernet4 → spine-2 Eth0
sudo config interface ip add \
  Ethernet4 172.16.3.1/31
sudo config interface mtu \
  Ethernet4 9100
sudo config interface description \
  Ethernet4 "to spine-2"
sudo config interface startup Ethernet4

sudo config save -y

verification

✓ verify — interfaces
# Port state and speed — equiv: show interfaces status
show interface status

# IP addresses — equiv: show ip interface brief
show ip interfaces

# Detailed counters on a specific interface
show interface counters Ethernet0

# Confirm connectivity to directly connected peer
ping 172.16.1.1 -I Ethernet0    # leaf-1 side: ping spine-1
Cisco verificationSONiC equivalent
show interfaces Eth1/1show interface status Ethernet0
show ip interface briefshow ip interfaces
show interfaces countersshow interface counters
show cdp neighborsshow lldp neighbors
3
Step 03

IS-IS underlay

L3 routing — IS-IS level-2, point-to-point links, loopback redistribution, BFD

⚡

Key difference. SONiC uses FRRouting (FRR) as its routing daemon — the same engine under the hood as many modern NOS platforms. You configure it via vtysh, which has a Cisco-like CLI feel. IS-IS in a leaf-spine fabric runs Level-2 only (no hierarchy needed), all links as point-to-point. Loopbacks are redistributed by IS-IS natively — they're just passive interfaces.

concept map — cisco → sonic (frr)

ActionCisco IOS-XE / NXOSSONiC / FRR vtysh
Enable IS-ISrouter isis UNDERLAY net 49.0001.0100.0000.0001.00 is-type level-2-onlyrouter isis UNDERLAY net 49.0001.0100.0000.0001.00 is-type level-2-only
Interface IS-ISinterface Eth1/1 ip router isis UNDERLAY isis network point-to-point isis circuit-type level-2-onlyinterface Ethernet0 ip router isis UNDERLAY isis network point-to-point isis circuit-type level-2-only
Passive loopbackrouter isis UNDERLAY passive-interface Loopback0interface Loopback0 ip router isis UNDERLAY isis passive
BFD on IS-ISinterface Eth1/1 isis bfdinterface Ethernet0 isis bfd
Metric stylemetric-style widemetric-style wide
Enter FRR CLI(direct CLI)sudo vtysh
▲

NET address convention. Area 49.0001 · System-ID derived from the loopback: 10.0.0.1 → 0100.0000.0001. Pad each octet to 3 digits, split into 4-digit groups.

isis config — spine-1 (vtysh)

vtysh — FRRspine-1
! Enter FRR shell
sudo vtysh

! Global IS-IS process
router isis UNDERLAY
 net 49.0001.0100.0000.0001.00
 is-type level-2-only
 metric-style wide
 log-adjacency-changes
!
! Fabric link to leaf-1
interface Ethernet0
 ip router isis UNDERLAY
 isis network point-to-point
 isis circuit-type level-2-only
 isis metric 10
 isis bfd
!
! Fabric link to leaf-2
interface Ethernet4
 ip router isis UNDERLAY
 isis network point-to-point
 isis circuit-type level-2-only
 isis metric 10
 isis bfd
!
! Loopback — passive, always advertised
interface Loopback0
 ip router isis UNDERLAY
 isis passive
!
write memory

isis config — leaf-1 (vtysh)

vtysh — FRRleaf-1
sudo vtysh

router isis UNDERLAY
 net 49.0001.0100.0000.0003.00
 is-type level-2-only
 metric-style wide
 log-adjacency-changes
!
interface Ethernet0
 ip router isis UNDERLAY
 isis network point-to-point
 isis circuit-type level-2-only
 isis metric 10
 isis bfd
!
interface Ethernet4
 ip router isis UNDERLAY
 isis network point-to-point
 isis circuit-type level-2-only
 isis metric 10
 isis bfd
!
interface Loopback0
 ip router isis UNDERLAY
 isis passive
!
write memory
ℹ

Repeat for spine-2 and leaf-2. NET addresses: spine-2 → 49.0001.0100.0000.0002.00, leaf-2 → 49.0001.0100.0000.0004.00. Interface numbering follows the address plan table above.

verification

✓ verify — IS-IS underlay
# Adjacency state — equiv: show isis neighbors
show isis neighbor              # expect: State UP on each p2p link

# IS-IS LSDB — equiv: show isis database
show isis database
show isis database detail

# Routes learned via IS-IS
show ip route isis              # expect: all 4 loopbacks + p2p subnets

# BFD sessions — verify liveness detection is up
show bfd peers

# Confirm loopback reachability (underlay)
ping 10.0.0.2 source 10.0.0.1   # spine-1 → spine-2 via IS-IS
ping 10.0.0.4 source 10.0.0.1   # spine-1 → leaf-2
Cisco verificationSONiC / FRR equivalent
show isis neighborsshow isis neighbor
show isis databaseshow isis database
show ip route isisshow ip route isis
show bfd neighborsshow bfd peers
debug isis adj-packetsdebug isis adj-packets (in vtysh)
4
Step 04

BGP overlay — loopback peering

eBGP between leaves and spines using loopback addresses over the IS-IS underlay

⚡

Key difference. In a disaggregated leaf-spine, BGP sessions ride the IS-IS underlay using loopbacks — not directly connected IPs. This decouples the overlay from physical topology. FRR makes this identical to Cisco syntax. The critical addition is update-source Loopback0 and ebgp-multihop 2. On a leaf-spine the spines are typically pure Route Reflectors — they don't originate prefixes, they reflect between leaves.

concept map — cisco → sonic (frr)

ActionCisco IOSSONiC / FRR vtysh
BGP processrouter bgp 65000router bgp 65000
Router-IDbgp router-id 10.0.0.1bgp router-id 10.0.0.1
Neighborneighbor 10.0.0.3 remote-as 65001neighbor 10.0.0.3 remote-as 65001
Update sourceneighbor 10.0.0.3 update-source Lo0neighbor 10.0.0.3 update-source Loopback0
Multihopneighbor 10.0.0.3 ebgp-multihop 2neighbor 10.0.0.3 ebgp-multihop 2
Route reflectorneighbor X route-reflector-clientneighbor X route-reflector-client
BFD on BGPneighbor X fall-over bfdneighbor X bfd
Advertise lonetwork 10.0.0.1 mask 255.255.255.255network 10.0.0.1/32

bgp config — spine-1 (route reflector)

vtysh — FRRspine-1 · AS 65000
sudo vtysh

router bgp 65000
 bgp router-id 10.0.0.1
 bgp log-neighbor-changes
 no bgp default ipv4-unicast
 !
 ! Peer with leaf-1
 neighbor 10.0.0.3 remote-as 65001
 neighbor 10.0.0.3 update-source Loopback0
 neighbor 10.0.0.3 ebgp-multihop 2
 neighbor 10.0.0.3 bfd
 neighbor 10.0.0.3 description leaf-1
 !
 ! Peer with leaf-2
 neighbor 10.0.0.4 remote-as 65002
 neighbor 10.0.0.4 update-source Loopback0
 neighbor 10.0.0.4 ebgp-multihop 2
 neighbor 10.0.0.4 bfd
 neighbor 10.0.0.4 description leaf-2
 !
 address-family ipv4 unicast
  ! Advertise own loopback
  network 10.0.0.1/32
  ! Reflect between leaves (spine acts as RR)
  neighbor 10.0.0.3 activate
  neighbor 10.0.0.3 route-reflector-client
  neighbor 10.0.0.4 activate
  neighbor 10.0.0.4 route-reflector-client
  ! No summarization
  no auto-summary
 exit-address-family
!
write memory

bgp config — leaf-1 (client)

vtysh — FRRleaf-1 · AS 65001
sudo vtysh

router bgp 65001
 bgp router-id 10.0.0.3
 bgp log-neighbor-changes
 no bgp default ipv4-unicast
 !
 ! Upstream spines as BGP peers
 neighbor 10.0.0.1 remote-as 65000
 neighbor 10.0.0.1 update-source Loopback0
 neighbor 10.0.0.1 ebgp-multihop 2
 neighbor 10.0.0.1 bfd
 neighbor 10.0.0.1 description spine-1
 !
 neighbor 10.0.0.2 remote-as 65000
 neighbor 10.0.0.2 update-source Loopback0
 neighbor 10.0.0.2 ebgp-multihop 2
 neighbor 10.0.0.2 bfd
 neighbor 10.0.0.2 description spine-2
 !
 address-family ipv4 unicast
  network 10.0.0.3/32
  neighbor 10.0.0.1 activate
  neighbor 10.0.0.2 activate
  no auto-summary
 exit-address-family
!
write memory
ℹ

Repeat for spine-2 (AS 65000) and leaf-2 (AS 65002). Spine-2 mirrors spine-1's config substituting its own loopback 10.0.0.2/32. Leaf-2 mirrors leaf-1 substituting 10.0.0.4/32 and AS 65002.

verification

✓ verify — BGP overlay
# Session state — equiv: show bgp summary
show bgp summary                # expect: all peers Established
show bgp ipv4 unicast summary

# BGP RIB — what routes are we receiving?
show bgp ipv4 unicast           # full table with best-path markers
show ip route bgp               # routes installed in FIB

# Specific prefix path detail
show bgp ipv4 unicast 10.0.0.4/32

# Confirm end-to-end: leaf-1 loopback → leaf-2 loopback
ping 10.0.0.4 source 10.0.0.3 count 5
traceroute 10.0.0.4 source 10.0.0.3
# Expect 2 hops via either spine
Cisco verificationSONiC / FRR equivalent
show bgp summaryshow bgp summary
show ip bgpshow bgp ipv4 unicast
show ip route bgpshow ip route bgp
show ip bgp neighbors X advertised-routesshow bgp neighbor 10.0.0.1 advertised-routes
show ip bgp neighbors X received-routesshow bgp neighbor 10.0.0.1 received-routes
clear ip bgp X softclear bgp 10.0.0.1 soft
debug ip bgpdebug bgp updates (in vtysh)

✓ EXPECTED END STATE

  • ✓4 IS-IS adjacencies (each node sees 2 neighbors)
  • ✓All 4 loopbacks in IS-IS LSDB and FIB
  • ✓4 BGP sessions Established (each leaf peers to both spines)
  • ✓All 4 loopbacks in BGP table on all nodes
  • ✓leaf-1 → leaf-2 ping via loopbacks succeeds (2-hop traceroute)
  • ✓BFD sessions up on all IS-IS and BGP peers