My dear friend Micheline Murphy posted an excellent question on OSPF in a Hub and Spoke topology at the Cisco Learning Network. The scenario is a Hub and Spoke topology with two Hub routers that are ABRs belonging to area 100 and area 200. SP-101 and SP-102 belong to area 100. SP-201 and SP-202 belong to area 200. The topology is shown below:
The OSPF areas are shown below:
Some facts about the setup and intent of this post:
- All routers are Catalyst8000v running IOS-XE 17.6.3.
- Hub routers are connected to area 0 where the prefix 198.51.100.0/24 is being advertised.
- Each spoke advertises a /28 from 192.0.2.0/24.
- All interfaces are point to point as the purpose is not to simulate a NBMA topology.
- The intent is to verify what happens in a failure scenario but lab first shows the stable topology.
The expectation is that in a stable topology each Spoke will have two ECMP routes, one via each Hub, to the other spokes. The router SP-202 will be used to demonstrate. First let’s verify that everything is working as expected. SP-202 is a router in area 200:
SP-202#show ip ospf 1
Routing Process "ospf 1" with ID 192.168.128.174
Start time: 00:40:26.530, Time elapsed: 00:00:52.890
Supports only single TOS(TOS0) routes
Supports opaque LSA
Supports Link-local Signaling (LLS)
Supports area transit capability
Supports NSSA (compatible with RFC 3101)
Supports Database Exchange Summary List Optimization (RFC 5243)
Event-log enabled, Maximum number of events: 1000, Mode: cyclic
Router is not originating router-LSAs with maximum metric
Initial SPF schedule delay 50 msecs
Minimum hold time between two consecutive SPFs 200 msecs
Maximum wait time between two consecutive SPFs 5000 msecs
Incremental-SPF disabled
Initial LSA throttle delay 50 msecs
Minimum hold time for LSA throttle 200 msecs
Maximum wait time for LSA throttle 5000 msecs
Minimum LSA arrival 100 msecs
LSA group pacing timer 240 secs
Interface flood pacing timer 33 msecs
Retransmission pacing timer 66 msecs
EXCHANGE/LOADING adjacency limit: initial 300, process maximum 300
Number of external LSA 0. Checksum Sum 0x000000
Number of opaque AS LSA 0. Checksum Sum 0x000000
Number of DCbitless external and opaque AS LSA 0
Number of DoNotAge external and opaque AS LSA 0
Number of areas in this router is 1. 1 normal 0 stub 0 nssa
Number of areas transit capable is 0
External flood list length 0
IETF NSF helper support enabled
Cisco NSF helper support enabled
Reference bandwidth unit is 100 mbps
Area 200
Number of interfaces in this area is 3
Area has no authentication
SPF algorithm last executed 00:00:09.013 ago
SPF algorithm executed 6 times
Area ranges are
Number of LSA 22. Checksum Sum 0x0AED55
Number of opaque link LSA 0. Checksum Sum 0x000000
Number of DCbitless LSA 0
Number of indication LSA 0
Number of DoNotAge LSA 0
Flood list length 0
It has full adjacencies to both Hub routers:
SP-202#show ip ospf nei Neighbor ID Pri State Dead Time Address Interface 192.168.128.169 0 FULL/ - 00:00:37 192.0.2.37 GigabitEthernet2.1509 192.168.128.164 0 FULL/ - 00:00:36 192.0.2.21 GigabitEthernet2.1505
The ABRs are HUB-1 and HUB-2:
SP-202#show ip ospf border-routers
OSPF Router with ID (192.168.128.174) (Process ID 1)
Base Topology (MTID 0)
Internal Router Routing Table
Codes: i - Intra-area route, I - Inter-area route
i 192.168.128.164 [1] via 192.0.2.21, GigabitEthernet2.1505, ABR, Area 200, SPF 6
i 192.168.128.169 [1] via 192.0.2.37, GigabitEthernet2.1509, ABR, Area 200, SPF 6
SP-202 has a two ECMP OSPF IA routes for 192.0.2.64/28 which is the network of SP-101:
SP-202#show ip route 192.0.2.64 255.255.255.240
Routing entry for 192.0.2.64/28
Known via "ospf 1", distance 110, metric 3, type inter area
Last update from 192.0.2.37 on GigabitEthernet2.1509, 00:00:57 ago
Routing Descriptor Blocks:
192.0.2.37, from 192.168.128.169, 00:00:57 ago, via GigabitEthernet2.1509
Route metric is 3, traffic share count is 1
* 192.0.2.21, from 192.168.128.164, 00:01:21 ago, via GigabitEthernet2.1505
Route metric is 3, traffic share count is 1
The summary LSAs can be seen below:
SP-202#show ip ospf data summary 192.0.2.64
OSPF Router with ID (192.168.128.174) (Process ID 1)
Summary Net Link States (Area 200)
LS age: 572
Options: (No TOS-capability, DC, Upward)
LS Type: Summary Links(Network)
Link State ID: 192.0.2.64 (summary Network Number)
Advertising Router: 192.168.128.164
LS Seq Number: 80000001
Checksum: 0x7543
Length: 28
Network Mask: /28
MTID: 0 Metric: 2
LS age: 574
Options: (No TOS-capability, DC, Upward)
LS Type: Summary Links(Network)
Link State ID: 192.0.2.64 (summary Network Number)
Advertising Router: 192.168.128.169
LS Seq Number: 80000001
Checksum: 0x575C
Length: 28
Network Mask: /28
MTID: 0 Metric: 2
SP-202 can ping SP-101:
SP-202#ping 192.0.2.65 so lo0 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 192.0.2.65, timeout is 2 seconds: Packet sent with a source address of 192.0.2.113 !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 1/5/21 ms
Now for the convergence scenario. What happens when the link between SP-101 and HUB-1 goes down?
Will one of the ECMP routes go away or will there be an alternate path from SP-202 to HUB-1 to SP-102 to HUB-2 and then to SP-101? The path would look like this:
Let’s bring the link down on HUB-1:
HUB-1(config)#int gi2.1502 HUB-1(config-subif)#shut HUB-1(config-subif)#^Z
SP-101 now only has one full OSPF adjacency:
SP-101#sh ip ospf nei Neighbor ID Pri State Dead Time Address Interface 192.168.128.169 0 FULL/ - 00:00:39 192.0.2.25 GigabitEthernet2.1506
How many routes does SP-202 have?
SP-202#show ip route 192.0.2.64 255.255.255.240
Routing entry for 192.0.2.64/28
Known via "ospf 1", distance 110, metric 3, type inter area
Last update from 192.0.2.37 on GigabitEthernet2.1509, 00:24:22 ago
Routing Descriptor Blocks:
* 192.0.2.37, from 192.168.128.169, 00:24:22 ago, via GigabitEthernet2.1509
Route metric is 3, traffic share count is 1
It now only has the single route. Does HUB-1 know about 192.0.2.64/28 at all?
HUB-1#show ip route 192.0.2.64 255.255.255.240
Routing entry for 192.0.2.64/28
Known via "ospf 1", distance 110, metric 4, type intra area
Last update from 192.0.2.14 on GigabitEthernet2.1503, 00:05:12 ago
Routing Descriptor Blocks:
* 192.0.2.14, from 192.168.128.171, 00:05:12 ago, via GigabitEthernet2.1503
Route metric is 4, traffic share count is 1
It does! The metric is now 4 to reach this network. Let’s do a traceroute:
HUB-1#traceroute 192.0.2.65 Type escape sequence to abort. Tracing the route to 192.0.2.65 VRF info: (vrf in name/id, vrf out name/id) 1 192.0.2.14 0 msec 0 msec 1 msec 2 192.0.2.29 1 msec 1 msec 1 msec 3 192.0.2.26 3 msec * 2 msec
Traffic is going to SP-102, then HUB-2 and finally SP-101. Is HUB-1 still advertising a summary LSA for this network?
HUB-1#show ip ospf data summary 192.0.2.64
OSPF Router with ID (192.168.128.164) (Process ID 1)
Summary Net Link States (Area 0)
LS age: 559
Options: (No TOS-capability, DC, Upward)
LS Type: Summary Links(Network)
Link State ID: 192.0.2.64 (summary Network Number)
Advertising Router: 192.168.128.164
LS Seq Number: 80000003
Checksum: 0x852F
Length: 28
Network Mask: /28
MTID: 0 Metric: 4
LS age: 593
Options: (No TOS-capability, DC, Upward)
LS Type: Summary Links(Network)
Link State ID: 192.0.2.64 (summary Network Number)
Advertising Router: 192.168.128.169
LS Seq Number: 80000002
Checksum: 0x555D
Length: 28
Network Mask: /28
MTID: 0 Metric: 2
Summary Net Link States (Area 200)
LS age: 559
Options: (No TOS-capability, DC, Upward)
LS Type: Summary Links(Network)
Link State ID: 192.0.2.64 (summary Network Number)
Advertising Router: 192.168.128.164
LS Seq Number: 80000003
Checksum: 0x852F
Length: 28
Network Mask: /28
MTID: 0 Metric: 4
LS age: 593
Options: (No TOS-capability, DC, Upward)
LS Type: Summary Links(Network)
Link State ID: 192.0.2.64 (summary Network Number)
Advertising Router: 192.168.128.169
LS Seq Number: 80000002
Checksum: 0x555D
Length: 28
Network Mask: /28
MTID: 0 Metric: 2
Indeed it is! SP-202 is receiving this LSA:
SP-202#show ip ospf data summary 192.0.2.64
OSPF Router with ID (192.168.128.174) (Process ID 1)
Summary Net Link States (Area 200)
LS age: 612
Options: (No TOS-capability, DC, Upward)
LS Type: Summary Links(Network)
Link State ID: 192.0.2.64 (summary Network Number)
Advertising Router: 192.168.128.164
LS Seq Number: 80000003
Checksum: 0x852F
Length: 28
Network Mask: /28
MTID: 0 Metric: 4
LS age: 645
Options: (No TOS-capability, DC, Upward)
LS Type: Summary Links(Network)
Link State ID: 192.0.2.64 (summary Network Number)
Advertising Router: 192.168.128.169
LS Seq Number: 80000002
Checksum: 0x555D
Length: 28
Network Mask: /28
MTID: 0 Metric: 2
It’s not in the OSPF RIB via HUB-1, though!
SP-202#show ip ospf rib 192.0.2.64 255.255.255.240
OSPF Router with ID (192.168.128.174) (Process ID 1)
Base Topology (MTID 0)
OSPF local RIB
Codes: * - Best, > - Installed in global RIB
LSA: type/LSID/originator
*> 192.0.2.64/28, Inter, cost 3, area 200
SPF Instance 7, age 00:11:27
contributing LSA: 3/192.0.2.64/192.168.128.169 (area 200)
contributing LSA: 3/192.0.2.64/192.168.128.164 (area 200)
Flags: RIB, PartialSPF
via 192.0.2.37, GigabitEthernet2.1509
Flags: RIB
LSA: 3/192.0.2.64/192.168.128.169
Hence only one route is available. The one via HUB-2 since it has a lower cost:
SP-202#show ip route 192.0.2.64 255.255.255.240
Routing entry for 192.0.2.64/28
Known via "ospf 1", distance 110, metric 3, type inter area
Last update from 192.0.2.37 on GigabitEthernet2.1509, 00:36:01 ago
Routing Descriptor Blocks:
* 192.0.2.37, from 192.168.128.169, 00:36:01 ago, via GigabitEthernet2.1509
Route metric is 3, traffic share count is 1
There you have it! There is no suboptimal routing in this scenario as OSPF will use the shortest path.
OSPF Convergence In a Hub and Spoke Topology
Interesting! Thank you for writing this, Daniel! 🙂
Thanks, Petru!
Nice blog post!
In a real world scenario you would of course tweak OSPF to limit the blast radius of link state changes from all of the spokes. Nick Russo wrote a nice paper on the subject for those that are interested: http://njrusmc.net/pub/ospf_dmvpn_anycast.pdf
Thanks, Robert!
Yes, link state protocols are often not the best in Hub and Spoke topologies but if you intend to use it you should do some tweaking. Nick is a dear friend and he has many good papers like the one you linked to.
Thanks for this Post, very interesting
Thanks!