I’ve been in a lot of interesting discussions the last couple of days on what protocol to use for the underlay when building a VXLAN datacenter network. Do you use an IGP such as OSPF or ISIS or do you use BGP? A common argument for BGP is that running one protocol is less complex than two. Is it, though?
We can argue about if OSPF or BGP is the more well known protocol. What I think is going on here though is that OSPF is perceived as complex due to the following reasons:
- Utilizes both unicast and multicast for messaging.
- Maintains a link state database and runs SPF to calculate best paths.
- Different LSA types and flooding behavior.
- Does not advertise routes.
On the other hand, BGP has the following characteristics:
- Utilizes only unicast for messaging.
- Rides over TCP.
- Advertises prefixes (NLRI).
Is OSPF complex? That’s debateable but everything is difficult if you don’t know it well enough. If you don’t know your way around the LSDB then it can be difficult to understand how routes get into the RIB and later FIB. Not knowing a protocol doesn’t make it complex, though. I would argue that someone with the title of Network Engineer or Network Architect should have at least a basic level of knowledge of OSPF.
Using an IGP in the underlay makes for a nice separation of underlay and overlay. There is nothing wrong with running BGP in the overlay but I oppose to the argument of it being simpler. Why?
- We are trying to make BGP behave like an IGP.
- Not as clear separation of underlay and overlay.
- Requires nerd knobs to get the desired behavior.
- More complicated to setup.
Running an IGP such as OSPF in the underlay is simple. It’s doing what it was designed to do. If running BGP, you need to understand AS paths, why it doesn’t accept prefixes with same AS as locally configured, retaining next-hop and why that matters, manipulating route targets, etc. This is much more complex than setting up OSPF. I have nothing against BGP, to be very clear, but the argument that BGP is simpler is not a very good one. What do you think?