BGP troubleshooting – peer address not matching

Yesterday I did some Internetwork Expert vol1 labs on BGP. I was having trouble getting some of the peers to come up and had to troubleshoot. This post will describe how to troubleshoot when peers won’t form. First, lets look at the topology. Thanks to DennisD on IEOC forums for the image.

R2 and R5 should peer with each other in AS 100. R2 is setup to peer with R5’s IP 155.1.45.5 and R5 is setup to peer with R2’s IP 155.1.23.2. It would have been better to peer over the 155.1.0.0/24 subnet directly but this is to show the steps of troubleshooting. So the session will not form, why? Lets look at some output from debug ip tcp transactions.

Rack1R5#*Mar 1 01:31:39.291: TCP: sending SYN, seq 478218125, ack 0
*Mar 1 01:31:39.291: TCP0: Connection to 155.1.23.2:179, advertising MSS 536
*Mar 1 01:31:39.291: TCP0: state was CLOSED -> SYNSENT [56275 -> 155.1.23.2(179)]
*Mar 1 01:31:39.311: Released port 56275 in Transport Port Agent for TCP IP type 1 delay 240000
*Mar 1 01:31:39.311: TCP0: state was SYNSENT -> CLOSED [56275 -> 155.1.23.2(179)]
*Mar 1 01:31:39.311: TCP0: bad seg from 155.1.23.2 — closing connection: port 56275 seq 0 ack 478218126 rcvnxt 0
rcvwnd 0 len 0
*Mar 1 01:31:39.311: TCP0: connection closed – remote sent RST
*Mar 1 01:31:39.311: TCB 0x651784FC destroyed

We can see that R5 is initiating the connection, it is sending a TCP SYN to R2 on port 179 but R2 responds with a TCP RST which resets the connection. This could indicate that either R2 is not running BGP or that their is a problem with the neighbor statements.

So we want to know what IP R5 is using when sending TCP packets to R2. Lets debug IP packets.

Rack1R5(config)#access-list 101 permit tcp any host 155.1.23.2
Rack1R5#debug ip packet 101
*Mar 1 01:36:01.611: IP: tableid=0, s=155.1.0.5 (local), d=155.1.23.2 (Serial0/0), routed via FIB
*Mar 1 01:36:01.615: IP: s=155.1.0.5 (local), d=155.1.23.2 (Serial0/0), len 44, sending

R5 is using its IP of 155.1.0.5 to communicate with 155.1.23.2 but R2 expects R5 to setup the BGP session from the IP of 155.1.45.5. Let’s verify why R5 is using 155.1.0.5 to get to 155.1.23.2. This is a look at the routing table.

Rack1R5#sh ip route 155.1.23.0
Routing entry for 155.1.23.0/24
Known via “eigrp 100”, distance 90, metric 2681856, type internal
Redistributing via eigrp 100
Last update from 155.1.0.3 on Serial0/0, 00:42:06 ago
Routing Descriptor Blocks:
155.1.0.3, from 155.1.0.3, 00:42:06 ago, via Serial0/0
Route metric is 2681856, traffic share count is 1
Total delay is 40000 microseconds, minimum bandwidth is 1544 Kbit
Reliability 255/255, minimum MTU 1500 bytes
Loading 1/255, Hops 1
* 155.1.0.2, from 155.1.0.2, 00:42:06 ago, via Serial0/0
Route metric is 2681856, traffic share count is 1
Total delay is 40000 microseconds, minimum bandwidth is 1544 Kbit
Reliability 255/255, minimum MTU 1500 bytes
Loading 1/255, Hops 1

We can see that R5 has two equal cost paths to reach the IP of R2. The next hop is either 155.1.0.2 or 155.1.0.3 and these are reachable via the connected subnet of Serial0/0. That is why R5 is using the IP of 155.1.0.5 to source packets. How can we solve this? Either we can setup the neighbor statement to point at 155.1.0.5 or we can change the update-source.

Rack1R5(config-router)#neighbor 155.1.23.2 update-source s0/1

A debug IP packet confirms that the right interface is now being used.

*Mar 1 01:36:31.663: IP: tableid=0, s=155.1.45.5 (local), d=155.1.23.2 (Serial0/0), routed via FIB
*Mar 1 01:36:31.663: IP: s=155.1.45.5 (local), d=155.1.23.2 (Serial0/0), len 44, sending

Show ip bgp confirms that they are now peers.

Neighbor   V AS    MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
155.1.23.2 4 100  5                  5        9          0      0     00:00:05            1

Show tcp brief is a good command to see TCP sessions to/from the router.

Rack1R5#show tcp brief
TCB Local Address Foreign Address (state)
651791FC 155.1.45.5.26655 155.1.23.2.179 ESTAB

And this is how to do basic BGP troubleshooting.