Carl Zellers asked an excellent question on how EIGRP works when run over FlexVPN with IP unnumbered, considering that routers will not be on a common subnet. I thought this was a great question so I took some help from my great friend, the EIGRP guru, Peter Palúch.
First, let’s examine behavior when EIGRP is run on numbered interface. I have built a very simple lab consisting of three routers, R1, R2, and R3, where R1 and R3 are separated by R2. To demonstrate that EIGRP checks that incoming hellos are received on a common subnet, the following simple configurations were applied to R1 and R2:
R1:
interface GigabitEthernet1 ip address 10.0.0.1 255.255.255.0 ! router eigrp LAB ! address-family ipv4 unicast autonomous-system 64512 ! topology base exit-af-topology network 10.0.0.0 0.0.0.255 exit-address-family
R2:
interface GigabitEthernet1 ip address 10.0.1.1 255.255.255.0 ! router eigrp LAB ! address-family ipv4 unicast autonomous-system 64512 ! topology base exit-af-topology network 10.0.1.0 0.0.0.255 exit-address-family
This results in the well familiar messages on the console:
*Jul 15 08:53:20.966: %DUAL-6-NBRINFO: EIGRP-IPv4 64512: Neighbor 10.0.0.1 (GigabitEthernet1) is blocked: not on common subnet (10.0.1.1/24) *Jul 15 08:53:34.831: %DUAL-6-NBRINFO: EIGRP-IPv4 64512: Neighbor 10.0.0.1 (GigabitEthernet1) is blocked: not on common subnet (10.0.1.1/24) *Jul 15 08:53:48.382: %DUAL-6-NBRINFO: EIGRP-IPv4 64512: Neighbor 10.0.0.1 (GigabitEthernet1) is blocked: not on common subnet (10.0.1.1/24) *Jul 15 08:54:01.911: %DUAL-6-NBRINFO: EIGRP-IPv4 64512: Neighbor 10.0.0.1 (GigabitEthernet1) is blocked: not on common subnet (10.0.1.1/24)
I then configured R1 to be a FlexVPN Hub, R3 a client, and R2 is just there to route packets between the two. Configs below:
R1:
crypto ikev2 proposal FLEX encryption aes-gcm-256 prf sha512 group 19 ! crypto ikev2 profile FLEX match identity remote any authentication remote pre-share key cisco authentication local pre-share key cisco virtual-template 1 ! crypto ipsec transform-set NULL esp-null esp-sha512-hmac mode tunnel ! crypto ipsec profile FLEX set transform-set NULL set ikev2-profile FLEX ! interface Loopback0 ip address 1.1.1.1 255.255.255.255 ! interface GigabitEthernet1 ip address 10.0.0.1 255.255.255.0 ! interface Virtual-Template1 type tunnel ip unnumbered Loopback0 tunnel source GigabitEthernet1 tunnel mode ipsec ipv4 tunnel protection ipsec profile FLEX ! ip route 0.0.0.0 0.0.0.0 10.0.0.2
R2:
interface GigabitEthernet1 ip address 10.0.0.2 255.255.255.0 ! interface GigabitEthernet2 ip address 10.0.1.2 255.255.255.0
R3:
crypto ikev2 proposal FLEX encryption aes-gcm-256 prf sha512 group 19 ! crypto ikev2 profile FLEX match identity remote any authentication remote pre-share key cisco authentication local pre-share key cisco ! crypto ipsec transform-set NULL esp-null esp-sha512-hmac mode tunnel ! crypto ipsec profile FLEX set transform-set NULL set ikev2-profile FLEX ! interface Loopback0 ip address 3.3.3.3 255.255.255.255 ! interface Tunnel0 ip unnumbered Loopback0 tunnel source GigabitEthernet1 tunnel mode ipsec ipv4 tunnel destination 10.0.0.1 tunnel protection ipsec profile FLEX ! interface GigabitEthernet1 ip address 10.0.1.1 255.255.255.0 ! ip route 0.0.0.0 0.0.0.0 10.0.1.2
Note the following when it comes to the configurations:
- These are not production grade configs, so don’t use them for that
- ESP Null is used so that we can see what is inside the ESP encapsulation
After applying configurations, the tunnel comes up, as can be seen below. There are no dynamic routes learned yet as we haven’t enabled EIGRP.
R1#show crypto ikev2 sa
IPv4 Crypto IKEv2 SA
Tunnel-id Local Remote fvrf/ivrf Status
1 10.0.0.1/500 10.0.1.1/500 none/none READY
Encr: AES-CBC, keysize: 256, PRF: SHA512, Hash: SHA512, DH Grp:19, Auth sign: PSK, Auth verify: PSK
Life/Active Time: 86400/128 sec
IPv6 Crypto IKEv2 SA
R1#show crypto ipsec sa
interface: Virtual-Access1
Crypto map tag: Virtual-Access1-head-0, local addr 10.0.0.1
protected vrf: (none)
local ident (addr/mask/prot/port): (0.0.0.0/0.0.0.0/0/0)
remote ident (addr/mask/prot/port): (0.0.0.0/0.0.0.0/0/0)
current_peer 10.0.1.1 port 500
PERMIT, flags={origin_is_acl,}
#pkts encaps: 0, #pkts encrypt: 0, #pkts digest: 0
#pkts decaps: 0, #pkts decrypt: 0, #pkts verify: 0
#pkts compressed: 0, #pkts decompressed: 0
#pkts not compressed: 0, #pkts compr. failed: 0
#pkts not decompressed: 0, #pkts decompress failed: 0
#send errors 0, #recv errors 0
local crypto endpt.: 10.0.0.1, remote crypto endpt.: 10.0.1.1
plaintext mtu 1438, path mtu 1500, ip mtu 1500, ip mtu idb GigabitEthernet1
current outbound spi: 0x7AEC8871(2062321777)
PFS (Y/N): N, DH group: none
inbound esp sas:
spi: 0xA9F458E9(2851363049)
transform: esp-null esp-sha512-hmac ,
in use settings ={Tunnel, }
conn id: 2041, flow_id: CSR:41, sibling_flags FFFFFFFF80000048, crypto map: Virtual-Access1-head-0
sa timing: remaining key lifetime (k/sec): (4608000/3598)
IV size: 0 bytes
replay detection support: Y
Status: ACTIVE(ACTIVE)
inbound ah sas:
inbound pcp sas:
outbound esp sas:
spi: 0x7AEC8871(2062321777)
transform: esp-null esp-sha512-hmac ,
in use settings ={Tunnel, }
conn id: 2042, flow_id: CSR:42, sibling_flags FFFFFFFF80000048, crypto map: Virtual-Access1-head-0
sa timing: remaining key lifetime (k/sec): (4608000/3598)
IV size: 0 bytes
replay detection support: Y
Status: ACTIVE(ACTIVE)
outbound ah sas:
outbound pcp sas:
To see the configuration of the virtual-access interface, which is derived from a virtual-template, use the command below:
R1#show derived-config interface virtual-access 1 Building configuration... Derived configuration : 218 bytes ! interface Virtual-Access1 ip unnumbered Loopback0 tunnel source GigabitEthernet1 tunnel mode ipsec ipv4 tunnel destination 10.0.1.1 tunnel protection ipsec profile FLEX no tunnel protection ipsec initiate end
To see the properties of the tunnel, use the command below:
R1#show interfaces virtual-access 1
Virtual-Access1 is up, line protocol is up
Hardware is Virtual Access interface
Interface is unnumbered. Using address of Loopback0 (1.1.1.1)
MTU 9938 bytes, BW 100 Kbit/sec, DLY 50000 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation TUNNEL
Tunnel vaccess, cloned from Virtual-Template1
Vaccess status 0x0, loopback not set
Keepalive not set
Tunnel linestate evaluation up
Tunnel source 10.0.0.1 (GigabitEthernet1), destination 10.0.1.1
Tunnel Subblocks:
src-track:
Virtual-Access1 source tracking subblock associated with GigabitEthernet1
Set of tunnels with source GigabitEthernet1, 2 members (includes iterators), on interface <OK>
Tunnel protocol/transport IPSEC/IP
Tunnel TTL 255
Tunnel transport MTU 1438 bytes
Tunnel transmit bandwidth 8000 (kbps)
Tunnel receive bandwidth 8000 (kbps)
Tunnel protection via IPSec (profile "FLEX")
Last input never, output 00:00:02, output hang never
Last clearing of "show interface" counters 00:03:21
Input queue: 0/375/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/0 (size/max)
5 minute input rate 0 bits/sec, 0 packets/sec
5 minute output rate 0 bits/sec, 0 packets/sec
0 packets input, 0 bytes, 0 no buffer
Received 0 broadcasts (0 IP multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
42 packets output, 2520 bytes, 0 underruns
0 output errors, 0 collisions, 0 interface resets
0 unknown protocol drops
0 output buffer failures, 0 output buffers swapped out
Note that when we try to ping R3’s loopback, that will fail since this traffic is not going through the tunnel.
R1#ping 3.3.3.3 so lo0 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 3.3.3.3, timeout is 2 seconds: Packet sent with a source address of 1.1.1.1 ..... Success rate is 0 percent (0/5)
We then enable EIGRP on the loopbacks of R1 and R3:
R1:
router eigrp LAB ! address-family ipv4 unicast autonomous-system 64512 ! topology base exit-af-topology network 1.1.1.1 0.0.0.0 exit-address-family
R3:
router eigrp LAB ! address-family ipv4 unicast autonomous-system 64512 ! topology base exit-af-topology network 3.3.3.3 0.0.0.0 exit-address-family
As soon as that’s done, the adjacency is established and we learn the route of each loopback:
*Jul 17 14:38:16.207: %DUAL-5-NBRCHANGE: EIGRP-IPv4 64512: Neighbor 3.3.3.3 (Virtual-Access1) is up: new adjacency[OK]
R1#show ip route
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
a - application route
+ - replicated route, % - next hop override, p - overrides from PfR
Gateway of last resort is 10.0.0.2 to network 0.0.0.0
S* 0.0.0.0/0 [1/0] via 10.0.0.2
1.0.0.0/32 is subnetted, 1 subnets
C 1.1.1.1 is directly connected, Loopback0
3.0.0.0/32 is subnetted, 1 subnets
D 3.3.3.3 [90/76800640] via 3.3.3.3, 00:00:25, Virtual-Access1
10.0.0.0/8 is variably subnetted, 2 subnets, 2 masks
C 10.0.0.0/24 is directly connected, GigabitEthernet1
L 10.0.0.1/32 is directly connected, GigabitEthernet1
There is now reachability between loopbacks:
R1#ping 3.3.3.3 so lo0 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 3.3.3.3, timeout is 2 seconds: Packet sent with a source address of 1.1.1.1 !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 2/5/18 ms
Notice that there were no EIGRP messages about not being on a common subnet. I even tried to change the mask of the loopback to something else than /32:
R1(config)#int lo0 R1(config-if)#ip add 1.1.1.1 255.255.255.254 % Warning: use /31 mask on non point-to-point interface cautiously R1(config-if)# *Jul 19 05:01:07.969: %DUAL-5-NBRCHANGE: EIGRP-IPv4 64512: Neighbor 3.3.3.3 (Virtual-Access1) is down: interface down *Jul 19 05:01:08.343: %DUAL-5-NBRCHANGE: EIGRP-IPv4 64512: Neighbor 3.3.3.3 (Virtual-Access1) is up: new adjacency R1(config-if)#ip add 1.1.1.1 255.255.255.0 R1(config-if)# *Jul 19 05:01:29.931: %DUAL-5-NBRCHANGE: EIGRP-IPv4 64512: Neighbor 3.3.3.3 (Virtual-Access1) is down: interface down *Jul 19 05:01:31.261: %DUAL-5-NBRCHANGE: EIGRP-IPv4 64512: Neighbor 3.3.3.3 (Virtual-Access1) is up: new adjacency
The adjacency flapped but came up immediately. This means that regardless of mask, when IP unnumbered is used, the check for common subnet is disabled for EIGRP.
That’s all great, but there is still some more digging to do. How does EIGRP even know where to send the packets? If you noticed, before the EIGRP adjacency was up, there were no routes for respective router’s loopback in the RIB and we couldn’t ping between loopbacks. To do some more digging, let’s head over to Wireshark, and this is why I used ESP Null as the encryption algorithm. Thanks to Peter and Nick for providing help on Wireshark.
Wireshark doesn’t try to decode ESP packets by default, so we need to enable this by checking Attempt to detect/decode encrypted ESP payloads:
Then we need to enter information about the SAs into Wireshark. We can get this information, such as the SPI from show crypto ipsec sa. Go to ESP SAs.
Enter information about SAs, such as below:
We can now start seeing some interesting information in Wireshark, such as the headers:
Note that there is both inner and outer header. The outer header is the tunnel sourced from 10.0.0.1 and the EIGRP packet is sent with a source of 1.1.1.1 which is R1's loopback. Note that Hello is sent as multicast and then unicast is sent between the two routers.
So let’s break down this a bit:
- Tunnel is using
IP unnumberedfrom loopback- IP packets sent over tunnel are sourced with (inner header)
IP of loopback
- IP packets sent over tunnel are sourced with (inner header)
EIGRPwas enabled for loopback- EIGRP
Hellos(multicast to 224.0.0.10) are sent over the tunnel with source IP of loopback
- EIGRP
But the peculiar thing here is, if there is no route in the RIB for the loopbacks of the routers, how does the router know where to send the unicast packets? The logic is as follows, unless Donnie has another opinion 😉
DVTIis basically a way of generating a “static” configuration dynamically- Hence each
virtual-accessinterface, cloned fromvirtual template, isuniqueinterface betweentwo peers- This is a
point to pointinterface
- This is a
- If packet is received with a certain
source IPover tunnel, that IP must bereachable over the tunnel- We can
send packets to that IPover the tunnel
- We can
EIGRP can send packets(unicast) to that IP over the tunnelAdjacenciescan form
- Hence each
Edit: After receiving some more information from Peter and Donnie, let’s get even more detailed on the process of how EIGRP can send these packets.
- IKEv2 is used to build the IPSec tunnel. R3 uses
tun0interface while R1 usesvirtual-accessinterface, cloned fromvirtual template- As mentioned previously, these are
point to pointinterfaces - Note that
1.1.1.1, and3.3.3.3, the loopbacks of each router, are NOT involved in setting up the tunnel
- As mentioned previously, these are
- Once the tunnel is up, it’s
availablefor sending data - Routers start sending multicast
Hellosto 224.0.0.10 overvirtual-accessandtun0interface- As enabled by
network 1.1.1.1andnetwork 3.3.3.3command, respectively Sourcingpackets from 1.1.1.1 and 3.3.3.3, respectively, due toIP unnumbered
- As enabled by
- When Hello is received, router learns
identityof neighbor andinterfaceto reach it- R1 learns of
3.3.3.3 over virtual-access - R3 learns of
1.1.1.1 over tun0 - This information is stored in the
EIGRP neighbor table
- R1 learns of
- Unicast packets are sent to 1.1.1.1 and 3.3.3.3, respectively, using the
EIGRP neighbor tableas information source- EIGRP has the ability to send packets
out an interface directlyBypassingthe routing table
- EIGRP has the ability to send packets
- This allows
adjacencyto come up between the EIGRP neighbors- When adjacency is up,
1.1.1.1/32and3.3.3.3/32, respectively, is advertised - Each router learns the other router’s loopback as an
EIGRP route
- When adjacency is up,
I actually spent a couple of days going down this rabbit hole. Why? I thought it was an interesting lesson to see the interaction of control- and data plane. That we can (successfully) send traffic between two IPs even though we have no route. The notion of point to point interfaces and just sending traffic out an interface, even though we don’t know exactly what is reachable there, may be lost on many people that have no background with serial interfaces, PPP, Frame Relay etc.
I hope this post was interesting. I certainly learned a lot myself. A big thanks to Peter for the immense help with this post.
This post is great for many reasons. Lets start with the obvious. I was unaware of wiresharks capability to accept spi information to shed the esp headers to better see traffic (still esp-null), Very cool.
So let’s say you’re a FlexVPN master, you can configure the hub/spokes with ease. What happens when someone asks you WHY you chose EIGRP over OSPF, BGP, or RIP?
This looks behind the curtain to provide important design/deployment considerations and context, which is what really matters here.
Great post!