To be able to show this scenario, I have added another server (SERVER-2) and will be using the topology below:
There is already existing configuration for VLAN 10 (L2 VNI) and for VLAN 100 (L3 VNI) which is shown below:
Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
vrf context Tenant1
vni 10001
rd auto
address-family ipv4 unicast
route-target both auto
route-target both auto evpn
!
interface Vlan10
no shutdown
vrf member Tenant1
ip address 198.51.100.1/24
fabric forwarding mode anycast-gateway
!
interface Vlan100
no shutdown
mtu 9216
vrf member Tenant1
ip forward
vrf context Tenant1
vni 10001
rd auto
address-family ipv4 unicast
route-target both auto
route-target both auto evpn
!
interface Vlan10
no shutdown
vrf member Tenant1
ip address 198.51.100.1/24
fabric forwarding mode anycast-gateway
!
interface Vlan100
no shutdown
mtu 9216
vrf member Tenant1
ip forward
vrf context Tenant1
vni 10001
rd auto
address-family ipv4 unicast
route-target both auto
route-target both auto evpn
!
interface Vlan10
no shutdown
vrf member Tenant1
ip address 198.51.100.1/24
fabric forwarding mode anycast-gateway
!
interface Vlan100
no shutdown
mtu 9216
vrf member Tenant1
ip forward
To get SERVER-2 connected the following is needed:
Configure VLAN 20 and map it to L2 VNI (VNI 10002).
Make the L2 VNI a member of the NVE.
SVI for VLAN 20.
Configure port towards SERVER-2 in VLAN 20.
This is shown below:
Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
vlan 20
vn-segment 10002
!
interface nve1
member vni 10002
ingress-replication protocol bgp
!
interface Vlan20
no shutdown
vrf member Tenant1
ip address 10.0.0.1/24
fabric forwarding mode anycast-gateway
!
interface Ethernet1/3
switchport access vlan 20
vlan 20
vn-segment 10002
!
interface nve1
member vni 10002
ingress-replication protocol bgp
!
interface Vlan20
no shutdown
vrf member Tenant1
ip address 10.0.0.1/24
fabric forwarding mode anycast-gateway
!
interface Ethernet1/3
switchport access vlan 20
vlan 20
vn-segment 10002
!
interface nve1
member vni 10002
ingress-replication protocol bgp
!
interface Vlan20
no shutdown
vrf member Tenant1
ip address 10.0.0.1/24
fabric forwarding mode anycast-gateway
!
interface Ethernet1/3
switchport access vlan 20
Now let’s try to ping 10.0.0.1 from SERVER-2:
Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
ping 10.0.0.1
PING 10.0.0.1(10.0.0.1)56(84) bytes of data.
64 bytes from 10.0.0.1: icmp_seq=2 ttl=255 time=1.16 ms
64 bytes from 10.0.0.1: icmp_seq=3 ttl=255 time=1.17 ms
ping 10.0.0.1
PING 10.0.0.1 (10.0.0.1) 56(84) bytes of data.
64 bytes from 10.0.0.1: icmp_seq=2 ttl=255 time=1.16 ms
64 bytes from 10.0.0.1: icmp_seq=3 ttl=255 time=1.17 ms
ping 10.0.0.1
PING 10.0.0.1 (10.0.0.1) 56(84) bytes of data.
64 bytes from 10.0.0.1: icmp_seq=2 ttl=255 time=1.16 ms
64 bytes from 10.0.0.1: icmp_seq=3 ttl=255 time=1.17 ms
That is working. Let’s see if we can ping SERVER-1:
Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
ping 198.51.100.11
PING 198.51.100.11(198.51.100.11)56(84) bytes of data.
64 bytes from 198.51.100.11: icmp_seq=1 ttl=62 time=5.07 ms
64 bytes from 198.51.100.11: icmp_seq=2 ttl=62 time=5.58 ms
ping 198.51.100.11
PING 198.51.100.11 (198.51.100.11) 56(84) bytes of data.
64 bytes from 198.51.100.11: icmp_seq=1 ttl=62 time=5.07 ms
64 bytes from 198.51.100.11: icmp_seq=2 ttl=62 time=5.58 ms
ping 198.51.100.11
PING 198.51.100.11 (198.51.100.11) 56(84) bytes of data.
64 bytes from 198.51.100.11: icmp_seq=1 ttl=62 time=5.07 ms
64 bytes from 198.51.100.11: icmp_seq=2 ttl=62 time=5.58 ms
That is now working as well. In a future post I will describe more on symmetric IRB and do a detailed packet walk, but for now let’s focus on the control plane. How is routing provided between LEAF-2 where SERVER-2 is connected and LEAF-1 where SERVER-1 is connected? LEAF-2 advertises two route type 2 routes with information about SERVER-2. One route is the MAC address only and the other route contains both the MAC and IP as can be seen below:
Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
Leaf2# show bgp l2vpn evpn 0050.56ad.f48d
BGP routing table information for VRF default, address family L2VPN EVPN
Route Distinguisher: 192.0.2.4:32787(L2VNI 10002)
BGP routing table entry for[2]:[0]:[0]:[48]:[0050.56ad.f48d]:[0]:[0.0.0.0]/216, version 971
Paths: (1 available, best #1)
Flags: (0x000102)(high32 00000000) on xmit-list, is not in l2rib/evpn
Advertised path-id 1
Path type: local, path is valid, is best path, no labeled nexthop
AS-Path: NONE, path locally originated
203.0.113.2(metric 0) from 0.0.0.0(192.0.2.4)
Origin IGP, MED not set, localpref 100, weight 32768
Received label 10002
Extcommunity: RT:65000:10002 ENCAP:8
Path-id 1 advertised to peers:
192.0.2.11192.0.2.12
BGP routing table entry for[2]:[0]:[0]:[48]:[0050.56ad.f48d]:[32]:[10.0.0.22]/272, version 972
Paths: (1 available, best #1)
Flags: (0x000102)(high32 00000000) on xmit-list, is not in l2rib/evpn
Advertised path-id 1
Path type: local, path is valid, is best path, no labeled nexthop
AS-Path: NONE, path locally originated
203.0.113.2(metric 0) from 0.0.0.0(192.0.2.4)
Origin IGP, MED not set, localpref 100, weight 32768
Leaf2# show bgp l2vpn evpn 0050.56ad.f48d
BGP routing table information for VRF default, address family L2VPN EVPN
Route Distinguisher: 192.0.2.4:32787 (L2VNI 10002)
BGP routing table entry for [2]:[0]:[0]:[48]:[0050.56ad.f48d]:[0]:[0.0.0.0]/216, version 971
Paths: (1 available, best #1)
Flags: (0x000102) (high32 00000000) on xmit-list, is not in l2rib/evpn
Advertised path-id 1
Path type: local, path is valid, is best path, no labeled nexthop
AS-Path: NONE, path locally originated
203.0.113.2 (metric 0) from 0.0.0.0 (192.0.2.4)
Origin IGP, MED not set, localpref 100, weight 32768
Received label 10002
Extcommunity: RT:65000:10002 ENCAP:8
Path-id 1 advertised to peers:
192.0.2.11 192.0.2.12
BGP routing table entry for [2]:[0]:[0]:[48]:[0050.56ad.f48d]:[32]:[10.0.0.22]/272, version 972
Paths: (1 available, best #1)
Flags: (0x000102) (high32 00000000) on xmit-list, is not in l2rib/evpn
Advertised path-id 1
Path type: local, path is valid, is best path, no labeled nexthop
AS-Path: NONE, path locally originated
203.0.113.2 (metric 0) from 0.0.0.0 (192.0.2.4)
Origin IGP, MED not set, localpref 100, weight 32768
Received label 10002 10001
Extcommunity: RT:65000:10001 RT:65000:10002 ENCAP:8 Router MAC:00ad.f3bb.1b08
Path-id 1 advertised to peers:
192.0.2.11 192.0.2.12
Leaf2# show bgp l2vpn evpn 0050.56ad.f48d
BGP routing table information for VRF default, address family L2VPN EVPN
Route Distinguisher: 192.0.2.4:32787 (L2VNI 10002)
BGP routing table entry for [2]:[0]:[0]:[48]:[0050.56ad.f48d]:[0]:[0.0.0.0]/216, version 971
Paths: (1 available, best #1)
Flags: (0x000102) (high32 00000000) on xmit-list, is not in l2rib/evpn
Advertised path-id 1
Path type: local, path is valid, is best path, no labeled nexthop
AS-Path: NONE, path locally originated
203.0.113.2 (metric 0) from 0.0.0.0 (192.0.2.4)
Origin IGP, MED not set, localpref 100, weight 32768
Received label 10002
Extcommunity: RT:65000:10002 ENCAP:8
Path-id 1 advertised to peers:
192.0.2.11 192.0.2.12
BGP routing table entry for [2]:[0]:[0]:[48]:[0050.56ad.f48d]:[32]:[10.0.0.22]/272, version 972
Paths: (1 available, best #1)
Flags: (0x000102) (high32 00000000) on xmit-list, is not in l2rib/evpn
Advertised path-id 1
Path type: local, path is valid, is best path, no labeled nexthop
AS-Path: NONE, path locally originated
203.0.113.2 (metric 0) from 0.0.0.0 (192.0.2.4)
Origin IGP, MED not set, localpref 100, weight 32768
Received label 10002 10001
Extcommunity: RT:65000:10001 RT:65000:10002 ENCAP:8 Router MAC:00ad.f3bb.1b08
Path-id 1 advertised to peers:
192.0.2.11 192.0.2.12
It is the second route which contains MAC and IP that can be used to provide host routing. As the name implies, host routing only works with /32 and /128 routes depending on IP protocol version. You can’t advertise a /24 in this manner, for example. That would require route type 5. The second route has route targets containing both the L2 VNI (65000:10002) and the L3 VNI (65000:10001). LEAF-1 has a L3 VNI that is importing routes that have a route target of 65000:10001. If we look at LEAF-1, we can first see the route that was originated by LEAF-2 with an RD of 192.0.2.4:32787 and then the imported route which has the RD of 192.0.2.3:3:
Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
Leaf1# show bgp l2vpn evpn 0050.56ad.f48d
BGP routing table information for VRF default, address family L2VPN EVPN
Route Distinguisher: 192.0.2.4:32787
BGP routing table entry for[2]:[0]:[0]:[48]:[0050.56ad.f48d]:[32]:[10.0.0.22]/272, version 1276
Paths: (2 available, best #2)
Flags: (0x000202)(high32 00000000) on xmit-list, is not in l2rib/evpn, is not in HW
Path type: internal, path is valid, not best reason: Neighbor Address, no labeled nexthop
Leaf1# show bgp l2vpn evpn 0050.56ad.f48d
BGP routing table information for VRF default, address family L2VPN EVPN
Route Distinguisher: 192.0.2.4:32787
BGP routing table entry for [2]:[0]:[0]:[48]:[0050.56ad.f48d]:[32]:[10.0.0.22]/272, version 1276
Paths: (2 available, best #2)
Flags: (0x000202) (high32 00000000) on xmit-list, is not in l2rib/evpn, is not in HW
Path type: internal, path is valid, not best reason: Neighbor Address, no labeled nexthop
AS-Path: NONE, path sourced internal to AS
203.0.113.2 (metric 81) from 192.0.2.12 (192.0.2.2)
Origin IGP, MED not set, localpref 100, weight 0
Received label 10002 10001
Extcommunity: RT:65000:10001 RT:65000:10002 ENCAP:8 Router MAC:00ad.f3bb.1b08
Originator: 192.0.2.4 Cluster list: 192.0.2.2
Advertised path-id 1
Path type: internal, path is valid, is best path, no labeled nexthop
Imported to 2 destination(s)
Imported paths list: Tenant1 L3-10001
AS-Path: NONE, path sourced internal to AS
203.0.113.2 (metric 81) from 192.0.2.11 (192.0.2.1)
Origin IGP, MED not set, localpref 100, weight 0
Received label 10002 10001
Extcommunity: RT:65000:10001 RT:65000:10002 ENCAP:8 Router MAC:00ad.f3bb.1b08
Originator: 192.0.2.4 Cluster list: 192.0.2.1
Path-id 1 not advertised to any peer
Route Distinguisher: 192.0.2.3:3 (L3VNI 10001)
BGP routing table entry for [2]:[0]:[0]:[48]:[0050.56ad.f48d]:[32]:[10.0.0.22]/272, version 1277
Paths: (1 available, best #1)
Flags: (0x000202) (high32 00000000) on xmit-list, is not in l2rib/evpn, is not in HW
Advertised path-id 1
Path type: internal, path is valid, is best path, no labeled nexthop
Imported from 192.0.2.4:32787:[2]:[0]:[0]:[48]:[0050.56ad.f48d]:[32]:[10.0.0.22]/272
AS-Path: NONE, path sourced internal to AS
203.0.113.2 (metric 81) from 192.0.2.11 (192.0.2.1)
Origin IGP, MED not set, localpref 100, weight 0
Received label 10002 10001
Extcommunity: RT:65000:10001 RT:65000:10002 ENCAP:8 Router MAC:00ad.f3bb.1b08
Originator: 192.0.2.4 Cluster list: 192.0.2.1
Path-id 1 not advertised to any peer
Leaf1# show bgp l2vpn evpn 0050.56ad.f48d
BGP routing table information for VRF default, address family L2VPN EVPN
Route Distinguisher: 192.0.2.4:32787
BGP routing table entry for [2]:[0]:[0]:[48]:[0050.56ad.f48d]:[32]:[10.0.0.22]/272, version 1276
Paths: (2 available, best #2)
Flags: (0x000202) (high32 00000000) on xmit-list, is not in l2rib/evpn, is not in HW
Path type: internal, path is valid, not best reason: Neighbor Address, no labeled nexthop
AS-Path: NONE, path sourced internal to AS
203.0.113.2 (metric 81) from 192.0.2.12 (192.0.2.2)
Origin IGP, MED not set, localpref 100, weight 0
Received label 10002 10001
Extcommunity: RT:65000:10001 RT:65000:10002 ENCAP:8 Router MAC:00ad.f3bb.1b08
Originator: 192.0.2.4 Cluster list: 192.0.2.2
Advertised path-id 1
Path type: internal, path is valid, is best path, no labeled nexthop
Imported to 2 destination(s)
Imported paths list: Tenant1 L3-10001
AS-Path: NONE, path sourced internal to AS
203.0.113.2 (metric 81) from 192.0.2.11 (192.0.2.1)
Origin IGP, MED not set, localpref 100, weight 0
Received label 10002 10001
Extcommunity: RT:65000:10001 RT:65000:10002 ENCAP:8 Router MAC:00ad.f3bb.1b08
Originator: 192.0.2.4 Cluster list: 192.0.2.1
Path-id 1 not advertised to any peer
Route Distinguisher: 192.0.2.3:3 (L3VNI 10001)
BGP routing table entry for [2]:[0]:[0]:[48]:[0050.56ad.f48d]:[32]:[10.0.0.22]/272, version 1277
Paths: (1 available, best #1)
Flags: (0x000202) (high32 00000000) on xmit-list, is not in l2rib/evpn, is not in HW
Advertised path-id 1
Path type: internal, path is valid, is best path, no labeled nexthop
Imported from 192.0.2.4:32787:[2]:[0]:[0]:[48]:[0050.56ad.f48d]:[32]:[10.0.0.22]/272
AS-Path: NONE, path sourced internal to AS
203.0.113.2 (metric 81) from 192.0.2.11 (192.0.2.1)
Origin IGP, MED not set, localpref 100, weight 0
Received label 10002 10001
Extcommunity: RT:65000:10001 RT:65000:10002 ENCAP:8 Router MAC:00ad.f3bb.1b08
Originator: 192.0.2.4 Cluster list: 192.0.2.1
Path-id 1 not advertised to any peer
The 10.0.0.22/32 route gets imported into VRF Tenant-1:
Leaf1# show ip route vrf Tenant1
IP Route Table for VRF "Tenant1"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>
10.0.0.22/32, ubest/mbest: 1/0
*via 203.0.113.2%default, [200/0], 00:34:14, bgp-65000, internal, tag 65000, segid: 10001 tunnelid: 0xcb007102 encap: VXLAN
Leaf1# show ip route vrf Tenant1
IP Route Table for VRF "Tenant1"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>
10.0.0.22/32, ubest/mbest: 1/0
*via 203.0.113.2%default, [200/0], 00:34:14, bgp-65000, internal, tag 65000, segid: 10001 tunnelid: 0xcb007102 encap: VXLAN
By advertising MAC and IP in route type 2, routing between hosts gets enabled automatically. There are some scenarios with silent/shy hosts where this doesn’t work. I will cover that in a future post. That’s all for this one!
first thanks for your excelent acticles regarding to VXLAN/EVPN. I am revising my implementation of VXLAN/EVPN a trying to better understand how does it work. So I do compare your findings with my network/implementation.
I am still not sure that I fully understand how Multiprotocol BGP/different families really works – but as far as I see from your “command output” there are two “RDs for MAC-IP”:
first: Route Distinguisher: 192.0.2.4:32787 — received by BGP
locally proccesed by BGP best path selection proccess
second: Route Distinguisher: 192.0.2.3:3 (L3VNI 10001) — local instance only?/imported
second run of BGP best path selection proccess
But there is another – third one BGP table for RD … for another AFI/SAFI – old good vpnv4/unicast pertaining to the vrf tenant1. Your specific output (I expect) can be shown by command “show bgp vpnv4 unicast 10.0.0.22″
third: **** — imported / potentionally advertised by BGP
second run of BGP best path selection proccess in parallel with L3VNI 10001
In my opinion it is this third BGP table/route that installs route to the rib -vrf tenant1. But if is that true – what is the purpose of L3VNI RD?
This snippet from your command output leads me to the previous assumptions:
” Imported to 2 destination(s)
Imported paths list: Tenant1 L3-10001″
and this one:
“Flags: (0x000202) (high32 00000000) on xmit-list, is not in l2rib/evpn, is not in HW”
The purpose of L3VNI RD is to distinguish the route populated/learned via L3VNI from a regular/vanilla IPv4 route. That is also the reason behind why you can see the output of “show bgp vpnv4 unicast x.x.x.x” in your example above – from router’s perspective, it treat the route learned via L3VNI as an VPN route, which was further imported to the VRF routing table/rib.
Hi Daniel,
first thanks for your excelent acticles regarding to VXLAN/EVPN. I am revising my implementation of VXLAN/EVPN a trying to better understand how does it work. So I do compare your findings with my network/implementation.
I am still not sure that I fully understand how Multiprotocol BGP/different families really works – but as far as I see from your “command output” there are two “RDs for MAC-IP”:
first: Route Distinguisher: 192.0.2.4:32787 — received by BGP
locally proccesed by BGP best path selection proccess
second: Route Distinguisher: 192.0.2.3:3 (L3VNI 10001) — local instance only?/imported
second run of BGP best path selection proccess
But there is another – third one BGP table for RD … for another AFI/SAFI – old good vpnv4/unicast pertaining to the vrf tenant1. Your specific output (I expect) can be shown by command “show bgp vpnv4 unicast 10.0.0.22″
third: **** — imported / potentionally advertised by BGP
second run of BGP best path selection proccess in parallel with L3VNI 10001
In my opinion it is this third BGP table/route that installs route to the rib -vrf tenant1. But if is that true – what is the purpose of L3VNI RD?
This snippet from your command output leads me to the previous assumptions:
” Imported to 2 destination(s)
Imported paths list: Tenant1 L3-10001″
and this one:
“Flags: (0x000202) (high32 00000000) on xmit-list, is not in l2rib/evpn, is not in HW”
The purpose of L3VNI RD is to distinguish the route populated/learned via L3VNI from a regular/vanilla IPv4 route. That is also the reason behind why you can see the output of “show bgp vpnv4 unicast x.x.x.x” in your example above – from router’s perspective, it treat the route learned via L3VNI as an VPN route, which was further imported to the VRF routing table/rib.