I’m currently designing and implementing a large network which will run MPLS.
This network will replace an old network that was mainly L2 based and did not
run MPLS, only VRF lite. There are a few customers that need to have diverse
paths in the network and quick convergence when a failure occurs.
This led me to consider MPLS-TE for those customers and to have plain MPLS
through LDP for other customers buying VPNs. What is the usage for MPLS-TE?
Weaknesses of IGP
When using normal IP forwarding a least cost path is calculated through an IGP,
such as OSPF or ISIS. The problem though is that only the least cost path will
be utilized, any links not on the best path will sit idle, which is a waste of
bandwidth. IGP metrics can be manipulated but that only moves the problem to
other links, it does not solve the root cause. Manipulating metrics is cumbersome
and prone to error. It’s difficult to think of all the traffic flows in the network
and get all the metrics correct. IGPs also lack the granularity in metrics to
utilize all the bandwidth in the network.
RSVP in the past was a protocol used for quality of service in the Intserv model.
It never got a lot of traction due to scalability issues of keeping state in the
core of the network. RSVP was modified to support MPLS and that protocol is known
as RSVP-TE. RSVP-TE provides support for:
- Explicit path configuration
- Path numbering
- Route recording
RSVP assigns labels to the LSPs. The headend of the tunnel sends PATH messages
towards the tailend and then from the tailend back, RESV messages are sent together
with a label to use for the LSP.
Constrained SPF (CSPF)
To overcome the limitations of IGPs mentioned above in the post, the SPF algorithm has
to be modified. This is called Constrained SPF (CSPF) and besides a simple metric it
can take other factors into account such as:
- Administrative weight
- Explicitly defined path
To support this modified SPF algorithm and to carry the information needed in the
LSU/LSP, the IGPs have been modified to support this. OSPF supports TE by using
an opaque LSA. ISIS which is easily modifiable with TLVs, supports TE through a new
TLV. When using ISIS as the IGP, a wide style metric must be used to support TE.
The CSPF path can either be calculated dynamically by the router or the user can
configure an explicit path. Both methods support the use of constraints to build
Routing Across an TE Tunnel
Once the tunnel has been built, traffic must be sent through the tunnel.
This can be achieved in a couple of different ways:
- Static routing
- Dynamic routing
- Policy-based routing
That was a brief overview of MPLS-TE. To test this out in a lab I have setup a
topology like this:
All routers are running IOS except for one router. This is to show the syntax of
IOS-XR and for me to practice using it. IOS1 and IOS6 advertise their loopbacks to
the PE routers. Normal routing has been setup as well as MPLS and BGP. This is the
configuration so far:
router bgp 1 bgp log-neighbor-changes network 220.127.116.11 mask 255.255.255.255 network 18.104.22.168 mask 255.255.255.255 neighbor 22.214.171.124 remote-as 100
ip vrf CUST_1 rd 1:1 route-target export 1:1 route-target import 1:1 ! interface Loopback0 ip address 126.96.36.199 255.255.255.255 ip router isis backbone ! interface Loopback1 ip address 188.8.131.52 255.255.255.255 ip router isis backbone ! interface GigabitEthernet0/0 ip vrf forwarding CUST_1 ip address 184.108.40.206 255.255.255.254 ! interface GigabitEthernet0/1 ip address 220.127.116.11 255.255.255.254 ip router isis backbone mpls ip isis circuit-type level-2-only ! interface GigabitEthernet0/2 ip address 18.104.22.168 255.255.255.254 ip router isis backbone mpls ip isis circuit-type level-2-only ! router isis backbone net 49.0001.0002.0002.0002.0002.00 is-type level-2-only metric-style wide passive-interface default no passive-interface GigabitEthernet0/1 no passive-interface GigabitEthernet0/2 ! router bgp 100 bgp router-id 22.214.171.124 bgp log-neighbor-changes neighbor 126.96.36.199 remote-as 100 neighbor 188.8.131.52 update-source Loopback0 ! address-family vpnv4 neighbor 184.108.40.206 activate neighbor 220.127.116.11 send-community extended exit-address-family ! address-family ipv4 vrf CUST_1 neighbor 18.104.22.168 remote-as 1 neighbor 22.214.171.124 activate neighbor 126.96.36.199 as-override ! mpls ldp router-id Loopback0 force
vrf CUST_1 address-family ipv4 unicast import route-target 1:1 ! export route-target 1:1 ! ! ! interface Loopback0 ipv4 address 188.8.131.52 255.255.255.255 ! interface GigabitEthernet0/0/0/0 ipv4 address 184.108.40.206 255.255.255.254 ! interface GigabitEthernet0/0/0/1 ipv4 address 220.127.116.11 255.255.255.254 ! interface GigabitEthernet0/0/0/2 vrf CUST_1 ipv4 address 18.104.22.168 255.255.255.254 ! route-policy PASS pass end-policy ! router isis backbone is-type level-2-only net 49.0001.0007.0007.0007.0007.00 address-family ipv4 unicast metric-style wide ! interface Loopback0 passive address-family ipv4 unicast ! ! interface GigabitEthernet0/0/0/0 address-family ipv4 unicast ! ! interface GigabitEthernet0/0/0/1 address-family ipv4 unicast ! ! ! router bgp 100 bgp router-id 22.214.171.124 address-family ipv4 unicast ! address-family vpnv4 unicast ! neighbor 126.96.36.199 remote-as 100 update-source Loopback0 address-family vpnv4 unicast ! ! vrf CUST_1 rd 1:1 address-family ipv4 unicast ! neighbor 188.8.131.52 remote-as 1 address-family ipv4 unicast route-policy PASS in route-policy PASS out as-override ! ! ! ! mpls ldp router-id 184.108.40.206 interface GigabitEthernet0/0/0/0 address-family ipv4 ! ! interface GigabitEthernet0/0/0/1 address-family ipv4
interface Loopback0 ip address 220.127.116.11 255.255.255.255 ! interface Loopback1 ip address 18.104.22.168 255.255.255.255 ! router bgp 1 bgp log-neighbor-changes network 22.214.171.124 mask 255.255.255.255 network 126.96.36.199 mask 255.255.255.255 neighbor 188.8.131.52 remote-as 100
The other configuration has been left out, it’s just plain IGP routing and enabling MPLS.
The lab is currently using MPLS VPN and taking the upper path due to the IGP path
being shorter. Let’s confirm this with a traceroute.
IOS1#traceroute 184.108.40.206 numeric source lo0 Type escape sequence to abort. Tracing the route to 220.127.116.11 VRF info: (vrf in name/id, vrf out name/id) 1 18.104.22.168 1 msec 0 msec 0 msec 2 22.214.171.124 [MPLS: Labels 24/16012 Exp 0] 8 msec 9 msec 4 msec 3 126.96.36.199 [MPLS: Label 16012 Exp 0] 23 msec 5 msec 8 msec 4 188.8.131.52 2 msec * 2 msec
The traceroute confirms this.
MPLS-TE tunnels are always unidirectional. To configure MPLS-TE we need to
go through the following steps:
- Enable CEF (default)
- Enable TE support in IGP
- Enable MPLS-TE tunnels globally
- Enable MPLS-TE tunnels on interface(s) in path
- Enable RSVP on interface(s) in path
The following configuration is added to all IOS routers:
IOS2(config)#mpls traffic-eng tunnels IOS2(config)#int range gi0/1 - 2 IOS2(config-if-range)#mpls traffic-eng tunnels IOS2(config-if-range)#ip rsvp bandwidth IOS2(config-if-range)#router isis backbone IOS2(config-router)#metric-style wide IOS2(config-router)#mpls traffic-eng level-2 IOS2(config-router)#mpls traffic-eng router-id lo0
The following configuration is added to the IOS-XR router:
router isis backbone address-family ipv4 unicast metric-style wide mpls traffic-eng level-2-only mpls traffic-eng router-id Loopback0 rsvp interface GigabitEthernet0/0/0/0 ! interface GigabitEthernet0/0/0/1 ! mpls traffic-eng interface GigabitEthernet0/0/0/0 ! interface GigabitEthernet0/0/0/1
This is enough to have ISIS support MPLS-TE. From the IOS-XR router we can
see that the IGP is now carrying more information in the LSPs.
RP/0/0/CPU0:ios#show mpls traffic-eng topology Sun Aug 24 17:16:57.583 UTC My_System_id: 0007.0007.0007.00 (IS-IS backbone level-2) My_BC_Model_Type: RDM Signalling error holddown: 10 sec Global Link Generation 19 IGP Id: 0002.0002.0002.00, MPLS TE Id: 184.108.40.206 Router Node (IS-IS backbone level-2) Link:Broadcast, DR:0002.0002.0002.01, Nbr Node Id:12, gen:10 Frag Id:0, Intf Address:220.127.116.11, Intf Id:0 Nbr Intf Address:0.0.0.0, Nbr Intf Id:0 TE Metric:10, IGP Metric:10 Attribute Flags: 0x0 Ext Admin Group: Length: 256 bits Value : 0x:: Attribute Names: Switching Capability:None, Encoding:unassigned BC Model ID:RDM Physical BW:1000000 (kbps), Max Reservable BW Global:750000 (kbps) Max Reservable BW Sub:0 (kbps) Global Pool Sub Pool Total Allocated Reservable Reservable BW (kbps) BW (kbps) BW (kbps) --------------- ----------- ---------- bw: 0 750000 0 bw: 0 750000 0 bw: 0 750000 0 bw: 0 750000 0 bw: 0 750000 0 bw: 0 750000 0 bw: 0 750000 0 bw: 0 750000 0
The next step is to create the tunnel. We will build a tunnel from IOS2 to XR1.
interface Tunnel0 ip unnumbered Loopback0 tunnel mode mpls traffic-eng tunnel destination 18.104.22.168 tunnel mpls traffic-eng autoroute announce tunnel mpls traffic-eng path-option 10 dynamic
interface tunnel-te0 ipv4 unnumbered Loopback0 autoroute announce destination 22.214.171.124 path-option 10 dynamic
Autoroute announce is used to advertise the tunnel into the IGP. Another option
would be a static route for the tunnel destination across the tunnel interface.
The tunnel is now up and traffic is forwarding across it.
IOS1#traceroute 126.96.36.199 numeric so lo0 Type escape sequence to abort. Tracing the route to 188.8.131.52 VRF info: (vrf in name/id, vrf out name/id) 1 184.108.40.206 1 msec 0 msec 0 msec 2 220.127.116.11 [MPLS: Labels 19/16012 Exp 0] 8 msec 1 msec 3 msec 3 18.104.22.168 [MPLS: Label 16012 Exp 0] 2 msec 3 msec 7 msec 4 22.214.171.124 9 msec * 3 msec
The label 19 is the label used for the tunnel and label 16012 is the VPN label.
We are still using the upper path though. Let’s configure an explicit path to
use the lower path of the topology.
ip explicit-path name TO_XR1 enable next-address 126.96.36.199 next-address 188.8.131.52 next-address 184.108.40.206 ! interface Tunnel0 tunnel mpls traffic-eng path-option 1 explicit name TO_XR1
explicit-path name TO_IOS2 index 1 next-address strict ipv4 unicast 220.127.116.11 index 2 next-address strict ipv4 unicast 18.104.22.168 index 3 next-address strict ipv4 unicast 22.214.171.124 interface tunnel-te0 path-option 1 explicit name TO_IOS2
We try a traceroute from IOS1 to see if the traffic is following the lower path.
IOS1#traceroute 126.96.36.199 numeric so lo0 Type escape sequence to abort. Tracing the route to 188.8.131.52 VRF info: (vrf in name/id, vrf out name/id) 1 184.108.40.206 1 msec 0 msec 0 msec 2 220.127.116.11 [MPLS: Labels 19/16012 Exp 0] 5 msec 2 msec 2 msec 3 18.104.22.168 [MPLS: Labels 21/16012 Exp 0] 2 msec 2 msec 4 msec 4 22.214.171.124 [MPLS: Label 16012 Exp 0] 1 msec 3 msec 1 msec 5 126.96.36.199 3 msec * 2 msec
This is the power of MPLS-TE. The ability to from the headend, define where
the traffic should go. This is essentially source routing. There are also new
features coming out such as segment routing which does something similar by
extending OSPF and ISIS and generating labels through the IGP.
What happens if the explicit path goes down? We can use a dynamic path as a
fallback by setting it to a higher ID in our tunnel configuration.
This is the current configuration of the tunnel:
interface Tunnel0 ip unnumbered Loopback0 tunnel mode mpls traffic-eng tunnel destination 188.8.131.52 tunnel mpls traffic-eng autoroute announce tunnel mpls traffic-eng path-option 1 explicit name TO_XR1 tunnel mpls traffic-eng path-option 10 dynamic
We will initiate a ping from IOS1, then I will shutdown IOS4 interface towards IOS3.
Traffic should then go over the upper path again.
IOS1#ping 184.108.40.206 so lo0 re 100000 Type escape sequence to abort. Sending 100000, 100-byte ICMP Echos to 220.127.116.11, timeout is 2 seconds: Packet sent with a source address of 18.104.22.168 !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Success rate is 99 percent (2876/2878), round-trip min/avg/max = 1/3/22 ms IOS1#
Only a single packet was lost. We confirm on IOS2 that the secondary path
is being used.
IOS2#show mpls traffic-eng tunnels Name: IOS2_t0 (Tunnel0) Destination: 22.214.171.124 Status: Admin: up Oper: up Path: valid Signalling: connected path option 10, type dynamic (Basis for Setup, path weight 20) path option 1, type explicit TO_XR1
IOS1 is using the upper path again.
IOS1#traceroute 126.96.36.199 numeric so lo0 Type escape sequence to abort. Tracing the route to 188.8.131.52 VRF info: (vrf in name/id, vrf out name/id) 1 184.108.40.206 2 msec 0 msec 0 msec 2 220.127.116.11 [MPLS: Labels 19/16012 Exp 0] 8 msec 8 msec 6 msec 3 18.104.22.168 [MPLS: Label 16012 Exp 0] 2 msec 6 msec 10 msec 4 22.214.171.124 1 msec * 2 msec
MPLS-TE can help overcome some of the limitations that come bundled with
IGPs, such as not utilizing links and not being able to consider constraints,
solely relying on a metric that says nothing about the bandwidth available.
MPLS-TE can be used to provide fast convergence by defining several path options.
It can also be combined with FRR to provide convergence times around 50ms.