A Quick Look at MPLS-TE

Introduction

I’m currently designing and implementing a large network which will run MPLS.
This network will replace an old network that was mainly L2 based and did not
run MPLS, only VRF lite. There are a few customers that need to have diverse
paths in the network and quick convergence when a failure occurs.
This led me to consider MPLS-TE for those customers and to have plain MPLS
through LDP for other customers buying VPNs. What is the usage for MPLS-TE?

Weaknesses of IGP

When using normal IP forwarding a least cost path is calculated through an IGP,
such as OSPF or ISIS. The problem though is that only the least cost path will
be utilized, any links not on the best path will sit idle, which is a waste of
bandwidth. IGP metrics can be manipulated but that only moves the problem to
other links, it does not solve the root cause. Manipulating metrics is cumbersome
and prone to error. It’s difficult to think of all the traffic flows in the network
and get all the metrics correct. IGPs also lack the granularity in metrics to
utilize all the bandwidth in the network.

RSVP-TE

RSVP in the past was a protocol used for quality of service in the Intserv model.
It never got a lot of traction due to scalability issues of keeping state in the
core of the network. RSVP was modified to support MPLS and that protocol is known
as RSVP-TE. RSVP-TE provides support for:

• Explicit path configuration
• Path numbering
• Route recording

RSVP assigns labels to the LSPs. The headend of the tunnel sends PATH messages
towards the tailend and then from the tailend back, RESV messages are sent together
with a label to use for the LSP.

Constrained SPF (CSPF)

To overcome the limitations of IGPs mentioned above in the post, the SPF algorithm has
to be modified. This is called Constrained SPF (CSPF) and besides a simple metric it
can take other factors into account such as:

• Bandwidth
• Affinity
• Administrative weight
• Explicitly defined path

To support this modified SPF algorithm and to carry the information needed in the
LSU/LSP, the IGPs have been modified to support this. OSPF supports TE by using
an opaque LSA. ISIS which is easily modifiable with TLVs, supports TE through a new
TLV. When using ISIS as the IGP, a wide style metric must be used to support TE.

The CSPF path can either be calculated dynamically by the router or the user can
configure an explicit path. Both methods support the use of constraints to build
the path.

Routing Across an TE Tunnel

Once the tunnel has been built, traffic must be sent through the tunnel.
This can be achieved in a couple of different ways:

• Static routing
• Dynamic routing
• Policy-based routing

The Lab

That was a brief overview of MPLS-TE. To test this out in a lab I have setup a
topology like this:

All routers are running IOS except for one router. This is to show the syntax of
IOS-XR and for me to practice using it. IOS1 and IOS6 advertise their loopbacks to
the PE routers. Normal routing has been setup as well as MPLS and BGP. This is the
configuration so far:

IOS1:

router bgp 1
 bgp log-neighbor-changes
 network 1.1.1.1 mask 255.255.255.255
 network 11.11.11.11 mask 255.255.255.255
 neighbor 12.12.12.1 remote-as 100

IOS2:

ip vrf CUST_1
 rd 1:1
 route-target export 1:1
 route-target import 1:1
!
interface Loopback0
 ip address 2.2.2.2 255.255.255.255
 ip router isis backbone
!
interface Loopback1
 ip address 22.22.22.22 255.255.255.255
 ip router isis backbone
!
interface GigabitEthernet0/0
 ip vrf forwarding CUST_1
 ip address 12.12.12.1 255.255.255.254
! 
interface GigabitEthernet0/1
 ip address 23.23.23.0 255.255.255.254
 ip router isis backbone
 mpls ip
 isis circuit-type level-2-only
!
interface GigabitEthernet0/2
 ip address 25.25.25.0 255.255.255.254
 ip router isis backbone
 mpls ip
 isis circuit-type level-2-only
!
router isis backbone
 net 49.0001.0002.0002.0002.0002.00
 is-type level-2-only
 metric-style wide
 passive-interface default
 no passive-interface GigabitEthernet0/1
 no passive-interface GigabitEthernet0/2
!
router bgp 100
 bgp router-id 2.2.2.2
 bgp log-neighbor-changes
 neighbor 7.7.7.7 remote-as 100
 neighbor 7.7.7.7 update-source Loopback0
 !        
 address-family vpnv4
  neighbor 7.7.7.7 activate
  neighbor 7.7.7.7 send-community extended
 exit-address-family
 !
 address-family ipv4 vrf CUST_1
  neighbor 12.12.12.0 remote-as 1
  neighbor 12.12.12.0 activate
  neighbor 12.12.12.0 as-override
!
mpls ldp router-id Loopback0 force

XR1:

vrf CUST_1
 address-family ipv4 unicast
  import route-target
   1:1
  !
  export route-target
   1:1
  !
 !
!
interface Loopback0
 ipv4 address 7.7.7.7 255.255.255.255
!
interface GigabitEthernet0/0/0/0
 ipv4 address 47.47.47.1 255.255.255.254
!
interface GigabitEthernet0/0/0/1
 ipv4 address 57.57.57.1 255.255.255.254
!
interface GigabitEthernet0/0/0/2
 vrf CUST_1
 ipv4 address 76.76.76.1 255.255.255.254
!
route-policy PASS
  pass
end-policy
!
router isis backbone
 is-type level-2-only
 net 49.0001.0007.0007.0007.0007.00
 address-family ipv4 unicast
  metric-style wide
 !
 interface Loopback0
  passive
  address-family ipv4 unicast
  !
 !
 interface GigabitEthernet0/0/0/0
  address-family ipv4 unicast
  !
 !
 interface GigabitEthernet0/0/0/1
  address-family ipv4 unicast
  !
 !
!
router bgp 100
 bgp router-id 7.7.7.7
 address-family ipv4 unicast
 !
 address-family vpnv4 unicast
 !
 neighbor 2.2.2.2
  remote-as 100
  update-source Loopback0
  address-family vpnv4 unicast
  !
 !
 vrf CUST_1
  rd 1:1
  address-family ipv4 unicast
  !
  neighbor 76.76.76.0
   remote-as 1
   address-family ipv4 unicast
 route-policy PASS in
    route-policy PASS out
    as-override
   !
  !
 !
!
mpls ldp
 router-id 7.7.7.7
 interface GigabitEthernet0/0/0/0
  address-family ipv4
  !
 !
 interface GigabitEthernet0/0/0/1
  address-family ipv4

IOS6:

interface Loopback0
 ip address 6.6.6.6 255.255.255.255
!
interface Loopback1
 ip address 66.66.66.66 255.255.255.255
!
router bgp 1
 bgp log-neighbor-changes
 network 6.6.6.6 mask 255.255.255.255
 network 66.66.66.66 mask 255.255.255.255
 neighbor 76.76.76.1 remote-as 100

The other configuration has been left out, it’s just plain IGP routing and enabling MPLS.
The lab is currently using MPLS VPN and taking the upper path due to the IGP path
being shorter. Let’s confirm this with a traceroute.

IOS1#traceroute 6.6.6.6 numeric source lo0
Type escape sequence to abort.
Tracing the route to 6.6.6.6
VRF info: (vrf in name/id, vrf out name/id)
  1 12.12.12.1 1 msec 0 msec 0 msec
  2 25.25.25.1 [MPLS: Labels 24/16012 Exp 0] 8 msec 9 msec 4 msec
  3 57.57.57.1 [MPLS: Label 16012 Exp 0] 23 msec 5 msec 8 msec
  4 76.76.76.0 2 msec *  2 msec

The traceroute confirms this.

MPLS-TE tunnels are always unidirectional. To configure MPLS-TE we need to
go through the following steps:

• Enable CEF (default)
• Enable TE support in IGP
• Enable MPLS-TE tunnels globally
• Enable MPLS-TE tunnels on interface(s) in path
• Enable RSVP on interface(s) in path

The following configuration is added to all IOS routers:

IOS2(config)#mpls traffic-eng tunnels 
IOS2(config)#int range gi0/1 - 2
IOS2(config-if-range)#mpls traffic-eng tunnels
IOS2(config-if-range)#ip rsvp bandwidth
IOS2(config-if-range)#router isis backbone
IOS2(config-router)#metric-style wide
IOS2(config-router)#mpls traffic-eng level-2
IOS2(config-router)#mpls traffic-eng router-id lo0

The following configuration is added to the IOS-XR router:

router isis backbone
 address-family ipv4 unicast 
  metric-style wide 
  mpls traffic-eng level-2-only 
  mpls traffic-eng router-id Loopback0 
rsvp 
 interface GigabitEthernet0/0/0/0
 ! 
 interface GigabitEthernet0/0/0/1
 ! 
mpls traffic-eng 
 interface GigabitEthernet0/0/0/0
 ! 
 interface GigabitEthernet0/0/0/1

This is enough to have ISIS support MPLS-TE. From the IOS-XR router we can
see that the IGP is now carrying more information in the LSPs.

RP/0/0/CPU0:ios#show mpls traffic-eng topology 
Sun Aug 24 17:16:57.583 UTC
My_System_id: 0007.0007.0007.00 (IS-IS backbone level-2)
My_BC_Model_Type: RDM 

Signalling error holddown: 10 sec Global Link Generation 19

IGP Id: 0002.0002.0002.00, MPLS TE Id: 2.2.2.2 Router Node  (IS-IS backbone level-2)

  Link[0]:Broadcast, DR:0002.0002.0002.01, Nbr Node Id:12, gen:10
      Frag Id:0, Intf Address:23.23.23.0, Intf Id:0
      Nbr Intf Address:0.0.0.0, Nbr Intf Id:0
      TE Metric:10, IGP Metric:10
      Attribute Flags: 0x0
      Ext Admin Group: 
          Length: 256 bits
          Value : 0x::
      Attribute Names: 
      Switching Capability:None, Encoding:unassigned
      BC Model ID:RDM
      Physical BW:1000000 (kbps), Max Reservable BW Global:750000 (kbps)
      Max Reservable BW Sub:0 (kbps)
                                 Global Pool       Sub Pool
               Total Allocated   Reservable        Reservable
               BW (kbps)         BW (kbps)         BW (kbps)
               ---------------   -----------       ----------
        bw[0]:            0         750000                0
        bw[1]:            0         750000                0
        bw[2]:            0         750000                0
        bw[3]:            0         750000                0
        bw[4]:            0         750000                0
        bw[5]:            0         750000                0
        bw[6]:            0         750000                0
        bw[7]:            0         750000                0

The next step is to create the tunnel. We will build a tunnel from IOS2 to XR1.

interface Tunnel0 
 ip unnumbered Loopback0 
 tunnel mode mpls traffic-eng 
 tunnel destination 7.7.7.7 
 tunnel mpls traffic-eng autoroute announce 
 tunnel mpls traffic-eng path-option 10 dynamic 

interface tunnel-te0 
 ipv4 unnumbered Loopback0 
 autoroute announce 
 destination 2.2.2.2 
 path-option 10 dynamic 

Autoroute announce is used to advertise the tunnel into the IGP. Another option
would be a static route for the tunnel destination across the tunnel interface.

The tunnel is now up and traffic is forwarding across it.

IOS1#traceroute 6.6.6.6 numeric so lo0
Type escape sequence to abort.
Tracing the route to 6.6.6.6
VRF info: (vrf in name/id, vrf out name/id)
  1 12.12.12.1 1 msec 0 msec 0 msec
  2 25.25.25.1 [MPLS: Labels 19/16012 Exp 0] 8 msec 1 msec 3 msec
  3 57.57.57.1 [MPLS: Label 16012 Exp 0] 2 msec 3 msec 7 msec
  4 76.76.76.0 9 msec *  3 msec

The label 19 is the label used for the tunnel and label 16012 is the VPN label.
We are still using the upper path though. Let’s configure an explicit path to
use the lower path of the topology.

ip explicit-path name TO_XR1 enable 
 next-address 23.23.23.1 
 next-address 34.34.34.1 
 next-address 47.47.47.1 
! 
interface Tunnel0 
 tunnel mpls traffic-eng path-option 1 explicit name TO_XR1 

explicit-path name TO_IOS2
 index 1 next-address strict ipv4 unicast 47.47.47.0 
 index 2 next-address strict ipv4 unicast 34.34.34.0 
 index 3 next-address strict ipv4 unicast 23.23.23.0 
interface tunnel-te0 
path-option 1 explicit name TO_IOS2

We try a traceroute from IOS1 to see if the traffic is following the lower path.

IOS1#traceroute 6.6.6.6 numeric so lo0
Type escape sequence to abort.
Tracing the route to 6.6.6.6
VRF info: (vrf in name/id, vrf out name/id)
  1 12.12.12.1 1 msec 0 msec 0 msec
  2 23.23.23.1 [MPLS: Labels 19/16012 Exp 0] 5 msec 2 msec 2 msec
  3 34.34.34.1 [MPLS: Labels 21/16012 Exp 0] 2 msec 2 msec 4 msec
  4 47.47.47.1 [MPLS: Label 16012 Exp 0] 1 msec 3 msec 1 msec
  5 76.76.76.0 3 msec *  2 msec

This is the power of MPLS-TE. The ability to from the headend, define where
the traffic should go. This is essentially source routing. There are also new
features coming out such as segment routing which does something similar by
extending OSPF and ISIS and generating labels through the IGP.

What happens if the explicit path goes down? We can use a dynamic path as a
fallback by setting it to a higher ID in our tunnel configuration.
This is the current configuration of the tunnel:

interface Tunnel0
 ip unnumbered Loopback0
 tunnel mode mpls traffic-eng
 tunnel destination 7.7.7.7
 tunnel mpls traffic-eng autoroute announce
 tunnel mpls traffic-eng path-option 1 explicit name TO_XR1
 tunnel mpls traffic-eng path-option 10 dynamic

We will initiate a ping from IOS1, then I will shutdown IOS4 interface towards IOS3.
Traffic should then go over the upper path again.

IOS1#ping 6.6.6.6 so lo0 re 100000
Type escape sequence to abort.
Sending 100000, 100-byte ICMP Echos to 6.6.6.6, timeout is 2 seconds:
Packet sent with a source address of 1.1.1.1 
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Success rate is 99 percent (2876/2878), round-trip min/avg/max = 1/3/22 ms
IOS1# 

Only a single packet was lost. We confirm on IOS2 that the secondary path
is being used.

IOS2#show mpls traffic-eng tunnels 

Name: IOS2_t0                             (Tunnel0) Destination: 7.7.7.7
  Status:
    Admin: up         Oper: up     Path: valid       Signalling: connected
    path option 10, type dynamic (Basis for Setup, path weight 20)
    path option 1, type explicit TO_XR1

IOS1 is using the upper path again.

IOS1#traceroute 6.6.6.6 numeric so lo0
Type escape sequence to abort.
Tracing the route to 6.6.6.6
VRF info: (vrf in name/id, vrf out name/id)
  1 12.12.12.1 2 msec 0 msec 0 msec
  2 25.25.25.1 [MPLS: Labels 19/16012 Exp 0] 8 msec 8 msec 6 msec
  3 57.57.57.1 [MPLS: Label 16012 Exp 0] 2 msec 6 msec 10 msec
  4 76.76.76.0 1 msec *  2 msec

Conclusion

MPLS-TE can help overcome some of the limitations that come bundled with
IGPs, such as not utilizing links and not being able to consider constraints,
solely relying on a metric that says nothing about the bandwidth available.
MPLS-TE can be used to provide fast convergence by defining several path options.
It can also be combined with FRR to provide convergence times around 50ms.