CCDE – Carrier Supporting Carrier

Introduction

In the previous post I showed some of the options to interconnect two AS so that a customer can buy a VPN in two different locations from two different SPs. There is another technology called Carrier Supporting Carrier or Carrier of Carriers. This technology is used when a customer buys a circuit from an SP, Internet service or L3 VPN and that SP uses another SP to carry their traffic between the locations. The SP connecting the customer is then the customer carrier and the SP providing the backbone is the backbone carrier. It is also possible to combine CSC with the Inter-AS options in the previous post, I will show an example of this being used in a real life network in the research world.

Carrier Supporting Carrier

CSC is a technology used to expand the reach of a SP by using another SP as transport. The concept is shown in the following diagram.

The customer carrier is providing a service to the customer. It can be an Internet service, MPLS switched or not or an MPLS L3 VPN. The CSC VPN service provides MPLS transport for the customer carrier. It is also sometimes referred to as a hierarchical VPN and is defined in RFC 4364. The CSC-CE is the device located in the customer carriers network, connecting to the backbone carrier. The CSC-PE is the device located in the backbone carriers network connecting to the customer carrier.

The first question someone might ask is “How will this scale if the backbone carrier must carry all the customer routes from the customer carrier?”. That’s a very valid question. The beauty of CSC is that it only requires the IP addresses of the PEs in the customer carrier to be advertised to the backbone carrier. The customer routes never make it into the backbone carrier which makes it a very scalable solution. Each customer carrier will belong to a VRF of its own.

How does CSC assign the labels? There are two options. Either IGP + LDP can be used where the CSC-PE would consider the CSC-CE to be a normal CE and it could use static routing, EIGRP, OSPF, etc to advertise the routes. It’s not common to run IGP + LDP on a link though unless it belongs to the same organization. The standard approach would be to use labelled BGP. Running IGP + LDP in CSC is less risky than in Inter-AS solutions though since it will only populate the VRF and not the global table. The following diagram shows the peerings involved in CSC.

The CSC-CE and CSC-PE exchange routes and labels via eBGP. The CSC-CE routers setup iBGP to propagate routes between the two islands. This can be IPv4 routes or VPNv4 routes. The BGP session could be between PE routers or RRs as well.

This technology can support both VPNs and non VPNs from the customer carrier. The label stack would be deeper for a VPN which may be considered for what MTU to use on the link between the CSC-PE and CSC-CE.

When covering technologies like this it’s rare that you see them deployed in the real world. A friend sent a link to a paper describing a solution based on this technology. It’s a joint effort from national research and education network (NREN), GEANT and NORDUnet. They call it a multi domain VPN (MD-VPN). The NREN will peer eBGP to the GEANT network and send labelled BGP routes.

The NREN can then peer with each other to exchange eBGP VPNv4 routes or IPv4 routes or even use it for L2 VPN. GEANT only needs reachability to the PEs and to receive the labels for those. To scale better, GEANT implemented route reflectors to reduce the number of peerings needed by each NREN and this RR supports both IPv4, VPNv4 and L2 VPNs. It also support the IPv6 address family.

This diagram shows how the NREN would enable iBGP labelled unicast within their AS. It would also be possible to redistribute between BGP and IGP to assign labels.

The next diagram shows the complete solution where the NREN peers with the RR in GEANTs network.

Like I mentioned it is also possible to use L2 VPNs, by using targeted LDP between the two NRENs.

It’s important for the GEANT RR to not modify the next-hop, which would normally be done on eBGP sessions. It’s nice to see that this technology is used in the real life. Once the peerings are in place, it’s quite simple to provide VPNs over this technology.