Introduction

In the previous post I showed some of the options to interconnect two AS so that a customer can buy a VPN in two different locations from two different SPs. There is another technology called Carrier Supporting Carrier or Carrier of Carriers. This technology is used when a customer buys a circuit from an SP, Internet service or L3 VPN and that SP uses another SP to carry their traffic between the locations. The SP connecting the customer is then the customer carrier and the SP providing the backbone is the backbone carrier. It is also possible to combine CSC with the Inter-AS options in the previous post, I will show an example of this being used in a real life network in the research world.

Carrier Supporting Carrier

CSC is a technology used to expand the reach of a SP by using another SP as transport. The concept is shown in the following diagram.

CSC-Overview
CSC-Overview

The customer carrier is providing a service to the customer. It can be an Internet service, MPLS switched or not or an MPLS L3 VPN. The CSC VPN service provides MPLS transport for the customer carrier. It is also sometimes referred to as a hierarchical VPN and is defined in RFC 4364. The CSC-CE is the device located in the customer carriers network, connecting to the backbone carrier. The CSC-PE is the device located in the backbone carriers network connecting to the customer carrier.

The first question someone might ask is “How will this scale if the backbone carrier must carry all the customer routes from the customer carrier?”. That’s a very valid question. The beauty of CSC is that it only requires the IP addresses of the PEs in the customer carrier to be advertised to the backbone carrier. The customer routes never make it into the backbone carrier which makes it a very scalable solution. Each customer carrier will belong to a VRF of its own.

How does CSC assign the labels? There are two options. Either IGP + LDP can be used where the CSC-PE would consider the CSC-CE to be a normal CE and it could use static routing, EIGRP, OSPF, etc to advertise the routes. It’s not common to run IGP + LDP on a link though unless it belongs to the same organization. The standard approach would be to use labelled BGP. Running IGP + LDP in CSC is less risky than in Inter-AS solutions though since it will only populate the VRF and not the global table. The following diagram shows the peerings involved in CSC.

CSC Detailed
CSC Detailed

The CSC-CE and CSC-PE exchange routes and labels via eBGP. The CSC-CE routers setup iBGP to propagate routes between the two islands. This can be IPv4 routes or VPNv4 routes. The BGP session could be between PE routers or RRs as well.

This technology can support both VPNs and non VPNs from the customer carrier. The label stack would be deeper for a VPN which may be considered for what MTU to use on the link between the CSC-PE and CSC-CE.

When covering technologies like this it’s rare that you see them deployed in the real world. A friend sent a link to a paper describing a solution based on this technology. It’s a joint effort from national research and education network (NREN), GEANT and NORDUnet. They call it a multi domain VPN (MD-VPN). The NREN will peer eBGP to the GEANT network and send labelled BGP routes.

MDVPN Overview
MDVPN Overview

The NREN can then peer with each other to exchange eBGP VPNv4 routes or IPv4 routes or even use it for L2 VPN. GEANT only needs reachability to the PEs and to receive the labels for those. To scale better, GEANT implemented route reflectors to reduce the number of peerings needed by each NREN and this RR supports both IPv4, VPNv4 and L2 VPNs. It also support the IPv6 address family.

This diagram shows how the NREN would enable iBGP labelled unicast within their AS. It would also be possible to redistribute between BGP and IGP to assign labels.

MDVPN2
MDVPN2

The next diagram shows the complete solution where the NREN peers with the RR in GEANTs network.

MDVPN3
MDVPN3

Like I mentioned it is also possible to use L2 VPNs, by using targeted LDP between the two NRENs.

MDVPN4
MDVPN4

It’s important for the GEANT RR to not modify the next-hop, which would normally be done on eBGP sessions. It’s nice to see that this technology is used in the real life. Once the peerings are in place, it’s quite simple to provide VPNs over this technology.

CCDE – Carrier Supporting Carrier
Tagged on:                         

12 thoughts on “CCDE – Carrier Supporting Carrier

  • March 5, 2016 at 9:10 am
    Permalink

    Hi Daniel,

    Personally I preferred to adopt eBGP labeled unicast approach. However, any potential issue if both customer carrier and backbone carrier are using private ip addresses as their PE loopback addresses? Is it possible to have conflict of ip addresses?

    Reply
    • March 5, 2016 at 9:59 am
      Permalink

      It should not be an issue since the backbone carrier is carrying the routes of the customer carrier in a VRF. The backbone carriers loopbacks will be in the global table.

      Reply
  • March 5, 2016 at 4:27 pm
    Permalink

    Just a minor nitpick:

    “Either IGP + LDP can be used where the CSC-PE would consider the CSC-CE to be a normal CE and it could use static routing, EIGRP, OSPF, BGP or any protocol to advertise the routes.”

    LDP will not allocate labels for routes learned via BGP 🙂 All the other protocols are valid.

    IGP+LDP can be useful for CSC mLDP (which I have never tested), assuming recursive mLDP FEC is supported (XR only).

    Reply
  • Pingback:Worth Reading: Carrier Supporting Carrier - 'net work

  • March 10, 2016 at 4:45 pm
    Permalink

    Do you know real world implementations of CsC in service providers? For example Telefonica, BT, ¿?

    Reply
    • March 10, 2016 at 4:48 pm
      Permalink

      I have no insight into those organizations. For the ISPs where I know people, I don’t believe that they run CSC.

      Reply
  • October 5, 2016 at 5:46 am
    Permalink

    Hi Daneil
    Thanks can u reply me as I need to deploy CSc in real world

    Reply
  • June 12, 2017 at 12:27 pm
    Permalink

    Hello Imran,

    Did you then manage in the end to deploy it in the real world ?

    cheers
    Andrea

    Reply
  • March 31, 2018 at 6:52 pm
    Permalink

    Hello, I am having a little trouble to figure out a CsC deployment with an Inter- AS option B in the backbone carrier cloud. Could you please elaborate on that?

    Reply
  • March 31, 2018 at 6:53 pm
    Permalink

    Hello, I am having a little trouble to figure out a CsC deployment with an Inter- AS option B in the backbone carrier cloud. Could you please elaborate on that? Thanks

    Reply
  • Pingback:CCDE V2.1 – Study Plan {Bonus Track} – The Network's Down

Leave a Reply to ERIC Cancel reply

Your email address will not be published. Required fields are marked *