The Customer Edge And Provider Edge

Published Date: 02 Nov 2017

Abstract

This document describes method of Service providers using "MPLS VPN" technology in backbone to provide VPN services to their customers. In this ``Peer Model`` is implemented in which customer edge devices (CE) provides routing information to the PE and CE routers do not itself participate in routing. Data packets are traversed via tunneled through the backbone, Backbone routers do not need to identify about the VPNs.

Introduction

Service provider can provide the public network to customers to access resources situated at other site while maintaining the private network. Service provider mostly uses "peer model" in which CE routers send their route to PE routers. As most of service provider uses BGP to exchange routes of a particular VPN among PE routers which are attached to particular VPNs so, in this document I will be using the same. This is performed in such a way that two distinct VPNs remain distinct even though they have overlapped address space. Each route for every VPN is assigned a Multiprotocol Label Switching (MPLS) label; BGP distributes a VPN route with MPLS label for that route as well. MPLS packet is further encapsulated so that it tunneled through backbone to proper PE router.

1.1 Virtual Private Network

Virtual Private Networks (VPNs) are a method of interconnecting multiple sites belonging to a customer using a Service Provider (SP) backbone network in place of dedicated leased lines. Each customer site is directly connected to the SP backbone. The SP can offer a VPN service more economically than if dedicated private WANs are built by each individual customer because the SP can share the same backbone network resources (bandwidth, redundant links) between many customers. The customer also gains by outsourcing the complex task of planning, provisioning and managing a geographically distributed network to the SP.

Figure 1: Two LSP in network providing VPN service with MPLS

Four types of VPN:

• Virtual Leased Lines (VLL) provide connection-oriented point-to-point links between customer sites. The customer perceives each VLL as a dedicated private (physical) link, although it is, in fact, provided by an IP tunnel across the backbone network The IP tunneling protocol used over a VLL must be capable of carrying any protocol that the customer uses between the sites connected by that VLL.

• Virtual Private LAN Segments (VPLS) provide an emulated LAN between the VPLS sites. As with VLLs, a VPLS VPN requires use of IP tunnels that are transparent to the protocols carried on the emulated LAN. The LAN may be emulated using a mesh of tunnels between the customer sites or by mapping each VPLS to a separate multicast IP address.

• Virtual Private Routed Networks (VPRNs) emulate a dedicated IP-based routed network between the customer sites. Although a VPRN carries IP traffic, it must be treated as a separate routing domain from the underlying SP network, as the VPRN is likely to make use of non-unique customer-assigned IP addresses. Each customer network perceives itself as operating in isolation and disjoint from the Internet – it is, therefore, free to assign IP addresses in whatever manner it likes. These addresses must not be advertised outside the VPRN since they cannot be guaranteed to be unique more widely than the VPN itself.

• Virtual Private Dial Networks (VPDNs) allow customers to outsource to the SP the provisioning and management of dial-in access to their networks. Instead of each customer setting up their own access servers and using PPP sessions between a central location and remote users, the SP provides a shared, or very many shared access servers. PPP sessions for each VPDN are tunneled from the SP access server to an access point into each customer’s network, known as the access concentrator.

MPLS uses "label switching" to forward data through the network. A small, fixed-format label is inserted in front of each data packet on entry into the MPLS network. Every hop in the backbone forward packet based on incoming interface and label and dispatched it with a new label.

1.2 Customer Edge and Provider Edge

There are different ways in which devices can connect with each other like PPP, ATM virtual circuits, Frame Relay, VLANs on Ethernet interfaces, L2TP, IPsec tunnels, etc. Devices like routers or end systems on customer site are called CE devices and routers present on providers’ side are called Provider’s devices. PEs is the devices (routers) which are attached to the CE devices and backbone routers are called "P" routers.

The attachment circuits over which packet traverse from CE to PE is packet’s ingress attachment circuit and PE as packet’s ingress PE and in opposite direction from PE to CE is packet’s "egress attachment circuit" and PE as the packet’s "egress PE"

1.3 Service provider Routers

Backbone routers do not need to maintain routing information for all the VPN supported by SP. It is maintained by the PE routers attached to particular VPN.

1.4 Security

VPNs of this sort are intended to provide a level of cryptographic techniques when layer 2 backbone (for e.g. frame relay) is used. This does not let two VPNs to interconnect even mistakenly they are misconfigured. Though for the security point of view IPsec tunnels are used in MPLS network in order to provide integrity and confidentiality of data over the backbone.

1.5 Virtual Routing and Forwarding

Internet service providers (ISPs) take advantage of VRF to create separate virtual private networks (VPNs) for customers; so thus called as VPN routing and forwarding. VRF acts like a logical router, which may include many routing tables. Moreover, VRF requires a forwarding table that points towards the next hop for each data packet, and rules and routing protocols that decides forwarding of packets. These tables prevent traffic from being forwarded outside a specific VRF path and keep out traffic that should be outside the VRF path.

Tunnels and Label Stacks

A key feature of MPLS, when considering VPNs, is that once the labels required for an LSP have been exchanged between the LSRs, intermediate LSRs transited by the LSP do not need to examine the content of the data packets flowing on the LSP. So, LSPs are considered to form tunnels across the backbone (MPLS) network.

In Fig.1, LSPs act as tunnels. LSR B forwards the packets based on label attached to each packet. An egress LSR may distribute labels for multiple FECs and set up multiple LSPs. These LSPs can be routed parallel or together in the network.

Fig. 2: Label Stacks across the backbone

Route Distinguisher

Route Distinguisher is simply a number; its purpose is mainly to allow one to create different routes to a common IPv4 address prefix. Other attributes are used to determine where to redistribute the route. Route Distinguisher usually makes the client’s addressing distinctive in the network and for attaining load balancing. This allows BGP to install multiple routes to same system in the MPLS-VPN network.

The RDs are managed so that every Service Provider can maintain its own "numbering space" without any confliction with the RD arrangements made by any other Service Provider. An RD consists of three fields: a 2-byte type field, an administrator field, and an assigned number field. The value of the type field determines the lengths of the other two fields, as well as the semantics of the administrator field. The administrator field identifies an assigned number authority, and the assigned number field contains a number that has been assigned, by the identified authority, for a particular purpose.

2.2 Route Distribution among PEs by BGP

Suppose two sites of a VPN attach to PEs that is in the same AS, the PEs can dispense VPN-IPv4 routes to each other with help of an IBGP connection among them. Each can have an IBGP connection to a route reflector. When a PE router distributes a VPN-IPv4 route via BGP, it allocates its own address as "BGP next hop". This address is determined as a VPN-IPv4 address with RD of 0. It also allocates and issues an MPLS label (PE routers not dispense VPN-IPv4 routes, just Labeled VPN-IPv4 routes). When PE processes a received packet that has corresponding label at the top of the stack, the PE will pop it up, and process the packet properly.

Route Reflectors

Use of Route Reflectors improves scalability instead of full-mesh capability. Route reflectors are the only devices that need to have routing information of the VPNs to which they are not directly attached. However, there is no need to have any one route reflector know all the VPN-IPv4 routes for all the VPNs supported by the backbone.

Route Target

It is a 64-bits BGP community used for tagging prefixes. When distributing prefixes from the VRF, we add prefixes to Route-Target community, so that the PE at the remote site has to import prefixes into the VRF, it can easily recognize which prefixes to import.

A RT attribute can be assumed as identifying a set of sites (VRFs.) Relating a particular Route Target attribute with a route allows that route to be put into the VRFs for routing traffic which is received from the matching site.

The set of RT that a PE router attaches to a route received from site S; known as "Export Targets". And set of Route Targets that a PE router uses to determine whether a route received from another PE router placed in the VRF associated with site S; known as "Import Targets".

Building VPNs Using Route Targets

By managing the Import Targets and Export Targets, we can make different kinds of VPNs.

Let us suppose we want to create a fully mesh closed network group. Then every site is related with a VRF, a single RT attribute is selected, that RT is assigned to each VRF as both the Import Target and the Export Target, and RT.

And not assigned to any other VRF. For, "hub and spoke" kind of VPN, we have to use two Route Target values, one for "Hub" and one for "Spoke". VRFs connected to the hub sites, "Hub" is Export Target and "Spoke" is Import Target. At the VRFs attached to the spoke site, "Hub" is the Import Target and "Spoke" is the Export Target.

Forwarding

When a PE router receives an IP packet from a CE device, it selects corresponding VRF in which to look up the packet’s destination address. Its judgment is based on the packet’s ingress attachment circuit.

Suppose backbone supports MPLS:

- The PE routers (and any ASBR) that redistribute VPN-IPv4 addresses insert /32 subnet masks address into the IGP routing tables of the backbone for themselves. MPLS, at each node in the backbone network assign a label corresponding to the route to each PE.

- In case of traffic engineering tunnels to the BGP next hop, to select one of tunnels associated with an MPLS label, the "tunnel label". The tunnel label gets pushed on the MPLS label stack, and the packet is forwarded to the tunnel’s next hop.

- Otherwise,

* The packet will be destined to "IGP Next Hop".

* If the BGP Next Hop and the IGP Next Hop are the same, and hop popping is used, the packet is then sent to the IGP Next Hop, transporting only with the VPN route label.

* Else, the IGP Next Hop will have assigned a label for the route that best matches the address of the BGP Next Hop. Call this the "tunnel label". The tunnel label gets pushed on as the packet’s top label stack. The packet is then sent to the IGP Next Hop.

- Then MPLS will carry the packet across the backbone to the BGP Next Hop, where the VPN label will be inspected. If the network cloud does not support MPLS, the MPLS packet carrying only the VPN route label may be tunneled to the BGP Next Hop. When the packet emerges from the tunnel, it will be at the BGP Next Hop, where the VPN route label will be examined. At the BGP Next Hop, the processing of the packet depends on the VPN route label.

Maintaining Proper Isolation of VPNs

For isolation of one VPN from another, it is important that no router in the backbone accept a tunneled packet from outside the backbone, unless it is sure that both endpoints of that tunnel are outside the backbone. If MPLS is being used as the tunneling technology, this means that a router in the backbone MUST NOT accept a labeled packet from any adjacent non-backbone device unless the following two conditions hold:

The label at the top of the label stack was actually distributed by that backbone router to that non-backbone device, and

2. The backbone router can determine that use of that label will cause the packet to leave the backbone before any labels lower in the stack will be inspected, and before the IP header will be inspected.

Route Learning by PE router from CE

The PE routers that attach to a particular VPN need to know, for each attachment circuit leading to that VPN, which of the VPN’s addresses should be reached over that attachment circuit. The PE translates these addresses into VPN-IPv4 addresses, using a configured RD. The PE then treats these VPN-IPv4 routes as input to BGP. Routes from a VPN site are NOT leaked into the backbone’s IGP. Exactly which PE/CE route distribution techniques are possible depends on whether or not a particular CE is in a "transit VPN". A "transit VPN" is one that contains a router that receives routes from a "third party" (i.e., from a router that is not in the VPN, but is not a PE router) and that redistributes those routes to a PE router. A VPN that is not a transit VPN is a "stub VPN". The vast majority of VPNs, including just about all corporate enterprise networks, would be expected to be "stubs" in this sense.

The possible PE/CE distribution techniques are:

Static routing (useful in stub networks)

If CE and PE are RIP routing peers then should obey "Split Horizon" rule as well.

In case of OSPF peers. A PE router which is an OSPF peer of a CE router seems, to the CE router, to be an area 0 router. If a PE router is an OSPF peer of CE routers that are in distinct VPNs, the PE must run multiple instances of OSPF.

IPv4 routes that the PE learns from the CE via OSPF are redistributed into BGP as VPN-IPv4 routes using extended Community attributes. All the information is required to enable the route to be distributed to other CE routers in the VPN in the proper type of OSPF Link State Advertisement (LSA). OSPF route tagging is used to ensure that routes received from the MPLS/BGP backbone are not sent back into the backbone.

The PE and CE routers may be BGP peers, and the CE router may use BGP (in particular, EBGP to tell the PE router the set of address prefixes that are at the CE router’s site.

How CEs Learn Routes from PEs

If the PE places a particular route in the VRF it uses to route packets received from a particular CE, then PE may distribute that route to the CE. The PE may distribute that route to the CE only if this is permitted by the protocol. There is restriction on distribution of routes from PE to CE: if a route’s originating point identifies a particular site, that route must never be redistributed to any CE at that site.

Operational Details

MPLS VPN Model:

Figure 3: MPLS VPN model

Some Important aspects:

- The CE routers should distribute only internal routes to the PE routers.

- The PE routers distribute labels for the routes they distribute to the CE routers. The PE routers should not distribute the same label to two different CEs unless one of the following conditions is true:

* The two CEs are related with the same set of VRFs;

* The PE upholds a different Incoming Label Map for each and every CE.

- PE Routers at the different sites should establish BGP.

- All the routes must be well-known to the CE routers.

Then when a Customer Edge router looks up a packet’s ip destination address, the routing lookup will resolve to an address, usually the address of the packet’s BGP next hop. The Customer Edge labels the packet properly and sends the packet to the Provider Edge. The PE uses the packet’s top MPLS label to select the BGP next hop. So, if the BGP next hop is more than one hop away, the top label will be switched by two labels, a tunnel label and a VPN route label. If the BGP next hop is one hop away, the top label may be swapped by just the VPN route label. When the packet is sent from its egress PE to a CE, the packet will have one fewer MPLS labels than it had when it was first received by its ingress PE.

Applicability of MPLS to VPN Types

MPLS LSP tunnels can be used to provide all or part of an implementation of any of the four types of VPN. The suitability of an MPLS solution to each VPN type is described below, including the scalability and management challenges such solutions present.

According to VPN types discussed above in section 1:

MPLS for VLL

In this each point-to-point VLL is provisioned as an LSP tunnel between the appropriate customer sites. The customer is explicitly looking for the equivalent of leased lines, so it is very important that the SP meets any bandwidth guarantees in the SLA. This means that the LSP tunnels used in a VLL solution may have to be dedicated to that specific customer rather than multiplexing the VLL traffic with other VPNs. It is also possible to subdivide the resources of an outer tunnel to provide the Quality of Service for inner LSPs.

MPLS for VPLS

The most immediately obvious means of implementing a VPLS is to map the LAN segment to a unique IP multicast address perhaps using IP encapsulation of the VPN IP traffic. Such a solution could use existing IP multicast technologies, rather than MPLS. Indeed, such approaches are offered by many ISPs today.

However, technologies such as MOSPF and (non-labels) RSVP do not provide the full TE capabilities of MPLS, so the SP has less control over how the VPLS traffic is routed across the backbone network.

Very large SPs with many VPLS customers may also eventually find that there are too few administratively scoped IPv4 multicast addresses to represent each of the VPN LAN segments that they need to support, forcing them either to move to IPv6 or to multiplex several VPLSs on one multicast address. There are 224 administratively scoped IP multicast addresses (239./8), but an SP may well wish to reserve only a portion of this address space for VPN services.

Current MPLS label distribution protocols are specified for unicast destination IP addresses only. This means that an MPLS-based implementation of a VPLS is, necessarily for now, based on one of the following network topologies.

• A full mesh of LSP tunnels connecting the customer sites, with each SP edge LSR responsible for the fan-out to all peers.

• Point-to-point or multipoint-to-point LSP tunnel connections to a "hub" LSR that handles fan-out to all sites using point-to-point LSP tunnels.

In both cases, but especially for the mesh of LSP tunnels, the MPLS-based topology may use more network bandwidth in total that the IP-multicast based solution. This is because multiple copies of each packet may be sent across any given link, each copy carried within one of several different LSP tunnels for the VPLS that transit that link. However, SPs may still choose to implement a VPLS using MPLS in order to exploit the TE capabilities of MPLS to give them better control of how the VPLS traffic is routed between SP edge LSRs.

Future standardization work on MPLS may extend the TE capabilities to cover point-to multipoint or multipoint-to-multipoint LSP tunnels. Such an extension would allow MPLS-based implementations of a VPLS to avoid the bandwidth overhead compared to an IP-multicast based implementation.

MPLS for VPRN

LSP tunnels provide an excellent solution to VPRNs. A VPRN is routed, rather than requiring point-to-multipoint connectivity. This means that even if the SP edge routers set up a full mesh of LSP tunnels to all the other SP edge routers for a given VPRN, they can route each packet onto a single LSP tunnel according to the destination address for that packet rather than fanning out copies to all peers for that VPRN. This avoids the bandwidth wastage that can occur when using an MPLS-based VPLS, as described in the previous section.

Note that the routing protocols used on a VPRN are independent of the routing protocols used on the SP backbone. It is perfectly possible for an SP to use OSPF and BGP4 but for a VPN customer to use a much simpler protocol such as RIP.

MPLS VPN Class of Service

Many VPN customers want to receive guaranteed minimum bandwidth on their VPN connections, but it would be inefficient and costly, for both the SP and their customers, to provision fixed bandwidth LSP tunnels that could support the maximum bandwidth needed between all VPN sites. A far better option is to provision the networks with spare capacity over and above the minimum bandwidth requirements and to share the spare capacity in the network between the VPN customers and public Internet traffic. The sharing of spare bandwidth could be on an unequal basis according to the CoS (Gold, Silver, Bronze etc.) that a customer has signed up for.

MPLS supports this style of provisioning. The sharing of spare bandwidth is similar to Diffserv, but this paper deliberately uses the CoS terminology to distinguish the service options available in the network core from the DSCPs used within any one VPN or the public Internet. In fact the Diffserv extensions to MPLS [12] can be used to signal the CoS that a tunnel carries as DSCPs within the SP network, but the interpretation of these DSCPs may well be different to that in the VPNs.

The ingress LSR is responsible for mapping the combination of VPN and DSCP (or equivalent for non-IP customer networks) to the LSP tunnels and CoS used to transport this data across the network core. The original DSCP is encapsulated and carried across the core network to the egress LSR, so the mapping to a different set of CoS in the core is transparent to the customer networks. The CoS range used within the network core is an administrative decision for each SP to make according to the services they wish to be able to offer to their customers. Correctly, this is not standardized for all SPs. If a customer uses multiple SPs, the gateway nodes between the two SP networks must map the CoS ranges used by inter-AS LSP tunnel into the ranges used by each SP.

Figure 4: Separate CoS Mappings per VPN Customer

MPLS VPN Security

Customers expect VPN data to remain private, including the topology and addressing scheme for their network as well as the data carried on the VPN. Historically, VPN implementations based on ATM or Frame Relay VCs have provided this security by virtue of the connection-oriented nature of the physical network. However, the connectionless public IP network cannot provide the same protection, and IP VPNs have relied on cryptographic means to provide security and authentication.

MPLS brings to IP security benefits similar to layer 2 VCs. This means that the customer equipment connected to the VPN does not need to run IPSEC or other cryptographic software, representing a considerable saving for the customer in terms of equipment expense and management complexity. MPLS VPN security is achieved as described below.

• At the ingress SP edge router, all data for a VPN is assigned a label stack that is unique to the VPN destination. This ensures that the data is delivered only to that destination, so data does not leak out of the VPN.

• Any other packet entering the SP network is either routed without the use of

MPLS or is assigned a different label stack, so a malicious third-party cannot insert data into the VPN from outside the SP network.

• SP routers can use the Cryptographic Algorithm MD5, or similar techniques, to protect against insertion of fake labels or LSRs into the label distribution protocols. There are two situations when a customer may still require the use of cryptographic security measures even when using an MPLS VPN solution.

• If the customer data is considered sufficiently sensitive that it must be protected against snooping even from within the SP network, IPSEC or similar cryptographic techniques must be applied to the VPN data before it enters the SP network. In this case the customer retains responsibility for distributing the cryptographic keys.

• When a VPN is served by more than one SP, the SPs may choose to use IPSEC based tunnels to carry the VPN traffic between their networks on the public IP network if a direct MPLS connection between the SPs is not available. In this case, the SPs are responsible for distributing cryptographic keys.

Standardization Efforts

11.1 Outstanding Items

Some further effort is required by the MPLS working group, and other IETF working groups, to define the standards needed for interoperable MPLS-based VPN solutions.

The main outstanding items that need to be resolved are listed below. At the time of writing, some of these efforts are already underway, but none has yet reached the standards track.

11.1.1 VPN ID

The VPN ID format defined in RFC 2685 guarantees uniqueness of VPN IDs, but it requires global knowledge of the VPNs served (and numbered) by an SP to assign these IDs correctly. This may be difficult if the SP network consists of a number of ASs that are administered separately, for example by separate organizations inherited from acquisitions. In this case the AS-assigned IDs from RFC 2547 may be more convenient. These VPN ID formats should either be merged to give a single format, say by modifying the RFC 2547 format to use a single-byte type field and assigning a new OUI-based type equivalent to RFC 2685, or a decision taken to use just one of these formats in all future standardization work (for example when specifying MPLS TE extensions for VPNs).

11.1.2 Overall Approach to MPLS VPNs

Many different options for implementing MPLS-based VPNs have been discussed in this paper, but some make rather better sense than others. Interoperability between router vendors will be achieved more quickly is the spectrum of implementation possibilities is restricted to a few options, such as:

• Outer labels should be per peer and, optionally, CoS. Inner labels should be per VPN.

• VPN route and label distribution using BGP or OSPF overlays.

• A directory-based solution for VPN peers and route determination.

• TE MIB and RSVP/CR-LDP extensions to allow setup of label stacks.

11.1.3 Directory Schema

A standard directory schema should be defined for the definition of VPN membership and routing information. This would allow interoperability between routers from multiple vendors with a single directory.

11.1.4 MPLS TE Extensions

Extensions to RSVP and CR-LDP are required to allow these protocols to be used for signaling outer and inner tunnels for MPLS VPNs:

• A new TLV is needed to carry the VPN ID for a tunnel between the ingress and egress points for the tunnel.

• The TE MIB explicit route hop objects should be extended to allow the specification of the outer tunnel through which a new inner tunnel should be routed.

Summary

MPLS provides a step-change improvement in the scalability and ease of provisioning of VPNs over IP networks. It also offers enhanced CoS support to allow SPs to offer differentiated service levels. By leveraging these MPLS facilities, SPs can offer highly cost-effective and competitive VPN solutions to their customers and maximize bandwidth usage across the core network.

LSP tunnels provide the encapsulation mechanism for VPN traffic. Automatic methods

for determining VPN routes allow the configuration complexity of an MPLS VPN to scale linearly (order(n)) with the number of sites in the VPN, as opposed to geometric (order(n2)) scaling for other IP-tunneling VPN solutions. Best scalability of peer discovery is achieved by overlaying the VPN peer and route discovery using a routing protocol or by use of a directory.

VPN traffic can be multiplexed onto common outer LSP tunnels in order that the number of tunnels scales according to the number of SP edge routers rather than the much larger number

of VPN sites serviced by these routers. This avoids the scalability problems seen in some ATM or Frame Relay VPN solutions by reducing the problem to order (m) where m is the number of LSRs providing access to n VPN sites, and m ≤ n.

Outer LSP tunnels can also be provisioned for different CoS ranges, allowing SPs to customize the way VPN traffic is treated in the network core to match the service levels they wish to make available to customers. This can be combined with bandwidth reservations for certain CoS ranges or particular dedicated LSP tunnels for a specific customer if required by their SLA.

In the short-term, RFC 2547 provides an efficient VPN implementation model. Longer term, a Virtual Router (VR) based implementation is likely to provide easier management of very complex VPN topologies. In the interests of having a single implementation and management model, SPs may also come to use VRs for smaller VPNs despite its lack of efficiency in that case.

The benefits of using MPLS for VPNs will be magnified if SPs have a choice of interoperable multi-vendor equipment that supports the VPN solutions. Standardization efforts are under way in the IETF MPLS Working Group for the technologies required for such solutions. The main challenge over the coming months will be to whittle down the number of different possible approaches for VPN membership determination and VPN/CoS multiplexing to a few generally applicable solutions to maximize interoperability.