Every chapter so far has forwarded packets by reading their destination address. This one is about the other way: decide once, at the edge of the network, where a packet should go, write that decision into a short label, and let the core follow labels. It is how most large carrier networks build private networks for customers and steer traffic away from congested links, and segment routing is the modern way to do it without per-path signalling.
Common mix-up: an MPLS label is not an address. Most labels mean something only to the router that handed them out, and only for packets arriving on that router; the next router down the path may use the same number for something completely different. That is why the label changes at every hop. The exception is a segment routing node SID, which is deliberately agreed across a whole domain.
If longest-prefix match and hop-by-hop forwarding are new, start with routing tables on networks.jelia.nyc, and for the idea of one packet carried inside another, VPNs and overlays. This chapter assumes both.
An ordinary IP router makes the same decision independently at every hop: read the destination address, find the longest matching prefix, send the packet to that prefix's next hop. Nothing a router upstream decided is remembered. That is the great strength of IP, because any router can be replaced and the network still works, but it also means the network cannot easily do two things carriers want: send two packets with the same destination along different paths, and carry customer addresses that the core has never heard of.
Multiprotocol Label Switching (MPLS), standardized in RFC 3031 in 2001, splits the job in two. The router where a packet enters the MPLS network, the ingress label edge router (LER), does the full classification once. It decides which forwarding equivalence class (FEC) the packet belongs to, meaning the group of packets that should all be treated the same way, such as "everything heading for the exit router PE2." Then it pushes a short label in front of the IP header. Every label switching router (LSR) in the core looks only at that label, does one exact-match lookup in its label forwarding information base (LFIB), replaces the label with a new one and sends the packet on. The path a labeled packet follows is a label switched path (LSP).
In the 1990s the selling point was speed, because an exact match on a small number was cheaper than longest-prefix match in the hardware of the day. Modern routers do longest-prefix match at line rate (Chapter 2, the lookup problem), so that argument has faded. What keeps MPLS everywhere is the decoupling. Because the ingress chose the label, it can choose a different label for traffic that should take a different path; and because the core never reads the IP header, the core does not need routes for whatever is inside. A carrier's core routers can forward a million customer prefixes while knowing only a few hundred routes to their own loopbacks.
| Term | Who | What it does |
|---|---|---|
| Ingress LER | First MPLS router (often a provider edge, PE) | Classifies the packet into a FEC and pushes one or more labels |
| LSR | Core router (often a provider router, P) | Looks up the top label only; swaps it, or pops it |
| Penultimate LSR | The core router before the exit | Usually pops the top label so the exit has less to do |
| Egress LER | Last MPLS router (another PE) | Removes what is left of the stack and forwards on what is underneath |
RFC 3032 defines the encoding. A label travels as a 32-bit label stack entry placed between the link-layer header and the IP header, which is why MPLS is sometimes called a "shim." On Ethernet, the EtherType 0x8847 tells the receiver a label comes next instead of IPv4 or IPv6. The entry has four fields.
| Bits | Field | Meaning |
|---|---|---|
| 0–19 | Label | The value the next router looks up. 220 = 1,048,576 possible values |
| 20–22 | Traffic Class (TC) | Priority for queueing, like the DSCP field in IP. Called "Experimental" (EXP) in RFC 3032 and renamed by RFC 5462 |
| 23 | Bottom of stack (S) | 1 on the last entry before the IP header, 0 on every entry above it |
| 24–31 | TTL | Time to live, decremented at each LSR so a looping labeled packet still dies |
Labels can be stacked: a packet can carry several entries, and routers act only on the top one. The S bit is how a router knows where the stack ends. Stacking is what makes MPLS VPNs and segment routing possible, as later sections show. The overhead is simple to compute.
The first sixteen label values, 0 to 15, are reserved and never handed out for ordinary forwarding. Four of them matter in daily operation.
| Label | Name | What it means |
|---|---|---|
| 0 | IPv4 Explicit NULL | "Pop me and treat what is underneath as IPv4." Lets the TC bits ride all the way to the egress |
| 1 | Router Alert | Hand the packet to the router's own software for a closer look |
| 2 | IPv6 Explicit NULL | The same as 0, for IPv6 |
| 3 | Implicit NULL | Never appears in a packet. An egress advertises it to ask its upstream neighbor to pop instead of swap |
| 7 | Entropy Label Indicator | The next label is an entropy label for load balancing (RFC 6790) |
| 4–15, others | Special purposes | Assigned and tracked in the IANA special-purpose label registry; 15 is an extension label that opens a further range |
The label stack has no "next protocol" field, unlike Ethernet's EtherType or IP's protocol number. RFC 3032 section 2.2 says the router that pops the bottom label must infer what lies beneath from the label value itself, possibly with a look at the first nibble of what follows. In practice the egress knows because it assigned the label: it advertised that label for an IPv4 FEC, an IPv6 FEC or a particular VPN, so popping it tells it which table to use next. Explicit null labels 0 and 2 exist precisely so the egress can be told "IPv4" or "IPv6" without having assigned anything.
Load balancing has the same problem. A core router hashing flows across equal-cost links (Chapter 14 shows how) would like to hash on IP addresses and ports, but it cannot be sure where the IP header starts. Many routers guess by peeking at the first nibble after the stack, which works until something that is not IP is carried. RFC 6790's entropy label solves it cleanly: the ingress computes a per-flow hash and pushes it as an extra label, preceded by label 7, so every core router can hash on the stack alone.
Everything a router does to a label stack is one of three operations, sometimes two at once. Push adds an entry on top. Swap replaces the top label with another. Pop removes the top entry and exposes whatever is underneath, either another label or the IP header. An LFIB row is just: incoming label, operation, outgoing label, outgoing interface and next hop.
Labels are usually of local significance. When P2 tells P1 "send me packets for PE2 with label 37," that 37 is an entry in P2's table and means nothing to anyone else. P1 has its own label for the same FEC, say 18, which it gave to PE1. So the packet arrives at P1 carrying 18 and leaves carrying 37: the swap translates from one router's numbering to the next.
At the far end there is a small inefficiency. If the egress PE2 receives a labeled packet, it must look up the label, find that the action is "pop," and then do an IP lookup on what is underneath: two lookups for one packet. Penultimate hop popping (PHP) avoids that. PE2 advertises the reserved label 3, implicit null, for its own FEC. The router before it, P3, reads that as "do not swap, pop," and hands PE2 an ordinary IP packet that needs only one lookup. The cost is that the TC bits, which carried the packet's priority through the core, are thrown away one hop early. If the operator needs them all the way to the egress, PE2 advertises explicit null (label 0) instead; P3 swaps to 0, and PE2 pops it with a trivial lookup.
An IP router decrements the IP TTL at every hop, and traceroute depends on it. Inside an LSP the routers decrement the TTL of the label, not the IP header they cannot see. RFC 3443 describes what happens to the IP TTL as a result, in two main models.
When a label TTL reaches zero in the uniform model, the LSR has to send an ICMP "time exceeded" back to the source. It may have no route to that source; in a VPN it certainly does not. RFC 3032 section 2.3.2 gives the trick: copy the label stack onto the ICMP message and send it on down the LSP, so it reaches the egress, which does know how to route it back. Traceroute through such a network therefore shows every core hop with roughly the same round-trip time as the egress, because every reply made the full trip. RFC 4950 adds an ICMP extension that quotes the received label stack, which is how a traceroute can print the label each hop saw. Instrument 1 lets you compare the two models and read the traceroute each would produce.
H1 sends a packet to H2 at 203.0.113.20. PE1 knows from BGP that 203.0.113.0/24 is reached through PE2 (192.0.2.5), so it maps the packet to the FEC "PE2's loopback" and pushes the label P1 gave it. Press Step to move one router at a time and watch the stack. Change how PE2 asks to be reached, the TTL model, and whether the packet belongs to an L3VPN; every table, header and traceroute line below is recomputed from those settings. Pick a router to read its label table.
| In label | Operation | Out label | Next hop | FEC |
|---|
What traceroute 203.0.113.20 from H1 would print, computed by sending probes with TTL 1, 2, 3 … through the same rules:
Something has to fill those label tables. The workhorse is the Label Distribution Protocol (LDP, RFC 5036). Neighboring routers find each other with UDP hellos on port 646 and then hold a TCP session on the same port. Over it, each router binds a local label to every FEC it knows, usually every loopback /32 in the IGP, and tells its neighbors. A router installs, as the outgoing label for a FEC, the label advertised by whichever neighbor is the IGP's next hop for that FEC.
The consequence is that LDP paths follow the IGP's shortest paths exactly. LDP adds labels but not choices: the packet goes wherever OSPF or IS-IS would have sent it. That is perfectly fine for what most networks use LDP for, which is a BGP-free core and VPN transport. It is no help at all for steering traffic, which needs the next protocol.
| LDP choice | Options | Usual setting and why |
|---|---|---|
| Advertisement | Downstream unsolicited, or downstream on demand | Unsolicited: tell every neighbor every binding, without being asked |
| Retention | Liberal, or conservative | Liberal: keep labels from neighbors that are not the current next hop, so a reroute needs no new signalling |
| Control | Independent, or ordered | Either is common. Ordered waits for the downstream label first, so an LSP is never advertised before it works end to end |
LDP binds labels to FECs that exactly match routes in the routing table. Provider edge routers are almost always reached by their /32 loopback addresses, because BGP next hops point at those loopbacks. If an area border summarizes the loopbacks into a shorter prefix, the routers beyond it no longer have an exact /32 route, and by default LDP will not bind a label for an FEC it cannot see. The LSP breaks at the summary even though IP reachability looks fine. This is one of the oldest operational traps in MPLS, and one reason operators keep loopbacks unsummarized or use segment routing, whose node SIDs are carried by the IGP itself.
RFC 2702, the 1999 requirements document for traffic engineering over MPLS, puts the problem bluntly: IGPs based on shortest-path algorithms contribute significantly to congestion, because they optimize a simple additive metric and ignore how much bandwidth is free. Draw two sources feeding one router that has a short path and a long path to the same destination, and you get a picture shaped like a fish. Every flow takes the short path, its links overload, and the long path idles. Raising the metric on the short path just moves the whole herd to the long path.
RSVP-TE (RFC 3209) answers it with explicitly routed, bandwidth-reserving tunnels. The ingress computes a path with constrained shortest path first (CSPF): Dijkstra run only over links that still have enough unreserved bandwidth, using bandwidth figures the IGP floods with traffic engineering extensions. It sends an RSVP PATH message hop by hop along that route, carrying an Explicit Route Object listing every hop and the bandwidth wanted. The tail end replies with a RESV message that travels back the same way; each router reserves the bandwidth and hands its upstream neighbor a label. When the RESV reaches the head, the tunnel is up.
It works, and many networks still run it. The cost is state. Every router along every tunnel holds that tunnel's PATH and RESV state and refreshes it periodically. A full mesh of tunnels among 300 edge routers is 300 × 299 = 89,700 LSPs, and every core router carries a share of them. Instrument 2 is a small fish: route flows by the IGP, steer them by hand, or let CSPF place them, and count the state that steering costs.
Three flows leave R1 and R2 for R8. Every core link holds 10 Gb/s; R7 to R8 holds 40. All IGP metrics are 10, so the upper path through R4 (cost 20 from R3 to R7) always beats the lower one through R5 and R6 (cost 30). Move the sliders to change each flow's rate, switch a flow onto an explicit lower path, or press CSPF to place every flow, largest first, on the shortest path that still has room. Overloaded links turn to the warning color with a dashed edge.
The most common reason a business buys "MPLS" from a carrier is the Layer 3 VPN of RFC 4364. Each customer's routes live in a separate virtual routing and forwarding instance (VRF) on every provider edge router, so two customers can both use 10.0.0.0/8 without colliding. The core P routers know nothing about any customer.
Three ideas make it work. A route distinguisher (RD), an 8-byte value, is glued in front of each customer prefix to make it unique inside BGP, producing a 12-byte VPN-IPv4 address; it only prevents collisions and carries no policy. A route target (RT), a BGP extended community, is the policy: a PE exports a VRF's routes tagged with some RTs and imports into each VRF the routes whose RTs it is configured to accept, which is how hub-and-spoke or shared-services designs are built. And PEs exchange these routes with multiprotocol BGP, each route carrying a VPN label chosen by the advertising PE.
A packet from customer site A to site B therefore gets two labels at PE1. The inner one is PE2's VPN label, which tells PE2 which VRF to look in. The outer one is the transport label that gets the packet to PE2, from LDP, RSVP-TE or segment routing. The core swaps the outer label and never looks under it; with PHP the outer label disappears one hop early, and PE2 sees only the VPN label. Instrument 1's L3VPN setting shows exactly this stack.
Early designs often set the RD and RT to the same value, which hides the fact that they do different jobs. The RD exists only because BGP picks one best path per prefix: if two customers both advertise 10.1.0.0/16, BGP must see them as different prefixes or one would vanish. Giving each VRF on each PE its own RD also lets a route reflector keep both paths of a dual-homed site instead of choosing one, which speeds failover. The RT, on the other hand, decides which VRFs end up with which routes. One VRF can import several RTs and export several more, so a single customer's topology can be a full mesh, a hub with spokes, or a set of sites sharing one services VRF, all without touching the RD.
Segment routing (SR, RFC 8402) keeps RSVP-TE's ability to steer and drops its state. The idea is old, source routing, but the details are new. The ingress, called the headend, writes an ordered list of instructions into the packet: a segment list. Each instruction is a segment, named by a segment identifier (SID). Routers in the middle execute the active segment, and when it is complete they move on to the next one. No router keeps any per-path state; the path exists only in the packets that carry it.
Two kinds of segment do most of the work.
The IGP carries the SIDs: OSPF and IS-IS were extended so that each router floods its own node SID and its adjacency SIDs alongside its links (Chapter 6 covers IS-IS, whose extensions are RFC 8667). There is no LDP and no RSVP. A traffic-engineered path becomes a short list, such as "go to R6, cross the R6 to R7 link, go to R8."
SR-MPLS (RFC 8660) encodes the segment list as an ordinary MPLS label stack, so existing hardware forwards it unchanged. Node SIDs come from a block of labels every router reserves, the segment routing global block (SRGB). A router's node SID is advertised as an index; the label is the SRGB base plus the index. If every router uses the same SRGB, which RFC 8402 strongly recommends, a node SID is the same label everywhere: in this chapter's examples the SRGB starts at 16000, so R5's node SID with index 5 is label 16005 on every router. Adjacency SIDs are ordinary local labels.
The MPLS operations map neatly. The headend pushes the whole list. A router forwarding toward a node SID that is not itself continues the segment by swapping the label for the same value. The penultimate router of a node segment pops it, as in PHP, and the router owning an adjacency SID pops it and sends the packet out of that link. The active segment is always the top label.
Because a node SID describes a destination, not a fixed sequence of links, a failure inside a node segment simply moves the segment onto the new shortest path once the IGP reconverges. Fast reroute builds on the same idea: Chapter 7's TI-LFA repairs a failed link within milliseconds by pushing a short segment list around it. Segment routing policies, defined in RFC 9256, wrap a list into a named object identified by the headend, a color standing for an intent such as "low latency," and an endpoint; BGP routes tagged with that color are steered into the policy automatically.
Hardware can only push and read so many labels; a router advertises this limit as its maximum SID depth. A path with twelve forced turns cannot be encoded if the headend can push only eight labels. Controllers therefore compress paths: rather than listing every link, they use node SIDs wherever the IGP's shortest path already goes the right way, and add an adjacency SID only where the path must deviate. Instrument 3 behaves the same way. Click a link that does not start where your path currently ends, and it adds the node SID needed to get there first, then the adjacency SID.
SRv6 uses the IPv6 data plane instead of MPLS. Each SID is a 128-bit IPv6 address. Its leading bits are a locator, a prefix routed to one router like any other route, and its trailing bits name a function that router performs (RFC 8986). End is the plain node segment, End.X is "end, then cross this adjacency," and service functions such as End.DT4 mean "decapsulate and look up the inner IPv4 packet in this VRF," replacing the VPN label.
The list rides in a Segment Routing Header (SRH, RFC 8754), an IPv6 routing extension header with an 8-byte fixed part followed by the SIDs, stored in reverse order, and a Segments Left counter that points at the active one. The active SID is always copied into the IPv6 destination address. Routers between segment endpoints need no SRv6 support at all: they route on the destination address as usual. At each endpoint, the router decrements Segments Left, copies the next SID into the destination address, and forwards.
The trade-off is size. When a headend steers an IPv4 or IPv6 packet into an SRv6 policy, it wraps it in a new 40-byte IPv6 header (the H.Encaps behavior) and adds an SRH with every segment in it. The reduced variant, H.Encaps.Red, leaves the first SID out of the SRH because it is already in the destination address, and drops the SRH entirely when there is only one segment.
| Segments | SR-MPLS stack | SRv6, H.Encaps | SRv6, H.Encaps.Red | SRv6 vs MPLS |
|---|---|---|---|---|
| 1 | 4 B | 64 B | 40 B | 10 × (reduced) |
| 2 | 8 B | 80 B | 64 B | 8 × |
| 3 | 12 B | 96 B | 80 B | 6.7 × |
| 5 | 20 B | 128 B | 112 B | 5.6 × |
| 8 | 32 B | 176 B | 160 B | 5 × |
Worked example for three segments: SR-MPLS pushes 3 × 4 = 12 bytes. SRv6 with H.Encaps adds the outer IPv6 header (40), the SRH fixed part (8) and 3 × 16 = 48 bytes of SIDs, 96 in all; the reduced form saves one SID, giving 80. On a link with a 1,500-byte MTU, that is the difference between 1,488 and 1,404 bytes left for the customer's packet. The "SRv6 vs MPLS" column compares the reduced form for each row. That gap is why RFC 9800 (2025) defines compressed SRv6 segment lists, which pack several short SIDs into one 128-bit address.
In SR-MPLS an adjacency SID is a local label, so the headend must first bring the packet to the router that owns it with a node SID, then add the adjacency SID: two labels. In SRv6 the End.X SID is a routable address under the owning router's locator, so the packet is routed there by ordinary IPv6 forwarding and the End.X function runs on arrival. One SID does both jobs. Instrument 3 lists segments as you build them, the MPLS way, so its SRv6 byte count is a fair upper bound; a real SRv6 controller would fold those pairs.
R1 is the headend. Click a router to add its node SID, or click a link to add an adjacency SID across it; the numbers on links are IGP metrics. The instrument runs Dijkstra from wherever the path currently ends to each node SID, follows each adjacency SID across exactly its link, and draws the result with the label stack the packet carries on every link. Switch to "fail links" and click a link to break it: node segments reroute, adjacency segments over a dead link cannot.
Segment list, active segment first:
| Link | Active segment | SR-MPLS labels on the wire | SRv6 destination · SL |
|---|
| Question | LDP | RSVP-TE | SR-MPLS | SRv6 |
|---|---|---|---|---|
| Path | IGP shortest only | Any explicit path, with bandwidth reserved | Any path expressible as segments | Same as SR-MPLS |
| Signalling | LDP sessions on every link | PATH and RESV per tunnel, refreshed | None beyond the IGP | None beyond the IGP |
| Per-path state in the core | Per FEC, not per path | Every tunnel, on every router it crosses | None | None |
| Bandwidth guarantee | No | Yes, admission-controlled | Only if a controller accounts for it | Only if a controller accounts for it |
| Bytes per hop of path | 4 per label | 4 per label | 4 per segment | 16 per segment, plus 48 of headers |
| Data plane | MPLS | MPLS | MPLS | IPv6 |
Most operators do not switch overnight. A common migration runs LDP and SR-MPLS side by side, prefers SR labels where both exist, and retires LDP router by router. SRv6 is chosen mostly where an operator wants a single IPv6 data plane from the data center to the wide area, or where it is building a new network without an MPLS history. Data center fabrics use related ideas with different encapsulations; Chapter 14, data center fabrics, covers VXLAN and EVPN.