2006.06.06 NANOG-NOTES MPLS TE tutorial

(still here, just been really busy at work today; will try to finish sending the
notes out tonight. --Matt)

2006.06.06 MPLS TE tutorial
Pete Templin, Nextlink
[slides are at:

He works in a Cisco shop, no JunOs experience
Operator perspective, no logos

Traffic engineering before MPLS
--the "fish" problem.

two parallel paths, one entry router,
one exit router, you end up with all
traffic taking one path, not using the
other path.

IGP metric adjustments
can lead to routing loops
hard to split traffic

No redundancy left over if both paths
filled, but can be good for using 2 out
of 3 paths.

MPLS TE fundamentals

Packets are forwarded based on FIB or LFIB
FIB/LFIBS built based on RIB

TE tunnels;
TE tunnel interface is a unidirectional logical link
from one router to another.
Once the tunnel is configured, a label is assigned for
the tunnel that corresponds to the path through the
MPLS network (LSP)

TE tunnel basics
Once traffic is routed onto the tunnel, the traffic
flows through the tunnel based on the path.
Return traffic could be placed onto
a tunnel going the opposite direction,
or simply routed by IGP

Key terms for TE
router on which the tunnel is configured
destination address of tunnel
router(s) along the path along the tunnel LSP

Basic TE config
mpls traffic-eng tunnels
IGP: must be OSPF or IS-IS
mpls traffic-eng rouer-id Loopback0
mpls traffic-eng <area X | level2>
physical interfaces
mpls ip
mpls traffic-eng tunnels
  tells IGP to share TE info with other TE nodes

interface TunnelX
ip unnumbered loopback0
  borrow the loopbak's address so we can forward traffic
    down the tunnel
tunnel mode mpls traffic-eng
tunnel destination <a.b.c.d>
  tunnel tail
tunnel mpls traffic-eng path-option 10 dynamic
  find a dynamic path through network
    best path
    with sufficient bandwidth
  will discuss path selection in a bit

Where are we at?
Tunnels go from headend to tail end through midpoint
routers over a deterministic path
we know what commands go on a router for the
physical interface
tunnel interface commands

TE and bandwidth
Physical interfaces can be told how much bandwidth can
  be reserved (used)
  ip rsvp bandwidth X X
TE tunenls can be configured with how much bandwidth
  they need:
  tun mpls traff bandw Y
Tunnels will reserve Y bw on outbound interfaces, and
  find a path across the network wth X(unused)>Y BW.
This prevents oversubscription, or at least helps
control it.

You can allow for burst room, but for now we'll stick
with static, non-oversubscribed links.

operators can adjust the tunnel bandwidth values over
time to match changes in traffic.
If tunnels are dynamically placed, the tunnels will
dynamically find a path through the network with
sufficient bandwidth, or will go down.

TE auto-bandwidth magic
Tunnels can be configured to watch their actual traffic
as in "shw int <blah>| inc rate" every five minutes,
and update their reservation to match, at periodic
Dynamic reservations to match the live network
Bandwidth is 'reserved' using RSVP
  but not "saved" for TE
Often buys enough time to identify the surge, see
where the traffic is coming/going.

The number is only a number in control plane; no
actual impact on data plane, no shaping, no control
on real data flows.

tunnel mpls traffic-eng auto-bw frequency Y
each auto-bw tunnel does "sh int" to capture
its rate every 300* seconds
each auto-bw tunnel updates "tunn mpls traff bandwidth X"
  every Y seconds
The config actually changes; this will impact your
RANCID tracking.

It uses highest measured rate during the interval Y

May want to tweak your load-interval, since it's a
decaying function over time; 5 minute is a fairly
smooth value.

May need to tweak config check-in system to avoid
getting flooded with bandwidth adjustments.

TE tunnel basics
router config basics
general concepts about TE and bandwidth
In this case, the shortest path that has X BW available
for reservation
  actually, bw X at or below priority Y, but that's later.

SPF calculations
step 0: create a PATH list and a TENT list
step 1: put "self" on PATH list.
step 2:
step 3: put PATH nodes' neighbors on TENT list
step 4: if TENT list is empty, stop.
step 5:
jump back to step 2:

Example exercise -- calculate router A's best path to
router D using the handout.

CSPF notes
No load sharing is performed within a tunnel; as soon
as a path is found, it wins
CSPF tiebreakers:
lowest IGP cost
largest minimum available bandwidth
lowest hop count
top node on the PATH list

Creating paths -- can be created dynamically,
or statically via static paths.

tunnel mpls traff path-option X dynamic

Explicit paths
paths can be crated manually by explicitly creating
a path
"ip explicit-path name <name?>"
next-address X
next-address Y
tunnel mpls traff path-option X explicit name blah

Paths can be created manually by explicity configuring
a path that excludes an address;
use any link EXCEPT this one.
for backup links, oob links, etc.
can't combine exclude-address and next-address on the
  same explicit path.
Q: if all other paths go away, will an excluded path
still be used?
can't both include and exclude on same path.

A TE tunnel can have multiple path options.
lowest cost path option is attempted
higher-cost paths attempted sequentially
  until a path can successfully established, or failure
Usually best to have a dynamic option as the highest-cost
option, to ensure you don't fall back to IGP (and lose
traffic matrix accounting!)

CSPF checks tunnel path options in sequence for one that
has sufficient information.

OK, we've got tunnels now--how do we share info around
the network?

TE info is shared around using IGP
available bw per interface
  eight priority levels
  high priority tunnels can push low priority tunnels
    out of the way
  some dynamics as far as tunnel vs interface sizing
administrative weight (TE specific IGP metric
Affinity (customizable)

You can configure different interfaces with
bits that indicate territorial restrictions,
or high-latency links, etc. Use the interface
affinity bits, and match/exclude tunnel affinity
bits to include or exclude certain links.

This information is distributed
immediately for "significant" changes
periodically for "insignificant" changes
immediately, if a change causes an error
Significant changes occur when available bandwidth
on an interface passes preset thresholds
customizable with 16 thresholds.
(based on percentages)
Key points where information is passed from device
to device.

Insignificant changes flooded every 3 minutes,
significant flooded immediately, by default.

If a new path (dynamic or static)
appears that's better than the current one
periodically, every hour from tunnel bringup by default
manually "mpls traffic-eng reoptimize"
  when a link comes up
  optional: requires "mpls traff reo events link-up"
   not so good with flapping links, though.

If you have a flapping link, TE tunnels will stay
off that link for about an hour; you have flap
dampening in your network.

Up next, how to put IP traffic on tunnels
static routes "ip route x.x.x.x y.y.y.y tuZ"
policy-based routing
route-map PBR permit 20
  match ip addr ACL
  set ip next-hop tunX

treat this tunnel as though it's a logically directly
connected link to the tunnel tail
updates the local router's RIB/FIB with "tunnelX"
in place of the IGP next-hop; preserves the IGP cost
all the way to the tail.

One line of config per tunnel; it updates the LOCAL
router's RIB/FIB; prior routers not made aware of
this router as a next-hop for the tunnel tail.
This is supported for IS-IS, but can be more
difficult; you increase number of links in your
IGP pretty quickly.

tunn mpls traff autoroute announce
autoroute and load-sharing
parallel tunnels will load-share inversely proportional
to their configured bandwidth
  auto-bandwith can really muck with these values!
load-sharing can be tuned separately with
"tunn mpls traff load-share X"

Limited to CEF 16 buckets, depending on when it
measures, can end with values drifting apart.
If you use same "X" on two tunnels, they will load share

IGPs can load-share over equal-cost paths
BUT TE tunnel cannot load-share over
multiple physical interfaces

TE diagnostics
show ip route X
sh run int tuX
sh ip rsvp reservation
show mpls traff tun suboptimal constr none
shows headend tunnels taking suboptimal paths to the
  tunnel tail (eg, different from IGP best path)

show mpls traff tun
detailed info for all tunnel headends
  bandwidth info (auto-bw)
  MPLS labels, hop by hop path
show mpls traff tun role <head|middle|remote|tail>

remote is non-headend (not originating or ending on
this router)

show ip rsvp interfaces
shows max allocated bw on an interface.

if a tunnel tail is not the egress PE, add
  "mpls ip" to the tunnel configuration
   --adds another label onto the stack.
  If the last router VPN isn't on penultimate router,
   the TE label won't be recognized, it'll be dropped.
   This adds the TE label back on, keeps the LDP label
   as well as the VPN label still intact
Add "mpls ldp discovery directed-hello accept"
   to config
  If you have unidirectional tunnels, that way when
   you're bringing up tunnels LDP info can be exchanged
   as you're going.

RPF calculations are normally based on unicast RIB
Unidirectional TE tunnels cause RPF failures
add "mpls traffic-engineering multicast-intact" to
  IGP config
Bases RPF checks on RIB BEFORE TE tunnels are substituted

templin at templin.org

Swap to looking at operational network for some
troubleshooting views.

About 150-200Mb traffic aggregate, 4 pops,
DAL and Houston,
four parallel DS3s, 2 between core1's and 2
between core2's

30mbit IP RSVP reservation; TE kicks in, and
moves traffic.
Houston intercore links tends to not have traffic
unless TE has kicked in.

Aggressive 15 minute auto recalculations, since
surprises can kick up fairly quickly.

Room heads for cookies at 1526 hours Pacific Time

Gyah!! Huge apologies to Pete, who really works for Texlink.
I used to work at Nextlink, and in taking notes, my fingers
went down their old familiar path a bit too easily.

Again, Pete Templin works at Texlink, not Nextlink--apologies
for that gaffe, Pete. ^_^;;