Agenda
Prerequisites Introduction How MPLS-TE Works Fast ReRoute Design Demo J
RST-251 5465_05_2002_c1
Copyright 2002, Cisco Systems, Inc. All rights reserved. Printed in USA. 5465_05_2002, RST-251
Prerequisites
RST-251 5465_05_2002_c1
Agenda
Prerequisites Introduction How MPLS-TE Works Fast ReRoute Design Demo J
RST-251 5465_05_2002_c1
Copyright 2002, Cisco Systems, Inc. All rights reserved. Printed in USA. 5465_05_2002, RST-251
IP IP MPLS MPLS
IP+ATM Integration
IP+Optical Integration
Frame Relay Frame Relay
ATM
RST-251 5465_05_2002_c1
Introduction
MPLS-TE was designed to move traffic along a path other than the IGP shortest path
Bring traffic engineering abilities to an IP network Bandwidth-aware connection setup
Copyright 2002, Cisco Systems, Inc. All rights reserved. Printed in USA. 5465_05_2002, RST-251
Some links are DS3, some are OC-3 Router A has 40Mb of traffic for Route F, 40Mb of traffic for Router G Massive (44%) packet loss at Router B->Router E!
Router B
Router A
OC-3
ffic Tra b 80M
Router G
OC-3
Router C
RST-251 5465_05_2002_c1
DS3 DS3
Router D
OC-3
Router A computes paths on properties other than just shortest cost No link oversubscribed!
Router B Router F
Router A
OC-3
b 40M
OC-3
Router E
DS3
40Mb
Router G
OC-3
Router C
RST-251 5465_05_2002_c1
DS3
Router D
OC-3
DS3
Copyright 2002, Cisco Systems, Inc. All rights reserved. Printed in USA. 5465_05_2002, RST-251
Fast ReRoute
FRR: A mechanism to minimize packet loss during a failure Pre-provision protection tunnels that carry traffic when a protected resource (link/node) goes down Use MPLS-TE to signal the FRR protection tunnels, taking advantage of the fact that MPLS-TE traffic doesnt have to follow the IGP shortest path Can protect MPLS traffic or IP traffic, depends on the type of protection See later slides on FRR
RST-251 5465_05_2002_c1
2002, Cisco Systems, Inc. All rights reserved.
Agenda
Prerequisites Introduction How MPLS-TE Works Fast ReRoute Design Demo J
RST-251 5465_05_2002_c1
10
Copyright 2002, Cisco Systems, Inc. All rights reserved. Printed in USA. 5465_05_2002, RST-251
Information distribution Path calculation Path setup Forwarding traffic down tunnels
RST-251 5465_05_2002_c1
11
Information Distribution
Need to tell the network about per-link resources (bandwidth, metrics (IGP and TE)) This is done using extensions to IGP (OSPF, ISIS) EIGRP, RIP not supported for MPLS-TE
EIGRP, RIP will work for other MPLS applications (like VPNs!), just not for TE.
RST-251 5465_05_2002_c1
12
Copyright 2002, Cisco Systems, Inc. All rights reserved. Printed in USA. 5465_05_2002, RST-251
Information Distribution
OSPF
Uses type 10 (opaque arealocal) LSAs See draft-katz-yeung-ospf-traffic
router ospf 1 mpls traffic -eng area <x> mpls traffic -eng router-id Loopback0
RST-251 5465_05_2002_c1
13
Information Distribution
IS-IS
Uses Type 22 TLVs See draft-ietf-isis-traffic
router isis foo mpls traffic -eng level-1|level-2 mpls traffic -eng router-id Loopback0 metric-style wide
RST-251 5465_05_2002_c1
14
Copyright 2002, Cisco Systems, Inc. All rights reserved. Printed in USA. 5465_05_2002, RST-251
Path Calculation
RST-251 5465_05_2002_c1
15
RtrB RtrA
RtrC
RST-251 5465_05_2002_c1
2002, Cisco Systems, Inc. All rights reserved.
RtrD
16
Copyright 2002, Cisco Systems, Inc. All rights reserved. Printed in USA. 5465_05_2002, RST-251
RtrA
RST-251 5465_05_2002_c1
17
RtrB RtrA
RtrC
RST-251 5465_05_2002_c1
2002, Cisco Systems, Inc. All rights reserved.
18
Copyright 2002, Cisco Systems, Inc. All rights reserved. Printed in USA. 5465_05_2002, RST-251
RtrB RtrA
RtrC
RST-251 5465_05_2002_c1
2002, Cisco Systems, Inc. All rights reserved.
RtrD
19
RtrC
RST-251 5465_05_2002_c1
2002, Cisco Systems, Inc. All rights reserved.
RtrD
20
Copyright 2002, Cisco Systems, Inc. All rights reserved. Printed in USA. 5465_05_2002, RST-251
RtrB RtrA
RtrC
RST-251 5465_05_2002_c1
2002, Cisco Systems, Inc. All rights reserved.
RtrD
21
RtrB RtrA
RtrC
RST-251 5465_05_2002_c1
2002, Cisco Systems, Inc. All rights reserved.
RtrD
22
Copyright 2002, Cisco Systems, Inc. All rights reserved. Printed in USA. 5465_05_2002, RST-251
RtrB RtrA
RtrC
RST-251 5465_05_2002_c1
2002, Cisco Systems, Inc. All rights reserved.
RtrD
23
RtrB RtrA
RtrC
RST-251 5465_05_2002_c1
2002, Cisco Systems, Inc. All rights reserved.
RtrD
24
Copyright 2002, Cisco Systems, Inc. All rights reserved. Printed in USA. 5465_05_2002, RST-251
Constrained SPF Find shortest path to a specific node Consider more than just link cost!
RtrF RtrE OC3
OC3
25
Path Calculation
Whats the Shortest Path to Router F with 40Mb Available?
Constrained SPF Find shortest path to a specific node Consider more than just link cost!
RtrA
RST-251 5465_05_2002_c1
26
Copyright 2002, Cisco Systems, Inc. All rights reserved. Printed in USA. 5465_05_2002, RST-251
Path Calculation
Whats the Shortest Path to Router F with 40Mb Available?
RtrB RtrA OC3
Constrained SPF Find shortest path to a specific node Consider more than just link cost!
OC3 RtrC
RST-251 5465_05_2002_c1
2002, Cisco Systems, Inc. All rights reserved.
27
Path Calculation
Whats the Shortest Path to Router F with 40Mb Available?
RtrB RtrA OC3
Constrained SPF Find shortest path to a specific node Consider more than just link cost!
OC3 RtrC
RST-251 5465_05_2002_c1
DS3
RtrD
28
Copyright 2002, Cisco Systems, Inc. All rights reserved. Printed in USA. 5465_05_2002, RST-251
Path Calculation
Whats the Shortest Path to Router F with 40Mb Available?
RtrB RtrA OC3 DS3 OC3 RtrC
RST-251 5465_05_2002_c1
Constrained SPF Find shortest path to a specific node Consider more than just link cost!
RtrE
DS3
RtrD
29
Path Calculation
Whats the Shortest Path to Router F with 40Mb Available?
RtrB RtrA OC3 DS3 RtrG OC3 RtrC
RST-251 5465_05_2002_c1
Constrained SPF Find shortest path to a specific node Consider more than just link cost!
RtrF RtrE OC3
Copyright 2002, Cisco Systems, Inc. All rights reserved. Printed in USA. 5465_05_2002, RST-251
Path Calculation
Whats the Shortest Path to Router F with 40Mb Available?
RtrB RtrA OC3 DS3 RtrG OC3 RtrC
RST-251 5465_05_2002_c1
Constrained SPF Find shortest path to a specific node Consider more than just link cost!
RtrF RtrE OC3
Path Calculation
Whats the Shortest Path to Router F with 40Mb Available?
RtrB RtrA OC3 DS3 RtrE OC3
Constrained SPF Find shortest path to a specific node Consider more than just link cost!
RtrF
RST-251 5465_05_2002_c1
32
Copyright 2002, Cisco Systems, Inc. All rights reserved. Printed in USA. 5465_05_2002, RST-251
Path Calculation
But wait! Theres nothing different between the two SPF results! but
RST-251 5465_05_2002_c1
33
Path Calculation
Whats the Shortest Path to Router G with 40Mb Available?
RtrB 115MB RtrA 5MB RtrG OC3 RtrC
RST-251 5465_05_2002_c1
OC3
34
Copyright 2002, Cisco Systems, Inc. All rights reserved. Printed in USA. 5465_05_2002, RST-251
Path Calculation
Whats the Shortest Path to Router G with 40Mb Available?
RtrA
RST-251 5465_05_2002_c1
35
Path Calculation
Whats the Shortest Path to Router G with 40Mb Available?
RtrB 115MB RtrA
OC3 RtrC
RST-251 5465_05_2002_c1
2002, Cisco Systems, Inc. All rights reserved.
36
Copyright 2002, Cisco Systems, Inc. All rights reserved. Printed in USA. 5465_05_2002, RST-251
Path Calculation
Whats the Shortest Path to Router G with 40Mb Available?
RtrB 115MB RtrA
OC3 RtrC
RST-251 5465_05_2002_c1
DS3
RtrD
37
Path Calculation
Whats the Shortest Path to Router G with 40Mb Available?
RtrB 115MB RtrA 5MB OC3 RtrC
RST-251 5465_05_2002_c1
RtrE
DS3
RtrD
38
Copyright 2002, Cisco Systems, Inc. All rights reserved. Printed in USA. 5465_05_2002, RST-251
Path Calculation
Whats the Shortest Path to Router G with 40Mb Available?
RtrB 115MB RtrA 5MB OC3 RtrC
RST-251 5465_05_2002_c1
RtrE
DS3
RtrD
39
Path Calculation
Whats the Shortest Path to Router G with 40Mb Available?
RtrB 115MB RtrA
OC3 RtrC
RST-251 5465_05_2002_c1
DS3
RtrD
40
Copyright 2002, Cisco Systems, Inc. All rights reserved. Printed in USA. 5465_05_2002, RST-251
Path Calculation
Whats the Shortest Path to Router G with 40Mb Available?
RtrA
OC3 RtrC
RST-251 5465_05_2002_c1
DS3
RtrD
41
Path Calculation
Whats the Shortest Path to Router G with 40Mb Available?
RtrA
RtrE
OC3 RtrC
RST-251 5465_05_2002_c1
Copyright 2002, Cisco Systems, Inc. All rights reserved. Printed in USA. 5465_05_2002, RST-251
Path Calculation
Whats the Shortest Path to Router G with 40Mb Available?
RtrA
RtrE115MB RtrG
OC3 RtrC
RST-251 5465_05_2002_c1
OC3
43
Path Calculation
Whats the Shortest Path to Router G with 40Mb Available?
RtrA
RtrE RtrG
OC3 RtrC
RST-251 5465_05_2002_c1
OC3
44
Copyright 2002, Cisco Systems, Inc. All rights reserved. Printed in USA. 5465_05_2002, RST-251
Path Calculation
Node B C D E F G Next-Hop B C C B Tunnel0 Tunnel1 Cost 10 10 20 20 30 30
End result:
Bandwidth used efficiently!
RtrF
OC3
45
Path Calculation
What if theres more than one path that meets the minimum requirements (bandwidth, etc.)? PCALC algorithm:
Find all paths with the lowest IGP cost Then pick the path with the highest minimum bandwidth along the path Then pick the path with the lowest hop count (not IGP cost, but hop count) Then just pick one path at random (take the top path on the TENT list)
RST-251 5465_05_2002_c1
2002, Cisco Systems, Inc. All rights reserved.
46
Copyright 2002, Cisco Systems, Inc. All rights reserved. Printed in USA. 5465_05_2002, RST-251
Path Calculation
{cost,available BW} {10,100M} Path Has Cost of 25, Not the Lowest Cost!
{8,90M}
{8,90M}
Path Calculation
{cost,available BW} Path Min BW Is Lower than the Other Paths!
{8,90M}
{8,90M}
Copyright 2002, Cisco Systems, Inc. All rights reserved. Printed in USA. 5465_05_2002, RST-251
Path Calculation
{cost,available BW} Hop Count Is 5, Other Paths Are 4!
RtrA {4,90M}
RtrZ
{8,90M}
{8,90M}
Path Calculation
{cost,available BW} Pick a Path at Random!
RtrA
RtrZ
{8,90M}
{8,90M}
Copyright 2002, Cisco Systems, Inc. All rights reserved. Printed in USA. 5465_05_2002, RST-251
Path Calculation
{cost,available BW}
RtrA
RtrZ
{8,90M}
Cisco MPLS-TE uses RSVP RFC2205 (base RSVP), RFC 3209 (TE extensions for RSVP) CR-LDP is dead as per RFC 3468 Once the path is calculated, it is handed to RSVP RSVP uses PATH and RESV messages to request an LSP along the calculated path
RST-251 5465_05_2002_c1
52
Copyright 2002, Cisco Systems, Inc. All rights reserved. Printed in USA. 5465_05_2002, RST-251
Path Setup
PATH message: Can I have 40Mb along this path? RESV message: Yes, and heres the label to use LFIB is set up along each hop
= PATH Messages = RESV Messages
Router B Router A Router E Router G Router F
Router C
RST-251 5465_05_2002_c1
2002, Cisco Systems, Inc. All rights reserved.
Router D
53
Path Setup
PATH message: Can I have 40Mb along this path? RESV message: Yes, and heres the label to use LFIB is set up along each hop
= PATH Messages = RESV Messages
Router B Router A Router E Router G
IMP L NU ICIT LL
Router F
15
23
Router C
RST-251 5465_05_2002_c1
2002, Cisco Systems, Inc. All rights reserved.
39
Router D
54
Copyright 2002, Cisco Systems, Inc. All rights reserved. Printed in USA. 5465_05_2002, RST-251
Path Setup
Errors along the way will trigger RSVP errors May also trigger re-flooding of TE information if appropriate
RST-251 5465_05_2002_c1
55
With all but PBR, MPLS-TE gets you unequal cost load balancing
RST-251 5465_05_2002_c1
2002, Cisco Systems, Inc. All rights reserved.
56
Copyright 2002, Cisco Systems, Inc. All rights reserved. Printed in USA. 5465_05_2002, RST-251
Static Routing
Router F
Router H
Tunnel1
Router G
Router C
RST-251 5465_05_2002_c1
2002, Cisco Systems, Inc. All rights reserved.
Router D
Router 1
57
Static Routing
Node Node Next-Hop Next-Hop Cost Cost B B 10 B B 10 C 10 C C 10 C D C 20 D C 20 E B 20 E B 20 B F B 30 F 30 G B 30 G B 30 H Tunnel 40 H Tunnel 1 1 40 II 40 B 40 B
Router H is known via the tunnel Router G is not routed to over the tunnel, even though its the tunnel tail!
Router F Router E Router H
Router B Router A
Tunnel1
Router G
Router C
RST-251 5465_05_2002_c1
2002, Cisco Systems, Inc. All rights reserved.
Router D
Router 1
58
Copyright 2002, Cisco Systems, Inc. All rights reserved. Printed in USA. 5465_05_2002, RST-251
Policy Routing
RtrA(config-if)#ip policy route-map set-tunnel RtrA(config)#route-map set-tunnel RtrA(config-route-map)#match ip address 101 RtrA(config-route-map)#set interface Tunnel1
Router F
Router H
Tunnel1
Router G
Router C
RST-251 5465_05_2002_c1
2002, Cisco Systems, Inc. All rights reserved.
Router D
Router 1
59
Policy Routing
Node Node Next-Hop Next-Hop Cost Cost B B 10 B B 10 C 10 C C 10 C D C 20 D C 20 E B 20 E B 20 B F B 30 F 30 G B 30 G B 30 40 H B 40 H B II 40 B 40 B
Routing table isnt affected by policy routing Need (12.0(16)ST or 12.2T) or higher for set interface tunnel to work
Router B Router F Router E
Router H
Router A
Tunnel1
Router G
Router C
RST-251 5465_05_2002_c1
2002, Cisco Systems, Inc. All rights reserved.
Router D
Router 1
60
Copyright 2002, Cisco Systems, Inc. All rights reserved. Printed in USA. 5465_05_2002, RST-251
Auto-Route
Auto-route = Use the tunnel as a directly connected link for SPF purposes Said otherwise: Add a route in the routing table of the headend that corresponds to the tunnel
Tunnel mpls traffic-eng autoroute announce command
This is not the CSPF (for path determination), but the regular IGP SPF (route determination) Behavior is intuitive, operation can be confusing
RST-251 5465_05_2002_c1
61
Auto-Route
Router F
Router H
Router G
Router C
RST-251 5465_05_2002_c1
2002, Cisco Systems, Inc. All rights reserved.
Router D
Router I
62
Copyright 2002, Cisco Systems, Inc. All rights reserved. Printed in USA. 5465_05_2002, RST-251
Auto-Route
This is Router As logical topology By default, other routers dont see the tunnel!
Router B Router A Router E Router F Router H
Tunnel1
Router G
Router C
RST-251 5465_05_2002_c1
2002, Cisco Systems, Inc. All rights reserved.
Router D
Router I
63
Auto-Route
Node Node Next-Hop Next-Hop Cost Cost B B 10 B B 10 C 10 C C 10 C D C 20 D C 20 E B 20 E B 20 B F B 30 F 30 G Tunnel 1 30 G Tunnel 1 30 H Tunnel 40 H Tunnel 1 1 40 II Tunnel 40 Tunnel 1 1 40
Router As routing table, built via auto-route Everything behind the tunnel is routed via the tunnel
Router B Router F Router E Router H
Router A
Tunnel1
Router G
Router C
RST-251 5465_05_2002_c1
2002, Cisco Systems, Inc. All rights reserved.
Router D
Router I
64
Copyright 2002, Cisco Systems, Inc. All rights reserved. Printed in USA. 5465_05_2002, RST-251
Auto-Route
RST-251 5465_05_2002_c1
65
Forwarding Adjacency
Autoroute metric change is purely local to the headend This makes MPLS TE different from TE with ATM
In ATM TE, the TE link (PVC) has its cost and neighbor advertised into the network In MPLS TE, no such thing is doneUntil FA
Forwarding Adjacency advertises the tunnel into IGP not only at the headend
RST-251 5465_05_2002_c1
66
Copyright 2002, Cisco Systems, Inc. All rights reserved. Printed in USA. 5465_05_2002, RST-251
ATM Model
H I
A C B D
Cost of ATM links (blue) is unknown to routers A sees two links in IGPE->H and B->D A can load-share between B and E
RST-251 5465_05_2002_c1
2002, Cisco Systems, Inc. All rights reserved.
67
Before FA
E F G H I
All links have cost of 10 As shortest path to I is A->B->C->D->I A doesnt see TE tunnels on {E,B}, alternate path never gets use d! Changing link costs is undesirable, can have strange adverse effects
RST-251 5465_05_2002_c1
2002, Cisco Systems, Inc. All rights reserved.
68
Copyright 2002, Cisco Systems, Inc. All rights reserved. Printed in USA. 5465_05_2002, RST-251
With forwarding-adjacency, A can see the TE tunnels as links A can then send traffic across both paths This is desirable in some topologies (looks just like ATM did, same methodologies can be applied)
RST-251 5465_05_2002_c1
2002, Cisco Systems, Inc. All rights reserved.
69
F-A heads up
In order for A to use F-A links, they need to be the best cost IGP path
Otherwise the physical topo gets used So make sure that the tunnels metric is better than the physical topology J
RST-251 5465_05_2002_c1
70
Copyright 2002, Cisco Systems, Inc. All rights reserved. Printed in USA. 5465_05_2002, RST-251
F-A heads up
F-A must be bidirectional F-A cost should probably be lower than lowest possible IGP path from head to tail, otherwise it might not always get used ( Yes I write this twice just to be sure I remember it myself. J )
RST-251 5465_05_2002_c1
71
IP routing has equal-cost load balancing, but not unequal cost* Unequal cost load balancing difficult to do while guaranteeing a loop-free topology
72
Copyright 2002, Cisco Systems, Inc. All rights reserved. Printed in USA. 5465_05_2002, RST-251
Since MPLS doesnt forward based on IP header, permanent IGP routing loops dont happen with unequal cost
Only one routing decision made, many forwarding, not many routing!!!
16 hash buckets for next-hop, shared in rough proportion to configured tunnel bandwidth or load-share value
RST-251 5465_05_2002_c1
73
Router E Router G
74
Copyright 2002, Cisco Systems, Inc. All rights reserved. Printed in USA. 5465_05_2002, RST-251
Router E Router G
Note That the Load Distribution Is 11:5Very Close to 2:1, but Not Quite!
RST-251 5465_05_2002_c1
75
Router E Router G
gsr1#sh ip rou 192.168.1.8 Routing entry for 192.168.1.8/32 Known via "isis", distance 115, metric 83, type level -2 Redistributing via isis Last update from 192.168.1.8 on Tunnel2, 00:00:08 ago Routing Descriptor Blocks: * 192.168.1.8, from 192.168.1.8, via Tunnel0 Route metric is 83, traffic share count is 100 192.168.1.8, from 192.168.1.8, via Tunnel1 Route metric is 83, traffic share count is 10 192.168.1.8, from 192.168.1.8, via Tunnel2 Route metric is 83, traffic share count is 1
76
Copyright 2002, Cisco Systems, Inc. All rights reserved. Printed in USA. 5465_05_2002, RST-251
Router E Router G
gsr1#sh ip cef 192.168.1.8 internal Load distribution: 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 (refcount 1) Hash 1 2 OK Y Y Interface Tunnel0 Tunnel1 Address point2point point2point Packets 0 0 T ags imposed {36} {37}
A: Any Way It Wants to! 15:1, 14:2, 13:2:1, It Depends on the Order the Tunnels Come Up
Deployment Guideline: Dont Use Tunnel Metrics That Dont Reduce to 16 Buckets!
RST-251 5465_05_2002_c1
2002, Cisco Systems, Inc. All rights reserved.
77
You can use any combination of auto-route, forwarding-adjacency, static routes, or PBR but simple is better unless you have a good reason Recommendation: autoroute, forwardingadjacency, or static indecreasinr order, depending on your needs
RST-251 5465_05_2002_c1
78
Copyright 2002, Cisco Systems, Inc. All rights reserved. Printed in USA. 5465_05_2002, RST-251
Agenda
Prerequisites Introduction How MPLS-TE Works Fast ReRoute Design Demo J
RST-251 5465_05_2002_c1
79
Fast ReRoute
FRR: A mechanism to minimize packet loss during a failure Pre-provision protection tunnels that carry traffic when a protected resource (link/node) goes down Use MPLS-TE to signal the FRR protection tunnels, taking advantage of the fact that MPLS-TE traffic doesnt have to follow the IGP shortest path Can protect MPLS traffic or IP traffic, depends on the type of protection
RST-251 5465_05_2002_c1
2002, Cisco Systems, Inc. All rights reserved.
80
Copyright 2002, Cisco Systems, Inc. All rights reserved. Printed in USA. 5465_05_2002, RST-251
Fast Reroute
In an IP network, a link failure causes several seconds of outage
Thing
Link Failure Detection Information Propagation Route Recalculation
Dependency
Media- and Platform-specific IGP Timers, Network Size, Collective Router Load LSDB Size, CPU Load
Time
~ secs (POS + APS) ~530 sec ~13 sec
RST-251 5465_05_2002_c1
81
Fast Reroute
In an MPLS network, theres more work to be done, so a (slightly) longer outage happens
Thing
Link Failure Detection Information Propagation Route Recalculation New LSP Setup
Dependency
Media- and Platform-specific IGP Timers, Network Size, Collective Router Load LSDB Size, CPU Load Network Size, CPU Load
Time
~Usecs (POS + APS) ~530 sec ~13 sec ~510 sec
RST-251 5465_05_2002_c1
82
Copyright 2002, Cisco Systems, Inc. All rights reserved. Printed in USA. 5465_05_2002, RST-251
Link protection
Implemented today
Node protection
Implemented today
Path protection
On development radar
RST-251 5465_05_2002_c1
83
Link Protection
Router A
Router B
Router D
Router E
Router C
RST-251 5465_05_2002_c1
2002, Cisco Systems, Inc. All rights reserved.
84
Copyright 2002, Cisco Systems, Inc. All rights reserved. Printed in USA. 5465_05_2002, RST-251
Link Protection
B has a pre-provisioned backup tunnel to the other end of the protected link (Router D) B relies on the fact that D is using global label space
10 34 POP
Router A
Router B
Protected Link 27 NHOP Backup Tunnel
Router D
Router E
PLR
MP NHOP POP
Router C
85
RST-251 5465_05_2002_c1
Link Protection
B -> D link fails, A -> E tunnel is encapsulated in B -> D tunnel Backup tunnel is used until A can re-compute tunnel path as A > B -> C -> D -> E (1030 seconds or so) not for longer.
10 POP
Router A
Router B
Router D
Router E
27, 34
34
Router C
RST-251 5465_05_2002_c1
2002, Cisco Systems, Inc. All rights reserved.
86
Copyright 2002, Cisco Systems, Inc. All rights reserved. Printed in USA. 5465_05_2002, RST-251
Link Protection
On tunnel head-end:
tunnel mpls traffic-eng fast-reroute
Router A
Router B
Router D
Router E
On protected link:
mpls traffic-eng backup-path <backup-tunnel>
RST-251 5465_05_2002_c1
2002, Cisco Systems, Inc. All rights reserved.
87
Node Protection
Router A has a tunnel A -> B -> D -> E -> F Router B has a protect tunnel B -> C -> E -> D
Router A Router B Router E
Router D
Router F
RST-251 5465_05_2002_c1
88
Copyright 2002, Cisco Systems, Inc. All rights reserved. Printed in USA. 5465_05_2002, RST-251
Node Protection
Link protection is OK if the B -> D link goes down What if Router D goes away?
Router A Router B Router E
Router D
Router F
RST-251 5465_05_2002_c1
89
Node Protection
Solution: Protect tunnel to the hop past the protected link
10 34 Protected Node 22 POP
Router A
Router B
Router D
Router E
Router F
NHOP POP
MP NNHOP
RST-251 5465_05_2002_c1
90
Copyright 2002, Cisco Systems, Inc. All rights reserved. Printed in USA. 5465_05_2002, RST-251
Node Protection
Node protection still has the same convergence properties as link protection Deciding where to place your backup tunnels is a much harder to problem to solve large-scale For small-scale protection, link may be better Cisco has developed tools to solve these hard problems for you (Tunnel Builder Pro)
RST-251 5465_05_2002_c1
91
Path Protection
Path protection: Multiple tunnels from TE head to tail, across diverse paths
Router A Router B Router D Router E Router F
RST-251 5465_05_2002_c1
92
Copyright 2002, Cisco Systems, Inc. All rights reserved. Printed in USA. 5465_05_2002, RST-251
Path Protection
Path protection: Least scalable, most resource-consuming, slowest convergence of all 3 protection schemes Path protection is useful in two places:
1. When you have more links than tunnels 2. When you need to protect links not using global label space
RST-251 5465_05_2002_c1
93
Dependency
MediaMedia- and and Platform-specific Platform-specific RP-> RP-> Communication Communication Time Time
Time
~Usecs ~Usecs (POS (POS + + APS) APS) ~Few ~Few msec msec or or Less Less
Path Protection
Thing
Link Link Failure Failure Detection Detection Information Information Propagation Propagation Head Head-end -end Switch Switch-over -over to to Protect Protect LSP LSP
RST-251 5465_05_2002_c1
2002, Cisco Systems, Inc. All rights reserved.
Dependency
MediaMedia- and and Platform-specific Platform-specific IGP IGP Timers, Timers, Network Network Size, Size, Collective Collective Router Router Load Load Network Network Size, Size, CPU CPU Load Load
Time
~Usecs ~Usecs (POS (POS + + APS) APS) ~530+ ~530+ sec sec
~Msec ~Msec
94
Copyright 2002, Cisco Systems, Inc. All rights reserved. Printed in USA. 5465_05_2002, RST-251
Dependency
MediaMedia- and and Platform-specific Platform-specific RP-> RP-> Communication Communication Time Time
Time
~Usecs ~Usecs (POS (POS + + APS) APS) ~Few ~Few msec msec or or Less Less
APS Protection
Thing
Link Link Failure Failure Detection Detection APS/MSP APS/MSP Cutover Cutover IGP IGP Reconverges Reconverges on on New New Link Link
RST-251 5465_05_2002_c1
Dependency
Media-and Media-and Platform-specific Platform-specific Generally Generally a a Fixed Fixed Time Time IGP IGP Timers, Timers, IGP IGP Size, Size, CPU CPU Load, Load, Etc. Etc.
Time
~Usecs ~Usecs (POS (POS + + APS) APS) <50ms, <50ms, per per spec spec
95
Agenda
Prerequisites Introduction How MPLS-TE Works Fast ReRoute Design Demo J
RST-251 5465_05_2002_c1
96
Copyright 2002, Cisco Systems, Inc. All rights reserved. Printed in USA. 5465_05_2002, RST-251
Design
Two ways to deploy MPLS-TE
As needed to clear up congestionTactical Full mesh between a set of routersStrategic
Strategic can be online or offline path calculation Both methods are valid, both have their pros and cons
RST-251 5465_05_2002_c1
97
Tactical
Case Study: A Large US ISP
Router A
Router B
Router C
All links are OC12 A has consistent 700MB to send to C ~100MB constantly dropped!
Router D
Router E
RST-251 5465_05_2002_c1
98
Copyright 2002, Cisco Systems, Inc. All rights reserved. Printed in USA. 5465_05_2002, RST-251
Tactical
Solution: Multiple tunnels, unequal cost load sharing!
Router A Router B Tunnels with bandwidth in 3:1 (12:4) ratio 25% of traffic sent the long way 75% sent the short way No out-of-order packet issues CEFs normal per-flow hashing is used! Router E Router C
Router D
RST-251 5465_05_2002_c1
99
Tactical
From Router As perspective, topology is:
Router A
Router B
Router C
Router D
Router E
RST-251 5465_05_2002_c1
100
Copyright 2002, Cisco Systems, Inc. All rights reserved. Printed in USA. 5465_05_2002, RST-251
Tactical
As neededEasy, quick, but hard to track over time Easy to forget why a tunnel is in place Inter-node BW requirements may change, tunnels may be working around issues that no longer exist Link protection pretty straightforward, node protection harder to track
RST-251 5465_05_2002_c1
2002, Cisco Systems, Inc. All rights reserved.
101
Strategic
Put a full mesh of TE tunnels between routers Initially deploy tunnels with 0 bandwidth Watch tunnel interface statistics, see how much bandwidth you are using between router pairs
Tunnels are interfacesUse IF-MIB! Make sure that tunnel <= network BW
RST-251 5465_05_2002_c1
2002, Cisco Systems, Inc. All rights reserved.
102
Copyright 2002, Cisco Systems, Inc. All rights reserved. Printed in USA. 5465_05_2002, RST-251
Strategic
Some folks deploy full mesh just to get router-to-router (pop-to-pop) traffic matrix Largest TE network ~80 routers full mesh (~6400 tunnels) As tunnel bandwidth is changed, tunnels will find the best path across your network
RST-251 5465_05_2002_c1
103
Strategic
Physical topology is:
Router A
Router B
Router C
Router D
Router E
RST-251 5465_05_2002_c1
104
Copyright 2002, Cisco Systems, Inc. All rights reserved. Printed in USA. 5465_05_2002, RST-251
Strategic
Logical topology is*
*Each link is actually 2 unidirectional tunnels
Router B
Router C
Router D
Router E
RST-251 5465_05_2002_c1
105
Strategic
Things to remember with full mesh
N routers, N*(N-1) tunnels Routing protocols not run over TE tunnels Unlike an ATM/FR full mesh! Tunnels are unidirectionalThis is a good thing Can have different bandwidth reservations in two different directions
RST-251 5465_05_2002_c1
106
Copyright 2002, Cisco Systems, Inc. All rights reserved. Printed in USA. 5465_05_2002, RST-251
Strategic
RST-251 5465_05_2002_c1
107
Scalability
How Many Tunnels on a Router?
Number of Head-End Tunnels 600 Number of Mid-Points 10,000 Number of Tunnel Tails 5,000
Code 12.0ST
Tests were done on a GSR RSP4, RSP8, VXR300, VXR400 will be similar 10,000 tunnels come up in 3-5 minutes
RST-251 5465_05_2002_c1
2002, Cisco Systems, Inc. All rights reserved.
108
Copyright 2002, Cisco Systems, Inc. All rights reserved. Printed in USA. 5465_05_2002, RST-251
Scalability
Largest TE network today = 80 routers, ~6400 tunnels full mesh 12.0ST600 head-ends, 360,000 tunnels full mesh with 10,000 tunnels per midpoint Bottom line: MPLS-TE is not a gating factor in scaling most networks!
RST-251 5465_05_2002_c1
109
Scalability
http://www.cisco.com/univercd/cc/td/doc/pro duct/software/ios120/120newft/120limit/120st /120st14/scalable.htm
Or just search CCO for Scalability Enhancements for MPLS Traffic Engineering
RST-251 5465_05_2002_c1
110
Copyright 2002, Cisco Systems, Inc. All rights reserved. Printed in USA. 5465_05_2002, RST-251
Demo
RST-251 5465_05_2002_c1
111
BW= 5 S2 .2
Process :102
E0 .2
BW= 2
Address : 10.1.1.0/24 Ethernet
E1 .1
BW= 5
S3 .1
S3 .1
S2 .1
BW= 3
BW= 3
S3 .1
E0 .1
E0 .2
Address : 130.1.0.0/24 Serial Name: Core5 Telnet Port:1105 Process :103 Name: PAGEN2 Telnet Port:1121 Process :111 Name: PAGEN1 Telnet Port:1120 Process :110
Bandwidth in Mbps
RST-251 5465_05_2002_c1
112
Copyright 2002, Cisco Systems, Inc. All rights reserved. Printed in USA. 5465_05_2002, RST-251
BW= 5 S2 .2
Process :102
E0 .2
BW= 2
Address : 10.1.1.0/24 Ethernet
E1 .1
BW= 5
S3 .1
S3 .1
S2 .1
BW= 3
BW= 3
S3 .1
E0 .1
E0 .2
Address : 130.1.0.0/24 Serial Name: Core5 Telnet Port:1105 Process :103 Name: PAGEN2 Telnet Port:1121 Process :111 Name: PAGEN1 Telnet Port:1120 Process :110
RST-251 5465_05_2002_c1
113
BW= 5 S2 .2
Process :102
E0 .2
BW= 2
Address : 10.1.1.0/24 Ethernet
E1 .1
BW= 5
S3 .1
S3 .1
S2 .1
BW= 3
BW= 3
S3 .1
E0 .1
E0 .2
Address : 130.1.0.0/24 Serial Name: Core5 Telnet Port:1105 Process :103 Name: PAGEN2 Telnet Port:1121 Process :111 Name: PAGEN1 Telnet Port:1120 Process :110
Tunnel
RST-251 5465_05_2002_c1
114
Copyright 2002, Cisco Systems, Inc. All rights reserved. Printed in USA. 5465_05_2002, RST-251
Recommended Reading
RST-251 5465_05_2002_c1
115
RST-251 5465_05_2002_c1
116
Copyright 2002, Cisco Systems, Inc. All rights reserved. Printed in USA. 5465_05_2002, RST-251