BRKARC-2003
Carrier Ethernet
Cable/MSO
1.
2.
3.
4.
Mobile Backhaul
Mobile Backhaul
L2/Metro Aggregation
CMTS Aggregation
Video Distribution &
Services
Web/OTT
Multiservice Edge
2. DC Gateway Router
DC gateway
Broadband
Gateway
1.
2.
3.
DC Interconnect
DC WAN Edge
WEB/OTT
3. Services Router
Large Enterprise
WAN
BRKARC-2003
1.
2.
3.
4.
Cisco Public
Business Services
Residential Broadband
Converged Edge/Core
Enterprise WAN
IOS XRv
XR virtualization
nV Satellite
9000v,901,903
BRKARC-2003
9001
9001-S
9904
9006
Cisco Public
9010
9912
9922
BRKARC-2003
Cisco Public
Agenda
ASR9000 Hardware System Architecture
HW Overview
HW Architecture
BRKARC-2003
Cisco Public
99xx: >2Tb/Slot*
9904, 6 RU
9912, 30RU
9922, 44RU
90xx: 880Gb/Slot*
9006, 10RU
9010, 21RU
Fixed
4 I/O
2 I/O
* Chassis capacity only, bandwidth also depends on the fabric and line cards
BRKARC-2003
Cisco Public
8 I/O
10 I/O
20 I/O
Front-to-back
airflow
RSP (0-1)
(integrated
switch fabric)
Side-to-back airflow, 10 RU
Line Card
(0-3, 4-7)
21RU
Cisco Public
2 power shelves
6 V1 or 8 V2 PS
Redundant
(AC or DC)
Power Supplies
Field Replaceable
BRKARC-2003
Supported MPAs:
20x1GE
2x10GE
4x10GE
1x40GE
Cisco Public
10
Fixed 4x10G
SFP+ ports
Fan Tray
Field Replaceable
Supported MPAs:
20x1GE
2x10GE
4x10GE
1x40GE
Cisco Public
11
Description
Power
Fan
4 Fan Trays
Front to back airflow
I/O Slots
20 I/O slots
Rack Size
44 RU
RP
1+1 RP redundancy
Fabric
Bandwidth
SW
BRKARC-2003
Cisco Public
12
Description
Fan
2 Fan Trays
Front to back airflow
I/O Slots
10 I/O slots
Rack Size
30 RU
RP
1+1 RP redundancy
Fabric
Power
Bandwidth
SW
BRKARC-2003
Cisco Public
13
ASR 9904
Shipping since 5.1.0, Sep 2013
Front-to-back air flow with air flow
baffles, 10RU
Feature
Description
I/O Slots
2 I/O slots
Rack size
6RU
Fan
RSPs
RSP440, 1+1
Power
Fabric
Bandwidth
SW
BRKARC-2003
Side-to-Side airflow
6RU
Cisco Public
14
ASR-9010-FAN
ASR-9006-FAN
DC Supplies
A
B
1.5 kW*
A
B
2.1 kW
AC Supplies
Power Supply
A
3 kW
3 kW
* Version 1 only
BRKARC-2003
Cisco Public
15
PEM1
Power Switch:
V1 in the back
V2 in the front
PEM1
PEM2
PEM2
PEM3
V1 Power
PEM3
PEM4
V2 Power
M3
M2
M1
M0
V2 AC power
PWR A-, M3
RTN A+, M3
BRKARC-2003
Cisco Public
16
PWR B-, M3
RTN B+, M3
V2 DC power
-L, -B, -E
A9K-40G
A9K-4T
A9K-8T/4
A9K-2T20G
A9K-8T
A9K-16T/8
A9K-MOD160
Second-gen LC
(Typhoon)
A9K-MOD80
-TR,
-SE
A9K-24x10GE
A9K-2x100GE
MPAs
20x1GE
2x10GE
4x10GE
8x10GE
1x40GE
2x40GE
A9K-36x10GE
* Trident 10G line cards EoS/EoL:
http://www.cisco.com/c/en/us/products/routers/asr-9000-series-aggregation-services-routers/eos-eol-notice-c51-731288.html
BRKARC-2003
Cisco Public
17
Trident
Typhoon *
nV Cluster
SP WiFi
MPLS-TP
1588v2 (PTP)
PBB-VPLS
PW-HE
E-VPN/ PBB-EVPN
Scale ACL
Y
* HW ready, See SW For Specific Release
Some features are not available yet in SW, although it will be supported on Typhoon hardware
This is not the complete feature list
BRKARC-2003
Cisco Public
18
128k Queues
128k Policers
H-QoS
Color Policing
Scalability
High Availability
IC-Stateful Switch Over
Capability
MR-APS
SIP-700
BRKARC-2003
SPAs
Cisco Public
19
SPA Support
ChOC-3/12/48 (STM1/4/16)
NEW
XR5.1.1
SFP, SFP+
XFP
QSFP
CFP
Cisco Public
20
Application Domain
Linux Based
Multi-Purpose Compute
Resource:
o Used for Network
Positioning System (NPS)
o Used for Translation Setup
and Logging of CGN
Applications
IOS-XR
Control Plane
Data Forwarding
L3, L2 (management)
IRB (4.1.1)
Hardware Management
BRKARC-2003
1M+ connections/second
validated for 14Gbps per ISM
Cisco Public
21
NAT 444
Dual Stack
DS-Lite
DS-Lite
Stateless 46
(dIVI/MAP-T)
Stateful NAT64
6RD
MAP-E
IOS XR
Releases
BRKARC-2003
Cisco Public
22
ISM
2nd Gen Eth Linecards
Service-1
VM-4
VM-1
Service-4
Service-2
VM-3
VM-2
VMM
OS / Hypervisor
Cisco Public
23
Anti-DDOS
DPI
NAT
Firewall
Transparent
Cache
vRouters
CDN
Virus Malware
Protection
Vmware/kvm
Vmware/kvm
Vmware/kvm
Vmware/kvm
Vmware/kvm
Vmware/kvm
Vmware/kvm
Vmware/kvm
Low Latency
Simplified Service
Chaining
Router integratedVSM
Management Plane
Hardware assists
VSM
SDN
SDN
VSM
VSM
Cisco Public
24
UCS
VSM Architecture
XAUI
PCIe
SFP+
SFP+
SFP+
SFP+
Quad
PHY
Crypto/DPI
Assist
32GB
DDR3
Ivy
Bridge
Niantic
Niantic
Crypto/DPI
Assist
32GB
DDR3
Ivy
Bridge
32GB
DDR3
Ivy
Bridge
Fabric
ASIC 0
Typhoon
NPU
Fabric
ASIC 1
Niantic
48
ports
10GE
Niantic
Niantic
Niantic
Crypto/DPI
Assist
32GB
DDR3
Typhoon
NPU
Ivy
Bridge
Niantic
Fabric
Niantic
Crypto/DPI
B
A
C
K
P
L
A
N
E Switch
Cisco Public
CPU
CPU
BITS/DTI
FIA
FIC
Switch Fabric
BRKARC-2003
Cisco Public
27
Integrated on RSP or
Separated fabric card
9904/9006/9010
RSP440
9912/9922-RP
PPC/Freescale
Intel x86
Intel x86
2 Core 1.5GHz
RSP-4G: 4GB
RSP440-TR: 6GB
-TR: 6GB
RSP-8G: 8GB
RSP440-SE: 12GB
-SE: 12GB
nV EOBC ports
No
Switch fabric
bandwidth
92G + 92G
660G+110G
(fabric integrated on
RSP)
Processors
RAM
BRKARC-2003
Cisco Public
28
Identical
GPS Interface
ToD, 1pps, 10Mhz
2x Mgmt Eth
Alarms
BITS/DTI/J.211 PTP*
nV Edge Sync*
* Future SW support
BRKARC-2003
Cisco Public
29
Status LEDs
Clock
Time
FPGA
MEM
CF card
or USB
HDD
CPU
Mgt Eth
EOBC
Internal communication between RPs and Line Cards
Mgt Eth
Console
Aux
Punt
FPGA
4G disk
Alarm
NVRAM
Front Panel
2014 Cisco and/or its affiliates. All rights reserved.
Boot Flash
CPU Complex
Cisco Public
FIA
Arbitration
Arbiter
Crossbar
Fabric
ASIC
I/O FPGA
BRKARC-2003
Timing Domain
30
Crossbar
Fabric
ASIC
Switch fabric
6+1 redundancy
1+1 redundancy
Integrated fabric/RP/LC
9904
RSP440: 385G+385G /slot
9006
9010
RSP440: 220G+220G /slot
9912
Cisco Public
31
9922
660G+110G /slot
Stage 2
Stage 3
fabric
fabric
8x55Gbps
fabric
Arbiter
FIA
FIA
FIA
FIA
FIA
FIA
RSP0
Ingress Linecard
fabric
8x55Gbps
Egress Linecard
Fabric bandwidth:
Arbiter
RSP1
2nd gen Fabric (RSP440)
BRKARC-2003
Cisco Public
32
FIA0
8x23G
bi-directional
= 184Gbps
2nd Generation
Fabric (RSP440)
FIA1
fabric
Arbiter
RSP0
FIA
FIA
FIA
FIA
2nd gen Line Card
fabric
4x23G
bi-directional
= 92Gbps
Arbiter
RSP1
BRKARC-2003
Egress Linecard
fabric
Dual-FIA
8xNPs 1st gen
Linecard
Single-FIA
4xNPs 1st gen
Linecard
8x55G
bi-directional
= 440Gbps
Cisco Public
33
8x23G
bi-directional
= 184Gbps
1nd Generation
Fabric (RSP)
FIA1
fabric
Dual-FIA
8xNPs 1st gen
Linecard
Arbiter
FIA
FIA
FIA
FIA
2nd gen Line Card
fabric
4x23G
bi-directional
= 92Gbps
Arbiter
RSP1
BRKARC-2003
8x23G
bi-directional
= 184Gbps
fabric
RSP0
Single-FIA
4xNPs 1st gen
Linecard
8x55G
bi-directional
= 440Gbps
Cisco Public
34
Stage 2
Stage 1
fabric
fabric
Arbiter
FIA
FIA
FIA
RSP0
14x55Gbps
fabric
FIA
FIA
FIA
3rd gen Line Card
Ingress Linecard
Stage 3
fabric
8x55Gbps
Egress Linecard
Fabric bandwidth for future Ethernet LC:
Arbiter
Note, if mix old and new future line card in the same
system, then fabric will fall back to 8x55Gbps mode
RSP1
RSP440
BRKARC-2003
Cisco Public
35
5x2x55G
bi-directional
= 550Gbps
fabric
FIA
FIA
FIA
FIA
FIA
FIA
fabric
2nd gen Line Card
BRKARC-2003
5x2x55G
bi-directional
= 550Gbps
Cisco Public
36
7x2x55G
bi-directional
= 770Gbps
fabric
FIA
FIA
FIA
FIA
FIA
FIA
fabric
3rd gen Line Card
7x2x55G
bi-directional
= 770Gbps
Note, if mix old and new future line card in the same
system, then old line card will fall back to 5 fabric plane
Fabric cards
BRKARC-2003
Cisco Public
37
A9K-4T
A9K-8T/4
A9K-2T20G
A9K-8T
A9K-16T/8
-TR, -SE
Second-gen LC
Typhoon NPU:
60Gbps, ~45Mpps,
bi-directional
A9K-MOD160
A9K-MOD80
A9K-24x10GE
A9K-2x100GE
(A9K-1x100G)
A9K-36x10GE
BRKARC-2003
2014-L:
Cisco
and/or
its affiliates.
All rights reserved.
low
queue,
-B: Medium
queue,
Publicqueue, 38
-E:Cisco
Large
-TR: transport optimized, -SE: Service edge optimized
MPAs
20x1GE
2x10GE
4x10GE
8x10GE
1x40GE
2x40GE
Trident
1x10GE
NP1
PHY
NP2
PHY
NP3
3x10GE
SFP +
Typhoon
3x10GE
SFP +
NP1
3x10GE
SFP +
NP2
3x10GE
SFP +
NP3
3x10GE
SFP +
NP4
3x10GE
SFP +
NP5
3x10GE
SFP +
NP6
3x10GE
SFP +
NP7
CPU
B0
4x23G =
92G
FIA0
B1
Trident LC example: A9K-4T
RSP0
CPU
FIA0
FIA1
FIA2
8x55G =
440G
Switch
Fabric
FIA3
Typhoon LC example: A9K-24x10G
BRKARC-2003
Switch
Fabric
Cisco Public
39
Switch
Fabric
RSP1
9010/9006
NP
3x10GE
SFP +
NP
3x10GE
SFP +
NP
3x10GE
SFP +
FIA
8x55G
FIA
NP
Switch
Fabric
RSP0
3x10GE
SFP +
NP
3x10GE
SFP +
NP
3x10GE
SFP +
NP
3x10GE
SFP +
NP
FIA
FIA
Each FIA: 60Gbps bidirectional
Cisco Public
40
Switch
Fabric
Switch
Fabric
RSP1
NP
FIA
8x55G
6x10GE
PHY (SFP+)
NP
FIA
6x10GE
PHY (SFP+)
NP
FIA
6x10GE
PHY (SFP+)
NP
FIA
6x10GE
PHY (SFP+)
NP
FIA
Switch
Fabric
RSP0
Switch
Fabric
Switch
Fabric
RSP1
6x10GE
PHY (SFP+)
BRKARC-2003
NP
Cisco Public
FIA
41
FIA
Ingress NP
100GE
MAC/PHY
8x55G
100G
FIA
Egress NP
Switch
Fabric
RSP0
100G
FIA
Ingress NP
100GE
MAC/PHY
100G
FIA
Egress NP
MUX FPGA
BRKARC-2003
Cisco Public
42
Switch
Fabric
Switch
Fabric
RSP1
NP
FIA
MPA Bay 0
8x55G
NP
FIA
Switch
Fabric
RSP0
NP
FIA
MPA Bay 1
NP
BRKARC-2003
FIA
Cisco Public
43
Switch
Fabric
Switch
Fabric
RSP1
NP
FIA
MPA Bay 0
8x55G
Switch
Fabric
RSP0
NP
FIA
MPA Bay 1
Switch
Fabric
Switch
Fabric
RSP1
BRKARC-2003
Cisco Public
44
MOD80
NP
NP
1
4 port 10GE
MPA
NP
4 port 10GE
MPA
NP
NP
1
2 port 10GE
MPA
2 port 10GE
MPA
NP
BRKARC-2003
Cisco Public
45
FIB
NP complex
STATS MEMORY
MAC
LOOKUP
MEMORY
FRAME MEMORY
TCAM
-TR/-SE, -L/-B/-E
Different TCAM/frame/stats memory size for different per-LC QoS, ACL, logical interface scale
Same lookup memory for same system wide scale mixing different variation of LCs doesnt impact system wide
scale
BRKARC-2003
-L: low queue, -B: Medium queue, -E: Large queue, -TR: transport optimized, -SE: Service edge optimized
Cisco Public
46
ASR9001 Architecture
Identical HW Components as the Modular Systems
MPAs
2,4x10GE
20xGE
1x40GE
FIA
NP
SFP+ 10GE
On-board
4x10 SFP+
ports
Internal
EOBC
SFP+ 10GE
LC
CPU
SFP+ 10GE
RP
CPU
SFP+ 10GE
MPAs
2,4x10GE
20xGE
1x40GE
NP
FIA
Disabled in ASR9001-S
BRKARC-2003
Cisco Public
47
Switch
Fabric
FIA
NP
SFP+ 10GE
On-board
4x10 SFP+ ports
SFP+ 10GE
Internal
EOBC
SFP+ 10GE
LC
CPU
RP
CPU
SFP+ 10GE
NP
FIA
Cisco Public
48
Switch
Fabric
I
O
S
LC CPU
CFM
LC CPU
NF
PIM
RP CPU
Routing
Independent Processes
X
R
Device
Driver
Micro-Kernel
TCP/IP
Process Mgmt
Memory Mgmt
Scheduler
HW Abstraction
OSPFv3
BGP
IOS-XR
File
System
Full Standard XR PI
Binaries
con
aux
Platform Layer
SPP Data Plane
QNX
disk
Cisco Public
50
Virtualization for
flexibility
Mgm
tEth
GE 0
GE 1
GE 2
GE n
MPLS
Multicast
RPLRouting
BGP
Composite
OSPF
ISIS
Manageability
Security
Forwarding
Host
Composite
Base
IOX Admin
OS
Line card
BRKARC-2003
Cisco Public
51
DRP 1)
Distributed data
management model
improves performance
and Scale
Local-DRP
Management
Applications
(CLI/XML/SNMP)
Global
RP-A
Local-Ra
IP
OSPF
Intf
ISIS
BGP
IP
Intf
LCa
Local-LCa
Intf
ARP
PPP
ACL
VLAN
Global
Consolidated
System View
Cisco Public
52
QOS
OSPF
BGP
ISIS
BRKARC-2003
Cisco Public
53
SMU Manager
Internet
Intranet
Customer
www.cisco.com
Secure
Connection
Automated
SW management
capabilities
Auto Discovery
Multi Node
Recommendations
Analysis and Optimization
BRKARC-2003
Cisco Public
54
Cisco Tools
PIMS
Release Ops
SMU Tool
BRKARC-2003
Cisco Public
55
Platform Virtualization
NX-OS
IOS XR
VM-based tool: IOS XRv
FCS Target: 5.1.1
IOS XE
VM-based tool: CSR1000v
FCS: Q2CY13
BRKARC-2003
IOS
VM-based tool: IOSv
H2FY13
Cisco Public
56
IOS-XRv
Cisco IOS XRv supported since 5.1.1
Control plane only. Virtual data plane on the roadmap
Initial application: BGP router reflect, Cisco Modeling Lab (CML)
Release Notes:
http://www.cisco.com/en/US/partner/docs/ios_xr_sw/iosxr_r5.1/general/release/notes/reln-xrv.html
Demo Image: https://upload.cisco.com/cgi-bin/swc/fileexg/main.cgi?CONTYPES=Cisco-IOS-XRv
Installation Guide:
http://www.cisco.com/en/US/docs/ios_xr_sw/ios_xrv/install_config/b_xrvr_432.html
Quick Guide to ESXi: https://supportforums.cisco.com/docs/DOC-39939
BRKARC-2003
Cisco Public
57
RP
CPU
Punt
FPGA
Switch Fabric
FIA
Switch Fabric
Punt Switch
Control
packet
3x10GE
SFP +
Typhoo
LPTS
n
3x10GE
SFP +
NP
BRKARC-2003
NP
Switch
Fabric ASIC
3x10G
E
SFP +
FIA
FIA
3x10GE
SFP +
NP
3x10GE
SFP +
NP
3x10GE
SFP +
NP
3x10GE
SFP +
NP
3x10GE
SFP +
NP
FIA
FIA
Cisco Public
59
Transit Traffic
Received Traffic
Forwarding
Information
Base (FIB)
LPTS
Internal FIB (IFIB)
Bad packets
Cisco Public
Application1
on RP
Local Stack
on LC
BRKARC-2003
Application1
on RP
60
RSVP-TE
Static
LSD
BGP
OSPF
ISIS
EIGRP
RIB
RP
ARP
FIB
SW FIB
Adjacency
AIB
LC NPU
LC CPU
BRKARC-2003
Cisco Public
61
Selective VRF
download per Line
card for high scale
NP
NP
FIA
FIA
3x10GE
SFP +
NP
3x10GE
SFP +
NP
3x10GE
SFP +
NP
3x10GE
SFP +
NP
3x10GE
SFP +
1
Typhoon
2
FIA
FIA
Switch
Fabric
Switch
Fabric
FIA
FIA
Egress
2
NP
FIA
Ingress
NP
Cisco Public
62
100G
100G
100GE
MAC/PH
Y
FIA
100G
100GE
MAC/PHY
Switch Fabric
ASIC
NP
Switch Fabric
ASIC
3x10GE
SFP +
Ingress
NP
Egress
NP
100G
L3 Unicast Forwarding
Packet Flow (Simplified) Example
from wire
LAGID
lookup key
L3: (VRF-ID, IP DA)
TCAM
rxIDB
L3FIB
rx-adj
Packet
classification
Source
interface info
L3 FIB
lookup
Next-hop
Rx LAG hashing
LAG
SFP
SFP
Switch Fabric Port
(egress NPU)
Packet rewrite
System headers added
rewrite
ECH Type:
L3_UNICAST
SFP
Ingress NPU
Fabric
Tx LAG hashing
LAG
rewrite
txIDB
tx-adj
L3FIB
destination
interface info
Next-hop
L3 FIB
lookup
ECH Type:
L3_UNICAST
=> L3FIB lookup
Egress NPU
BRKARC-2003
to wire
63
MRIB
PIM
MFIB PI
MFIB PI
MFIB PI
MFIB PD
MFIB PD
MFIB PD
LC1
LC2
LC0
BRKARC-2003
Cisco Public
64
Fabric to LC replication
Egress NP OIF replication
ASR9k doesnt use inferior binary tree or root uniary tree replication model
BRKARC-2003
Cisco Public
65
BRKARC-2003
Cisco Public
66
FGID (Slotmask)
FGIDs: 10 Slot Chassis
LC 5
LC 4
RSP 0
RSP 1
LC 3
LC 2
LC 0
LC 6
LC 1
LC 7
Logical
Slot
Logical
Slot
LC 3
LC 2
LC 1
LC 0
RSP 1
RSP 0
Slot
Slot
Logical
Slot Mask
Physical
Binary
Hex
LC7
1000000000
0x0200
LC6
0100000000
0x0100
LC5
0010000000
0x0080
LC4
0001000000
0x0040
RSP0
0000100000
0x0020
RSP1
0000010000
0x0010
LC3
0000001000
0x0008
LC2
0000000100
0x0004
LC1
0000000010
0x0002
LC0
BRKARC-2003
0 Cisco Systems,0000000001
2006
Inc. All rights reserved.
Physical
Binary
Hex
LC3
0000100000
0x0020
LC2
0000010000
0x0010
LC1
0000001000
0x0008
LC0
0000000100
0x0004
RSP1
0000000010
0x0002
FGID Calculation
0
0000000001
Examples
0x0001
RSP0
Cisco0x0001
Confidential
Cisco Public
Slot Mask
Logical
Target Linecards
LC6
0x0100
LC1 + LC5
67
MGID Tables
MGID Bitmasks
MGID
FIA
MGID
Bit 1
Bit 0
Bridge1
0
NP3
BRKARC-2003
MGID
Bridge0
NP2
Cisco Public
NP1
68
Bit 1
Bit 0
Bit 1
Bit 0
NP0
BRKARC-2003
Cisco Public
69
Ingress NPU:
1. MFIB (S,G) route lookup yields {FGID, MGID, Olist, 4-bit RBH} data-structures
2. Ingress NPU adds FGID, MGID, 4-bit RBH in fabric header to FIA
BRKARC-2003
Cisco Public
70
Ingress FIA:
1. Load-balance multicast traffic from FIA to LC Fabric
BRKARC-2003
Cisco Public
71
Ingress LC Fabric:
1. Reads FPOE bits in the fabric header AND reads 3-bits of derived RBH
2. It will load-balance MGID towards any of the 8 fabric channels
3. Now it send traffic to central fabric over 1 of the fabric channels per MGID
(Note: there are only upto 8 fabric-channel links to central fabrlc)
BRKARC-2003
Cisco Public
72
BRKARC-2003
Cisco Public
73
BRKARC-2003
Cisco Public
74
BRKARC-2003
Cisco Public
75
BRKARC-2003
Cisco Public
76
L3 SubI/F
Routing
EoMPLS PW
Bridging
Flexible VLAN
tag classification
(H-)VPLS
EoMPLS PW
IRB
Flexible VLAN
tag rewrite
Flexible
Ethertype (.1Q,
QinQ, .1ad)
EoMPLS PW
Bridging
IRB
Routing and Bridging
1
L2 or L3 subinterfaces
(802.1a/qinq/.1ad)
BRKARC-2003
77
dot1q 10
RP/0/RSP0/CPU0:PE2-asr(config-subif)#encapsulation ?
default Packets unmatched by other service instances
dot1ad
dot1q
dot1q 10
second
100
dot1q 10
second 128133
RP/0/RSP0/CPU0:PE2-asr(config-subif)#encapsulation dot1q 10
comma comma
EFP or
L2 sub-interface
Cisco Public
78
Pop tag 1 or 2
Push tag 1 or 2
Tag translation
1-1
1-2
Cisco Public
79
2-1
2-2
VLAN:
A
VLAN:
B
VFI
VFI
VFI
VLAN:
C
L2VPN P2P
EFP configuration example
L2VPN P2P service configuration example
Interface gig 0/0/0/1.101 l2transport
encapsulation dot1q 101 second 10
l2vpn
Internal
logical port
AC
AC
AC
PW
PW
PW
BRKARC-2003
Cisco Public
80
Bridge
domain
VXLAN
VPLS PW
Local bridging
L2 port
PBB-VPLS
PBB-EVPN
* Not in 5.2.0
BRKARC-2003
Cisco Public
81
l2vpn
bridge group cisco
bridge-domain domain1 local bridging
Interface gig 0/0/0/1.101
Interface gig 0/0/0/2.101
AC
Interface gig 0/0/0/3.101
Cisco Public
82
AC
AC
AC
BRKARC-2003
AC
PW
Default B-MAC SA
Auto RT for EVI
Auto RD for EVI
Auto RD for Segment Route
PE1
interface Bundle-Ether1.777 l2transport
encapsulation dot1q 777
l2vpn
bridge group gr1
bridge-domain bd1
interface Bundle-Ether1.777
pbb edge i-sid 260 core-bridge-domain core_bd1
bridge group gr2
bridge-domain core_bd1
pbb core
evpn evi 1000
PE1
CE1
PBB B-component
No need to define BVLAN
Mandatory - Globally
unique identifier for all
PEs in a given EVI
router bgp 64
address-family l2vpn evpn
!
neighbor <x.x.x.x>
remote-as 64
address-family l2vpn evpn
BRKARC-2003
Cisco Public
MINIMAL
Configuration
83
BundleEth1.777
MPLS
Core
RP/0/0/CPU0:r1(config)#l2vpn
RP/0/0/CPU0:r1(config-l2vpn)#bridge group customer1
RP/0/0/CPU0:r1(config-l2vpn-bg)#bridge-domain cu-l3vpn
RP/0/0/CPU0:r1(config-l2vpn-bg-bd)#member vni 65001
RP/0/0/CPU0:r1(config-l2vpn-bg-bd)#routed interface 101
BRKARC-2003
Cisco Public
84
RP/0/0/CPU0:r1(config)#l2vpn
RP/0/0/CPU0:r1(config-l2vpn)#bridge group customer1
RP/0/0/CPU0:r1(config-l2vpn-bg)#bridge-domain cu-l2vpn
RP/0/0/CPU0:r1(config-l2vpn-bg-bd)#interface GigabitEthernet0/2/0/0.100
RP/0/0/CPU0:r1(config-l2vpn-bg-bd)#member vni 65001
RP/0/0/CPU0:r1(config)#interface GigabitEthernet0/2/0/0.100 l2transport
RP/0/0/CPU0:r1(config-subif)#dot1q vlan 100
BRKARC-2003
Cisco Public
85
RP
CPU
Data
packet
1NP 2
CPU
CPU
FIA
NP
FIA
NP
NP
FIA
NP
Cisco Public
86
3x10GE
SFP +
3x10GE
SFP +
3x10GE
SFP +
3x10GE
SFP +
3x10GE
SFP +
3x10GE
SFP +
3x10GE
SFP +
3x10GE
SFP +
LC2
NP
NP
FIA
NP
NP
FIA
NP
FIA
NP
NP
FIA
NP
Switch
Fabric ASIC
FIA
NP
Switch Fabric
LC1
NP
NP
FIA
Switch Fabric
Switch
Fabric ASIC
BRKARC-2003
3x10GE
SFP +
3x10GE
SFP +
3x10GE
SFP +
3x10GE
SFP +
3x10GE
SFP +
3x10GE
SFP +
3x10GE
SFP +
3x10GE
SFP +
Punt
FPGA
PWHE
virtual
interface
L2 PW
Access PE (A-PE)
Internet
Peering
CE
Service PE (S-PE)
L2 (port or
vlan)
Aggregation
LDP domain
Business L3
VPNs
LDP Core /
Internet Core
L3PE
CE
Cisco Public
87
MPLS
xconnect
interface pw-ether 200
vrf vrf0001
ipv4 address 11.0.0.1 255.255.255.0
ipv6 address 2001:da1::1/64
load-interval 30
PW
CE
PE
PE
l2vpn
xconnect group pwhe
p2p pwhe-red
interface pw-ether 100
neighbor 100.100.100.100 pw-id 1
PWHE L3 interface
Example
xconnect
MPLS
PW
CE
PE
PE
BRKARC-2003
l2vpn
xconnect group pwhe
p2p pwhe-red
interface pw-ether 100
neighbor 100.100.100.100 pw-id 1
xconnect group cisco
p2p service2
Interface pw-ether 100.200
neighbor 1.1.1.1 pw-id 22
bridge-domain domain2
Interface pw-ether 100.300
vfi cisco
neighbor 192.0.0.1 pw-id 100
neighbor 192.0.0.2 pw-id 100
Cisco Public
88
Egress side of LC
CPU
CPU
3
PHY
NP
PHY
NP1
FIA
BRKARC-2003
NP
PHY
PHY
Switch
Fabric
1
Ingress Port
QoS
FIA
NP
Cisco Public
90
4
Egress Port
QoS
L3 sub-interface
L2 sub-interface (EFP)
* Some logical interface could apply qos policy, for example PWHE and BVI
** it could have main interface level simple flat qos co-exist with sub-interface level H-QoS on ingress direction
BRKARC-2003
Cisco Public
91
TM
FIA
Switch Fabric
ASIC
High scale
Up to 3 Million queues per system (with -SE linecard)
Up to 2 Million policers per system (with -SE linecard
Highly flexible: 4 layer hierarchy queuing/scheduling support
Four layer scheduling hierarchy Port, Subscriber Group,
Subscriber, Class
Egress & Ingress, shaping and policing
Three strict priority scheduling with priority propagation
Flexible & granular classification, and marking
Full Layer 2, Full Layer 3/4 IPv4, IPv6
* 8 queues per port
BRKARC-2003
Cisco Public
92
L4
L1
L2
L3
Port
Level
Subscriber
group Level
Subscribe
r Level
Clas
s
Leve
l
PQ1
PQ1
Telepresence
Internet Best Effort
PQ2
Telepresence
EVC 4
PQ1
93
Customer2 - egress
BW
EVC3
Business Critical
Note: We count
hierarchies as follows:
4L hierarchy = 3 Level
nested p-map
3L hierarchy = 2 level
nested p-map
L1 level is not configurable
but is implicitly assumed
BW
BW
EVC 2
BW
Customer1 - egress
EVC1
Business Critical
BW
PQ1
BRKARC-2003
BW
PQ2
CPU
Packet
process in NP
FIA
NP
Switch
Fabric
NP
PHY
Egress LC
CPU
1
2
1.
2.
3.
4.
5.
6.
BRKARC-2003
Cisco Public
PHY
4
94
Packet
process in NP
Ingress LC
FIA
NP
PHY
Switch
Fabric
NP
BRKARC-2003
Egress LC
Cisco Public
CPU
PHY
PHY
PHY
95
Typhoon
8x55G
Fabric
Fabric
ASIC
FIA
Fabric
ASIC
MOD80/MOD160
Typhoon
8x55G
Fabric
10GE or
10x1GE
FIA
Fabric
ASIC
50G
30G
FIA
Typhoon
FIA
Typhoon
100GE
2x100GE
36x10GE
Typhoon
Fabric
ASIC
FIA
Fabric
ASIC
Typhoon
MOD80/MOD160
24x10GE
BRKARC-2003
FIA
Cisco Public
96
Typhoon
40GE
8 VQI per
40GE port
36x10GE
Fabric
ASIC
FIA
FIA
Typhoo
n
FIA
Typhoo
n
100G
E
2x100GE
Fabric
ASIC
Typhoo
n
Fabric
FIA
Fabric
ASIC
Typhoo
n
Fabric
ASIC
MOD80/MOD160
16 VQI per
100GE port
8x55G
8x55G
Fabric
10GE or
10x1GE
FIA
Fabric
ASIC
Typhoo
n
FIA
MOD80/MOD160
24x10GE
BRKARC-2003
97
Cisco Public
Load balancing
over
FIA-NP links per VQI
Typhoo
n
40GE
8 VQI per
40GE port
Typhoon
FIA
Fabric
ASIC
8x55G
8x55G
Fabric
ASIC
Fabric
FIA
FIA
Typhoon
1.
Fabric
100GE
2x100GE
Fabric
ASIC
BRKARC-2003
7
Typhoon
FIA
MOD80/MOD160
98
Typhoon
40GE
Cisco Public
99
PW Load-balancing scenarios
45 for ipv4
BRKARC-2003
Cisco Public
100
L2
MPLS
MPLS
45 (ipv4)
0000 (CW)
41-22-33 (mac)
4111.0000.
Control Word inserts additional zeros after the inner label showing the P nodes to go for label based balancing.
In EoMPLS, the inner label is VC label. So LB per VC then. More granular spread for EoMPLS can be achieved with
FAT PW (label based on FLOW inserted by the PE device who owns the service
BRKARC-2003
Cisco Public
101
BRKARC-2003
Cisco Public
102
BRKARC-2003
Cisco Public
103
Great references
Understanding NP counters
https://supportforums.cisco.com/docs/DOC-15552
BRKARC-2003
Cisco Public
104
Cisco Public
106
Open Flow
Agent
API Infrastructure
SDN Platform Independent Layer
SDN Platform Dependent Layer
Flow tables supporting full-match and wild card entries
Packet
classification
Match
ACL/EXP/BGP
Community String,
AS Path
BRKARC-2003
Packet
modification
actions
Forwarding
actions
Re-direct/copy to
IP, PW, GRE, vPath
Service chaining
Drop
Forward
Set DSCP/EXP/.1P
NAT actions
Cisco Public
107
Packet QOS
actions
Rate limit
Shape
Monitoring
actions
Counter Updates
Sampling rate for
copy
Cisco Public
108
BRKARC-2003
Cisco Public
109
Example 1:
Complex, mesh
network topologies,
multiple paths, need
network protocols
Example 2:
Ring topology, traffic
direction: East or
West, do I still need
those network
protocols?
Example 3:
Even a simpler case: P2P topology.
Why it need to run any protocol on
the access device? Why it even
need any forwarding table like FIB or
MAC?
BRKARC-2003
Cisco Public
111
ASR9K
local ports
nv fabric links
Satellite
(9000v, asr901, asr903)
Host (asr9k)
Satellite and ASR 9000 Host run satellite protocol for auto-discovery, provisioning and management
Satellite and Host could be co-located or in different location. There is no distance limitation
between satellite and Host
The connection between satellite and host is called nv fabric link, which could be L1 or over L2
virtual circuit (future)
Satellite access port have feature parity with ASR9K local ports
it works/feels just as local port
BRKARC-2003
Cisco Public
112
Redundant Fans
ToD/PSS Output
Bits Out
44x10/100/1000 Mbps
Pluggables
4x10G SFP+
Cisco Public
113
BITS 1)
Mgmt 1)
Ethernet
4x GE
(RJ45)
Console 2)
Cisco Public
114
4x GE Combo Ports
(SFP or RJ45)
4x GE
(SFP)
Fan Module
2x Power Modules
Cisco Public
MAC-DA
MAC-SA
Control VID
Payload/FCS
CPU
Discovery Phase
A CDP-like link-level protocol that discovers satellites and maintains a periodic heartbeat
Heartbeat sent once every second, used to detect satellite or fabric link failures. CFM based fast
failure detection plan for future release
Control Phase
Used for Inter-Process Communication between Host and Satellite
Cisco proprietary protocol over TCP socket, it could get standardized in the future
Get/Set style messages to provision the satellites and also to retrieve notifications from the
satellite
BRKARC-2003
Cisco Public
116
MAC-SA
VLANs (OPT)
Payload
MAC-DA
MAC-DA
nV-tag
MAC-SA
VLANs (OPT)
VLANs (OPT)
MAC-SA
Payload
Payload/FCS
On the Host
On the Satellite
BRKARC-2003
Cisco Public
117
Host
nv
satellite 101 define satellite
type asr9000v
interface TenGigE 0/2/0/2 configure satellite fabric port
nv
satellite-fabric-link satellite 101
remote-ports satellite to fabric port mapping
GigabitEthernet 0/0/0-9
BRKARC-2003
Cisco Public
118
Remote port:
int gig 101/0/0/1
Host
Local port
configuration examples
119
44x1GE
Access ports
4x10GE
Fabric ports
4x10GE
Fabric ports
Fabric link failure keeps all Access Ports up, rehashing of Traffic
Cisco Public
120
Low level debug could still be done directly on the satellite device
User can telnet into satellite via out-of-band management console, or in-band from Host,
and run regular show/debug CLIs
BRKARC-2003
Cisco Public
121
Cisco Public
122
Go
BRKARC-2003
Cisco Public
123
nV Satellite Evolution
High Dense
10G Satellite**
Topology expansion*
Feature offload***
Cisco Public
124
*** QoS offload in 5.1.1, SyncE offload in 5.2.0, Multicast offload in 5.2.2,
others are on roadmap
Linecard
chassis
Ctrl Links
Linecard
chassis
Fabric
chassis
CRS Multi-Chassis
InterChassis
Links
Ctrl
Links
Cisco Public
125
1
Active
RSP
LC
Standby
RSP
Secondary
RSP
LC
LC
LC
LC
LC
Secondary
RSP
LC
Internal
EOBC
LC
Control plane connection: Active RSP and standby RSP are on the different chassis, they
communicate via external EOBC links
Data plane connection: bundle regular data links into special nV fabric link to simulate switch fabric
function between two physical chassis for data packet
Flexible co-located or different location deployment (upto 10msec latency)
BRKARC-2003
Cisco Public
126
nV Edge Configuration
Configure nV Edge globally
nv
edge-system
serial FOX1437GC1R rack 1
serial FOX1439G63M rack 0
NO
to configure the
inter-chassis
control
plane
EOBC
Its plug-and-play
After
thisneed
configuration,
rack
1 will reload
and
then
joinports.
cluster
after it bootup
Now you successfully convert two standalone ASR 9000 into one ASR 9000 nV Edge
As simple as this !!!
BRKARC-2003
Cisco Public
127
unassigned
unassigned
unassigned
unassigned
Up
Shutdown
Up
Down
Up
Shutdown
Up
Down
GigabitEthernet100/1/1/0
Interfaces on a Satellite connected
to the nVUp
Edge Virtual SystemUp
unassigned
GigabitEthernet100/1/1/1.123
...
BRKARC-2003
Cisco Public
unassigned
128
Up
Up
Cisco Public
129
IOS-XR:
True
modular
OS
Superior
multicast
replication
Full HW
portfolio
with nV
Carrier Class,
Scalable
System
Fully
distributed
control
plane
Advanced
internal
system
QoS
Fully
distributed,
2-stag
forwarding
References
ASR9000/XR Feature Order of operation
Cisco Public
131
References
ASR9000/XR What is the difference between the -p- and -px- files ?
ASR9000/XR Flexible VLAN matching, EVC, VLAN-Tag rewriting, IRB/BVI and defining L2 services
ASR9000/XR Using Task groups and understanding Priv levels and authorization
Using Embedded Event Manager (EEM) in IOS-XR for the ASR9000 to simulate ECMP "min-links"
BRKARC-2003
Cisco Public
132
BRKARC-2003
Cisco Public
133