BRKARC-3470
Session Goal
To provide you with a thorough understanding of the Cisco
Nexus 7000 switching architecture, supervisor, fabric, and
I/O module design, packet flows, and key forwarding engine
functions
This session will not examine Unified I/O, DCB, FCoE, NXOS software architecture, or other Nexus platforms
Related sessions:
BRKARC-3471: Cisco NXOS Software
Architecture
Presentation_ID
Cisco Public
Agenda
Chassis Architecture
Supervisor Engine Architecture
I/O Module Architecture
Forwarding Engine Architecture
Fabric Architecture
Layer 2 Forwarding
IP Forwarding
IP Multicast Forwarding
ACLs
QoS
NetFlow
Presentation_ID
Cisco Public
System status
LEDs
ID LEDs on
all FRUs
Front-toback airflow
Integrated cable
management
with cover
Air exhaust
Optional
locking front
doors
Locking
ejector
levers
Supervisor
slots (5-6)
Two chassis
per 7 rack
Crossbar fabric
modules
Power supplies
Air intake with
optional filter
Presentation_ID
Front
N7K-C7010Cisco Public
Rear
Common equipment
removes from rear
4
System
fan trays
System status
LEDs
Optional front
door
Side-to-side
airflow
Supervisor
slots (9-10)
Crossbar
fabric
modules
25RU
Common equipment
removes from rear
Power supplies
Power supply
air intake
Presentation_ID
Front
N7K-C7018
Cisco Public
Rear
Agenda
Chassis Architecture
Supervisor Engine Architecture
I/O Module Architecture
Forwarding Engine Architecture
Fabric Architecture
Layer 2 Forwarding
IP Forwarding
IP Multicast Forwarding
ACLs
QoS
NetFlow
Presentation_ID
Cisco Public
Supervisor Engine
Performs control plane and management functions
Dual-core 1.66GHz Intel Xeon processor with 4GB DRAM
2MB NVRAM, 2GB internal bootdisk, compact flash slots
Out-of-band 10/100/1000 management interface
Always-on Connectivity Management Processor (CMP)
for lights-out management
Console and auxiliary serial ports
USB ports for file transfer
ID LED
Status
LEDs
N7K-SUP1
AUX Port
Console Port
Presentation_ID
USB Ports
Management
Ethernet
Compact Flash
Slots
Cisco Public
CMP Ethernet
Reset Button
7
Management Interfaces
Management Ethernet
10/100/1000 interface used exclusively for system
management
Belongs to dedicated management VRF
Prevents data plane traffic from entering/exiting from mgmt0
interface
Cannot move mgmt0 interface to another VRF
Cannot assign other system ports to management VRF
Presentation_ID
Cisco Public
To Modules
To Modules
n * 23G
Switched
Gigabit
Ethernet
Arbitration
Path
Fabric
ASIC
Switched
EOBC
Arbitration
Path
VOQs
1GE EOBC
Central
Arbiter
1GE Inband
128MB
16MB
DRAM
Flash
System Controller
CMP
4GB
DRAM
Link
Encryption
2MB
2GB
PHY
NVRAM
Internal CF
Console
AUX
Mgmt
Enet
Security
Processor
Main
CPU
1.66GHz
Dual-Core
OBFL
Flash
10/100/1000
PHY
10/100/1000
slot0:
usb
usb
log-flash:
Presentation_ID
266MHz
Cisco Public
usb
CMP
Enet
Agenda
Chassis Architecture
Supervisor Engine Architecture
I/O Module Architecture
Forwarding Engine Architecture
Fabric Architecture
Layer 2 Forwarding
IP Forwarding
IP Multicast Forwarding
ACLs
QoS
NetFlow
Presentation_ID
Cisco Public
10
Presentation_ID
Cisco Public
11
To Central Arbiter
To Fabric Modules
Fabric ASIC
LC
CPU
VOQs
Forwarding
Engine
VOQs
Forwarding
Engine
Replication
Engine
Replication
Engine
Replication
Engine
Replication
Engine
10G MAC
10G MAC
10G MAC
10G MAC
10G MAC
10G MAC
10G MAC
10G MAC
Linksec
Linksec
Linksec
Linksec
Linksec
Linksec
Linksec
Linksec
Cisco Public
12
802.1AE LinkSec
Presentation_ID
Cisco Public
13
10G
11
13
15
To Fabric
rate-mode dedicated
Shared mode
10G
Dedicated mode
9
11
13
15
Presentation_ID
Cisco Public
14
To Fabric Modules
To Central Arbiter
Fabric ASIC
LC
CPU
VOQs
VOQs
Forwarding Engine
Replication
Engine
Replication
Engine
Replication
Engine
Replication
Engine
10G MAC
10G MAC
10G MAC
10G MAC
10G MAC
10G MAC
10G MAC
10G MAC
4:1 Mux +
Linksec
4:1 Mux +
Linksec
4:1 Mux +
Linksec
4:1 Mux +
Linksec
4:1 Mux +
Linksec
4:1 Mux +
Linksec
4:1 Mux +
Linksec
4:1 Mux +
Linksec
1 3 5 7
9 11 13 15
17 19 21 23
25 27 29 31
2 4 6 8
10 12 14 16 18 20 22 24 26 28 30 32
Cisco Public
15
N7K-M148GT-11
Release 4.0(1) and later
N7K-M148GS-11
Release 4.1(2) and later
Cisco Public
16
EOBC
To Central Arbiter
Fabric ASIC
VOQs
LC
CPU
Forwarding Engine
Replication
Engine
12 x 1G MAC
Linksec
1-12
12 x 1G MAC
Linksec
Linksec
Replication
Engine
12 x 1G MAC
Linksec
13-24
25-36
12 x 1G MAC
Linksec
Linksec
37-48
Cisco Public
17
Agenda
Chassis Architecture
Supervisor Engine Architecture
I/O Module Architecture
Forwarding Engine Architecture
Fabric Architecture
Layer 2 Forwarding
IP Forwarding
IP Multicast Forwarding
ACLs
QoS
NetFlow
Presentation_ID
Cisco Public
18
RACL/VACL/PACLs
Policy-based routing (PBR)
Hardware Table
M1 Modules
M1-XL Modules
without License
M1-XL Modules
with License
FIB TCAM
128K
128K
900K
64K
64K
128K
128K
128K
128K
NetFlow Table
512K
512K
512K
Presentation_ID
Cisco Public
19
Lic
Status Expiry Date Comments
Count
------------------------------------------------------------------------SCALABLE_SERVICES_PKG
Yes
In use Never
LAN_ADVANCED_SERVICES_PKG
Yes
In use Never
LAN_ENTERPRISE_SERVICES_PKG
Yes
In use Never
------------------------------------------------------------------------N7K#
Presentation_ID
Ins
Cisco Public
20
Layer 3 Engine
IPv4/IPv6 Layer 3 lookups
ACL, QoS, NetFlow and other processing
Linear, pipelined architectureevery packet subjected to both
ingress and egress pipeline
Enabling features does not affect forwarding engine performance
Presentation_ID
Cisco Public
21
Forwarding Engine
Pipelined Architecture
FE Daughter
Card
Ingress Pipeline
Ingress NetFlow
collection
Egress Pipeline
Layer 3
Engine
Ingress ACL
and QoS
classification
lookups
Egress policing
Egress NetFlow
collection
Unicast RPF
check
Ingress MAC
table lookups
IGMP snooping
lookups
IGMP snooping
redirection
Ingress policing
Egress MAC
lookups
IGMP snooping
lookups
Final lookup result to
I/O Module Replication Engine
Cisco Public
22
Agenda
Chassis Architecture
Supervisor Engine Architecture
I/O Module Architecture
Forwarding Engine Architecture
Fabric Architecture
Layer 2 Forwarding
IP Forwarding
IP Multicast Forwarding
ACLs
QoS
NetFlow
Presentation_ID
Cisco Public
23
N7K-C7010-FAB-1
N7K-C7018-FAB-1
Presentation_ID
Cisco Public
24
Crossbar
Fabric
ASICs
46Gbps/slot
Crossbar
Fabric
ASICs
46Gbps/slot
Crossbar
Fabric
ASICs
46Gbps/slot
Crossbar
Fabric
ASICs
46Gbps/slot
Crossbar
Fabric
ASICs
230Gbps
46Gbps
184Gbps
138Gbps
92Gbps
per slot bandwidth
Nexus 7018
Presentation_ID
Cisco Public
25
1G modules
Require 1 fabric for full
bandwidth
Require 2 fabrics for N+1
redundancy
46Gbps/slot
Crossbar
Fabric
ASICs
46Gbps/slot
Crossbar
Fabric
ASICs
46Gbps/slot
Crossbar
Fabric
ASICs
46Gbps/slot
Crossbar
Fabric
ASICs
46Gbps/slot
Crossbar
Fabric
ASICs
230Gbps
46Gbps
184Gbps
138Gbps
92Gbps
per slot bandwidth
4th and 5th fabric
modules provide
additional redundancy
and future-proofing
10G modules
Require 2 fabrics for full
bandwidth
Require 3 fabrics for N+1
redundancy
Presentation_ID
Cisco Public
26
Presentation_ID
Cisco Public
27
Presentation_ID
Cisco Public
28
Cisco Public
29
Agenda
Chassis Architecture
Supervisor Engine Architecture
I/O Module Architecture
Forwarding Engine Architecture
Fabric Architecture
Layer 2 Forwarding
IP Forwarding
IP Multicast Forwarding
ACLs
QoS
NetFlow
Presentation_ID
Cisco Public
31
Layer 2 Forwarding
MAC table is 128K entries (115K effective)
Hardware MAC learning
CPU not directly involved in learning
Presentation_ID
Cisco Public
32
Supervisor Engine
L2FM
L2FM-C
L2FM-C
L2FM-C
Hardware
Hardware
Hardware
I/O Module
I/O Module
I/O Module
Hardware
MAC Table
Cisco Public
33
Presentation_ID
Cisco Public
34
= Packet Headers
HDR
L2 Packet Flow
= Packet Data
DATA
= Internal Signaling
Supervisor Engine
Credit grant
for fabric
access
CTRL
Fabric Module 1
Fabric Module 2
Fabric Module 3
Fabric ASIC
Fabric ASIC
Fabric ASIC
Receive from
fabric
Return buffer
credit
11
VOQ arbitration
and queuing
Fabric ASIC
Layer 3
Engine
VOQs
Submit packet
headers for
lookup
Receive
packet
from wire
Layer 2
Engine
Replication
Engine
L2-only
SMAC/DMAC
lookup
14
Layer 3
Engine
Layer 2
Engine
L2 SMAC/
DMAC lookups
Forwarding
Engine
Return result
VOQs
Replication
Engine
Egress
port QoS
15
Module 2
Module 1
Fabric ASIC
Submit packet
headers for
egress L2
lookup
13
10G MAC
LinkSec decryption
1st stage ingress port QoS
e1/1
Presentation_ID
7
2
4:1 Mux +
Linksec
Forwarding
Engine
ACL/QoS/
NetFlow
lookups
10G MAC
12
Transmit
to fabric
Return
credit
to pool
Central Arbiter
10
Transmit
packet on
Cisco Publicwire
17
Linksec
LinkSec
encryption
16
e2/1
35
Agenda
Chassis Architecture
Supervisor Engine Architecture
I/O Module Architecture
Forwarding Engine Architecture
Fabric Architecture
Layer 2 Forwarding
IP Forwarding
IP Multicast Forwarding
ACLs
QoS
NetFlow
Presentation_ID
Cisco Public
36
IP Forwarding
Nexus 7000 decouples control plane and data
plane
Forwarding tables built on control plane using
routing protocols or static configuration
OSPF, EIGRP, IS-IS, RIP, BGP for dynamic routing
Presentation_ID
Cisco Public
37
IP Forwarding Architecture
Routing protocol processes learn routing
information from neighbors
IPv4 and IPv6 unicast RIBs calculate
routing/next-hop information
Supervisor Engine
BGP OSPF
ISIS
RIP
EIGRP
URIB/U6RIB
UFDM
IP FIB
IP FIB
IP FIB
Hardware
Hardware
Hardware
sh processes
Runtime(ms)
93
sh processes
117
150
sh processes
1272
Process
ospf
u6rib
urib
I/O Module
I/O Module
I/O Module
Hardware
ufdm
FIB TCAM
ADJ Table
ipfib
Cisco Public
38
Presentation_ID
Cisco Public
39
Ingress
unicast IPv4
packet header
Generate
Lookup Key
10.1.1.10
HIT!
Compare
lookup
key
10.1.1.2
Index, # next-hops
10.1.1.3
Index, # next-hops
10.1.1.4
Index, # next-hops
10.10.0.10
Index, # next-hops
10.10.0.100
Index, # next-hops
10.10.0.33
Index, # next-hops
10.1.1.xx
10.1.2.xx
Index, # next-hops
10.1.3.xx
Index, # next-hops
10.10.100.xx
Index, # next-hops
10.1.1.xx
Index, # next-hops
10.100.1.xx
Index, # next-hops
10.10.0.xx
Index, # next-hops
10.100.1.xx
FIB TCAM
Presentation_ID
Forwarding Engine
Flow
Data
Hit in FIB
Index,
returns
result#
in FIB DRAM
Load-Sharing
Hash
Offset
Adj Index
next-hops
Adjacency
index
identifies ADJ
block
FIB DRAM
2010 Cisco and/or its affiliates. All rights reserved.
Cisco Public
Hash
selects
exact next
hop entry
Result
Adjacency Table
41
Presentation_ID
Cisco Public
42
Displaying Routing
and Forwarding Information
show routing [ipv4|ipv6] [<prefix>]
[vrf <vrf>]
Displays software routing (URIB) information
Can also use traditional show ip route command
Presentation_ID
Cisco Public
43
Displaying Routing
and Forwarding Information (Cont.)
n7010# sh routing ipv4 10.100.7.0/24
IP Route Table for VRF "default"
10.100.7.0/24, 1 ucast next-hops, 0 mcast next-hops
*via 10.1.2.2, Ethernet9/2, [110/5], 00:02:30, ospf-1, type-1
n7010# show forwarding ipv4 route 10.100.7.0/24 module 9
IPv4 routes for table default/base
------------------+------------------+--------------------Prefix
| Next-hop
| Interface
------------------+------------------+--------------------10.100.7.0/24
10.1.2.2
Ethernet9/2
rewrite info
interface
--------------
----------
Cisco Public
44
10.10.0.0/16
10.10.0.0/16
via Rtr-A
via Rtr-B
Cisco Public
45
| Next-hop
| Interface
------------------+------------------+--------------------10.200.0.0/16
10.1.1.2
Ethernet9/1
10.1.2.2
Ethernet9/2
n7010#
Presentation_ID
Cisco Public
46
Cisco Public
47
= Packet Headers
HDR
L3 Packet Flow
= Packet Data
DATA
CTRL
= Internal Signaling
Supervisor Engine
9
Fabric Module 2
Fabric Module 3
Fabric ASIC
Fabric ASIC
Fabric ASIC
Receive from
fabric
Return buffer
credit
Transmit to
fabric
10
6
Fabric ASIC
Layer 3
Engine
VOQs
Submit packet
headers for
lookup
Forwarding
Engine
L2-only
SMAC/DMAC
lookup
L2 ingress and
egress SMAC/
DMAC lookups
Layer 2
Engine
Forwarding
Engine
VOQs
Replication
Engine
Egress port
QoS
15
Module 2
Module 1
2010 Cisco and/or its affiliates. All rights reserved.
Fabric ASIC
Submit packet
headers for
egress L2
lookup
13
10G MAC
LinkSec decryption
1st stage ingress port QoS
e1/1
Presentation_ID
Layer 3
Engine
7
2
4:1 Mux +
Linksec
14
Return result
10G MAC
L3 FIB/ADJ
lookup
Ingress and
egress ACL/QoS/
NetFlow lookups
Layer 2
Engine
Replication
Engine
Receive
packet from
wire
11
12
Central Arbiter
Fabric Module 1
VOQ arbitration
and queuing
Return
credit to
pool
Transmit
packet on
Cisco Public wire
17
Linksec
LinkSec
encryption
16
e2/1
48
Agenda
Chassis Architecture
Supervisor Engine Architecture
I/O Module Architecture
Forwarding Engine Architecture
Fabric Architecture
Layer 2 Forwarding
IP Forwarding
IP Multicast Forwarding
ACLs
QoS
NetFlow
Presentation_ID
Cisco Public
49
IP Multicast Forwarding
Forwarding tables built on control plane using
multicast protocols
PIM-SM, PIM-SSM, PIM-Bidir, IGMP, MLD
Presentation_ID
Cisco Public
50
Supervisor Engine
PIM
MFDM
IP FIB
IP FIB
IP FIB
Hardware
Hardware
Hardware
Adjacency table
Multicast Expansion Table (MET)
n7010#
PID
3842
3850
n7010#
3843
3847
n7010#
3846
sh processes
Runtime(ms)
109
133
sh processes
177
115
sh processes
2442
I/O Module
I/O Module
Hardware
FIB TCAM
I/O Module
MET
ADJ Table
ipfib
Cisco Public
51
Hardware Programming
IP FIB process on I/O modules programs hardware:
FIB TCAM
Part of Layer 3 Engine ASIC on forwarding engine
Consists of (S,G) and (*,G) entries as well as RPF interface
Presentation_ID
Cisco Public
52
Multicast FIB
TCAM Lookup
Ingress
multicast
packet header
Generate TCAM
lookup key (source
and group IP address)
Generate
Lookup Key
10.1.1.10, 239.1.1.1
Forwarding Engine
Compare
lookup
key
10.1.1.12, 239.1.1.1
MET Index
10.1.1.10, 232.1.2.3
MET Index
10.4.7.10, 225.8.8.8
MET Index
MET Index
10.6.6.10, 239.44.2.1
FIB TCAM
FIB DRAM
MET Index
Result
Adjacency Table
Hit in FIB
returns result
in FIB DRAM
Replication
Engine
Adj Index
Identifies multicast
adjacency entry
Return
lookup
result
OIFs
OIFs
Replication for
each OIF in
MET block
Replicate
OIFs
OIFs
MET
Presentation_ID
Cisco Public
Presentation_ID
Cisco Public
55
packets
bytes
aps
pps
bit-rate
oifs
(*,G)
767
84370
110
0 bps
10.1.1.2
9917158
1269395810
127
4227
4 mbps
10.1.1.3
9917143
1269393890
127
4227
4 mbps
10.1.1.4
9917127
1269391824
127
4227
4 mbps
<>
Presentation_ID
Cisco Public
56
n7010# sh forwarding ipv4 multicast route group 239.1.1.1 source 10.1.1.2 module 9
Presentation_ID
Cisco Public
57
Egress Replication
IIF
Module 1
Replication
MET
Engine
Fabric ASIC
Local
OIF
Fabric
Module
Fabric
Copy
Fabric ASIC
Fabric ASIC
Fabric ASIC
Fabric ASIC
Replication
MET
Engine
Replication
MET
Engine
Replication
MET
Engine
Local
OIFs
Local
OIFs
Local
OIFs
Cisco Public
58
= Packet Headers
HDR
DATA
= Packet Data
Fabric Module 2
Fabric Module 3
Fabric ASIC
Fabric ASIC
Fabric ASIC
Transmit to
fabric
Dequeue multicast
distribution copy
from fabric
11
VOQ queuing
10
Fabric ASIC
Transmit
multicast fabric
distribution
packet
VOQs
Submit packet
headers for
lookup
Replication
Engine
e1/1
Egress ACL/QoS/
NetFlow lookups
Layer 2
Engine
Forwarding
Engine
L2 ingress
snooping
lookup
Return MET
result
Replicate for
fabric delivery
4:1 Mux +
Linksec
Layer 3
Engine
L3 multicast FIB
lookup
Ingress ACL/QoS/
NetFlow lookups
10G MAC
Receive
packet from
wire
12
8
Module 1
16
Layer 3
Engine
15
Forwarding
Engine
13
Replication
Engine
Egress port
QoS
L2 egress
snooping
lookup
17
Module 2
2
LinkSec decryption
1st stage ingress port QoS
VOQs
Replicate for
local OIF
delivery
Submit packet
headers for
egress lookups
14
10G MAC
19
Presentation_ID
Layer 2
Engine
Fabric ASIC
Transmit
packet on
wire
Cisco Public
Linksec
LinkSec
encryption
18
e2/1
59
Multicast traffic
Locally arbitrated
Load balanced over available fabric channels using hash
Presentation_ID
Cisco Public
60
Agenda
Chassis Architecture
Supervisor Engine Architecture
I/O Module Architecture
Forwarding Engine Architecture
Fabric Architecture
Layer 2 Forwarding
IP Forwarding
IP Multicast Forwarding
ACLs
QoS
NetFlow
Presentation_ID
Cisco Public
61
Security ACLs
Enforce security policies based on Layer 2, Layer 3,
and Layer 4 information
Classification TCAM (CL TCAM) provides ACL lookups
in forwarding engine
64K hardware entries
Cisco Public
62
ACL Architecture
ACL manager receives
policy via configuration
Supervisor Engine
CLI
Presentation_ID
XML
ACL Manager
ACL/QoS-C
ACL/QoS-C
ACL/QoS-C
Hardware
Hardware
Hardware
I/O Module
I/O Module
I/O Module
Hardware
CL TCAM
aclqos
Cisco Public
63
Security ACL
ip access-list example
Packet header:
SIP: 10.1.1.1
DIP: 10.2.2.2
Protocol: TCP
SPORT: 33992
DPORT: 80
Generate
Lookup Key
Forwarding Engine
Compare
lookup key
X=Mask
Presentation_ID
xxxxxxx| 10.1.2.100
| 10.2.2.2 | |xx
xxxxxxx
xx| xxx
| xxx| xxx
| xxx
Permit
Deny
Deny
xxxxxxx
xxxxxxx| |06
06| |xxx
xxx| 0050
| 0016
xxxxxxx | xxxxxxx
Permit
Deny
Deny
Permit
Permit
CL TCAM
CL SRAM
Hit in CL TCAM
returns result in
CL SRAM
Cisco Public
Result
Result affects
final packet
handling
Return
lookup
result
64
Hardware Modules
Used
Free
Percent
Utilization
----------------------------------------------------Tcam 0, Bank 0
16383
0.000
Tcam 0, Bank 1
4121
12263
25.000
Tcam 1, Bank 0
4013
12371
24.000
Tcam 1, Bank 1
4078
12306
24.000
LOU
102
1.000
TCP Flags
16
0.000
Protocol CAM
57.000
14
0.000
6140
0.000
6140
0.000
L4 op labels, Tcam 0
2047
0.000
L4 op labels, Tcam 1
2046
0.000
Presentation_ID
n7010#
Cisco Public
65
Agenda
Chassis Architecture
Supervisor Engine Architecture
I/O Module Architecture
Forwarding Engine Architecture
Fabric Architecture
Layer 2 Forwarding
IP Forwarding
IP Multicast Forwarding
ACLs
QoS
NetFlow
Presentation_ID
Cisco Public
68
Quality of Service
Comprehensive LAN QoS feature set
Ingress and egress queuing and scheduling
Applied in I/O module port ASICs
Presentation_ID
Cisco Public
69
QoS Architecture
QoS manager receives
policy via configuration
Supervisor Engine
CLI
Presentation_ID
Process
ipqosmgr
XML
QoS Manager
ACL/QoS-C
ACL/QoS-C
ACL/QoS-C
Hardware
Hardware
Hardware
I/O Module
I/O Module
I/O Module
Hardware
CL TCAM
aclqos
Cisco Public
70
Queue Structure
8q2t ingress
1p7q4t egress
Presentation_ID
Cisco Public
71
Ingress
12345678
8q2t
Replication
Engine
Port 1
96MB
Port 1
80MB
Port ASIC
12345678
1p7q4t
Egress
Presentation_ID
Cisco Public
72
Egress
Dedicated mode: 80MB per port
Shared mode: 80MB per port-group
Queue Structure
8q2t + 2q1t ingress
1p7q4t egress
Presentation_ID
Cisco Public
73
Ingress
(Fixed)
12
Ports 1,3,5,7
65MB
2q1t
12345678
Replication
Engine
Port 1,3,5,7
80MB
Port ASIC
Port 1
1MB
12345678
1p7q4t
Port 3
1MB
4:1 Mux
8q2t
Port 5
1MB
Port 7
1MB
1,3,5,7
Port Group
Presentation_ID
Cisco Public
Egress
74
Ingress
(Fixed)
12
Port 1
65MB
2q1t
12345678
Replication
Engine
Port 1
80MB
Port ASIC
12345678
1p7q4t
Port 1
1MB
4:1 Mux
8q2t
1,3,5,7
Port Group
Presentation_ID
Cisco Public
Egress
75
Queue Structure
2q4t ingress
1p3q4t egress
Presentation_ID
Cisco Public
76
Ingress
Replication
Engine
12
2q4t
Port 10
7.6MB
PortPort
7 4
7.6MB
7.6MB
Port 1
Port 11
7.6MB
7.6MB
PortPort
8 5
7.6MB
7.6MB
Port 2
Port 12
7.6MB
7.6MB
PortPort
9 6
7.6MB
7.6MB
Port 3
7.6MB
Port 10
6.2MB
PortPort
7 4
6.2MB
6.2MB
Port 1
Port 11
6.2MB
6.2MB
PortPort
8 5
6.2MB
6.2MB
Port 2
Port 12
6.2MB
6.2MB
PortPort
9 6
6.2MB
6.2MB
Port 3
6.2MB
Port ASIC
1-12
1234
1p3q4t
Egress
Presentation_ID
Cisco Public
77
Presentation_ID
Cisco Public
78
Packet header:
SIP: 10.1.1.1
DIP: 10.2.2.2
Protocol: TCP
SPORT: 33992
DPORT: 80
Generate
Lookup Key
ip access-list remark-prec-3
permit tcp any 10.5.5.0/24 eq 23
Forwarding Engine
xxxxxxx | 10.2.2.xx
10.3.3.xx | xx | xxx | xxx
Policer ID 1
Policer ID 1
10.1.1.xx | xxxxxxx | 11
xxx
06 || xxx
xxx|| xxx
Remark DSCP 32
Remark DSCP 40
Remark IP Prec 3
CL TCAM
Presentation_ID
Result
Result affects
final packet
handling
CL SRAM
Hit in CL TCAM
returns result in
CL SRAM
Cisco Public
Return
lookup
result
79
enabled
Ethernet9/1
Service-policy (qos) input:
policy statistics status:
Class-map (qos):
mark
enabled
udp-mcast (match-all)
432117468 packets
Match: access-group multicast
set dscp cs4
Class-map (qos):
udp (match-all)
76035663 packets
Match: access-group other-udp
police cir 2 mbps bc 1000 bytes pir 4 mbps be 1000 bytes
conformed 587624064 bytes, 3999632 bps action: transmit
exceeded 293811456 bytes, 1999812 bps action: set dscp dscp table cir-markdown-map
violated 22511172352 bytes, 153221133 bps action: drop
n7010#
Presentation_ID
Cisco Public
80
Agenda
Chassis Architecture
Supervisor Engine Architecture
I/O Module Architecture
Forwarding Engine Architecture
Fabric Architecture
Layer 2 Forwarding
IP Forwarding
IP Multicast Forwarding
ACLs
QoS
NetFlow
Presentation_ID
Cisco Public
81
NetFlow
NetFlow table is 512K entries (490K effective),
shared between ingress/egress NetFlow
Hardware NetFlow entry creation
CPU not involved in NetFlow entry creation/update
Presentation_ID
Cisco Public
82
NetFlow Architecture
Supervisor Engine
NetFlow manager
receives configuration via
CLI/XML
NetFlow manager
distributes configuration
to NetFlow-Clients on
I/O modules
NetFlow-Clients apply
policy to hardware
CLI
XML
NetFlow Manager
NF-C
NF-C
NF-C
Hardware
Hardware
Hardware
I/O Module
I/O Module
I/O Module
Hardware
NF Table
nfp
Cisco Public
83
Presentation_ID
Cisco Public
86
SrcAddr
DstAddr
L4 Info
PktCnt
TCP Flags
-+-----+---------------+---------------+---------------+----------+----------I 9/1
I 9/1
I 9/1
<>
n7010# sh hardware flow ip interface e9/1 detail module 9
D - Direction; IF - Intf/VLAN; L4 Info - Protocol:Source Port:Destination Port
TCP Flags: Ack, Flush, Push, Reset, Syn, Urgent; FR - FRagment; FA - FastAging
SID - Sampler/Policer ID; AP - Adjacency/RIT Pointer
CRT - Creation Time; LUT - Last Used Time; NtAddr - NT Table Address
D IF
SrcAddr
DstAddr
L4 Info
PktCnt
TCP Flags
-+-----+---------------+---------------+---------------+----------+----------ByteCnt
TOS FR FA SID
AP
CRT
LUT
NtAddr
-------------+---+--+--+-----+--------+-----+-----+-------I 9/1
0000218460416 000 N
Presentation_ID
Cisco Public
89
Generate NetFlow v5 or
v9 export packets
To NetFlow Collector
I/O Module
Fabric
ASIC
LC
CPU
Supervisor
Engine
NetFlow
Table
Aged Flows
Hardware
Flow Creation
Forwarding
Engine
via Inband
VOQs
I/O Module
Main
CPU
LC
CPU
Switched
EOBC
NetFlow
Table
Aged Flows
Forwarding
Engine
via mgmt0
I/O Module
Mgmt
Enet
LC
CPU
NetFlow
Table
Aged Flows
To NetFlow Collector
Presentation_ID
Forwarding
Engine
Hardware
Flow Creation
Cisco Public
Hardware
Flow Creation
90
Conclusion
You should now have a thorough
understanding of the Nexus
7000 switching architecture, I/O
module design, packet flows,
and key forwarding engine
functions
Any questions?
Presentation_ID
Cisco Public
92
Q and A
Presentation_ID
Cisco Public
94
Presentation_ID
95