Frankfurt 2009
Nexus Family
Virtual Port Channel
Dieter Hadwiger
Systems Engineer Team Finance Germany
vPC peer
Standalone
Port-channel vPC vPC member port
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 8
vPC Configuration Commands
vPC_PL
vPC1 vPC2
Definition:
Port-channel member of a vPC peer.
Requirements:
Configuration needs to match other vPC
peer’s member port config.
In case of inconsistency a VLAN or the
entire port-channel may suspend (i.e.
MTU mismatch, inconsistent set of Vlans,
values and config).
Number of member ports on both vPC
vPC
member
peers is not required to match. port
ALWAYS
dual attach
devices to a vPC
Domain!!!
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 19
Attaching to a vPC Domain
IEEE 802.3ad and LACP
Definition:
Port-channel for devices for devices dual-attached to
the vPC pair.
Provides local load balancing for port-channel
members
STANDARD 802.3ad port channel
Access Device Requirements
STANDARD 802.3ad capability
LACP Optional
vPC
Recommendations:
vPC
Regular
Use LACP when available for better failover and mis- member
Port-
port
channel
configuration protection (config consistency check) port
* VLAN that is NOT part of any vPC and not present on vPC peer-link
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 21
Attaching to a vPC Domain
vPC and non-vPC VLANs (i.e. single attached .. )
P S P S
Orphan
Ports
P S
P S
P Primary vPC
S Secondary vPC
P S SR PR
P S
1. All devices Dual Attached via vPC 2. Separate vPC and STP VLANs
SR PR
P S
P Primary vPC
S Secondary vPC
Switch Switch
Po2 Po2
7k1 7k2
L3 ECMP
Po1
Router Router
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 28
Layer 3 and vPC
What can happen… (1 of 3)
7k vPC
7k1 7k2 7k1 7k2
R
R
R
R could be any router, Port-channel looks like Layer 3 will use ECMP
L3 switch or VSS a single L2 pipe. for northbound traffic
building a port-channel Hashing will decide
which link to chose
1) Packet arrives at R
S
2) R does lookup in routing table and sees 2
Po2
equal paths going north (to 7k1 & 7k2)
3) Assume it chooses 7k1 (ECMP decision)
4) R now has rewrite information to which
router it needs to go (router MAC 7k1 or
7k2)
5) L2 lookup happens and outgoing 7k1 7k2
interface is port-channel 1
Po1
6) Hashing determines which port-channel
member is chosen (say to 7k2)
7) Packet is sent to 7k2
R
8) 7k2 sees that it needs to send it over the
peer-link to 7k1 based on MAC address
Po1
L Loopguard
Primary Secondary
vPC vPC
vPC
HSRP Domain HSRP Layer 3
ACTIVE STANDBY
Aggregation
N N Secondary
Primary
Root Root
Layer 2 (STP + Rootguard)
- - - - - - - -
R R R R R R R R
-
Access
- - L
E E E E E
B B B B B
Layer 2 (STP + BPDUguard)
DC 1 vPC domain 11
Long Distance
vPC domain 21 DC 2
CORE
CORE
- F F -
- -
N N
N N
- - F F - -
R R
- R -
- R -
AGGR
AGGR
N N N N
ACCESS
E No Bridge Assurance on interconnecting vPCs E
B BPDU Filter on the edge devices to avoid BPDU propagation B
No L3 peering between DCs (i.e. L3 over vPC)
vPC vPC
CTS Manual Mode
(802.1AE 10GE line-rate
encryption)
No ACS is required
Cautions:
Not recommended using HSRP link tracking in a vPC configuration
Reason: vPC will not forward a packet back on a vPC once it has
crossed the peer-link, except in the case of a remote member port
failure
L3 CORE
VLAN 200
VLAN 100 VLAN 200
VLAN 100
OSPF
VLAN 99
L3 OSPF
L2
Primary Secondary
vPC vPC
Design considerations:
• Access switches requiring services are connected to sub-
aggregation VDC
• Access switches not requiring services may be connected to
aggregation VDC
• May be extended to support multiple virtualized service
contexts by using multiple VRF instances in the sub-
aggregation VDC
Design Cautions:
• Be aware of the Layer 3 over vPC design caveat. If Peering at
Layer 3 is required across the two vPC layers an alternative
solution should be explored (i.e. using STP rather than vPC to
attach service chassis)
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 47
Agenda
Nexus 7000 vPC Feature Overview & Terminology
Nexus 7000 vPC Design Guidance & Best Practices
Building a vPC domain
Attaching to a vPC domain
Layer 3 and vPC
Spanning Tree Recommendations
Data Center Interconnect (& Encryption)
HSRP with vPC
vPC and Services
vPC latest enhancements
ISSU
Problem/Impact:
Single attached devices that are not
connected via a vPC but still carry
vPC VLANs are also known as Port #1 Port #2
orphan ports. L3
vPC PL
OSPF L3 Core
Nexus 7000
NOTE: Convergence numbers may vary depending on the specific configuration (i.e. scaled
number of VLANs/SVIs or HSRP groups) and traffic patterns (i.e. L2 vs L3 flows).
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 59
vPC on Nexus 7000
Scalability Number Improvements
Release Supported Scalability
L3 Core
N7K-1 N7K-2
L2/L3 Aggregation
E1/26 E1/25
Po100 Po100
Te1/2/1 Te2/2/1
Please refer to CCO for more detailed information or refer to your CiscoSE
VSS VSS
L3 L3
L2 L2
4x10G uplinks
FE
FEX100 FEX102
FEX connectivity between
Nexus 5000 and Nexus FEX101
Port-channeling is the
Host Ports N2k01
pinning max-links 1
FEX101
http://www.cisco.com/en/US/partner/products/ps9670/products_installation_and_configuration_guides_list.html
mgmt0 vrf
nexus5k01 nexus5k02
vPC
Peer Keepalive
2-Ports vPCs
4+ Ports vPCs
5k01 5k02
5k01 5k02
eth2/1 eth2/2
eth2/1,2/2 eth2/3,2/4
vPC vPC
access
Peer Keepalive
Peer Link
mgmt0 mgmt0
mgmt0 mgmt0
Peer-link
primary secondary
Peer-link
primary secondary
5k01 5k02 5k01 5k02
“fabric links” “fabric links”
vPC 1 vPC 2
FEX100 FEX120
HIF HIF
FEX100 FEX120
2 ports
HIF HIF
vPC
Po10
max 12 FEXes
vPC
2-ports 4+ ports
vPC member port host host
2-ports port channel port channel
Peer Keepalive or FT link switch 4+ ports
port channel switch
vPC Peer Link aka MCT
port channel
Management Network
mgmt0 mgmt0
5k01 5k02
primary secondary
FEX100 FEX120
FEX101 FEX121
2-GigE ports host port channel single attached servers and/or A/S
Max 16 Ports
1 2 3 4 1 2 3 4
1 2 3 1 2 3
Max 16 Ports
1 2 3 4 1 2 3 4
5 6 7 8 5 6 7 8
1 2 3 1 2 3
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 82
Double-sided vPC between Nexus 7000 and Nexus5000 and FEX A/A
DESIGN 5 DESIGN 6
Max 16 Ports
1 2 3 4 1 2 3 4
1 3 1 3
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 83
16-ports Port-Channel
16 x 10 GigE ports
5k01 5k02
Peer-link
primary secondary
“fabric links”
vPC 1 vPC 2
FEX100 FEX120
HIF HIF
5k01 5k02
5k01 5k02
Failure
IP Cloud Boundary
Core
L3
L3 Aggregation
L2 vPC
L2MP
Access
L2
vPC vPC
Servers