Anda di halaman 1dari 86

BRK-3035

Advanced Enterprise Campus Design : Virtual Switching System (VSS)


Rahul Kachalia

BRKDCT-2256

2013 Cisco and/or its affiliates. All rights reserved.

Cisco Public

Enhancing Campus HA

Most Common Causes of Downtime


Network Design and Best Practices
Network 20%

Operational Process 40% Software Application 40%

System and Network Level Resiliency


Hardware Other Power Failure 8% Failure 12% 14% Human Error 31% Telco/ISP 35%

Sources of Network Downtime*

Embedded Management

Common Causes of Enterprise Network Downtime** *Source: Gartner Group **Source: Yankee Group
BRKDCT-2256 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public

Enterprise Class Availability

Resilient Campus Communication Fabric


Systems Design Approach to High Availability

VOIP availability is the baseline for the enterprise networks


Human ear notices the difference in voice within 150200 msec, which translates only ten consecutive packet loss with G711 codec

Ultimate Goal..100%
Next-Generation Apps Video Conf., Unified Messaging, Global Outsourcing, E-Business, Wireless Ubiquity Mission-Critical Apps, Databases, Order-Entry, CRM, ERP

Video loss is even more noticeable and it is rapidly becoming new frontier for jitter and delay requirements 200 msec end-to-end campus convergence is the design goal
BRKDCT-2256

Desktop Apps E-Mail, File, and Print

Applications Drive Requirements for High Availability Networking


Cisco Public

2013 Cisco and/or its affiliates. All rights reserved.

Cisco VSS Key Benefits

Reduce 50% of Managed Nodes Loop-free topology LMS 3.0 integration

Simplifies Operational Manageability

Boosts Non-Stop Communications

Deterministic sub-sec network recovery Business continuity with no service disruption

Si

Si

Supported Platforms Catalyst 6500E Catalyst 4500E Catalyst 4500X

Maximize system usage Maximize server usage NIC standardization

Maximizes Bandwidth Utilization

Lowers Latency

Optimized path selection Increased throughput

Design Guide: www.cisco.com/go/srnd http://www.cisco.com/en/US/docs/solutions/Enterprise/Campus/Borderless_Campus_Network_1.0/Borderless_Campus_1.0_Design_Guide.html http://www.cisco.com/en/US/docs/solutions/Enterprise/Campus/VSS30dg/campusVSS_DG.html


BRKDCT-2256 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public

VSS Enabled Campus Design


End-to-End VSS Design Option

Si

Si

Si

Si

Si

Si

Si

Si

Si

Si

Si

Si

WAN
BRKDCT-2256

Data Center

Internet
Cisco Public 6

2013 Cisco and/or its affiliates. All rights reserved.

Advance Virtual Switching System Design


Agenda
Cisco VSS Architecture VSS Architecture Overview Unified System Architecture Designing VSS System Redundancy VSS Dual and Quad-Sup Redundancy Design Virtual Switch Link Design and Best Practices Designing VSS Network Redundancy Multi-Chassis EtherChannel and ECMP Design Load Sharing and Resiliency Designing VSS Enabled Campus Network Access Layer

Distribution and Core Layer Design, Best Practices and Failure Analysis
VSS Dual Active Detection Understanding Dual Active and Recovery Mechanics Dual Active Best Practices and Failure Analysis Summary
7 BRKDCT-2256 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public

Cisco VSS Architecture Overview


Catalyst 6500E/4500E
Line Card
SF PFC RP Inter-Chassis SSO Redundancy SF

Catalyst 6500E/4500E
Line Card
PFC RP

Active Sup
Intra-Chassis SSO Redundancy Internal EOBC External EOBC (VSL)

Standby Sup
Internal EOBC

SF

PFC

RP

Standby Sup

Line Card

Line Card

Standalone VSS-SW1

VSS-SW2

Internal EOBC : Internal communication control channel between supervisor and linecards within single-chassis External EOBC : External communication control channel between supervisors between two-chassis

SF : Switch Fabric PFC : Policy Feature Card RP : Route Processor EOBC : Ethernet Out-of-Band Channel
8 Cisco Public

BRKDCT-2256

2013 Cisco and/or its affiliates. All rights reserved.

Unified System Architecture

Catalyst 6500E/4500E

Line Card

Catalyst 6500E/4500E
Line Card
SF PFC

Simplified Control-Plane
VSS#show switch virtual redundancy My Switch Id = 1 Peer Switch Id = 2 Switch 1 Slot 5 Processor Information : ----------------------------------------------Current Software state = ACTIVE <snip> Configuration register = 0x2 Fabric State = ACTIVE Control Plane State = ACTIVE

Line Card
Line Card
PFC RP

Line Card Line Card Active Sup


VSL

Common Management
RP

Single Control-Plane to manage two Active Sup physical systems Consistent IOS software feature Switch 2 Slot 5 Processor : parity as Information Standalone ---------------------------------------------- Centralized Programming for distributed forwarding
Current Software state = STANDBY HOT (switchover target) <snip> Configuration register = 0x2 Fabric State = ACTIVE Control Plane State = STANDBY

SF

Standby Sup

Standby Sup Line Card Line Card

Single virtual system for OOB/In-Band management of two physical systems


Common SNMP MIBs, Traps with advance VSS MIBS

Line Card

Line Card
Line Card

Line Card

Single troubleshooting point

VSS-SW1

VSS-SW2 SW1

BRKDCT-2256

2013 Cisco and/or its affiliates. All rights reserved.

Cisco Public

Unified Forwarding Architecture


Catalyst 4500E (Centralized Forwarding Architecture) Catalyst 4500X (Centralized Forwarding Architecture) Catalyst 6500E (Distributed Forwarding Architecture)

Line Card
Line Card Line Card Line Card Active Sup Standby Sup Line Card Line Card Line Card Line Card Active Switch Standby Switch

Line Card Line Card Line Card Line Card

Layer 2 / 3 Network

Layer 2 / 3 Network

Active Sup Standby Sup Line Card Line Card Line Card Line Card

Layer 2 / 3 Network

SW1

SW1

SW1

Catalyst 4500E
VSS Active Supervisor builds and maintain network topologies Programs Forwarding Engine on both virtual switch supervisor module

Catalyst 4500X
Same Forwarding Architecture as Catalyst 4500E

Catalyst 6500E
Hybrid Forwarding Design Distributed/Centralized

VSS Active supervisor builds and maintain network topologies


Distributed Inter + Intra-Chassis Forwarding Centralized Intra-Chassis Forwarding
Cisco Public 10

Distributed Inter-Chassis Forwarding. Centralized Intra-Chassis Forwarding design


BRKDCT-2256 2013 Cisco and/or its affiliates. All rights reserved.

Advance Virtual Switching System Design


Agenda
Cisco VSS Architecture VSS Architecture Overview Unified System Architecture Designing VSS System Redundancy

VSS Dual and Quad-Sup Redundancy Design


Virtual Switch Link Design and Best Practices Designing VSS Network Redundancy Multi-Chassis EtherChannel and ECMP Design Load Sharing and Resiliency Designing VSS Enabled Campus Network Access Layer Distribution and Core Layer Design, Best Practices and Failure Analysis VSS Dual Active Detection

Understanding Dual Active and Recovery Mechanics


Dual Active Best Practices and Failure Analysis Summary
11 BRKDCT-2256 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public

VSS Dual-Sup Inter-Chassis Redundancy


VSS Dual-Sup (single sup per chassis) supports interchassis SSO redundancy.
Single in-chassis supervisor - SSO Active or Standby role.

Stateful SSO synchronization and redundancy between virtual-switches


Single Sup System Design
VSL

Reduced NSF Recovery Capacity Reduced Capacity

Supervisor switchover requires chassis reset, including all linecard and service modules Network capacity reduced until system returns to operational state

Active Standby

Standby Active

Reduced Reduced Capacity Capacity

Consistent redundancy design between modular Catalyst 6500E/4500E and fixed Catalyst 4500X system

Si

12 BRKDCT-2256 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public

Catalyst 6500E VSS Quad-Sup with RPR-WARM


Sup720-10GE Quad-Sup Redundancy
Starting 12.2(33)SXI4 Sup720-10GE VSS supports two sup redundancy modes :
Dual-Sup One Sup per virtual-switch
Quad-Sup Two Sups per virtual-switch

Dual Sup offers single redundancy option


Inter-Chassis only. Resetting Active or Standby supervisor reboots all installed modules Sup hardware failure may increase MTTR, reduce network capacity, services availability and may build un-reliable network
Self Recovery Fail New Active Supervisor

NSF Recovery Reduced Capacity

VSL

Quad Sup offers dual redundancy options


Inter-Chassis Same design as dual-sup

Single Point of Failure

Intra-Chassis Allows virtual switch to return in-service, reduce MTTR and stabilize network from major fault
Si

Reduced Capacity

13 BRKDCT-2256 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public

VSS Quad Sup Supports Dual HA Mode


Sup720-10GE Quad-Sup Redundancy
Inter-Chassis Sup Redundancy

Intra-Chassis Sup Redundancy ICA SSO Active ICS RPR-WARM

VSL

Intra-Chassis Sup Redundancy ICA SSO Standby ICS RPR-WARM

Si
SW1

Si
SW2

Dual in-chassis supervisors, each in different redundancy modes


In-chassis Active Supervisor (ICA) In SSO Active OR Standby Mode In-chassis Standby Supervisor (ICS) RPR-WARM Mode

Stateful SSO synchronization from SSO Active to Standby supervisor


System configuration synchronization between ICA and ICS supervisors Chassis reset when ICA supervisor reset
14 BRKDCT-2256 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public

Catalyst 6500E Quad-Sup NSF/SSO Redundancy


Non-Stop Network Availability and Performance
Inter-Chassis Sup Redundancy
Shipping in March 2013

Intra-Chassis Sup Redundancy ICA SSO Active ICS STANDBY-HOT ( Chassis)

VSL

Intra-Chassis Sup Redundancy ICA SSO Standby ICS STANDBY-HOT(Chassis)

Si
SW1

Si
SW2

Dual in-chassis Sup2T supervisors, each in different redundancy modes


In-chassis Active Supervisor (ICA) SSO Active OR Standby-Hot (switchover target) In-chassis Standby Supervisor (ICS) Standby-Hot (Chassis)

VSS Quad-Sup protects network availability and capacity with dual redundancy domain Stateful SSO synchronization between multiple redundancy domains Complete system configuration and parameters synchronization between Quad supervisors Chassis and modules remains operational when Active or Standby-Hot supervisor resets
BRKDCT-2256 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 15

Catalyst 4500E VSS Quad-Sup


Intra-Chassis Sup Redundancy ICA SSO Active ICS ROMMON

Inter-Chassis Sup Redundancy

Intra-Chassis Sup Redundancy ICA SSO Standby ICS ROMMON

Catalyst 4500E VSS software leverages existing standalone supervisor redundancy architecture ICS supervisor must be manually forced to go in ROMMON mode No VSS capability in software release:
Cannot synchronize VSS parameters Cannot synchronize system configuration Cannot synchronize Cisco IOS software during migration

Si
SW1

VSL

Si
4500E SW2

4500E-VSS#show module | inc Switch|Sup Switch Number: 1 Role: Virtual Switch Active 3 4 Sup 7-E 10GE (SFP+), 1000BaseX (SFP) WS-X45-SUP7-E 4 4 Sup 7-E 10GE (SFP+), 1000BaseX (SFP) 3 Active Supervisor SSO Active Switch Number: 2 Role: Virtual Switch Standby 3 4 Sup 7-E 10GE (SFP+), 1000BaseX (SFP) WS-X45-SUP7-E 4 4 Sup 7-E 10GE (SFP+), 1000BaseX (SFP) 3 Standby Supervisor SSO Standby hot

CAT1634L277

CAT1633L09W

Not supported feature and not recommended system design.

4500E-VSS#show switch virtual redundancy | inc Id|Mode|Slot|Fabric|Control My Switch Id = 1 Peer Switch Id = 2 Configured Redundancy Mode = Stateful Switchover Operating Redundancy Mode = Stateful Switchover Switch 1 Slot 3 Processor Information : Fabric State = ACTIVE Control Plane State = ACTIVE Switch 2 Slot 3 Processor Information : Fabric State = ACTIVE Control Plane State = STANDBY

16 BRKDCT-2256 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public

Standalone to VSS Conversion


System and Sup Redundancy Independent Process
Step-1 : Configure VSS Domain ID
Common Domain ID between two pairing systems Unique Domain ID network-wide. Duplicate ID may fail L2 protocols Range 1-255 Step-2 : Configure Switch ID Unique Switch ID per switch in same VSS Domain Range 1-2
SW1 Po1 SW1 Po2 SW2

Si

VSL VSL

Si

Step-3 : Configure Switch Priority (Optional)


Range 1-255. Default 100 Step-4 : Configure VSS Virtual MAC-Address Virtual MAC Address for reliable Layer 3 communication Step-5 : Configure VSL EtherChannel Unique Port-Channel per switch

SW1
Step-1 SW1(config)#switch virtual domain 10

SW2
SW2(config)#switch virtual domain 10

Unique Switch Priority per switch in same VSS Domain Step-2 SW1(config-vs)#SW1 switch 1
Step-3 SW1(config)# SW1(config-vs)# switch priority 110 switch convert mode virtual Step-4 Step-5

SW2(config-vs)#SW2 switch 2 SW2(config-vs)# switch priority 100 SW2(config)# switch convert mode virtual SW2(config-vs)# mac-address use-virtual SW2(config)#interface Port-Channel 2 SW2(config-if)#switch virtual 1 ! SW2(config-if)#interface range Ten5/1 2 SW2(config-if-range)#channel-group 2 mode on

SW1(config-vs)#mac-address use-virtual SW1(config)#interface Port-Channel 1 SW1(config-if)#switch virtual 1 ! SW1(config-if)#interface range Ten5/1 2 SW1(config-if-range)#channel-group 1 mode on

Up to 8 physical ports bundle in VSL EtherChannel


BRKDCT-2256 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 17

VSS Supervisor Redundancy Summary


Catalyst 6500E Sup2T Catalyst 6500E Sup720-10GE Catalyst 4500E/4500X/6500E

Quad-Sup (SSO)
Supported Platforms Switch Fabric Switching Capacity Catalyst 6500E Sup2T Inter-Chassis(ICA) Active Intra-Chassis (ICS) Ready 4 Tbps

Quad-Sup (RPR-WARM)
Catalyst 6500E Sup720-10GE Inter-Chassis (ICA) Active Intra-Chassis (ICS) Inactive 1.4 Tbps

Dual-Sup
Catalyst 6500E, 4500E and 4500X Inter-Chassis Active 4500E / 4500X 1.6 Tbps 6500E Sup720-10GE 1.4 Tbps 6500E Sup2T 4 Tbps

Policy Feature
BOOT, VLAN Dbase and Startup config Sync Running configuration SSO State Synchronization eFSU Software Upgrade
BRKDCT-2256

Inter-Chassis(ICA) Active Intra-Chassis (ICS) Inactive

Inter-Chassis (ICA) Active Intra-Chassis (ICS) Inactive

Inter-Chassis Active
Inter-Chassis Inter-Chassis Inter-Chassis Inter-Chassis
18

Inter-Chassis (ICS) + Intra-Chassis (ICA) Inter-Chassis (ICA) + Intra-Chassis (ICS) Inter-Chassis (ICA) Inter-Chassis (ICA) Inter-Chassis (ICA) Inter-Chassis (ICA)

Inter-Chassis (ICA) + Intra-Chassis (ICS) Inter-Chassis (ICA) + Intra-Chassis (ICS)


2013 Cisco and/or its affiliates. All rights reserved. Cisco Public

Understanding Virtual Switch Link


Inter-Chassis System Link No network protocol operations Invisible in network topology Transparent to network level troubleshooting VSL Control Link Carries all system internal control traffic Single member-link and dynamic election during bootup Shared interface for network/data traffic < 50 msec switchover to pre-determined VSL path Payload Overhead Every single packet encapsulated with Virtual Switch Header (VSH) Non-bridgeable and Non-routeable. VSL must be directly connected between two virtual switch systems
Control Link Control Link

VSL
VSH L2 L3 Payload CRC

4500E-VSS#show switch virtual link Executing the command on VSS member switch role = VSS Active, id = 1 VSL Status : UP VSL Uptime : 1 day, 1 hour, 16 minutes VSL Control Link : Te1/3/1 Executing the command on VSS member switch role = VSS Standby, id = 2

VSL Status : UP VSL Uptime : 1 day, 1 hour, 17 minutes VSL Control Link : Te2/3/1

19 BRKDCT-2256 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public

Virtual Switching System


VSLP Framework Building Virtual System
Link Management Protocol (LMP)
LMP protocol operates on each VSL member-link for peer-switch detection, link integrity and bi-directionality health check Default hello and dead timers are non-tunable and are optimal for various purpose. LMP hello timers (aka VSLP timers) :
4500-VSS#show vslp lmp timer LMP hello timer Hello Tx (T4) ms Hello Rx (T5*) ms Interface State Cfg Cur Rem Cfg Cur Rem --------------------------------------------------------------------------------------------Te1/3/1 operational 1000 700 30000 29416 Te1/4/1 operational 1000 472 30000 29692 6500-VSS#show vslp lmp timer LMP hello timer Hello Tx (T4) Hello Rx (T5*) ms Interface State Cfg Cur Rem Cfg Cur Rem --------------------------------------------------------------------------------------------Te2/5/4 operational 500 156 60000 59952 Te2/2/8 operational 500 156 60000 59952
6500-VSS#show switch virtual role Switch Switch Status Preempt Priority Role Session ID Number Oper(Conf) Oper(Conf) Local Remote ----------------------------------------------------------------------------------------------------------------------------------LOCAL 1 UP FALSE(N) 110(110) ACTIVE 0 0 REMOTE 2 UP FALSE(N) 100(100) STANDBY 9924 7656
20 BRKDCT-2256 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public

LMP

LMP

RRP

VSL

RRP

Catalyst 6500E LMP Hello / Dead Timer = 0.5 sec / 60 sec


Catalyst 4500E/4500X LMP Hello / Dead Timer = 1 sec / 30 sec For older 6500E VSS deployments, it is strongly recommended not to modify default LMP(VSLP) timer

Role Resolution Protocol (RRP)


RRP runs on control link of the VSL bundle

Determines whether software versions allow a virtual switch to form


Determines which chassis will become Active or Hot Standby from a control plane perspective by checking configuration of switch priority or pre-emption RRP roles are negotiated when either of the switch member initializes or when VSL link is restored

6500E VSS Dual Sup VSL Design


Sup2T and Sup720-10GE Design

Two Cisco recommended designs


Profile 1 VSL on Supervisor (Sup2T/Sup720-10GE) Profile 2 Diversified VSL between Supervisor (Sup2T/Sup720-10GE) and VSL capable Linecard

Sup

Sup

Sup

Sup

VSL VSL

Cost-effective solution to leverage both uplinks. Continue to use non-VSL capable linecard for 10G core connection.

Redundant and diversified fibers between supervisor and next-gen VSL capable linecards. Same design as Profile 1 but increases system reliability as each VSL port are diversified across different fabric/ASICs. Optimal and preset VSL parameters Load-Balancing, QoS, HA, Traffic-engg, Dual-Active etc. Flexible to scale up to 8 x VSL for high-dense system to aggregate uplink, service modules, single-home etc.
Cisco Public 21

Redundant fibers connects thru common fabric and ASICs, this could result vulnerability in system stability.
Optimal and preset VSL parameters Load-Balancing, QoS, HA, Traffic-engg, Dual-Active etc. Restricted to bundle 2 x VSL ports or 20G switching capacity on per virtual-switch node basis.
BRKDCT-2256

2013 Cisco and/or its affiliates. All rights reserved.

6500E VSS VSL Design Quad-Sup (SSO / RPR-WARM)


Sup2T Quad-Sup NSF/SSO VSL Redundancy
Recommended Full-Mesh VSL on Quad-Sup
Sup-1 Sup-2
Sup-4 Sup-4

Sup-1

Sup-2
Sup-4 Sup-4

Sup-3 Sup-3

Sup-3 Sup-3

VSL

VSL

Si
SW1

Si
SW2

Si
SW1

Si
SW2

Same Design Profile 1 Dual Sup Flexible to increase VSL Capacity Continue to leverage existing non-VSL 10G linecard for uplink connection Retains all original VSL benefits Vulnerable design during any supervisor selfrecovery fault incident

Highly Redundant and cost-effective VSL Design. Increases overall VSL Capacity Maintains 20G VSL Capacity during supervisor failure. Increases network reliability by minimizing the dual-active probability

BRKDCT-2256

2013 Cisco and/or its affiliates. All rights reserved.

Cisco Public

22

4500E VSS Dual-Sup VSL Network Design


Sup7E and Sup7-LE Design
Two Cisco recommended designs
Profile 1 VSL on Sup7-E Profile 2 Diversified VSL between Supervisor (Sup7-E/Sup7-LE) and VSL capable Linecard
Sup

Sup

Sup

Sup

VSL

VSL

Cost-effective solution to leverage Quad uplinks for VSL and Core connections For reliable internal connection diversify fibers between Uplink ports groups thru different fabric and ASICs connection Optimal and preset VSL parameters Load-Balancing, QoS, HA, Traffic-engg, Dual-Active etc.

Redundant and diversified fibers between supervisor and VSL capable linecards.
Same design as Profile 1 but increases system reliability as each VSL port are diversified across different ASICs.

Optimal and preset VSL parameters Load-Balancing, QoS, HA, Traffic-engg, Dual-Active etc.
Flexible to scale up to 8 x VSL for high-dense system to aggregate uplink, service modules, single-home etc.

Restricted to bundle 2 x VSL ports or 20G switching capacity on per virtual-switch node basis.

BRKDCT-2256

2013 Cisco and/or its affiliates. All rights reserved.

Cisco Public

23

Catalyst 4500E Sup7LE VSL Uplink Select Best Practices

4500E Sup7LE supervisor module supports following uplink interfaces :


VSL

2 Port 10G Uplink (Default) 4 Port 1G Uplink


SW-1 SW-2

The default 10G uplink ports can be modified to 1G using hw-module uplink select gigabit CLI Prior rebooting the existing VSL port configuration must be manually copied to new ports to successfully make new configuration effective
Step Step-1 Step-2 Step-3 Task Connect cables to new VSL uplink ports Copy all current VSL member-link configuration to new VSL uplink member-links ports Modify uplink port configuration using hw-module uplink select (gigabit | tengig) CLI in global exec mode

VSS switches may enter in dual active and de-stabilize the network if configuration not copied correctly

Step-4

Save configuration and reload both systems using redundancy reload shelf CLI

BRKDCT-2256

2013 Cisco and/or its affiliates. All rights reserved.

Cisco Public

24

4500X VSS VSL Network Design

Fixed switch hardware architecture 24 or 48 10G/1G Front Panel Ports 8 port 1G/10G Pluggable Uplink Module Any ports can be bundled into VSL EtherChannel. Recommended to use front-panel ports to build VSL connections. Minimizes system instability during accidental uplink module OIR/reset Recommended to use odd or even front-panel port numbers. Splits VSL member-link interfaces to different internal ASICs. Consistent software design and VSL function as 4500E

Si

Si

Front / Uplink Ports


Ten1/1/1 Ten2/1/1

4500-X

Ten1/1/5 VSL

Ten2/1/5

4500-X

Front Panel Ports

SW-1

SW-2

BRKDCT-2256

2013 Cisco and/or its affiliates. All rights reserved.

Cisco Public

25

Understanding VSL Forwarding Design


The VSL control and data plane software design is intelligent and optimal
Builds neighbor adjacencies and maintains system virtualization thru remote chassis physical port connection Develops distributed hardware forwarding design and use VSL as last-resort interface
SW-1
VSL

SW-2

6500-vss#show int vsl VSL Port-channel: Po1 Port: Te1/5/4 Port: Te1/5/5 VSL Port-channel: Po2 Port: Te2/5/4 Port: Te2/5/5 6500-vss#show vsl lmp neighbor Instance #1: LMP neighbors Peer Group info: # Groups: 1 (* => Preferred PG) PG # MAC Switch Ctrl Interface Interfaces -------------------------------------------------------------------------------------------*1 001a.30e1.6800 2 Te1/5/4 Te1/5/4, Te1/5/5 6500-vss#remote command switch-id 2 mod 5 show vsl lmp neighbor Instance #2: LMP neighbors Peer Group info: # Groups: 1 (* => Preferred PG) PG # MAC Switch Ctrl Interface Interfaces ------------------------------------------------------------------------------------------*1 001a.30f1.e800 1 Te2/5/4 Te2/5/4, Te2/5/5

VSL carries following traffic categories:


System Control Traffic VSS Control protocols, i.e. LMP, IPC, SCP etc Network Control Traffic Per-Port L2/L3 protocols, i.e. PAgP, CDP, EIGRP/OSPF etc User Data Plane Single Homed Devices traffic Services Traffic Integrated Services Module, SPAN etc

Common EtherChannel load sharing and hash mechanics for control and data traffic

BRKDCT-2256

2013 Cisco and/or its affiliates. All rights reserved.

Cisco Public

26

Virtual Switch Link Capacity Planning


Plan VSL capacity to reduce congestion point, handle failures and specific configurations Supported VSL interfaces types :
Catalyst 6500E : 10G and 40G Catalyst 4500E/4500X : 1G and 10G

Four major factors :


Total Uplink BW Per Chassis. Ability to handle data re-route during uplink failures without network congestion Handling egress data to single-homed devices (Nonrecommended design) Catalyst 6500E services module integration may require centralized forwarding on remote chassis
VSL

Analyzer

Remote network services such as SPAN

Up to 8 member-links supported in VSL EtherChannel. Recommended to implement in power of 2 for optimal forwarding decision

BRKDCT-2256

2013 Cisco and/or its affiliates. All rights reserved.

Cisco Public

27

Advance Virtual Switching System Design


Agenda
Cisco VSS Architecture VSS Architecture Overview Unified System Architecture Designing VSS System Redundancy VSS Dual and Quad-Sup Redundancy Design Virtual Switch Link Design and Best Practices Designing VSS Network Redundancy Multi-Chassis EtherChannel and ECMP Design Load Sharing and Resiliency Designing VSS Enabled Campus Network Access Layer Distribution and Core Layer Design, Best Practices and Failure Analysis VSS Dual Active Detection Understanding Dual Active and Recovery Mechanics Dual Active Best Practices and Failure Analysis

Summary
BRKDCT-2256 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public

28

VSS Single Home Connections

Independent of system modes (VSS or Standalone), single-home connection is non-recommended Cannot leverage any distributed VSS architecture benefits. Non-congruent Layer 2 or Layer 3 network design with Centralized network control-plane processing over VSL
VSL

Si

Si

Asymmetric forwarding plane. Ingress data may traverse over VSL interface and oversubscribe the ports Single-point of failure in various faults Link/SFP/Module failure, SSO switchover, ISSU etc.

SW-1 (ACTIVE)

SW-2 (HOT-STANDBY)

Single Point Of Failure


A1 A2

Cannot be trusted switch for dual active detection purpose

29
BRKDCT-2256 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public

VSS Multi-Home Physical Connections


Si Si

Redundant network paths per system delivers best architectural approach Enables optimal data load sharing and protects network availability during various types of planned/unplanned network outages Parallel Layer 2 paths between bridges builds sub-optimal topology :
Creates STP Loop. Except root port all other ports are in blocking mode Slow network convergence
SW-1 (ACTIVE) VSL SW-2 (HOT-STANDBY)

A1

A2

Parallel Layer 3 doubles control-plane processing load :


ACTIVE switch needs to handle control plane load of local and remotechassis interfaces
Multiple unicast and multicast neighbor adjacencies Redundant routing and forwarding topologies
STP Loop 30
BRKDCT-2256 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public

VSS Multi-Chassis EtherChannel


Multi-Chassis EtherChannel (MEC) in VSS enables distributed link bundling into single logical L2/L3 Interface Combining VSS with MEC builds simplified, scalable and highly resilient campus network MEC is an imperative network design component to enable
Simplified STP loop-free network topology
Consistent L3 control-plane and network design as traditional Standalone mode system Deterministic sub-second network recovery
SW-1 (ACTIVE) VSL

Si

Si

SW-2 (HOT-STANDBY)

MECs can be deployed in two modes


Layer 2 = Supported on 6500E, 4500E and 4500X Layer 3 = Supported on 6500E *
A1 A2

MEC scalability support varies on system basis


Catalyst 6500E supports 512 L2/L3 MEC
Catalyst 4500E and 4500X supports 256 L2 MEC

* L3 MEC is in 4500E/4500X roadmap


BRKDCT-2256 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public

31

Understanding MEC Load Sharing


VSL MEC hash algorithm is computed independently by each virtual-switch toSi perform load share via its Si local physical ports.

8 bits computation on each member link of an MEC is independently done 4 on 8 per virtual-switch 4 8 4 4 8 node basis. Total number of member link bundling in single MEC recommendation remains consistent as described in single chassis Etherchannel section. Recommended to deploy EtherChannel in ratio of n2 that are evenly distributed to each virtualSi switch for best load-sharing result.
Per Switch MEC Flow Distribution Matrix
Member Links 1 2 3 Port1 Bit 8 4 3 Port2 Bit X 4 3 Port3 Bit X X 2 Port4 Bit X X X Port5 Bit X X X Port6 Bit X X X

SW-1

SW-2

Port7 Bit X X X

Port8 Bit X X X

4
5 6 7 8

2
2 2 2 1

2
2 2 1 1

2
2 1 1 1

2
1 1 1 1

X
1 1 1 1

X
X 1 1 1

X
X X 1 1

X
X X X 1

Recommended MEC Bundle link configuration


32
BRKDCT-2256 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public

Optimize EtherChannel Load Balancing

Load share egress data traffic based on input hash Optimal load sharing results with :
Multiple variation of input for hash (L2 to L4)
Default : src-dst-ip vlan
Core

: src-dst-mixed-ip-port Bucket-based load-sharing Bundle member-links in power-of-2 Recommended (2/4/8)

Recommended algorithm * :
Access Src/Dst IP
6500E Dist/Core Src/Dst IP + Src/Dst L4 Ports 4500E / 4500X Dist Src/Dst IP
Default : src-dst-ip vlan Recommended : src-dst-mixed-ip-port vlan
Dist

Default : src-mac Recommended : src-dst-ip


Si

Access

* May vary based on your network traffic pattern


BRKDCT-2256 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public

33

6500E VSS MEC EtherChannel Hash Algorithm


Cat6500 in VSS or in non-VSS configuration mode has common support of EtherChannel Hash algorithms. 6500E EtherChannel Hash result computation mode:
Fixed Recomputes hash results and programs each time when member-link flaps. May impact network convergence time. This is default mode and can be kept default if each virtual-switch node has single physical port bundled in same L2/L3 MEC. Adaptive Pre-computes hash results and programs member-link ports. Do not recompute when member-link flaps and improves network convergence. Best practice to modify to adaptive hash method only if each virtual-switch has >=2 physical port in same L2/L3 MEC.
6500-vss#show etherchannel 10 detail | inc Hash Last applied Hash Distribution Algorithm: Fixed 6500-vss#show interface po10 etherchannel | inc Load|Gi Index Load Port 0 FF Gi1/4/1 2 FF Gi2/4/1 EC state Desirable-Sl Desirable-Sl No of bits 8 8

6500-vss#show etherchannel 10 detail | inc Hash Last applied Hash Distribution Algorithm: Fixed
6500-vss#conf t 6500-vss(config)#port-channel hash-distribution adaptive 6500-vss(config)#do show etherchannel 10 detail | inc Hash Last applied Hash Distribution Algorithm: Fixed 6500-vss(config)#interface port-channel <id> 6500-vss(config-if)#shutdown 6500-vss(config-if)#no shutdown

Unlike EtherChannel load sharing, the EtherChannel Hash can be globally enabled for entire system or it can be on per MEC basis. Modifying EtherChannel Hash algorithm requires manually EtherChannel reset to make effective.

6500-vss#show etherchannel 10 detail | inc Hash Last applied Hash Distribution Algorithm: Adaptive

BRKDCT-2256

2013 Cisco and/or its affiliates. All rights reserved.

Cisco Public

34

Layer 3 Load Balancing Can Be Randomized with a Unique ID Associated with Switch
Universal ID concept (also called Unique ID) is used to prevent CEF polarization
Universal ID generated at bootup (32-bit pseudo-random value seeded by routers base IP address)
Si Si

Universal ID used as input to ECMP hash, introduces variability of hash result at each network layer Universal ID supported on Catalyst 6500 Sup-720-10GE and Sup2T
Si Si

Universal ID supported on Catalyst 4500E Sup7E, Sup7LE and Catalyst 4500X


Si

Hash using Source IP (SIP), Destination IP (DIP) &Universal ID

Catalyst 4500E/4500X Load-Sharing Options


Original Universal* Include Port Src IP + Dst IP Src IP + Dst IP + Unique ID Src IP + Dst IP + (Src or Dst Port) + Unique ID

Catalyst 6500 PFC3** Load-Sharing Options


Default* Full Src IP + Dst IP + Unique ID Src IP + Dst IP + Src Port + Dst Port

Full Exclude Port


Simple

Src IP + Dst IP + (Src or Dst Port)


Src IP + Dst IP Src IP + Dst IP + Src Port + Dst Port
Cisco Public 35

* = default load-sharing mode


BRKDCT-2256

Full Simple

2013 Cisco and/or its affiliates. All rights reserved.

Cisco PAgP and IETF LACP Best Practices


Link bundling protocols builds reliable logical network connections between two systems Cisco PAgP and IETF LACP protocol provides consistent solution
Ensure link aggregation parameters consistency and compatibility between the VSS and neighbor switch. Ensure interface compliance with various aggregation requirements. Dynamically react to runtime changes and failures on local and remote Etherchannel systems
SW1 VSL

Si
LACP Layer 3 Port-Channel

interface TenG1/2/1 , TenG2/2/1 channel-protocol lacp channel-group <id> mode active

Si

Si
SW2

Detect and remove unidirectional links and multidrop connections from the Etherchannel bundle

PAgP Layer 2 Port-Channel

Cisco PAgP MEC can be use for in-direct dual-active detection Recommended to implement in following modes for Layer 2 or Layer 3 EtherChannel :
Cisco PAgP = Desirable / Desirable on both MEC end IETF LACP = Active / Active on both MEC end Keep PAgP and LACP timers to default settings
Si

interface TenG1/1/1 , TenG2/1/1 channel-protocol pagp channel-group <id> mode desirable

Catalyst 2K/3K/4K

4500E-VSS#show pagp neighbor Flags: S - Device is sending Slow hello. C - Device is in Consistent state. A - Device is in Auto mode. P - Device learns on physical port. Channel group 101 neighbors Partner Partner Port Name Device ID Gi1/2/4 M09-3750-3 6073.5c8c.a780 Gi2/2/4 M09-3750-3 6073.5c8c.a780

Implement non-negotiable EtherChannel mode (ON) only when remote device do not support PAgP or LACP protocols, i.e. multi-home PC
BRKDCT-2256

Partner Port Age Gi1/1/1 17s Gi1/1/2 4s

Flags SC SC

PartnerGroup Cap. 10001 10001

2013 Cisco and/or its affiliates. All rights reserved.

Cisco Public

36

LACP Secondary Aggregator Interface


VSL

Active

Standby

SW-1

SW-2
Po20A Po20 Gi2/1 Po20B STP Block port Gi2/2

During EtherChannel bundling process, LACP performs configuration check between physical bundle ports and port-channel and takes 2 following sequential actions :
If configuration check pass, both end system establishes control and forwarding-plane information on user-defined port-channel group and both system function normally. If configuration check fails than it automatically generate an EtherChannel interface with unique alphabetical ID on each end device of an EtherChannel.

MEC config check fail

Switch#show etherchannel 20 summary | inc Gi 20Po20(SU) LACP Gi2/1(P) Gi2/2(P) Switch#show spanning-tree | inc Po20 Po20 Root FWD 3 128.1667 P2p Switch(config)#int gi2/2 Switch(config-if)#switchport nonegotiate Switch(config-if)#shut Switch(config-if)#no shut
%EC-SPSTBY-5-CANNOT_BUNDLE_LACP: Gi2/2 is not compatible with aggregators in channel 20 and cannot attach to them (trunk mode of Gi2/2 is trunk, Gi2/1 is dynamic) %EC-SP-5-BUNDLE: Interface Gi2/2 joined port-channel Po20B

System generated LACP MEC will bundle all the physical ports into an MEC that failed configuration check. All control, forwarding and management-plane will be independently operated over system generated LACP MEC. Such type of EtherChannel configuration mis-match condition will trigger dual individual layer 2 EtherChannel paths between access and virtual-switch nodes. STP topology will consider such network as a loop and block high STP port priority.

Recommendation keep member-link configuration consistent to minimize network impact

Switch#show etherchannel 20 summary | inc Gi 20Po20(SU) LACP Gi2/1(P) 21Po20B(SU) LACP Gi2/2(P) 6500-access#show spanning-tree | inc Po20 Po20 Root FWD 4 128.1667 P2p Po20B Altn BLK 4 128.1668 P2p

BRKDCT-2256

2013 Cisco and/or its affiliates. All rights reserved.

Cisco Public

37

Protocol Comparison PAgP vs LACP


PAgP
Standards Interoperability Cisco Port-Aggregation Protocol PAgP capable Cisco platforms

LACP
IEEE 802.1ad Port-Aggregation Protocol With LACP capable Cisco and third-party vendor device. 8 ports Additional port remains in HOT-STANDBY mode 01-80-c00-00-02 Slow Rate 30 sec / 105 sec Fast Rate 1 sec / 3 sec No Yes No. May create LACP Secondary Aggregator and STP loop with VSS Yes

Max. ports in bundle


Multicast MAC Hello/Hold Timer Dual ACTIVE Detection Capable Per Port operation Local MEC inconsistency check Uni-directional Link Detection Capability Traffic Load-sharing Mechanism Hello Timer Operational

8 ports
01-80-c00-00-00 Slow Rate 30 sec / 105 sec Fast Rate 1 sec / 3 sec Yes Yes Yes Yes

Link-aggregation Protocol independent with up to different 16 permutation traffic load-share across each bundle port in an PAgP or LACP enabled EtherChannel Symmetric Symmetric or Asymmetric

BRKDCT-2256

2013 Cisco and/or its affiliates. All rights reserved.

Cisco Public

38

EtherChannel Link Convergence


Hardware-Based Fault Detection and Recovery
1 2 3 4
Si
VSL

Si
SW2

Link failure detection


SW1

Removal of the Portchannel entry in the software Update of the hardware Portchannel indices Notify the spanning tree and/or routing protocol processes of path cost change

1
Link Failure Detected

Si

Hardware-Based Deterministic Sub-Secondary Recovery


System Independent Catalyst 6500, 4500E, 4500X, 3xxx etc. MEC Type Layer 2 or Layer 3 Protocol Independent STP, EIGRP, OSPF, BGP, PIM, MPLS etc. Protocol Tuning Independent Timer Tunings, Fast Hello, BFD etc. Prefix-Scale Independent MAC or Routes Table Size Fault Independent Link Failure, System Reboot/Failure, ISSU etc.

2
Failed Link Unbundle

Routing Protocol Process

Po1

G1/3/1, G2/3/1, G1/4/1, G2/4/1

4
Update Protocols

Spanning Tree Process

Layer 2 Forwarding Table


VLAN 10 11 MAC AA BB Destination Index Portchannel 1 G5/1

3
Update HW Hash

Destination Port G1/3/1 G2/3/1 G1/4/1 G2/4/1


39

Load-Balancing Hash
BRKDCT-2256 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public

Cisco VSS System Design Summary

Catalyst 6500E
Network Layer Design Network Scale Distribution and Core Large

Catalyst 4500E
Distribution Mid/Small/Collapsed

Catalyst 4500X
Distribution Mid/Small/Collapsed

Sup Redundancy
Network Design Alternatives Inter-Chassis Forwarding Policy Features Design

Dual-Sup (Inter-Chassis) Quad-Sup (NSF/SSO and RPR-WARM)


ECMP and MEC (L2/L3) Distributed Distributed

Dual-Sup (Inter-Chassis)
ECMP and MEC (L2) * Distributed Distributed

Dual-Sup (Inter-Chassis)
ECMP and MEC (L2) * Distributed Distributed

Software Upgrade

eFSU (Dual and Quad-Sup)

ISSU (Dual-Sup)

ISSU (Dual-Sup)

* = Layer 3 MEC is in roadmap


BRKDCT-2256 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 40

Advance Virtual Switching System Design


Agenda
Cisco VSS Architecture VSS Architecture Overview Unified System Architecture Designing VSS System Redundancy

VSS Dual and Quad-Sup Redundancy Design


Virtual Switch Link Design and Best Practices Designing VSS Network Redundancy Multi-Chassis EtherChannel and ECMP Design Load Sharing and Resiliency Designing VSS Enabled Campus Network Access Layer Distribution and Core Layer Design, Best Practices and Failure Analysis VSS Dual Active Detection

Understanding Dual Active and Recovery Mechanics


Dual Active Best Practices and Failure Analysis Summary
41
BRKDCT-2256 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public

VSS in Access Layer Key Benefits

Single Management Plane to manage up to 768 end points and ports with Catalyst 4500E switch Unified Control Plane to two large modular 4500E switches Distributed rich access-layer network technologies:
SW1

4500E

Si

Si
SW2 SW1

Access Layer

Power over Ethernet (PoE) Quality of Service Security ACLs, Identity etc Flexible NetFlow

Scalable Forwarding Architecture to deliver 1.696 Tbps

BRKDCT-2256

2013 Cisco and/or its affiliates. All rights reserved.

Cisco Public

42

VSS in Access Layer Asymmetric Forwarding

Distribution Layer

No protocol or topological difference between Standalone and VSS modes Asymmetric downstream data plane forwarding design. Heavy traffic over VSL as most end points are single-homed connections Depending on distribution layer design the upstream traffic may also traverse over VSL in certain condition Cannot leverage any distributed VSS architecture benefits.
SW-1 (ACTIVE)

Si

Si

VSL

Access Layer

SW-2 (HOT-STANDBY)

BRKDCT-2256

2013 Cisco and/or its affiliates. All rights reserved.

Cisco Public

43

VSS in Access Layer System Redundancy Challenge


Access Layer Standalone Mode Access Layer VSS Mode

System level redundancy in access is base requirement for single-home endpoints Standalone access design delivers non-disruptive network communication with supervisor redundancy

Si

Si

VSL

VSS require Quad-sup NSF/SSO software to deliver equal redundancy. Dual sup VSS design have similar impact as single-sup Standalone access switch
SW1 SW1

Si Si
SW2

Si Si

BRKDCT-2256

2013 Cisco and/or its affiliates. All rights reserved.

Cisco Public

44

Distribution Layer Design Alternatives Standalone vs VSS

Si

Si

Si

Si

Vlan 10

Vlan 20

Vlan 30

Vlan 10

Vlan 20

Vlan 30

Traditional Distribution Block Design


Dual Standalone System Distributed Planes Protocol dependent fault detection and recovery
BRKDCT-2256 2013 Cisco and/or its affiliates. All rights reserved.

Evolution Network Design Single Virtual System Unified Control and Management plane. Distributed Forwarding plane. Deterministic Network Recovery.

Cisco Public

45

Traditional Distribution Design


Redundant design with sub-optimal topology and complex operation. Stabilize network topology with several L2 :
STP Primary and Backup Root Bridge Rootguard Loopguard or Bridge Assurance STP Edge Protection STP Root HSRP Active

Bridge Assurance

Si

Si

Rootguard
Loopguard or Bridge Assurance

Protocol restricted forwarding topology


STP FWD/ALT/BLK Port Single Active FHRP Gateway Asymmetric forwarding Unicast Flood

Protocol dependent driven network recovery


PVST/RPVST+ FHRP Tunings BPDU Guard or PortFast Port Security

BRKDCT-2256

2013 Cisco and/or its affiliates. All rights reserved.

Cisco Public

46

Simplify STP Network Topology with VSS

STP Root

VSS simplifies STP. VSS does not eliminate STP. Never disable STP Multiple parallel Layer 2 network path builds STP loop network
Rootguard

VSS with MEC builds single loop-free network to utilize all available links.
Distributed EtherChannel minimizes STP complexities compared to standalone distribution design STP toolkit should be deployed to safe-guard multilayer network

BPDU Guard or PortFast Port Security


STP BLK Port Loop-free L2 EtherChannel

BRKDCT-2256

2013 Cisco and/or its affiliates. All rights reserved.

Cisco Public

47

Even with Faster Convergence from RPVST+ We Still Have to Wait on FHRP Convergence
VRRP Config
interface Vlan4 ip address 10.120.4.1 255.255.255.0 ip helper-address 10.121.0.5 no ip redirects vrrp 1 description Master VRRP vrrp 1 ip 10.120.4.1 vrrp 1 timers advertise msec 250 vrrp 1 preempt delay minimum 180

FHRP Active

FHRP Standby

Si

Si

HSRP Config
interface Vlan4 ip address 10.120.4.2 255.255.255.0 standby 1 ip 10.120.4.1 standby 1 timers msec 250 msec 750 standby 1 priority 150 standby 1 preempt standby 1 preempt delay minimum 180

GLBP Config
interface Vlan4 ip address 10.120.4.2 255.255.255.0 glbp 1 ip 10.120.4.1 glbp 1 timers msec 250 msec 750 glbp 1 priority 150 glbp 1 preempt glbp 1 preempt delay minimum 180

GLBP offers load balancing within a VLAN For Voice, sub-second Hello timer enables < 1 Sec traffic recovery upstream Sub-Second protocol timers must be avoided on SSO capable network

BRKDCT-2256

2013 Cisco and/or its affiliates. All rights reserved.

Cisco Public

48

PIM Needs Timer Tuning Too

Multicast recovery depends on PIM DR failure detection in Layer 2 network PIM routers exchanges PIM expiration time in query message
Default Query-Interval 30 seconds Expiration Query Interval x 3 DR Failure Detection ~90 seconds
Si Si

PIM DR

Tune PIM query interval to sub-sec as FHRP for faster multicast convergence Sub-second protocol timer must be avoided on SSO capable network

interface Vlan4 ip pim sparse-mode ip pim query-interval 250 msec

BRKDCT-2256

2013 Cisco and/or its affiliates. All rights reserved.

Cisco Public

49

Simplified, Scalable and Reliable L3 Gateway with VSS

Single logical Layer 3 gateway. Eliminates complete need of implementing FHRP protocols.

Removes FHRP dependencies and increases Layer 3 network scalability.


Hardware based rapid fault-detection and network recovery with default protocol timers.
Single IP Gateway

Single PIM Router

Deterministic network sub-second network convergence in multiple fault conditions.


Standalone
interface Vlan4 ip address 10.120.4.2 255.255.255.0 ip pim sparse-mode standby 1 ip 10.120.4.1 standby 1 timers msec 250 msec 750 standby 1 priority 150 standby 1 preempt standby 1 preempt delay minimum 180 ip pim query-interval 250 msec

VSS
interface Vlan4 ip address 10.120.4.2 255.255.255.0 ip pim sparse-mode

BRKDCT-2256

2013 Cisco and/or its affiliates. All rights reserved.

Cisco Public

50

HSRP and VRRP Design Consideration

Asymmetric Routing (Unicast Flooding)


Alternating HSRP Active between distribution switches can be used for upstream load balancing, however downstream traffic hits both distribution block switches ARP (4 hours) and CAM (5 min) table timer mismatch may build inconsistent tables and cause unicast flooding VSS eliminates unicast flooding problem by automatically synchronizing ARP and CAM tables in local and remote switch hardware
SW1: Single Root Bridge and Gateway for VLAN 2 and VLAN 3 SW1: Active HSRP and SW2: Active HSRP and Root Bridge VLAN 3 Root Bridge VLAN 2 CAM Table Empty for VLAN 2 SW1
Single auto synchronized CAM ARP and CAM Table Table Empty for Si

Si

B B B

SW1

SW2

VLAN 3

VLAN 3 VLAN 2 VLAN 3 VLAN 2

VLAN 3 VLAN 3

VLAN 2 VLAN 2

BRKDCT-2256

2013 Cisco and/or its affiliates. All rights reserved.

Cisco Public

51

Multi-Chassis EtherChannel Performs Better In Any Network Design


Network Recovery mechanic varies in different distribution design
Standalone Protocol and Timer dependent VSS Hardware dependent
Convergence (sec)

0.8

VSS logical distribution system


Single P2P STP Topology
Single Layer 3 gateway Single PIM DR system

0.6

0.4

0.2

Distributed and synchronized forwarding table MAC address, ARP cache, IGMP All links are fully utilized based on Ether-channel load balancing

0
L2-FHRP
Upstream Downstream Multicast

L2-MEC

BRKDCT-2256

2013 Cisco and/or its affiliates. All rights reserved.

Cisco Public

52

The Best Deployment for Standalone Is Routed Access


Bridge Assurance OSPF SPF Tuning STP Root timers throttle spf 10 100 5000 timers throttle lsa all 10 Active 100 5000 HSRP timers lsa arrival 80 Rootguard Loopguard or Bridge Assurance BPDU Guard or PortFast Port Security EIGRP/OSPF

Layer 3
Si

Si

Layer 2

Simplified Operation with single control-plane Routing Protocols Improved Network Design No FHRP, STP, Trunk, VTP etc. Optimized Forwarding Topology Layer 3 ECMP Improved convergence with fewer protocols
BRKDCT-2256 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 53

VSS Simplifies Routed Access

Builds single point-to-point routing peer adjacency with MEC EtherChannel delivers deterministic hardware-based network recovery Eliminates adjusting protocol timers and parameters Eliminates additional protocols requirements for rapid fault detection
Single Adjacency EIGRP / OSPF

BRKDCT-2256

2013 Cisco and/or its affiliates. All rights reserved.

Cisco Public

54

Routed Access Optimized Multicast Operation

Layer 2 access has two multicast routers on the access subnet, causing one to have to discard frames Routed Access has a single multicast router which simplifies management of multicast topology

IGMP Querier (Low IP address)

Si

Si

Si

Si

Non-DR has to drop all non-RPF Traffic

Designated Router (High IP Address) Designated Router & IGMP Querier

BRKDCT-2256

2013 Cisco and/or its affiliates. All rights reserved.

Cisco Public

55

VSS Optimizes Multicast Performance with Routed Access


Single logical L3 path to RP from access to join multicast distribution tree Single OIL/IIL PIM interface in Multicast Routing Table Increases multicast bandwidth capacity with all MEC member-links programmed for switching
Single OIL

Single PIM Join Message

Transparent to network faults and provides deterministic sub-second multicast data recovery

6500E-VSS#show ip mroute sparse (*, 239.192.51.8), 3d22h/00:03:20, RP 10.100.100.100, flags: S Incoming interface: Null, RPF nbr 0.0.0.0 Outgoing interface list: Port-channel105, Forward/Sparse, 00:16:54/00:02:54 Port-channel101, Forward/Sparse, 00:16:56/00:03:20 (10.125.31.147, 239.192.51.8), 00:16:54/00:02:35, flags: A Incoming interface: Port-channel105, RPF nbr 10.125.0.21 Outgoing interface list: Port-channel101, Forward/Sparse, 00:16:54/00:03:20

OIL = Outgoing Interface List IIL = Incoming Interface List


BRKDCT-2256 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 56

Routed Access Provides Rapid Convergence with Optimized Traffic Flow and Ease of Mgmt
CEF and protocol based network recovery in Standalone Routed Access Design
EIGRP converges in <200 msec OSPF with sub-second tuning converges in <200 msec Multicast with sub-second tuning convergences in ~600 msec
Convergence (sec)

0.7

0.6

0.5

EtherChannel hash based network recovery in VSS Routed Access Design


Deterministic sub-second unicast & multicast network convergence

0.4

0.3

EtherChannel does not require any further protocol tunings

0.2

0.1

0
EIGRP-ECMP EIGRP-MEC OSPF-ECMP OSPF-MEC

Upstream

Downstream

Multicast

BRKDCT-2256

2013 Cisco and/or its affiliates. All rights reserved.

Cisco Public

57

Diversify Links For Module Redundancy

Distribute multiple connections to single or logical remote system between different linecard module when possible. Recovery mechanic same as link failure. Prevents topology changes or forwarding updates and provides intra-chassis sub-second recovery. Depending network load it minimize the network congestion

Inter-Chassis Recovery Intra-Chassis Recovery

VSL

Si Si

VSL

Si Si

BRKDCT-2256

2013 Cisco and/or its affiliates. All rights reserved.

Cisco Public

58

Best Practice for Module OIR


Module OIR is supported on all modular systems. Network recovery have higher impact with Module OIR due to
OIR detection
2 2.5

Hardware Synchronization
Convergence (sec)

Protocol Dependencies Forwarding Updates

1.5

Minimize network impact with following techniques :


Admin Power Down Admin Reset

0.5

OIR 6500 Standalone


6500E(config)# no power enable module <slot-id>

Power Down Downstream

Soft Reset Multicast

Upstream

6500 VSS
6500-VSS(config)# no power enable switch <1|2> module <slot-id>
BRKDCT-2256 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 59

Summary VSS vs Standalone


STP Loop FHRP FHRP Tunings PIM DR Priority PIM Tunings Protocol Dependent Scale Unicast Flooding Asymmetric Forwarding L2 Hardening Network/System Redundancy Tradeoff Protocol Dependent Recovery CAM/ARP Tunings OSPF LSA/SPF Tuning Control/Mgmt/Forwarding Complexities
BRKDCT-2256 2013 Cisco and/or its affiliates. All rights reserved.

Scale-independent Recovery Network/System Level Redundancy Hardware Driven Recovery


Si Si

Increase Unicast Capacity Increase Multicast Capacity Simplified Network Topologies Control-plane Simplicity Operational Simplicity L2-L4 Load Sharing Flat L2 Network

Cisco Public

60

VSS Enabled Campus Core Design

Extend VSS architectural benefits to campus core layer network VSS enabled core increases capacity, optimizes network topologies and simplifies system operations
Si Si Si Si Si Si

Key VSS enable core best practices :


Protect network availability and capacity with Catalyst 6500E Sup2T Quad-Sup NSF/SSO Simplify network topology and routing database with single MEC Leverage self-engineer VSS and MEC capabilities for deterministic network fault detection and recovery
Si Si Si Si Si

Si

WAN WAN
BRKDCT-2256 2013 Cisco and/or its affiliates. All rights reserved.

Data Center Data Center


Cisco Public

Internet Internet
61

VSS Core Network Design Alternatives


Single Link Network Design Full-Mesh Network Design
VSL VSL

Si
SW1

Si
SW2 SW1

Si

Si
SW2

Physical Design
VSL VSL

Si
SW1

Si
SW2 SW1

Si

Si
SW2

ECMP

MEC

ECMP

Dual MEC

Single MEC

Routing Design

Recommended Design : Full-Mesh Physical Network with Single MEC


BRKDCT-2256 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 62

VSS Core Network Design Analysis


Single Link ECMP Total physical links Total logical links Total layer 3 links ECMP routing path Per switch local forwarding path Routing Peers 2 0 2 2 1 Double ECMP No No No No Variable Variable No Single Link MEC 2 1 1 0 1 Single via VSL Yes No Yes Yes ~600 msec ~600 msec No Full-Mesh ECMP 4 0 4 4 2 Quadrupled ECMP Yes No No No ~200-msec ~200-msec No Full-Mesh Dual-MEC 4 2 2 2 2 Double MEC Yes Yes Yes Yes <=100 msec <=100 msec No Full-Mesh Single MEC 4 1 1 0 2 Single MEC Yes Yes Yes Yes <=100 msec <=100 msec Yes

Single link failure recovery mechanic


NSF/SSO benefits MEC Load-sharing benefits Dual-Active Trust Support Fast-Link Notification capability Single Link Failure Upstream Network Convergence (ave) Single Link Failure Downstream Network Convergence (ave) Recommended Best Practice Core routing Design

BRKDCT-2256

2013 Cisco and/or its affiliates. All rights reserved.

Cisco Public

63

Optimizing Core Performance


HW Driven Forwarding Topology & High Availability

Unicast Forwarding Path

MEC Design ECMP Design

ECMP Design EC Design

Multicast Forwarding Path

VSS-Core
Si
Si Si

Standalone-Core
Si Si Si

VSS-Dist
Si Si Si Si

VSS-Dist

Single MEC between network layer reduces 50% ECMP network design doubles control-plane load control-plane load on VSSon ACTIVE system system and redundant topologies VSS ACTIVE Single L3 unicast/multicast neighbor and best path in Unicast routing protocol installs ECMP best path table between two chassis. Multicast routing installs single Consistent unicast forwarding design. Increase in OIL multicast switching capacity in core Egress data forwarding decision is localized with Increased unicast and multicast load sharing input is 6500E. Catalyst 4500E egress forwarding decision variables across all ECMP links Protocol and scale-independent network recovery scale-dependent network recovery
BRKDCT-2256

Same as VSS enabled system Dual challenges MEC between network layercore maintains original ECMP network load design control-plane control-plane on doubles VSS ACTIVE system load and topologies on VSS ACTIVE system redundant Dual MEC L3 unicast/multicast neighbor and ECMP Unicast routing protocol installs ECMP best path best path in table two unicast chassis. Multicast routing single between Consistent forwarding design. installs Increase in OIL Egress data forwarding decision is localized with 6500E. multicast switching capacity in core 4500E egress decision is across Catalyst Increased unicast andforwarding multicast load sharing input all ECMP links variables Protocol and scale-dependent network recovery Protocol and scale-independent network recovery
Cisco Public 64

2013 Cisco and/or its affiliates. All rights reserved.

Simple Core Network Design Delivers Deterministic Network Recovery


Routing Protocol Independent network convergence in large scale campus core ECMP Prefix-Independent Convergence (PIC) for with 6500 (VSS/Standalone) from 12.2(33)SXI2 Cisco Express Forwarding (CEF) optimization in IOS software. No additional configuration or tunings required Hardware-based fault detection and recovery in MEC/EC designs
3.5 3 2.5 2 1.5 1 0.5 0 500 1000 5000
ECMP (W/o PIC)

Time for ECMP/MEC Unicast Recovery

Convergence (sec)

10000
ECMP (With PIC)

15000
MEC

20000

25000

Number or Unicast Routes Core/Distribution Sup720-10GE


BRKDCT-2256 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 65

VSS Core Simplifies Multicast Operation, Improve Performance and Redundancy

AnyCast - MSDP

Standalone Core needs AnyCast MSDP peering for RP

Redundancy.
VSS based Core simplifies PIM RP Redundancy with

Single Logical PIM RP PIM RP


Si Si

Core Core PIM RP Multiple Multicast Forwarding Paths Single OIL

NSF/SSO/MMLS technologies.
ECMP builds single Multicast forwarding path. MEC increases multicast forwarding capacity by utilizing all

Single Logical PIM Interface PIM Router Single Logical PIM Router
Si
VSL

Single Logical OIL PIM Join PIM Join PIM Router


Si

Dist Dist

member-links.

BRKDCT-2256

2013 Cisco and/or its affiliates. All rights reserved.

Cisco Public

66

Simplified Multicast Network Design Delivers Deterministic Network Recovery


ECMP multicast recovery is mroute scale dependent could range in seconds. MEC/EC multicast recovery is hardware-based and recovery is scale-independent in sub-seconds
Time for ECMP/MEC Multicast Recovery
6 Convergence (sec) 5 4

3
2 1 0 100 500 1000 5000

ECMP MEC/EC

Number or Multicast Routes Core/Distribution Sup720-10GE


BRKDCT-2256 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 67

End-to-End VSS Design


Single System and Network Path Per Campus Layer

Single Unified Core System Single Point-to-Point routing peers between network tiers. Reduced control-plane load and redundant topology database Increased Multicast Switching Capacity and Simplified PIM RP Design Protocol and scale-independent sub-second deterministic network recovery Catalyst 6500E VSS Quad-Sup NSF/SSO protects core network availability and capacity
BRKDCT-2256 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public

Core

Dist

68

Understanding Non Stop Forwarding Design


Neighbor Loss, Graceful Restart
NSF-Aware
Hello
Si Si

Non Stop Forwarding (NSF) functions with Stateful Switch Over (SSO) to protect data connectivity Recovering supervisor and linecard modules uses lastknown forwarding information while gracefully rebuilding L3 protocol state-machines NSF support variation :
NSF Capable An redundant system with dual supervisor or routeprocessor that offers 1+1 redundancy during primary failure, i.e. Catalyst 4500E, 6500E etc. NSF Helper The peer system of NSF-capable system that understands and assist in L3 protocols graceful restart process. NSF-Helper system itself can be redundant or non-redundant, i.e. Catalyst 3560X

NSF Restart RP Restart OSPF First Hello

NSF Capable

BRKDCT-2256

2013 Cisco and/or its affiliates. All rights reserved.

Cisco Public

69

Implementing NSF
VSS software design is built on NSF/SSO architecture. Catalyst 4500E, 4500X and 6500E deployed in VSS mode must enabled NSF. No configuration required on NSF Helper system
EIGRP NSF Configuration
4500E(config)#router eigrp <AS#> 4500E(config-router)#nsf ! 4500E#show ip protocols | inc Routing|EIGRP NSF *** IP Routing is NSF aware *** Routing Protocol is "eigrp 100" EIGRP NSF enabled <snip>

NSF capability must be manually enabled for all Layer 3 routing protocols :
EIGRP, OSPF, ISIS, BGP, MPLS etc.

In VRF environment the NSF must be manually enabled on per-VRF IGP instance Multicast NSF capability is default ON
Inter-Chassis NSF/SSO Recovery Analysis
16 14

OSPF NSF Configuration


6500E(config)#router ospf <PID#> 6500E(config-router)#nsf (cisco | ietf) ! 6500E#show ip ospf | inc Routing|Non-Stop|NSF Routing Process "ospf 100" with ID 10.125.100.1 Non-Stop Forwarding enabled IETF NSF helper support enabled Cisco NSF helper support enabled

Convergence (sec)

12 10 8 6 4 2 0 Without NSF With NSF 2013 Cisco and/or its affiliates. All rights reserved.

Multicast Redundancy Configuration


4500E#show ip multicast redundancy state Multicast IPv4 Redundancy Mode: SSO <snip>

BRKDCT-2256

Cisco Public

70

Sub-second Protocol Timers and NSF/SSO


NSF is intended to provide availability through route convergence avoidance Fast IGP timers are intended to provide availability through fast route convergence In an NSF environment dead timer must be greater than:
SSO recovery + Routing Protocol restart + time to send first hello
Si

Core

interface Port-Channel 10 ip ospf dead-interval minimal multiplier 4

OSPF dead timer expired

Recommendation
Do not configure aggressive timer Layer 2 protocols, i.e. Fast UDLD Do not configure aggressive timer Layer 3 protocols, i.e. OSPF Fast Hello, BFD etc. Keep all protocol timers at default settings
0.25
Link and Switch Failure Analysis Default OSPF Timer

VSL

Dist

SW1 ACTIVE

SW2 HOT-STANDBY SW2 ACTIVE

0.25

Link Failure Analysis Aggressive OSPF Timer

0.2

0.2

UDLD dead timer expired


Si

Access

0.15

0.15

0.1

0.1

Catalyst 2K/3K/4K

0.05

0.05

0
Upstream Downstream

Upstream

Downstream

BRKDCT-2256

2013 Cisco and/or its affiliates. All rights reserved.

Cisco Public

71

Advance Virtual Switching System Design


Agenda
Cisco VSS Architecture VSS Architecture Overview Unified System Architecture Designing VSS System Redundancy

VSS Dual and Quad-Sup Redundancy Design


Virtual Switch Link Design and Best Practices Designing VSS Network Redundancy Multi-Chassis EtherChannel and ECMP Design Load Sharing and Resiliency Designing VSS Enabled Campus Network Access Layer Distribution and Core Layer Design, Best Practices and Failure Analysis VSS Dual Active Detection

Understanding Dual Active and Recovery Mechanics


Dual Active Best Practices and Failure Analysis Summary

BRKDCT-2256

2013 Cisco and/or its affiliates. All rights reserved.

Cisco Public

72

Understanding VSS Dual Active Condition


VSL links between VSS switches carries in-band control plane to maintain various types of virtual-chassis statemachines
Duplicate Interface IP Duplicate IGP/BGP RID Duplicate Control-Plane (ARP, ICMP) Core
Si

Failure of all VSL link breaks system virtualization and leads HOT-STANDBY switch to transition in ACTIVE role while original ACTIVE switch is still operational. This system state is known as Dual-Active Dual-Active condition confuses neighbor devices and destabilizes L2 and L3 network with duplicate system information
Unstable L2 and L3 network topologies directly impacts forwarding-plane causing network outage

Control Link

Control Link

VSL

Dist

SW1 ACTIVE

SW2 HOT-STANDBY SW2 ACTIVE

Duplicate PAGP/LACP System ID STP BPDU Duplicate L2 Control-Plane (CDP, UDLD)


Si

Access

BRKDCT-2256

2013 Cisco and/or its affiliates. All rights reserved.

Cisco Public

73

VSS Dual-Active Detection Redundancy


Dual-Sup or Quad-Sup VSL Redundancy
Two Detection and Recovery Mechanic : In-Direct Detection = Enhanced PAgP (ePAgP) Direct Detection = Dual-Active Fast Hello Recommended to use ePAgP and Fast-Hello mechanic for redundancy on Catalyst 6500E VSS
Core

Si

ePAgP Trusted L3 Port-Channel

Recommended to use multiple trusted ePAgP MECs for redundancy on Catalyst 4500E / 4500X VSS
6500E VSS BFD detection mechanic is deprecated starting 15.0(SY1)
Enhanced PAgP Dual Active Fast Hello BFD
SW1 ACTIVE

VSL

Dist

Fast Hello
SW2 HOT-STANDBY

Platform Catalyst 6500E Catalyst 4500E Catalyst 4500X

ePAgP Trusted L2 Port-Channel


Access
Si

* *

(Deprecated)


Cisco Public

Catalyst 2K/3K/4K

* Dual Active Fast-Hello is in Catalyst 4500E/4500X roadmap


BRKDCT-2256 2013 Cisco and/or its affiliates. All rights reserved. 74

Cisco PAgP Dual Active Detection and Recovery


Trusted ePAgP EtherChannel includes single ACTIVE switch ID and unique backplane MAC address information. Neighbor switch caches advertised information In dual active condition both switches advertises ePAgP messages to neighbor with common VSS domain, different Switch ID and different backplane MAC address
SW1 : ACTIVE SW1 : MAC=A.B.C
Si

SW2 : ACTIVE SW2 : MAC=X.Y.Z

Core

ePAgP Trusted L3 Port-Channel

Neighbor switch proxies ePAgP message to old ACTIVE switch.


Old ACTIVE enters in Recovery mode upon receiving ePAgP message with different switch ID and backplane MAC address Trusted ePAgP EtherChannels can be L2 or L3
SW1 ACTIVE RECOVERY

VSL

Dist

Multiple ePAgP EtherChannels can be trusted. Recommended minimum 2 trusted EtherChannel for redundancy
Configuring dual active ePAgP trust EtherChannel requires admin down. Plan and implement during migration or downtime
Catalyst 4500E/4500X/6500E ePAgP Configuration
!Enable Enhanced PAgP on trusted L2/L3 Port-Channel interface 4500-VSS(config-vs-domain)#dual-active detection pagp trust channel-group 101 !
BRKDCT-2256 2013 Cisco and/or its affiliates. All rights reserved.

SW2 ACTIVE SW2 HOT-STANDBY

ePAgP Trusted L2 Port-Channel


SW1 : ACTIVE SW1 : MAC=A.B.C

Si

SW2 : ACTIVE SW2 : MAC=X.Y.Z

Access

Catalyst 2K/3K/4K

Cisco Public

75

Implementing and Monitoring Dual Active ePAgP

Si

ePAgP Trusted L3 Port-Channel

Po101

Catalyst 4500E/4500X/6500E VSS ePAgP Configuration


!Enable Enhanced PAgP on trusted L2/L3 Port-Channel interface 6500E-VSS(config-vs-domain)#dual-active detection pagp trust channel-group 101 6500E-VSS(config-vs-domain)#dual-active detection pagp trust channel-group 102 !
SW1 ACTIVE

VSL

SW2 HOT-STANDBY

ePAgP Client Catalyst Systems Catalyst 2960 * Catalyst 3560X Catalyst 3750X * Catalyst 3850 ** Catalyst 4500E Catalyst 4500X Catalyst 6500E

ePAgP Client Verification ePAgP Trusted L2 Port-Channel Po102


4500E-Access#show pagp dual-active PAgP dual-active detection enabled: Yes PAgP dual-active version: 1.1 Channel group 4 Dual-Active Port Detect Capable Te1/1 Yes Te2/1 Yes

Si

Catalyst 2K/3K/4K

Partner Name cr2-6500-VSS cr2-6500-VSS

Partner Port Te2/2/6 Te1/2/6

Partner Version 1.1 1.1

* Cisco Catalyst 2960 FlexStack and 3750X StackWise-Plus cross-stack do not support ePAgP ** Cisco Catalyst 3850 StackWise-480 cross-stack supports ePAgP
BRKDCT-2256 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 76

Dual Active Fast Hello Detection and Recovery


Direct dual active detection technique over dedicated fiber/copper 10/100/1000 connection In single active state fast hello messages are bi-directionally processed at every 2 second interval. Accelerates at 200 msec rate upon loosing all VSL interface Dual active is detected if all VSL connections are lost and fast hello message from peer switch is detected. Old ACTIVE switch enters in recovery mode
SW1

Core
Si

VSL

Dist
SW2

Fast Hello interfaces operates on restricted configuration mode and remains transparent network topologies

Fast Hello
SW1 SW1 ACTIVE RECOVERY

ACTIVE SW2 SW2 HOT-STANDBY

Up to four Fast Hello interfaces can be configured. Cannot be in EtherChannel mode


Supported on Catalyst 6500E*
Catalyst 6500E Dual Active Fast Hello Configuration
6500-VSS(config#interface range Gi1/5/1 , Gi2/5/1 6500-VSS(config-if)#dual-active fast-hello
6500-vss#show switch virtual dual-active fast-hello Fast-hello dual-active detection enabled: Yes

Access
Si

Fast-hello dual-active interfaces: Port Local State Peer Port Remote State --------------------------------------------------Gi1/5/1 Link up Gi2/5/1 Link up 6500-vss#remote command standby-rp show switch virtual dual-active fast-hello Fast-hello dual-active detection enabled: Yes Fast-hello dual-active interfaces: Port Local State Peer Port Remote State --------------------------------------------------Gi2/5/1 Link up Gi1/5/1 Link up

Catalyst 2K/3K/4K

* Dual Active Fast-Hello is in Catalyst 4500E/4500X roadmap


BRKDCT-2256 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 77

6500E Dual-Active Recovery Analysis


Dual-Sup or Quad-Sup VSL Redundancy
6500E VSS Dual-Active Recovery Analysis ePAgP
35 30 Convergence (sec)

Dual-Active Network Recovery depends on


Uplink Network Design ECMP vs MEC Routing Protocols EIGRP vs OSPF Detection Mechanic Fast-Hello vs ePAgP

25 20 15 10 5 0 EIGRP - ECMP EIGRP - MEC Upstream OSPF - ECMP Downstream OSPF - MEC

OSPF ECMP faster in failure detection then ePAgP. Slow network convergence
Starting 12.2(33)SXI3 Dual-Active Fast-Hello performs rapid failure detection and delivers deterministic recovery independent of network design and protocol
0.5

6500E VSS Dual-Active Recovery Analysis Fast-Hello

0.4 Convergence (sec)

0.3

0.2

0.1

0 EIGRP - ECMP EIGRP - MEC Upstream OSPF - ECMP Downstream OSPF - MEC

BRKDCT-2256

2013 Cisco and/or its affiliates. All rights reserved.

Cisco Public

78

Dirty Configuration during dual ACTIVE


During the dual ACTIVE restoration if configuration on old ACTIVE chassis is unchanged, old ACTIVE will reboot itself after at least one VSL member link is restored
*Apr 6 17:36:33.809: %VSLP-SW1_SP-5-VSL_UP: Ready for Role Resolution with Switch=2, MAC=001a.30e1.6800 over Te1/5/5 *Apr 6 17:36:36.109: %dual ACTIVE-1-VSL_RECOVERED: VSL has recovered during dual ACTIVE situation: Reloading switch 1 snip

*Apr 6 17:36:36.145: %VSLP-SW1_SP-5-RRP_MSG: Role change from ACTIVE to HOT_STANDBY and hence need to reload *Apr 6 17:36:36.145: %VSLP-SW1_SP-5-RRP_MSG: Reloading the system... *Apr 6 17:36:37.981: %SYS-SW1_SP-5-RELOAD: Reload requested Reload Reason: VSLP HA role change from ACTIVE to HOT_STANDBY.

When VSL recovers, a switch in recovery mode will reload and come up as HOT_STANDBY. However, if the configuration is changed (marked dirty by RF config_sync process), the switch will not reload automatically. Manual reload must be issues on old ACTIVE after configuration has been corrected and saved. Even just entering in configuration mode and exiting will mark the configuration dirty and will force manual intervention
*Aug 13 04:24:34.716: %dual ACTIVE-1-VSL_RECOVERED: VSL has recovered during dual ACTIVE situation: Reloading switch 2 *Aug 13 04:24:34.716: %VS_GENERIC-5-VS_CONFIG_DIRTY: Configuration has changed. Ignored reload request until configuration is saved

The configuration change on VSL link will parsed during the initialization . The configuration check helps ensure that the VSL-related configurations on the two switches are compatible. If it fails, then the standby chassis comes up in route-processor redundancy (RPR), mode where all modules are powered down. VSL related configuration changes can be viewed via show switch virtual redundancy config-mismatch The best practice recommendation is to NOT to enter into configuration mode while in dual active however one can not avoid the accidental shut down of VSL link and thus required configuration changes to have proper VSL restoration

BRKDCT-2256

2013 Cisco and/or its affiliates. All rights reserved.

Cisco Public

79

VSS Best Practices Summary

Design each VSS domain with unique ID Configure mac-address use-virtual under virtual switch configuration mode Select appropriate VSS capable system that fits in network and solution requirements Deploy 6500E Quad-sup NSF/SSO for mission-critical networks to protect network availability and capacity Do not compromise network foundation baselines. Deploy full-mesh physical connections for redundancy and load sharing across the network MEC enables network benefits with VSS. Bundle all physical connections into single logical connection for simplified and resilient network topologies Always use link bundling protocols Cisco PAgP or IETF LACP Plan and design VSL with appropriate capacity, diversification and redundancy
80
BRKDCT-2256 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public

VSS Best Practices Summary

Configure nsf under L3 routing protocols

Keep Layer 2 and Layer 3 protocol timers at factory default. Do not enable protocols with aggressive timers
Configure redundant dual active trusted ePAgP neighbors (L2/L3)

Configure redundant dual active mechanics ePAgP and Fast Hello


Exclude dual active management interface for connectivity and troubleshooting

81
BRKDCT-2256 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public

Summary

Simplify and Optimize your campus network design with system and network consolidation to maintain application performance even during common network faults Leverage hardware-based fault detection for scale-independent and deterministic network recovery Build non-stop communication network with system-level redundancy in all campus layer Access / Distribution / Core Design mission-critical campus backbone that offers scale flexibility, key foundational services and uncompromised high-availability. Reduce maintenance window and upgrade system while maintaining network availability
82
BRKDCT-2256 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public

Recommended Reading

Continue your Cisco Live learning experience with further reading from Cisco Press Check the Recommended Reading flyer for suggested books
End-to-End QoS Network Design: Quality of Service in LANs, WANs and VPNs ISBN: 1-58705-176-1 Building Resilient IP Networks ISBN: 1-58705-215-6 Top-Down Network Design, Second Ed. ISBN: 1-58705-152-4

Available Onsite at the Cisco Company Store


83
BRKDCT-2256 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public

Call to Action

Visit the Cisco Campus at the World of Solutions


to experience Cisco innovations in action

Get hands-on experience attending one of the Walk-in Labs


Schedule face to face meeting with one of Ciscos engineers at the Meet the Engineer center Discuss your projects challenges at the Technical Solutions Clinics

BRKDCT-2256

2013 Cisco and/or its affiliates. All rights reserved.

Cisco Public

84

BRKDCT-2256

2013 Cisco and/or its affiliates. All rights reserved.

Cisco Public

85