(kchew@cisco.com)
Presentation_ID
Cisco Confidential
Agenda
Introduction Embedded Event Monitoring (EEM) Generic Online Diagnostics (GOLD) Smart Call Home (SCH) Global Balancing Protocol (GLBP)
Presentation_ID 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential
Wiring Closet
Backbone
Data Center
EWAN
Metro
NEW
SP Network
VS-S720-10G
Enhanced Object 16-way IOS Software 12.2(33)SXH Software SHIPPING! Tracking Loadbalancing Modularity HSRP and GLBP SSO Multiple SPAN Enhancements Fast Fabric Switchover EEM BFD with BGP IP SLA
MPLS HA MPLS FRR link and Node protection Multiplexed UNI E-OAM (802.1ag and 802.3ah) MPLS MIBs Private Hosts
Major Virtualization
Application Intelligence Integrated Security
Presentation_ID
Per interface NDE Virtual Switching & L2 Scalability Innovations NBAR on PISA NetFlow Top Sophisticated Sophisticated
Cisco Confidential
Presentation_ID
Cisco Confidential
Presentation_ID
Simplified Operation - Embedded Event Manager provides a means to automate the operational management in real time - EEM monitors for specific events on the switch and can invoke pre defined actions to correct, take remedial action and report the event to network operations 5
2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential
Presentation_ID
Cisco Confidential
EEM
Basic Architecture
Presentation_ID
Cisco Confidential
Presentation_ID
Cisco Confidential
Presentation_ID
Cisco Confidential
10
Presentation_ID
Cisco Confidential
11
Presentation_ID
Cisco Confidential
12
Health monitoring tests (background non-disruptive) On-Demand tests (disruptive and non-disruptive) User scheduled tests (disruptive and non-disruptive) CLI access to data via management interface
Cisco Confidential
13
Forwarding Engine
Active Supervisor
Standby Supervisor
Other types of diagnostics tests including memory and error correlation tests are also available
Presentation_ID 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential
Line card
16
Presentation_ID
Cisco Confidential
17
Action
Si
Default corrective action Supervisor reset Supervisor switch-over Fabric switch-over Port shut down Line card reset Line card power down Generate a call-home message Trigger Syslog Trigger EEM policies Generate SNMP Trap
18
Run During System Bootup, Line Card OIR or Supervisor Switchover Makes Sure Faulty Hardware Is Taken out of Service
Runtime Diagnostics
Health-Monitoring
Switch(config)#diagnostic monitor module 5 test 2 Switch(config)#diagnostic monitor interval module 5 test 2 00:00:15
On-Demand
Switch#diagnostic start module 4 test 8 Module 4: Running test(s) 8 may disrupt normal system operation Do you want to continue? [no]: y Switch#diagnostic stop module 4
Scheduled
Switch(config)#diagnostic schedule module 4 test 1 port 3 on Jan 3 2005 23:32 Switch(config)#diagnostic schedule module 4 test 2 daily 14:45
All Diagnostics Tests Can Be Run on Demand, for Troubleshooting Purposes. It Can Also Be Used As A Pre-deployment Tool Schedule Diagnostics Tests, for Verification and Troubleshooting Purposes
19
Presentation_ID
Cisco Confidential
000 00:00:30.00
000 00:00:15.00 not configured not configured not configured not configured
7) TestDontConditionalLearn --------> M**N****I*** not configured Diagnostics test suite attributes: 8) TestBadBpduTrap -----------------> M**D****I*** not configured M/C/* - Minimal bootup level test / Complete bootup level 9) TestMatchCapture ----------------> M**D****I*** test /not configured NA 10) TestProtocolMatchChannel --------> M**D****I*** - not configured test / NA B/* Basic ondemand 11) TestFibDevices ------------------> M**N****I*** P/V/* -not configured / Per device test / NA Per port test 12) TestIPv4FibShortcut -------------> M**N****I*** D/N/* -not configured Disruptive test / Non-disruptive test / NA 13) TestL3Capture2 ------------------> M**N****I*** - not configured to standby unit / NA S/* Only applicable 14) TestIPv6FibShortcut -------------> M**N****I*** - not configured X/* Not a health monitoring test / NA 15) TestMPLSFibShortcut -------------> M**N****I*** - not configured F/* Fixed monitoring interval test / NA 16) TestNATFibShortcut --------------> M**N****I*** - not configured monitoring test / NA E/* Always enabled 17) TestAclPermit -------------------> M**N****I*** - not configuredactive / Monitoring is inactive A/I Monitoring is 18) TestAclDeny ---------------------> M**N****A*** - 000 00:00:05.00 cards and need reset supervisor / NA R/* Power-down line 19) TestQoSTcam ---------------------> M**D****I*** - not configured K/* Require resetting the line card after the test has completed / NA <snip> T/* - Shut down all ports and need reset supervisor / NA 20
Presentation_ID
Cisco Confidential
32) TestAsicSync --------------------> ***N****A*** 000 00:00:15.00 10 Diagnostics test suite attributes:
Pay Extra Attention to Memory Tests: Memory Tests Can Take Hours to Complete and a Reset Is Required After Running These Tests
S/* - Only applicable to standby unit / NA X/* - Not a health monitoring test / NA F/* - Fixed monitoring interval test / NA
Presentation_ID
Cisco Confidential
21
Port ASIC
RP CPU SP CPU
L2 Engine
Switch Fabric
16 Gbps Bus
Switch(config)#diagnostic monitor module 5 test 2 Switch(config)#diagnostic monitor interval module 5 test 2 00:00:15
Presentation_ID 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential
22
----------------------------------------------------------------------------
2) TestLoopback: Port 1 . 2 . 3 . 4 . 5 . 6 . 7 . 8 . 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 . . . . F . . . . . . . . . . .
----------------------------------------------------------------------------
Presentation_ID
Cisco Confidential
23
GOLD
Operation Example
GOLD generic Syslog messages start with the string DIAG; CONST_DIAG messages platform specific
Bootup Test Failure: %CONST_DIAG-SP-3-BOOTUP_TEST_FAIL: Module 2: TestL3VlanMet failed Health Monitoring Test Failure: %CONST_DIAG-SP-3-HM_TEST_FAIL: Module 5 TestSPRPInbandPing consecutive failure count:10 %CONST_DIAG-SP-6-HM_TEST_INFO: CPU util(5sec): SP=3% RP=12% Traffic=0% %CONST_DIAG-SP-4-HM_TEST_WARNING: Sup switchover will occur after 10 consecutive failures On Demand Diagnostics Test Failure: %DIAG-SP-3-TEST_FAIL: Module 5: TestTrafficStress{ID=24} has failed. Error code = 0x1 Scheduled Diagnostics Test Failure: %DIAG-SP-3-TEST_FAIL: Module 3: TestLoopback{ID=1} has failed. Error code = 0x1 Generic Minor and Major Failure: %DIAG-SP-3-MINOR: Module 3: Online Diagnostics detected a Minor Error. Please use 'show diagnostic result <target>' to see test results. %DIAG-SP-3-MAJOR: Module 6: Online Diagnostics detected a Major Error. Please use 'show diagnostic Module 6' to see test results.
Presentation_ID
Cisco Confidential
24
Reducing Downtime Thru Automation GOLD Integration With EEM and Call Home
Automates problem diagnosis and information gathering
EEM applets and scripts can initiate GOLD tests
Configure User Policies
Presentation_ID
Cisco Confidential
25
Presentation_ID
Cisco Confidential
26
Presentation_ID
Cisco Confidential
27
On demand diagnostics:
Use as a pre-deployment tool: run complete diagnostics before putting hardware into production environment
Si
Scheduled diagnostics:
Schedule key diagnostics tests periodically
Health-monitoring diagnostics:
Key tests running by default Enable additional non-disruptive tests for specific functionalities enabled in your network: IPv6, MPLS, NAT
Presentation_ID
Cisco Confidential
28
Presentation_ID
Cisco Confidential
29
Presentation_ID
Cisco Confidential
30
Sends message to Cisco TAC with precise information and diagnostics Detects GOLD events and sends to Call Home GOLD runs diags, isolates fault and precise location
Presentation_ID 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential
31
TAC
3
Secure Transport* Messages Received:
Diagnostics Environmental Syslog Inventory and Configuration
2
Call Home DB *Ensures data protection HTTPS Encryption Certificate-based authentication
32
Call Home
IOS 12.2(33)SXH
Presentation_ID
Cisco Confidential
Minor hardware failureundetected Customers Ops team discovers IP multicast configuration problem
Look into various known issues and bugs on WSX6548-GE-TX. Find nothing. Request logs from customer
Logs received and analyzed Identify online diagnostics failure for test TestL3VlanMet RMA created
After
12 min 42 min 1.2 hrs 5.5 hrs
P3 SR opened due to GOLD failure. Diag. info attached Cisco LAN SW team takes ownership
Presentation_ID
Cisco Confidential
35
Devices continually monitored with secure, connected service Real-time alerts for early detection of potential network problems Automatic, accurate fault diagnosis
Fast, web-based access to information Call Home messages, diagnostics and recommendations Inventory and configuration for all Call Home devices Security alerts, Field and End-of-life Notices
Presentation_ID
Cisco Confidential
37
Presentation_ID
Cisco Confidential
38
Presentation_ID
Cisco Confidential
39
$1295/Mb
Active
Presentation_ID
Cisco Confidential
42
With GLBP
Load is shared
Packet rate
Buffer Load balancing improves threshold throughput & reduces potential of packet loss
43
AVG
R1
R2
R3
Gateway Routers
ARP
ARP Reply
ARP
ARP
ARP Reply
Clients
IP: MAC: GW: ARP:
Presentation_ID
CL1 ARP Reply 10.0.0.1 aaaa.aaaa.aa01 10.0.0.10 0007.B400.0101 IP: MAC: GW: ARP:
CL2
CL3
Cisco Confidential
AVG
R1
R2
R3
Gateway Routers
Clients
IP: MAC: GW: ARP:
Presentation_ID
CL1
CL2
CL3
Cisco Confidential
where yy.yyyy equals the lower 24 bits; these bits consist of 6 zero bits, 10 bits that correspond to the GLBP group number, and 8 bits that correspond to the virtual forwarder number
0007.b400.0102 : last 24 bits = 0000 0000 0000 0001 0000 0010 = GLBP group 1, forwarder 2
46
Round-robin
Each virtual forwarder MAC takes turns Weighted Directed load determined by advertised weighting factor
Host-dependent
Ensures that each host is always given the same vMAC If no load balance algorithm is specified, default is round-robin MD5 authentication security (Releases 12.3(2)T and 12.2(18)S))
Presentation_ID 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential
47
interface FastEthernet2/0
ip address 10.88.49.1 255.255.255.0 duplex full glbp 1 ip 10.88.49.10 glbp 1 priority 105 glbp 1 authentication text magicword glbp 1 weighting 100 lower 95
Presentation_ID
Cisco Confidential
48
Cisco Catalyst 6500 Series and Cisco 7600 Series GLBP Specifics
GLBP reserves 4 MAC filter entries
The number of forwarders in the group is limited to 4* Active Virtual Gateway will allocate these to GLBP group members (Virtual Forwarders)
There is a restriction on GLBP group number for the MSFC2/PFC2 Only a single group may be defined
The single group may be reused on all VLAN Sup720 supports both plain text & MD5 auth; Sup2 plain text only
1024 / 4 1/4
* Note: 1024 group limit is an arbitrary cap, the protocol design actually allows for 4096; as is the forwarder limit of 4 the design could allow for up to 16. Customers haveInc. All rights reserved. Cisco Confidential additional capacity. Presentation_ID 2006 Cisco Systems, not requested the
50
Presentation_ID
Cisco Confidential
51