FSP 150CC-GE20x Product Training Course 10 - Troubleshooting

FSP 150CC-GE206 R4.4.x FSP 150CC-GE201 R4.3.x October 2010 V1.3

Module Contents
Alarms and Conditions Last Reset Cause Recovery Command DB Maintenance Mode

2010 ADVA Optical Networking. All rights reserved. Confidential.

Alarms and Conditions

Transient Condition:
Represents a single event that has a notification generated off the device No state information is maintained, thus it does not show up in show alarms or show conditions Example is a PM threshold crossing alert (TCA)

Alarmable Condition:
Can be configured with one of the following notification codes:
CR/MJ/MN: an alarm that is visible via show conditions and show alarms has separate raise and clear events NR: a standing condition visible only via show conditions but no notifications sent off the device. NA: treated as a transient condition. Event generated only when the condition occurs. Nothing generated on the clear. Not retrievable via the show conditions command. A change to the alarm attributes definition (CR/MJ/MN/NA/NR) effect alarms that are currently active.

Example is a link down raised against an Ethernet interface.

3
2010 ADVA Optical Networking. All rights reserved. Confidential.

Show Alarms

ADVA--> show alarms |Entity Id |ConditionType |Svc|Loc |Dir |NC |TimeStamp |---------------------------|------------------------------|---|-----|-----|---|--------------------| |network-1-1-1-2 |sfp-rmvd |nsa|near |rcv |mn |2009-11-12,17:12:08 | |psu-1-1-2 |eqpt-rmvd |nsa|near |rcv |mn |2009-11-11,18:58:25 | |access-1-1-1-1 |farend-duplex-unknown |sa |near |rcv |mj |2009-11-12,18:45:17 |

2010 ADVA Optical Networking. All rights reserved. Confidential.

Show Conditions

ADVA--> show conditions |Entity Id |ConditionType |Svc|Loc |Dir |NC |TimeStamp |----------------------------|------------------------------|---|-----|-----|---|--------------------| |sync-1-1-1-1 |frngsync |nsa|near |na |nr |2009-11-11,18:58:30 | |network-1-1-1-2 |sfp-rmvd |nsa|near |rcv |mn |2009-11-12,17:12:08 | |psu-1-1-2 |eqpt-rmvd |nsa|near |rcv |mn |2009-11-11,18:58:25 | |access-1-1-1-1 |farend-duplex-unknown |sa |near |rcv |mj |2009-11-12,18:45:17 |

2010 ADVA Optical Networking. All rights reserved. Confidential.

Alarms and Conditions

System
Condition
Primary NTP Server Failed Secondary NTP Server Failed SNMP Dying Gasp Host Unresolved

Description
The Primary NTP server is not reachable The Secondary NTP server is not reachable

The unit is unable to ping the resolved SNMP Dying Gasp Trap host.

2010 ADVA Optical Networking. All rights reserved. Confidential.

Alarms and Conditions

Power Supply Units
Condition Equipment Fault Input Power Fault Equipment Provisioned But Removed Description This can be a result of a defective PSU or a supply voltage that is out of spec. The voltage supplied by this PSU to the chassis is outside the permissible range. The PSU has been physically removed from the system.

2010 ADVA Optical Networking. All rights reserved. Confidential.

Alarms and Conditions

Ethernet Interfaces
Media

Type

Condition
SFP Removed

Description
The SFP is not populated or not properly seated.

All

Fiber Copper or Fiber

8

All
All All All All

SFP TX Fail
SFP Mismatch Link Down RX Jabber RFI

The SFP transmitter has failed.

The SFP inventory cant be read or it does not support the provisioned rate. The link is down. Cable unplugged, cut, etc. One or more jabbers received per second for 3 consecutive seconds. Remote Failure Indication has been detected on a GbE interface

All
All All

Auto Negotiation Duplex Unknown

Dying Gasp LCP Failure

The interface is provisioned for auto negotiation and has not been able to negotiate the duplicity.
The device at the far end of the link has suffered a catastrophic failure and generated an EFM Dying Gasp message. The Link Control Protocol running as part of the PPP Mgmt tunnel has detected a problem.

All
All All All WAN LAN

Link Deactivated
EFM Discovery Fail Remote Critical Event Remote Link Status Indication Remote Link Down MAC Table Full

The link has been deactivated as a result of the Fault Propagation feature.
EFM-OAM discovery protocol has failed. The far end has detected a critical failure and has sent an EFM-OAM remote critical event indication. The far end has detected a link error and has sent an EFM-OAM remote link status indication. An EFM AOM Remote Link Down indication is being received. The MAC lookup table used on connectionless mode is full.
2010 ADVA Optical Networking. All rights reserved. Confidential.

Alarms and Conditions

Maintenance End Points (MEPs)

Condition
Crossconnect CCM

Description
One or more CCMs received where: The MAID does not match the MAID of the receiving MEP The MD is less than the level of the receiving MEPs level One or more CCMs received where: MEP ID is not configured in the MA The MEP ID is the same as the receiving MEP The CCM interval does not match the receiving MEPs MA value No CCM received for one or more MEPs defined in the MA for 3 consecutive intervals One or more CCMs received that contain: An interface status TLV (Type/Length/Value tuple) A port status TLV These TLVs are populated interface or port associated with the sending MEP are not up.

Erroneous CCM

Some Remote MEP CCM Some MAC Status

Some RDI

One or more CCMs received that have the RDI bit set. The Remote Defect Indicator bit is set by the sending MEP when it has detected a problem.

2010 ADVA Optical Networking. All rights reserved. Confidential.

Last Reset Cause (GE201)

System provides a last reset cause such as warm restart or cold restart. This is available on CLI/GUI/SNMP.

System captures the last 3 instances of an abnormal event. The 3 debug files (binary) are stored on a single debug image which can be downloaded for further investigation.

10

2010 ADVA Optical Networking. All rights reserved. Confidential.

Recovery Command
The intent of the recovery command is to allow for the recovery of a unit in the case where passwords are lost.

This will allow for the reset of the unit back to factory defaults. All configuration and user data will be reset.

11

2010 ADVA Optical Networking. All rights reserved. Confidential.

Recovery Commands
Accessing the Command
Connect to the serial port and hold the space bar down while rebooting/power cycling.
Copyright (c) 2007-2010 ADVA Optical Networking. All rights reserved. Login: U-Boot 2009.03-22427 (Jan 27 2010 - 18:06:48) MPC83XX

CPU:

e300c3, MPC8313A, Rev: 2.1 at 330 MHz, CSB: 165 MHz

Board: FSP150CC-GE206 I2C: SPI: DRAM: ready ready 256 MB

Testing DRAM... passed FLASH: 4 MB NAND: Net: => 512 MiB TSEC0, TSEC1

12

2010 ADVA Optical Networking. All rights reserved. Confidential.

Recovery Commands
Accessing the Command

Once the bootup sequence has been interrupted, at the => prompt, type boot factorydefaults

=> boot factorydefaults Warning: all saved configuration will be erased. Continue (yes/no)?

13

2010 ADVA Optical Networking. All rights reserved. Confidential.

DB Maintenance Mode
Login: root Password: ******************************************************************************** **** DB Maintenance Mode **** The system has encountered an unrecoverable error with the configuration database. None of the available database files on the system can be used to restore the system to it's previous state. The operations permitted in this mode are: o restore factory/system defaults (results in loss of all provisioning) o configure local DCN o backup and transfer current DB from the system o download and restore DB onto the system o restart cold/warm o show logs (audit/alarm/security)

******************************************************************************** DB Maintenance mode-->

The commands to restore defaults, configure DCN etc. are the same as in the normal mode

14

2010 ADVA Optical Networking. All rights reserved. Confidential.

End of Troubleshooting

IMPORTANT NOTICE The content of this presentation is strictly confidential. ADVA Optical Networking is the exclusive owner or licensee of the content, material, and information in this presentation. Any reproduction, publication or reprint, in whole or in part, is strictly prohibited. The information in this presentation may not be accurate, complete or up to date, and is provided without warranties or representations of any kind, either express or implied. ADVA Optical Networking shall not be responsible for and disclaims any liability for any loss or damages, including without limitation, direct, indirect, incidental, consequential and special damages, alleged to have been caused by or in connection with using and/or relying on the information contained in this presentation. Copyright for the entire content of this presentation: ADVA Optical Networking.