Solution ID: emc223491 Date Created: 09/18/2009 Last Modified Date: 02/07/2011 Here is the solution: 8 Gb link does

not initialize to FA adapter when using ARB(FF) as fill word on Brocade switches. The Brocade 8 Gb-capable switches have the default fill words set to IDLE. The fill word can be changed to ARB(FF) using a CLI command. Once changed to ARB(FF), the link does not initialize when connected to an EMC FA adapter. The link does come up when the fill word is switched back to IDLE. The EMC perferred idle mode is 3, ARB(ff)/ARB(ff) then IDLE/ARB(ff). The CLI command to check the fill word setting is: portcfgshow <slot/port> The CLI command to change the fill word is: portcfgfillword <slot/port> <0|1|2|3> This problem occurred on 8 Gb Brocade switches connected to 8 Gb Open System and IBM z10 8 Gb FCP links. There are no issues using either fill word on IBM 8 Gb FICON channels to a Brocade 8 Gb-capable switch. Here is the problem or goal: 8 Gb link does not initialize to FA adapter when using ARB(FF) as fill word on Brocade switches. The 8 Gb link will not come up to an EMC Fiber Adapter (FA) Problem Environment: Product: Connectrix DS-300B Product: Connectrix DS-5100B Product: Connectrix DS-5300B Product: Connectrix ED-DCX-B Product: Connectrix ED-DCX-4S-B Director: Fiber Adapter (FA) Product: Symmetrix VMAX Series Causes of this problem: The T11 fiber committee changed the FC-PI-3 standard for 8 Gb attachment on February 4, 2008. This change uses the ARB(FF) primitive signal in place of IDLE as a fill word to maintain link synchronization. During link initialization, 6 IDLE frames are sent before sending ARB(FF) at port state LR3. Slower link speeds only use the IDLE fill word and do not have this issue. Changes affecting this problem: An 8 Gb capable Brocade switch was installed. The default fill word was changed from IDLE to ARB(FF) to comply with T11 standards. Additional Information: At FOS 6.3.1 there are now the three following options when using the portcfgfillword CLI command: 0 | -idle-idle Sets IDLE mode in the Link Init and IDLE as the fill word (default). 1 | -arbff-arbff Sets ARB(ff) in the Link Init and ARB(ff) as the fill word. 2 | -idlef-arbff Sets IDLE mode in the Link Init and ARB(ff) as the fill word. 3 | -aa-then-ia Attempts hardware arbff-arbff (mode 1) first. If the attempt fails to go into active state, this command executes software idle-arb (mode 2). Mode 3 is preferable to modes 1 and 2 as it captures more cases. For the Symmetrix VMAX Series to eliminate problems with the counter for er_bad_os increasing on the switch port when running at 8GB, set fill word to 3. FICON_Pluto_upper_174-20:dennis> portcfgshow 2/3 Area Number: 67 Speed Level: AUTO(HW) Fill Word: 1(Arbff-Arbff) AL_PA Offset 13: OFF

Trunk Port Long Distance VC Link Init Locked L_Port Locked G_Port Disabled E_Port ISL R_RDY Mode RSCN Suppressed Persistent Disable NPIV capability QOS E_Port Port Auto Disable: Rate Limit EX Port Mirror Port Credit Recovery F_Port Buffers FICON_Pluto_upper_174-20:dennis>

ON LE ON OFF OFF OFF OFF OFF OFF ON ON OFF OFF OFF OFF ON OFF

Script to disruptively change fillword to mode 3 for all ports on card. Repeat command for each port card slot in switch. for ((i=0;i<32;i++)); do (echo <slot>/$i;portcfgfillword <slot>/$i 3); done

Solution ID: emc281517 Date Created: 11/08/2011 Last Modified Date: 02/29/2012 Here is the solution: Connectrix Brocade: VMware host not logging in as N-port or intermittently loosing access to LUN's Recommend changing the Fill Word mode on the 8GB ports to 3 as per EMC Knowledgebase solution emc223491 "8 Gb link does not initialize to FA adapter when using ARB (FF) as Fill Word on Brocade switches".

1. Find the host in question using the 'switchshow' command.

Below is an example where the HBA which is configured for NPIV support logged in as a F port, when it should not be. Note this port is running at 8GB. Index Slot Port Address Media Speed State Proto =================================================== 124 12 12 027c00 id N8 Online FC F-Port

10:00:00:05:XX:XX:XX:XX

2. Check the current Fill Word setting on the port using the 'portcfgshow <slot/port>' command. #> portcfgshow 12/12 Area Number: Speed Level: Fill Word(On Active) Fill Word(Current) AL_PA Offset 13: Trunk Port Long Distance VC Link Init Locked L_Port Locked G_Port [output truncated...] 81 8G 0(Idle-Idle)=========>Currently Fill word is mode 0 0(Idle-Idle) OFF ON OFF OFF OFF OFF

3. Clear the counters on the switch and check to see if the "er_bad_os" field is incrementing. Example: #> statsclear #> slotstatsclear #> portstatsshow 12/12 stat_wtx stat_wrx stat_ftx stat_frx stat_c2_frx stat_c3_frx stat_lc_rx stat_mc_rx stat_mc_to stat_mc_tx tim_rdy_pri tim_txcrd_z tim_txcrd_z_vc 0- 3: tim_txcrd_z_vc 4- 7: tim_txcrd_z_vc 8-11: tim_txcrd_z_vc 12-15: er_enc_in er_crc er_trunc er_toolong er_bad_eof er_enc_out er_bad_os incrementing extremely fast [output truncated..] 4. Change the Fill Word setting to mode 3. Reference EMC Knowledgebase solution emc223491 "8 Gb link does not initialize to FA adapter when using ARB(FF) as fill word on Brocade switches". 5. Clear the counters and check the port counter to make sure its not incrementing. You may need to block and unblock the port to force a full re-login to get your N-port login. Example: #> portdisable 12/12 #> portenable 12/12 #> statsclear #> slotstatsclear #> portstatsshow 12/12 stat_wtx stat_wrx stat_ftx stat_frx stat_c2_frx stat_c3_frx stat_lc_rx stat_mc_rx stat_mc_to stat_mc_tx tim_rdy_pri tim_txcrd_z tim_txcrd_z_vc 0- 3: tim_txcrd_z_vc 4- 7: tim_txcrd_z_vc 8-11: tim_txcrd_z_vc 12-15:

4-byte words transmitted 4-byte words received Frames transmitted Frames received Class 2 frames received Class 3 frames received Link control frames received Multicast frames received Multicast timeouts Multicast frames transmitted Time R_RDY high priority Time TX Credit Zero (2.5Us ticks) 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Encoding errors inside of frames 0 Frames with CRC errors 0 Frames shorter than minimum 0 Frames longer than maximum 0 Frames with bad end-of-frame 0 Encoding error outside of frames 3456363056 Invalid ordered set=============>this is

0 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

4-byte words transmitted 4-byte words received Frames transmitted Frames received Class 2 frames received Class 3 frames received Link control frames received Multicast frames received Multicast timeouts Multicast frames transmitted Time R_RDY high priority Time TX Credit Zero (2.5Us ticks) 0 0 0 0 0 0 0 0 0 0 0 0

er_enc_in er_crc er_trunc er_toolong er_bad_eof er_enc_out er_bad_os incrementing

0 0 0 0 0 0 0

Encoding errors inside of frames Frames with CRC errors Frames shorter than minimum Frames longer than maximum Frames with bad end-of-frame Encoding error outside of frames Invalid ordered set=============>No longer

Here is the problem or goal: VMware host not logging in as N-port. VMware host intermittently loosing access to LUN's. Fill Word Problem Environment: Product: Connectrix Brocade Switches All System: VMware vSphere OS: VMware ESX

Solution ID: emc228910 Date Created: 12/04/2009 Last Modified Date: 03/06/2012 Here is the solution: Brocade B-series: er_bad_os counter incrementing when 8 Gb device is attached Issue portcfgfillword command against port where 8 Gb device is to be attached. In FOS 6.3.1 and later there are now the following options when using the CLI portcfgfillword command: 0 | -idle-idle Sets IDLE mode in the Link Init and IDLE as the fill word (default). 1 | -arbff-arbff Sets ARB(ff) in the Link Init and ARB(ff) as the fill word. 2 | -idlef-arbff Sets IDLE mode in the Link Init and ARB(ff) as the fill word. 3 | -aa-then-ia Attempts hardware arbff-arbff (mode 1) first. If the attempt fails to go into active state, this command executes software idle-arb (mode 2). Mode 3 is the preferable to modes 1 and 2 as it captures more cases.

Here is the problem or goal: er_bad_os counter incrementing Problem Environment: Product: Connectrix ED-DCX-B Product: Connectrix DCX-4S-B Product: Connectrix DS-300B Product: Connectrix DS-5100B Product: Connectrix DS-5300B Host adapter: Emulex LPe12002-E Host adapter: Emulex LPe12000-E Host Adapter 8 Gb HBA logged in at 8 Gb

Causes of this problem: The default implementation is to use Idles for link initialization and fill words. Some newer 8 Gb devices may experience problems with the use of Idle/Idle sequence at 8 Gb. These devices may require the ARB (FF) / ARB (FF) sequence to have a successful link initialization. Changes affecting this problem: Newly installed 8Gb HBA / device Additional Information: No errors seen in porterrshow Index Slot Port Address Media Speed State Proto =================================================== 257 1 33 0fbf80 id 8G Online portstatsshow 257 er_bad_os 3277982983 Invalid ordered set /fabos/cliexec/porterrshow : frames enc crc crc too too bad enc disc link loss loss frjt fbsy tx rx in err g_eof shrt long eof out c3 fail sync sig ============================================================================ 257: 2.8g 2.2g 0 0 0 0 0 0 0 0 0 0 0 0 0 This command is not applicable to a non 8G FC port. This command disables and re-enables the port and the port comes online with the new fill word setting. The configuration is stored in nonvolatile memory and is persistent across switch reboots or power cycles.

F-Port

10:00:00:00:c9:xx:xx:xx