SSSSSSCCCCCCSI SI SI SI SI SI oooooovvvvvveeeeeerrrrrr FFFFFFCCCCCCPPPPPP ffffffoooooorrrrrr LLLLLLiiiiiinnnnnnuuuuuuxxxxxx oooooonnnnnn

SSSSSSyyyyyysssssstttttteeeeeemmmmmm zzzzzz IIIIIInnnnnnttttttrrrrrroooooodu du du du du duccccccttttttiiiiiioooooonnnnnn aaaaaannnnnndddddd

NNNNNNeeeeeew F w F w F w F w F w Feeeeeeaaaaaatu tu tu tu tu turrrrrreeeeeessssss
Christof Schmitt <christof.schmitt@de.ibm.com>
IBM Germany Research & Develoment
!""#$"#$%&
Session '!('
AAAAAAbstr bstr bstr bstr bstr bstraaaaaacccccctttttt
)he *in+, -fc device driver adds s+ort for .ibre Channel
attached SCSI devices to *in+, on System -. )he .ibre
Channel rotocol is an oen/ standard$based alternative and
s+lement to e,istin0 1SC23 or .IC23 connections and
becomes more and more imortant. )he intention of this
resentation is to 0ive an introd+ction to the SCSI 4orld on a
System - mainframe. )his resentation 4ill cover hard4are
and soft4are re5+irements/ confi0+ration/ erformance
considerations/ I6* and d+m. 2ther oints 4ill be .C6
s+ort in recent *in+, distrib+tions and ne4 feat+res to
imrove the .C6 s+ort in *in+, on System -.
AAAAAAggggggeeeeeennnnnnddddddaaaaaa

Introd+ction

Channel I72 vs. SCSI

8ard4are re5+irements

Soft4are/ Confi0+ration

SCSI I6*

SCSI d+m

*in+, distrib+tion s+ort

36I9

3e4 feat+res
IIIIIInnnnnntr tr tr tr tr trod od od od od oduuuuuuccccccti ti ti ti ti tion on on on on on

connect System - to 2en

Systems stora0e :dis;/ tae/ ...<

inte0rate in e,istin0 Stora0e

=rea 3et4or;s :S=3s<

reso+rce sharin0 4ith 2en

Systems

I72 +ses SCSI over .ibre

Channel 6rotocol :.C6<

System - .IC23 1,ress card

-fc as device driver in *in+,

S=3
SSSSSSAAAAAANNNNNN top top top top top topo o o o o oog og og og og ogiiiiiieeeeeessssss aaaaaannnnnndddddd SSSSSSyyyyyysssssste te te te te temmmmmm zzzzzz
oint$to$oint s4itched fabric
direct attached
arbitrated loo
)ae )ae
Dis; Dis; )ae Dis;
nnnnnnooooootttttt
ssssssuuuuuupppppppo po po po po porrrrrrte te te te te tedddddd
FC FC FC FC FC FCP P P P P P cccccc!!!!!!an an an an an annnnnnneeeeee aaaaaannnnnndddddd su su su su su subc bc bc bc bc bc!!!!!!aaaaaannnnnnnnnnnneeeeee
*in+,
FCP
subchannel
FCP channel
*in+, connects thro+0h .C6
s+bchannels to .C6 attached
stora0e.
A subchannel is identified
in Linux - by its bus identifier
which is derived from the
subchannel's device
number.
samle .C6 s+bchannel
:as seen in *in+,<>
7sys7b+s7cc47drivers7-fc7".".("d?
"or "or "or "or "or "ordddddd "i "i "i "i "i "iddddddeeeeee PPPPPPoooooorrrrrrt t t t t t NNNNNNaaaaaammmmmmeeeeeessssss ######""PN ""PN ""PN ""PN ""PN ""PNs$ s$ s$ s$ s$ s$
other 2S
FCP
*in+,
.C6
s+bchannel
FCP channel
Stora0e devices and servers
attach thro+0h
.ibre Channel orts
:called 3@6orts<.
An N_Port is identified by its
World-Wide Port Name
WWPN!.
"or redundancy# servers or
stora$e may attach throu$h
several N_Ports.
samle AA63>
",(""("BC&""c!"b#e
LLLLLLog og og og og ogiiiiiiccccccaaaaaa %%%%%%nnnnnniiiiiitttttt NNNNNNuuuuuummmmmmbe be be be be berrrrrrssssss ######LLLLLL%N %N %N %N %N %Nssssss$$$$$$
Stora0e devices +s+ally comrise
many lo0ical +nits :vol+mes/ tae
drives/ ...<.
= lo0ical +nit is identified by its
.ibre Channel 6rotocol *o0ical
Dnit 3+mber :.C6 *D3<.
samle .C6 *D3>
",?"%"?""!""""""""
AAAAAAcccccccccccceeeeeessssssssssssiiiiiinnnnnngggggg SA SA SA SA SA SANNNNNN st st st st st stor or or or or oraaaaaaggggggeeeeee
.C6 device id
e.0. ".".e'C!
Aorld4ide 6ort 3ame
:AA63<
e.0. ",(""("BC&""ce'&aB
*o0ical Dnit 3+mber
:*D3<
e.0. ",?"%!?"&?""""""""
S=3
SSSSSSCCCCCCSI SI SI SI SI SI ccccccom om om om om omppppppaaaaaarrrrrreeeeeedddddd to to to to to to CCCCCC!!!!!!aaaaaannnnnnnnnnnneeeeee IIIIII&&&&&&''''''

.C6 adater is defined in System - I72 confi0+ration

6orts and *D3s attachment handled in 2eratin0 Systems

M+ltiathin0 handled in 2eratin0 System

3o dis; si-e restrictions for SCSI dis;s

=dditional confi0+ration o+tside System - necessary

Eonin0 in the S=3 fabric

*D3 mas;in0 on the stora0e server

((((((ar ar ar ar ar arddddddwwwwwwaaaaaarrrrrreeeeee rrrrrreeeeee)u )u )u )u )u )uiiiiiirrrrrreeeeeemmmmmmeeeeeennnnnnts ts ts ts ts ts

IBM -Series #""/ #'"/ '"" or ''"

IBM System -' or -%"

.IC23 or .IC23 1,ress adater cards

.ibre Channel stora0e system

2tional>

.ibre Channel s4itch :for .abric toolo0y<

IBM System -' or -%" :for 36I9 s+ort<

((((((ar ar ar ar ar arddddddwwwwwwaaaaaarrrrrreeeeee* * * * * * ++++++eeeeeeffffffiiiiiinnnnnneeeeee FC FC FC FC FC FCP P P P P P aaaaaaddddddaaaaaappppppte te te te te terrrrrr iiiiiinnnnnn IIIIII''''''CCCCCC++++++FFFFFF
CCCCCCHHHHHHPPPPPPIIIIIIDDDDDD PPPPPPAAAAAATTTTTTHHHHHH======((((((CCCCCCSSSSSSSSSSSS((((((000000,,,,,,111111,,,,,,222222,,,,,,333333)))))),,,,,,555555111111)))))),,,,,,SSSSSSHHHHHHAAAAAARRRRRREEEEEEDDDDDD,,,,,, ******
NNNNNNOOOOOOTTTTTTPPPPPPAAAAAARRRRRRTTTTTT======((((((((((((CCCCCCSSSSSSSSSSSS((((((111111)))))),,,,,,((((((TTTTTTRRRRRRXXXXXX111111)))))),,,,,,((((((======)))))))))))),,,,,,((((((CCCCCCSSSSSSSSSSSS((((((333333)))))),,,,,,((((((TTTTTTRRRRRRXXXXXX222222,,,,,,TTTTTT222222999999CCCCCCFFFFFFAAAAAA)))))),,,,,,((((((======))))))))))))))))))******
,,,,,,PPPPPPCCCCCCHHHHHHIIIIIIDDDDDD======111111CCCCCC333333,,,,,,TTTTTTYYYYYYPPPPPPEEEEEE======FFFFFFCCCCCCPPPPPP
CCCCCCNNNNNNTTTTTTLLLLLLUUUUUUNNNNNNIIIIIITTTTTT CCCCCCUUUUUUNNNNNNUUUUUUMMMMMMBBBBBBRRRRRR======333333DDDDDD000000000000,,,,,, ******
PPPPPPAAAAAATTTTTTHHHHHH======((((((((((((CCCCCCSSSSSSSSSSSS((((((000000)))))),,,,,,555555111111)))))),,,,,,((((((CCCCCCSSSSSSSSSSSS((((((111111)))))),,,,,,555555111111)))))),,,,,,((((((CCCCCCSSSSSSSSSSSS((((((222222)))))),,,,,,555555111111)))))),,,,,,((((((CCCCCCSSSSSSSSSSSS((((((333333)))))),,,,,,555555111111)))))))))))),,,,,, ******
UUUUUUNNNNNNIIIIIITTTTTT======FFFFFFCCCCCCPPPPPP
IIIIIIOOOOOODDDDDDEEEEEEIIIIIICCCCCCEEEEEE AAAAAADDDDDDDDDDDDRRRRRREEEEEESSSSSSSSSSSS======((((((333333DDDDDD000000000000,,,,,,000000000000111111)))))),,,,,,CCCCCCUUUUUUNNNNNNUUUUUUMMMMMMBBBBBBRRRRRR======((((((333333DDDDDD000000000000)))))),,,,,,UUUUUUNNNNNNIIIIIITTTTTT======FFFFFFCCCCCCPPPPPP
IIIIIIOOOOOODDDDDDEEEEEEIIIIIICCCCCCEEEEEE AAAAAADDDDDDDDDDDDRRRRRREEEEEESSSSSSSSSSSS======((((((333333DDDDDD000000111111,,,,,,000000000000!!!!!!)))))),,,,,,CCCCCCUUUUUUNNNNNNUUUUUUMMMMMMBBBBBBRRRRRR======((((((333333DDDDDD000000000000)))))),,,,,, ******
PPPPPPAAAAAARRRRRRTTTTTTIIIIIITTTTTTIIIIIIOOOOOONNNNNN======((((((((((((CCCCCCSSSSSSSSSSSS((((((000000)))))),,,,,,TTTTTT222222999999LLLLLLPPPPPP111111111111,,,,,,TTTTTT222222999999LLLLLLPPPPPP111111222222,,,,,,TTTTTT222222999999LLLLLLPPPPPP111111333333,,,,,,TTTTTT222222999999LLLLLLPPPPPP111111"""""",,,,,,TTTTTT222222999999LLLLLLPPPPPP******
111111555555)))))),,,,,,((((((CCCCCCSSSSSSSSSSSS((((((111111)))))),,,,,,TTTTTT222222999999LLLLLLPPPPPP222222######,,,,,,TTTTTT222222999999LLLLLLPPPPPP222222!!!!!!,,,,,,TTTTTT222222999999LLLLLLPPPPPP222222999999,,,,,,TTTTTT222222999999LLLLLLPPPPPP333333000000)))))),,,,,,((((((CCCCCCSSSSSSSSSSSS((((((222222)))))),,,,,,TTTTTT222222999999******
LLLLLLPPPPPP""""""111111,,,,,,TTTTTT222222999999LLLLLLPPPPPP""""""222222,,,,,,TTTTTT222222999999LLLLLLPPPPPP""""""333333,,,,,,TTTTTT222222999999LLLLLLPPPPPP"""""""""""",,,,,,TTTTTT222222999999LLLLLLPPPPPP""""""555555)))))),,,,,,((((((CCCCCCSSSSSSSSSSSS((((((333333)))))),,,,,,TTTTTT222222999999LLLLLLPPPPPP555555######,,,,,,TTTTTT222222******
999999LLLLLLPPPPPP555555!!!!!!,,,,,,TTTTTT222222999999LLLLLLPPPPPP555555$$$$$$,,,,,,TTTTTT222222999999LLLLLLPPPPPP555555999999,,,,,,TTTTTT222222999999LLLLLLPPPPPP######000000)))))))))))),,,,,,UUUUUUNNNNNNIIIIIITTTTTT======FFFFFFCCCCCCPPPPPP
IIIIIIOOOOOODDDDDDEEEEEEIIIIIICCCCCCEEEEEE AAAAAADDDDDDDDDDDDRRRRRREEEEEESSSSSSSSSSSS======((((((333333DDDDDD000000$$$$$$,,,,,,000000555555######)))))),,,,,,CCCCCCUUUUUUNNNNNNUUUUUUMMMMMMBBBBBBRRRRRR======((((((333333DDDDDD000000000000)))))),,,,,, ******
PPPPPPAAAAAARRRRRRTTTTTTIIIIIITTTTTTIIIIIIOOOOOONNNNNN======((((((((((((CCCCCCSSSSSSSSSSSS((((((000000)))))),,,,,,TTTTTT222222999999LLLLLLPPPPPP111111555555)))))),,,,,,((((((CCCCCCSSSSSSSSSSSS((((((111111)))))),,,,,,TTTTTT222222999999LLLLLLPPPPPP333333000000)))))),,,,,,((((((CCCCCCSSSSSSSSSSSS((((((222222)))))),,,,,,TTTTTT222222999999******
LLLLLLPPPPPP""""""555555)))))),,,,,,((((((CCCCCCSSSSSSSSSSSS((((((333333)))))),,,,,,TTTTTT222222999999LLLLLLPPPPPP######000000)))))))))))),,,,,,UUUUUUNNNNNNIIIIIITTTTTT======FFFFFFCCCCCCPPPPPP
SSSSSSof of of of of oftw tw tw tw tw twaaaaaarrrrrreeeeee rrrrrreeeeee)u )u )u )u )u )uiiiiiirrrrrreeeeeemmmmmmeeeeeennnnnnttttttssssss

-fc is art of standard *in+, ;ernel and standard distrib+tions

s+orted *in+, distrib+tions

S*1S'

S*1S%"

R81*?

R81*(

Recommendations

start 4ith latest available +date 7 service ac;

incl+de *in+, in maintenance lannin0

chec; for ossibly related -79M 6).s

IIIIII&&&&&&'''''' sta sta sta sta sta stacccccc,,,,,, ffffffor or or or or or SSSSSSCCCCCCSSSSSSIIIIII an an an an an andddddd LLLLLLiiiiiinnnnnnuuuuuuxxxxxx
file system
Bloc; Devices 7 Device Maer 7 *9M
*in+, SCSI layer
-fc
*in+, 5dio mod+le
-79M
.C6 adater
:otional<
SSSSSSAN AN AN AN AN AN
common *in+, code
inside *in+, system
System - hard4are
CCCCCCom om om om om ommmmmmmaaaaaannnnnndddddd iiiiiinnnnnneeeeee sssssseeeeeetu tu tu tu tu tupppppp ######SSSSSSLLLLLL------SSSSSS......&/ &/ &/ &/ &/ &/000000111111
2( 2( 2( 2( 2( 2(------LLLLLL333333&4 &4 &4 &4 &4 &4$$$$$$
# cd /sys/bus/ccw/drivers/zfcp/0.0.3c00/
# echo 1 > online
# echo 0x500507630313c562 > port_add
# echo 0x401040cc00000000 > 0x500507630313c562/unit_add
# lszfcp -D
0.0.3c00/0x500507630313c562/0x401040cc00000000
0:0:0:1087127568
# lsscsi -t
[0:0:0:1087127568]disk fc:0x500507630313c562,0x650d13 /dev/
sda
.C6 adater
AA63
*D3
Pe Pe Pe Pe Pe Perrrrrrssssssiiiiiisssssste te te te te tennnnnntttttt se se se se se settttttuuuuuupppppp* * * * * * SSSSSSLLLLLL------SSSSSS

thro+0h FaS) or

set+ in 7etc7sysconfi07hard4are7h4cf0$-fc$b+s$cc4$".".G
Pe Pe Pe Pe Pe Perrrrrrssssssiiiiiisssssste te te te te tennnnnntttttt se se se se se settttttuuuuuupppppp* * * * * * 2( 2( 2( 2( 2( 2(------LLLLLL

Installer GDI or

7etc7-fc.conf

SCSI and SCSI *D3 are +n+sed fields :+se "/ %/ ...<
555555uuuuuuti ti ti ti ti tippppppaaaaaat! t! t! t! t! t!iiiiiinnnnnngggggg ccccccon on on on on onssssssiiiiiiddddddeeeeeerrrrrraaaaaati ti ti ti ti tion on on on on onssssss

stora0e controllers allo4 different aths

ath failover re5+ired for

stora0e system maintenance

S=3 fabric maintenance :4ith d+al fabrics<

imlemented inside *in+,

dis; stora0e> m+ltiath$tools

IBM tae drives> lin@tae driver

more details> Session '!#'/ H=dditional .eet for the 6en0+in $

SCSI over .C6 M+ltiathin0 for *in+, on System -I
SSSSSSCCCCCCSI SI SI SI SI SI IIIIIIPPPPPPLLLLLL

Similar to I6* from D=SD

Re5+ires to address the SCSI dis;

.C6 adater id

Remote ort

*D3
SSSSSSAN AN AN AN AN AN
00:
00: CP SET LOADDEV PORTNAME 50050763 0313C562 LUN 401040CF 00000000
00:
00: CP Q LOADDEV
PORTNAME 50050763 0313C562 LUN 401040CF 00000000 BOOTPROG 0
BR_LBA 00000000 00000000
00:
00: CP IPL 3C00
00: HCPLDI2816I Acquiring the machine loader from the processor controller.
00: HCPLDI2817I Load completed from the processor controller.
00: HCPLDI2817I Now starting the machine loader.
01: HCPGSP2630I The virtual machine is placed in CP mode due to a SIGP stop and
store status from CPU 00.
00: MLOEVL012I: Machine loader up and running (version 0.18).
00: MLOPDM003I: Machine loader finished, moving data to final storage location.
Linux version 2.6.16.60-0.9-default (geeko@buildhost) (gcc version 4.1.2 2007011
5 (SUSE Linux)) #1 SMP Mon Mar 17 17:16:31 UTC 2008
We are running under VM (64 bit mode)
Detected 2 CPU's
Boot cpu address 0
Built 1 zonelists
Kernel command line: root=/dev/disk/by-id/scsi-36005076303ffc56200000000000010ce
-part1 TERM=dumb
SSSSSSCCCCCCSI SI SI SI SI SI IIIIIIPPPPPPLLLLLL eeeeeexxxxxxam am am am am amppppppeeeeee wwwwwwiiiiiit! t! t! t! t! t! zzzzzz&&&&&&666666555555
*D3 AA63
.C6 adater
SSSSSSCCCCCCSI SI SI SI SI SI IIIIIIPPPPPPLLLLLL ffffffor or or or or or LLLLLLPA PA PA PA PA PA2s 2s 2s 2s 2s 2s
SSSSSSCCCCCCSI SI SI SI SI SI dddddduuuuuummmmmmpppppp

D+m memory of one *6=R to dis; for roblem analysis

Similar to 9MDDM6 and d+m to D=SD

B+t> SCSI d+m only s+orted for *6=Rs/ not -79M

6rearation s+mmary>

lar0e SCSI dis; :*6=R memory J %"MB<

fdis; 7dev7sda

m;e!fs 7dev7sda%

mo+nt 7dev7sda% 7mnt

-il $D 7dev7sda% $t 7mnt

+mo+nt 7mnt
IIIIIIssssssssssssuuuuuueeeeee SC SC SC SC SC SCSSSSSSIIIIII dddddduuuuuummmmmmpppppp ffffffrrrrrrom om om om om om ((((((555555CCCCCC

Select C6C ima0e for *6=R to d+m

Goto *oad anel

Iss+e SCSI d+m

.C6 device

AA63

*D3
NNNNNNPI PI PI PI PI PI666666

3@6ort Identifier 9irt+ali-ation :36I9<

4itho+t 36I9> one AA63 for .C6 channel

4ith 36I9> +ni5+e AA63 for each .C6 s+bchannel

enables roer -onin0 in S=3 fabrics

enables roer *D3 mas;in0 in stora0e devices

sec+rity

access control
LLLLLL%N %N %N %N %N %N mmmmmmaaaaaas, s, s, s, s, s,iiiiiinnnnnngggggg wwwwwwiiiiiit! t! t! t! t! t! NNNNNNPI PI PI PI PI PI666666
*in+, =
*in+, B
*in+, C
*in+, K
.
.
.
.C6
Channe
l
Dis; %
:e,cl+sive<
Dis; !
:e,cl+sive<
Dis; &
:e,cl+sive<
Dis; ?
:shared<
Stora0e server can identify *in+, 0+ests via AA63s
SSSSSSAAAAAANNNNNN zzzzzzon on on on on oniiiiiinnnnnngggggg wwwwwwiiiiiit! t! t! t! t! t! NNNNNNPI PI PI PI PI PI666666

Different *in+, 0+ests in different -ones

LLLLLLiiiiiinnnnnnuuuuuuxxxxxx AAAAAA
LLLLLLiiiiiinnnnnnuuuuuuxxxxxx 777777
LLLLLLiiiiiinnnnnnuuuuuuxxxxxx CCCCCC
FFFFFFCCCCCCPPPPPP
CCCCCC!!!!!!aaaaaannnnnnnnnnnneeeeee
zzzzzzoooooonnnnnneeeeee 888888bbbbbbuuuuuueeeeee999999
zzzzzzoooooonnnnnneeeeee 888888rrrrrreeeeeed9 d9 d9 d9 d9 d9
zzzzzzoooooonnnnnneeeeee 888888ggggggrrrrrreeeeeeeeeeeennnnnn999999
NNNNNNPI PI PI PI PI PI6 6 6 6 6 6 rrrrrreeeeee)u )u )u )u )u )uiiiiiirrrrrreeeeeemmmmmmeeeeeennnnnnts ts ts ts ts ts
J J

36I9 is available on System - servers.

.IC23 1,ress ! adater r+nnin0 4ith MC*""& on 1C L''C(#

-79M

-79M (.! or (.&

-79M (.% 4ith the 6). for =6=R 9MC&B??

*in+, Distrib+tion

C+rrently S*1S' S6&7?/ S*1S%"/ R81*( :*6=R mode or -79M<

36I9$Caable S4itch

only re5+ired for s4itch adMacent to System -

Pe Pe Pe Pe Pe Perrrrrrffffffoooooorrrrrrmmmmmman an an an an ancccccceeeeee ccccccon on on on on onssssssiiiiiiddddddeeeeeerrrrrrati ati ati ati ati ation on on on on onssssss

DS#&""

-''" *6=R

S*1S' S6!

# C6Ds

# .IC23

# .C6

!(C MB

Io-one &.'C
8i0her erformance
4ith SCSI dis;s
comared to 1CND
NNNNNNeeeeeewwwwww ffffffeeeeeeaaaaaatu tu tu tu tu turrrrrreeeeeessssss ov ov ov ov ov oveeeeeerrrrrrvvvvvviiiiiieeeeeewwwwww

Bl;trace for I72 and latency tracin0

Channel statistics from sysfs :!.C.!C<

Channel and fabric latencies in +stream code :!.C.!B<

=+tomatic ort discovery :!.C.!B<

Messa0e clean+ :!.C.!B<

b b b b b b,,,,,,tr tr tr tr tr trac ac ac ac ac aceeeeee

HoldI -fc secific statistics only available as Hadd$onI atch

ne4 ;ernel infrastr+ct+re and tools> bl;trace

common I72 tracin0 infrastr+ct+re in *in+,

re5+est si-es 7 latencies

f+nctional relacement for most of the -fc secific statistics

==================== All Devices ====================
ALL MIN AVG MAX N
--------------- ------------- ------------- ------------- -----------
Q2Q 0.000000072 0.000086313 5.453721801 1257686
Q2I 0.000000359 0.000000516 0.023150311 1257687
I2D 0.000000933 0.003573727 0.487170508 1267275
D2C 0.000363719 0.034028080 0.708048174 1257687
Q2C 0.000395336 0.037609824 0.708064315 1257687
:O+e+ed/ Iss+ed/ Disatched/ Comlete<
D!C PP Disatched to Comlete
NNNNNNeeeeeewwwwww ssssssta ta ta ta ta tati ti ti ti ti tissssssti ti ti ti ti ticcccccssssss ffffffrrrrrrom om om om om om ssssssyyyyyyssssssffffffssssss
Q cat 7sys7class7scsi@host7host"7me0abytes
%C %
Q cat 7sys7class7scsi@host7host"7re5+ests
&'C& &B !
Q cat 7sys7class7scsi@host7host"7seconds@active
!#B%
Q cat 7sys7class7scsi@host7host"7+tili-ation
! %" "
SSSSSSuuuuuubbbbbbcccccc!!!!!!aaaaaannnnnnnnnnnneeeeee
FFFFFFCCCCCCPPPPPP CCCCCC!!!!!!aaaaaannnnnnnnnnnneeeeee uuuuuuttttttiiiiiiiiiiiizzzzzzaaaaaattttttiiiiiioooooonnnnnn
channel rocessor/ channel b+s/ adater
in+t/ o+t+t/ control re5+ests
more channel data in
7sys7class7fc@host7host"7statistics7
read/ 4ritten
# cat /sys/block/sda/device/write_latency
273 4562 67446 32 616 9212 37
# cat /sys/block/sda/device/read_latency
129 77471 18507807 21 67 110541 3963
# cat /sys/block/sda/device/cmd_latency
92 98 190 23 28 52 2
CCCCCC!!!!!!an an an an an annnnnnneeeeee an an an an an andddddd ffffffaaaaaabr br br br br briiiiiicccccc aaaaaate te te te te tennnnnncccccciiiiiieeeeeessssss

revio+sly art of -fc statistics add$on

available thro+0h sysfs :!.C.!B<

fabric> min/ ma,/ s+m
:micro seconds<
channel> min/ ma,/ s+m
:micro seconds<
re5+est co+nt
AAAAAAuuuuuutom tom tom tom tom tomaaaaaati ti ti ti ti ticccccc ppppppor or or or or ort t t t t t ddddddiiiiiisc sc sc sc sc scov ov ov ov ov oveeeeeerrrrrryyyyyy

Discover and attach orts a+tomatically

4hen settin0 adater online/

on chan0es in S=3.
# cd /sys/bus/ccw/drivers/zfcp/0.0.181d/
# echo 1 > online
# ls -d 0x*
0x500308c141699001 0x5005076300cbb130 0x5005076303048335
0x500507630310c562 0x500507630e0202aa 0x500308c141699004
0x5005076300cc0b8e 0x5005076303098335 0x500507630313c562
# echo 0x401040C300000000 > 0x500507630310c562/unit_add

Does not chan0e handlin0 of *D3s

*D3s have to be attached man+ally>

Man+al tri00er available>

# echo 1 > port_rescan
555555eeeeeessssssssssssag ag ag ag ag ageeeeee cccccceeeeeeaaaaaannnnnnuuuuuupppppp

removed deb+0 and trace messa0es

removed information also available in sysfs or ls-fc

standard format :-fc> ".".KKKK> ...<

-fc> )he adater ".".%#%d reorted the follo4in0 characteristics>
AA33 ",(""("BC?""c!d"'e/ AA63 ",(""("BC?"%e"B%b!/ S@ID ",""C#'&%&/
adater version ",&/ *IC version ",%B"b/ .C lin; seed ! Gb7s
-fc> S4itched fabric fibrechannel net4or; detected at adater ".".%#%d.
-fc> adater ".".%#%d> no ath
-fc> adater ".".%#%d> oerational a0ain
-fc> ".".%#%d> S4itched fabric fibrechannel net4or; detected.
-fc> ".".%#%d> no ath
-fc> ".".%#%d> oerational a0ain
old
ne4
2e 2e 2e 2e 2e 2effffffeeeeeerrrrrreeeeeennnnnncccccceeeeeessssss

S+orted devices
htt>77444.ibm.com7systems7-7hard4are7connectivity7rod+cts7fc.html

Stora0e device interoerability

htt>77444.ibm.com7systems7stora0e7dis;7dsC"""7df7intero.df
htt>77444.ibm.com7systems7stora0e7dis;7ds#"""7intero.df
htt>77444.ibm.com7systems7stora0e7soft4are7virt+ali-ation7svc7intero.html
htt>77444.ibm.com7systems7s+ort7stora0e7confi07ssic7inde,.Ms

*in+, on System - Doc+mentation

htt>77444.ibm.com7develoer4or;s7lin+,7lin+,&'"7

Device Drivers/ .eat+res/ and Commands

8o4 to +se .C$attached SCSI devices 4ith *in+, on System -

Dsin0 the D+m )ools

)+nin0 hints & tis

htt>77444.ibm.com7develoer4or;s7lin+,7lin+,&'"7erf7

Device driver for IBM tae drives

htt>77ft.soft4are.ibm.com7stora0e7devdrvr7Doc7
::::::uuuuuueeeeeessssssti ti ti ti ti tioooooonnnnnnssssss;;;;;;
<<<<<<rrrrrraaaaaaddddddeeeeeemmmmmmaaaaaarrrrrr,,,,,,ssssss
<<<<<<!!!!!!eeeeee ffffff ooooooo o o o o owwwwwwing are ing are ing are ing are ing are ing are ttttttrrrrrraaaaaaddddddeeeeeemmmmmmar, ar, ar, ar, ar, ar,ssssss of of of of of of tttttt!e !e !e !e !e !e Int Int Int Int Int Inteeeeeerrrrrrnnnnnnat at at at at atio io io io io ionnnnnna 7 a 7 a 7 a 7 a 7 a 7uuuuuussssssin in in in in ineeeeeessssssssssss 555555ac! ac! ac! ac! ac! ac!ine ine ine ine ine inessssss Co Co Co Co Co Corpo rpo rpo rpo rpo rporat rat rat rat rat ratio io io io io ion i n i n i n i n i n in t n t n t n t n t n t!!!!!!eeeeee %%%%%%nnnnnnit it it it it iteeeeeed St d St d St d St d St d Stat at at at at ateeeeeessssss1 1 1 1 1 1 ot ot ot ot ot ot!!!!!!eeeeeer co r co r co r co r co r coun un un un un unttttttrie rie rie rie rie riessssss1 1 1 1 1 1 or bo or bo or bo or bo or bo or botttttt!!!!!!======
<<<<<<!!!!!!eeeeee ffffffo o o o o oow ow ow ow ow owiiiiiinnnnnngggggg aaaaaarrrrrreeeeee ttttttrrrrrraaaaaade de de de de demmmmmmaaaaaarrrrrr,,,,,,ssssss oooooorrrrrr rrrrrreeeeeegi gi gi gi gi gist st st st st steeeeeerrrrrreeeeeed d d d d d ttttttrrrrrrad ad ad ad ad adeeeeeemmmmmmar ar ar ar ar ar,,,,,,ssssss of of of of of of ot ot ot ot ot ot!!!!!!eeeeeerrrrrr ccccccom om om om om ompa pa pa pa pa pannnnnniiiiiieeeeees= s= s= s= s= s=
G =ll other rod+cts may be trademar;s or re0istered trademar;s of their resective comanies.
NNNNNNooooootttttt eeeeeessssss>
6erformance is in Internal )hro+0h+t Rate :I)R< ratio based on meas+rements and roMections +sin0 standard IBM benchmar;s in a controlled environment. )he act+al thro+0h+t that any +ser 4ill
e,erience 4ill vary deendin0 +on considerations s+ch as the amo+nt of m+ltiro0rammin0 in the +serRs Mob stream/ the I72 confi0+ration/ the stora0e confi0+ration/ and the 4or;load rocessed.
)herefore/ no ass+rance can be 0iven that an individ+al +ser 4ill achieve thro+0h+t imrovements e5+ivalent to the erformance ratios stated here.
IBM hard4are rod+cts are man+fact+red from ne4 arts/ or ne4 and serviceable +sed arts. Re0ardless/ o+r 4arranty terms aly.
=ll c+stomer e,amles cited or described in this resentation are resented as ill+strations of the manner in 4hich some c+stomers have +sed IBM rod+cts and the res+lts they may have achieved.
=ct+al environmental costs and erformance characteristics 4ill vary deendin0 on individ+al c+stomer confi0+rations and conditions.
)his +blication 4as rod+ced in the Dnited States. IBM may not offer the rod+cts/ services or feat+res disc+ssed in this doc+ment in other co+ntries/ and the information may be s+bMect to chan0e
4itho+t notice. Cons+lt yo+r local IBM b+siness contact for information on the rod+ct or services available in yo+r area.
=ll statements re0ardin0 IBMRs f+t+re direction and intent are s+bMect to chan0e or 4ithdra4al 4itho+t notice/ and reresent 0oals and obMectives only.
Information abo+t non$ IBM rod+cts is obtained from the man+fact+rers of those rod+cts or their +blished anno+ncements. IBM has not tested those rod+cts and cannot confirm the erformance/
comatibility/ or any other claims related to non$ IBM rod+cts. O+estions on the caabilities of non$ IBM rod+cts sho+ld be addressed to the s+liers of those rod+cts.
6rices s+bMect to chan0e 4itho+t notice. Contact yo+r IBM reresentative or B+siness 6artner for the most c+rrent ricin0 in yo+r 0eo0rahy.
=dobe/ t he =dobe lo0o/ 6ost Scrit / and t he 6ost Scrit lo0o are eit her re0ist ered t rademar;s or t rademar;s of =dobe Syst ems Incororat ed in t he Dnit ed St at es/ and7or ot her
co+nt ries.
Cell Broadband 1n0ine is a t rademar; of Sony Com+t er 1nt ert ainment / Inc. in t he Dnit ed St at es/ ot her co+nt ries/ or bot h and is +sed +nder license t heref rom.
Lava and all Lava$based t rademar;s are t rademar;s of S+n Microsyst ems/ Inc. in t he Dnit ed St at es/ ot her co+nt ries/ or bot h.
Microsof t / Aindo4s/ Aindo4s 3)/ and t he Aindo4s lo0o are t rademar;s of Microsof t Cororat ion in t he Dnit ed St at es/ ot her co+nt ries/ or bot h.
Int el/ Int el lo0o/ Int el Inside/ Int el Inside lo0o/ Int el Cent rino/ Int el Cent rino lo0o/ Celeron/ Int el Keon/ Int el SeedSt e/ It ani+m/ and 6ent i+m are t rademar;s or re0ist ered t rademar;s of
Int el Cororat ion or it s s+bsidiaries in t he Dnit ed St at es and ot her co+nt ries.
D3IK is a re0ist ered t rademar; of )he 2en Gro+ in t he Dnit ed St at es and ot her co+nt ries.
*in+, is a re0ist ered t rademar; of *in+s )orvalds in t he Dnit ed St at es/ ot her co+nt ries/ or bot h.
I)I* is a re0ist ered t rademar;/ and a re0ist ered comm+nit y t rademar; of t he 2f f ice of Government Commerce/ and is re0ist ered in t he D.S. 6at ent and )rademar; 2f f ice.
I) Inf rast r+ct +re *ibrary is a re0ist ered t rademar; of t he Cent ral Com+t er and )elecomm+nicat ions =0ency/ 4hich is no4 art of t he 2f f ice of Government Commerce.
.or a comlete list of IBM )rademar;s/ see 444.ibm.com7le0al7coytrade.shtml>
G/ =S7?""S/ e b+siness:lo0o<S/ DB1/ 1SC2/ eServer/ .IC23/ IBMS/ IBM :lo0o<S/ iSeriesS/ M9S/ 2S7&'"S/ SeriesS/ RS7C"""S/ S7&"/ 9M71S=S/ 9S17
1S=/ AebShereS/ ,SeriesS/ -72SS/ -SeriesS/ -79MS/ Syst em i/ Syst em i(/ Syst em / Syst em (/ Syst em ,/ Syst em -/ Syst em -'S/ BladeCent erS
3ot all common la4 mar;s +sed by IBM are list ed on t his a0e. .ail+re of a mar; t o aear does not mean t hat IBM does not +se t he mar; nor does it mean t hat t he rod+ct is
not act ively mar;et ed or is not si0nif icant 4it hin it s relevant mar;et . )hose t rademar;s f ollo4ed by S are re0ist ered t rademar;s of IBM in t he Dnit ed St at esT all ot hers are
t rademar;s or common la4 mar;s of IBM in t he Dnit ed St at es.