Storage Area Networks

SAN Concepts Hardware Compatibility Configuration for ESX Server SAN Failover

ESX Server System Management II Module 6

Storage Area Networks, Network-Attached Storage

SCSI Fibre Channel copper or fiber media NFS or SMB IP LAN or WAN links

Disk array + intelligent controller

File server

Host
SAN

Host

Host
NAS

Host

2
2

For ESX Server 2.0.1 2003-11-05 Copyright 2003 VMware, Inc. All rights reserved.

ESX Server, SAN, and NAS

VMkernel cannot use virtual disks on NAS
Because VMkernel does not know NFS or SMB NAS can be used for /vmimages-type storage
Dont forget to export virtual disks!

VMkernel can use virtual disks on a SAN

VM storage is centralized for easy management Required for clustering between VMs on different ESX Servers

VMkernel can also use raw LUNs on a SAN

Required for physical-to-virtual clustering

3
3

For ESX Server 2.0.1 2003-11-05 Copyright 2003 VMware, Inc. All rights reserved.

Parts of Fibre Channel SAN

Host Bus Adapter (HBA)
This card connects the server to the FC network

Disk Array with Intelligent Controller(s)

Supports the creation of volumes at various RAID levels Controllers are often called storage ports or storage processors (SPs)

SAN Switches
Intelligent devices that interconnect HBAs and SPs

Other devices
Tape drives, jukeboxes may also be connected to a SAN switch
4
4
For ESX Server 2.0.1 2003-11-05 Copyright 2003 VMware, Inc. All rights reserved.

SAN Topology: Direct

Low cost No expansion without system shutdown No redundancy; many single points of failure Storage SP

HBA Server

5
5

For ESX Server 2.0.1 2003-11-05 Copyright 2003 VMware, Inc. All rights reserved.

SAN Topology: Trivial Fabric

Switch allows for later connection of more storage or more servers (robust topology) No redundancy; many single points of failure FC Switch Storage SP

HBA Server

6
6

For ESX Server 2.0.1 2003-11-05 Copyright 2003 VMware, Inc. All rights reserved.

SAN Topology: Arbitrated Loop (FCAL)

Splits transmit and receive into two physical connections Similar to a Token Ring data configuration Not supported by ESX Server

HBA Server Storage SP

HBA Server

7
7

For ESX Server 2.0.1 2003-11-05 Copyright 2003 VMware, Inc. All rights reserved.

SAN Topology: Fabric with Multipath

Storage SP SP

FC Switch

HBA

HBA

HBA

HBA

Server

Server

8
8

For ESX Server 2.0.1 2003-11-05 Copyright 2003 VMware, Inc. All rights reserved.

SAN Concepts
SAN devices are identified by a World Wide Number
Unique, 64-bit address assigned to each piece of Fibre Channel equipment: 210000e08b05672d

A LUN (logical unit number)

Identifies individual units of storage behind a SCSI target ID: a RAID group or a single disk drive

Switched fabric SANs are partitioned into zones

Member devices cannot communicate across zone boundaries Zoning is implemented in terms of switch ports (hard) or WWN (soft)

9
9

For ESX Server 2.0.1 2003-11-05 Copyright 2003 VMware, Inc. All rights reserved.

ESX SAN Hardware Compatibility

HBAs:
QLogic 2200, 2300 series (including OEMed HP cards) Emulex 8000, 9000 series

Switches: Brocade Silkworm, others Storage:

Work closely with pre-sales or support!

IBM FAStT, ESS (Shark); HP MSA, EVA; EMC Symmetrix Fujitsu-Siemens S60; HP XP; Hitachi (clustering configurations not yet supported) EMC Clariion (failover and clustering configurations not yet supported)

10
10

For ESX Server 2.0.1 2003-11-05 Copyright 2003 VMware, Inc. All rights reserved.

Key ESX Server configuration checks

Because LUNs may be discontiguous
Ensure that DiskSupportSparseLUN is 1

Because VMkernel by default only scans LUNs up to 7

Set DiskMaxLUN to desired maximum
Caution: large values will slow boot markedly

Use DiskMaskLUNs to control LUN visibility vmhba0:0:4,6-255;

Because masking overrides max LUN
11
11

For ESX Server 2.0.1 2003-11-05 Copyright 2003 VMware, Inc. All rights reserved.

SAN Configuration Tips

Check the release notes!

Set DiskRetryUnitAttention to 1 if connecting to IBM FAStT

On FAStT storage, unit attention signals can be noncritical

Some HBAs require further configuration

Example: QLogic cards:
If IBM storage: Set Port Down Retry Count to 8 If clustering: Set Enable Target Reset to Yes Set Full LIP Login to Yes Set Full LIP Reset to No

12
12

For ESX Server 2.0.1 2003-11-05 Copyright 2003 VMware, Inc. All rights reserved.

Making SAN storage available to VMs

Scan for new LUNs with MUI or command line
vmkfstools s vmhba0 No need to reboot or shut down VMs When rescanning QLogic adapters from command line, you must first clear the cards cache of LUNs echo scsi-qlascan > /proc/scsi/qla2?00/number
Do not rescan SCSI adapters

Allocate Fibre Channel HBAs only to VMkernel

To administer, use VMware Management Interface or vmkfstools

13
13

For ESX Server 2.0.1 2003-11-05 Copyright 2003 VMware, Inc. All rights reserved.

ESX Server 2.0.1 and Multipathing

Multipathing allows a SAN LUN to continue to be available in the event of a hardware failure
Manual load-balancing support in ESX 2.0: administrator may set preferred paths for each LUN

ESX Server supports failover with any supported HBAs

Failover occurs automatically, with a 30-60 second delay

14
14

For ESX Server 2.0.1 2003-11-05 Copyright 2003 VMware, Inc. All rights reserved.

Setting Preferred Paths

View list of paths to storage # cat /proc/vmware/scsi/vmhba0/0:1 [] Active Paths:fixed vmhba0:0:1 on*# Preferred vmhba1:0:1 on Set your desired policy echo "preferred vmhba1:0:1" > 0:1 echo "policy mru" > 0:1 Save with vmkmultipath -S
15
15
For ESX Server 2.0.1 2003-11-05 Copyright 2003 VMware, Inc. All rights reserved.

Questions?

ESX Server System Management II Module 6