IBM PureFlex Implementation Best Practices

2011
IBM Power Systems Technical University

October 10-14 | Fontainebleau Miami Beach | Miami, FL
IBM
Best Practices for PureFlex

systems implementation
pPS149
Arthur Torok, Luca Comparini
Copyright IBM Corporation 2013
Materials may not be reproduced in whole or in part without the prior written permission of IBM.
5.3
Agenda Logical view

IBM Power Systems Technical University Athens 2013
Project
Project Planning
Planning
Architecture / Project Management

BaU configuration FSM not mandatory
Network
Network Integration
Integration
PureFlex
PureFlex PowerVM
PowerVM Tuning
Tuning
SAN
SAN Integration
Integration
Initialization
Initialization
PureFlex
PureFlex upgrade
upgrade
FCoE
FCoE
FSM systems management
FSM
FSM Discovery
Discovery and
and Inventory
Inventory
Automation
Automation Manager
Manager
SMIA
SMIA Discover
Discover Fabrics
Fabrics
Private Cloud
Using
Using VMControl
VMControl
Analytics on our past engagements: talk the walk

42 customers
77 chassis
43% has > 1 chassis

20% has IBMi
70% has AIX
99% has x86 also
Most represented sectors:

23% Banking
23% IT Outsourcing (MSP)
17% GB
2011

IBM
High Level Planning

project planning
5.3
Before we start understanding Flex vs PureFlex

Project Planning
Project Planning
Ensure to do a project Kick-Off
Plan the right skill around the table
Don't assume that attendants already know; take the time to explain high level concepts
Define success criteria
Define time planning
When project is complex, better to take the time to produce a technical architecture document
To document expectations
To get stakeholders acceptance
Define responsibilities (RACI matrix)
Skill planning - Education

2011

IBM
Network Planning

5.3
Network planning is all about choosing...

Understanding the chassis internal topology

10
Understanding I/O - Feature on Demand

Mezzanine Card
I/O Module 1
I/O Module 2
2 mezzanine per node

16 lanes per node
14 nodes
224 total lanes
11
Understanding pnic vs vnic (x86)

With 10Gb networking, each lane could be used as:

- 1 physical nic
- up to 4 virtual nics (with dynamic bandwidth allocation and QoS)
Mezzanine Card
10Gb
pnic
1Gb 2.5Gb 2.5Gb

vnic vnic
vnic
4Gb
vnic
224 physical lanes
Up to 896 virtual lanes
12
Looped and blocking design, no host NIC teaming

Each I/O module in the Flex System Chassis is shown with two direct aggregations to a pair of
upstream Top-of-Rack (ToR) switches
This topology is suggested for environments in which hosts need network redundancy, but they are
not themselves performing any NIC teaming.
Offers complete network-level redundancy out of the chassis but if not balanced correctly, you lose
half of the available links and bandwidth due to Spanning Tree Protocol (STP) blocking them
13
Non-looped, single upstream, host NIC teaming

Each I/O module in the Flex System Chassis connects directly to a single ToR switch through single or
aggregated links.
Topology is highly suggested when compute nodes use some form of NIC teaming.
This architecture with NIC teaming on the host, and Layer 2 Failover on the I/O modules, provides high
availability with no loops
No wasted bandwidth to spanning-tree blocked links.
14
Trunk failover in action

Layer 2 failover is enabled on each Flex I/O Module
15
Fully redundant network configuration

Features
Uses aggregated links or port-channels.
Provides loop-free network design
Recommended for environments that wish

to leverage a best-practice implementation
utilizing next generation networking
features such as VSS (Virtual Switching
System) or vPC (Virtual Port Channel)
technology.
Advantages of this approach are:
Active/Active uplinks helps to avoid the

wasted bandwidth associated with links
blocked by spanning tree
Maximum redundancy and fault tolerance

Disadvantages of this approach are:
Requires upstream equipment that

supports virtualization features
More cabling and connections are

necessary
Careful implementation and planning to

ensure correct operation by a network
architect / administrator
16
Double wide Power node

17
Double wide Power slot assignment example

VIOS A: Slot 1: C34

Port 1: L1-T1
Port 2: L1-T2
VIOS B: Slot 1: C34
Port 3: L2-T1
Port 4: L2-T2
VIOS B: Slot 3: C36

Port 1: L1-T1
Port 2: L1-T2
VIOS A: Slot 3: C36
Port 3: L2-T1
Port 4: L2-T2
http://www.redbooks.ibm.com/technotes/tips0880.pdf
18
Double wide Power Without ScSE Upgrade 1

VIOS1
S1_INT5A
S1_EXT1
en0 S1_INT5A
en1 S2_INT5A
S1_EXT2
P7-460 SLOT 5-6

P1-C34-L1-T1
-T2
S1_INT6A
S1_EXT3
S1_EXT4
VIOS2
en2 S1_INT6A
en3 S2_INT6A
P1-C36-L1-T1
-T2
SWITCH 1
S2_INT5A
S2_EXT1
S2_EXT2
S2_INT6A
S2_EXT3
S2_EXT4
SWITCH 2
19
Double wide Power With ScSE Upgrade 1

VIOS1
S1_EXT1
S1_EXT2
S1_EXT3
S1_INT5A
en0 S1_INT5A
S1_INT5B
en1 S2_INT5A
S1_INT6A
en2 S1_INT6B
S1_INT6B
S1_EXT4
en3 S2_INT6B
VIOS2
en0 S1_INT5B
en1 S2_INT5B
en2 S1_INT6A
en3 S2_INT6A
P7-460 SLOT 5-6

P1-C34-L1-T1
-T2
P1-C36-L2-T1
-T2
P1-C34-L2-T1
-T2
P1-C36-L1-T1
-T2
SWITCH 1
S2_INT5A
S2_EXT1
S2_INT5B
S2_EXT2
S2_INT6A
S2_EXT3
S2_INT6B
S2_EXT4
SWITCH 2
20
Double wide Power Without ScSE Upgrade 1

P7-460 SLOT 5-6
VIOS1
S1_INT5A
S1_EXT1
P1-C34-L1-T1
-T2
en0 S1_INT5A
en1 S2_INT5A
S1_EXT2
S1_EXT3
S1_EXT4
en4 =lnagg en0 + en1

MGT
DEV
SWITCH 1
PVID=1
S2_INT5A
S2_EXT1
SEA
S2_EXT2
S2_EXT3
S2_EXT4
en4
SWITCH 2
Switch 1
Switch 2
Backbone
Backbone
21
Double wide Power With ScSE Upgrade 1

P7-460 SLOT 5-6
VIOS1
S1_INT5A
S1_EXT1
en0 S1_INT5A
en1 S2_INT5A
S1_EXT2
S1_EXT3
en2 S1_INT6B
S1_INT6B
S1_EXT4
en3 S2_INT6B
P1-C34-L1-T1
-T2
P1-C36-L2-T1
-T2

MGT
DEV
PVID=1
PVID=2
SEA
SEA
en4
en5
SWITCH 1
SWITCH 2
Backbone
Backbone
SWITCH 1
S2_INT5A
S2_EXT1
S2_EXT2
S2_EXT3
S2_EXT4
S2_INT6B
SWITCH 2
22
Network planning questions

Who is the current installed switch vendor?

What upstream networking devices will Flex switches be connected to?
Do you have any previous experience with IBM/BNT networking equipment?
What switch CLI does the customer prefer to use? IBM/BNT or Cisco?
How many uplinks and what cables are to be used (RJ45, SFP+, QSFP, DAC)?
Network Switch uplinks (x2 recommended per switch minimum)
Use of shared uplinks reduces the number of uplink cables
Will ScSEs be used as Switches or Pass-thru ScSEs?
Number of physical management network connections
CMM, FSM, External Management Port on ScSE, Storwize V7000
Number of IPs required
23
Network planning questions

What are the performance characteristics of the Data network in use?

VLANs, load balancing, availability
Will vNICs be implemented? Virtual Fabric, Switch independent & Universal Fabric port modes
Each vNIC pair will need to be on its own separate VLAN
Assign VLAN designation at the OS level (Windows & VMWare)
Is FCoE / iSCSI being used?
Feature must be activated on the 2 port LOM or 4 port CN4054 mezzanine adapter
FoD key must be applied in the IMM, OneCommand Manager or FSM
Are there enough Switch Partitions? Switch partition upgrades determine internal and external
usable switch ports.
NIC Teaming for high availability and/or bandwidth aggregation.
24
2011

IBM
SAN Planning

5.3
Fiber Channel options for connectivity

Pass-Thru NPIV Mode (FC5022 or FC3171)

Use when you already have an existing FC fabric in your environment and dont want
additional switches added to that fabric
Downfall You cannot directly connect devices to the chassis like a tape library
Full-Fabric Mode (FC5022 or FC3171)

Full Fiber Channel switch that you have to zone and add to you fabric.
You are able to directly connect devices like a tape library
Can add complexity as another switch is added to the fabric
Breakdown of FC Switch Vendors
FC5022 Brocade (16GB)
FC3171 Qlogic (8GB)
26
How to set NPIV mode (Qlogic)

27
Fiber Channel port types & NPIV

N_Port, node port.

A host, HBA, or storage device port connected to the F_Port of the switch.
F_Port, fabric port.
A switch port that connect a host, HBA, or storage device to the SAN.
E_Port, known as Inter Switch Link (ISL).
A switch port that connect the switch to another switch directly.
N_Port ID Virtualization (NPIV) is a Fibre Channel protocol that facilitates sharing a single
physical N_Port among multiple N_Port IDs.
Virtualization refers to the ability of a single physical N_Port, thereby allowing multiple,
distinguishable entities on the same physical ports. In other words, it makes a single
Fibre Channel port appear as multiple virtual ports, each having its own N-Port ID and
virtual worldwide name (WWN).
The NPIV protocol requires an N_Port (typically a host bus adapter any device that acts as
an NPIV gateway) and a fabric (generally an FC switch) so that the N_Port can request
and acquired multiple addresses from the fabric.
28
Dual VIOS vtSCSI without TOR

CLIENT
LPAR
hdisk0
mpio
hdisk0
hdisk0
vscsi0
vscsi0
VIOS A
mpio/sddpcm
VIOS B
vhost0
fcs0
fcs1
SWITCH 3
vhost0
fcs0
fcs1
SWITCH 4
V7000
node1
V7000
node2
29
Dual VIOS NPIV without TOR

CLIENT
LPAR
hdisk0
mpio/sddpcm
VIOS A
fcs0
fcs1
vfchost0
vfchost1
fcs0
fcs1
SWITCH 3
VIOS B
fcs2
fcs3
vfchost0
vfchost1
fcs0
fcs1
SWITCH 4
V7000
node1
V7000
node2
30
Dual VIOS NPIV with TOR (ex: with Passthrough)

CLIENT
LPAR
hdisk0
mpio/sddpcm
VIOS A
fcs0
fcs1
vfchost0
vfchost1
fcs0
fcs1
VIOS B
SWITCH /
PASST 3
fcs2
fcs3
vfchost0
vfchost1
fcs0
fcs1
SWITCH /
PASST 4
TOR
TOR
V7000
node1
V7000
node2
31
Zoning considerations
vtSCSI
Zone VIOS with Storage
NPIV
Zone LPAR virtual WWPN with Storage
If you need LPM, ensure to zone both WWPN
lssyscfg -r prof -m Server-7895-42X-xxxxxx -F virtual_fc_adapters --filter
lpar_names=AIX
"""10/client/1/VIOS1/10/c050760586f60000,c050760586f60001/1"",
""11/client/2/VIOS2/11/c050760586f60002,c050760586f60003/1"",
""12/client/1/VIOS1/12/c050760586f60004,c050760586f60005/1"",
""13/client/2/VIOS2/13/c050760586f60006,c050760586f60007/1"""
Number of paths
Keep a reasonable number of paths (4 or 8 ok; 16 max)
32
2011

IBM
What to do to start

5.3
Configure the chassis components

34
Login into CMM to start initialization wizard

CMM IP address
192.168.70.100 if Flex
192.168.93.100 if PureFlex
35
Set Component IP address

36
Systems Management
FSM
Network
SAN
Storage
Power nodes
PowerVM
(VIOS)
Intel nodes
IBMi
AIX
Chassis
(CMM)
Linux
KVM
VMWare VCenter
Win / Hyper-V
37
Understanding FSM networks

Hardware
Management
Network (GOLD)
eth0
eth1
FSM code
Active
CMM
I/O Module
1
KVM host
bond
Stand-by
CMM
bond
I/O Module
2
nodes
nodes
nodes
IMM /
FSP
I/O
nodes
nodes
Modules
INT-MGT
Customer Data
Network
(PURPLE / GREEN)
38
Initialize FSM management network

39
Initialize FSM management network

Eth0 - Private Management Network (In-band)
Private and secure Gigabit Ethernet network.
Used to complete management-related functions throughout the chassis including:

Compute nodes
Switches
Chassis
Flex V7000 & Storwize V7000 Storage
Connects the Chassis Management Module (CMM), the IMM/FSP of the compute nodes,
switches in the I/O bays, the Flex V7000 storage and the Flex System Manager (FSM)
Uses a special Broadcom 5718-based management network adapter
Eth1 - Data Network (Out of band)
FSM bonded pair of NIC ports connecting to ports 1 of both switch 1 and switch 2
Connects to the Internet for product updates and other related management functions
Note: The Eth0 or CMM management networks in multiple chassis are connected together
through the external ports of the CMMs in each chassis by using a GbE top-of-rack switch.
40
Select Chassis to be Managed

41
2011

IBM
What to do with the FSM

(discover, access, inventory,
manageV7000)
5.3
FSM Best Practices step by step

After the Chassis has been managed it usually takes around 15 minutes for the inventory to be
collected,meaning the FSM's database will be updated with all the information regarding the
hardware elements present in the managed chassis. It's a good idea to wait for this process to
finish, if you have issues with patience (like I do) go to the Plugins tab and click the
active&scheduled jobs link, you should see the jobs running.
43

Usually access to the components should show OK, but just to make sure, have a look at the
Plugins tab, you should see immediately under the Inventory link if there are any systems with
No Access or Partial Access, in case there are make sure you request access to avoid any
future inventory collection or update issues.
44

If using V7000 storage you should discover&manage it from the FSM. If you have an internal Flex
System V7000 system you can use the Launch IBM Flex System V7000 EZSetup task and follow
the wizard, alternatively you can use the quick link, on the Additional Setup tab, Manage System
Storage. This will give you several possibilities to manage and monitor your storage subsystem, get
an overview of the disk usage, pools, hardware alerts, create and delete volumes.
45

46

First you might have to generate a public and private pair of rsa keys
ssh-keygen -t rsa -f $HOME/v7000_rsa, then using browse select the private key.
After the task is complete inventory will automatically be collected.
47

As inventory is a central component of great importance from the FSM's usability perspective I
recommend to do a complete collection after all the hardware has been discovered/managed,
my preference is the command line from a terminal/putty ssh session: smcli collectinv -p All
Inventory. Inventory collection is task that highly utilizes the database so I wouldn't
recommend doing too many things during this process.
48
2011

IBM
Upgrade best practices

5.3
Updating Best Practices order of operations

There are a number of methods to update the component firmware of your Flex System. Below
is the recommended order of operations for performing these depending on the hardware you
have.
For the latest information on firmware updating best practices, please see the Firmware Update
Guide at the link below.
https://www-947.ibm.com/support/entry/myportal/docdisplay?lndocid=MIGR-5091991
Chassis without a FSM :

CMM
X-Architecture compute nodes
Flex V7000
I/O Modules
Chassis with a FSM :

Flex System Manager Node
FSP on Power System Nodes
CMM
Adapters / Hard Drives on Power System Nodes
X-Architecture compute nodes
Flex V7000
I/O Modules
50
Updating Best Practices order of operations

Obtaining Firmware
In addition to updating the firmware on the nodes, you should also update the CMM and
all the switch modules you have in the chassis. To make it easy, IBM has a single page
where you can easily access the latest firmware for all of the components.
http://www-01.ibm.com/software/brandcatalog/puresystems/centre/details?uid=S_PUREFLEX
Log in locally to each component and update the corresponding firmware.
51
Updating CMM
To update the Chassis Management Module (CMM), log in to the CMM, click on Mgt Module
Management, and select Firmware.
If you have two CMMs, you only need to update the firmware on your primary CMM. Once
the update process is complete, and the CMM has rebooted, it will automatically update the
secondary CMM.
When you download the firmware package, you will need to extract the zip file. The update
process will require the file with the .uxp extension within that package.
52
Updating x86 nodes

Update Servers via Bootable Media

Creator
Download IBM Bootable Media Creator from the
link below :
http://www-947.ibm.com/support/entry/portal/docdisplay?
lndocid=TOOL-BOMC iSCSI
Once the download is complete, right click on the

file and choose run as administrator.
Redhat and SLES version of the tool are also
available and will run in a similar format.
53
Updating FC3171 Switch

To update the Qlogic FC3171 fiber channel switch, log in to (default user/pass is
USERID/PASSW0RD) and click on Switch > Load Firmware. Select the folder where the .bin
file firmware is located. You do not need to select the exact file. The switch will automatically
access the firmware file in that directory.
54
Updating EN4093 Switch

The networking switch has two components that need to be updated. The Boot code and the
Image code. Both the boot and image code are included in the single download package.
Typically we recommend updating the boot code first and then one of the image banks. It is
recommend that you keep one level back in one of the image banks.
Updating the firmware is as easy as clicking the browse button and then selecting the firmware
package.
55
Updating via FSM

If you have the Flex System Manager, updating chassis components is a easy process by
selecting the Update Chassis Components option and following the order provided. We
always recommend checking the Firmware Update Guide for the latest information and any
changes to the order listed in the Flex System Manager.
56
2011

IBM
IBM SMIA step by step

5.3

To fully utilize the advanced features and capabilities of the FSM, like automatic workload
deployments, you should be able to read and write zoning information on your SAN
infrastructure.
Since most storage vendors developed their own language a need for commonality arose. The
Storage Networking Industry Association developed a standard, Storage Management
Initiative Specification, shortly SMI-S, to enable interoperable management of storage
systems.
If you have a Qlogic switch in full fabric mode all you need to do is discover the switch and
configure SNMP access. Qlogic switches have an SMI-S provider built in their firmware that
can be used to discover the zoning information. However due to current limitations this access
is read only, meaning you will not be able to deploy NPIV based workloads as the FSM can
not create new zones.
Brocade switches do not have this built in component thus an external tool, SMI-S provider is
required to be able to manage a Brocade based SAN infrastructure from the FSM. IBM
decided to preinstall this java based tool in the FSM, you have to to start and configure it as
shown in the next slide:
58

From the applications tab click Start under SMIA Configuration Tool:
59

Tip: Make sure you have a dns entry or you have the name in your local host file, credentials are
the same as for the FSM.
60

Click Options and configure https authentication:
61

After changing to https Stop and Start form the Applications tab:
62

Do a Fabric discovery:
63

Click add, put the IP address of the Brocade switch and USERID/PASSW0RD by default:
64

After this you have to use the Discover Storage System task as with the V7000 but this time
select FC switch and provide the details:
65

The SMI-S IP will be the FSM itself:
66

After successfully configuring the IBM Fabric Manager, go to the Plugins tab, click on Resource
Explorer and click on the All Systems dynamic group. Type Farm in the search box, this is a
container object that represents all storage components known to the FSM, like the V7000
and SAN switches. Right click this object and select Collect Inventory. This should update the
FSM's database with all the zoning and SNMP information from the fabric switches, and also
with the information from the V7000. Since the fabric switches are now known to the FSM any
alerts generated by these will be received and displayed in the FSM.
67
2011

IBM
Tips for PowerVM

(tuning)
5.3
Tips for PowerVM

If you have Power nodes in your Chassis I would assume you will also have Virtual I/O Servers,
unless you want to use the Power nodes in full system partition mode, in which case more
specific tuning will have to be considered.
For quite some time now VIOS and AIX have the so called Common Agent installed by default.
Though it's not mandatory for basic management, I highly recommend making use of it's
features, especially for more advanced functions, like automating deployments of Virtual
Servers. To do this, after you have successfully installed and configured your VIOS, run a
discovery on the IP address of the VIOS, authenticate (request access), then collect inventory.
This will give the FSM access and visibility to the Operating System component of the Power
node, specifically of the VIOS. The procedure is the same with AIX systems.
69
Tips for PowerVM

Tips for creating the Virtual Server profile for the VIOS:
- use 1 or 2 dedicated processors in donating mode, this way you ensure the best locality and
availability of CPU and memory resources and also that the idle cycles can be used by the
other partitions. If you plan to have high network traffic without jumbo frames and vSCSI
storage you should have at least 2 CPU's, if you use jumbo frames and NPIV 1 CPU should
be enough. Memory should be between 6-8Gb to start and 12Gb maximum in case you have
to increase it later on the fly. Physical I/O can be split between 2 VIO servers depending on
the physical hardware capabilities, in a p460 for example you would typically have 4
mezzanine adapters, thus you can split 2-2 for creating a dual VIOS configuration. The
network adapters have 2 asics, thus even one adapter can be split in two, the 2 port fibre
channel adapters have one while the 4 port adapters have 2 asiscs. In the new p270 node
there is also a special adapter for splitting the on board SAS controller into 2 in which case
you could potentially install 2 separate VIO servers on local disks. This is not possible with
any other nodes as they only have 1 SAS controller.
70
Tips for PowerVM

Networking tips:
For high availability and redundancy reasons I suggest to consider creating etherchannel network
adapters on the VIOS. As LACP is natively supported this is what I prefer to use. It does imply
you will need VLAG and will have to create the LACP port groups on the chassis network
switches. I usually put INTAx on IOM1 with INTAx on IOM2 in one LACP group and INTBx
with INTBx in a different LACP group. If it's a full wide node with Dual VIOS I would use INTAy
with INTAy and INTBy with INTBy on the second VIOS. This configuration will give you
2x20Gbs adapters per VIOS. Then you have to create the etherchannel adapter making sure
you select type 802.3ad, and the en adapters corresponding to the ports you configured
previously. In most cases they would be en0 with en1 and en2 with en3 but it's best to double
check the location codes first.
Jumbo frames are very useful in reducing processing overhead and increasing throughput though
it should be used extensively throughout the infrastructure. Good news is the PureSystems
network switches all support this by default. Fine tuning for jumbo frames is quite an extensive
topic that usually has it's own session, please check the agenda for more information.
71
Tips for PowerVM

Next step is to create the SEA adapters. If you haven't done so while creating the virtual server
definition for the VIOS you should now create the required virtual network adapters,
depending on how many etherchannel adapters you had created in the previous step.I
suggest to have one virtual adapter for the management IP of the VIOS, then 1 or 2 for each
Shared Ethernet Adapter. If you are using Dual VIOS you will need at least 2 adapters/SEA,
one will be the trunk adapter, this one needs to have access to the external network, the
other one will be the control channel (heartbeat) adapter used for communication between the
2 VIO servers. The trunk adapter can be IEEE 802.1Q compatible and it can manage up to 20
VLANs. If you need to serve more VLAN's you will have to create additional adapters for each
bundle of 20.
This is an example command for creating the SEA:
mkvdev -sea ent9 -vadapter ent4 -default ent4 -defaultid 1 -attr
ha_mode=auto ctl_chan=ent5 largesend=1 large_receive=yes
Ent9 is the physical adapter, in our case the new etherchannel adapter, ent4 is the virtual trunk
adapter, the one that has the option to access external networks and ent5 is the control
channel adapter. Note that both ctl_chan virtual adapters, on VIOS1 and VIOS2 should have
the same unique PVID. If you plan to use jumbo frames, which I highly recommend also add
this parameter to the command: jumbo_frames=yes
72
Tips for PowerVM

Useful parameters:
For fibre channel adapters you should use dynamic tracking of devices and fast fail error
recovery, this will ensure that any missing paths will be noticed immediately and the multipath
I/O driver will initiate a failover and prevent loss of data. It is also recommended to install the
sdd-pcm driver that builds on the default mpio and adds extra functionality and usability.
chdev -dev fscsi0 -attr dyntrk=yes fc_err_recov=fast_fail -perm
Since the adapters are usually active these changes cannot be done on-line thus the -P
parameter which also implies you will have to reboot the VIOS to activate the changes.
For specific applications you might have to change additional parameters.
For vScsi disks in a dual VIOS setup you have to make sure to change the reservation policy
before assigning the disks to a partition:
chdev -dev hdisk# -attr reserve_policy=no_reserve
73
2011

IBM
Tips for VMControl

5.3
Tips for VMControl

VMControl is the first step into a Power systems based Cloud environment on the FlexSystem
platform, it provides the functionality to capture existing workloads and automatically deploy
new systems. Deployment includes the creation of the Virtual Server, creation of required
virtual network adapters, creation of required disks and their corresponding virtual adapters, all
the mappings plus the OS installation.
Given everything has been set-up correctly this process will complete under 5 minutes after a few
easy steps where you are required to provide the name, future IP address, hostname, dns
servers, etc.
The reason for being so quick is because it's based on flashcopy technology, either when using
V7000 for direct LUN allocation or VIOS Shared Storage Pools based virtual disks.
75
Tips for VMControl

For this to work there a few key factors that have to be considered:
- the VIOS Common Agent has to be discovered and access has to be OK.
- the correct repository subagent has to be installed on the VIOS, and if using Shared Storage
Pools there is an additional subagent that you will have to install. This will ensure that you can
create an image repository to store all the metadata required to successfully catalog and
manage OS images that you will use in the future to deploy new workloads.
- the SAN switches and the V7000 have to be discovered and successfully accessed
- the system that you will capture has to have the Common Agent discovered and has to have the
activation engine code installed and the activation engine has to be configured. The
activation engine code resides on the FSM and can be copied using scp after which it has ot
be installed on the host using specific java code:
opt/ibm/director/proddata/activation-engine/vmc.vsae.tar
root@1.1.1.1:/tmp/
export JAVA_HOME=/usr/java5/jre
untar and execute the AE install script
After installing the activation engine and before trying to capture make sure you run an inventory
collection on the FSM, Farm, VIOS, host system.
76
Tips for VMControl

After you customize the OS that you will capture you have to run the AE.sh script:
/opt/ibm/ae/AE.sh --reset
This will prepare the OS for capture and will also shut down the virtual server. After the system is
shut down you can initiate the capture by right clicking the Virtual Server element in resource
explorer or by following the VMControl links.
In case you want to make some changes and capture the server again you have to reset the
activation engine by issuing these commands:
rm /opt/ibm/ae/AP/*
cp /opt/ibm/ae/AS/vmc-network-restore/resetenv /opt/ibm/ae/AP/ovfenv.xml
If you have done everything correctly, which I know you did, you will now have your customized
image ready to deploy. Please note that the CPU, Memory, Network, Storage information is all
captured form the original image, all new deployments will use this information as default,
which of course you can change later.
77
2011

IBM
Tips for automating actions

(EAP, ESA, alerting...)
5.3
Tips for automation

The FSM comes with a very useful feature called Electronic Service Agent, also known as call
home. I highly recommend enabling this, it will automatically create a service call with IBM
Support in case of any hardware failures, it will also collect and upload the logs.You do need to
have an active support agreement contract and you have to provide the customer number. It's
also a good idea to check on the IBM Support website that the correct product is registered.
79
Tips for automation

Part of the advanced RAS features of the FSM include predictive hardware alerting which can be
enabled by creating an Event Automation Plan that will filter out from the event log the specific
alerts and will log them in the error log. This plan can also include an action to send and email
or integrate into IBM Tivoli Monitoring, for example.
80
Tips for automation

Go to the Plugins tab, Automation Manager, Event Automation Plans, create new:
81
Tips for automation

Select All Systems as target, this will ensure that all predictive hardware alerts are captured from
all systems:
82
Tips for automation

Change to Advanced Event Filters and select Hardware Predictive Failure Alerts:
83
Tips for automation

Next step is to select what action to do in case an alert is raised. If you don't see an appropriate
action in the list click create:
84
Tips for automation

Select from the list send email, this way the recipient will be informed:
85
Tips for automation

Note you can only use one address for action, and you can customize what to include in the
message using variables:
86
Tips for automation

Next step is to finish the creation and activate the plan. Since this is related to predictive
hardware alerts it will be active 24/7, with other plans you have the flexibility to chose the
timeframe. For example if your service agreement is only from 9 to 5 you on weekdays you
can set the plan to be active only during this timeframe.
Another useful feature is monitoring, which can include filesystem usage, CPU, memory, process
monitors etc.
For each monitor you can define a threshold with a severity and you can create automation plans
for these as well.
87
2011

IBM
FCoE

5.3
FCoE
Support for FCoE in Power nodes comes with the new 8 port, dual asic Converged Ethernet
Adapter and the CN4093 10Gb converged switch. Due to current limitations of the available
CN4093 switch only 6 of these can actually be used and if you plan to split the adapter for 2
VIO servers you will only be able to use 2 ports for each.
When used in converged mode each physical port is split into 2 providing Fibre Channel and
Ethernet traffic.
The CN4093 converged switch can act as the primary SAN director when running in full fabric
mode or as an NPV gateway to provide pass-through traffic to a top of rack SAN director. It
does use a Qlogic fabric thus full fabric integration with Brocade is not possible, in this case I
recommend using NPV. It has 12 omniports that can be configured in pairs only, and you can
attach a SAN SFP+ for direct connection of storage systems or SAN directors.
89
FCoE
Useful commands to configure Converged Enhanced Ethernet on the CN4093:

#cee enable: this will enable FCoE forwarding
#fcoe fips enable: this enables snooping for FCoE frames
To change an omniport for fibre channel (note it has to be done in pairs)
#system port EXT6-EXT7 type fc
All internal ports where you want to use FCoE must have tagging enabled:
#interface port INTA2-INTA14
#tagging
90
FCoE
You have to define a new vlan for FCoE traffic only, the accepted industry default is 1002, and
configure it accordingly:
#vlan 1002
#member INTA2-INTA14,EXT6-EXT7
You have to include all host ports and the omniports configured as FC in this vlan. Then if you
plan to use the switch in pass through (NPV) mode:
#npv enable
#npv traffic-map external-interface EXT6-EXT7 (providing the FC ports that will be connected
to your SAN Director switch)
Note that it is not supported to transport vlan 1002 through an ISL link.
91
FCoE
For full fabric management, the FCF mode has to be used:

#vlan 1002
#fcf enable
Then you have to create the aliases, zones and zoneset:
#fcalias V7K_N1_P1 wwn 50:05:07:68:02:35:da:f5
#fcalias Node5_2_P1 wwn 10:00:34:40:b5:ef:d0:19
#zone name Node5_2_v7k_F1
#member fcalias V7K_N1_P1
#member fcalias node5_2_P1
#zoneset name Fabric1
#member Node5_2_v7k_F1
#zoneset activate name Fabric1
92
FCoE
After doing the configuration I recommend to save it:

#write
Then do a reload.
To check the status:
#show fcoe database
#show fcoe fips fcoe
#show npv flogi-table
93
FcoE support using FC targets

Flex System Interoperability Guide:

http://www.redbooks.ibm.com/abstracts/redpfsig.html
94
FCoE support using FCoE targets (end to end)

Hardware based iSCSI support
95
ibmtechu.com/gr
Win prizes by
submitting
evaluations online.
The more evalutions
submitted, the
greater chance of
winning
96
Thank you for attending this session

Dont miss these valuable opportunities:
Keep the momentum going
We value your feedback
ibm.com/training
Now is the time to explore your options for
additional training. One stop shopping for all
your technical training needs.
1.
Submit Session Evaluations on

ibmtechu.com and receive an entry to
WIN prizes. The more evaluations
submitted, the greater your chance of
winning!
1.
Submit your Overall Session

Evaluation and receive a free gift at
the registration desk.
ibm.com/training/trainingpaths
IBM Training Paths These flowcharts map
out the sequence of classes you need, to
obtain a specific skill or professional
certification. Get started today!
ibm.com/certify
Stand out from the crowd when you earn a
valuable credential of an IBM Certified
Professional. Find certifications by product or
solution.
97

IBM PureFlex Implementation Best Practices

Diunggah oleh

Informasi Dokumen

Deskripsi Asli:

Hak Cipta

Format Tersedia

Bagikan dokumen Ini

Bagikan atau Tanam Dokumen

Opsi Berbagi

Apakah menurut Anda dokumen ini bermanfaat?

Apakah konten ini tidak pantas?

Hak Cipta:

Format Tersedia

IBM PureFlex Implementation Best Practices

Diunggah oleh

Hak Cipta:

Format Tersedia

2011

IBM Power Systems Technical University

Best Practices for PureFlex

Agenda Logical view

Architecture / Project Management

FSM systems management

Copyright IBM Corporation 2013

Analytics on our past engagements: talk the walk

43% has > 1 chassis

Most represented sectors:

Copyright IBM Corporation 2013

IBM Power Systems Technical University

High Level Planning

Before we start understanding Flex vs PureFlex

Copyright IBM Corporation 2013

Copyright IBM Corporation 2013

Skill planning - Education

Copyright IBM Corporation 2013

IBM Power Systems Technical University

Copyright IBM Corporation 2013

Network planning is all about choosing...

Copyright IBM Corporation 2013

Understanding the chassis internal topology

Copyright IBM Corporation 2013

Understanding I/O - Feature on Demand

Copyright IBM Corporation 2013

2 mezzanine per node

Understanding pnic vs vnic (x86)

With 10Gb networking, each lane could be used as:

1Gb 2.5Gb 2.5Gb

Copyright IBM Corporation 2013

Looped and blocking design, no host NIC teaming

Copyright IBM Corporation 2013

Non-looped, single upstream, host NIC teaming

Copyright IBM Corporation 2013

Trunk failover in action

Layer 2 failover is enabled on each Flex I/O Module

Copyright IBM Corporation 2013

Fully redundant network configuration

Uses aggregated links or port-channels.

Provides loop-free network design

Recommended for environments that wish

Active/Active uplinks helps to avoid the

Maximum redundancy and fault tolerance

Requires upstream equipment that

More cabling and connections are

Careful implementation and planning to

Copyright IBM Corporation 2013

Double wide Power node

Copyright IBM Corporation 2013

Double wide Power slot assignment example

VIOS A: Slot 1: C34

VIOS B: Slot 3: C36

Double wide Power Without ScSE Upgrade 1

P7-460 SLOT 5-6

Copyright IBM Corporation 2013

Double wide Power With ScSE Upgrade 1

P7-460 SLOT 5-6

Copyright IBM Corporation 2013

Double wide Power Without ScSE Upgrade 1

en4 =lnagg en0 + en1

Copyright IBM Corporation 2013

Double wide Power With ScSE Upgrade 1