Session Title:
Trademarks
The following are trademarks of the International Business Machines Corporation in the United States, other countries, or both.
Not all common law marks used by IBM are listed on this page. Failure of a mark to appear does not mean that IBM does not use the mark nor does it mean that the product is not actively marketed or is not significant within its relevant market. Those trademarks followed by are registered trademarks of IBM in the United States; all others are trademarks or common law marks of IBM in the United States.
Agenda
A walk through Live Partition Mobility Application Migration using Live Partition Mobility Migrating Logical Partitions between POWER7 and POWER6 Systems Live Partition Mobility with N_Port ID Virtualization (NPIV)
Basic Requirements
Migrating partition must have fully virtualized resources (i.e no dedicated physical adapters) All systems in a migration domain must be connected on a Storage Area network (SAN) to shared physical disks No internal or VIOS-based disks SAN disks must be configured to be visible from both systems HMC(s) requires network connectivity to: VIO Servers providing virtual SCSI for the migrating partition VIO Servers providing virtual networking for the migrating partition VIO Servers providing Mover Service Partition (MSP) function Migrating partitions Both MSPs must have network connectivity to each other Mobile partition must be connected to same subnet after migration
HMC CLI: lssyscfg -r lpar -m "Ops-Rack92-MMA-SN100F6B0" filter=name=zlab040-VIOS -F \ name zlab040-VIOS msp 0 time_ref 0
lshwres -r io --rsubtype slot -m "Ops-Rack92-MMA-SN100F6B0" --filter "lpar_names=zlab046" -F "lpar_name drc_index drc_name description" zlab046 21010222 U789D.001.DQDXYKM-P1-C5 "PCI 10/100/1000Mbps Ethernet UTP 2-port"
NOTE: Though most partitions will pass the following checks, all the same for active migration go through the partitions properties and ensure: - That the mobile partition is not the redundant error path reporting logical partition - That the mobile partition is not configured with barrier synchronization registers (BSR) - That the mobile partition is not configured with huge pages
11 2010 IBM Corporation
Now use the mkauthkeys command to generate security key to authenticate one MSP with the other. mkauthkeys ip 9.3.131.144 -u hscroot -g Enter the password for user hscroot on the remote host 9.3.131.144:
Note: use the -g option so that authentication keys are set up in both directions (from source to destination and vice-versa)
12 2010 IBM Corporation
13
Source System Destination System Source Virtual I/O Server Destination Virtual I/O Server Mobile Partition New LPAR Creation
LPAR removal
Validation
Notification of completion
Time
14
HMC
POWER Hypervisor
POWER Hypervisor
Source system
Destination system
15
Destination System Source Virtual I/O Server Destination Virtual I/O Server Mobile Partition
Source System
System resource Availability check RMC connection check Virtual Adapter Mapping Partition Readiness Check OS and Apps Readiness check
Time
16
Validation
MSP Setup
Memory copy
Notification of completion
Time
17
1 POWER Hypervisor
3 POWER Hypervisor
Source system
Destination system
18
19
20
21
Validate will return a detailed error explaining the loss of redundancy If we continue with the migration it will result in an MPIO failed path Please note that redundancy is not restored when migrating back!
22
23
24
Bandwidth Testing
During migration validation the network bandwidth will be tested Results are logged in the configuration log alog t cfg o > cfglog MSP_MSP_BANDWIDTH=xxx,xxx is in Mb/s If bandwidth is less than 50 Mb/s xxx will be set to LOW Results are not 100% accurate, they are simply an idea as to what kind of bandwidth was available during the validation phase of migration. 64K-sized ping packets are used to test bandwidth Useful information if the migration takes a long time or fails
25
MSP_MSP_BANDWIDTH Equivalent MB/s value 1000 500 250 167 125 100 83 71 62 55 50 LOW 125.0 62.6 32.2 20.8 16.1 12.5 10.4 8.9 7.8 6.9 6.2 Less than 6.2
2010 IBM Corporation
Trial-By-Fire Session
Live Partition Mobility Between POWER7 and POWER6
27
Environment
1. POWER7- 8233-E8B, 12 CPUs and 64GB memory (firmware- AL710_065) 2. POWER6- 9117-MMA, 2 CPUs and 8GB memory (firmware- EM340_116) 3. HMC V7 Release 7.1.0, 7310CR4 4. Dual VIO Server each running 2.1.2.13-FP-22.1 SP02 with IZ70375 using NPIV, VIOS 2.1.3+ is highly recommended) 5. AIX 6.1 LPAR 6100-04-03-1009 6. AIX 5.3 LPAR 5300-11-03-1013 7. Storage-DS4K (1742-900) disks for both virtual SCSI and virtual FC 8. Logical memory block (LMB) size on both systems (32MB) 9. AIX native MPIO is used for multi-pathing 10. Ethernet adapters: HEA on POWER7 and 10/100/1000 Base-TX PCI-X adapters on POWER6 11. NPIV - 8Gb PCI Express Dual Port Adapters (df1000f114108a03) (If
28
Test scenarios
1. 2. 3. 4. 5. 6. 7. 8. 9. Test Inactive Partition Migration Test Active Partition Migration Test assigning more CPU capacity to the mobile partition than is available on destination system Test CPU compatibility modes POWER6/POWER6+/POWER7 on the moving LPAR Test parallel active migrations between POWER7 and POWER6 systems Test effect of the reserve_policy when it is set to single_path Test interaction of partition migration and Active Memory Sharing Simulate a crash of the mobile while migration is active While LPM is running crash the VIO server on the source MSP
10. Crash the HMC during partition a partition migration 11. NPIV virtual FC adapter load test 12. Create a mirrored VG and run syncvg during partition migration 13. NFS mount files between two LPARs and do a partition migration of the NFS server RESULT: LPM passed all the tests failing a migration only when it is expected (for example, test 3, with not enough memory on the destination system)
29 2010 IBM Corporation
Lessons Learned
When doing multiple migration in parallel it is recommend to use HMC level V7R7.2.0+ so you can specify IP addresses for source and destination MSP communication. When migrating between POWER6 and POWER7 system it is recommended to set processor compatibility mode to default so the HMC can set suitable mode on migration. Please check the reserve_policy on all MPIO disks and make sure it is set to no_reserve or SCSI-3 PR_shared (in which case use different keys for source and destination MSPs) For NPIV to work, make sure you are using recommended VIOS levels and SAN switch firmware is at the recommended level
30
Minimal change in response time except for close to the end of LPM when LPAR is switched to the new system. Notice workload is very CPU intensive (95%) yet could still be transferred. Source: http://www-304.ibm.com/partnerworld/wps/servlet/ContentHandler/whitepaper/power6_lpar/aix/v6r1
31 2010 IBM Corporation
33
Power Hypervisor
Storage Controller
SAN Switch Physical FC HBA NPIV Adapter SAN Switch
34
3
VIO Client
LPM
2
fcsZ
VIO Client
1
fcsY
4
fcsY
Power Hypervisor
Steps to migrate a VIO client with a dedicated FC Adapter using NPIV (1) (2)
SAN Switch
(3) (4)
Add virtual FC adapter using NPIV/MPIO Unconfigure the physical FC adapter (using rmdev) and remove slot using DLPAR Migrate the partition using LPM A virtual FC adapter is automatically configured at the destination
35
Migrating partition: zlab019-Migration2-Client (hostname zlab019) Destination system: Ops-Left-9117-MMA-SN10B32C0 Destination MSP:
36
37
38
39
At this point we shutdown both VIO Client and VIO Server, then we reactivated them using the updated profiles (the partitions are brought up in sequence, the VIO Server then the Client)
40 2010 IBM Corporation
41
WWPN
By using standard SAN configuration techniques, assign the mobile partitions storage to the virtual Fibre Channel adapters that use the WWPN pair generated and properly zone the virtual Fibre Channel WWPNs with the storage subsystems WWPN.
42 2010 IBM Corporation
Pre-Migration
Source MSP
Destination MSP
43
44
45
46
47
48
Thank you!