SUN LDOMS aka oracle VM for SPARC is a complete virtual machines that runs an independent operating systems and contains its own virtual CPUs,Storage,Memory, Cryptographic and console.Unlike Solaris zones,LDOMS can run its own operating system with different kernel patch levels. You can also install Solaris 8,Solaris 9 ,Solaris 10 and Solaris 11 on LDOMS. LDOM also provides the OBP for each logical domains but we wont get that in zones. LDOM also allow to move the resources across the logical domains or virtual hosts exclusively. LDOM also allows you to configure zones under that.
Tutorials Topics: Installation and Configuring the Primary domain (You are here) Creating Guest Domains Configuring I/O Domain Dynamically adding/removing Resources Live Migration for Guest Domains Adding ISO image to Multiple Guests Configuration Backup & Restore Removing the LDOM and Services LDOM Quick Reference
Prerequisite: Oracle/Sun T-series Server (T5140 Firmware:7.3.3 ) Solaris 10 or Solaris 11 (To be directly installed on T-series server) LDOM Packages (Ex: 2.1) Server Hardware Console Access Contents: Basic Concepts of LDOM Installation of LDOM Software Configuring the Service Domains Configuring the Control Domains Saving the SP-Profile Basic Concepts of LDOM or Oracle VM for SPARC: LDOM is a hardware based virtualization which works on top the hypervisor. To understand better, you should know the type of terms used in LDOM or oracle VM for SPARC. Control Domain is the place where you are going to install the LDOM software and managing the complete logical domain environment.It used to configure the resources and guest domains Service domain provides the various virtual services to guest domains.Virtaul services can be virtual disks,network switches or virtual consoles.Mostly we will be configuring the control domain as service domain too. I/O Domains have a direct ownership of PCI Bus.You need to directly map the I/O devices to guest domain to improve the application performance in some cases. There is an another domain called root domain and this domain has direct ownership of PCI devcies. So it also called I/O domain. Guest domains doesnt performing any of the above mentioned roles.It jsut runs the operating system instances. Control domain and service domain can be run together.But the user application should not be installed on this domain in a order to protect domain stability and performance.
LDOM Basic concept
Installation of LDOM software: Download the LDOM packages from the oracle support website. For your information ,you cant download without having the oracle support login credentials. Once you have downloaded the packages ,just copy the packages to the Solaris host which you have already installed on T- series server. 1.The below mentioned packages needs to be installed .
-rwxr-xr-x 1 root root 3564851 Jan 16 23:26 OracleVM-Server-for-SPARC-2.0.0_(64-bit).zip drwxrwxrwx 5 root root 512 Jan 16 23:26 SUNWldomr.v drwxrwxrwx 5 root root 512 Jan 16 23:26 SUNWldomu.v 2.Install the SUNWldomr and SUNWldomu packages first. -bash-3.2# pkgadd -d . SUNWldom*
Processing package instance from </>
Solaris Logical Domains (Root)(sparc.sun4v) 11.10.0,REV=2006.10.04.00.26 Copyright (c) 2011, Oracle and/or its affiliates. All rights reserved.
This appears to be an attempt to install the same architecture and version of a package which is already installed. This installation will attempt to overwrite this package.
Using </> as the package base directory. ## Processing package information. ## Processing system information. 42 package pathnames are already properly installed. ## Verifying package dependencies. ## Verifying disk space requirements. ## Checking for conflicts with packages already installed. ## Checking for setuid/setgid programs.
This package contains scripts which will be executed with super-user permission during the process of installing this package.
Do you want to continue with the installation of [y,n,?] y
Installing Solaris Logical Domains (Root) as
## Executing preinstall script. ## Installing part 1 of 1. [ verifying class ] ## Executing postinstall script.
Installation of was successful.
Processing package instance from </>
Solaris Logical Domains (Usr)(sparc.sun4v) 11.10.0,REV=2006.08.08.12.13 Copyright (c) 2011, Oracle and/or its affiliates. All rights reserved.
This appears to be an attempt to install the same architecture and version of a package which is already installed. This installation will attempt to overwrite this package.
Using </> as the package base directory. ## Processing package information. ## Processing system information. 11 package pathnames are already properly installed. ## Verifying package dependencies. ## Verifying disk space requirements. ## Checking for conflicts with packages already installed. ## Checking for setuid/setgid programs.
This package contains scripts which will be executed with super-user permission during the process of installing this package.
Do you want to continue with the installation of [y,n,?] y
Installing Solaris Logical Domains (Usr) as
## Installing part 1 of 1.
Installation of was successful. -bash-3.2# 3.Install the LDOM software . Unzip the package and continue the installation like below -bash-3.2# cd OVM_Server_SPARC-2_0 -bash-3.2# ls -lrt total 12 -r--r--r-- 1 root root 3368 Sep 17 2010 README drwxr-xr-x 2 root root 512 Sep 18 2010 Install drwxr-xr-x 7 root root 512 Dec 16 2010 Product -bash-3.2# cd Install/ -bash-3.2# ls -lrt total 56 -rwxr-xr-x 1 root root 28471 Sep 18 2010 install-ldm -bash-3.2# ./install-ldm
Welcome to the Oracle VM Server for SPARC installer.
You are about to install the LDoms Manager package, SUNWldm, that will enable you to create, destroy and control other domains on your system. You will also be given the option of running the Oracle VM Server for SPARC Configuration Assistant (ldmconfig) to setup the control domain and create guest domains.
If the Solaris Security Toolkit (SST) is installed, you will be prompted to optionally harden your control domain.
Installing LDoms packages. pkgadd -n -d "/OVM_Server_SPARC-2_0/Product" -a pkg_admin SUNWldm.v Copyright (c) 2010, Oracle and/or its affiliates. All rights reserved. Feb 17 10:37:54 sol10-10 sendmail[2631]: [ID 702911 mail.alert] unable to qualify my own domain name (sol10-10) -- using short name
Installation of was successful. pkgadd -n -d "/OVM_Server_SPARC-2_0/Product" -a pkg_admin SUNWldmp2v Copyright (c) 2010, Oracle and/or its affiliates. All rights reserved. Feb 17 10:38:54 sol10-10 sendmail[2705]: [ID 702911 mail.alert] unable to qualify my own domain name (sol10-10) -- using short name
Installation of was successful.
Oracle VM Server for SPARC Configuration
Once installed, you may configure your system for a basic LDoms deployment. If you select "y" for the following question, the Oracle VM Server for SPARC Configuration Assistant (tty) will be launched following a successful installation of the packages.
(You may launch the Configuration Assistant at a later time with the command: /usr/sbin/ldmconfig, or use the GUI Configuration Assistant which is bundled in the Oracle VM Server for SPARC zip file - see README.GUI for more details)
Select an option for configuration:
y) Yes, launch the Configuration Assistant after install n) No thanks, I will configure the system manually later
Enter y or n [y]: n
Verifying that all packages are fully installed. OK. You have new mail in /var/mail//root -bash-3.2# Configuring the Service Domains: Login to the Solaris host where you have installed the LDOM or oracle VM software. 1.List the configured services.By default it will be nothing.
-bash-3.2# ldm list-services 2.Create a new virtual disk service.This service is responsible to provide the virtual disks to guest domains. -bash-3.2# ldm add-vdiskserver primary-vds0 primary -bash-3.2# ldm list-services VDS NAME LDOM VOLUME OPTIONS MPGROUP DEVICE primary-vds0 primary -bash-3.2# 3.Create a new virtual switch. By creating this , we can create N-number of VNIC and share to the guest domains. -bash-3.2# dladm show-dev nxge0 link: up speed: 1000 Mbps duplex: full nxge1 link: up speed: 1000 Mbps duplex: full nxge2 link: up speed: 1000 Mbps duplex: full nxge3 link: up speed: 1000 Mbps duplex: full -bash-3.2# ifconfig -a lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1 inet 127.0.0.1 netmask ff000000 nxge0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2 inet 192.168.2.11 netmask ff000000 broadcast 192.168.2.255 ether 0:21:98:57:cd:16 -bash-3.2# ldm add-vswitch net-dev=nxge1 primary-vsw0 primary -bash-3.2# -bash-3.2# ldm list-services VSW NAME LDOM MAC NET-DEV ID DEVICE LINKPROP DEFAULT-VLAN-ID PVID VID MTU MODE primary-vsw0 primary 00:14:4f:f9:05:b1 nxge1 0 switch@0 1 1 1500
VDS NAME LDOM VOLUME OPTIONS MPGROUP DEVICE primary-vds0 primary
-bash-3.2# Here i have configured virtual switch using nxge1 . You can also use nxge0 configure the virtual swtich but you need to remove the IP from nxge0 and need to configure the IP over virtual switch. -bash-3.2# ifconfig -a lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1 inet 127.0.0.1 netmask ff000000 vsw0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3 inet 198.168.2.11 netmask ff000000 broadcast 192.168.2.255 ether 0:14:4f:98:83:f3 -bash-3.2# 4.Create a virtual service for terminal consoles. -bash-3.2# ldm add-vconscon port-range=5000-5100 primary-vc0 primary -bash-3.2# ldm list-services VCC NAME LDOM PORT-RANGE primary-vc0 primary 5000-5100
VDS NAME LDOM VOLUME OPTIONS MPGROUP DEVICE primary-vds0 primary
-bash-3.2# VSW NAME LDOM MAC NET-DEV ID DEVICE LINKPROP DEFAULT-VLAN-ID PVID VID MTU MODE primary-vsw0 primary 00:14:4f:f9:05:b1 nxge1 0 switch@0 1 1 1500 Now you have successfully configured the default services or service domain. (Creating the spconfig is must after making any changes on the service.Otherwise you will loose the changes after the system power cycle.) Configuring the control Domains: You need to login to Solaris host where you have installed the ldom software to configure the control domain. 1.List the logical domain devices using ldm list-device -a command.You can see all the resource has been allocated to the primary domain. bash-3.2# ldm list-devices -a CORE ID %FREE CPUSET 0 0 (0, 1, 2, 3, 4, 5, 6, 7) 1 0 (8, 9, 10, 11, 12, 13, 14, 15) 2 0 (16, 17, 18, 19, 20, 21, 22, 23) 3 0 (24, 25, 26, 27, 28, 29, 30, 31) 4 0 (64, 65, 66, 67, 68, 69, 70, 71) 5 0 (72, 73, 74, 75, 76, 77, 78, 79) 6 0 (80, 81, 82, 83, 84, 85, 86, 87) 7 0 (88, 89, 90, 91, 92, 93, 94, 95)
VCPU PID %FREE PM 0 0 no 1 0 no 2 0 no 3 0 no 4 0 no 5 0 no 6 0 no 7 0 no 8 0 no 9 0 no 10 0 no 11 0 no 12 0 no 13 0 no 14 0 no 15 0 no 16 0 no 17 0 no 18 0 no 19 0 no 20 0 no 21 0 no 22 0 no 23 0 no 24 0 no 25 0 no 26 0 no 27 0 no 28 0 no 29 0 no 30 0 no 31 0 no 64 0 no 65 0 no 66 0 no 67 0 no 68 0 no 69 0 no 70 0 no 71 0 no 72 0 no 73 0 no 74 0 no 75 0 no 76 0 no 77 0 no 78 0 no 79 0 no 80 0 no 81 0 no 82 0 no 83 0 no 84 0 no 85 0 no 86 0 no 87 0 no 88 0 no 89 0 no 90 0 no 91 0 no 92 0 no 93 0 no 94 0 no 95 0 no
bash-3.2# 2.List what are the resources assigned to the primary domain or control domains . bash-3.2# ldm list-bindings primary NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -n-c-- SP 64 32544M 0.1% 9m
bash-3.2# You have successfully configured the control domain. Note:Here you havent saved the configuration.These are all in-memory configuration.After the power cycle, you will loose the configuration if its not saved in sp config profile. Saving the sp-config file: You have to save the configuration after every ldom settings that you made. For an example, if you didnt save, ldom configuration will be flushed after server power cycle. 1.Create a new profile the current configuration. bash-3.2# ldm add-spconfig unixarena bash-3.2# ldm ls-spconfig factory-default unixarena [current] bash-3.2# In the above command output, you can see that currents settings are saved in unixarena and this settings will come up after the system reboot and power cycle. For an example , if i do any small changes on ldom ,it will not update in the spconfig file. bash-3.2# ldm set-vcpu 12 primary bash-3.2# ldm ls-spconfig factory-default unixarena [next poweron] bash-3.2# Here we just increased the number vpcus to 12 . The previous number of vpcu was 8. Here sp-config file unixarena will be activated on next power cycle. Which mean you will loose the cpu setting you just done. In a order to save the configuration, just re-create the sp-config like below. bash-3.2# ldm ls-spconfig factory-default unixarena [next poweron] bash-3.2# ldm remove-spconfig unixarena bash-3.2# ldm ls-spconfig factory-default [next poweron] bash-3.2# ldm add-spconfig unixarena_new bash-3.2# ldm ls-spconfig factory-default unixarena_new [current] bash-3.2# Checking the OBP,Hypervisor and LDOM version: -bash-3.2# ldm -V
Logical Domain Manager (v 2.0) Hypervisor control protocol v 1.4 Using Hypervisor MD v 1.1
System PROM: Hypervisor v. 1.7.4. @(#)Hypervisor 1.7.4.a 2009/09/21 08:25\015
OpenBoot v. 4.30.4 @(#)OBP 4.30.4 2009/08/19 07:25 -bash-3.2# Hope this article is first step to learn LDOM for Solaris beginners.In this article we have seen the basic concepts of oracle VM for SPARC (LDOM), packages installation,configuration the service domain,configuring the control domain or primary domain and saving the sp-configuration. I will try to complete the tutorial by adding another 6 to 8 articles. Hope you will like it . Please share it in social media to reach maximum number of people. Thank you for visiting UnixArena LDOM Tutorial Part 2 Configuring the guest domain
How to configure the Guest domains on LDOM ? February 18, 2014 in LDOM
Guest domain is a top level virtual machine which we can create using the control domain. Guest domain will not perform any administrative operation on ldom but runs its own operating system on it.Once you have completed the Oracle VM for SPARC tutorial 1 ,then you can start following this article.We will see how we can create a new guest domain and allocate the resources to it. Oracle VM for SPARC Tutorial: Installation and Configuring the Primary domain Creating Guest Domains (You Are Here) Configuring I/O Domain Dynamically adding/removing Resources Live Migration for Guest Domains Adding ISO image to Multiple Guests Configuration Backup & Restore Removing the LDOM and Services
Creating the new Guest Domain 1. Login to the control domain which we have already configured . (Refer Oracle VM for SPARC tutorial 1 ) 2. You need to find out how much resources are available in the system.Use ldm list-devices -a command to get those details. PD-bash# ldm list-devices -a CORE ID %FREE CPUSET 0 0 (0, 1, 2, 3, 4, 5, 6, 7) 1 100 (8, 9, 10, 11, 12, 13, 14, 15) 2 100 (16, 17, 18, 19, 20, 21, 22, 23) 3 100 (24, 25, 26, 27, 28, 29, 30, 31) 4 100 (64, 65, 66, 67, 68, 69, 70, 71) 5 100 (72, 73, 74, 75, 76, 77, 78, 79) 6 100 (80, 81, 82, 83, 84, 85, 86, 87) 7 100 (88, 89, 90, 91, 92, 93, 94, 95)
# 3.Create a new Guest domain using the below command. PD-bash# ldm add-domain ua1 PD-bash# ldm list-domain NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -n-cv- SP 8 4128M 0.3% 17m ua1 inactive ------ PD-bash# 4.Add 8 vcpus to guest domain ua1. PD-bash# ldm add-vcpu 8 ua1 PD-bash# ldm list-domain NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -n-cv- SP 8 4128M 0.5% 17m ua1 inactive ------ 8 PD-bash# 5.Add 4G memory to logical domain ua1. PD-bash# ldm add-memory 4G ua1 PD-bash# ldm list-domain NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -n-cv- SP 8 4128M 0.4% 18m ua1 inactive ------ 8 4G PD-bash# 6.Create a new virtual NIC on top the virtual switch and assigned to ua1. List the logical domain bindings to verify the configuration. PD-bash# ldm add-vnet vnet0 primary-vsw0 ua1 PD-bash# ldm list-bindings ua1 NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME ua1 inactive ------ 8 4G
UUID 1e8baa72-a123-6184-f724-feebad1585eb
CONTROL failure-policy=ignore
DEPENDENCY master=
NETWORK NAME SERVICE ID DEVICE MAC MODE PVID VID MTU LINKPROP vnet0 primary-vsw0 0 00:14:4f:fb:54:92 1
PD-bash# 7.Now its time to add the new disks to logical domain ua1 . Here I am using /dev/dsk/c4t0456RTE000005400002d0s2 LUN for ua1 domain . 8.First i am allocating the LUN to virtual disk server. PD-bash# ldm add-vdiskserverdevice /dev/dsk/c4t600144F0B8E706DA00005300F4350005d0s2 vol0@primary-vds0 PD-bash# ldm list-services vds0 LDom "vds0" was not found PD-bash# ldm list-services VCC NAME LDOM PORT-RANGE primary-vc0 primary 5000-5100
VSW NAME LDOM MAC NET-DEV ID DEVICE LINKPROP DEFAULT-VLAN-ID PVID VID MTU MODE primary-vsw0 primary 00:14:4f:fa:12:0e nxge1 0 switch@0 1 1 1500
PD-bash# 9.Now bind the virtual disk to logical domain ua1 which you have created in step 8 . PD-bash# ldm add-vdisk vdisk0 vol0@primary-vds0 ua1 PD-bash# ldm list-bindings ua1 NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME ua1 inactive ------ 8 4G
UUID 1e8baa72-a123-6184-f724-feebad1585eb
CONTROL failure-policy=ignore
DEPENDENCY master=
NETWORK NAME SERVICE ID DEVICE MAC MODE PVID VID MTU LINKPROP vnet0 primary-vsw0 0 00:14:4f:fb:54:92 1
DISK NAME VOLUME TOUT ID DEVICE SERVER MPGROUP vdisk0 vol0@primary-vds0 0
PD-bash# 10.The same way you can map the ISO image file to guest domain using the below set of commands. PD-bash# ldm add-vdiskserverdevice /export/home/sol-10-u10-ga2-sparc-dvd.iso iso0@primary-vds0 PD-bash# ldm add-vdisk iso0 iso0@primary-vds0 ua1 PD-bash# ldm list-bindings ua1 NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME ua1 inactive ------ 8 4G
UUID 1e8baa72-a123-6184-f724-feebad1585eb
CONTROL failure-policy=ignore
DEPENDENCY master=
NETWORK NAME SERVICE ID DEVICE MAC MODE PVID VID MTU LINKPROP vnet0 primary-vsw0 0 00:14:4f:fb:54:92 1
DISK NAME VOLUME TOUT ID DEVICE SERVER MPGROUP vdisk0 vol0@primary-vds0 0 iso0 iso0@primary-vds0 1
PD-bash# 11.Just bind the logical domain to the primary domain. PD-bash# ldm list-domain NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -n-cv- SP 8 4128M 0.4% 26m ua1 inactive ------ 8 4G PD-bash# PD-bash# ldm bind-domain ua1 LDom ua1 started PD-bash# ldm list-domain NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -n-cv- SP 8 4128M 0.7% 30m ua1 bind ------ 5000 8 4G PD-bash# 12.Set the auto-boot is false to logical domain ua1 for Operating system installation. PD-bash# ldm set-variable auto-boot\?=false ua1 13.Boot/start the logical domain ua1. PD-bash# ldm start-domain ua1 LDom ua1 started PD-bash# ldm list-domain NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -n-cv- SP 8 4128M 0.7% 30m ua1 active -t---- 5000 8 4G 12% 1s PD-bash# 14. Take the console of the logical domain ua1 .It uses the console port 5000 (see step:13) ok boot iso0 PD-bash# telnet 192.168.2.11 5000 Trying 1192.168.2.11... Connected to 192.168.2.11. Escape character is '^]'.
Connecting to console "ua1" in group "ua1" .... Press ~? for control options ..
{0} ok 15. List the devices to boot. {0} ok devalias iso0 /virtual-devices@100/channel-devices@200/disk@1 vdisk0 /virtual-devices@100/channel-devices@200/disk@0 vnet0 /virtual-devices@100/channel-devices@200/network@0 net /virtual-devices@100/channel-devices@200/network@0 disk /virtual-devices@100/channel-devices@200/disk@0 virtual-console /virtual-devices/console@1 name aliases {0} ok 16.Boot the system from iso. {0} ok boot iso0 Boot device: /virtual-devices@100/channel-devices@200/disk@1 File and args: SunOS Release 5.10 Version Generic_147440-01 64-bit Copyright (c) 1983, 2011, Oracle and/or its affiliates. All rights reserved. Configuring devices. Using RPC Bootparams for network configuration information. Attempting to configure interface vnet0... | You have successfully created the guest domain and assigned the resources to it. You have also mapped the virtual disks to the virtualdisk server and mapped to the guest domain or logical domain. In the end of the article ,we have see how to access the console of the guest domains and mapped the ISO file for OS installation. Hope this article is informative to you .Please share it in social networks. Thank you for visiting UnixArena.
How to Configure the I/O Domain on Oracle VM for SPARC ? February 19, 2014 in LDOM
If you allow any of the guest domain or control domain to access the any I/O (PCI) cards directly from the hardware ,usually that domain is called I/O domain or root domain.By default, two virtual I/O services that you need to configure on primary domain to provide the I/O to the guest domains.One is virtual disk (vds) for storage and another one is virtual switch(vsw) for network communication.But in I/O domain,you can directly access the real physical LUN or physical network port.But the typical guest domain will be using the virtual I/O services.For an example, SAN LUNs will be mapped to control domain and from there , you can map the physical LUNS to guest domains as virtual services. Oracle VM for SPARC Tutorial: Installation and Configuring the Primary domain Creating Guest Domains Configuring I/O Domain (You Are Here) Dynamically adding/removing Resources Live Migration for Guest Domains Adding ISO image to Multiple Guests Configuration Backup & Restore Removing the LDOM and Services
IO Domain Example In the above image, control domain aka primary domain and SOL10 Guest domain also called I/O domains since its directly accessing the PCI card(HBA). Here we will see how we can configure the guest domain as I/O domain. 1. Login to control domain and list the complete hardware details of the server . -bash-3.2# ldm list-devices -a CORE ID %FREE CPUSET 0 0 (0, 1, 2, 3) 1 0 (4, 5, 6, 7) 2 0 (8, 9, 10, 11) 3 0 (12, 13, 14, 15) 4 0 (16, 17, 18, 19) 5 0 (20, 21, 22, 23) 6 0 (24, 25, 26, 27) 7 0 (28, 29, 30, 31)
VCPU PID %FREE PM 0 0 no 1 0 no 2 0 no 3 0 no 4 0 no 5 0 no 6 0 no 7 0 no 8 0 no 9 0 no 10 0 no 11 0 no 12 0 no 13 0 no 14 0 no 15 0 no 16 0 no 17 0 no 18 0 no 19 0 no 20 0 no 21 0 no 22 0 no 23 0 no 24 0 no 25 0 no 26 0 no 27 0 no 28 0 no 29 0 no 30 0 no 31 0 no
-bash-3.2# In the bottom of the above command output shows that IO (pci@780 and pci7c0) have already bind to some of the domain. 2.To find where the PCI IO card has been bound , you need to use ldm list-bindings command. -bash-3.2# ldm list-bindings NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -ndcv- SP 8 2G 0.6% 4h 58m
IO DEVICE PSEUDONYM OPTIONS pci@780 bus_a pci@7c0 bus_b The above outputs confirms both the PCI card has been assigned to primary domain.So our primary domain is also called I/O domain or root domain. 3.We will remove one the PCI card from primary and we will assign to one of the guest domain. Before removing,please make sure that card is not used in primary .If its in use, please un-configure it .Let me confirm which card can be removed .(pci@780 and pci@7c0) pci@780 is used for internal disks. We are good to move the pci@7c0 to one of the guest domains since there is no device is currently using it from primary domain. AVAILABLE DISK SELECTIONS: 0. c0t0d0 /pci@780/pci@0/pci@9/scsi@0/sd@0,0 1. c0t1d0 /pci@780/pci@0/pci@9/scsi@0/sd@1,0 -bash-3.2# dladm show-dev vsw0 link: up speed: 1000 Mbps duplex: full e1000g0 link: up speed: 1000 Mbps duplex: full e1000g1 link: unknown speed: 0 Mbps duplex: half e1000g2 link: unknown speed: 0 Mbps duplex: half e1000g3 link: unknown speed: 0 Mbps duplex: half -bash-3.2# cat /etc/path_to_inst |grep e1000g "/pci@780/pci@0/pci@1/network@0" 0 "e1000g" "/pci@780/pci@0/pci@1/network@0,1" 1 "e1000g" "/pci@7c0/pci@0/pci@2/network@0" 2 "e1000g" "/pci@7c0/pci@0/pci@2/network@0,1" 3 "e1000g" -bash-3.2# 4.List the configured guest domain. -bash-3.2# ldm list-domain NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -n-cv- SP 8 2G 0.5% 4h 51m ldom1 bound ------ 5000 8 512M ldom2 bound ------ 5001 8 896M ldom3 active -n---- 5002 8 512M 0.0% 3h 20m -bash-3.2# 5. Remove or unbind the pci@7c0 to from primary domain.You can see couple of physical network adapters has been removed from primary.You may need to reboot the server to release the pci cards sometimes. -bash-3.2# ldm remove-io pci@7c0 -bash-3.2# dladm show-dev vsw0 link: up speed: 1000 Mbps duplex: full e1000g0 link: up speed: 1000 Mbps duplex: full e1000g1 link: unknown speed: 0 Mbps duplex: half -bash-3.2# -bash-3.2# ldm list-bindings NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -n-cv- SP 8 2G 0.2% 3h 57m
IO DEVICE PSEUDONYM OPTIONS pci@780 bus_a 6.Now add the pci card pci7c0 to ldom1 .sometimes you may need to reboot the system to see the new pci cards. # ldm add-io pci@7c0 ldom1 Initiating a delayed reconfiguration operation on the primary domain. All configuration changes for other domains are disabled until the primary domain reboots, at which time the new configuration for the primary domain will also take effect. -bash-3.2# 7.You can list the binding using ldm list-bindings command to confirm the changes.sometimes you need to reboot the system to see the devices in operating system level. ldm list-bindings ldom1 NAME SERVICE PORT SP
------------------------------------------------------------------------------ NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME ldom1 bound ------ 5000 8 512M
VARIABLES auto-boot?=false boot-device=vdisk1:a disk net keyboard-layout=US-English
IO DEVICE PSEUDONYM OPTIONS pci@7c0 bus_b 8. Login to ldom1 and see the pci card devices . -bash-3.2# dladm show-dev vsw0 link: up speed: 1000 Mbps duplex: full e1000g2 link: up speed: 1000 Mbps duplex: full e1000g3 link: up speed: 1000 Mbps duplex: full -bash-3.2# Now ldom1 is also called as I/O domain since its accessing the network port directly without hypervisor. The same way if your system have multiple HBA-cards , you can assign one of the cards to guest domain and SAN team can directly allocate tothe luns to guest domain.(ohhh sorry now its called I/O domain) You have successfully removed the pci card from primary domain and assigned to the guest domain. So the guest domain also become I/O domain now. These I/O domain will be used on special application requirements. Otherwise we will configure only the guest domains.
How to Dynamically add the resources in Oracle VM for SPARC ? February 20, 2014 in LDOM
The beauty of LDOM or Oracle VM for SPARC is that resources can be dynamically added or removed from the domains(Primary/Guest). This will help you to perform the load balancing between the guest domains on the physical server. Dynamic resource allocation is only possible if the guest domain operating system is booted.If you add the resource during the guest domain OS start-up, allocation will fail. Note:Solaris operating system support dynamic resources allocation. Oracle VM for SPARC Tutorial: Installation and Configuring the Primary domain Creating Guest Domains Configuring I/O Domain Dynamically adding/removing Resources (You Are Here) Live Migration for Guest Domains Adding ISO image to Multiple Guests Configuration Backup & Restore Removing the LDOM and Services
Here we will see how to add/remove resources dynamically when the guest domain is up and running.Before performing any resource management, please make sure you have enough resources on the system .(use # ldm list-devices -a to find free resources) 1.List the running domains from the control domain aka primary domain. bash-3.2# ldm ls-domain NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -n-cv- SP 14 4128M 0.2% 7h 48m ldom2 active -n---- 5000 4 4G 0.1% 9h 28m ldom2clone active -n---- 5001 8 8G 0.1% 7h 7m As per the above output,all the domains are in active state.Which means domains are in powered.(Login to the ldoms and verify OS is up or not .) 2.Add four more virtual CPUs to ldom2 guest domain. -bash-3.2# ldm add-vcpu 4 ldom2 -bash-3.2# ldm ls-domain NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -n-cv- SP 14 4128M 0.3% 7h 48m ldom2 active -n---- 5000 8 4G 0.3% 9h 28m ldom2clone active -n---- 5001 8 8G 0.1% 7h 7m You can see number of vcpu has been increased from 4 to 8 for ldom2. 3.Number CPUs can added by directly setting the new value.For an example, Currently ldom2 has just 8 vcpu .If you want to add four more, you just give the below command. -bash-3.2# ldm set-vcpu 12 ldom2 -bash-3.2# ldm ls-domain NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -n-cv- SP 14 4128M 0.3% 7h 49m ldom2 active -n---- 5000 12 4G 0.1% 9h 29m ldom2clone active -n---- 5001 8 8G 0.1% 7h 8m 4.CPU also can be removed using set command. Currently ldom2 has 12 vcpus. If you want to remove 8 from that, just give the below command. -bash-3.2# ldm set-vcpu 4 ldom2 -bash-3.2# ldm ls-domain NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -n-cv- SP 14 4128M 0.4% 7h 49m ldom2 active -n---- 5000 4 4G 3.4% 9h 29m ldom2clone active -n---- 5001 8 8G 0.1% 7h 8m -bash-3.2# 5.Removing vcpu also can be done using remove-vpcu command. -bash-3.2# ldm remove-vcpu 2 ldom2 -bash-3.2# ldm ls-domain NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -n-cv- SP 14 4128M 0.2% 7h 52m ldom2 active -n---- 5000 2 6G 0.2% 9h 32m ldom2clone active -n---- 5001 8 8G 0.1% 7h 11m -bash-3.2# 6. Dynamic memory management also same like the cpu management. You need to use memory keyword instead of vcpu. To set the new memory size to the running domain,use the below commands.If the new value size is higher than the current one ,memory will be added .If the new value is lesser than the existing one, memory size will be reduced to the guest domains. -bash-3.2# ldm ls-domain NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -n-cv- SP 14 4128M 0.2% 7h 51m ldom2 active -n---- 5000 4 4G 0.1% 9h 30m ldom2clone active -n---- 5001 8 8G 0.1% 7h 10m -bash-3.2# ldm set-memory 8G ldom2 -bash-3.2# ldm ls-domain NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -n-cv- SP 14 4128M 0.2% 7h 51m ldom2 active -n---- 5000 4 8G 0.2% 9h 31m ldom2clone active -n---- 5001 8 8G 0.1% 7h 10m 7.We can also use the below method to increase the physical memory of guest domains. -bash-3.2# ldm add-memory 2G ldom2 -bash-3.2# ldm ls-domain NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -n-cv- SP 14 4128M 0.3% 7h 51m ldom2 active -n---- 5000 4 10G 0.1% 9h 31m ldom2clone active -n---- 5001 8 8G 0.2% 7h 10m 8. Like remove option in vcpu, remove-memory option is used to reduce the memory size to the guest domain. -bash-3.2# ldm remove-memory 4G ldom2 -bash-3.2# ldm ls-domain NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -n-cv- SP 14 4128M 0.4% 7h 52m ldom2 active -n---- 5000 4 6G 0.5% 9h 32m ldom2clone active -n---- 5001 8 8G 0.2% 7h 11m -bash-3.2# Other resources like virtual disk also can be added on fly to the virtual machine.Here I am creating a ZFS level LUN for test purpose. -bash-3.2# zfs create -V 25G pool1/uavol1 -bash-3.2# zfs list NAME USED AVAIL REFER MOUNTPOINT pool1 25.4G 248G 31K /pool1 pool1/clonevol1 96.3M 248G 4.75G - pool1/uavol1 25.3G 248G 4.70G - pool1/vol1 25.3G 269G 4.70G - Adding the disk to virtual disk service. (vds0) -bash-3.2# ldm add-vdiskserverdevice /dev/zvol/dsk/pool1/uavol1 uavol1@primary-vds0 Assigning the LUN to guest domain from the virtual disk service. -bash-3.2# ldm add-vdisk uavdisk uavol1@primary-vds0 ldom2 Connect to the server and scan the newly assiged LUN. -bash-3.2# ldm ls-domain NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -n-cv- SP 14 4128M 0.2% 8h ldom2 active -n---- 5000 2 6G 0.2% 9h 40m ldom2clone active -n---- 5001 8 8G 0.1% 7h 19m -bash-3.2# telnet 192.168.2.28 5000 Trying 192.168.2.28... Connected to 192.168.2.28. Escape character is '^]'.
Connecting to console "ldom2" in group "ldom2" .... Press ~? for control options ..
ldom2-208 console login: root Password: # devfsadm # echo |format To terminate the ldom console, use ctrl + ] key sequence to back to the control domain. You can also see the lun from the guest domain OK prompt. Let me halt the domain and start it. -bash-3.2# ldm stop-domain ldom2 LDom ldom2 stopped -bash-3.2# ldm start-domain ldom2 LDom ldom2 started Connect to the guest domain console again. -bash-3.2# telnet 192.168.2.28 5000 Trying 192.168.2.28... Connected to 192.168.2.28. Escape character is '^]'.
Connecting to console "ldom2" in group "ldom2" .... Press ~? for control options ..
{0} ok devalias uavdisk /virtual-devices@100/channel-devices@200/disk@0 iso_ldom2 /virtual-devices@100/channel-devices@200/disk@2 vdisk1_dom2 /virtual-devices@100/channel-devices@200/disk@1 vnet0 /virtual-devices@100/channel-devices@200/network@0 net /virtual-devices@100/channel-devices@200/network@0 disk /virtual-devices@100/channel-devices@200/disk@1 virtual-console /virtual-devices/console@1 name aliases You can see the newly assigned disk in the OBP level as well.So the bottom line is that resource management is very simple and effective in LDOM aka oracle VM for SPARC. Hope you too liked the Oracle VM for SPARC resource management. Thank you for visiting UnixArena.
How to perform Live domain migration on LDOM ? Oracle VM SPARC February 20, 2014 in LDOM
In Oracle VM for SPARC,we can migrate the guest domains to another system without any downtime. This is similar to vmotion on VMware ESXi. This feature provides the operational flexibility to the customers who dont want the system to be down for system maintenance.Prior to the system maintenance ,we can easily move the guest domains to another system without any downtime using this live migration.Once you have completed the system maintenance , you can just migrate the guest domains back to the original systems. Oracle VM for SPARC Tutorial: Installation and Configuring the Primary domain Creating Guest Domains Configuring I/O Domain Dynamically adding/removing Resources Live Migration for Guest Domains (You Are Here) Adding ISO image to Multiple Guests Configuration Backup & Restore Removing the LDOM and Services
Normally we would allocate the same set of LUNS to source and target machines. The live-migration feature is just only for the Guest domain migration not for the storage.So when the control domains verify the remote system, it will check whether the guest machine storage is accessible or not. Prerequisites: Network access between Control Domains Shared Storage Compatible versions of Oracle VM Server for SPARC and Firmware. Similar Hardware configuration on both the systems Identical Virtual services between the nodes. How the migration is happening without any downtime ? Migration has been initiated by one of the control domain root user Control domain performs the nessasary check prior to the migration.It checks server compatibitliy,network connnectivity and available resources Virtual machine configuration is copied from source to target Virtaul machines in-memory contents will be copied to target system Guest domain will be suspended on source system Finally guest will be started on the target system Here we will see how we can perform the live migration between two T5140 systems. 1. Login to the source system and list the guest domains. -bash-3.2# ldm ls-domain NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -n-cv- SP 12 4128M 0.5% 1d 15h 45m ua1guest active -n---- 5000 12 6G 0.1% 1h 35m -bash-3.2# 2.List the guest domain ua1guest bindings.Make sure the remote system should has enough enough resource to handle this guest domains. (Use # ldm list-devices -a) -bash-3.2# ldm ls-bindings ua1guest NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME ua1guest active -n---- 5000 12 6G 0.1% 1h 36m
NETWORK NAME SERVICE ID DEVICE MAC MODE PVID VID MTU LINKPROP vnet0 primary-vsw0@primary 0 network@0 00:14:4f:fa:b2:d6 1 1500 PEER MAC MODE PVID VID MTU LINKPROP primary-vsw0@primary 00:14:4f:fa:12:0e 1 1500
DISK NAME VOLUME TOUT ID DEVICE SERVER MPGROUP ua1guestdsk ua1guest@primary-vds0 0 disk@0 primary
VCONS NAME SERVICE PORT ua1guest primary-vc0@primary 5000
-bash-3.2# 3.List the source systems services. You should also have the same services name configured on the target system. -bash-3.2# ldm list-services VCC NAME LDOM PORT-RANGE primary-vc0 primary 5000-5100
VSW NAME LDOM MAC NET-DEV ID DEVICE LINKPROP DEFAULT-VLAN-ID PVID VID MTU MODE primary-vsw0 primary 00:14:4f:fa:12:0e nxge1 0 switch@0 1 1 1500
-bash-3.2# 4.Now login to target system and check whether the live migration can be done or not. -bash-3.2# ldm list-services VCC NAME LDOM PORT-RANGE primary-vc0 primary 5000-5100
VSW NAME LDOM MAC NET-DEV ID DEVICE LINKPROP DEFAULT-VLAN-ID PVID VID MTU MODE primary-vsw0 primary 00:14:4f:fb:a5:6d nxge1 0 switch@0 1 1 1500
-bash-3.2# ldm ls-domain NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -n-cv- SP 8 8G 0.2% 1d 6h 23m subuldom inactive ------ 4 4G -bash-3.2# As per the above command output, all the services names are matching with source system. We also have the enough resource to handle the guest domain. 5. Login to the source systems control domain and execute the migration command.It will prompt for target system root password. Just enter continue. -bash-3.2# ldm ls-domain NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -n-cv- SP 12 4128M 0.3% 1d 15h 53m ua1guest active -n---- 5000 12 6G 0.0% 1h 42m -bash-3.2# ldm migrate ua1guest 192.168.2.29 Target Password: 6.Login to the target systems control domain and list the domains. -bash-3.2# ldm ls-domain NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -n-cv- SP 8 8G 0.2% 1d 6h 23m subuldom inactive ------ 4 4G -bash-3.2# ldm ls-domain NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -n-cv- SP 8 8G 14% 1d 6h 26m ua1guest bound -----t 5000 1 6G subuldom inactive ------ 4 4G -bash-3.2# In the above output, you can see , ua1guest domain is copying from the source system. 7.In the source system , you may get a warning like the below one after successfully completion of live-migration. Domain ua1guest is not running an operating system that is compatible with the latest migration functionality.
Upgrading to the latest software will remove restrictions on a migrated domain that are in effect until it is rebooted. Consult the product documentation for a full description of these restrictions. -bash-3.2# 8.In target system, the guest domain flag will be changed from t to n . -bash-3.2# ldm ls-domain NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -n-cv- SP 8 8G 44% 1d 6h 28m ua1guest bound -----t 5000 1 6G subuldom inactive ------ 4 4G -bash-3.2# ldm ls-domain NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -n-cv- SP 8 8G 0.9% 1d 6h 32m ua1guest active -n---- 5000 12 6G 0.1% 12h 11m subuldom inactive ------ 4 4G -bash-3.2# 9.You didnt believe the live migration ? Let me double check. -bash-3.2# telnet localhost 5000 Trying ::1... telnet: connect to address ::1: Network is unreachable Trying 127.0.0.1... Connected to localhost. Escape character is '^]'.
Connecting to console "ua1guest" in group "ua1guest" .... Press ~? for control options ..
UAMIGD console login: root Password: Last login: Thu Feb 20 00:50:39 on console Oracle Corporation SunOS 5.10 Generic Patch January 2005 # uptime 2:02am up 1:43, 1 user, load average: 0.00, 0.02, 0.01 # As per the above command output, nothing has happened to the guest domain. The system is up from last 2hr. The migration has been initiated few minutes back.So the guest domain was up during the migration time too. Hope you will also give a try on LDOM guest live migration . Thank you for visiting UnixArena.
How to Add the Same ISO image file in Multiple LDOMS ? February 20, 2014 in LDOM
In oracle VM for SPARC (LDOM), you need to attach the ISO file to each logical domain, for the first time operating system installation. If you have pre-configured jumpstart , You can use it for the guest domain operating system installation. If you dont have jumpstart,then you need to install it using the ISO image. By default LDOM will not allow to map the same device file to multiple guest domains .Here we will see how we can map the same ISO image to multiple LDOMS. Oracle VM for SPARC Tutorial: Installation and Configuring the Primary domain Creating Guest Domains Configuring I/O Domain Dynamically adding/removing Resources Live Migration for Guest Domains Adding ISO image to Multiple Guests (You Are Here) Configuration Backup & Restore Removing the LDOM and Services
1. Login to the control domain and list the ISO image which you need to map it to ldoms. # ls /export/home/sol-10-u10-ga2-sparc-dvd.iso sol-10-u10-ga2-sparc-dvd.iso 2.List the available services. Create a new virtual device for the virtual disk server using the below command. -bash-3.2# ldm list-services VCC NAME LDOM PORT-RANGE primary-vcc0 primary 5000-5100
VSW NAME LDOM MAC NET-DEV ID DEVICE LINKPROP DEFAULT-VLAN-ID PVID VID MTU MODE primary-vsw0 primary 00:14:4f:fa:fe:49 e1000g0 0 switch@0 1 1 1500
VSW NAME LDOM MAC NET-DEV ID DEVICE LINKPROP DEFAULT-VLAN-ID PVID VID MTU MODE primary-vsw0 primary 00:14:4f:fa:fe:49 e1000g0 0 switch@0 1 1 1500
VDS NAME LDOM VOLUME OPTIONS MPGROUP DEVICE primary-vds0 primary vol1 /dev/zvol/dsk/pool1/vol1 clonevol1 /dev/zvol/dsk/pool1/clonevol1 vol0 /dev/dsk/c2t600144F0B8E706DA00005300F47B0007d0s2 vol10 /dev/dsk/c2t600144F0C1135B4900005303839A0002d0s2 iso0 ro /export/home/sol-10-u10-ga2-sparc-dvd.iso 3.Add the newly created virtual read-only iso image to guest domain. -bash-3.2# ldm list-domain NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -n-cv- SP 12 4128M 0.3% 5h 45m ua1 active -n---- 5000 8 4G 0.1% 58m ua2 active -n---- 5001 8 2G 0.0% 58m ua3 active -n---- 5002 4 1G 0.1% 30m -bash-3.2# ldm add-vdisk viso1 newiso@primary-vds0 ua1 -bash-3.2# 4.You can see that ISO file has been attached to the ua1 ldom. -bash-3.2# ldm ls-bindings ldom1 NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME ldom1 bound ------ 5000 8 512M
DISK NAME VOLUME TOUT ID DEVICE SERVER MPGROUP vdisk1 vol1@primary-vds0 2 disk@2 primary viso1 newiso@primary-vds0 0 disk@0 primary 5.When try to map the same iso virtual image to another domain, you will get below error. -bash-3.2# ldm add-vdisk viso2 newiso@primary-vds0 ua2 The volume "newiso" is already in use. 6.To avoid the above error, you need to create a new virtual read-only ISO image again. -bash-3.2# ldm add-vdsdev options=ro /export/home/sol-10-u10-ga2-sparc-dvd.iso newiso2@primary-vds0 -bash-3.2# 7.Add the second virtual iso image to the another LDOM using the below command. -bash-3.2# ldm add-vdisk viso2 newiso2@primary-vds0 ua2 -bash-3.2# -bash-3.2# ldm list-bindings ua2 NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME ua2 active -n---- 5001 8 2G 0.1% 1h 3m DISK NAME VOLUME TOUT ID DEVICE SERVER MPGROUP vdisk1 ua1ldom2@primary-vds0 0 disk@0 primary viso2 newiso2@primary-vds0 1 disk@1 primary The same way you can map the same ISO file to N-number of guest domains.(6 and 7 ). Hope this article is informative to you. Thank you for visiting UnixArena.
Oracle VM SPARC LDOM configuration Backup and Restore February 21, 2014 in LDOM
Oracle VM for SPARC configuration can be backed up in XML file and it can be restored it whenever required.This configuration backup will be very useful for future rebuilding. It also provides the option to take the backup of particular domain and that can be restored using ldm add-domain command.In this article we will how take the individual domain backup,how to take the complete ldom configuration backup and restore methods. Oracle VM for SPARC Tutorial: Installation and Configuring the Primary domain Creating Guest Domains Configuring I/O Domain Dynamically adding/removing Resources Live Migration for Guest Domains Adding ISO image to Multiple Guests Configuration Backup & Restore (You Are Here) Removing the LDOM and Services 1. Login to control Domain and list the current configuration file. -bash-3.2# ldm list-spconfig factory-default config_new [current] -bash-3.2# 2.To take the complete LDOM configuration backup,use below command. # ldm list-constraints -x >config.xml 3.You can also take the configuration backup for specific LDOM, # ldm list-constraints -x ldom_name > ldom_name.xml How to restore the system to factory default ? 1. Login to the control domain console and remove the spconfig profile. bash-3.2# ldm ls-spconfig factory-default config_new [current] bash-3.2# ldm remove-spconfig config_new bash-3.2# ldm ls-spconfig factory-default [next poweron] bash-3.2# 2.Power cycle the server to take the changes effect. bash-3.2# init 5 bash-3.2# svc.startd: The system is coming down. Please wait. svc.startd: 101 system services are now being stopped. Feb 18 10:00:49 sol10-11 syslogd: going down on signal 15 svc.startd: The system is down. syncing file systems... done
Serial console stopped.
-> show /SYS Properties: type = Host System ipmi_name = /SYS keyswitch_state = Normal product_name = T5140 product_part_number = 4613802-42 product_serial_number = FML1015013 product_manufacturer = SUN MICROSYSTEMS fault_state = Faulted power_state = Off -> start /SYS Are you sure you want to start /SYS (y/n)? y Starting /SYS
-> -> start /SP/console Are you sure you want to start /SP/console (y/n)? y
Serial console started. To stop, type #.
Done 0:0:0>Master CPU Tests Basic....Done 0:0:0>Init MMU..... 0:0:0>L2 Tests....Done 0:0:0>Extended CPU Tests....Done 0:0:0>Scrub Memory....Done 0:0:0>Functional CPU Tests....Done sol10-11 console login: 3. Login to the system and see whether factory default settings has been restored or not . sol10-11 console login: root Password: Feb 18 10:10:58 sol10-11 login: ROOT LOGIN /dev/console Last login: Tue Feb 18 01:19:27 on console Oracle Corporation SunOS 5.10 Generic Patch January 2005 You have new mail. -bash-3.2#
-bash-3.2# ldm list-spconfig factory-default [current] -bash-3.2# Restoring the sp-config : 1.Once the system has restored to the factory default,then you can restore the system backup using the XML files which we have created. Login to the control domain and restore it. # ldm init-system -i config.xml # ldm add-spconfig unixarencfg1 # ldm list-spconfig factory-default unixarenacfg1 (current) 2.To restore the specific ldom/primary domain configuration # ldm init-system -i ldom_name.xml # ldm add-spconfig unixarencfg2 # ldm list-spconfig factory-default unixarenacfg1 unixarenacfg2 (current) 3.To restore only the virtual service, # ldm init-system -s -i filename.xml -bash-3.2# ldm list-spconfig factory-default unixarenacfg1 unixarenacfg2[active on Reboot] -bash-3.2# ldm add-spconfig unixarencfg3 -bash-3.2# ldm list-spconfig factory-default unixarenacfg1 unixarenacfg2 unixarenacfg2 (current) The old configuration copies can be removed using ldm remove-spconfig profile_name . If you make any small changes on LDOM configuration, you must create a new profile to save it.Otherwise you may loose the new changes after the system power cycle. Thank you for visiting UnixArena.
How to Remove Logical Domains and Services ? Oracle VM for SPARC February 20, 2014 in LDOM
Destroying the infrastructure is a easy task in all the environments.In oracle VM for SPARC also ,you can easily remove the guest domains in few seconds. The same way you can also remove the virtual disk resources and virtual network resources. In some cases you may not able to remove the resources if its in use or already bound to any guest domains. In this article ,we will see how to remove the guest domains and how to remove the virtual services like vds and vswitch. Oracle VM for SPARC Tutorial: Installation and Configuring the Primary domain Creating Guest Domains Configuring I/O Domain Dynamically adding/removing Resources Live Migration for Guest Domains Adding ISO image to Multiple Guests Configuration Backup & Restore Removing the LDOM and Services (You Are Here) 1. Login to the control domain and list the available domains. -bash-3.2# ldm ls-domain NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -n-cv- SP 14 4128M 0.2% 8h 7m ldom2 active -t---- 5000 2 6G 50% 4m ldom2clone active -n---- 5001 8 8G 0.1% 7h 26m 2.Halt the domain by logging in to the guest domain operating system. (init 5) . You can also perform the force shutdown from the control domain using the below command. -bash-3.2# ldm stop-domain ldom2clone LDom ldom2clone stopped 3.Un-bind the domain from the hypervisor. -bash-3.2# ldm unbind-domain ldom2clone -bash-3.2# ldm ls-domain NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -n-cv- SP 14 4128M 6.8% 8h 8m ldom2 active -t---- 5000 2 6G 50% 5m ldom2clone inactive ------ 8 8G -bash-3.2# 4.Now you can remove the inactive guest domain from the hypervisor. -bash-3.2# ldm remove-domain ldom2clone -bash-3.2# ldm ls-domain NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -n-cv- SP 14 4128M 0.2% 8h 10m ldom2 active -t---- 5000 2 6G 50% 7m -bash-3.2# You can see ldom2clone has been removed from the hypervisor completely. If you want to remove the specific resources from the guest domain,follow the below steps. 1. List the bounded resources for one of the guest domain. (ldom2) -bash-3.2# ldm list-bindings ldom2 NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME ldom2 active -t---- 5000 2 6G 50% 8m
MEMORY RA PA SIZE 0x10000000 0x10000000 4G 0x480000000 0x390000000 2G
VARIABLES auto-boot?=false boot-device=disk:a disk net keyboard-layout=US-English
NETWORK NAME SERVICE ID DEVICE MAC MODE PVID VID MTU LINKPROP vnet0 primary-vds0@primary 0 network@0 00:14:4f:f9:53:57 1 1500 PEER MAC MODE PVID VID MTU LINKPROP primary-vds0@primary 00:14:4f:fb:e2:f3 1 1500
DISK NAME VOLUME TOUT ID DEVICE SERVER MPGROUP vdisk1_dom2 vol1@primary-vds0 1 disk@1 primary iso_ldom2 iso0@primary-vds0 2 disk@2 primary uavdisk uavol1@primary-vds0 0 disk@0 primary
VCONS NAME SERVICE PORT ldom2 primary-vcc0@primary 5000 2.Try to remove the one of the virtual disk resource from the guest domain. -bash-3.2# ldm remove-vdisk uavdisk ldom2 Please perform the operation while the LDom is bound or inactive -bash-3.2# ldm remove-vdisk -f uavdisk ldom2 Please perform the operation while the LDom is bound or inactive You are not able to remove the virtual disk since its in use on ldom2. 3.You can stop the guest domain and then remove it . -bash-3.2# ldm stop-domain ldom2 LDom ldom2 stopped -bash-3.2# ldm remove-vdisk uavdisk ldom2 -bash-3.2# -bash-3.2# ldm list-bindings ldom2 NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME ldom2 bound ------ 5000 2 6G
MEMORY RA PA SIZE 0x10000000 0x10000000 4G 0x480000000 0x390000000 2G
VARIABLES auto-boot?=false boot-device=disk:a disk net keyboard-layout=US-English
NETWORK NAME SERVICE ID DEVICE MAC MODE PVID VID MTU LINKPROP vnet0 primary-vds0@primary 0 network@0 00:14:4f:f9:53:57 1 1500 PEER MAC MODE PVID VID MTU LINKPROP primary-vds0@primary 00:14:4f:fb:e2:f3 1 1500
DISK NAME VOLUME TOUT ID DEVICE SERVER MPGROUP vdisk1_dom2 vol1@primary-vds0 1 disk@1 primary iso_ldom2 iso0@primary-vds0 2 disk@2 primary
VCONS NAME SERVICE PORT ldom2 primary-vcc0@primary 5000
-bash-3.2# 4. Removing the devices from the vds service -bash-3.2# ldm list-services VCC NAME LDOM PORT-RANGE primary-vcc0 primary 5000-5100
VSW NAME LDOM MAC NET-DEV ID DEVICE LINKPROP DEFAULT-VLAN-ID PVID VID MTU MODE primary-vds0 primary 00:14:4f:fb:e2:f3 nxge0 0 switch@0 1 1 1500 primary-vsw0 primary 00:14:4f:fa:bf:27 nxge1 1 switch@1 1 1 1500
-bash-3.2# We have seen how to remove the assigned virtual disk from the guest and removing the disk from the virtual disk server. Now we will see how to remove the complete service from the service domain aka primary domain. 1.Try to remove the virtual switch service from service domain -bash-3.2# ldm remove-vsw primary-vds0 Service is active and being used by bound client(s) Failed to remove virtual switch 2.The above command failed because,that virtual service is used for some of the guest domains. First remove the those assigned virtual resources from guest domain the n try to remove it. -bash-3.2# ldm remove-vsw primary-vsw0 -bash-3.2# ldm list-services VCC NAME LDOM PORT-RANGE primary-vcc0 primary 5000-5100
VSW NAME LDOM MAC NET-DEV ID DEVICE LINKPROP DEFAULT-VLAN-ID PVID VID MTU MODE primary-vds0 primary 00:14:4f:fb:e2:f3 nxge0 0 switch@0 1 1 1500
VDS NAME LDOM VOLUME OPTIONS MPGROUP DEVICE primary-vds0 primary vol1 /dev/zvol/dsk/pool1/vol1 iso0 /export/home/sol-10-u10- ga2-sparc-dvd.iso clonevol1 /dev/zvol/dsk/pool1/clonevol1 vol0 /dev/dsk/c3t600144F0B8E706DA00005300F4270004d0s2 vol2 /dev/dsk/c3t600144F0C1135B490000530383D70006d0 Using the same setup commands (showed in this article), you can remove the other guest domains and virtual services. Once you have cleared the all services and domains, you can uninstall the oracle VM for SPARC software using below method. bash-3.2# pkgrm SUNWldm
The following package is currently installed: SUNWldm LDoms Manager software (sparc.sun4v) 2.0,REV=2010.09.02.10.08
Do you want to remove this package? [y,n,?,q] y
## Removing installed package instance
This package contains scripts which will be executed with super-user permission during the process of removing this package.
Do you want to continue with the removal of this package [y,n,?,q] y ## Verifying package dependencies in global zone ## Processing package information. ## Removing pathnames in class Waiting up to 60 seconds for svc:/ldoms/ldmd:default to stop... ## Removing pathnames in class /var/svc/manifest/platform/sun4v /var/svc/manifest/platform /var/svc/manifest /var/svc <<<<<<<<>>>> /opt/SUNWldm/bin/schemas/ovf-strings.xsd /opt/SUNWldm/bin/schemas/ovf-section.xsd /opt/SUNWldm/bin/schemas/ovf-envelope.xsd /opt/SUNWldm/bin/schemas/ovf-core.xsd /opt/SUNWldm/bin/schemas/ldmd-property.xsd /opt/SUNWldm/bin/schemas/ldmd-binding.xsd /opt/SUNWldm/bin/schemas/event-v3.xsd /opt/SUNWldm/bin/schemas/combined-v3.xsd /opt/SUNWldm/bin/schemas/cli-list-constraint-v3.xsd /opt/SUNWldm/bin/schemas/cim-vssd.xsd /opt/SUNWldm/bin/schemas/cim-rasd.xsd /opt/SUNWldm/bin/schemas/cim-common.xsd /opt/SUNWldm/bin/schemas /opt/SUNWldm/bin/ldmd_start /opt/SUNWldm/bin/ldmd /opt/SUNWldm/bin/ldmconfig /opt/SUNWldm/bin/ldm /opt/SUNWldm/bin /opt/SUNWldm /opt ## Updating system information.
Removal of was successful. bash-3.2# pkgrm SUNWldomu
The following package is currently installed: SUNWldomu Solaris Logical Domains (Usr) (sparc.sun4v) 11.10.0,REV=2006.08.08.12.13
Do you want to remove this package? [y,n,?,q] y
## Removing installed package instance ## Verifying package dependencies in global zone ## Processing package information. ## Removing pathnames in class /usr/lib/sparcv9/libpri.so.1 /usr/lib/sparcv9/libds.so.1 /usr/lib/sparcv9 /usr/lib/libpri.so.1 /usr/lib/libds.so.1 /usr/lib/ldoms/vntsd /usr/lib/ldoms/ldmad /usr/lib/ldoms/drd /usr/lib/ldoms /usr/lib /usr ## Updating system information.
Removal of was successful. bash-3.2# pkgrm SUNWldomr
The following package is currently installed: SUNWldomr Solaris Logical Domains (Root) (sparc.sun4v) 11.10.0,REV=2006.10.04.00.26
Do you want to remove this package? [y,n,?,q] y
## Removing installed package instance
This package contains scripts which will be executed with super-user permission during the process of removing this package.
Do you want to continue with the removal of this package [y,n,?,q] y ## Verifying package dependencies in global zone ## Processing package information. ## Executing preremove script. ## Removing pathnames in class Waiting up to 60 seconds for svc:/ldoms/vntsd:default to stop... Waiting up to 60 seconds for svc:/ldoms/agents:default to stop... Waiting up to 60 seconds for svc:/platform/sun4v/drd:default to stop... ## Removing pathnames in class /var/svc/manifest/platform/sun4v /var/svc/manifest/platform /var/svc/manifest /var/svc /var /platform/sun4v/kernel/misc/sparcv9/platsvc /platform/sun4v/kernel/misc/sparcv9/ldc /platform/sun4v/kernel/misc/sparcv9/fault_iso /platform/sun4v/kernel/misc/sparcv9/ds /platform/sun4v/kernel/misc/sparcv9/dr_mem /platform/sun4v/kernel/misc/sparcv9/dr_io /platform/sun4v/kernel/misc/sparcv9/dr_cpu /platform/sun4v/kernel/misc/sparcv9 /platform/sun4v/kernel/misc /platform/sun4v/kernel/drv/sparcv9/vsw /platform/sun4v/kernel/drv/sparcv9/vnet /platform/sun4v/kernel/drv/sparcv9/vlds /platform/sun4v/kernel/drv/sparcv9/vldc /platform/sun4v/kernel/drv/sparcv9/vds /platform/sun4v/kernel/drv/sparcv9/vdc /platform/sun4v/kernel/drv/sparcv9/vcc /platform/sun4v/kernel/drv/sparcv9/ds_snmp /platform/sun4v/kernel/drv/sparcv9/ds_pri /platform/sun4v/kernel/drv/sparcv9/drctl /platform/sun4v/kernel/drv/sparcv9/cnex /platform/sun4v/kernel/drv/sparcv9 /platform/sun4v/kernel/drv/ds_snmp.conf /platform/sun4v/kernel/drv/ds_pri.conf /platform/sun4v/kernel/drv/drctl.conf /platform/sun4v/kernel/drv /platform/sun4v/kernel /platform/sun4v /platform /lib/svc/method/svc-vntsd /lib/svc/method/svc-drd /lib/svc/method/ldoms-agents /lib/svc/method /lib/svc /lib ## Updating system information.
Removal of was successful. bash-3.2# pkginfo | grep -i ldom application SUNWldmp2v LDoms P2V tool bash-3.2# pkgrm SUNWldmp2v
The following package is currently installed: SUNWldmp2v LDoms P2V tool (sparc) 2.0,REV=2010.09.02.10.08
Do you want to remove this package? [y,n,?,q] y
## Removing installed package instance
This package contains scripts which will be executed with super-user permission during the process of removing this package.
Do you want to continue with the removal of this package [y,n,?,q] y ## Verifying package dependencies in global zone ## Processing package information. ## Removing pathnames in class ## Removing pathnames in class /usr/share/man/man1m/ldmp2v.1m /usr/share/man/man1m /usr/share/man /usr/share /usr/sbin/ldmp2v /usr/sbin /usr /opt/SUNWldmp2v/man/man1m/ldmp2v.1m /opt/SUNWldmp2v/man/man1m /opt/SUNWldmp2v/man /opt/SUNWldmp2v/lib/ldmp2v_convert /opt/SUNWldmp2v/lib /opt/SUNWldmp2v/etc/ldmp2v.conf.sample /opt/SUNWldmp2v/etc /opt/SUNWldmp2v/bin/ldmp2v_prepare /opt/SUNWldmp2v/bin /opt/SUNWldmp2v /opt ## Updating system information.
Removal of was successful. bash-3.2# pkginfo | grep -i ldom* bash-3.2# pkginfo | grep -i ldm* bash-3.2# You have successfully uninstall the oracle VM for SPARC software. You can safely recycle the control domain to remove the complete ldom stuffs. Thank you for visiting UnixArena.
LDOM console write access issue Solaris March 27, 2014 in LDOM
Now a days i have mostly dealing with oracle VM for SPARC on T-4 servers.(LDOM). As we all know that hardware console can be writable for only one user at a time and if you take the second session , it will automatically connect in read only mode unless you use the force option. The same way oracle VM for SPARC console also behaves even though its a virtual consoles on the hypervisor levels. Sometimes , the first console session may terminate unexpectedly due to network issues and if you try take the console for second time ,you may get error that You do not have write access on the console screen. root@sun5#ldm list-domain NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -n-cv- SP 8 2G 0.3% 2h 34m ua1 active -n---- 1 12 3G 0.1% 1h 37m ua2 active -n---- 2 12 2560M 0.1% 1h 37m root@sun5# root@sun5#telnet 127.0.0.1 1 Trying 127.0.0.1... Connected to 127.0.0.1. Escape character is '^]'.
Connecting to console "ua1" in group "ua1" .... Press ~? for control options .. You do not have write access You do not have write access You do not have write access You do not have write access You do not have write access You do not have write access You do not have write access You do not have write access You do not have write access
telnet> quit Connection to 127.0.0.1 closed. To fix this issue, login to the control domain and list the telnet session. root@sun5#ps -ef |grep telnet root 6259 6118 0 12:07:22 pts/4 0:00 telnet 127.0.0.1 1 root 6035 374 0 12:01:21 ? 0:00 /usr/sbin/in.telnetd root 1645 374 0 09:47:55 ? 0:00 /usr/sbin/in.telnetd root 5867 374 0 11:57:25 ? 0:00 /usr/sbin/in.telnetd root 6004 5870 0 12:00:49 pts/1 0:00 telnet localhost 2 root 6262 6038 0 12:07:46 pts/2 0:00 grep telnet Kill the process which is associated to the guest domain. In this case , i am having issue with ua1 guest domain which has the console port-1.Let me terminate the process of that and try to connect the console again. root@sun5#kill -9 6259 root@sun5#telnet 127.0.0.1 1 Trying 127.0.0.1... Connected to 127.0.0.1. Escape character is '^]'.
Connecting to console "ua1" in group "ua1" .... Press ~? for control options ..
ua1 console login: root Password: Mar 24 13:44:13 ua1 login: ROOT LOGIN /dev/console Last login: Mon Mar 24 13:16:41 from 192.168.0.117 Oracle Corporation SunOS 5.11 11.1 September 2012 root@ua1:~# logout
ua1 console login: telnet> quit Connection to 127.0.0.1 closed. root@sun5# Awesome .We have fixed the issue .This is small trick but you need to remember this when you face this kind of issues while connecting the console. Share it ! comment it !! Be sociable !!!
How to configure IPMP on Solaris 10 ? June 26, 2013 in Solaris Networking
This article describes how to configure link based IPMP interfaces in Solaris 10. IPMP eliminates single network card failure and it ensures system will be always accessible via network.You can also configure the failure detection seconds in /etc/default/mpathd file and the default value is 10 seconds.In this file there is an option called FAILBACK to specify IP behavior when primary interface recovered from the fault. in.mpathd is a daemon which handles IPMP (Internet Protocol Multi-Pathing) operations.There are two type of IPMP configuration available in Solaris 10.
1.Link Based IPMP The link based IPMP detects network errors by checking the IFF_RUNNING flag.Normally it doesnt require any test IP address like probe based IPMP. 2.Probe Based IPMP The probe based IPMP detects network errors by sending ICMP ECHO_REQUEST messages.It requires test IP Addresses unlike link based IPMP. 1.Link Based IPMP Request: Configure IP address 192.168.2.50 on e1000g1 & e1000g2 using Link based IPMP.
Step:1 Find out the installed NICs on the systems and its status.Verify the ifconfig output as well. Make sure the NIC status are up and not in use. Arena-Node1#dladm show-dev e1000g0 link: up speed: 1000 Mbps duplex: full e1000g1 link: up speed: 1000 Mbps duplex: full e1000g2 link: up speed: 1000 Mbps duplex: full Arena-Node1#ifconfig -a lo0: flags=2001000849 mtu 8232 index 1 inet 127.0.0.1 netmask ff000000 e1000g0: flags=1000843 mtu 1500 index 2 inet 192.168.2.5 netmask ffffff00 broadcast 192.168.2.255 ether 0:c:29:ec:b3:af Arena-Node1#
Step:2 Add the IP address in /etc/hosts and specify the netmask value in /etc/netmasks like below one. Arena-Node1#cat /etc/hosts |grep 192.168.2.50 192.168.2.50 arenagroupIP Arena-Node1#cat /etc/netmasks |grep 192.168.2 192.168.2.0 255.255.255.0 Arena-Node1#eeprom "local-mac-address?=true"
Step:3 Plumb the interfaces which you are going to use for new IP address. check the status in ifconfig output. Arena-Node1#ifconfig e1000g1 plumb Arena-Node1#ifconfig e1000g2 plumb Arena-Node1#ifconfig -a lo0: flags=2001000849 mtu 8232 index 1 inet 127.0.0.1 netmask ff000000 e1000g0: flags=1000843 mtu 1500 index 2 inet 192.168.2.5 netmask ffffff00 broadcast 192.168.2.255 ether 0:c:29:ec:b3:af e1000g1: flags=1000842 mtu 1500 index 3 inet 0.0.0.0 netmask 0 ether 0:c:29:ec:b3:b9 e1000g2: flags=1000842 mtu 1500 index 4 inet 0.0.0.0 netmask 0 ether 0:c:29:ec:b3:c3
Step:4 Configure IP on Primary interface and add the interfaces to IPMP group with your own group name. Arena-Node1#ifconfig e1000g1 192.168.2.50 netmask 255.255.255.0 broadcast + up Arena-Node1#ifconfig e1000g1 e1000g1: flags=1000843 mtu 1500 index 3 inet 192.168.2.50 netmask ffffff00 broadcast 192.168.2.255 ether 0:c:29:ec:b3:b9
Arena-Node1#ifconfig e1000g1 group arenagroup-1 Arena-Node1#ifconfig e1000g2 group arenagroup-1 Arena-Node1#ifconfig -a lo0: flags=2001000849 mtu 8232 index 1 inet 127.0.0.1 netmask ff000000 e1000g0: flags=1000843 mtu 1500 index 2 inet 192.168.2.5 netmask ffffff00 broadcast 192.168.2.255 ether 0:c:29:ec:b3:af e1000g1: flags=1000843 mtu 1500 index 3 inet 192.168.2.50 netmask ffffff00 broadcast 192.168.2.255 groupname arenagroup-1 ether 0:c:29:ec:b3:b9 e1000g2: flags=1000842 mtu 1500 index 4 inet 0.0.0.0 netmask 0 groupname arenagroup-1 ether 0:c:29:ec:b3:c3
Step:5 Now we have to ensure IPMP is working fine.This can be done in two ways.
i.Test:1 Remove the primary LAN cable and check it.Here i have removed the LAN cable from e1000g1 and let see what happens. Arena-Node1#ifconfig -a lo0: flags=2001000849 mtu 8232 index 1 inet 127.0.0.1 netmask ff000000 e1000g0: flags=1000843 mtu 1500 index 2 inet 192.168.2.5 netmask ffffff00 broadcast 192.168.2.255 ether 0:c:29:ec:b3:af e1000g1: flags=19000802 mtu 0 index 3 inet 0.0.0.0 netmask 0 groupname arenagroup-1 ether 0:c:29:ec:b3:b9 e1000g2: flags=1000842 mtu 1500 index 4 inet 0.0.0.0 netmask 0 groupname arenagroup-1 ether 0:c:29:ec:b3:c3 e1000g2:1: flags=1000843 mtu 1500 index 4 inet 192.168.2.50 netmask ffffff00 broadcast 192.168.2.255 Again i have connected back the LAN cable to e1000g1. Arena-Node1#dladm show-dev e1000g0 link: up speed: 1000 Mbps duplex: full e1000g1 link: up speed: 1000 Mbps duplex: full e1000g2 link: up speed: 1000 Mbps duplex: full Arena-Node1#ifconfig -a lo0: flags=2001000849 mtu 8232 index 1 inet 127.0.0.1 netmask ff000000 e1000g0: flags=1000843 mtu 1500 index 2 inet 192.168.2.5 netmask ffffff00 broadcast 192.168.2.255 ether 0:c:29:ec:b3:af e1000g1: flags=1000843 mtu 1500 index 3 inet 192.168.2.50 netmask ffffff00 broadcast 192.168.2.255 groupname arenagroup-1 ether 0:c:29:ec:b3:b9 e1000g2: flags=1000842 mtu 1500 index 4 inet 0.0.0.0 netmask 0 groupname arenagroup-1 ether 0:c:29:ec:b3:c3 Here the configured IP is going back to original interface where it was running before. Here I had specified FALLBACK=yes . Thats why IP is moving back to original interface.The same way you can also specify failure detection time to mpathd using parameter FAILURE_DETECTION_TIME in ms. Arena-Node1#cat /etc/default/mpathd |grep -v "#" FAILURE_DETECTION_TIME=10000 FAILBACK=yes TRACK_INTERFACES_ONLY_WITH_GROUPS=yes Arena-Node1#
ii.Test:2 Normally most of the Unix admins will be sitting in remote site. So you will be not able to perform the above test.In this case ,you can use if_mpadm command to disable the interface in OS level. Fist i am going to disable e1000g1 and let see what happens. Arena-Node1#if_mpadm -d e1000g1 Arena-Node1#ifconfig -a lo0: flags=2001000849 mtu 8232 index 1 inet 127.0.0.1 netmask ff000000 e1000g0: flags=1000843 mtu 1500 index 2 inet 192.168.2.5 netmask ffffff00 broadcast 192.168.2.255 ether 0:c:29:ec:b3:af e1000g1: flags=89000842 mtu 0 index 3 inet 0.0.0.0 netmask 0 groupname arenagroup-1 ether 0:c:29:ec:b3:b9 e1000g2: flags=1000842 mtu 1500 index 4 inet 0.0.0.0 netmask 0 groupname arenagroup-1 ether 0:c:29:ec:b3:c3 e1000g2:1: flags=1000843 mtu 1500 index 4 inet 192.168.2.50 netmask ffffff00 broadcast 192.168.2.255
Now i am going to enable it back. Arena-Node1#if_mpadm -r e1000g1 Arena-Node1#ifconfig -a lo0: flags=2001000849 mtu 8232 index 1 inet 127.0.0.1 netmask ff000000 e1000g0: flags=1000843 mtu 1500 index 2 inet 192.168.2.5 netmask ffffff00 broadcast 192.168.2.255 ether 0:c:29:ec:b3:af e1000g1: flags=1000843 mtu 1500 index 3 inet 192.168.2.50 netmask ffffff00 broadcast 192.168.2.255 groupname arenagroup-1 ether 0:c:29:ec:b3:b9 e1000g2: flags=1000842 mtu 1500 index 4 inet 0.0.0.0 netmask 0 groupname arenagroup-1 ether 0:c:29:ec:b3:c3 The same way you can manually failover the IP to one interface to another interface.
In the both tests,we can clearly see IP is moving from e1000g1 to e1000g2 automatically without any issues.So we have successfully configured Link based IPMP on Solaris. These failover logs will be logged in /var/adm/messages like below. Jun 26 20:57:24 node1 in.mpathd[3800]: [ID 215189 daemon.error] The link has gone down on e1000g1 Jun 26 20:57:24 node1 in.mpathd[3800]: [ID 594170 daemon.error] NIC failure detected on e1000g1 of group arenagroup-1 Jun 26 20:57:24 node1 in.mpathd[3800]: [ID 832587 daemon.error] Successfully failed over from NIC e1000g1 to NIC e1000g2 Jun 26 20:57:57 node1 in.mpathd[3800]: [ID 820239 daemon.error] The link has come up on e1000g1 Jun 26 20:57:57 node1 in.mpathd[3800]: [ID 299542 daemon.error] NIC repair detected on e1000g1 of group arenagroup-1 Jun 26 20:57:57 node1 in.mpathd[3800]: [ID 620804 daemon.error] Successfully failed back to NIC e1000g1 Jun 26 21:03:59 node1 in.mpathd[3800]: [ID 832587 daemon.error] Successfully failed over from NIC e1000g1 to NIC e1000g2 Jun 26 21:04:07 node1 in.mpathd[3800]: [ID 620804 daemon.error] Successfully failed back to NIC e1000g1 To make the above work persistent across the reboot create the configuration files for both the network interfaces. Arena-Node1#cat /etc/hostname.e1000g1 arenagroupIP netmask + broadcast + group arenagroup up Arena-Node1#cat /etc/hostname.e1000g2 group arenagroup up Thank you for reading this article. Please leave a comment if its useful for you.