Anda di halaman 1dari 10

VMware vSphere - Scripted Installer for ESX Server

Wednesday, 05 August 2009 11:10 - Last Updated Wednesday, 05 August 2009 12:53

The ESX 4.0 Server is VMware's customised operating system (a linux-based distribution) that
is installed onto a server to provide an interface between the virtual machines and the physical
hardware.  It also consists of a small VM called the Service Console that is used to both control
the VM's running on that host, but also to talk to wider vSphere cluster.

The install for ESX couldn't be much simpler.  You have to make some choices regarding the
sizes of the various disk partitions, much like any other Linux install, set some networking
options, and off you go.

The problem comes a bit later in the game, because to make some of the magic work, all the
hosts have to have exactly the same settings for their networking, virtual switches and storage
controllers, and they all need to have hosts files pointing at each other and the Virtual Center
managing it all.  Remembering to do all that when it's 9pm on a Friday evening and you need to
build a new host on the quick is not all that reliable.

Read on to see how this can be overcome...

Automatic Installation - Single Host

When you've installed an ESX server with the graphical installer, it leaves a small config file in
root's home directory called ks.cfg.  This is a record of all the choices made during setup.  The
easiest (but least-powerful) way to get a "recovery" plan in place is to just pop that file on a USB
memory stick, and then the next time you need to re-create the ESX host, just pop that in and
select the "USB ks.cfg" option when the DVD boots up.

The problems with that are two-fold:

- It has the server name, IP address and other device-specific settings hard-wired into it

1 / 10
VMware vSphere - Scripted Installer for ESX Server
Wednesday, 05 August 2009 11:10 - Last Updated Wednesday, 05 August 2009 12:53

- Doesn't cover any of the configuration past the creation of the disks and the installation of
the system

To overcome some of these issues, we need to delve deeper into the ks.cfg, and what it really
is...

Automatic Installation - Many Hosts

The problem with multiple-hosts (and really, who has just one ESX host?) is that we need
different IP addresses and names for all sorts of things in the configuration.  This is where a
more complex ks.cfg can help.

In this example, I'll describe the case for four hosts, which will be numbered 1 to 4.

ESXID Host name IP address


1 ESX-01 10.20.30.1
2 ESX-02 10.20.30.2
3 ESX-03 10.20.30.3
4 ESX-04 10.20.30.4

There's no easy way to do this, so what happens is kind of like this

1. Insert in the USB stick with the configuration we're going to create on it
2. Insert the ESX bootable DVD and boot the server
3. Select (but DO NOT press enter yet) the option "ESX Scripted Install using USB ks.cfg"
4. Press [F2]
5. At the end of the command-line, enter ESXID=X, where X is the ESXID of the server
being built
6. Press [Enter]

What happens now is all of this:

1. The server boots into install mode


2. The config's PRE section creates the network configuration file
3. The config's PRE section creates the disk configuration file

2 / 10
VMware vSphere - Scripted Installer for ESX Server
Wednesday, 05 August 2009 11:10 - Last Updated Wednesday, 05 August 2009 12:53

4. The config's PRE section creates a call-script.sh to save the ESXID for later use

5. The installer reads the main config, the contents of the network and the contents of the
disk config files
6. The config's POST section copies the new call-script.sh to the new installation
7. The config's POST section copies the esx-post-script.sh to the new installation
8. The installer calls call-script.sh from the new installation
9. call-script.sh calls esx-post-script.sh X, where X is the ESXID
10. esx-post-script.sh creates a new customised script esxcfg.sh
11. esx-post-script.sh backs up the current /etc/rc.init file
12. esx-post-script.sh puts esxcfg.sh into rc.init
13. The installer is finished, and so reboots the system
14. rc.init now calls esxcfg.sh, which contains customised configuration
15. rc.init then copies the original rc.init back over itself
16. Done

ks.cfg - The Kickstart File

The ks.cfg file is not actually a simple configuration file as one might expect.  The installation
process is actually a scripted interface with quite a lot of power, but not quite enough to easily
utilised!

The basic part of ks.cfg is the simple and generally-consistent stuff like time-zones, keyboard
layouts, eula's, etc

{geshibot lang="bash" head="ks.cfg Basic Information"}# Regional Settings


keyboard uk
timezone --utc 'Europe/London'

# Installation settings
reboot
install cdrom

# Unencrypted root password (replace by your own)


rootpw --iscrypted $1$6uSi8SxC$Gvr0QbKV.4DZhgmT07qCj/

# Licensing
accepteula
serialnum --esx=50090-0H04Q-18C39-0F8K2-9JCN0

# Authentication

3 / 10
VMware vSphere - Scripted Installer for ESX Server
Wednesday, 05 August 2009 11:10 - Last Updated Wednesday, 05 August 2009 12:53

auth --enablemd5 --enableshadow

# Partitioning
%incude /tmp/diskconfig

# Network
%include /tmp/networkconfig
{/geshibot}

The clever part starts in the last two items there.  You can't have dynamic stuff in the normal
configuration section, but you can include other configuration files.  So what's happening here is
that we're telling the installer to include two files, diskconfig and networkconfig, which will just
contain more options.

What's the point of that, you ask?  Well, whilst we can't be dynamic in the main config files,
there are additional magical sections to this file that can be dynamic.  It's in these sections that
we'll generate those configs dynamically.

The PRE section

The special marker "%pre" identifies a section that should be executed before the installer runs
the actual configuration file.  The thing that makes this work is that anything in it can be bash
script.

So, the first thing we do is define a %pre section, read in the options that were given at the
installer prompt, and then go on to create the above two configuration files.

{geshibot lang="bash" head="ks.cfg PRE"}# This is the PRE section


$pre --interpreter=bash

set -- `cat /proc/cmdline`


for I in $*; do case "$I" in *=*) eval $I;; esac; done
ESXCODE=`printf "%02dn" "$ESXID"`
ESXHOSTNAME="ESX-$ESXCODE"
ESXIP="10.20.30.$ESXID"{/geshibot}

4 / 10
VMware vSphere - Scripted Installer for ESX Server
Wednesday, 05 August 2009 11:10 - Last Updated Wednesday, 05 August 2009 12:53

What we've done here is quite simple.

- Line 2 tells the installer to start a new PRE section, and to use bash to execute it.
- Line 4 and Line 5 take the input specified on bootup, and turns them all into variables. 
This means we now have an ESXID variable, which will be populated with a number.
- Lines 6, 7 and 8 take the single-digit ESXID and format it in various ways to get the
hostname, and ipaddress.

Now, we can't just run the networking and partitioning options within this PRE section, because
we're in a bash shell, so it doesn't know the commands. So what we do is create the files in
here, then include them like we saw above.

{geshibot lang="bash" head="ks.cfg PRE"}# Create the disk configuration


cat << EOFdisk >> /tmp/diskconfig
clearpart --alldrives --overwritevmfs
part '/boot' --fstype=ext3 --size=1100 --onfirstdisk
part 'none' --fstype=vmkcore --size=110 --onfirstdisk
part 'LocalStorage-${ESXHOSTNAME}' --fstype=vmfs3 --size=8804 --grow --onfirstdisk
virtualdisk 'esxconsole' --size=7804 --onvmfs='LocalStorage-${ESXHOSTNAME}'
part 'swap' --fstype=swap --size=800 --onvirtualdisk='esxconsole'
part '/var/log' --fstype=ext3 --size=2000 --onvirtualdisk='esxconsole'
part '/' --fstype=ext3 --size=5000 --grow --onvirtualdisk='esxconsole'
EOFdisk

# Create the networking config


cat << EOFnic >> /tmp/networkconfig
network --ip=${ESXSCIP} --hostname=${ESXHOSTNAME} --nameserver=${ESXSCIP}
--netmask=255.255.0.0 --gateway=10.20.0.1 --addvmportgroup=true --device=vmnic1
--bootproto=static
EOFnic{/geshibot}

This will

- Line 2: Create a new /tmp/diskconfig file.


- Lines 3-10: output a bunch of part commands.
- Line 13: Create a new /tmp/networkconfig file.

5 / 10
VMware vSphere - Scripted Installer for ESX Server
Wednesday, 05 August 2009 11:10 - Last Updated Wednesday, 05 August 2009 12:53

- Line 15: output the network command.

The last (and confusing) bit in the PRE section is required because later on when we switch into
the POST section, we loose access to the ESXID.  So what we do is create a small
runner-script which will call out main post-configuration script later with the ESXID as a
parameter:

{geshibot lang="bash" head="ks.cfg PRE"}# The ESXID var is lost later, so keep it somewhere
cat << EOFcs >> /call-script.sh
/esx-post-script.sh ${ESXID}
EOFcs
chmod a+x /call-script.sh{/geshibot}

Now we move onto the two POST sections.  These are run after all the packages are installed,
and we also have access to the new filesystem that the ESX box will be running.

The POST Section

The first POST section is small, and all it does is copy the call-script.sh that we created in the
PRE section (which has our ESXID in it) and our main esx-post-script.sh (which we'll come to a
bit later), into the real soon-to-be-running filesystem.

{geshibot lang="bash" head="ks.cfg POST 1"}# This is the POST section


%post --interpreter=bash --nochroot
cp /call-script.sh /mnt/sysimage/call-script.sh
chmod a+x /mnt/sysimage/call-script.sh

cp /mnt/usbdisk/esx-post-script.sh /mnt/sysimage/esx-post-script.sh
chmod a+x /mnt/sysimage/esx-post-script.sh{/geshibot}

The main bit of cleverness here is that in line 2, where we start the section, we're telling the
installer not to run the script as if it's on the live system, but to stay in the installer environment. 
That gives us access to both the installer config files (in /mnt/usbdrive) and also the production

6 / 10
VMware vSphere - Scripted Installer for ESX Server
Wednesday, 05 August 2009 11:10 - Last Updated Wednesday, 05 August 2009 12:53

filesystem (in /mnt/systemdrive).  That means we can copy stuff to the production system!

The second, normal, POST section will be executed by the installer in a chrooted environment,
so it'll think it's actually running on the real production operating system.  The key thing here
though is that we don't actually have a running ESX server, so none of the esx commands will
work.  So all we do is run the call-script.sh file, which uses the esx-post-script.sh file to create a
script that can be run after the reboot once all the services are running.

{geshibot lang="bash" head="ks.cfg POST 2"}# This is the POST section


/call-script.sh{/geshibot}

What this final step does is

1. call esx-post-script.sh with the ESXID passed as a parameter


2. esx-post-script then creates a final, customised, configuration scipt called esxfg.sh
3. Backs up the current rc.local configuration script

4. Adds a call to esxcfg.sh to the rc.init script (which runs after reboot)

The ESX-POST-SCRIPT script

That's everything that happens before the reboot sorted out.  What we've go so far is a network
configuration that's personalised for the server, a disk-partition and VMDK container that's
personalised for the server, and a configuration-script.

Now that the server is also rebooted, we have full access to all the ESX tools, so we can do the
final configuration.

The sole role of esx-post-script.sh is basically to create a runnable esxcfg.sh file that contains
all the standard ESX commands for setting up a host.

7 / 10
VMware vSphere - Scripted Installer for ESX Server
Wednesday, 05 August 2009 11:10 - Last Updated Wednesday, 05 August 2009 12:53

{geshibot lang="bash" head="esx-post-script"}ESXID=$1


ESXCODE=`printf "%02dn" "$ESXID"`
ESXHOSTNAME="ESX-$ESXCODE"
ESXSCIP="10.20.30.$ESXID"

ESXSTORAGEIP="10.10.30.$ESXID"

cat > /esxcfg.sh <

# MORE CONFIG WILL GO HERE, THAT COMES LATER

EOFcfg

cp -f /etc/rc.d/rc.local /etc/rc.d/rc.local.bak
cat > /etc/rc.d/rc.local << EOFrc
chmod a+x esxcfg.sh
bash /esxcfg.sh > /root/post_install.log
cp -f /etc/rc.d/rc.local /etc/rc.d/rc.postinstall
cp -f /etc/rc.d/rc.local.bak /etc/rc.d/rc.local
EOFrc {/geshibot}

1. esx-post-script.sh creates a new customised script esxcfg.sh


2. esx-post-script.sh backs up the current /etc/rc.init file
3. esx-post-script.sh puts esxcfg.sh into rc.init

The actual configuration

All the rest is pretty much standard ESX config stuff, and goes into the "MORE CONFIG..."
section in the esx-post-script.sh file.  The main exception is that we now have access to the
ESXID variables.

{geshibot lang="bash" head="esxcfg"}# During auto-install a vSwitch0 is created, but I wanted a


different naming
echo "Deleting default vSwitch"
esxcfg-vswif --del vswif0
esxcfg-vswitch --del-pg='Service Console' vSwitch0
esxcfg-vswitch --delete vSwitch0

8 / 10
VMware vSphere - Scripted Installer for ESX Server
Wednesday, 05 August 2009 11:10 - Last Updated Wednesday, 05 August 2009 12:53

# Create the Service Console network


echo "Creating new Service Console switch"
esxcfg-vswitch --add VMwareSwitch
esxcfg-vswitch --link=vmnic1 VMwareSwitch
esxcfg-vswitch --link=vmnic3 VMwareSwitch
esxcfg-vswitch --add-pg='Service Console' VMwareSwitch
esxcfg-vswitch --add-pg="Service Console Management" VMwareSwitch
esxcfg-vswif --add vswif0 --portgroup 'Service Console' --ip $ESXSCIP --netmask 255.255.0.0

# Create the Storage network


echo "Creating Storage vSwitch and port-groups"
esxcfg-vswitch --add StorageSwitch
esxcfg-vswitch --link=vmnic0 StorageSwitch
esxcfg-vswitch --link=vmnic2 StorageSwitch
esxcfg-vswitch --add-pg="Storage Management" StorageSwitch
esxcfg-vswitch --add-pg="Storage" StorageSwitch

# Create the Storage VM Kernel Port, and enable Jumbo frames on it


esxcfg-vmknic -a -i $ESXSTORAGEIP -n 255.255.0.0 -m 9000 Storage
esxcfg-vswitch -m 9000 StorageSwitch

# Create the Production network


echo "Creating Production vSwitch and port-groups"
esxcfg-vswitch --add ProductionSwitch
esxcfg-vswitch --link=vmnic4 ProductionSwitch
esxcfg-vswitch --link=vmnic5 ProductionSwitch
esxcfg-vswitch --link=vmnic6 ProductionSwitch
esxcfg-vswitch --link=vmnic7 ProductionSwitch
esxcfg-vswitch --add-pg="Production" ProductionSwitch
esxcfg-vswitch --add-pg="DMZ" ProductionSwitch
esxcfg-vswitch --add-pg="Testing" ProductionSwitch
esxcfg-vswitch --vlan=40 --pg=DMZ ProductionSwitch
esxcfg-vswitch --vlan=50 --pg=Production ProductionSwitch
esxcfg-vswitch --vlan=60 --pg=Testing ProductionSwitch

# Enable the Software iSCSI adapter


# The adapter name given to the iSCSI HBA was changing sometimes, so
I've
# determined it from the esxcfg-swiscsi command's "scan" subcommand now.
# (note that the inlined functions need to be escaped)
echo "Adding iSCSI interfaces"
esxcfg-swiscsi -e >> /tmp/post.log

9 / 10
VMware vSphere - Scripted Installer for ESX Server
Wednesday, 05 August 2009 11:10 - Last Updated Wednesday, 05 August 2009 12:53

HBANAME=`esxcfg-swiscsi -s | awk '{print $2}' | sed 's/...$//'`


vmkiscsi-tool -D -a 10.10.10.10 $HBANAME >> /tmp/post.log
esxcfg-rescan $HBANAME >> /tmp/post.log

# Activate changes to reflect them into Virtual Center


echo "Restarting the Management interface"
service mgmt-vmware restart

#Configure Firewall and startup services


echo "Configuring firewall"

chkconfig snmpd on
esxcfg-firewall -e ntpClient
esxcfg-firewall -e snmpd
esxcfg-firewall -e sshClient
esxcfg-firewall -e CIMSLP
esxcfg-firewall -e VCB
esxcfg-firewall -e swISCSIClient
esxcfg-firewall -e CIMHttpsServer
esxcfg-firewall -e vpxHeartbeats
esxcfg-firewall -e sshServer
esxcfg-firewall -e updateManager

#Configure hosts files to prevent DNS failure


echo "Configuring hosts file"
echo "127.0.0.1 localhost.localdomain localhost" > /etc/hosts
echo "10.20.30.1 esx-01.mydomain.vm.local esx-01" >> /etc/hosts
echo "10.20.30.2 esx-02.mydomain.vm.local esx-02" >> /etc/hosts
echo "10.20.30.3 esx-03.mydomain.vm.local esx-03" >> /etc/hosts
echo "10.20.30.4 esx-04.mydomain.vm.local esx-04" >> /etc/hosts
echo "10.20.10.1 vc-01.mydomain.com vc-01" >> /etc/hosts {/geshibot}

The full scripts can be downloaded here .

10 / 10

Anda mungkin juga menyukai