Wednesday, 05 August 2009 11:10 - Last Updated Wednesday, 05 August 2009 12:53
The ESX 4.0 Server is VMware's customised operating system (a linux-based distribution) that
is installed onto a server to provide an interface between the virtual machines and the physical
hardware. It also consists of a small VM called the Service Console that is used to both control
the VM's running on that host, but also to talk to wider vSphere cluster.
The install for ESX couldn't be much simpler. You have to make some choices regarding the
sizes of the various disk partitions, much like any other Linux install, set some networking
options, and off you go.
The problem comes a bit later in the game, because to make some of the magic work, all the
hosts have to have exactly the same settings for their networking, virtual switches and storage
controllers, and they all need to have hosts files pointing at each other and the Virtual Center
managing it all. Remembering to do all that when it's 9pm on a Friday evening and you need to
build a new host on the quick is not all that reliable.
When you've installed an ESX server with the graphical installer, it leaves a small config file in
root's home directory called ks.cfg. This is a record of all the choices made during setup. The
easiest (but least-powerful) way to get a "recovery" plan in place is to just pop that file on a USB
memory stick, and then the next time you need to re-create the ESX host, just pop that in and
select the "USB ks.cfg" option when the DVD boots up.
- It has the server name, IP address and other device-specific settings hard-wired into it
1 / 10
VMware vSphere - Scripted Installer for ESX Server
Wednesday, 05 August 2009 11:10 - Last Updated Wednesday, 05 August 2009 12:53
- Doesn't cover any of the configuration past the creation of the disks and the installation of
the system
To overcome some of these issues, we need to delve deeper into the ks.cfg, and what it really
is...
The problem with multiple-hosts (and really, who has just one ESX host?) is that we need
different IP addresses and names for all sorts of things in the configuration. This is where a
more complex ks.cfg can help.
In this example, I'll describe the case for four hosts, which will be numbered 1 to 4.
1. Insert in the USB stick with the configuration we're going to create on it
2. Insert the ESX bootable DVD and boot the server
3. Select (but DO NOT press enter yet) the option "ESX Scripted Install using USB ks.cfg"
4. Press [F2]
5. At the end of the command-line, enter ESXID=X, where X is the ESXID of the server
being built
6. Press [Enter]
2 / 10
VMware vSphere - Scripted Installer for ESX Server
Wednesday, 05 August 2009 11:10 - Last Updated Wednesday, 05 August 2009 12:53
4. The config's PRE section creates a call-script.sh to save the ESXID for later use
5. The installer reads the main config, the contents of the network and the contents of the
disk config files
6. The config's POST section copies the new call-script.sh to the new installation
7. The config's POST section copies the esx-post-script.sh to the new installation
8. The installer calls call-script.sh from the new installation
9. call-script.sh calls esx-post-script.sh X, where X is the ESXID
10. esx-post-script.sh creates a new customised script esxcfg.sh
11. esx-post-script.sh backs up the current /etc/rc.init file
12. esx-post-script.sh puts esxcfg.sh into rc.init
13. The installer is finished, and so reboots the system
14. rc.init now calls esxcfg.sh, which contains customised configuration
15. rc.init then copies the original rc.init back over itself
16. Done
The ks.cfg file is not actually a simple configuration file as one might expect. The installation
process is actually a scripted interface with quite a lot of power, but not quite enough to easily
utilised!
The basic part of ks.cfg is the simple and generally-consistent stuff like time-zones, keyboard
layouts, eula's, etc
# Installation settings
reboot
install cdrom
# Licensing
accepteula
serialnum --esx=50090-0H04Q-18C39-0F8K2-9JCN0
# Authentication
3 / 10
VMware vSphere - Scripted Installer for ESX Server
Wednesday, 05 August 2009 11:10 - Last Updated Wednesday, 05 August 2009 12:53
# Partitioning
%incude /tmp/diskconfig
# Network
%include /tmp/networkconfig
{/geshibot}
The clever part starts in the last two items there. You can't have dynamic stuff in the normal
configuration section, but you can include other configuration files. So what's happening here is
that we're telling the installer to include two files, diskconfig and networkconfig, which will just
contain more options.
What's the point of that, you ask? Well, whilst we can't be dynamic in the main config files,
there are additional magical sections to this file that can be dynamic. It's in these sections that
we'll generate those configs dynamically.
The special marker "%pre" identifies a section that should be executed before the installer runs
the actual configuration file. The thing that makes this work is that anything in it can be bash
script.
So, the first thing we do is define a %pre section, read in the options that were given at the
installer prompt, and then go on to create the above two configuration files.
4 / 10
VMware vSphere - Scripted Installer for ESX Server
Wednesday, 05 August 2009 11:10 - Last Updated Wednesday, 05 August 2009 12:53
- Line 2 tells the installer to start a new PRE section, and to use bash to execute it.
- Line 4 and Line 5 take the input specified on bootup, and turns them all into variables.
This means we now have an ESXID variable, which will be populated with a number.
- Lines 6, 7 and 8 take the single-digit ESXID and format it in various ways to get the
hostname, and ipaddress.
Now, we can't just run the networking and partitioning options within this PRE section, because
we're in a bash shell, so it doesn't know the commands. So what we do is create the files in
here, then include them like we saw above.
This will
5 / 10
VMware vSphere - Scripted Installer for ESX Server
Wednesday, 05 August 2009 11:10 - Last Updated Wednesday, 05 August 2009 12:53
The last (and confusing) bit in the PRE section is required because later on when we switch into
the POST section, we loose access to the ESXID. So what we do is create a small
runner-script which will call out main post-configuration script later with the ESXID as a
parameter:
{geshibot lang="bash" head="ks.cfg PRE"}# The ESXID var is lost later, so keep it somewhere
cat << EOFcs >> /call-script.sh
/esx-post-script.sh ${ESXID}
EOFcs
chmod a+x /call-script.sh{/geshibot}
Now we move onto the two POST sections. These are run after all the packages are installed,
and we also have access to the new filesystem that the ESX box will be running.
The first POST section is small, and all it does is copy the call-script.sh that we created in the
PRE section (which has our ESXID in it) and our main esx-post-script.sh (which we'll come to a
bit later), into the real soon-to-be-running filesystem.
cp /mnt/usbdisk/esx-post-script.sh /mnt/sysimage/esx-post-script.sh
chmod a+x /mnt/sysimage/esx-post-script.sh{/geshibot}
The main bit of cleverness here is that in line 2, where we start the section, we're telling the
installer not to run the script as if it's on the live system, but to stay in the installer environment.
That gives us access to both the installer config files (in /mnt/usbdrive) and also the production
6 / 10
VMware vSphere - Scripted Installer for ESX Server
Wednesday, 05 August 2009 11:10 - Last Updated Wednesday, 05 August 2009 12:53
filesystem (in /mnt/systemdrive). That means we can copy stuff to the production system!
The second, normal, POST section will be executed by the installer in a chrooted environment,
so it'll think it's actually running on the real production operating system. The key thing here
though is that we don't actually have a running ESX server, so none of the esx commands will
work. So all we do is run the call-script.sh file, which uses the esx-post-script.sh file to create a
script that can be run after the reboot once all the services are running.
4. Adds a call to esxcfg.sh to the rc.init script (which runs after reboot)
That's everything that happens before the reboot sorted out. What we've go so far is a network
configuration that's personalised for the server, a disk-partition and VMDK container that's
personalised for the server, and a configuration-script.
Now that the server is also rebooted, we have full access to all the ESX tools, so we can do the
final configuration.
The sole role of esx-post-script.sh is basically to create a runnable esxcfg.sh file that contains
all the standard ESX commands for setting up a host.
7 / 10
VMware vSphere - Scripted Installer for ESX Server
Wednesday, 05 August 2009 11:10 - Last Updated Wednesday, 05 August 2009 12:53
ESXSTORAGEIP="10.10.30.$ESXID"
EOFcfg
cp -f /etc/rc.d/rc.local /etc/rc.d/rc.local.bak
cat > /etc/rc.d/rc.local << EOFrc
chmod a+x esxcfg.sh
bash /esxcfg.sh > /root/post_install.log
cp -f /etc/rc.d/rc.local /etc/rc.d/rc.postinstall
cp -f /etc/rc.d/rc.local.bak /etc/rc.d/rc.local
EOFrc {/geshibot}
All the rest is pretty much standard ESX config stuff, and goes into the "MORE CONFIG..."
section in the esx-post-script.sh file. The main exception is that we now have access to the
ESXID variables.
8 / 10
VMware vSphere - Scripted Installer for ESX Server
Wednesday, 05 August 2009 11:10 - Last Updated Wednesday, 05 August 2009 12:53
9 / 10
VMware vSphere - Scripted Installer for ESX Server
Wednesday, 05 August 2009 11:10 - Last Updated Wednesday, 05 August 2009 12:53
chkconfig snmpd on
esxcfg-firewall -e ntpClient
esxcfg-firewall -e snmpd
esxcfg-firewall -e sshClient
esxcfg-firewall -e CIMSLP
esxcfg-firewall -e VCB
esxcfg-firewall -e swISCSIClient
esxcfg-firewall -e CIMHttpsServer
esxcfg-firewall -e vpxHeartbeats
esxcfg-firewall -e sshServer
esxcfg-firewall -e updateManager
10 / 10