Er.

Rohit Handa
Lecturer, CSE-IT Department
IBM-ICE Program, BUEST Baddi
Topic-Load Balancing, Machine Imaging

Topic-3: Load Balancing and Virtualization

Overloading
People put load on your web site by making lots of requests to it. If the load is too high
(because too many people are visiting), your web site is going to take a performance
hit. Its going to slow down, and with more and more users, it will get slower and
slower until it fails completely.
To get around this, you need more resources. You can either buy a bigger machine to
replace your current server (scale up) or you can buy another small machine to work
alongside your existing server (scale out).

Working of Processor
1. Keeping the database server alive and watching for requests.
2. Taking requests for data from the database.
3. Calling up the database and looking through it for each specific piece of information to
match your query
4. Making changes to the existing database tables (if applicable) for the query that it has
received.
5. Altering the data collected for the query into one cohesive response in the format you
requested, even if it is the default format.
6. Packaging and transferring the collated response to the portion of the system that
requested it.
7. Tracking that this request has been made, by whom, how it went, and what Happened.

Balancing the performance of virtual machines

Mixing of virtual machines whose peak times are different
Mixing of light loaded and heavy loaded machines.
Dont put too many virtual machines on a single server.

Overprovisioning
Overprovisioning means allocating more resources and (normally) more virtual
machines that your host has available to it.
All that space that is not being used at any one given moment when combined form a
significant volume of resources, and all of them when combined are enough to power
several virtual machines.
This is what you call overprovisioning, adding more servers than you have provisioned
for, because you know that you will have spare extra capacity to use with these
servers.

Load Balancing
Load Balancing means the ability to spread the load of processing over a number of
separate systems for an overall performance increase in processing incoming requests.
 Er. Rohit Handa
Lecturer, CSE-IT Department
IBM-ICE Program, BUEST Baddi
Load balancing is a computer networking method for distributing workloads across
multiple computing resources, such as computers, a computer cluster, network links,
central processing units or disk drives.
It represents the ability to transfer any portion of the processing for a system request
to another independent system that will handle it concurrently. Eg. Web/Database
Server.
One characteristic of cloud computing is virtualized network access to a service. No
matter where you access the service, you are directed to the available resources. The
technology used to distribute service requests to resources is referred to as load
balancing.
Load balancing Load balancing is an optimization technique which can be
implemented in hardware or in software.
Using multiple components with load balancing instead of a single component may
increase reliability through redundancy.

Benefits of Load Balancing

Increase resource utilization
Maximize throughput
Lower latency
Reduce response time
Avoid system overload
Increased Reliability

Network resources which can be load balanced:

Network interfaces and services such as DNS, FTP, and HTTP
Connections through intelligent switches
Processing through computer system assignment
Storage resources
Access to application instances

Without load balancing, cloud computing would very difficult to manage. Load
balancing provides the necessary redundancy to make an intrinsically unreliable
system reliable through managed redirection.
It also provides fault tolerance when coupled with a failover mechanism.
Load balancing is nearly always a feature of server farms and computer clusters and
for high availability applications.
A load-balancing system can use different mechanisms to assign service direction.
In the simplest load-balancing mechanisms, the load balancer listens to a network
port for service requests. When a request from a client or service requester arrives, the
load balancer uses a scheduling algorithm to assign where the request is sent.
Typical scheduling algorithms in use today are round robin and weighted round robin,
fastest response time, least connections and weighted least connections, and custom
assignments based on other factors.
A session ticket is created by the load balancer so that subsequent related traffic from
the client that is part of that session can be properly routed to the same resource.
 Er. Rohit Handa
Lecturer, CSE-IT Department
IBM-ICE Program, BUEST Baddi
Without this session record or persistence, a load balancer would not be able to
correctly failover a request from one resource to another.
Persistence can be enforced using session data stored in a database and replicated
across multiple load balancers.
Other methods can use the client's browser to store a client-side cookie or through the
use of a rewrite engine that modifies the URL.
Of all these methods, a session cookie stored on the client has the least amount of
overhead for a load balancer because it allows the load balancer an independent
selection of resources.
The algorithm can be based on a simple round robin system where the next system in
a list of systems gets the request. Round robin DNS is a common application, where IP
addresses are assigned out of a pool of available IP addresses.

Advanced load balancing

Load balancers are workload managers. They determine the current utilization of the
resources in their pool, the response time, the work queue length, connection latency
and capacity, and other factors in order to assign tasks to each resource.
Among the features you find in load balancers are polling resources for their health,
the ability to bring standby servers online (priority activation), workload weighting
based on a resource's capacity (asymmetric loading), HTTP traffic compression, TCP
offload and buffering, security and authentication, and packet shaping using content
filtering and priority queuing.

Application Delivery Controller (ADC)

An Application Delivery Controller (ADC) is a combination load balancer and
application server that is a server placed between a firewall or router and a server farm
providing Web services.
An Application Delivery Controller is assigned a virtual IP address (VIP) that it maps to
a pool of servers based on application specific criteria.
An ADC is a combination network and application layer device. You also may come
across ADCs referred to as a content switch, multilayer switch, or Web switch.
An ADC is considered to be an advanced version of a load balancer as it not only can
provide the features of load balancing, but it conditions content in order to lower the
workload of the Web servers.
Services provided by an ADC include:
o Data compression
o Content caching
o Server health monitoring
o Security
o SSL offload
o Advanced routing based on current conditions
An ADC is considered to be an application accelerator, and is focused on two areas of
technology: network optimization, and an application or framework optimization.
For example, you may find ADC's that are tuned to accelerate ASP.NET or AJAX
applications.
 Er. Rohit Handa
Lecturer, CSE-IT Department
IBM-ICE Program, BUEST Baddi
An architectural layer containing ADCs is described as an Application Delivery
Network (ADN), and is considered to provide WAN optimization services.
Often an ADN is comprised of a pair of redundant ADCs. The purpose of an ADN is to
distribute content to resources based on application specific criteria.
ADN provide a caching mechanism to reduce traffic, traffic prioritization and
optimization, and other techniques.

Topic-4: Understanding Machine Imaging

A mechanism (other than using hypervisor & load balancing) commonly used to
provide system portability, instantiate applications, and provision and deploy systems
in the cloud is through storing the state of systems using a system image.
A system image makes a copy or a clone of the entire computer system inside a single
container such as a file.
The system imaging program is used to make this image and can be used later to
restore a system image.
Some imaging programs can take snapshots of systems, and most allow you to view
the files contained in the image and do partial restores.
A prominent example of a system image and how it can be used in cloud computing
architectures is the Amazon Machine Image (AMI) used by Amazon Web Services to
store copies of a virtual machine.
An AMI is a file system image that contains an operating system, all appropriate device
drivers, and any applications and state information that the working virtual machine
would have.
When you subscribe to AWS, you can choose to use one of its hundreds of canned
AMIs or to create a custom system and capture that system's image to an AMI.
An AMI can be for public use under a free distribution license, for pay-per-use with
operating systems such as Windows, or shared by an EC2 user with other users who
are given the privilege of access.
An Amazon Machine Image (AMI) is a template that contains a software configuration
for your server (for example, an operating system, an application server, and
applications). You specify an AMI when you launch an instance, which is a virtual
server in the cloud. The AMI provides the software for the root volume of the instance.
You can launch as many instances from your AMI as you need.
The AMI file system is not a standard bit-for-bit image of a system that is common to
many disk imaging programs. AMI omits the kernel image and stores a pointer to a
particular kernel that is part of the AWS kernel library. Among the choices are Red Hat
Linux, Ubuntu, Microsoft Windows, Solaris, and others.
Files in AMI are compressed and encrypted, and an XML file is written that describes
the AMI archive.
AMIs are typically stored in your Amazon S3 (Simple Storage System) buckets as a set
of 10MB chunks.
Machine images are sometimes referred to as virtual appliancessystems that are
meant to run on virtualization platforms. Running Virtual Machines are known as
Instances.
AWS EC2 runs on the Xen hypervisor, for example.
 Er. Rohit Handa
Lecturer, CSE-IT Department
IBM-ICE Program, BUEST Baddi
Virtual appliances are provided to the user or customer as files, via either electronic
downloads or physical distribution. The file format most commonly used is the Open
Virtualization Format (OVF).
The Distributed Management Task Force (DMTF) publishes the OVF specification
documentation. Most virtualization vendors, including VMware, Microsoft, Oracle, and
Citrix, support OVF for virtual appliances.
The term virtual appliance is meant to differentiate the software image from an
operating virtual machine. The system image contains the operating system and
applications that create an environment.
Virtual appliances are a subset of the broader class of software appliances. Installation
of a software appliance on a virtual machine and packaging that into an image creates
a virtual appliance. Like software appliances, virtual appliances are intended to
eliminate the installation, configuration and maintenance costs associated with
running complex stacks of software.
A virtual appliance is not a complete virtual machine platform, but rather a software
image containing a software stack designed to run on a virtual machine platform
which may be a Type 1 or Type 2 hypervisor.
Most virtual appliances are used to run a single application and are configurable from
a Web page.
Virtual appliances are a relatively new paradigm for application deployment, and cloud
computing is the major reason for the interest in them and for their adoption. This
area of WAN application portability and deployment, and of WAN optimization of an
application based on demand, is one with many new participants.

Porting Applications
Cloud computing applications have the ability to run on virtual systems and for these
systems to be moved as needed to respond to demand.
Systems (VMs running applications), storage, and network assets can all be virtualized
and have sufficient flexibility to give acceptable distributed WAN application
performance.
Developers who write software to run in the cloud will undoubtedly want the ability to
port their applications from one cloud vendor to another, but that is a much more
difficult proposition. Cloud computing is a relatively new area of technology, and the
major vendors have technologies that don't interoperate with one another.

VM Migration
Live migration refers to the process of moving a running virtual machine or
application between different physical machines without disconnecting the client or
application.
Memory, storage, and network connectivity of the virtual machine are transferred from
the original host machine to the destination.

VM memory migration
Two techniques for moving the virtual machine's memory state from the source to the
destination are pre-copy memory migration and post-copy memory migration.
1. Pre-copy memory migration
 Er. Rohit Handa
Lecturer, CSE-IT Department
IBM-ICE Program, BUEST Baddi
a. Warm-up phase: In pre-copy memory migration, the Hypervisor typically copies
all the memory pages from source to destination while the VM is still running on
the source. If some memory pages change (become 'dirty') during this process,
they will be re-copied until the rate of re-copied pages is not less than page
dirtying rate.
b. Stop-and-copy phase: After the warm-up phase, the VM will be stopped on the
original host, the remaining dirty pages will be copied to the destination, and the
VM will be resumed on the destination host. The time between stopping the VM
on the original host and resuming it on destination is called "down-time", and
ranges from a few milliseconds to seconds according to the size of memory and
applications running on the VM. There are some techniques to reduce live
migration down-time, such as using probability density function of memory
change.
2. Post-copy memory migration
Post-copy VM migration is initiated by suspending the VM at the source.
With the VM suspended, a minimal subset of the execution state of the VM is
transferred to the target.
The VM is then resumed at the target, even though most of the memory state of the VM
still resides at the source.
At the target, when the VM tries to access pages that have not yet been transferred, it
generates page-faults.
These faults are trapped at the target and redirected towards the source over the
network.
Such faults are referred to as network faults. The source host responds to the network-
fault by sending the faulted page.
Since each page fault of the running VM is redirected towards the source, this
technique can degrade performance of applications running inside the VM.
However, pure demand-paging accompanied with techniques such as pre-paging can
reduce this impact by a great extent.
When down-time of a VM during a live migration is not noticeable by the end user, it is
called a seamless live migration.

Xen Hypervisor
Xen is an open-source type-1 or baremetal hypervisor, which makes it possible to run
many instances of an operating system or indeed different operating systems in
parallel on a single machine (or host).
Xen is the only type-1 hypervisor that is available as open source. Xen is used as the
basis for a number of different commercial and open source applications, such as:
server virtualization, Infrastructure as a Service (IaaS), desktop virtualization, security
applications, embedded and hardware appliances.
The University of Cambridge Computer Laboratory developed the first versions of Xen.
The Xen community develops and maintains Xen as free and open-source software,
subject to the requirements of the GNU General Public License (GPL), version 2.
 Er. Rohit Handa
Lecturer, CSE-IT Department
IBM-ICE Program, BUEST Baddi
Xen is currently available for the IA-32, x86-64 and ARM instruction sets.
Xen runs in a more privileged CPU state than any other software on the machine.
It is responsible for CPU scheduling and memory partitioning of the various virtual
machines running on the hardware device.
Xen boots from a boot loader such as GNU GRUB, and then usually loads a
paravirtualized host operating system into the host domain (dom0).
The system administrator can log into dom0 in order to manage any other guest
operating systems, called "domain U" (domU) in Xen terminology.
The device accesses from all the domU are managed through the dom0.
The hypervisor not only
abstracts the hardware
for the virtual machines
but also controls the
execution of virtual
machines as they share
the common processing
environment.
It has no knowledge of
networking, external
storage devices, video,
or any other common
I/O functions found on
a computing system.
Features:
Small footprint and interface (is around 1MB in size). Because Xen uses a
microkernel design, with a small memory footprint and limited interface to the guest, it
is more robust and secure than other hypervisors.
Operating system agnostic: Most installations run with Linux as the main control
stack (aka "domain 0"). But a number of other operating systems can be used instead,
including NetBSD and OpenSolaris.
Driver Isolation: Xen has the capability to allow the main device driver for a system to
run inside of a virtual machine. If the driver crashes, or is compromised, the VM
containing the driver can be rebooted and the driver restarted without affecting the
rest of the system.
Paravirtualization:
Fully paravirtualized guests
have been optimized to run
as a virtual machine. This
allows the guests to run
much faster than with
hardware extensions (HVM).
Additionally, Xen can run on
hardware that doesn't
support virtualization
extensions.
Xen Architecture
 Er. Rohit Handa
Lecturer, CSE-IT Department
IBM-ICE Program, BUEST Baddi
Components:
1. The Xen Hypervisor:
It is an exceptionally lean (<150,000 lines of code) software layer that runs directly
on the hardware and is responsible for managing CPU, memory, and interrupts.
It is the first program running after the boot loader exits.
The hypervisor itself has no knowledge of I/O functions such as networking and
storage.
2. Guest Domains/Virtual Machines:
They are virtualized environments, each running their own operating system and
applications.
Xen supports two different virtualization modes:
Paravirtualization (PV) and Hardware-assisted or
Full Virtualization (HVM).
Both guest types can be used at the same time
on a single Xen system.
It is also possible to use techniques used for
Paravirtualization in an HVM guest: essentially
creating a continuum between PV and HVM.
This approach is called PV on HVM.
Xen guests are totally isolated from the
hardware: in other words, they have no privilege
to access hardware or I/O functionality. Thus,
they are also called unprivileged domain (or DomU).
All paravirtualized virtual machines running on a Xen hypervisor are referred to as
Domain U PV Guests and are modified Linux operating systems, Solaris, FreeBSD, and
other UNIX operating systems.
All fully virtualized machines running on a Xen hypervisor are referred to as Domain U
HVM Guests and run standard Windows or any other unchanged operating system.
A Domain U PV Guest contains two drivers for network and disk access, PV Network
Driver and PV Block Driver.
3. The Control Domain (or Domain 0):
It is a specialized Virtual Machine that has special privileges like the capability to
access the hardware directly, handles all access to the systems I/O functions and
interacts with the other Virtual Machines.
It also exposes a control interface to the outside world, through which the system is
controlled.
The Xen hypervisor is not usable without Domain 0, which is the first VM started by
the system.
The dom0 is essentially the "host" operating system or a "service console.
 Er. Rohit Handa
Lecturer, CSE-IT Department
IBM-ICE Program, BUEST Baddi
All Xen virtualization environments require
Domain 0 to be running before any other virtual
machines can be started.
Two drivers are included in Domain 0 to support
network and local disk requests from Domain U
Guests:
o The Network Backend Driver
communicates directly with the local
networking hardware to process all virtual
machines requests coming from the
Domain U guests.
o The Block Backend Driver communicates
with the local storage disk to read and
write data from the drive based upon
Domain U requests
4. Toolstack and Console:
Domain 0 contains a control stack (also called Toolstack) that allows a user to manage
virtual machine creation, destruction, and configuration.
The toolstack exposes an interface that is either driven by a command line console, by
a graphical interface or by a cloud orchestration stack such as OpenStack or
CloudStack.
5. Xen-enabled operating systems:
A Xen Domain 0 requires a Xen-enabled kernel.
Paravirtualized guests require a PV-enabled kernel.
Linux distributions that are based on recent Linux kernel are Xen-enabled and usually
contain packages that contain the Xen Hypervisor and Tools Xen.
All but legacy Linux kernels are PV-enabled: in other words, they will run Xen PV
guests.