Comparison of Open Source Virtualization Technologies

Fernando Laudares Camargos
Gabriel Girard
Benoit des Ligneris, Ph. D.
bdl@RevolutionLinux.com
Comparative study of
Open Source virtualization & contextualization
technologies
© Révolution Linux 2008. Ce document est confidentiel.
Context (1)
 Introduction
 Why virtualize the server infrastructure
 Virtualization technologies
 The experiments
 Explanations & anomalies
 Which technology is best for ... you ?

Context (2)
 Research executed by Fernando L. Camargos in pursuit

of his Masters degree in Computer Science under the
direction of Gabriel Girard (Université de Sherbrooke)
and Benoît des Ligneris (Révolution Linux)
 This being a research work, some questions remain
unsolved... maybe you can help !

Why virtualize the server infrastructure (1) ?
Server consolidation is the most mentionned argument

Why virtualize the server infrastructure (2) ?
 Reduction of the purchase and maintenance costs
 Compatibility with legacy applications and OSs
 Security : environment to execute untrusty applications
 Low cost environment for software development
 Centralized control/management
 Easy backup/restore procedures
 Live migration
 Quick server fail-over
 High availability
 Virtual appliances
 Controled sharing of ressources
 Cloud computing
 Hardware abstraction
 It's ... cool !
Virtualization technologies (1)
Hardware emulation
●
● Full virtualization Binary translation

●
Classic virtualization
●
● Para-virtualization
● OS-level virtualization (contextualization)

● Hardware emulation KQEMU
● Full virtualization ● binary translation VirtualBox
● Classic virtualization KVM
● Para-virtualization Xen
Linux-VServer
● OS-level virtualization
(contextualization/
containers) OpenVZ

virtualization != emulation
QEMU is an emulator

Virtualization technologies
partial emulation no emulation
KQEMU KVM VirtualBox OpenVZ Xen (Linux) Linux-VServer

2 types of hypervisors:
 Hypervisors type I: KVM, Xen

 Hypervisors type II: VirtualBox, KQEMU

The experiments (1)
virtualization layer overhead
But of how much ?
To discover, we need to mesure the efficiency of

the virtualization technologies
where :
efficiency = performance + scalability

The experiments (2)
2 types of experiments:
1) Performance (overhead): one virtual machine only

2) Scalability: several virtual machines

The experiments (3)
Virtualization solutions evaluated in this study
Virt. solution Version Server kernel VMs kernel
Host -- 2.6.22-14-server --
Linux-VServer 2.2.0.5 2.6.22.14 2.6.22.14
Xen 3.1.0 2.6.22-14-xen 2.6.22-14-xen
KVM 58 2.6.22-14-server 2.6.15-26-amd64
OpenVZ 5.1 2.6.22-ovz005 2.6.22-ovz005
KQEMU 1.3.0~pre11-6 2.6.22.14-kqemu 2.6.15-26-amd64
VirtualBox 1.5.4_OSE / 1.5.51_OSE 2.6.22-14-server 2.6.22.14
Test bed:
Chosen OSs:
 Intel Core 2 Duo 6300, 1.86GHz
(x86_64 / VT-x)
 Host: Ubuntu 7.10
 4G Memory
 VMs: Ubuntu 6.06
 Hard drive SATA 80G
The experiment (4)
Bits & Bytes & VMs :
 64 bit kernel for all technologies

 Use of VT extension for KVM, Xen
 32 bit VM for VirtualBox
 Identical memory allocation per VM for every
technology but Vserver : 2039 Mo

The experiments – Performance (1)
Methodology :
 7 benchmarks (different workloads)

 Reference : executed in the Linux host (scale = 1)
 executed inside the virtual machines
 4 execution sets
 results = the average of the 3 last sets normalized by
the result obtained by the Linux host

Compilation of the Linux kernel
 An equilibrated workload – a little bit of everything without

stressing one particular ressource too much
 Metric: given time for the completion of the compilation
tar xvzf linux-XXX.tar.gz

cd linux-XXX
make defconfig # ("New config with default answer to all options")
---
date +%s.%N && make && date +%s.%N
...
make clean
date +%s.%N && make && date +%s.%N 3x
...

Compilation of the Linux kernel
0.9
Li nux-VServer
Xen
0.8
OpenVZ
KVM
Virtual Box (-hwvirtex off)
0.7 KQEMU
Virtual Box (-hwvirtex on)
QEMU
0.6
Nomalized time
(kernel vanilla)
0.5
0.4
0.3
0.2
0.1

Bzip2
 Software for file compression

 Using option that yields maximal compression which
considerably increases the memory utilisation per process
 Metric: given time for the completion of the compression
cd /var/tmp
cp /home/fernando/Iso/ubuntu-6.06.1-server-i386.iso .
date +%s.%N && bzip2 -9 ubuntu-6.06.1-server-i386.iso && date +%s.%N 4x
rm ubuntu-6.06.1-server-i386.iso.bz2
...

Bzip2
1.00
0.90
0.80 Xen
Li nux-VServer
VirtualBox (-hwvirtex on)
Vir tual Box (-hwvirtex off)
0.70
KVM
KQEMU
OpenVZ
0.60 QEMU
Normalized time
(kernel vanilla)
0.50
0.40
0.30
0.20
0.10
0.00

Dbench
 Derived from the Netbench benchmark

 Emulates the load imposed in a file server by n Windows 95
clients
 n(umber of clients)=100, t(ime)=300
 Metric: throughput (Mb/sec)
/usr/local/bin/dbench -t 300 -D /var/tmp 100 # 4x

Dbench
1.20
Normalizes throughput
(kernel vanilla) 1.00
Linux-VServer
0.80
Xen
OpenVZ
KVM
0.60 KQEMU
QEMU
0.40
0.20
0.00
* no results for VirtualBox

dd
 Application for low level (bit by bit) data copy
 Mesures the performance of the I/O system (hard drive
access)
 2 tests:
copy of a single big file

 copy of 60G of /dev/zero to /dev/null
 Metric : throughput
...
dd if=/opt/iso/ubuntu-6.06.1-server-i386.iso of=/var/tmp/out.iso
...
dd if=/dev/zero of=/dev/null count=117187560 # 117187560 = 60G
...
rm -fr /var/tmp/* # between execution sets
...

dd (copy of ISO file)
1.4
1.2
1.0
Li nux-VServer
Xen
KVM
OpenVZ
Normalized throughput
0.8
(kernel vanilla)
0.6
0.4
0.2
0.0
* no results for KQEMU nor VirtualBox

dd (60G /dev/zero --> /dev/null)
0.9
0.8
0.7
Li nux-VServer
KVM
OpenVZ
0.6 Xen
0.5
(kernel vanilla)
0.4
0.3
0.2
0.1
* no results for KQEMU nor VirtualBox

Netperf
 A benchmark that can be used to measure several aspects of

the network performance
 TCP Stream test: measure the speed of the exchange of TCP
packets through the network (10 sec.)
 Metric: throughput (bits/sec)
netserver # in the server

..
netperf -H <nom du serveur> # in the client, 4x

Netperf (TCP Stream Test)

1.2
1 Vir tual Box (-hwvir tex off)

Vir tual Box (-hwvir tex on)
Li nux-VServer
Xen
OpenVZ
0.8 QEMU (-no-kqemu)
KQEMU
KQEMU (-kernel-kqemu)
KVM
(kernel vanilla)
0.6
0.4
0.2

Rsync
 Similar to Netperf's TCP Stream Test, measures the

performance of file exchange through the network
 2 tests:
 ISO file: 1 big file (433M)
 Linux kernel tree: several small files (294M)
 Metric: time (sec.)
..
date +%s.%N && rsync -av <server>::kernel /var/tmp && date +%s.%N
...
date +%s.%N && rsync -av <server>::iso /var/tmp && date +%s.%N
...
rm -fr /var/tmp/* # between execution sets
...

Rsync (kernel tree)
1.2
1.0 OpenVZ
VirtualBox (-hwvirtex off)
Li nux-VServer
Xen
QEMU
0.8
KQEMU
KVM
Normalized time
(kernel vanilla)
0.6
0.4
0.2
0.0

Rsync (ISO file)
1.2
OpenVZ
1.0
KQEMU
Li nux-VServer
Xen
QEMU
0.8
KVM
Normalized time
(kernel vanilla)
0.6
0.4
0.2
0.0

Sysbench
 Measures the performance of a DB server
 Workload centered in I/O operations in the file system
 Metric: throughput (transactions/sec)
sysbench --test=oltp --mysql-user=root --mysql-host=localhost --debug=off prepare # (1x)
sysbench --test=oltp --mysql-user=root --mysql-host=localhost --debug=off run # (4x)
On-Line Transaction
T Processing
P
OLTP test statistics:
queries performed:
read: 140000
write: 50000
other: 20000
total: 210000
transactions: 10000 (376.70 per sec.)
deadlocks: 0 (0.00 per sec.)
read/write requests: 190000 (7157.28 per sec.)
other operations: 20000 (753.40 per sec.)

Sysbench
1
0.9
0.8 Linux-VServer
Xen
0.7 KVM
(kernel vanilla)
OpenVZ
0.6 VirtualBox (-hwvirtex on)
0.5 KQEMU
0.4 QEMU
0.3
0.2
0.1

Conclusion :
 Linux-VServer: excellent performance. Has presented minimal

to no overhead when compared to Linux.
 Xen: has shown a great performance in all but the Dbench
benchmark (I/O bound benchmark).
 KVM's performance was fairly good for a full virtualization
solution but should be avoided to run applications that
strongly rely on I/O.

Conclusion (cont) :
 OpenVZ has shown a very variable performance (from weak to

excellent). Certainly because of accounting for I/O and
because of some network optimization for the Network related
tests.
 VirtualBox has presented a good performance for the file
compression and network based benchmarks. Poor
performance for all the other situations.
 KQEMU has shown a poor performance for all benchmarks. It
is clear that this virtualization solution does not make a good
use of the available ressources and its application in
production servers should be avoided.

The experiments – Scalability (1)
Methodology:
 1 benchmark (Sysbench, kernel compilation) executed by n

VMs concurrently
 n = 1, 2, 4, 8, 16 et 32
 4 execution sets:
 results = average of the last tree execution sets
 Memory allocation per VM :
n Mémoire (Mb)
1 2039*
2 1622
4 811
8 405
16 202
32 101
* 1536 Mb (KQEMU)
Sysbench at scale: average throughput per VM

400
Throughput (transactions/sec)
350
300
250 Linux-VServer
Xen
200 KVM
OpenVZ
VirtualBox
150
KQEMU
100
50
0
10
11
12
13
16
17
18
19
20
21
22
23
25
26
27
28
29
30
31
32
33
14
15
24
1
2
3
4
5
6
7
8
9
Number of VMs (n)

Sysbench at scale: total throughput

700
Throughput (transactions/sec)
600
500
Linux-VServer
400 Xen
KVM
OpenVZ
300
VirtualBox
KQEMU
200
100
0
10
11
12
13
14
15
16
18
20
21
23
25
27
28
29
30
31
32
33
17
19
22
24
26
1
2
3
4
5
6
7
8
9
Number of VMs (n)

Kernel compilation (VMs average)

8000
7000
6000
time (seconds)
5000
Host
4000 Vserver
Xen
3000
2000
1000
0
1 2 4 8 16 32
number of VMs (n)

Kernel scalability
25
20
Kernel / Hours
15
Host
Vserver
10
Xen
0
1 2 4 8 16 32
Number of VM

Conclusion - Scalability (1)
Conclusion :
 The efciency of virtualization solutions is strongly related to

the number of VMs executing concurrently (scalability)
 Most of the time, one additional VM helps to get the maximum
performance out of a given server (link with the number of
CPU)
 More decreases performance as a bottleneck is limiting
performance (CPU/core number important !)
 Linux-VServer has shown the best global performance for up
to 5-7 VMs.
 Xen has proved to be the best full virtualization based
solution.
Conclusion – Scalability (2)
 KVM has shown a reasonable performance for a full

virtualization solution.
 OpenVZ's performance was not what we would expect of a
contextualization solution. Our hypothesis is that the
accounting (beancounter) is the root cause of th overhead
 VirtualBox has shown an impressive performance, the total
throughput has more than doubled when the number of VMs
has pass from 8 to 16. However, we were unaible to execute
this experiment with 32 VirtualBox VMs executing
concurrently.
 KQEMU's performance was weak when compared to all other
solutions, independently of the number of VMs in execution.

Which technology to use in each case ?
A technological point of view ...
 OpenVZ : pure network related applications, thanks to

the optimizations done in the network layer. Not
indicated for I/O applications.
 Linux-VServer: all kinds of situations (a priori).
 Xen can also be used in all kinds of situations but
requires important modifications in the guest OS kernels
OR the use of the VT virtualization instructions.
 KVM and VirtualBox have proven to be good options for
development environments.
 KQEMU has shown weak performance. It is indicated for
development only.

Conclusion : Which technology to use in each
case ?
 Results are very different for every
benchmark/technology ,so benchmark the technology
you plan to use with your own mission critical
application BEFORE virtualizing your servers (ex: File
Servers benchmark).
 Only Xen is actually supported by the industry
(RedHat, SuSE, Mandrake, IBM, etc.)
 KVM is available in the standard Linux kernel : Yeah!
But poor performance overall ;-(
 Linux-VServer and OpenVZ: they both need a modified
kernel that is not ofcially supported by the afore
mentioned giants of the industry but . . .
Future / contextualisation
 Since last OLS, key players like IBM, Intel, and Google
are working hard to include a container based
technology in the Linux kernel
 Lots of patches from OpenVZ gets integrated into the
kernel recently and everyone expects that we will
have Really Soon Now a contextualization in the linux
kernel without the need for any kernel hacking
 We strongly believe that the integration of a
contextualization/container solution is the best way to
go for Linux-on-Linux virtualization needs
 It will offer VMWare a very strong and completely
open-source competition


Comparison of Open Source Virtualization Technologies

Diunggah oleh

Informasi Dokumen

Deskripsi Asli:

Hak Cipta

Format Tersedia

Bagikan dokumen Ini

Bagikan atau Tanam Dokumen

Opsi Berbagi

Apakah menurut Anda dokumen ini bermanfaat?

Apakah konten ini tidak pantas?

Hak Cipta:

Format Tersedia

Comparison of Open Source Virtualization Technologies

Diunggah oleh

Hak Cipta:

Format Tersedia

Fernando Laudares Camargos

© Révolution Linux 2008. Ce document est confidentiel.

 Research executed by Fernando L. Camargos in pursuit

© Révolution Linux 2008. Ce document est confidentiel.

Server consolidation is the most mentionned argument

© Révolution Linux 2008. Ce document est confidentiel.

● Full virtualization Binary translation

● OS-level virtualization (contextualization)

© Révolution Linux 2008. Ce document est confidentiel.

● Hardware emulation KQEMU

● Full virtualization ● binary translation VirtualBox

● Classic virtualization KVM

© Révolution Linux 2008. Ce document est confidentiel.

© Révolution Linux 2008. Ce document est confidentiel.

partial emulation no emulation

KQEMU KVM VirtualBox OpenVZ Xen (Linux) Linux-VServer

© Révolution Linux 2008. Ce document est confidentiel.

 Hypervisors type I: KVM, Xen

© Révolution Linux 2008. Ce document est confidentiel.

virtualization layer overhead

But of how much ?

To discover, we need to mesure the efficiency of

efficiency = performance + scalability

© Révolution Linux 2008. Ce document est confidentiel.

1) Performance (overhead): one virtual machine only

© Révolution Linux 2008. Ce document est confidentiel.

Linux-VServer 2.2.0.5 2.6.22.14 2.6.22.14

Xen 3.1.0 2.6.22-14-xen 2.6.22-14-xen

KVM 58 2.6.22-14-server 2.6.15-26-amd64

OpenVZ 5.1 2.6.22-ovz005 2.6.22-ovz005

KQEMU 1.3.0~pre11-6 2.6.22.14-kqemu 2.6.15-26-amd64

VirtualBox 1.5.4_OSE / 1.5.51_OSE 2.6.22-14-server 2.6.22.14

Bits & Bytes & VMs :

 64 bit kernel for all technologies

© Révolution Linux 2008. Ce document est confidentiel.

 7 benchmarks (different workloads)

© Révolution Linux 2008. Ce document est confidentiel.

Compilation of the Linux kernel

 An equilibrated workload – a little bit of everything without

tar xvzf linux-XXX.tar.gz

© Révolution Linux 2008. Ce document est confidentiel.

Compilation of the Linux kernel

© Révolution Linux 2008. Ce document est confidentiel.

 Software for file compression

© Révolution Linux 2008. Ce document est confidentiel.

© Révolution Linux 2008. Ce document est confidentiel.

 Derived from the Netbench benchmark

/usr/local/bin/dbench -t 300 -D /var/tmp 100 # 4x

© Révolution Linux 2008. Ce document est confidentiel.

* no results for VirtualBox

 copy of 60G of /dev/zero to /dev/null

© Révolution Linux 2008. Ce document est confidentiel.

dd (copy of ISO file)

* no results for KQEMU nor VirtualBox

© Révolution Linux 2008. Ce document est confidentiel.

dd (60G /dev/zero --> /dev/null)

* no results for KQEMU nor VirtualBox

© Révolution Linux 2008. Ce document est confidentiel.

 A benchmark that can be used to measure several aspects of

netserver # in the server

© Révolution Linux 2008. Ce document est confidentiel.

Netperf (TCP Stream Test)

1 Vir tual Box (-hwvir tex off)