Hacking on Apache (Incubating) CloudStack

Tutorial Outline

Session 1: Introduction to CloudStack

Murali Reddy: Committer Apache CloudStack

Session 2: Architecture of CloudStack

Murali Reddy: Committer Apache CloudStack

Session 3: Hands on with DevCloud

Kishan Kavala: Committer Apache CloudStack Rajesh Battala: Contributor Apache CloudStack

Session 1 Introduction to CloudStack

Cloud Computing
Virtualization is not Cloud computing
Server Virtualization++ Cloud

Built for traditional enterprise apps & client-server compute

Designed around big data, massive scale & next-gen apps

Enterprise arch for 100s of hosts Scale-up (pool-based resourcing) IT management-centric 1 administrator for Dozens of servers Apps assume reliability Proprietary vendor stack

Cloud arch for 1000s of hosts Scale-out (horizontal resourcing) Autonomic management 1 administrator for 1,000s of servers Apps assume failure Open, value-added stack

Cloud Computing (contd..)

Tenets of Cloud
o Shared infrastructure and Multi-tenancy o Self Service o Elasticity o Built for massive Scale o Service agility o Pay-as-you-go o APIs and Extreme Automation

IAAS/PAAS/SAAS Public/Private/Hybrid clouds

What is Apache CloudStack

Turnkey orchestration platform for delivering IAAS clouds

o Secure, multi-tenant o Self-service o Service agility and elasticity o Built for large scale o Pay-as-you-go

Deploys on premise (private) or as a hosted (public) cloud Can be used for hybrid clouds built in java, provides native REST APIs and EC2 API Has python, Ruby clients and CLI as well

A bit of History

Original company Cloud.com (2008) Open source (GPLv3) as CloudStack (2010) Acquired by Citrix (July 2011) Relicensed under ASL v2 April 3, 2012 Accepted as Apache IncubaKng Project April 16,

2012 First Apache (ACS 4.0) released Many non-Citrix contributors, commiRers, PPMC members

Who is contributing

Sungard: Unit test cases Carnigo: Object store plug-in Ceph/Rbd support by Wido CLVM/KVM by Marcus Nicira NVP: Schuberg Philis Basho: Object Store Brocade ADX ADC support Midokura midonet SDN controller integration

How to contribute

Its not just about code! As community member

you can engage in o Discussions: Design, Use Case, deployment issues o Bug reporting, feature requests o Code reviews o Build, tools, infrastructure o Helping out on the IRC o Documentation o Submit bug fixes, features

How to contribute (contd..)

Git repo, bug tracker, wiki are on ASF infra Project website IRC
o http://incubator.apache.org/cloudstack/ o http://www.cloudstack.org o #cloudstack on irc.freenode.net

Mailing lists (cloudstack.org/discuss/mailinglists.html)

o Wednesday - 10:30 PM IST, 5:00 UTC

http://www.slideshare.net/cloudstack

o cloudstack-dev-subscribe@incubator.apache.org o cloudstack-users-subscribe@incubator.apache.org

On-demand infrastructure as a service

Org A

Admin
Users

Org B

Admin
Users

Cloud
Admin
End User

Provision resources UI Cli REST API CloudStack Management Server manage resources Consume resources EC2

CloudStack managed cloud

Compute Network Storage

Core CloudStack Components

VM

Hosts
Servers onto which services will be provisioned

Host
Network
VM

Primary Storage
VM storage

Host
Primary Storage

Cluster
A grouping of hosts and their associated storage

Pod
Collection of clusters

Cluster
Secondary Storage

Network
Logical network associated with service offerings

Cluster CloudStack Pod CloudStack Pod Zone

Secondary Storage
Template, snapshot and ISO storage

Zone
Collection of pods, network offerings and secondary storage

Management Server Farm

Responsible for all management and provisioning tasks

CloudStack Deployment Architecture

CloudStack Management Server Zone 1 L3 core Internet

Hypervisor is the basic unit of scale. Cluster consists of one ore more hosts of same hypervisor

Pod 1

Access Layer

Pod N .
Secondary Storage

All hosts in cluster have access to shared (primary) storage Pod is one or more clusters, usually with L2 switches. Availability Zone has one or more pods, has access to secondary storage.

Cluster N

. Cluster 1 Host 1 Host 2

Primary Storage

One or more zones represent cloud

CloudStack Managing Multiple Zones

Data Center 1
Management Server

Data Center 2 Data Center 2 Data Center 3

Single Management Server can manage multiple zones Zones can be geographically distributed but low latency links are expected for better performance Single MS node can manage up to 5K hosts. Multiple MS nodes can be deployed as cluster for scale or redundancy

Zone1

Zone Zone 2 2 Zone Zone Zone 3 4 3

Data Center 2 Data Center 2 Data Center 2

Zone Zone 2 Zone 2 Zone 2 Zone 3 Zone 3 3

Infrastructure provisioning

Infrastructure provisioning (contd.)

Compute/Disk/Network Offering

Create Virtual Machines via Offerings

Select Operating System Windows, Linux

Select Compute Offering CPU & RAM

Select Disk Offering Volume Size

Select Network Offering Network & Services

Create VM

Virtual Machine Management

Users

VM Operations

Console Access

VM Status

Change Service Offering

Start Stop Restart Destroy

CPU Utilized Network Read Network Writes

2 CPUs 1 GB RAM 20 GB 20 Mbps

4 CPUs 4 GB RAM 200 GB 100 Mbps

Volume & Snapshot Management

Add / Delete Volumes
VM 1
Volume

Create Templates from Volumes

Volume

Template

Schedule Snapshots

Hourly

Weekly Monthly

Now
Daily

View Snapshot History

A Very Flexible IaaS Pla5orm

Compute Hypervisor
XenServer VMware Oracle VM KVM Bare metal

Storage

Block & Object

Local Disk iSCSI Fiber Channel NFS Swift Ceph Riak

Primary Storage

Secondary Storage

Network

Network & Network Services

Network Type Isolation Firewall Load balancer VPN

CloudStack Storage
Primary Storage Configured at Cluster-level. Close to hosts for better performance Stores all disk volumes for VMs in a cluster Cluster can have one or more primary storages Local disk, iSCSI, FC or NFS Pod 1 L2 switch
Secondary Storage

L3 switch

Cluster 1 Host 1
Local storage

Secondary Storage Configured at Zone-level Stores all Templates, ISOs and Snapshots Zone can have one or more secondary storages NFS, OpenStack Swift Local Storage Storage available on hypervisor hist

Primary Storage

Host 2

Availability zone

Role of Storage and Templates

Primary Storage
Cluster level storage for VMs Connected directly to hosts NFS, iSCSI, FC and Local
Host Host
Primary Storage

Secondary Storage
Zone level storage for template, ISOs and snapshots NFS or OpenStack Swift via CloudStack System VM

Cluster Pod

Templates and ISOs

Imported into CloudStack Can be private or public
Template

Secondary Storage

Zone

Provisioning Process
1. 2. 3. User Requests Instance Provision Optional Network Services Copy instance template from secondary storage to primary storage on appropriate cluster Create any requested data volumes on primary storage for the cluster Create instance Start instance
Secondary Storage
VM

Host Host
Primary Storage

Cluster Pod
Template

4.

5. 6.

Zone

Object Store
CloudStack Mgmt Server

Object store used to store templates and snapshots VMs can be distributed across the availability zones

Availability Zone

Availability Zone

Availability Zone

For DR create instances in different zones

Object Storage

Multi-tenancy & Account Management

Domain
Org A

Resources
VMs, IPs, Snapshots

Admin

Domain
Reseller A

Domain is a unit of isolation that represents a customer org, business unit or a reseller Domain can have arbitrary levels of sub-domains A Domain can have one or more accounts An Account represents one or more users and is the basic unit of isolation
User 1
User 2

Admin

Sub-Domain
Org C

Resources
VMs, IPs, Snapshots

Admin

Account
Group A
Account
Group B

Admin can limit resources at the Account or Domain levels

User Dashboard: Consumed Resources

Running, Stopped & Total VMs Public IPs Private networks Latest Events

Admin Dashboard: Consumed Resources

Provides zone wide resource consumption Also provides latest alerts and events

Edge services with System VMs

System VMs optimize and scale the datapath on behalf of CloudStack

o o o o Stateless, can be destroyed and recreated from database state Highly Available Communicates with Management Server over management network Usually have 3 interfaces: control, guest and public

Console Proxy VM o Provides AJAX-style HTTP-only console viewer o Grabs VNC output from hypervisor o Scales out (more spawned) as load increases o Java-based server Communicates with MS over message bus

Secondary Storage VM
o o o o o Provides image (template) management services Download from HTTP file share or Swift Copy between zones Scale out to handle multiple NFS mounts Java-based server communicates with MS over message bus

Edge services with System VMs (contd.)

Virtual Router VM o Provides multiple network services o IPAM (DHCP), DNS, NAT, Source NAT, Firewall, PF, VPN o User-data, Meta-data, SSH keys and password change server o Redundancy via VRRP o MS configures VR over SSH
Proxied via the hypervisor on XS and KVM

Network & Network Services

Create Networks and attach VMs Acquire public IP address for NAT & load balancing Control traffic to VM using ingress and egress firewall rules Set up rules to load balance traffic between VMs

Networking feature overview

Orchestration of L2 L7 network services o IPAM, DNS, Gateway, Firewall, NAT, LB, VPN, etc Mix-and-match services and providers Out-of-the-box integration with automated deployment of virtual
routers

Orchestrate external providers such as hardware firewalls and

load balancers

o Highly available network services using CloudStack HA and VRRP

Multiple multi-tenancy [network isolation] options Integrated traffic accounting Access control Software Defined Networking (Nicira NVP)

o Devices can provide multiple services o Admin API to configure external devices o Plugin-based extensions for network behavior and admin API extensions

L2 Features

Choice of network isolation

o Physical, VLAN, L3 (anti-spoof), Overlay[GRE] o Physical isolation through network labels [limited to # of nics or bonds] Multi-nic o Deploy instance in multiple networks o Control default route Access control o Shared networks, project networks QoS [max rate] Traffic monitoring Hot-plug / detach of nics

L3 Features

IPAM [DHCP], Public IP address management
o o o o o o o VR acts as DHCP server Can request multiple public IPs per tenant Redundant VR (using VRRP) Inter-subnet routing Static routing control L2TP over IPSec using PSK Virtual Router only

Gateway (default gateway)

Remote Access VPN Firewall based on source cidr Static NAT [1:1]
o o o o o Including Elastic IP in Basic Zone Per-network, or interface NAT Monitoring on the Virtual Router / External network device Integration with sFlow collectors IPSec VPN based on VR

Source NAT Public Traffic usage

Site-to-Site VPN L3 ACLs

L4 Features

Security groups for L3-isolation Stateful firewall for TCP, UDP and ICMP Port forwarding [Advanced Zone]
o Conserve public Ips o Basic Zone in docs o Default AWS-style networking o Scales much better than VLANs

L7 features

Loadbalancer

User-data & meta-data o Fetched from virtual router Password change server

o VR has HAProxy built in o External Loadbalancer support Netscaler (MPX/SDX/VPX) F5 BigIP Can dedicate an LB appliance to an account or share it among tenants o Loadbalancer supported with L3-isolation as well o Stickiness support o SSL support [future] o Health Checks [future]

CloudStack Terminology

Guest network
o The tenant network to which instances are attached

Storage network
o The physical network which connects the hypervisor to primary storage

Management network
o Control Plane traffic between CloudStack management server and hypervisor clusters

Public network
o Outside the cloud [usually Internet] o Shared public VLANs trunked down to all hypervisors

All traffic can be multiplexed on to the same underlying physical network using VLANs
o Usually Management network is untagged o Storage network usually on separate nic (or bond)

Admin informs CloudStack how to map these network types to the underlying physical network
o Configure traffic labels on the hypervisor o Configure traffic labels on Admin UI

CloudStack Network Service Providers

A Network Service Provider is hardware or virtual appliance that makes a network service possible in CloudStack ; for example, a Citrix NetScaler appliance can be installed in the cloud to provide Load-Balancing services. Administrators can have multiple instances of the same service provider in a network; for example, more than one Citrix NetScaler or Juniper SRX device can be added to CloudStack CloudStack supports the following Network Providers:
o o o o CloudStack Virtual Router (default) Citrix NetScaler SDX, VPX and MPX models Juniper SRX F5 BigIP

Network Service Providers Matrix

Network offerings is basically a definition of what Network Services are available when this offering is used. The available Network Services are: VPN, DHCP, DNS, Firewall, Load Balancer, User Data, Source NAT, Static NAT, Port Forwarding and Security Groups*
Feature Remote Access VPN Firewall Source NAT Static NAT Load Balancing Port Forwarding Elastic IP Elastic LB DHCP/DNS/User Data Virtual Router YES YES YES YES YES YES N/A N/A YES Citrix NetScaler N/A N/A N/A YES YES N/A YES YES N/A Juniper SRX N/A YES YES YES N/A YES N/A N/A N/A F5 BigIP N/A N/A N/A N/A YES N/A N/A N/A N/A

Network Offerings

Cloud provider defines the feature set for guest networks Toggle features or service
levels
o o o o

User chooses network offering when creating network Enables upgrade between network offerings Default offerings built-in
o For classic CloudStack networking

Security groups on/off Load balancer on/off Load balancer software/hardware VPN, firewall, port forwarding

Add Guest Networks

Choice to choose L3 subnet, default gateway Choice of network offerings

Editing Guest Networks

When editing a guest network users can change the network offering. They can either upgrade to a premium network offering (for example offering that uses hardware Load-balancer) or downgrade to a cheaper network.

Restarting/Cleaning Up a Guest Network

Restarting the network will simply resend all the LB, Firewall and Port-Forwarding rules to the network provider

Restarting the Network with Clean up:

restarKng network elements - virtual routers, DHCP
servers If virtual router is used, it will be destroyed and recreated Reapplying all public IPs to the network provider Reapplying load-Balancing/Port-Forwarding/Firewall rules

Deleting a Guest Network

An Isolated Guest Network can only be deleted if no VMs are using these network (e.g. Completely destroyed and expunged) Deleting a Network will Destroy the Virtual Router (if used) and will release the Public IPs back to the IP Pool

Basic vs Advanced Networking

Segmentation based on feature set and ease-of

deployment Both are feature-rich Basic implements true AWS-style L3-isolation
o Tenants do not get contiguous IP addresses or subnets o Network segmentation based on Security Groups o Tremendous scale (tens of thousands)

Advanced Zone offers full L3 subnets and L2

isolation
o VLANs are default implementation (4K limit) o More features (source NAT, PF, LB, VPN)

Physical Network in Zone

Core (L3) Network Pod 1
Cloudstack Server Cloudstack Servers Access Switch(es)

Pod 2

Pod N

CLUSTER 1
Hypervisor 1

VM Traffic Control Plane Traffic Storage Traffic Public Traffic

Hypervisor 8

Storage 2 Storage 1

CLUSTER 4
Hypervisor N
Hypervisor N+1

Storage k

Layer 3 cloud networking

Web VM

Web Security Group

DB VM

DB Security Group

Web VM

Web VM

Web VM

DB VM

Web VM

Web VM

Guest Networks with L3 isolation

Public Internet Public IP address 65.37.141.11 65.37.141.24 65.37.141.36 65.37.141.80 L3 Core Switch Guest 1 VM 1 Pod 1 L2 Switch 10.1.0.1 Guest 2 VM 1 Guest 1 VM 2 Guest address 10.1.0.2 Guest address 10.1.0.3 Guest address 10.1.0.4

Pod 2 L2 Switch

10.1.8.1

Load Balancer

Pod 3 L2 Switch

10.1.16. 1

Guest 2 VM 2 Guest 2 VM 3 Guest 1 VM 3 Guest 1 VM 4

Guest address 10.1.16.12 Guest address 10.1.16.21 Guest address 10.1.16.47 Guest address 10.1.16.85

Guest Networks with L2 isolation

Core (L3) Network Pod K
Access Switch(es)
Hypervisor V R

Pod M

Pod N
Hypervisor V V

CLUSTER 1
Hypervisor 1

VM Traffic Public Traffic

Hypervisor 8

CLUSTER 4
Hypervisor V V N
Hypervisor N+1

V V R Tenant VM Tenant Virtual Router

L2 isolation: VLAN networking

User 1 User 1 User 2 User 1

User 1 User 2 User 1

User 2

User 1

SDN at Work
CloudStack Mgmt Server
SDN Controlle r

Host 1
VM 1 VM 1

OVS

Host 3
VM 3

OVS

V R

GRE Tunnel

GRE Tunnel

Host 2
VM 2 VM 2

OVS

Host 4
VM 3

OVS

V R

GRE Tunnel

GRE Tunnel

Guest virtual layer-2 network

Guest Virtual Network 10.1.1.0/24 Public IP Public Guest Gateway Network address 1 VM 1 address 65.37.141.11 10.1.1.1 65.37.141.36 Guest 1 Guest Virtual 1 VM 2 Router NAT Guest DHCP 1 VM 3 Load Balancing Guest VPN 1 VM 4 Public IP address 65.37.141.24 65.37.141.80 Guest Virtual Network 10.1.1.0/24 Guest Gateway 2 VM 1 address 10.1.1.1 Guest 2 Guest Virtual 2 VM 2 Router NAT Guest DHCP 2 VM 3 Load Balancing VPN Guest address 10.1.1.2 Guest address 10.1.1.3 Guest address 10.1.1.4 Guest address 10.1.1.5 Guest address 10.1.1.2 Guest address 10.1.1.3 Guest address 10.1.1.4

Public Internet

Layer-2 Guest Virtual Network

CS Virtual Router provides Network Services External Devices provide Network Services

Guest Virtual Network 10.1.1.1/8 VLAN 100 Public Network/ Internet Public IP 65.37.141.11 Gateway address 10.1.1.1 10.1.1.1 Public Network/ Internet Public IP 65.37.141.111 Juniper SRX Firewall Public IP 65.37.141. 112

Guest Virtual Network 10.1.1.1/8 VLAN 100

Guest VM 1

Private IP 10.1.1.111

10.1.1.1

Guest VM 1

CS Virtual Router

10.1.1.3

Guest VM 2

10.1.1.3
NetScaler Load Blancer

Guest VM 2

DHCP, DNS NAT Load Balancing VPN

Private IP 10.1.1.112 10.1.1.4 Guest VM 3

10.1.1.4

Guest VM 3

10.1.1.5

Guest VM 4

10.1.1.5
CS

Guest VM 4

DHCP, Virtual Router DNS

Layer-3 Guest Network

Network Services Managed Externally

Network Services Managed by CS

Public Network 65.11.0.0/16 Security Group 1 65.11.1.2 Guest VM 1 65.11.1.2 65.11.1.3 65.11.1.4
NetScaler Load Blancer
10.2.12.4

Public Network/ Internet

Security Group 1
10.1.2.3

Guest VM 1

65.11.1.3

Guest VM 2

L3 switch

Guest VM 2

65.11.1.4

EIP, ELB
Guest VM 3

10.5.2.99

Guest VM 3

65.11.1.5

10.1.2.18

Guest VM 4 Security Group 2 DHCP, DNS

CS Virtual Router

Guest VM 4 Security Group 2

DHCP, DNS

CS Virtual Route r

Multi-tier network
Internet

IPSec or SSL site-to-site VPN Loadbalancer

CS Virtual Router

Customer Premises

Monitoring VLAN

Virtual Router Services IPAM DNS LB [intra] S-2-S VPN Static Routes ACLs NAT, PF FW [ingress & egress] BGP

10.1.1.1

Web VM 1

10.1.2.31

App VM 1

10.1.1.3

Web VM 2

10.1.2.24

App VM 2

10.1.1.4

Web VM 3

10.1.3.24

DB VM 1

10.1.1.5 Virtual Network 10.1.1.0/24 VLAN 100

Web VM 4 Virtual Network 10.1.2.0/24 VLAN 1001 Virtual Network 10.1.3.0/24 VLAN 141

Session 2 Architecture of CloudStack

Problem Definition

Offer a scalable, flexible, manageable IAAS platform

that orchestrate physical and virtual resources to offer self-service infrastructure provisioning and monitoring

Flexible
o Handle new physical resource types Hypervisors, storage, networking o Add new APIs o Add new services o Add new networking models

Problem Definition (contd..)

Manageable
o Hide complexity of underlying resources o Rich functional end-user and admin UI o Admin API to automate operations o Easy install, upgrade for small -> large clouds o Simple scaling, automated resilience

Scalable architecture
o 1 -> N hypervisors / VMs / virtual resources o 1 -> N end users

Problem Definition (contd..)

Resource Allocation
o Hypervisor CPU, Memory o Storage space o Avoid set of pods, clusters, hosts

Capacity scanning
o Snapshot of resources consumed o Trigger capacity threshold violations

Garbage collection
o Network resources (IP, VLAN, CIDR etc) o Compute (VM, CPU, memory) o Storage (volumes)

Synchronizing the resource states Infrastructure resource failures Fencing

Scaling: Horizontal Scaling

Single-node Deployment Multi-node Deployment
Manage ment Server

User API
Manage ment Server
MySQL DB

User API
Load Balancer Manage ment Server Manage ment Server
MySQL DB Back Up DB Replication

Admin API

Admin API

MS is stateless. MS can be deployed as physical server or VM Single MS node can manage up to 10K hosts. Multiple nodes can be deployed for scale or redundancy Infrastructure Resources

Infrastructure Resources

Resource Load Balancing

As management server is added into the cluster, resources are

rebalanced seamlessly.
o o o o o o o MS2 signals to MS1 to hand over a resource MS1 wait for the commands on the resources to finish MS1 holds further commands in a queue MS1 signals to MS2 to take over MS2 connects MS2 signals to MS1 to complete transfer MS1 discards its resource and flows the commands being held to MS2

Listeners are provided to business logic to listen on connection status and adjusts work based on whos connected. By only working on resources that are connected to the
management server the process is on, work is auto-balanced between management servers. Also reduces the message routing between the management servers.

Cloud Portal

CLI

UI

Other Clients

Management Server
REST API
OAM&P API Console Proxy Management Template Access Services API HA Usage Calculations Additional Services Cluster Managemen t End User API EC2 API Other APIs Pluggable Service API Engine Security Adapters Account Management Connectors Plugin API Deployment Planning Network Configurations Network Elements Hypervisor Gurus Resource Managemen t Job Management Alert & Event Management Database Access

ACL & Authentication - Accounts, Domains, and Projects - ACL, limits checking Services API

- Drives long running VM operations - Syncs between resources managed and DB - Generates events

Kernel

Job Queue

DB

Event Bus Message Bus Hypervisor Resources Network Resources Storage Resources Image Resources Snapshot Resources

Interactions
OVM Cluster
vcenter

Primary Storage

Monitoring
End User UI Admin UI Domai n Admin UI

CS API

Storage vSphere Cluster

Primary

XS Cluster CS Admin &

Primary Storage

End-user API

Clustered CloudStack CloudStack CloudStack Management Server

XAPI
Primary

JSON

KVM Cluster Storage

NetConf

Cloud user {API client (Fog/etc)}

Juniper SRX Nitro API JSON JSON Console Console Proxy VM Proxy VM {Proxied} SSH Netscaler VNC

ec2 API
Cloud user {ec2 API client }

MySQL
HTTPS Router VM Router VM Router VM

Ajax Console

Sec. NFS NFS Sec. Storage Storage VM VM HTTP (Template Download) HTTP (Template Copy) HTTP (Swift)

NFS Server

Cloud user

Management Server Layering

Balancing Incoming Requests

Each management server has two worker thread pools for

incoming requests: effectively two servers in one.
o Executor threads provided by tomcat o Job threads waiting on job queue

All incoming requests that requires mostly DB operations are

short in duration and are executed by executor threads because incoming requests are already load balanced by the load balancer All incoming requests needing resources, which often have long running durations, are checked against ACL by the executor threads and then queued and picked up by job threads. # of job threads are scaled to the # of DB connections available to the management server Requests may take a long time depending on the constraint of the resources but they dont fail.

Inside a Management Server

CloudStack API

API Servlet
Plugins Plugins Plugins

Commands

cmd.execute()

Async Job Queue Mgr

Services API
Responses

Kernel

Agent API (Cmds)

Mess age Bus

Agent Manager

Local Or Remote

Resources

Hypervisor Native APIs

Network Device API

MySQL

CloudStack API Sync/Async commands

Package and Location cloudstack-oss/api/src/com/cloud/api/

BaseCmd (base class) All commands descend from the BaseCmd base class

CloudStack API
Configuration Commands are configured in cloudstack-oss/client/command.properties.in Format: <command name>=<java classname>;<ACL> *note* ACL is calculated as a bitmap with the following, 1 = ADMIN, 2 = RESOURCE_DOMAIN_ADMIN, 4 = DOMAIN_ADMIN, 8 = USER Example:

### snapshot commands! createSnapshot=com.cloud.api.commands.CreateSnapshotCmd;15! listSnapshots=com.cloud.api.commands.ListSnapshotsCmd;15! deleteSnapshot=com.cloud.api.commands.DeleteSnapshotCmd;15! createSnapshotPolicy=com.cloud.api.commands.CreateSnapshotPoli cyCmd;15! deleteSnapshotPolicies=com.cloud.api.commands.DeleteSnapshotPo liciesCmd;15! listSnapshotPolicies=com.cloud.api.commands.ListSnapshotPolici esCmd;15!

CloudStack API: adding API

Adding a new command Determine type of command Synchronous Synchronous List Based Asynchronous Asynchronous Create based Create your command Define request parameters Implement the execute() method Implement an appropriate ResponseObject Add new command to command.properties.in

Management Layer

Management layer is collection of Managers

o Managers are responsible for directing a specific area of the cloud Storage Manager Manages primary storage server (allocation, life-cycle, attach, detach, user volumes, life-cycle of the primary storage server itself) Network Manager Manages network configurations, IP Allocations, Port Forwarding, Load Balancers etc. User Vm Manager Manages life-cycle of VMs created in the cloud And many more!!!

Managers coordinate with each other to achieve a task

Management Layer: Adapters

Modularization and customization within the CloudStack

management server is achieved through the use of the Adapter framework.

Each Adapter is uniquely identified by the interface it exposes Adapters provide extensibility and in many cases device

and represents the boundary between CloudStack and the individual component and/or processes that can be configured into the system specific implementation details while maintaining a simple and consistent interface.

Management Layer: Adapters

Adapters are executed as a chain in the order that they are configured Defined in cloudstack-oss/client/tomcatconf/components.xml.in
<adapter name="StorageNetworkGuru class="com.cloud.network.guru.StorageNetworkGuru"/> <adapter name="ExternalGuestNetworkGuru" class="com.cloud.network.guru.ExternalGuestNetworkGuru"/> <adapter name="PublicNetworkGuru" class="com.cloud.network.guru.PublicNetworkGuru"/> <adapter name="PodBasedNetworkGuru" class="com.cloud.network.guru.PodBasedNetworkGuru"/> <adapter name="ControlNetworkGuru" class="com.cloud.network.guru.ControlNetworkGuru"/> <adapter name="DirectNetworkGuru" class="com.cloud.network.guru.DirectNetworkGuru"/> <adapter name="DirectPodBasedNetworkGuru" class="com.cloud.network.guru.DirectPodBasedNetworkGuru"/> <adapter name="OvsGuestNetworkGuru" class="com.cloud.network.guru.OvsGuestNetworkGuru"/>

<adapters key="com.cloud.network.guru.NetworkGuru>

</adapters>

Adapter Interfaces Available

Discoverer StoragePoolDiscoverer StoragePoolAllocator ConsoleProxyAllocator Investigator FenceBuilder DeploymentPlanner NetworkGuru NetworkElement And more

VirtualMachineGuru HypervisorGuru Listener UserAuthenticator SecurityChecker

Adapters: VM orchestration

Deployment Planner o First Fit planner Host Allocator

Storage Allocator
o First Fit o Random o First Fit o Random

Adapters: Network Orchestration

Network Guru (Responsible for L2-L3) o Design o Implement o Allocate o Release o Shutdown e.g. guest network guru, OVS network guru etc

Network Element (Responsible for L4-L7) o Implement o Shutdown e.g. F5, SRX, NetScaler, Virtual Router

Extending CloudStack Networking

1. prepare (part of start vm) 2. prepare (Network, Nic, DeployDestination, VmInfo)

Network Manager

Network Element

PluggableServi ce

DnsService

MyDnsDeviceS ervice

Device Configuration Admin API (CRUD)

3. addDnsRecord(ip, fqdn)

Demonstrates one way to inform an external DNS server when an instance starts. Classes shaded blue form a plugin / service bundle to integrate an external DNS server. Clients of the instance can then use DNS names to access the instance.

MyDnsElement

MyDnsDeviceM anager

MySQL
AgentMana ger Queue

4.Enqueue AddDnsRecord

MyDnsDeviceR esource
5.API call to Dns Device

Sequence Flow for VM Creation

Job Threads Services API User VM Mgr VirtualMac hine Mgr Network Mgr Storage Mgr Network Guru Network Element Templat e Mgr
Deployme nt Planner

Server Resource s

Start VM Start User VM

Start VM Get a Deployment Plan (Host and StoragePool) Prepare Nics Reserve resources for Nic Notify that Nic is about to be started in network Agent Calls Prepare Volumes Prepare template on Primary Storage Agent Start VM Call

Agent Calls

Stores job result

Management Layer: Adapters flow

Server Resources
Agent
Hypervisor Resources

Resources are carried in

Network Resources Storage Resources Image & Template Resources Snapshot Resources

service VMs to be in close network proximity to the physical resources it manages Easily scales to utilize the most abundant resource in data center (CPU & RAM) Communicates with Orchestration Server over message bus (JSON) Can be replicated for fault tolerance Control gateway to resources within data center

Resource API

Resource Layer

Working toward 4.1 release

4.1 is next major release

o Moving away from monolithic architecture to loosely coupled subsystems o Spring for IOC container and AOP o Storage subsystem refactoring o Network subsystem refactoring o New orchestration engine o Regions support

Session 3 Developing with DevCloud

DevCloud

CloudStack requires
o Hypervisor o Network o Storage

DevCloud

self-contained CloudStack runs in the

appliance

DevCloud

Several use cases

o Try CloudStack in an isolated sandbox. Runs within the appliance o Develop CloudStack on own machine, build locally and deploy new version in DevCloud (Build and test) o Develop and Run locally, use DevCloud as Xen hosts

Thanks