Microservices,

Containers and Docker

by Joost Hietbrink
April, 2015
 Joost?Linkedin - Github

YelloYello
Sold in 2008 to Yellow Pages Fastest Site Builder
StartupBootcamp Batch 2014
Goal?
Make you
look smart
in the pub!
Buzzword bingo..
& bignumber bingo..
 Agenda:
Netflix
Microservices?
Containers?
Docker?
 Netflix
Microservices?
Containers?
Docker?
 $32.900.000.000 company (32.9B)

$2.390.000 revenue per employee (#1) (Apple $2,21M)

57.000.000 subscribers

40-60 countries (only US until 2010)

1997 (yep.. 18 years ago)

Reed Hastings
CEO Netflix
1997 (yep.. 18 years ago)
1998 (yep.. 17 years ago)
2007 (10 years later..)

1.000.000.000
DVDs delivered
2007 (10 years later..)
2008
2010

Move to cloud
(AWS)
2010
2010

link
2015
10x customers
20x visits
100x traffic
5x devs
200 devs 1.000 product devs

2x costs of IT Infrastructure in 2008

 > 1/3 of US traffic is Netflix
7.000.000.000 hours streamed (Q3 2014)
10.000.000.000.000.000.000 bytes (10.000PB)

10.000 starts / second

2.000.000.000 edge API requests per day

2.000.000 payment transactions per day

60.000 instances (AWS)

40.000.000.000.000.000 bytes S3 storage (40PB)

>500 microservices

1.500 employees (1.000 devs)

~30 engineering teams (microservices)

 Netflix
Microservices?
Containers?
Docker?
 Loosely coupled service oriented
architecture with bounded contexts

Adrian Cockcroft, ex-Cloud Architect @ Netflix

SOA > Microservice
link
link
Characteristics of a Microservice Architecture

Componentization via Services

Organized around Business Capabilities
Products not Projects
Smart endpoints and dumb pipes
Decentralized Governance
Decentralized Data Management
Infrastructure Automation
Design for failure
Evolutionary Design
Martin Fowler, loud-mouth
 Any organization that designs a system (defined broadly) will
produce a design whose structure is a copy of the organization's
communication structure.

Melvyn Conway, 1967

link
Characteristics of a Microservice Architecture

Componentization via Services

Organized around Business Capabilities
Products not Projects
Smart endpoints and dumb pipes
Decentralized Governance
Decentralized Data Management
Infrastructure Automation
Design for failure
Evolutionary Design
Martin Fowler, loud-mouth
 Unix Philosophy
- 1978 -

Write programs that do one thing

and do it well.
Write programs to work together.

~ tr 'A-Z' 'a-z' < doc.txt | tr -cs 'a-z' '\n'

| sort | uniq | comm -23 - /usr/share/dict/words

M. Douglas McIlroy (Inventor Unix Pipe)

(Head Bell Labs Computing Sciences Research Center)

Characteristics of a Microservice Architecture

Componentization via Services

Organized around Business Capabilities
Products not Projects
Smart endpoints and dumb pipes
Decentralized Governance
Decentralized Data Management
Infrastructure Automation
Design for failure
Evolutionary Design
Martin Fowler, loud-mouth
Why?
 Smaller footprint:
easier to develop
easier deploy
own responsibility
Loosely coupled:
develop independently (own stack)
update independently (own versions)
scale independently
fail independently (own bugs)
How?
 Netflix
Microservices?
Containers?
Docker?
March 2013
docker

jan 2014 - $15.000.000

sep 2014 - $40.000.000
apr 2015 - $95.000.000
 Docker is an open-source project that automates the
deployment of applications inside software containers, by
providing an additional layer of abstraction and automation
of operating-system-level virtualization on Linux.
 The Challenge
User DB

Do services and apps

Multiplicity of Stacks

appropriately?
Static website postgresql + pgv8 + v8
Queue Analytics DB

interact
nginx 1.5 + modsecurity + openssl + bootstrap 2 hadoop + hive + thrift + OpenJDK
Redis + redis-sentinel

Web frontend
Background workers
Ruby + Rails + sass + Unicorn
Python 3.0 + celery + pyredis + libcurl + ffmpeg + libopencv + nodejs + phantomjs
API endpoint
Python 2.7 + Flask + pyredis + celery + psycopg + postgresql-client

Production Cluster
Multiplicity of

Development VM
environments

Public Cloud
hardware

smoothly and
Can I migrate
quickly?
QA server
Disaster recovery
Customer Data Center Contributors laptop
Production Servers
The Matrix From Hell
Static website
? ? ? ? ? ? ?
Web frontend
? ? ? ? ? ? ?
Background workers
? ? ? ? ? ? ?
User DB
? ? ? ? ? ? ?
Analytics DB
? ? ? ? ? ? ?
Queue
? ? ? ? ? ? ?
Development Single Prod Contributors Customer
QA Server Onsite Cluster Public Cloud
VM Server laptop Servers
 Do I worry about how Can I transport quickly
goods interact (e.g. and smoothly
coffee beans next to (e.g. from boat to train
spices) to truck)
Cargo Transport Pre-1960
transporting/storing
Multiplicity of Goods methods for
Multipilicity of
Also a matrix from hell

? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
 Solution: Intermodal Shipping Container

Do I worry about how

coffee beans next to
goods interact (e.g.
Multiplicity of Goods

spices)
A standard container that is
loaded with virtually any
goods, and stays sealed until
it reaches final delivery.

in between, can be loaded and

unloaded, stacked, transported

Can I transport quickly

efficiently over long distances,
transporting/storing

(e.g. from boat to

and transferred from one mode of
Multiplicity of

train to truck)

and smoothly
methods for

transport to another
 Docker is a shipping container system for code

Do services and apps

Static website User DB Web frontend Queue Analytics DB
Multiplicity of Stacks

appropriately?
interact
An engine that enables any
payload to be encapsulated
as a lightweight, portable,
self-sufficient container

that can be manipulated using

standard operations and run

smoothly and quickly

Multiplicity of

environments

consistently on virtually any

hardware

Can I migrate
hardware platform

Development QA server Customer Data Public Cloud Production Contributors

VM Center Cluster laptop
Docker eliminates the matrix from Hell
Static website

Web frontend

Background workers

User DB

Analytics DB

Queue

Development Single Prod Contributors Customer

QA Server Onsite Cluster Public Cloud
VM Server laptop Servers
Docker > Containers
images (Dockerfile)
layered filesystem (versioning)
index/hub/repository (push, pull)
machine: create hosts
compose: linking
swarm: clustering
so.. what?
for you as an
Enterprise
 deploy faster
separate per microservices
no downtime
microservice teams
scale faster
a-b testing
make more money
for us all
greener
for you as a
Developer
 get it running your project
(microservices :))
on your computer
on every computer (production, test, that
designer that needs to work a few hours)
no more works on my machine
easily deploy everywhere
 stand on the shoulder of giants
deploy like Netflix
stand on the shoulders of many midgets
try things super quickly
click to deploy (tutum.co)