By Andreas Jansson (@ndreasa)
# fabfile.py
from headintheclouds.tasks import *
from headintheclouds import ensemble
from headintheclouds import ec2
from headintheclouds import digitalocean
from headintheclouds import docker
$ fab create:provider=ec2,name=myserver,size=m1.small
Waiting for instance to start [pending]
Waiting for instance to start [pending]
Waiting for instance to start [pending]
Waiting for instance to start [pending]
Waiting for instance to become accessible
Waiting for instance to become accessible
Waiting for instance to become accessible
Waiting for instance to become accessible
Done.
$ fab nodes
[54.205.54.241] Executing task 'nodes'
ec2
name size ip internal_ip state created
myserver m1.small 54.205.54.241 10.211.67.146 running 2014-04-29 22:48:36-04:00
digitalocean
name size ip state
Done.
$ fab -R myserver ssh
[54.205.54.241] Executing task 'ssh'
[localhost] local: ssh -o StrictHostKeyChecking=no -i "/home/andreas/.ssh/ec2.pem" ubuntu@54.205.54.241 ""
Welcome to Ubuntu 14.04 LTS (GNU/Linux 3.13.0-24-generic x86_64)
[snip]
Last login: Wed Apr 30 02:57:09 2014 from 209-6-206-91.c3-0.smr-ubr2.sbo-smr.ma.cable.rcn.com
ubuntu@domU-12-31-39-0A-40-64:~$
$ fab pricing
[54.205.54.241] Executing task 'pricing'
ec2
size memory cores storage gpu recent median stddev max cost
t1.micro 0.615 1 ebsonly 0.003 0.003 0.0 0.003 0.02
m1.small 1.7 1 1 x 160 0.007 0.007 0.0 0.007 0.044
m3.medium 3.75 1 1 x 4 SSD 0.008 0.008 0.0 0.008 0.07
c3.large 3.75 2 2 x 16 SSD 0.016 0.016 0.0 0.016 0.105
m3.large 7.5 2 1 x 32 SSD 0.016 0.016 0.002 0.02 0.14
r3.large 15.0 2 1 x 32 SSD 0.175
c3.xlarge 7.5 4 2 x 40 SSD 0.032 0.032 0.0 0.032 0.21
m3.xlarge 15.0 4 2 x 40 SSD 0.032 0.032 0.0 0.032 0.28
r3.xlarge 30.5 4 1 x 80 SSD 0.35
c3.2xlarge 15.0 8 2 x 80 SSD 0.064 0.064 0.0 0.065 0.42
m3.2xlarge 30.0 8 2 x 80 SSD 0.066 0.067 0.001 0.069 0.56
g2.2xlarge 15.0 8 60 SSD gpu 0.65
r3.2xlarge 61.0 8 1 x 160 SSD 0.7
c3.4xlarge 30.0 16 2 x 160 SSD 4.8 0.195 0.969 4.8 0.84
i2.xlarge 30.5 4 1 x 800 SSD 0.853
r3.4xlarge 122.0 16 1 x 320 SSD 1.4
c3.8xlarge 60.0 32 2 x 320 SSD 0.279 0.298 0.176 1.0 1.68
i2.2xlarge 61.0 8 2 x 800 SSD 1.705
r3.8xlarge 244.0 32 2 x 320 SSD 2.8
i2.4xlarge 122.0 16 4 x 800 SSD 3.41
hs1.8xlarge 117.0 16 24 x 2048 4.6
i2.8xlarge 244.0 32 8 x 800 SSD 6.82
digitalocean
size memory cores disk transfer cost
512MB 0.5 1 20 1TB 0.007
1GB 1 1 30 2TB 0.015
2GB 2 2 40 3TB 0.03
4GB 4 2 60 4TB 0.06
8GB 8 4 80 5TB 0.119
16GB 16 8 160 6TB 0.238
32GB 32 12 320 7TB 0.476
48GB 48 16 480 8TB 0.705
64GB 64 20 640 9TB 0.941
96GB 96 24 960 10TB 1.411
Done.
$ fab -R myserver terminate
[54.205.54.241] Executing task 'terminate'
Sleeping for ten seconds so you can change your mind if you want to!!!
Terminating EC2 instance i-040ec554
Done.
automatically installs Docker if it's not there already
$ fab -R myserver docker.run:image=andreasjansson/graphite,name=graphite,ports=80
[54.198.87.139] Executing task 'docker.run'
[snip - installing docker and pulling image]
[54.198.87.139] out: 3891a87d3bed: Download complete
[54.198.87.139] out: 0fef1dcfa1fe84db43d9d0f03b4d89b203744f559add27e181bf8775f26b24e3
[54.198.87.139] out:
[54.198.87.139] sudo: iptables -t nat -A DOCKER -p tcp --dport 80 -j DNAT --to-destination 172.17.0.2:80
Done.
$ fab docker.ps
[54.198.87.139] Executing task 'docker.ps'
name ip ports created image
graphite 172.17.0.2 80:80, 2003:None, 22:None, 7002:None, 2004:None 2014-04-30 03:18:21 andreasjansson/graphite
Done.
Disconnecting from ubuntu@54.198.87.139... done.
you can bind and unbind while the container is running
$ fab -R myserver docker.bind:graphite,2003
[54.198.87.139] Executing task 'docker.bind'
[54.198.87.139] sudo: iptables -t nat -A DOCKER -p tcp --dport 2003 -j DNAT --to-destination 172.17.0.2:2003
Done.
only works if you have an SSH server running inside the container, default user/pass is root/root
$ fab -R myserver docker.ssh:graphite
[54.198.87.139] Executing task 'docker.ssh'
[localhost] local: ssh -A -t -o StrictHostKeyChecking=no -i "/home/andreas/.ssh/ec2.pem" ubuntu@54.198.87.139 sshpass -p 'root' ssh -A -t -o StrictHostKeyChecking=no 'root'@172.17.0.2
Warning: Permanently added '172.17.0.2' (ECDSA) to the list of known hosts.
Welcome to Ubuntu 12.04 LTS (GNU/Linux 3.13.0-24-generic x86_64)
* Documentation: https://help.ubuntu.com/
The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.
Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.
root@0fef1dcfa1fe:~#
$ fab -R myserver docker.kill:graphite
[54.198.87.139] Executing task 'docker.kill'
[54.198.87.139] sudo: iptables -t nat -D DOCKER -p tcp -m tcp --dport 80 -j DNAT --to-destination 172.17.0.2:80
[54.198.87.139] sudo: iptables -t nat -D DOCKER -p tcp -m tcp --dport 2003 -j DNAT --to-destination 172.17.0.2:2003
[54.198.87.139] sudo: docker kill graphite
[54.198.87.139] out: graphite
[54.198.87.139] out:
[54.198.87.139] sudo: docker rm graphite
[54.198.87.139] out: graphite
[54.198.87.139] out:
Done.
# my-environment.yml
wordpress:
provider: digitalocean
size: 512MB
containers:
wordpress:
image: jbfink/docker-wordpress
ports:
- 80
firewall:
80: "*"
$ fab ensemble.up:my-environment
[54.198.87.139] Executing task 'ensemble.up'
Calculating changes...
The following servers will be created:
wordpress
The following containers will be created:
wordpress (wordpress)
The following servers will get new firewalls:
wordpress (1 rules total)
Do you wish to continue? [Y/n] y
>>>>>>>>>>>>>>>>>>>>>>>>> starting <Server: wordpress>
Creating 1 Digital Ocean 512MB droplets
Waiting for droplet to start [pending: 1, running: 0]
[snip - creating server, installing docker, setting up firewall, etc]
$ fab ensemble.up:my-environment
[54.198.87.139] Executing task 'ensemble.up'
Calculating changes.....
Done.
Dependencies, templates, etc.
# wordpress.yml
wordpress:
provider: digitalocean
size: 512MB
containers:
wordpress:
image: jbfink/docker-wordpress
ports:
- 80
serverstats:
image: andreasjansson/collectd-write-graphite
environment:
HOST_NAME: ${host.name}
GRAPHITE_HOST: ${graphite.ip}
# hack to make sure the graphite.containers.graphite container
# is up before we start streaming data to it
_DEPENDS: ${graphite.containers.graphite.ip}
firewall:
template: default_firewall
graphite:
provider: digitalocean
size: 512MB
containers:
graphite:
image: andreasjansson/graphite
ports:
- 80
- 2003
firewall:
template: default_firewall
templates:
default_firewall:
22: "*"
80: "*"
"*/*": $internal_ips
are turned into a DAG, which is traversed twice. Once when figuring out what needs to change to meet the YAML file. And then when actually making those changes.
isn't stored anywhere. The state of the cluster is defined by what is actually in the cluster.
Makes it less error prone (but also a little slower, since you have to interrogate that state every time you ensemble.up
)