Someone at Rancher must have some cattle in the middle of Arizona or in the backcountry of California. Or one of their VCs might be in Montana sitting in a big ranch while Docker is eating the IT world. In any case, this post is short and sweet like veggies and not like cattle (TFW) and is about Rancher and the newly announced RancherOS. Check out the rancheros announcement.

Let's keep this short, shall we ? Docker is great, but it is a daemon running on a single host. Since you want to scale :) and operate multiple servers, you need something to manage your Docker containers across multiple hosts. Several solutions are emerging, of course Docker Swarm but also Kubernetes, Lattice from Cloudfoundry and even Apache Mesos. Rancher is one of these cluster management solutions for Docker. It does some nice things like cross-hosts container linking through a custom built network overlay (think Flannel, Weave, Socketplane).

You can use Rancher with any set of Docker hosts. However, a new type of operating systems have started to appear. Container optimized OS. Or Just Enough Operating System for Docker. CoreOS, ProjectAtomic from RedHat, Ubuntu Snappy fit in that space. They aim to provide rolling atomic upgrades to the OS and run everything in it as a container. No more package manager, magic happens and you are always up to date. Package all your apps in containers, and use Rancher to run them in your cluster. End of story. Wait, enters rancherOS.

RancherOS

A couple lines of bash make all the talking:

$ git clone https://github.com/rancherio/os-vagrant.git
$ cd os-vagrant
$ vagrant up
$ vagrant ssh
[rancher@rancher ~]$ docker version
Client version: 1.5.0
…

rancherOS is a super minimalistic OS exclusively for Docker. It goes further and also runs system services as container themselves. And I will let @ibuildthecloud talk about systemd and Docker as PID 1.

[rancher@rancher ~]$ sudo system-docker ps
CONTAINER ID        IMAGE               COMMAND                ...      NAMES
32607470eb78        console:latest      "/usr/sbin/console.s   ...      console             
d0420165c1c0        userdocker:latest   "/docker.sh"           ...      userdocker          
375a8de12183        syslog:latest       "/syslog.sh"           ...      syslog              
d284afd7f628        ntp:latest          "/ntp.sh"              ...      ntp   

The next logical question is of course....drum roll... Can I run rancher on rancheros. RinR not R&R ? And the answer is a resounding yes. I expect Rancher to come out in the next weeks maybe months with a solid product based on the two.

Rancher

If you are interested to try out RinR then check out the Ansible playbook I just made. You can use use it to deploy a cluster of rancherOS instances in AWS, and use one of them as a master and the others as workers. The master runs in a container:

$ docker run -d -p 8080:8080 rancher/server 

And the workers can register with their agent:

$ sudo docker run --rm -it --privileged -v /var/run/docker.sock:/var/run/docker.sock rancher/agent http://<master_ip>:8080

Once all the workers have registered you can use the UI or the API to start containers.


As you can see I tested this at web scale with two nodes :)

Notes

In this very early super bleeding-edge testing phase (as you can tell in my good spirit today), I did find a few things that were a bit strange. Considering rancherOS was announced just last week, I am sure things will get fixed. Cloud-init support is minimal, not able to add second network interface, support for both keypair and userdata at the same time seems off. The UI was a bit slow to start and building the overlay was also a bit slow. It is also possible that I did something wrong.

Overall though, rancher is quite nice. It builds on years of experience in the team with developing CloudStack and operating clouds at scale and applies it to the Docker world. It does seem that they want to integrate with and provide the native Docker API, this would mean that users will be able to use Docker machine to add hosts to a rancher cluster, or even Docker swarm and that launching a container would also be a docker command away. How that differentiates from Swarm itself is not yet clear, but I would bet we will see additional networking and integration services in Rancher. Blurring the lines with Kubernetes ? Time will tell.

There has been a lot of discussions lately within the OpenStack community on the need for an AWS API interface to OpenStack nova. I followed the discussion from far via a few tweets, but I am of the opinion that any IaaS solution does need to expose an AWS interface. AWS is the leader in Cloud and has been since 2006 -yes that's seven years- Users are accustomed to it and the AWS API is the de-factor standard.

When Eucalyptus started, it's main goal was to become an AWS clone and in 2012 signed an agreement with Amazon to offer seamless AWS support in Eucalyptus. Opennebula has almost always offered an AWS bridge and CloudStack has too, even though in total disclosure the interface was broken in the Apache CloudStack 4.1 release. Thankfully the AWS interface is now fixed in 4.1.2 release and will also be in the upcoming 4.2 release. To avoid breaking this interface we are developing a jenkins pipeline which will test it using the Eucalyptus testing suite.

Opennebula recently ran a survey to determine where to best put its efforts in API development. The results where clear with 47% of respondents asking for better AWS compatibility. There are of course developing official standards from standard organizations, most notably OCCI from OGF and CIMI from DMTF. The opennebula survey seems to indicate a stronger demand for OCCI than CIMI, but IMHO this is due to historical reasons: Opennebula early efforts in being the first OCCI implementation and Opennebula user base especially within projects like HelixNebula.

CIMI was promising and probably still is but it will most likely face an up-hill battle since RedHat announced it's scaling back on supporting Apache DeltaCloud. I recently heard about a new CIMI implementation project for Stratuslab from some of my friends at Sixsq, it is interesting and fun because written in Clojure and I hope to see it used with Clostack to provide a CIMI interface to CloudStack. We may be couple weeks out :)

While AWS is the de-facto standard, I want to make sure that CloudStack offers choices for its users. If someone wants to use OCCI and CIMI or AWS or the native CloudStack API they should be able to. I will be at the CloudPlugfest Interoperability week in Madrid Sept 18-20 and I hope to demonstrate a brand new OCCI interface to CloudStack using rOCCI and CloudStack ruby gem. A CloudStack contributor from Taiwan has been working on it.

The main issue with all these "standard" interfaces is that they will never give you the complete API of a given IaaS implementation. They by nature provide the lowest common denominator. That roughly means that the user facing APIs could be standardized but the administrator API will always remain hidden and non-standard. In CloudStack for instance, there are over 300 API calls. While we can expose a compatible interface, this will always only cover a subset of the overall API. It also brings the question of all the other AWS services: EMR, Beanstalk, CloudFormation... Standardizing on those will be extremely difficult if not impossible.

So yes, we should expose an AWS compatible interface, but we should also have OCCI, CIMI and of course our native API. Making those bridges is not hard, what's hard is the implementation behind it.

All of this would leave us with Google Compute Engine (GCE), and I should be able to bring back some good news by the end of september, stay tuned !!!

A vote has started on the libcloud dev list for the 0.13 release. The release notes detail all the new features and fixes. I am stoked about this because a few patches that I submitted are included in this release candidate. I patched the CloudStack driver quite a bit to improve the support for Clouds with a basic zone like Exoscale. There is more work to do on this driver including better support for Advanced zone especially for port forwarding, firewall rules and more unit tests. A the CloudStack hackathon last sunday @pst418 submitted a few patches for unit tests and they made it as well into 0.13 RC, terrific.

If you don't know libcloud, it's a python based API wrapper to abstract the various cloud APIs. With libcloud you can create connections to multiple clouds, potentially using different API. At a high level, it is similar to jclouds for JAVA or deltacloud written in ruby. There was already a CloudStack driver but its functionality was limited. If you grab my quickie libcloud shell, you can try to follow this walkthrough of what you can do with libcloud and a CloudStack basic zone. Of course you will need a CloudStack endpoint.

Start the libshell and check what zone you are on:

$ python ./libshell.py 
Hello LibCloud Shell !!
You are running at: https://api.exoscale.ch/compute
>>> conn.list_locations()
[<NodeLocation: id=1128bd56-b4d9-4ac6-a7b9-c715b187ce11, name=CH-GV2, country=AU, driver=CloudStack>]

You might notice a wrong country code, it is hard-coded in libcloud, I need to file a bug for this. Get the list of templates (or images in libcloud speak):

>>> conn.list_images()
[<NodeImage: id=01df77c3-0150-412a-a580-413a50924a18, name=Windows Server 2008 R2 SP1, driver=CloudStack  ...>,
 <NodeImage: id=89ee852c-a5f5-4ab9-a311-89f39d133e88, name=Windows Server 2008 R2 SP1, driver=CloudStack  ...>,
 <NodeImage: id=ccd142ec-83e3-4108-b1c5-8e1fdb353ff9, name=Windows Server 2008 R2 SP1, driver=CloudStack  ...>,
 <NodeImage: id=f2101a0c-eaf7-4760-a143-0a5c940fd864, name=Windows Server 2008 R2 SP1, driver=CloudStack  ...>,
 <NodeImage: id=77d32782-6866-43d4-9524-6fe346594d09, name=CentOS 5.5(64-bit) no GUI (KVM), driver=CloudStack  ...>,
 <NodeImage: id=29cba09f-4569-4bb3-95e7-71f833876e3e, name=Windows Server 2012, driver=CloudStack  ...>,
 <NodeImage: id=ee241b47-4303-40c8-af58-42ed6bf09f8c, name=Windows Server 2012, driver=CloudStack  ...>,
 <NodeImage: id=754ea486-d649-49e5-a70e-5e5d458f0df0, name=Windows Server 2012, driver=CloudStack  ...>,
 <NodeImage: id=0f9f4f49-afc2-4139-b26b-b05a9f51ea74, name=Windows Server 2012, driver=CloudStack  ...>,
 <NodeImage: id=954752a8-0486-46bb-8e3f-0adb3e01c619, name=Linux CentOS 6.4 64-bit, driver=CloudStack  ...<]

I cut the previous output, but there are also Ubuntu and CentOS images on that cloud...You can then list the various instance types or sizes in libcloud speak.

>>> conn.list_sizes()
[<NodeSize: id=71004023-bb72-4a97-b1e9-bc66dfce9470, name=Micro, ram=512 disk=0 bandwidth=0 price=0 driver=CloudStack ...>,
 <NodeSize: id=b6cd1ff5-3a2f-4e9d-a4d1-8988c1191fe8, name=Tiny, ram=1024 disk=0 bandwidth=0 price=0 driver=CloudStack ...>,
<NodeSize: id=21624abb-764e-4def-81d7-9fc54b5957fb, name=Small, ram=2048 disk=0 bandwidth=0 price=0 driver=CloudStack ...>,
 <NodeSize: id=b6e9d1e8-89fc-4db3-aaa4-9b4c5b1d0844, name=Medium, ram=4096 disk=0 bandwidth=0 price=0 driver=CloudStack ...>, 
<NodeSize: id=c6f99499-7f59-4138-9427-a09db13af2bc, name=Large, ram=8182 disk=0 bandwidth=0 price=0 driver=CloudStack ...>, 
<NodeSize: id=350dc5ea-fe6d-42ba-b6c0-efb8b75617ad, name=Extra-large, ram=16384 disk=0 bandwidth=0 price=0 driver=CloudStack ...>,
 <NodeSize: id=a216b0d1-370f-4e21-a0eb-3dfc6302b564, name=Huge, ram=32184 disk=0 bandwidth=0 price=0 driver=CloudStack ...>]

What I added is the management of ssh key pairs and security groups, you can now list, create and delete both keypairs and security groups, and use those when deploying nodes. (Don't bother trying to do anything with the keys below, I deleted everything.)

>>> conn.ex_list_keypairs() 
[{u'name': u'foobar', u'fingerprint': u'b9:2d:4b:07:db:e7:3e:42:17:11:22:33:44:55:66:77'}]

>>> conn.ex_list_security_groups()
[{u'egressrule': [], u'account': u'foobar@gmail.com', u'domainid': u'ab53d864-6f78-4993-bb28-9b8667b535a1', u'description': u'Default 
Security Group', u'tags': [], u'domain': u'foobar@gmail.com', u'ingressrule': [{u'startport': 22, u'cidr': u'0.0.0.0/0', u'protocol':
 u'tcp', u'endport': 22, u'ruleid': u'b83428c0-7f4c-44d1-bc96-4e1720168fdf'}, {u'startport': 80, u'cidr': u'0.0.0.0/0', u'protocol':
 u'tcp', u'endport': 80, u'ruleid': u'042124dd-652d-4fa2-8bee-d69973380f21'}], u'id': u'ebfa2339-e9ae-4dcb-b73c-a76cd3fce39e', u'name': 
u'default'}, {u'account': u'foobar@gmail.com', u'domainid': u'ab53d864-6f78-4993-bb28-9b8667b535a1', u'description': u'this is a test',
 u'domain': u'foobar@gmail.com', u'id': u'b7f5fbad-4244-491f-9547-c91a010e0c9d', u'name': u'toto'}]

>>> conn.ex_create_keypair('test')
{u'privateKey': u'-----BEGIN RSA PRIVATE KEY-----\nMIICXAIBAAKBgQCAuPufJnbzFkyIUaMymkXwfilA5qS2J9fx2Q3EWNDs1m89FLFq\n19ERjG43ZvRuI/KGwWZwHbhqoqJth7WQYNe
lYCmOUYRaepxTrpU4TGDGuhqMh9D9\
noNMIFx3ktkcTitxkSY/5h/pXqSB2XXURLpZWwZxjEYwWCpWE8i7uVtIR7wIDAQAB\nAoGAOv0Kik9lOVbhsaK/yAO8w039d7l6h+NQaYtPbMhKzg4iofomp9DJBWK2a3sp\
nzoN4s9pTKFPmXC+1gb4sLULD72ENSgyozrPMCJ0tytNa3ebVCCYDvlagEZ83KwGn\nnr2BIRLul1xyHVa+amvTemuxi7OOx0u/0aZ6jPVW9ocPahkCQQDnUACG3Q2p60Mx\
nCttW7Go2wz0BhaI8ibG8rHorhdrFJUrsTI6QrLh340uHCIEAuWUXjYDX2u5MTPBG\njWbL/A/9AkEAjnX8EzO/B/RooFTxhYdxv7D1Dzcx4H59mFzjez6N/mjAHjKL4p66\
nqGFgLa9HMhDPFOs83VetBXcihu1vueffWwJBALQUD2TvAS0wz82FYz8nrITXuE3Q\nCH7Sv8FgEXiCq89hehO+ghrVrIMBPBJzJ2M18iLE8fKaKXzTRRfYC5hwss0CQHY8\
nBdIKCGoZtxwaY7lnCEkIHNtb+9FOKf7iWQpYiJC1b32ghei3xEMrTh+ccYJj4PqD\noigyNC9tCQLi3O92OjECQGYR2z8IDlfnl7G8p7eiyBciuoDyyq5/oJHhkHbkbwzr\
nW0Uun+rEcjRnXbUN8wUQ6FSFrxk2VSajbCBteTOrF24=\n-----END RSA PRIVATE KEY-----\n', u'name': u'test', u'fingerprint': u'43:59:7e:00:16:45:fc:ab:81:55:03:47:12:22:1e:d5'}

>>> conn.ex_list_keypairs()
[{u'name': u'test', u'fingerprint': u'43:59:7e:00:16:45:fc:ab:81:55:03:47:12:22:1e:d5'}, {u'name': u'foobar',
 u'fingerprint': u'b9:2d:4b:07:db:e7:3e:42:17:11:22:33:44:55:66:77'}]

>>> conn.ex_delete_keypair('test')
u'true'

>>> conn.ex_list_keypairs()
[{u'name': u'foobar', u'fingerprint': u'b9:2d:4b:07:db:e7:3e:42:17:11:22:33:44:55:66:77'}]

>>> conn.ex_create_security_group('heyhey')
{u'account': u'runseb@gmail.com', u'domainid': u'ab53d864-6f78-4993-bb28-9b8667b535a1', u'id':
 u'77ad73b5-a383-4e13-94be-a38ef9877996', u'domain': u'runseb@gmail.com', u'name': u'heyhey'}

>>> conn.ex_list_security_groups()
[{u'egressrule': [], u'account': u'foobar@gmail.com', u'domainid': u'ab53d864-6f78-4993-bb28-9b8667b535a1',
 u'description': u'Default Security Group', u'tags': [], u'domain': u'foobar@gmail.com', u'ingressrule': [{u'startport': 22, u'cidr': 
u'0.0.0.0/0', u'protocol': u'tcp', u'endport': 22, u'ruleid': u'b83428c0-7f4c-44d1-bc96-4e1720168fdf'}, {u'startport': 80, u'cidr': u'
0.0.0.0/0', u'protocol': u'tcp', u'endport': 80, u'ruleid': u'042124dd-652d-4fa2-8bee-d69973380f21'}], u'id': u'ebfa2339-e9ae-4dcb-b73c-a76cd3fce39e',
 u'name': u'default'}, {u'account': u'foobar@gmail.com', u'domainid': u'ab53d864-6f78-4993-bb28-9b8667b535a1', u'description':
 u'this is a test', u'domain': u'foobar@gmail.com', u'id': u'b7f5fbad-4244-491f-9547-c91a010e0c9d', u'name': u'toto'}, 
{u'account': u'foobar@gmail.com', u'domainid': u'ab53d864-6f78-4993-bb28-9b8667b535a1', u'id': u'77ad73b5-a383-4e13-94be-a38ef9877996',
 u'domain': u'foobar@gmail.com', u'name': u'heyhey'}]

>>> conn.ex_delete_security_group('heyhey')
u'true'

>>> conn.ex_list_security_groups()
[{u'egressrule': [], u'account': u'runseb@gmail.com', u'domainid': u'ab53d864-6f78-4993-bb28-9b8667b535a1',
 u'description': u'Default Security Group', u'tags': [], u'domain': u'foobar@gmail.com', u'ingressrule': [{u'startport': 22, u'cidr':
 u'0.0.0.0/0', u'protocol': u'tcp', u'endport': 22, u'ruleid': u'b83428c0-7f4c-44d1-bc96-4e1720168fdf'}, {u'startport': 80, u'cidr': 
u'0.0.0.0/0', u'protocol': u'tcp', u'endport': 80, u'ruleid': u'042124dd-652d-4fa2-8bee-d69973380f21'}], u'id': u'ebfa2339-e9ae-4dcb-b73c-a76cd3fce39e',
 u'name': u'default'}, {u'account': u'foobar@gmail.com', u'domainid': u'ab53d864-6f78-4993-bb28-9b8667b535a1',
 u'description': u'this is a test', u'domain': u'foobar@gmail.com', u'id': u'b7f5fbad-4244-491f-9547-c91a010e0c9d', u'name': u'toto'}]

With a security group created you can now add an ingress rule:

>>> conn.ex_authorize_security_group_ingress(securitygroupname='toto',protocol='TCP',cidrlist='0.0.0.0./0',
startport=99)
{u'egressrule': [], u'account': u'runseb@gmail.com', u'domainid': u'ab53d864-6f78-4993-bb28-9b8667b535a1', 
u'description': u'this is a test', u'domain': u'runseb@gmail.com', u'ingressrule': [{u'startport': 99, u'cidr': u'0.0.0.0./0', 
u'protocol': u'tcp', u'endport': 99, u'ruleid': u'a13a21f9-1709-431f-9c7d-e1a2c2caacdd'}], 
u'id': u'b7f5fbad-4244-491f-9547-c91a010e0c9d', u'name': u'toto'}

>>> conn.ex_list_security_groups()
[{u'egressrule': [], u'account': u'runseb@gmail.com', u'domainid': u'ab53d864-6f78-4993-bb28-9b8667b535a1', 
u'description': u'Default Security Group', u'tags': [], u'domain': u'runseb@gmail.com', u'ingressrule': [{u'startport': 22, u'cidr':
 u'0.0.0.0/0', u'protocol': u'tcp', u'endport': 22, u'ruleid': u'b83428c0-7f4c-44d1-bc96-4e1720168fdf'}, {u'startport': 80, u'cidr':
 u'0.0.0.0/0', u'protocol': u'tcp', u'endport': 80, u'ruleid': u'042124dd-652d-4fa2-8bee-d69973380f21'}], u'id':
 u'ebfa2339-e9ae-4dcb-b73c-a76cd3fce39e', u'name': u'default'}, {u'egressrule': [], u'account': u'runseb@gmail.com', u'domainid': 
u'ab53d864-6f78-4993-bb28-9b8667b535a1', u'description': u'this is a test', u'tags': [], u'domain': u'runseb@gmail.com', u'ingressrule': 
[{u'startport': 99, u'cidr': u'0.0.0.0./0', u'protocol': u'tcp', u'endport': 99, u'ruleid': u'a13a21f9-1709-431f-9c7d-e1a2c2caacdd'}], 
u'id': u'b7f5fbad-4244-491f-9547-c91a010e0c9d', u'name': u'toto'}]

Great, keypair and security group work fine. Now let's do the basics of starting an instance and let's test the suspend and resume which was not in the Driver until now. Nothing earth shattering but it's an improvement.

>>> size=conn.list_sizes()
>>> image=conn.list_images()

>>> n=conn.create_node(name='yoyo',size=size[0],image=image[0])

>>> n.ex_stop()
u'Stopped'
>>> n.ex_start()
u'Running'

And that's it for now, CloudStack support is getting better, there is still tons of work to do: improve the advanced zone support, check the load-balancer and storage support, add unit-tests and new CloudStack features like auto-scaling. Carry on !!!!