Docker is a relatively new technology platform that helps teams develop, deploy and scale applications with greater ease and speed. Docker has become stable, more accessible and is well-supported by key industry players. While the platform and tools are still maturing, one day Docker may become commonplace in development and deployment workflows.
My experience with Docker
I heard about Docker a couple of years ago and thought it could be a useful tool to help developers package, deploy and run applications. While I do explore and play with new technology, I am generally not an early adopter of significant pieces of technology until it has proven to be stable and supported.
Since its initial release in 2013, Docker has grown up. It is now well-funded by VC money and is recognised by industry giants including Amazon, Google and Microsoft. The tools are fairly stable and documentation is plentiful. It was time for me to experiment and learn more about the Docker platform.
Before Docker, my experience of application deployment and server administration was limited to using technologies and services such as Amazon EC2, DigitalOcean, Heroku, Cloud66, Capistrano, Vagrant and shell scripting. At the time of writing, I have not had much experience with configuration management tools such as Chef, Puppet, Salt and Ansible, but I understand Ansible and Docker work well together so maybe I will look at Ansible in the near future.
What is Docker?
From Docker's website:
Docker allows you to package an application with all of its dependencies into a standardised unit (container images) for software development.
Containers running on a single host machine share resources more efficiently than traditional VMs while still being isolated from one another. Docker container images are composed of layers of file systems making them more lightweight and portable.
As configs, dependencies and code are packaged together, containers are guaranteed to always run the same, regardless of the environment it is running in. This should allow teams to develop, deploy and scale faster.
Docker is not a completely new technology platform. It has been built upon the hard work of open source communities, who have contributed to technologies including UnionFS, cgroups and LXC. What makes Docker a compelling platform is the toolset and ecosystem Docker Inc. has created to allow developers to easily leverage this technology to improve their development workflows.
I got started reading and skimming the documentation on Docker's website. The documentation has a great page which explains Docker's architecture and its major components.
I installed Docker and went through their practical 'Get Started' guide which covers installation and an introduction to images and containers. I ran a 'hello world' image, built my own image, and pushed my image to Docker Hub (a GitHub for hosting, building and distributing container images).
Next, I went through their 'Self-Paced Training' video series. It consists of three 1-hour practical videos which cover all the various Docker components and tools. I learnt how to:
- Write Dockerfiles and build images using Docker Engine.
- Leverage GitHub and Docker Hub to host my Dockerfiles, Docker images and automate my image builds.
- Run, network, manage and troubleshoot containers.
- Provision and manage a Docker host in the cloud using Docker Machine.
- Create and manage multi-container applications using Docker Compose.
- Cluster Docker hosts and schedule containers using Docker Swarm.
- And much more.
I highly recommend the video series to anyone interested in getting hands-on with the Docker toolset and ecosystem.
Using Docker for Real
At the time of writing, I was operating a couple of Twitterbot containers and also a collection of containers running this Ghost (blogging platform) blog. The containers are sharing a single Docker host in the cloud. I look forward to packing more containers on this small VPS and attempting to run Dockerized Ruby on Rails applications in a production environment.
Docker Inc. and others companies have created tools and services to automate and enhance workflows, making it even easier for teams to develop and deploy with Docker. As Docker becomes mainstream, I hope to see a richer ecosystem as companies aggressively compete in this new space.
Automated Image Builds with Docker Hub and Dockerfiles
The Docker Hub automated build workflow and the Dockerfile DSL allows me to easily build, maintain, update and deploy my container images.
On GitHub, I have a repository of Dockerfiles that I have created. A Dockerfile is a text document containing a series of commands to build a Docker image. Container images are hosted on a Registry (e.g. Docker Hub, Quay) where they can be easily shared and distributed.
Docker's layered file system allow Dockerfiles to be built upon another image. If I required Ruby I would simply use one of the official Ruby images as a base for my own image. As a result, my image builds faster and I avoid having to write commands to download and compile Ruby in my Dockerfile.
In addition to hosting and distribution of images, Docker Hub can also automatically build images from Dockerfiles on GitHub or Bitbucket. All my Docker Hub image repositories will automatically start a new image build whenever I push any changes to my GitHub Dockerfiles repository.
Docker Compose for Multi-Container Applications and Microservices
If you are using Docker for non-trivial purposes, you are likely building applications comprised of many containers. Each container would be responsible for one thing and would run a single process or service. With monolithic applications being viewed more as an anti-pattern today, the microservices architecture is becoming more widely adopted. Docker's platform enables and assists in the distribution and deployment of microservice applications.
Docker Compose allows us to configure, deploy and manage multi-container applications. With a Compose file, we can define the configuration of an application's services, how containers are networked together, what resources are shared and much more. With a single command, we can build and start all the services from the Compose file.
I have a GitHub repository of Docker Compose files that I have created. I use Docker Compose and my Compose file, 'ghost-ssl' to build, start and manage this multi-container blogging application.
Running this Blog on Docker
This blog is configured and managed using the Docker platform and toolset. It is a multi-container application, composed of four services (containers):
- Ghost (blogging platform) application using a custom image configured for Mailgun (email service).
- Nginx web server using a custom image configured for Ghost and SSL.
- Data volume container to hold persistent Ghost data (database, images, custom themes).
- Data volume container to hold the SSL certificate and private key for Nginx.
The custom images are based on the official images which are maintained by Docker Inc. The Dockerfiles for my custom images are on my GitHub repository, which are automatically built and distributed by Docker Hub.
Data Volume Containers
For this blogging application, I am using data volume containers to hold persistent data such as the blog database and the SSL private key. This means my containers can be ephemeral. I can stop, destroy and rebuild my containers as I wish without loss of data or need for reconfiguration. The new container simply mounts the existing data volume so there is no need to transfer or restore files.
For security reasons (and to separate concerns), the SSL certificate and key is on its own data volume, only accessible by the Nginx container. If an adversary gains complete control of the Ghost container, they would not be able to access the coveted SSL private key.
Containers as Cattle, not Pets
My containers are not special snowflakes. I can treat my containers like cattle and not like pets. This allows tasks such as upgrading Ghost to be an easy and simple process. There is no need to follow that tedious manual process for non-Dockerized Ghost instances. To upgrade Ghost to the latest version, all I need to do is:
- Update my GitHub hosted Dockerfile to use the latest image of Ghost. Docker Hub will automatically build a new version of my custom image.
- Use Docker Compose to recreate the Ghost container with the command
docker-compose up -d.
What is next?
Clustering, Scheduling and Scaling
At the time of writing, Docker Swarm recently got out of beta and became 'production-ready.' Docker Swarm promises to allow teams to create and manage a cluster of Docker hosts in which containers can be scheduled and scaled.
Compose/Swarm integration is still experimental and work is in progress to allow Compose applications to work out of the box. With the recent announcement of multi-host networking, hopefully we will see better Compose/Swarm integration and seamless workflows.
While we wait for Docker Inc. to enhance their clustering and orchestration tools, there are other tools and services filling this gap. We have paid services including Cloud66 and Docker-acquired Tutum which provides an all-in-one solution for deploying and scaling applications on orchestrated clusters. There is also Amazon EC2 Container Service and Google Container Engine which interfaces with their own respective cloud platform.
There are also open-source tools including CoreOS's fleet and Google's Kubernetes, which help teams manage clusters. I expect Docker Inc. are working hard on Docker Swarm to eventually catch up and provide a native experience and better integration than their third-party alternatives.
I am looking forward to future Docker releases and perhaps I will explore this subject further once Docker Swarm matures.
Dockerizing Ruby on Rails Applications for Production
While there are already proprietary services that help teams manage and deploy Dockerized applications, the real test for me is being able to do this for myself. At the time of writing, there was not much public information on how to deploy and manage a Ruby on Rails application for a production environment.
The next step for me is to come up with a sensible workflow using the Docker tools and platform to allow me to deploy a Ruby on Rails application for a production environment. I should be able to perform typical deployment and administrative tasks including:
- Monitoring and logging
- Database backup, replication and restoration
- Zero-downtime deployment
- Continuous integration and deployment
I look forward to this challenge and hopefully I can share my successful experiences and learnings.
Competitors, Open Standards and the Future
There was a bit of tension in late 2014 when CoreOS announced their own container format, rkt (Rocket). But since then, CoreOS and Docker have reconciled and the Open Container Initiative (OCI) was established in June 2015. With the support of industry leaders, the OCI's purpose is to create an open standard for container formats and runtime. Docker has donated its container format and runtime to the cause.
With the financial and technological backing of big industry players, Docker has the potential to disrupt the virtualisation space. However, it is too early to declare that Docker will be a commonplace technology platform. With rapid changes, new competitors and new developments with Open Container Format, it will definitely be an exciting story to follow.
I have been using Docker and have been impressed with its platform. It is early days for Docker and there is still some work to enable containerization to become a common practice in application development. I look forward to experimenting and reading more about Docker and their competing technologies.