Vagrant with Docker: How to set up Postgres, Elasticsearch and Redis on Mac OS X

🚀 Coinbase is looking for DevOps and Software Engineers 🚀

After some time spent looking at Docker from afar, hearing everyone talk about how awesome it is and how all the cool kids are already using it. I decided to test drive Docker out by using it in my development environment. In this post I will describe how to set up Postgres, Elasticsearch, and Redis as Docker containers with Vagrant on Mac OS X.

What is Docker?

Docker uses lightweight containers to separate an application from the operating system it is running in. It puts the application in an isolated box that only exposes selected folders or ports required for that application to be used.

This makes each container is a reusable, shareable, knowledge base of how to setup and use an application. There already exists over 15,000 containers ready to be used at the Docker Hub. Docker is like a shopping cart, where you go and pick out the services you need to build the application you want, then just download and turn them on.

Setting Docker Up in OS X

Docker does not run in OS X natively, it requires a Linux kernel with LXC (LinuX Containers). So if you are on OS X like me, you will require some virtualisation.

Don't use boot2docker

While trying to get docker working I found the "easy" install described here. This uses a tool called boot2docker which is a thin wrapper on a virtual machine like VirtualBox.

I soon discovered this tool has some significant problems, like this, which halted any progress towards getting Docker in a stable state. I did not feel like hitting my head against a virtual wall any longer, so I continued looking for an alternate solution.

Use Vagrant

I eventually found out that since Vagrant version 1.6, it has built-in support for Docker. Vagrant is a thin wrapper around virtualisation software like VirtualBox, and it uses a declarative Ruby DSL to describe the environment you want to have.

I like this way of defining the virtual environment, because if something fails you can burn it down and start again without a lot of left over mess, like environment variables, littering your machine.

Installing stuff

First, lets quickly go over all the things you need to have installed.

Homebrew, installed with:

ruby -e "$(curl -fsSL https://raw.github.com/mxcl/homebrew/go)"  

Cask, installed with:

brew tap caskroom/homebrew-cask  
brew install brew-cask  

Vagrant and VirtualBox, installed with:

brew cask install virtualbox  
brew cask install vagrant  

The Vagrant Files

A Vagrant file describes the required virtual machine environment using a Ruby DSL. When describing Docker containers, Vagrant makes each container look like it is its own virtual machine. But this is a lie, as each Docker container is actually running in a "proxy" virtual machine.

Therefore, two Vagrant files are required, one to define the proxy virtual machine (the provisioner) and one to define the the Docker containers (the providers).

The Proxy VM Vagrant file

The proxy Vagrant file is called Vagrantfile.proxy:

VAGRANTFILE_API_VERSION = "2"

Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|  
  config.vm.box = "hashicorp/precise64"
  config.vm.provision "docker"
  config.vm.provision "shell", inline:
    "ps aux | grep 'sshd:' | awk '{print $2}' | xargs kill"

  config.vm.network :forwarded_port, guest: 6379, host: 6379
  config.vm.network :forwarded_port, guest: 5432, host: 5432
  config.vm.network :forwarded_port, guest: 9200, host: 9200
end

This uses the hashicorp/precise64 Ubuntu 12.04 64 bit image for the proxy VM. It also provisions docker and does some magic with the shell to make docker to work (explained here).

The last thing is to set up the port forwarding. This uses config.vm.network to map the ports for Redis, Elasticsearch and Postgres from the proxy VM to OS X.

The Docker Containers Vagrant File

This is the main Vagrantfile:

VAGRANTFILE_API_VERSION = "2"  
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|

  config.vm.define "redis" do |v|
    v.vm.provider "docker" do |d|
      d.image = "dockerfile/redis"
      d.volumes = ["/var/docker/redis:/data"]
      d.ports = ["6379:6379"]
      d.vagrant_vagrantfile = "./Vagrantfile.proxy"
    end
  end

  config.vm.define "elasticsearch" do |v|
    v.vm.provider "docker" do |d|
      d.image = "dockerfile/elasticsearch"
      d.ports = ["9200:9200"]
      d.vagrant_vagrantfile = "./Vagrantfile.proxy"
    end
  end

  config.vm.define "postgres" do |v|
    v.vm.provider "docker" do |d|
      d.image = "paintedfox/postgresql"
      d.volumes = ["/var/docker/postgresql:/data"]
      d.ports = ["5432:5432"]
      d.env = {
        USER: "root",
        PASS: "abcdEF123456",
        DB: "root"
      }
      d.vagrant_vagrantfile = "./Vagrantfile.proxy"
    end
  end

end  

This file defines the three containers for Redis , Elasticsearch, and Postgres with the images dockerfile/redis, dockerfile/elasticsearch and paintedfox/postgresql.

Each file defines the vagrant_vagrantfile as the proxy VM file, this makes them all run in the same proxy virtual machine.

The volumes for Redis and Postgres are defined so that their information is stored in the proxy VM, and not in the container. This is so the container could be deleted or upgraded and the data will not be lost. The next step would be to map those folders from the proxy VM to OS X, but this is not necessary to get things working.

The ports on each container defines which ports to forward to the proxy VM. These need to match up with the ports that the proxy VM forwards to OS X.

The Postgres container also defines the environment variables needed to set up its server. These can be used to set up the default Postgres server in OS X by setting the environment variables PGHOST=localhost PGUSER=root PGPASSWORD=abcdEF123456.

Working with Vagrant

In the same directory as your Vagrant file, you can now run:

vagrant up --provider=docker  

The first time you run this, Vagrant will download then start the proxy VM, then it will download and start the Docker containers. Each time Vagrant is run after these initial downloads it will reuse the existing images.

The status of the Docker containers can be seen with:

vagrant status  

This should output something like:

Current machine states:

redis                     running (docker)  
elasticsearch             running (docker)  
db                        running (docker)  

To test the Docker containers are working correctly, the Redis and Postgres clients, and curl for Elasticsearch can be used. Just check that redis-cli and psql connect to their servers, and curl http://localhost:9200 responds.

If you need to connect to the proxy VM, which can be very useful for debugging, run vagrant global-status which will list all VMs including the proxy. Then call vagrant ssh <ID> with the ID of the proxy. I would recommend not changing this proxy VM manually, instead use a Chef (or similar) script so that the changes can be more easily tested and distributed.

Performance

When using virtualisation, the first question always asked is "How much of a performance hit is there?". To find out how bad this performance hit is, my colleague and I both ran a Postgres, Elasticsearch and Redis intensive test suite on identical hardware. The only difference was one test suite had natively installed software and the other had Docker-ized containers. The native suite ran in 2 minutes and the containers ran in 3 minutes.

This performance hit is not as small as I would like, but it could be worse. Even with this, I will continue using Docker for development, but not recommend it to everybody as a panacea for all development environment problems.

Note: Some other limitations of using Vagrant and Docker are listed here.

Conclusion

I cannot yet see where this "Vagrant with Docker" path is going. However, after seeing what is possible I cannot help but think about how else it can be used. Plus, it is the most fun I have ever had with virtualisation, and fun is what programming is all about.

Further Reading

The Docker Book: Containerization is the new virtualization

Vagrant: Up and Running

comments powered by Disqus