Using Packer for faster provisioning

Speeding up the provisioning process by installing things ahead of time

terraform packer cloud
2018-04-08
Thomas Kooi

As a frequent user of Terraform, I have a variety of modules tailored for different purposes, many of which rely on CentOS with a Docker install script. However, while demoing a setup to a co-worker, we encountered a significant delay — about seven minutes — for a Terraform apply to complete with an install script. Determined to improve this, I dedicated my Sunday to exploring Packer.

Given my extensive use of Terraform, delving into Packer seemed like a natural next step. Despite finding the initial documentation somewhat confusing, expecting it to be similar to Terraform, I quickly discovered that Packer is both straightforward and powerful.

Use Case

My primary use case involves installing specific versions of Docker CE and Kubeadm, as most of my Terraform projects require a Docker setup. Running the full installation script usually takes around five minutes, not counting additional configuration steps or host-specific installs. While I could use the pre-configured Docker image on DigitalOcean, I prefer the flexibility and control of installing specific Docker CE versions.

Docker Swarm

Typically, my process involves provisioning a set of master machines (such as Docker Swarm mode masters, etcd, or Kubernetes API servers), followed by a series of worker nodes. This entire setup often results in a waiting period of around ten minutes, in addition to the time required for other installations and configurations. By integrating Packer into my workflow, I aim to significantly reduce these delays, enhancing efficiency and productivity.

I figured I’d start out simple and take the DigitalOcean builder example from Packer.

{
  "type": "digitalocean",
  "api_token": "YOUR API KEY",
  "image": "ubuntu-14-04-x64",
  "region": "nyc3",
  "size": "512mb",
  "ssh_username": "root"
}

And turn it into a packer config file:

{
  "builders": [
    {
      "type": "digitalocean",
      "api_token": "MY_TOKEN",
      "image": "centos-7-x64",
      "region": "ams3",
      "size": "s-1vcpu-1gb",
      "ssh_username": "root"
    }
  ],

  "provisioners": [
    {
      "type": "shell",
      "script": "digitalocean/scripts/install-kubeadm.sh"
    }
  ]
}

And the contents of digitalocean/scripts/install-kubeadm.sh:

#!/bin/bash

# install docker
yum update -y

yum install -y docker
systemctl enable docker && systemctl start docker

# get kubernetes repo
cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
EOF
setenforce 0

# install kubernetes
yum install -y kubelet kubeadm kubectl
systemctl enable kubelet && systemctl start kubelet

cat <<EOF >  /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
sysctl --system

Building the image turns out to be as simple as:

$ packer build docker-kubeadm.json
digitalocean output will be in this color.

==> digitalocean: Creating temporary ssh key for droplet...
==> digitalocean: Creating droplet...
==> digitalocean: Waiting for droplet to become active...
==> digitalocean: Waiting for SSH to become available...
==> digitalocean: Connected to SSH!
==> digitalocean: Provisioning with shell script: digitalocean/scripts/install-kubeadm.sh
    digitalocean: Loaded plugins: fastestmirror
    digitalocean: Determining fastest mirrors
    digitalocean:  * base: mirror.denit.net
    digitalocean:  * extras: mirror.nforce.com
    digitalocean:  * updates: centos.mirror.triple-it.nl
...
==> digitalocean: Gracefully shutting down droplet...
==> digitalocean: Creating snapshot: packer-1523190279
==> digitalocean: Waiting for snapshot to complete...
==> digitalocean: Destroying droplet...
==> digitalocean: Deleting temporary ssh key...
Build 'digitalocean' finished.

==> Builds finished. The artifacts of successful builds are:
--> digitalocean: A snapshot was created: 'packer-1523190279' (ID: 33289213) in regions ''

After successfully running the Packer build step, I can now see the available images using doctl:

$ doctl compute image ls
ID          Name                 Type        Distribution    Slug    Public    Min Disk
33289213    packer-1523190279    snapshot    CentOS                  false     25

Creating some flavours

I typically use a couple of different configurations: one with plain Docker CE, using one of the latest stable builds, and another with Docker alongside kubeadm.

By making a small modification to the standard Docker CE installation script, I created a shared script that efficiently installs Docker CE for both configurations:

#!/bin/bash
# Installing Docker CE On CentOS distributions
# https://docs.docker.com/install/linux/docker-ce/centos/#install-docker-ce

export VERSION=${VERSION:-18.03.0.ce-1.el7.centos}

yum update -y
yum install -y yum-utils device-mapper-persistent-data lvm2

yum-config-manager \
  --add-repo \
  https://download.docker.com/linux/centos/docker-ce.repo

yum install -y docker-ce-$VERSION
systemctl start docker

Now I just needed to add some configuration for creating an image for multiple docker versions. I started with Docker v17.12.0 and Docker v18.03.0.

{
  "builders": [
    {
      "type": "digitalocean",
      "api_token": "MY_TOKEN",
      "image": "centos-7-x64",
      "region": "ams3",
      "size": "s-1vcpu-1gb",
      "ssh_username": "root",
      "snapshot_name": "docker-v17.12.0-ce",
      "snapshot_regions": ["ams3"]
    }
  ],
  "provisioners": [
    {
      "type": "shell",
      "environment_vars": ["VERSION=17.12.0.ce-1.el7.centos"],
      "script": "digitalocean/scripts/install-docker-ce.sh"
    }
  ]
}

When building with Packer for several different Docker versions, I ended up with the following images:

$ doctl compute image ls
ID          Name                 Type        Distribution    Slug    Public    Min Disk
33289213    packer-1523190279    snapshot    CentOS                  false     25
33289232    docker-v17.12.0-ce   snapshot    CentOS                  false     25
33289248    docker-v18.03.0-ce   snapshot    CentOS                  false     25

Reusing in Terraform

In Provisioning a Swarm mode cluster I talked about the Terraform module I use for creating a quick Docker swarm mode lab. I always either used the Docker image provided by DigitalOcean or ran an install script using user_data.

Instead I am now able to point towards the image I just created using Packer:

data "digitalocean_image" "docker_image" {
  name = "docker-v17.12.0-ce"
}

module "swarm-cluster" {
    source            = "thojkooi/docker-swarm-mode/digitalocean"
    version           = "0.1.0"
    total_managers    = 1
    total_workers     = 1
    domain            = "do.example.com"
    do_token          = "${var.do_token}"
    manager_ssh_keys  = "${var.ssh_keys}"
    worker_ssh_keys   = "${var.ssh_keys}"
    manager_os        = "${data.digitalocean_image.docker_image.image}"
    worker_os         = "${data.digitalocean_image.docker_image.image}"
    provision_user    = "root"
    manager_tags      = ["${digitalocean_tag.cluster.id}", "${digitalocean_tag.manager.id}"]
    worker_tags       = ["${digitalocean_tag.cluster.id}", "${digitalocean_tag.worker.id}"]
}

Conclusion

Packer has proven to be a straightforward and effective tool that I will definitely be incorporating more into my workflow. The cost efficiency is notable as well; storing a snapshot on DigitalOcean costs only around $0.05 at the time of writing.

Considering the size of the images I’ve created, this amounts to approximately $0.30 a month. This small investment translates to a significant time savings of about five minutes every time I provision a new droplet. By using Packer for creating Kubernetes base images, I’ve streamlined my setup process, making it more efficient and cost-effective.

$ doctl compute snapshot ls
ID          Name                  Created at              Regions    Resource ID    Resource Type    Min Disk Size    Size
33289213    packer-1523190279     2018-04-08T12:29:23Z    [ams3]     88805641       droplet          25               1.57 GiB
33289232    docker-v17.12.0-ce    2018-04-08T12:36:51Z    [ams3]     88806114       droplet          25               1.34 GiB
33289248    docker-v18.03.0-ce    2018-04-08T12:45:46Z    [ams3]     88806835       droplet          25               1.37 GiB

5 min read
Share this post:

Related posts