Using Packer to create AMI (Amazon Machine Images)

, ,

Terraform is an awesome tool - It writes, plans and creates infrastructure as code.

An Amazon Machine Image (AMI) provides the information required to launch an instance. You must specify an AMI when you launch an instance.

With Terraform AWS provider you can create multiple EC2 instances from a specific AMI.

Packer is an open source tool for creating identical machine images for multiple platforms from a single source configuration. In fact, you can create multiples machine images contains different software.

In this post, I will share the way I use Packer to create 4 kinds of machine images:

  • Ubuntu server
  • Ubuntu server + Docker
  • Ubuntu server + Nginx
  • Ubuntu server + Ruby on Rails framework + Elixir-Phoenix framework

Getting started

Install Packer and json CLI tool. I’m using MacOS btw

1
brew install packer
1
npm install -g json

The configuration file used to define what image we want to be built and how is called a template in Packer terminology. The format of a template is simple JSON.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
{
  "variables": {
    "aws_access_key": "",
    "aws_secret_key": ""
  },
  "builders": [{
    "type": "amazon-ebs",
    "access_key": "{{user `aws_access_key`}}",
    "secret_key": "{{user `aws_secret_key`}}",
    "region": "us-east-1",
    "source_ami_filter": {
      "filters": {
        "virtualization-type": "hvm",
        "name": "ubuntu/images/*ubuntu-xenial-16.04-amd64-server-*",
        "root-device-type": "ebs"
      },
      "owners": ["099720109477"],
      "most_recent": true
    },
    "instance_type": "t2.micro",
    "ssh_username": "ubuntu",
    "ami_name": "packer-example {{timestamp}}"
  }]
}

From this template, we can easily create the first machine image: latest Ubuntu server.

The rest machine images are latest Ubuntu server + dependencies, we can setup by using Provisioner. Provisioners are configured as part of the Packer JSON template.

Regarding the AWS credential, we can save it as a simple json file and load it by -var-file option.

1
2
3
4
{
  "aws_access_key": "xxx",
  "aws_secret_key": "xxx"
}

Finally, make a simple CLI application to ask AWS credential variables, image information before using json CLI to render these variables to the Packer JSON template.

Demo example

  • Clone the source code, run make to see all available commands
1
2
3
4
5
╰─$ make
build                          build the AMI from json
generate                       generate json-credential file
init                           init base
validate                       validate AMI json
  • Run the make init and choose one of kind of AMI you want to create including the image information such as instance type, image name, description and region. In this example, I choose base
1
2
3
4
5
6
7
8
9
10
11
12
╰─$ make init
./init.sh
Please enter machine image type (e.g: base, base-docker, base-nginx, base-full-stack):
> base
Enter the instance type: (https://aws.amazon.com/ec2/instance-types/) (e.g: t2.micro):
> t2.micro
Enter the image name:
> Base Image with latest Ubuntu
Enter the image description:
> Ubuntu Server 18.04
Enter the ami region: (https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Concepts.RegionsAndAvailabilityZones.html) (e.g: ap-southeast-1):
> ap-southeast-1
  • Now, run make generate to generate the AWS credential file
1
2
3
4
5
6
7
╰─$ make generate
./generate.sh
Generate AWS Credential configuration
Enter AWS Access Key:
> AKIAJABCDEF29340929320
Enter AWS Secret Key:
> pXULnB209Jkdlwkkew/V20930dk+8wdalEKelE
  • Run make validate to check whether the JSON configuration is valid or new_post_name
1
2
3
4
5
╰─$ make validate
./validate.sh
Please enter machine image type (e.g: base, base-docker, base-nginx, base-full-stack):
> base
Template validated successfully.
  • Finally, run the make build, and waiting for the machine image creating process finished then you will have a new AMI.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
./build.sh
Please enter machine image type (e.g: base, base-docker, base-nginx, base-full-stack):
> base
amazon-ebs output will be in this color.

==> amazon-ebs: Prevalidating AMI Name: Base Image with latest Ubuntu-1562084759
    amazon-ebs: Found Image ID: ami-026c8acd92718196b
==> amazon-ebs: Creating temporary keypair: packer_5d1b8597-427d-7d4a-148d-5cf043c14b95
==> amazon-ebs: Creating temporary security group for this instance: packer_5d1b859c-f16d-81cc-c437-4a431ed0dd1e
==> amazon-ebs: Authorizing access to port 22 from 0.0.0.0/0 in the temporary security group...
==> amazon-ebs: Launching a source AWS instance...
==> amazon-ebs: Adding tags to source instance
    amazon-ebs: Adding tag: "Name": "Packer Builder"
    amazon-ebs: Instance ID: i-0a903cb50cee6bbe4
==> amazon-ebs: Waiting for instance (i-0a903cb50cee6bbe4) to become ready...
==> amazon-ebs: Using ssh communicator to connect: 3.81.131.159
==> amazon-ebs: Waiting for SSH to become available...
==> amazon-ebs: Connected to SSH!
==> amazon-ebs: Stopping the source instance...
    amazon-ebs: Stopping instance, attempt 1
==> amazon-ebs: Waiting for the instance to stop...
==> amazon-ebs: Creating unencrypted AMI Base Image with latest Ubuntu-1562084759 from instance i-0a903cb50cee6bbe4
    amazon-ebs: AMI: ami-0a72e23df7c8078cc
==> amazon-ebs: Waiting for AMI to become ready...
==> amazon-ebs: Copying AMI (ami-0a72e23df7c8078cc) to other regions...
    amazon-ebs: Copying to: ap-southeast-1
    amazon-ebs: Waiting for all copies to complete...
==> amazon-ebs: Adding tags to AMI (ami-0a72e23df7c8078cc)...
==> amazon-ebs: Tagging snapshot: snap-0e49336b3550fb9be
==> amazon-ebs: Creating AMI tags
    amazon-ebs: Adding tag: "Name": "Base Image with latest Ubuntu"
    amazon-ebs: Adding tag: "Description": "Ubuntu Server 18.04"
==> amazon-ebs: Creating snapshot tags
==> amazon-ebs: Terminating the source AWS instance...
==> amazon-ebs: Cleaning up any extra volumes...
==> amazon-ebs: No volumes to clean up, skipping
==> amazon-ebs: Deleting temporary security group...
==> amazon-ebs: Deleting temporary keypair...
Build 'amazon-ebs' finished.

==> Builds finished. The artifacts of successful builds are:
--> amazon-ebs: AMIs were created:
ap-southeast-1: ami-09bf72a56cbc77bd2
us-east-1: ami-0a72e23df7c8078cc

NOTE:

Packer wasn’t able to assign an instance profile to an EC2 machine that it provisions. This is because the instance profile itself could have permissions different from Packer.
AWS intends it to be this way because this could easily become an attack vector — Packer can be used to create machines that are assigned sensitive permissions and hence, to escalate privileges indirectly.
To allow Packer to be able to assign the profile to the instance, you must give it 3 additional permissions:

1
2
3
iam:PassRole
ec2:AssociateIamInstanceProfile
ec2:ReplaceIamInstanceProfileAssociation

Thanks for @theMadKing for permissions problem.