Daemonic Dispatches

Introducing configinit

I have been working on bringing FreeBSD to the Amazon EC2 platform since 2006, and for the past three years I've been blogging about my progress: First FreeBSD on t1.micro instances, then cluster compute instances, then "m1" and "m2" family large and xlarge instances, and finally in early 2012, FreeBSD could finally run on all EC2 instance types. Once I had a hacked-up version of FreeBSD which ran smoothly, I turned my attention towards polishing it: First moving my EC2 scripts into the ports tree, then using binaries from the release ISOs for the FreeBSD world, and finally in early October (with FreeBSD 10.0-ALPHA4) all the necessary bits had been merged to make it possible for me to build EC2 images completely (including the kernel) with "straight off the ISO" binaries. Next on my agenda was taking my images from "pure FreeBSD" to "FreeBSD set up to be used in the cloud", and for that I'm happy to now announce that starting from 10.0-RC1, my FreeBSD AMIs have a new feature: configinit.

Anyone who has been around the world of cloud computing for long is likely to have heard of CloudInit. It is a system originally written for Ubuntu which performs configuration of a system at boot-time based on "user-data" provided via EC2 or from a similar environment (e.g., OpenStack). CloudInit works well for its original purpose, but is less than ideal for FreeBSD systems, for two reasons: First, it relies on Python, which is not part of the FreeBSD base system; and second, it is designed around a concept of configuring a system by running commands rather than editing configuration files.

Now, there are merits to both approaches — most notably, configuring a system by running commands is easier to script, while configuring a system by editing text files has the advantage that given a working configuration there's no doubt about how to reproduce it — but the fact that BSD systems are far more edit-configuration-files oriented (to the point that /etc/rc.conf might be the only configuration file which needs to be edited on some systems), and thus CloudInit is less than optimal for configuring FreeBSD systems.

Enter configinit. Rather than providing instructions such as "tell apt to use this mirror" or "run this python code", configinit handles four types of input:

1. If the configuration data starts ">/path/to/file" then the data, minus the first line, will be written to the specified location.
2. If the configuration data starts ">>/path/to/file". then the data, minus the first line, will be appended to the specified location.
3. If the configuration data starts "#!", it will be executed (in most cases this would be a shell script).
4. For any other inputs, configinit attempts to extract the file as an archive, and (if extraction was successful) runs on each part in turn. The extraction is performed using bsdtar, so archives in tar, pax, cpio, zip, jar, ar, xar, rpm and ISO 9660 formats, optionally compressed using gzip, bzip2, xz, or compress can all be used.

This is much simpler than CloudInit, but — in combination with other tools which are already available on FreeBSD, such as my firstboot-pkgs port, it provides very powerful yet easy-to-use functionality. For example, launching a FreeBSD 10.0-RC1 EC2 instance with the following user-data:

>>/etc/rc.conf
firstboot_pkgs_list="apache22"
apache22_enable="YES"

will provide a system with Apache 2.2 installed and running (in my test, within 150 seconds of when I clicked "launch" in the EC2 Management Console) — in addition to performing the other default system initialization behaviours of my EC2 images: checking for updates to the FreeBSD base system, downloading an SSH public key to allow SSH logins as ec2-user, logging SSH host keys to the EC2 console, and autoconfiguring swap space using EC2 ephemeral disks.

I know for my purposes this will be very useful — for example, while I have the process of configuring a new Portsnap mirror mostly scripted now, using configinit I could have it entirely scripted and avoid the need to ever SSH into the mirrors — and from what other FreeBSD users have told me, I don't think I will be alone. Is there anything else I could do to make FreeBSD even more usable in EC2? Quite likely — but I don't know what. If I'm missing something important, please let me know!

blog comments powered by Disqus