Deploy tools

I've been looking at the world of deployment tools lately. Outside of Puppet and Chef (and ignoring the old beards Bcfg2 & Cfengine), what other things are there?

Fabric lets you write Python functions to describe "tasks" to be run. The Python functions are run on a client machine -- for example, the sysadmin's laptop -- and each task can be directed to operate on hosts or roles (groups of hosts), over SSH. The functions can run remote commands with run("echo hello, world") and sudo("chmod u=rw,go=r /etc/passwd"). Fabric is a very useful piece of the puzzle, but doing more complex operations one shell command at a time gets frustrating. I keep wanting something that can run whole chunks of Python easily, on the target machine. Fabric also does nothing to solve the problems of e.g. multiple admins running a deploy command at the same time.

Kokki is closer to Chef in it's style (and literally, "Chef" in Finnish). It's a framework for writing cookbooks, with actions like File("/etc/greeting", content="hello, world") in them. Then a configuration for a machine can invoke certain recipes. Kokki seems to be aimed fully at running things locally; that is, if you're deploying things, you'd run Kokki on the target machine. Kokki is still in fairly early development, it's website and source code don't match each other at all, and many of the cookbooks no longer work with the current version. It also still inherits a bit too many non-Pythonic elements from Chef, for my tastes. Still, to this Pythonista, it looks very promising, and I will be exploring it further.

Poni ("Pony" in Finnish -- I want one too!) is another Python project that takes a very different tack. It is built on a command-line tool that lets you define your infrastructure through hierarchical collections of key-value settings; that is, you describe the whole multi-machine service with database servers, load balancers, app servers and all. You can use inheritance much like other deploy tools use cookbooks. The command-line tool seems to be meant to be used for everything; the stored data is not really meant to be edited directly. While I appreciate the polish of the command-line tool, the editor-hostility comes off a bit odd. Especially so when the getting started guide has me "uploading" template files and Python source code to Poni's internal configuration storage. Am I really supposed to have two copies of these files?

Once you have your infrastructure defined, Poni provides you two main methods to actually make changes: you can create files based on templates (that have a strong mechanism for referring to any values from the configuration, including things like sharing the database connection information between the DB server and the client config), or you can run custom functions ("control commands"). The Python functions run locally, but Poni provides a remote execution framework very similar to the one in Fabric, though at least for now it is significantly more verbose. And, to my disappointment, doesn't really allow running full Python functions remotely either.

Somewhat confusing is the difference between the "create a file" and the "run control command" functionality. It is not quite clear how the whole is intended to orchestrate the full deployment, and the examples are both lacking and misleading. For example, right now the Puppet deployment example requires you to run a command to create some files, watch it fail, run another command to install software, then run the first command again to create the rest of the files. (Kind of weird to deploy Puppet with Poni in the first place..)

There is one thing about Poni that I am already starting to dislike. Currently, every identifier you need to refer to on the command line is given as a regexp, and commands act on all matches. This leads to high risk for operator errors: for example, the documentation itself uses $find("webshop/frontend") as an example; yet that would also match webshop/frontendforsomethingelse. I do hope the author changes his mind about regexps everywhere.

Much like Kokki, Poni is very early on in its development; it's command-line tools and things like variable referencing are top notch, but the picture is very much not complete yet. But this one is definitely a project to watch.

For posterity: I filed a bunch of issues about the things I bumped into:

  • deploy
  • software
  • fabric
  • kokki
  • poni