Apr 6th 2009

Simple backups can be simple!

vitaly astrails, it, astrails-safe, opensource

Everyone needs a backup, right? Unfortunately almost no one does though. Why?!

We needed something for ourselves and our customers. Something simple, free, configure-and-forget. Most of the time there is no need for something fancy, a simple tar + mysqldump can do the job for many small/medium sites.

We did some reasearch but every solution we found had one of the following problems:

too complicated to use/configure
not open source
does only filesystem or only mysql backup but not both
no Amazon S3 support
no backup rotation support

So we wrote our own. Basically we cleaned up and refactored some custom backup scripts that we had for ages on our own servers.

We had the following requirements in mind when we wrote it:

opensource :)
simple to install and configure
support for simple ‘tar’ backups of directories (with includes/excludes)
support for simple mysqldump of mysql databases
support for symmetric or public key encryption (see README for instructions)
support for local filesystem and Amazon S3 for storage
support for backup rotation. we don’t want backups filling all the diskspace or cost a fortune on S3

So lets dive right in:

# gem install astrails-safe --source http://gems.github.com/
Successfully installed astrails-safe-0.1.4
1 gem installed
Installing ri documentation for astrails-safe-0.1.4...
Installing RDoc documentation for astrails-safe-0.1.4...

# astrails-safe my-backup.conf
ERROR: Created default /root/my-backup.conf. Please edit and run again.

For configuration file format we use Ruby DSL (yeah, we probably got too excited writing it and went a little overboard with the implementation, a simple hash probably would suffice here :), but we actually like the result.

Check it out (you can see more configuration options in the generated template config file):

safe do
  local :path => "/backup/:kind/:id"

  s3 do
    key "...................."
    secret "........................................"
    bucket "backup.astrails.com"
    path "servers/alpha/:kind/:id"
  end

  gpg do
    # symmetric encryption key
    # password "qwe"

    # public GPG key (must be known to GPG, i.e. be on the keyring)
    key "[email protected]"
  end

  keep do
    local 2
    s3 30
  end

  mysqldump do
    options "-ceKq --single-transaction --create-options"

    user "root"
    password "............"
    socket "/var/run/mysqld/mysqld.sock"

    database :blog
    database :servershape
    database :astrails_com
    database :secret_project_com
  end

  tar do
    archive "git-repositories", :files => "/home/git/repositories"
    archive "dot-configs",      :files => "/home/*/.[^.]*"
    archive "etc",              :files => "/etc", :exclude => "/etc/puppet/other"

    archive "blog-astrails-com" do
      files "/var/www/blog.astrails.com/"
      exclude ["/var/www/blog.astrails.com/log", "/var/www/blog.astrails.com/tmp"]
    end

    archive "astrails-com" do
      files "/var/www/astrails.com/"
      exclude ["/var/www/astrails.com/log", "/var/www/astrails.com/tmp"]
    end
  end
end