This blog-post is mostly targeted at non-Rails developers. Rails devs should
know all this by heart :) Many times we need to explain to our customers what
is ‘proper deployment’ and why their current one sucks :) Now we’ll be able to
just point them to this post…
Proper deployment is almost not found anywhere. At least anywhere we looked.
Very few places really ‘get’ it and assign enough importance to it. Some
project owners just don’t know how it is supposed to be, so they accept their
developer’s practice of ftp sync to production server :)
If you are a developer, make sure you implement it all.
If you have a project developed for you, then this is a checklist that you can
bring to your developers/consultants and require ‘yes’ to every single
one.
If your deployment procedure misses any of the qualities listed below, you are
asking for trouble.
Production deployment must be:
- Automated
- Atomic
- Clean
- Reversible
- Have a release log
- Manage DB migrations
Automated
This is I think the most important requirement. Deployments shouldn’t be done
by hand. There are still a lot of companies where a deployment process looks
something like this:
- connect to the remote windows server with “remote desktop”
- upload a zip file via ftp
- unzip it on the desktop on the remote pc
- manually drag it to the production deployment, effectively creating a mix of new and old files
- manually connect to the sql server and fiddle with schema until it looks similar to the local one.
- restart something.
This is soooo wrong. The deployment must be done by a single execution of a
deployment script, that should do all the work. The maximum developers
intervention allowed is entering passwords.
Atomic
This one is not always possible (for example when doing database migrations),
but the idea is that you switch old version with the new one instantly, as
opposed to syncing a whole tree of files over ftp. With later you have a
noticeable amount of time when some files are already from the new version and
some others are still from the old one.
Clean
By clean deployment I mean that no files from the old version should be
accessible after deploying the new version. Too many times we’ve seen
deployments by simply dumping new files into production directory. So files
that were removed in the new version remain in place.
Reversable
It should be fairly easy to revert to the previous version in a case when a
major bug found in the new one. (and in case of no db-migrations it should be
trivial)
Have a release log.
It should be possible to answer questions like:
- what version were we running at 17th september last year? (I have a db record
time-stamped with this date, and I want to see the code that created it)
- did we have any code changes 17 days ago at 13:40? (I’m looking at the
performance graphs for the application and notice that CPU usage rised by 20%
at this time.)
Manage DB migrations
Database schema must be managed by scripts that run during deployment.
You should NEVER manually modify production database schema in a course of
a regular deployment. (you might still need to do it to ‘repair’ an automated
migration that failed midway, but this should be an exception, not the rule).
Reversibility requirement also means that your migration scripts should be able
to both upgrade and downgrade the schema.
Credit and more war stories
To give credit where it is due Rails +
Capistrano /
Vlad actually suppost all of
the above almost ‘out of the box’.
But even with Rails we’ve seen projects that do deployments by running ‘svn up’
in the production directory, or even with just ftp-ing new .css file directly
to production w/o even checking it into the version control.
The HOWTO
Below is one of the possible ways to do it that passes all of the above (this
is how its usually done with capistrano or vlad):
- The production directory is just a symbolic link to deployment directory
which is different for every deployment.
- The files that should be shared between deployments (database configuration
file, user uploaded images etc) are stored in a separate ‘shared’ directory
that can be accessed directly or can be linked into each release directory
during deployment (we do the later)
- exact source code version and exact deployment time-stamp are logged into a
simple text file
- database has a version attached to it, which is stored in a separate database
table (with just numeric version field). There is a script for each database
version that knows how to bring the database up from the previous, or return
back to the previous version (i.e. both ‘up’ and ‘down’)
- The deployment process is performed as follows
- new code is uploaded/unpacked/checked out directly from source control
into a NEW directory with a unique name (just use time-stamp)
- directory name, code version and current timestamp are logged in the
releases.log file
- shared directory linked into the release directory
- deployment-specific configuration files are copied from the shared
directory to the release directory
- database migrations are run if needed.
- symlink is replaced pointing to the new directory
- web server is restarted
The resulting directory structure might look like this:
deployment-root/
|-- current -> releases/20081210-1703
|-- releases
| |-- 20081201-1415
| | |-- config
| | | |-- databse.yml <== this is copied from the shared/config/database.yml
| | | `-- shared -> ../../../shared
| | |-- log -> ../../shared/log
| | `-- public
| | `-- images
| | `-- avatars -> ../../config/shared/avatars
| `-- 20081210-1703
| |-- config
| | |-- databse.yml
| | `-- shared -> ../../../shared
| |-- log -> ../../shared/log
| `-- public
| `-- images
| `-- avatars -> ../../config/shared/avatars
|-- revisions.log
`-- shared
|-- avatars
|-- config
| `-- databse.yml
`-- log
`-- production.log
