Lately, my prefered Postgres distribution of choice for OSX is Postgres.app.

Its very easy to download and install (just drag-n-drrop, like most other OSX Apps), no need to wait for it to compile, and it comes with a nice menubar icon for control.

In its default configuration it only listens on a port, but not on a unix socket. The problem is, Rails recipes with postgres create a config/database.yml file that assumes a socket presence.

Of example:

✗ rails new test1 -d postgresql
      create
      create  README.rdoc
      ...
Your bundle is complete!
Use `bundle show [gemname]` to see where a bundled gem is installed.
✗ cd test1
✗ rake db:create
could not connect to server: No such file or directory
        Is the server running locally and accepting
        connections on Unix domain socket "/var/pgsql_socket/.s.PGSQL.5432"?

This problem was bothering me for a while as I tried to use rails_apps_composer to bootstrap new applications and it was failing on those database errors.

I didn’t want to mess with Postgres.app configs, as I suspected they’d be overwriten on each version upgrade, so at first I tried to somehow trick it to stop and let me replace the config/database.yml file in time. The solution turned out to be much simpler though.

The Rails Postgres driver recognizes standard Postgres environment variables, one of which is PGHOST. When set to localhost Rails will go for the port instead of the Unix socket, even if there is no host: localhost setting in the YAML file.

✗ PGHOST=localhost rake db:create
✗ _

So just add export PGHOST=localhost to your shell start file, e.g. ~/.zshrc and you are set.

Im actually went further with it, and now I have shell aliases to reset postgres env config, configure it for localhost, or read current application Heroku configs and pre-set Postgres environment for a direct access to Heroku db, but that is a topic for another blog post.

I just recently reinstalled my MacBook Pro, this time with Snow Leopard.

So I’m again going through various installation problems I already forgot about from few years back when I installed Leopard.

Anyway, just had to hunt down a problem with mysql gem installation on Snow Leopard.

Apparently if you just do ‘gem install mysql’ it seems to install fine, but doesn’t work.

You get “uninitialized constant MysqlCompat::MysqlRes” error when trying to run rails.

The solution is to pass it some arguments for installation:

ARCHFLAGS="-arch x86_64" gem install mysql -- --with-mysql-config=/usr/local/mysql/bin/mysql_config

Oh, and of course install 64bit mysql version before that. I usually just download the binaries from mysql.com.

We just started a project for a client that involves Cassandra.

If you’ve been living under a rock and don’t know what Cassandra is let me tell you :)

Cassandra is a “second-generation distributed database” that was built for web scale.

Its is one of the many distributed nosql databases that appear everywhere lately like mushrooms after a heavy rain :).

What sets Cassandra apart is that it comes from a recognizable entity - Facebook.

But I digress.

This is not meant to be a Cassandra introduction, there are enough of those on the net. I Just created a new nosql section on this blog where I’m going to post various tidbits of information about cassandra (and probably others) as I learn them while working on this new project.

Here is the first one: Cassandra gem is just an installer

If you are on Mac OSX and interested in Cassandra you probably know that its just a gem installation away (almost):

gem install cassandra

First thing to note though is that this will not install Cassandra. It will install cassandra installer!

I got bitten by this when I took my laptop with me to my doughter’s dancing class. You see, parents are not allowed “in the room” to not interfere with the process :), so I have 45 minutes to find myself something to do each time. I installed cassandra gem at home and intended to play with Cassandra while there.

Not so fast.

When I tried to run cassandra_helper cassandra which is supposed to start a test cassandra instance it went to connect to a github repository to download and install the actual database.

Duh!

and the 2nd one: Use java preferences

When I got back and finally built Cassandra I got the following message when starting it for the first time:

~ > cassandra_helper cassandra
Set the CASSANDRA_INCLUDE environment variable to use a non-default cassandra.in.sh and friends.
(in /Library/Ruby/Gems/1.8/gems/cassandra-0.5.6.2)
You need to configure your environment for Java 1.6.
If you're on OS X, just export the following environment variables:
  JAVA_HOME="/System/Library/Frameworks/JavaVM.framework/Versions/1.6/Home"
  PATH="/System/Library/Frameworks/JavaVM.framework/Versions/1.6/Home/bin:$PATH"

First thing to note is that just typing JAVA_HOME="/System/Library/Frameworks/JavaVM.framework/Versions/1.6/Home" in the terminal wont help.

You need to export it:

export JAVA_HOME="/System/Library/Frameworks/JavaVM.framework/Versions/1.6/Home"
PATH="/System/Library/Frameworks/JavaVM.framework/Versions/1.6/Home/bin:$PATH"

No need to export PATH as its already exported.

But there is a better and simpler way!

Start “Java Preferences” (find it in /Applications/Utilities, or just use spotlight):

Then reorder the entries in the bottom “Java Applications” section so that Java 6 will be the 1st:

Now cassandra starts right away w/o any exports:

~ > cassandra_helper cassandra
Set the CASSANDRA_INCLUDE environment variable to use a non-default cassandra.in.sh and friends.
(in /Library/Ruby/Gems/1.8/gems/cassandra-0.5.6.2)
CASSANDRA_HOME: /Users/vitaly/cassandra/server
CASSANDRA_CONF: /Library/Ruby/Gems/1.8/gems/cassandra-0.5.6.2/conf
Listening for transport dt_socket at address: 8888
...

Cool, now go write your killer application!

I just did something pretty stupid. I edited /etc/sudoers file directly from within my non-root user account.

I did

sudo vim /etc/sudoers

and added the following to it:

Cmnd_Alias GEM_INSTALL = /usr/bin/gem install *
Cmnd_Alias GEM_UNINSTALL = /usr/bin/gem uninstall *
vitaly ALL=NOPASSWD GEM_INSTALL
vitaly ALL=NOPASSWD GEM_UNINSTALL

The intention was to grant myself permissions to install gems w/o entering password. I know its insecure, but this is security-vs-convinience kind of thing and I only intended to leave it there for a couple of hours while I do some heavy gem development.

Anyway, experienced unix users might have spotted the syntax error in my sudoers edits. I forgot the : just after the NOPASSWD. But the problem is even more basic then that. I shouldn’t have beed editing the file directly. I should have known better. And now I’m paying the price:

$ sudo
>>> sudoers file: syntax error, line 36 <<<
>>> sudoers file: syntax error, line 37 <<<
sudo: parse error in /private/etc/sudoers near line 36

$ sudo vim /etc/sudoers
>>> sudoers file: syntax error, line 36 <<<
>>> sudoers file: syntax error, line 37 <<<
sudo: parse error in /private/etc/sudoers near line 36

OOPS!

Now the sudoers file is broken and I can’t even fix it since I was using sudo to edit it!

Never do that! :)

Use the visudo command. it will check the file syntax before ‘commiting’ it.

Now what?

I looked at the net and the general consensus is that you need to boot into a single-user mode to fix it. I really really didn’t want to do it. I have 4G of RAM and so I’m usually running dozens of programs and its a pain to close and reopen them all after boot. I’m lazy :)

Then I thought there might be a better way.

First I checked the permissions on the sudoers file:

$ ls -l /etc/sudoers
-r--r-----+ 1 root  wheel  1302 Sep 28 17:20 /etc/sudoers

and only ‘root’ is in the group wheel, so no luck here.

I also couldn’t ‘su root’ since my root user doesn’t have a password. duh!

But then it appeared to me that I might be able to circumvent this protection by leveraging my OS X ‘admin’ status. After all it ought to count for something :).

I opened “/etc” folder in finder (Go -> Go to Folder...), then opened sudoers file properties. Opening the lock there doesn’t require to be a root. Its enough to be an Admin and my Admin user does have a password! So I was easily able to grant myself permission to edit the file:

after that I just edited the file with vim again to comment the edits

vim /etc/sudoers

Then I did what I was supposed to do from the beginning, I used the ‘visudo’ at last:

sudo visudo

Last thing was to restore original permissions on the file in finder.

DONE

Everyone needs a backup, right? Unfortunately almost no one does though. Why?!

We needed something for ourselves and our customers. Something simple, free, configure-and-forget. Most of the time there is no need for something fancy, a simple tar + mysqldump can do the job for many small/medium sites.

We did some reasearch but every solution we found had one of the following problems:

• too complicated to use/configure
• not open source
• does only filesystem or only mysql backup but not both
• no Amazon S3 support
• no backup rotation support

So we wrote our own. Basically we cleaned up and refactored some custom backup scripts that we had for ages on our own servers.

We had the following requirements in mind when we wrote it:

• opensource :)
• simple to install and configure
• support for simple ‘tar’ backups of directories (with includes/excludes)
• support for simple mysqldump of mysql databases
• support for symmetric or public key encryption (see README for instructions)
• support for local filesystem and Amazon S3 for storage
• support for backup rotation. we don’t want backups filling all the diskspace or cost a fortune on S3

So lets dive right in:

# gem install astrails-safe --source http://gems.github.com/
Successfully installed astrails-safe-0.1.4
1 gem installed
Installing ri documentation for astrails-safe-0.1.4...
Installing RDoc documentation for astrails-safe-0.1.4...

# astrails-safe my-backup.conf
ERROR: Created default /root/my-backup.conf. Please edit and run again.

For configuration file format we use Ruby DSL (yeah, we probably got too excited writing it and went a little overboard with the implementation, a simple hash probably would suffice here :), but we actually like the result.

Check it out (you can see more configuration options in the generated template config file):

safe do
  local :path => "/backup/:kind/:id"

  s3 do
    key "...................."
    secret "........................................"
    bucket "backup.astrails.com"
    path "servers/alpha/:kind/:id"
  end

  gpg do
    # symmetric encryption key
    # password "qwe"

    # public GPG key (must be known to GPG, i.e. be on the keyring)
    key "[email protected]"
  end

  keep do
    local 2
    s3 30
  end

  mysqldump do
    options "-ceKq --single-transaction --create-options"

    user "root"
    password "............"
    socket "/var/run/mysqld/mysqld.sock"

    database :blog
    database :servershape
    database :astrails_com
    database :secret_project_com
  end

  tar do
    archive "git-repositories", :files => "/home/git/repositories"
    archive "dot-configs",      :files => "/home/*/.[^.]*"
    archive "etc",              :files => "/etc", :exclude => "/etc/puppet/other"

    archive "blog-astrails-com" do
      files "/var/www/blog.astrails.com/"
      exclude ["/var/www/blog.astrails.com/log", "/var/www/blog.astrails.com/tmp"]
    end

    archive "astrails-com" do
      files "/var/www/astrails.com/"
      exclude ["/var/www/astrails.com/log", "/var/www/astrails.com/tmp"]
    end
  end
end

UPDATE: There is new updated version with PostgreSQL and svn support.

Thanks a lot to Amit Hurvitz for providing a file of Virtual Disk Image (VDI) of VirtualBox, containing an up and running JRuby on Rails on Glassfish with Mysql. Image also contains some examples (actually solutions to the code camp exercises), all running on top of an OpenSolaris guest OS (can be run on many host systems).

Grab the image ~1.5GB archive.

Grab the exercises ~9.7MB archive.

We recently moved our DNS to dnsmadeeasy.com from godaddy.com name servers.

After the transfer some internal CNAME records had a problem. So after fixing the problem and checking in the terminal that the changes propagated to the DNS server (host xxx.astrails.com) I tried to type the address in the browser, but it kept giving me the “can’t find host” error.

Nice idea and implementation of a schema-less data store on top of MySQL .

Recently we looked for video transcoding/hosting solution to use in one of our client’s projects.

The best thing we’ve found is Panda. It runs on Amazon stack of services including ec2, s3, and simpledb.

Using amazon has many advantages. no contracts, pay as you go, easy and fast scaling in case your site explodes :)

Unfortunately the image that is refered in the Getting Started (ami-05d7336c) is not safe for production - it has openssh version with a serious security bug, but don’t worry, we will explain how to fix it.

This blog-post is mostly targeted at non-Rails developers. Rails devs should know all this by heart :) Many times we need to explain to our customers what is ‘proper deployment’ and why their current one sucks :) Now we’ll be able to just point them to this post…

In the process of installing Mephisto I’ve got a problem with image_science gem.

It installed OK but when trying to require it the was a problem with RubyInline compilation:

We really like Debian and we usually use the current “stable” distribution for our production servers.

It all works great with one little problem: if you need very current soft it is probably not in the ‘stable’ yet.

The current Debian stable (“etch”) includes rubygems 0.9.0-5 which is way too old.

We needed to upgrade to at least 1.2.

Once again I’ve hit a problem of installing gems on a machine with very little memory.