– we create awesome web applications

We usually use dragonfly to handle user generated assets in almost all the projects. But sometimes dragonfly with ImageMagick doesn't play nicely in a limited environments like heroku.

We were getting tons of R14 - Memory quota exceeded errors after analyzing even small images using ImageMagick's identify command.

Here is how we solved it.

First of all the context.

We use direct S3 upload on the client side in order to reduce the heroku servers load. Client goes to the Rails server with a sign request and gets back a policy, a signature and a key (aka path) of the resource to be uploaded to S3. There are more details about jQuery-File-Upload flow here.

Once the file is uploaded client goes to ImagesController and creates a record for the image.

class My::ImagesController < InheritedResources::Base
  before_filter :authenticate_user!
  actions :create, #...
  respond_to :json

  # ...
end

"create" action receives the only parameter original_image_uid which is passed to the model.

class Image < ActiveRecord::Base
  belongs_to :user
  dragonfly_accessor :original_image

  def original_uid=(value)
    self.original_image_uid = value
    self.original_image_width = original_image.analyse(:width)
    self.original_image_height = original_image.analyse(:height)
    self.original_image_size = original_image.file.size
  end
end

This is where all the (image-) magic happens. Before the model is saved we analyze the image width, height and the file size in order to use later according to the application business needs.

First call to the original_image will download the file from S3, files can be upto a few megabytes, so it takes about a second in the production environment. Than original_image.analyse calls the ImageMagick's identify command and cache its results.

So everything is quite straightforward. But, we started to get R14 errors on heroku after the images#create requests. We were under impression that some huge memory leak eats up all the memory, but it turned out that it was not garbage collected memory bloat that happens right after the identify command returns.

It looks like ImageMagick's identify tries to get as much memory as possible with no particular reason from my perspective. So we had to fight these bloats in a few different ways.

First is to run garbage collection. Check out gctools, the only thing we had to do is to add these lines to config.ru

require 'gctools/oobgc'
if defined?(Unicorn::HttpRequest)
  use GC::OOB::UnicornMiddleware
end

It works with unicorn running on ruby 2.1. Learn more about it in the Aman Gupta's blog.

And everything got back to normal, no R14 any more because the memory was cleaned up properly after each request.

But, why should we allow identify to take so much memory at the first place? And here comes the solution: passing limits to the identify command.

Another line of code added to initializers/dragonfly.rb

Dragonfly.app.configure do
  plugin :imagemagick, identify_command: "identify -limit memory 0 -limit map 0"
  #...
end

So, the ImageMagick doesn't eat so much memory any more, and even if it does the bloat will be garbage collected after the request.

I'm going to start a series of short digest blog posts that will cover a few things worth mentioning. I sumble upon a lot of things reading different sources, here I will share the most interesting ones. Well, at least most intersting for me.

Here we go.

Tools.

http://breach.cc/

A browser for the HTML5 era Entirely written in Javascript. Free. Modular. Hackable.

The browser is really nice especially its minimalistic design. Chrome Development Console works as usual, so it can really be an alternative for Incognito Tabs/Windows when debugging web applications that require to work on flows involving different logged in users.

https://github.com/google/ios-webkit-debug-proxy/

The ios_webkit_debug_proxy allows developers to inspect MobileSafari and UIWebViews on real and simulated iOS devices via the DevTools UI and WebKit Remote Debugging Protocol.

I'm really an Apple fun, but Google's DevTools rock.

http://macdown.uranusjr.com/

MacDown. The open source Markdown editor for OS X.

I used MarkdownPro for a while, but this one is really cool and free.

http://iterm2.com/

iTerm2 is a replacement for Terminal and the successor to iTerm.

2.0 is released.

Security.

http://words.zemn.me/csp

Detecting login state for almost any website on the internet

I'm not sure if it worths it to re-implement Location-based redirects in all the projects I participated but big guys probably have to consider this.

http://minilock.io/

File encryption software that does more with less.

In short, users have only remember the passphrase, keys pair will be generated automatically based on the passphrase.

Development.

Where were a lot of buz about 2 charting libraries recently: http://fastly.github.io/epoch/ and http://www.chartjs.org/. Both are cool, I'm looking forward to testing both of them in a next project that will require charting.

http://component.kitchen/

Component Kitchen - Great ingredients for your web apps

Tons of useful code, all searchable.

http://customelements.io/

Custom Elements - a web components gallery for modern web apps

More or less the same.

http://www.polymer-project.org/

Welcome to the future - Web Components usher in a new era of web development based on encapsulated and interoperable custom elements that extend HTML itself.

Most exciting thing for front end development ever happened since Backbone.js. I'm trying to use it in the mobile verion of http://isratracker.com which will comes out shortly.

Design.

http://www.google.com/get/noto/

Google Noto Fonts - Beautiful and free fonts for all languages

Beautiful indeed, looks even better then Roboto.

http://designairs.com/inside-brand-evolution/

Inside our Brand Evolution

An interesting reading about airbnb rebranding.

Reading

http://venturebeat.com/2014/07/18/how-yo-became-one-of-the-most-viral-apps-of-all-time-step-by-step/

How Yo became one of the most viral apps of all time — step by step

http://www.logolounge.com/article/2013logotrends/

2013 Logo Trends