Using monit to manage backgroundrb
Somehow these days I had to deal with long running tasks in my rails projects. By far, BackgrounDRb seems like the right tool for the job. However, some gotchas gave me one of those typical sleepless nights trying to figure out how to deploy the damn thing.
Enter monit
The production environment I set up includes nginx, and monit with 4 thin instances, running on ubuntu server. Not being a professional sysadmin, I had some troubles with mongrel clusters. Besides, thin seems to work better for my case. Inserting BackgrounDRb in the game seemed like a trivial task. Like, monit is meant to do that?
Google first, than think
Trying not to be a smartass, I did some searches for monit configuration, stumbling upon Capitate, which seemed like the right thing to use. Wrong. I tried
cap backgroundrb:monit:install
Which indeed installed a monit file that may work on fedora or other systems, which have /sbin/service. Ubuntu does not. So I reluctantly tried to create my own config. How hard can it be?
check process backgroundrb
with pidfile /var/www/apps/myapp/shared/pids/backgroundrb_11006.pid
start program = "/usr/local/bin/ruby /var/www/apps/myapp/current/script/backgroundrb start
-e production > /var/www/apps/myapp/current/log/backgroundrb-cap.log 2>&1"
stop program = "/usr/local/bin/ruby /var/www/apps/myapp/current/script/backgroundrb stop"
group myapp
Let’s not get too fancy with restarts and so on, I am not sure what are the typical resources it needs. Let’s reboot, and check what goes on. 4 ruby processes running, no sign of BackgrounDRb.
cd /var/www/apps/myapp/current/log && less log/backgroundrb_debug_11006.log
Hm, some vague error about No such file or directory – packet_worker_runner. This result pointed me to the right direction – although the executable was available, Monit somehow did not see it. Of course, there is a good reason to do so – security. Say no more:
For security reasons monit purges the environment and only set a
spartan PATH variable that contains /bin, /usr/bin, /sbin and
/usr/sbin. If your program or script dies, the reason could be
that it expects certain environment variables or to find certain
programs via the PATH. If this is the case you should set the
environment variables you need directly in the start or stop
script called by monit.
No sign of /usr/local/bin, where the packet_worker_runner is situated (installed ruby from source). Fortunately the FAQ entry was a helpful, and adding this line at the top of script/backgroundrb file solved my troubles:
ENV['PATH'] = "#{ENV['PATH']}:/usr/local/bin"
The good news: I did not have to do anything about my cap recipe, since I already restarted monit with the group of my add. Grabbed a beer from the fridge.
Hi Petyo -
Another possibility is to add /usr/local/bin to Monit’s internal path:
http://tomcopeland.blogs.com/juniordeveloper/2009/07/monit-mongrel-and-usrlocalbin.html
Yours,
Tom
Tom Copeland
September 3, 2009 at 4:38 pm
Nice to see ya Tom! Neat solution, but you will be tied up to your compiled version of monit, and you should not forget to apply the patch every time
.
petyo
September 3, 2009 at 5:42 pm
@petyo, ha! I was wondering if that was the same Petyo
Yeah, true, that is the downside.
Tom Copeland
September 3, 2009 at 6:00 pm