Saturday, January 31, 2009

Benchmarking JRuby on Rails

Last night, while working on a project I found a really neat use of Rails Components, but I also noticed that this part of Rails is deprecated, among other reasons because it's slow.

Well, how slow? During my quest to find out, I collected some interesting data, and even more importantly put JRuby and MRI Ruby face to face.

Disclaimer: the benchmarks were not done on a well isolated and specially configured test harness, but I did my best to gather data with informational value. All the components were used with OOB settings.

Setup

  • ruby 1.8.6 (2008-03-03 patchlevel 114) [universal-darwin9.0] + Mongrel Web Server 1.1.4
  • jruby 1.1.6 (ruby 1.8.6 patchlevel 114) (2008-12-17 rev 8388) [x86_64-java] + GlassFish gem version: 0.9.2
  • common backend: mysql5 5.0.75 Source distribution (InnoDB table engine, Rails pool set to 30)


Benchmarks

I used an excellent high quality benchmarking framework Faban for my tests. I was lazy, so I only used fhb (very similar to ab, but without its flaws) to invoke simple benchmarks:
  • simple request benchmark: bin/fhb -r 60/120/5 -c 10 http://localhost:3000/buckets/1
  • component request benchmark: bin/fhb -r 60/120/5 -c 10 http://localhost:3000/bucket1/object1
Both tests were run with JRuby as well as with RMI Ruby and in addition to that I ran the tests with Rails in single-threaded as well as multi-threaded modes. I didn't use mongler clusters or glassfish pooled instances - there was always only one Ruby instance serving all the requests.

Results

ruby 1.8.6 + mongrel
---------------------------------
simple action + single-threaded:
ops/sec: 210.900
% errors: 0.0
avg. time: 0.047
max time: 0.382
90th %: 0.095

simple action + multi-threaded:
ops/sec: 226.483
% errors: 0.0
avg. time: 0.044
max time: 0.180
90th %: 0.095

component action + single-threaded:
ops/sec: 132.950
% errors: 0.0
avg. time: 0.075
max time: 0.214
90th %: 0.130

component action + multi-threaded:
ops/sec: 131.775
% errors: 0.0
avg. time: 0.076
max time: 0.279
90th %: 0.125

jruby 1.2.6 + glassfish gem 0.9.2
----------------------------------
simple action + single-threaded:
ops/sec: 141.417
% errors: 0.0
avg. time: 0.070
max time: 0.259
90th %: 0.115

simple action + multi-threaded:
ops/sec: 247.333
% errors: 0.0
avg. time: 0.040
max time: 0.318
90th %: 0.065

component action + single-threaded:
ops/sec: 107.858
% errors: 0.0
avg. time: 0.092
max time: 0.595
90th %: 0.145

component action + multi-threaded:
ops/sec: 179.042
% errors: 0.0
avg. time: 0.055
max time: 0.357
90th %: 0.085


Platform/ActionSimple+/-Component+/-
Ruby ST 210ops0%132ops0%
Ruby MT226ops7.62%131ops-0.76%
JRuby ST141ops-32.86%107ops-18.94%
JRuby MT247ops17.62%179ops35.61%
(ST - single-threaded; MT - multi-threaded)

Conclusion

From my tests it appears that MRI is faster in single threaded mode, but JRuby makes up for the loss big time in the multi-threaded tests. It's also interesting to see that the multi-threaded mode gives MRI(green threads) a performance boost, but it's nowhere close to the boost that JRuby(native threads) can squeeze out from using multiple threads.

During the tests I noticed that rails was reporting more times spent in the db when using JRuby (2-80ms) compared to MRI (1-3ms). I don't know how reliable this data is but I wonder if this is the bottleneck that is holding JRuby back in the single threaded mode.

Tuesday, January 13, 2009

Using ZFS with Mac OS X 10.5

A few days ago I got a new MacBook Pro. While waiting for it to be delivered, I started thinking about how I want to layout the installation of the OS. For a long long time I wanted to try to use ZFS file system on Mac and this looked like a wonderful opportunity. Getting rid of HFS+, which was causing me lots of problems (especially its case insensitive re-incarnation), sounds like a dream come true.

If you've never heard of ZFS before, check out this good 5min screencast of some of the important features.

A brief google search revealed that there are several people using and developing ZFS for Mac. There is a Mac ZFS porting project at http://zfs.macosforge.org and I found a lot of good info at AlBlue's blog.

Some noteworthy info:
  • The current ZFS port (build 119) is based on ZFS code that shipped with Solaris build 72
  • It's currently not possible to boot Mac OS X from a ZFS filesystem
  • Finder integration is not perfect yet - Finder lists a ZFS pool as an unmountable drive under devices
  • There are several reports of kernel panics, most of which appeared in connection to the use of cheap external USB disks (I haven't experienced any)
  • There are a bunch of minor issues, which I'm sure will eventually go away.
None of the above was a show stopper for me, so I went ahead with the installation. My plan was simple - repartition the internal hard drive to a small bootable partition and a large partition used by ZFS, which will hold my home directory and other filesystems.

Install ZFS

Even though MacOS X 10.5 comes with ZFS support, it's only a read-only support. In order to be able to really use ZFS, full ZFS implementation must be installed.

The installation is very simple and can be done by following these instructions: http://zfs.macosforge.org/trac/wiki/downloads. Alternatively, AlBlue created a fancy installer for the lazy ones out there.

Repartition Disk

Once ZFS is installed and the OS was rebooted, I could repartition the internal disk. If you are using an external hard drive, you'll most likely need to use zpool command instead.

First let's check what the disk looks like:
$ diskutil list
/dev/disk0
#:                       TYPE NAME                    SIZE       IDENTIFIER
0:      GUID_partition_scheme                        *298.1 Gi   disk0
1:                        EFI                         200.0 Mi   disk0s1
2:                  Apple_HFS boot                    297.8 Gi   disk0s2
Good, the internal disk was identified as /dev/disk0 and it currently contains an EFI (boot) slice and ~300G data slice/partition. Let's repartition the disk so that it contains two data partitions.
$ sudo diskutil resizeVolume disk0s2 40G ZFS tank 257G
Password:
Started resizing on disk disk0s2 boot
Verifying
Resizing Volume
Adjusting Partitions
Formatting new partitions
Formatting disk0s3 as ZFS File System with name tank
[ + 0%..10%..20%..30%..40%..50%..60%..70%..80%..90%..100% ]
Finished resizing on disk disk0
/dev/disk0
#:                       TYPE NAME                    SIZE       IDENTIFIER
0:      GUID_partition_scheme                        *298.1 Gi   disk0
1:                        EFI                         200.0 Mi   disk0s1
2:                  Apple_HFS boot                    39.9 Gi    disk0s2
3:                        ZFS tank                    252.0 Gi   disk0s3


Great, the disk was repartitioned and the existing data partition, which I call boot, was resized into a smaller 40GB partition and the extra space was used to create a ZFS pool called tank. Btw all the data on the boot partition was preserved.

Let's check my new pool:
$ zpool list
NAME                    SIZE    USED   AVAIL    CAP  HEALTH     ALTROOT
tank                    256G    360K    256G     0%  ONLINE     -
$ zpool status
pool: tank
state: ONLINE
status: The pool is formatted using an older on-disk format.  The pool can
 still be used, but some features are unavailable.
action: Upgrade the pool using 'zpool upgrade'.  Once this is done, the
 pool will no longer be accessible on older software versions.
scrub: none requested
config:

 NAME        STATE     READ WRITE CKSUM
 tank        ONLINE       0     0     0
   disk0s3   ONLINE       0     0     0

errors: No known data errors
The warning above just means that a new ZFS storage format is available but is not used by the current pool. As far as I could find there are no benefits for upgrading to the new format on Mac, but if I did, I would lose compatibility with Macs that have only the read-only ZFS support.

Create Filesystems

So now that the new pool exists, I can create a shiny new filesystem using a single command:
$ sudo zfs create tank/me3x
$ zfs list
NAME        USED  AVAIL  REFER  MOUNTPOINT
tank        388K   252G   270K  /Volumes/tank
tank/me3x    19K   252G    19K  /Volumes/tank/me3x
To configure this new filesystem as my home directory, I created a temporary admin account, logged in under this account and mounted the ZFS fs as /Users/me3x:
$ sudo mv /Users/me3x /Users/me3x.hfs
$ sudo zfs set mountpoint=/Users/me3x tank/me3x
$ sudo cp -rp /Users/me3x.hfs /Users/me3x
That's it. My Mac account now resides on a ZFS file system. Now I can finally enjoy all the benefits of using ZFS on my OpenSolaris box in my office as well as on my Mac. Bye bye HFS, I won't miss you! 

Sunday, January 11, 2009

Freezing activerecord-jdbc Gems into a Rails Project

Over the Christmas break a Slovak friend of mine (Hi Martin!) asked me to build a simple book library management app for a school in the Philippines where he's been volunteering for the past year. I thought to my self that if someone can volunteer one year of his life in such an amazing way, I could spend a few hours to help him out too.

Since from his description it was obvious that he was looking for a low maintenance solution, I though that a rails application with an embedded database would be a good choice. I worked with derby (JavaDB) in the past and I knew that derby drivers were already available as an active-record adapter gem, so I thought that it would be pretty simple to set up dev environment using Rails, JRuby, and embedded derby db. Surprisingly there were a few issues along the way.

I started with defining the database config in config/database.yml:
development:
adapter: jdbcderby
database: db/library_development
pool: 5
timeout: 5000
...
...
The database files for the dev db will be stored under RAILS_ROOT/db/library_development

Secondly I specified the gem dependency in config/environment.rb (you gotta love this Rails 2.1+ feature):
Rails::Initializer.run do |config|
...
config.gem "activerecord-jdbcderby-adapter", :version => '0.9', :lib => 'active_record/connection_adapters/jdbcderby_adapter'
...
end
Note that you must specify the :lib parameter, otherwise Rails won't be able to initialize the gem and you'll end up with:
no such file to load -- activerecord-jdbcderby-adapter
So far so good. Now let's install the gems we depend on:
$ jruby -S rake gems:install
(in /Users/me3x/Development/library)
rake aborted!
Please install the jdbcderby adapter: `gem install activerecord-jdbcderby-adapter` (no such file to load -- active_record/connection_adapters/jdbcderby_adapter)

(See full trace by running task with --trace)
Huh? I asked rake to install gems and I get an error that I need to install gems first? It turns out that this error comes from ActiveRecord, which tries to initialize db according to database.yml, and only then environment.rb gets to be read.

Ok, so let's install the db dependencies manually:
$ sudo jruby -S gem install activerecord-jdbcderby-adapter
Password:
JRuby limited openssl loaded. gem install jruby-openssl for full support.
http://wiki.jruby.org/wiki/JRuby_Builtin_OpenSSL
Successfully installed activerecord-jdbc-adapter-0.9
Successfully installed jdbc-derby-10.3.2.1
Successfully installed activerecord-jdbcderby-adapter-0.9
3 gems installed
Installing ri documentation for activerecord-jdbc-adapter-0.9...
Installing ri documentation for jdbc-derby-10.3.2.1...
Installing ri documentation for activerecord-jdbcderby-adapter-0.9...
Installing RDoc documentation for activerecord-jdbc-adapter-0.9...
Installing RDoc documentation for jdbc-derby-10.3.2.1...
Installing RDoc documentation for activerecord-jdbcderby-adapter-0.9...
Cool, let's check if all the dependencies are available:
$ jruby -S rake gems
(in /Users/me3x/Development/library)
- [I] activerecord-jdbcderby-adapter = 0.9
- [I] activerecord-jdbc-adapter = 0.9
- [I] jdbc-derby = 10.3.2.1

I = Installed
F = Frozen
R = Framework (loaded before rails starts)
Yay, all dependencies are installed.

In the past when dependencies couldn't be declared in environment.rb, I found developing with frozen rails and gems much more manageable, especially when the app is being developed by more than one person. This also made for less deployment surprises. With the config.gem defined dependencies, the situation changes quite a bit, but there are situations when it still makes sense to freeze gems into the project. So let's freeze the gems:
$ jruby -S rake gems:unpack:dependencies
(in /Users/me3x/Development/library)
WARNING:  Installing to ~/.gem since /usr/local/jruby/jruby-1.1.6/lib/ruby/gems/1.8 and
   /usr/local/jruby/current/bin aren't both writable.
Unpacked gem: '/Users/me3x/Development/library/vendor/gems/activerecord-jdbcderby-adapter-0.9'
Unpacked gem: '/Users/me3x/Development/library/vendor/gems/activerecord-jdbc-adapter-0.9'
Unpacked gem: '/Users/me3x/Development/library/vendor/gems/jdbc-derby-10.3.2.1'
Looks good, let's check it:
$ jruby -S rake gems
(in /Users/me3x/Development/library)
- [F] activerecord-jdbcderby-adapter = 0.9
- [F] activerecord-jdbc-adapter = 0.9
- [F] jdbc-derby = 10.3.2.1

I = Installed
F = Frozen
R = Framework (loaded before rails starts)
Nice all the dependencies are now frozen!