Well, how slow? During my quest to find out, I collected some interesting data, and even more importantly put JRuby and MRI Ruby face to face.
Disclaimer: the benchmarks were not done on a well isolated and specially configured test harness, but I did my best to gather data with informational value. All the components were used with OOB settings.
Setup
- ruby 1.8.6 (2008-03-03 patchlevel 114) [universal-darwin9.0] + Mongrel Web Server 1.1.4
- jruby 1.1.6 (ruby 1.8.6 patchlevel 114) (2008-12-17 rev 8388) [x86_64-java] + GlassFish gem version: 0.9.2
- common backend: mysql5 5.0.75 Source distribution (InnoDB table engine, Rails pool set to 30)
Benchmarks
I used an excellent high quality benchmarking framework Faban for my tests. I was lazy, so I only used fhb (very similar to ab, but without its flaws) to invoke simple benchmarks:- simple request benchmark: bin/fhb -r 60/120/5 -c 10 http://localhost:3000/buckets/1
- component request benchmark: bin/fhb -r 60/120/5 -c 10 http://localhost:3000/bucket1/object1
Results
ruby 1.8.6 + mongrel --------------------------------- simple action + single-threaded: ops/sec: 210.900 % errors: 0.0 avg. time: 0.047 max time: 0.382 90th %: 0.095 simple action + multi-threaded: ops/sec: 226.483 % errors: 0.0 avg. time: 0.044 max time: 0.180 90th %: 0.095 component action + single-threaded: ops/sec: 132.950 % errors: 0.0 avg. time: 0.075 max time: 0.214 90th %: 0.130 component action + multi-threaded: ops/sec: 131.775 % errors: 0.0 avg. time: 0.076 max time: 0.279 90th %: 0.125 jruby 1.2.6 + glassfish gem 0.9.2 ---------------------------------- simple action + single-threaded: ops/sec: 141.417 % errors: 0.0 avg. time: 0.070 max time: 0.259 90th %: 0.115 simple action + multi-threaded: ops/sec: 247.333 % errors: 0.0 avg. time: 0.040 max time: 0.318 90th %: 0.065 component action + single-threaded: ops/sec: 107.858 % errors: 0.0 avg. time: 0.092 max time: 0.595 90th %: 0.145 component action + multi-threaded: ops/sec: 179.042 % errors: 0.0 avg. time: 0.055 max time: 0.357 90th %: 0.085
| Platform/Action | Simple | +/- | Component | +/- |
|---|---|---|---|---|
| Ruby ST | 210ops | 0% | 132ops | 0% |
| Ruby MT | 226ops | 7.62% | 131ops | -0.76% |
| JRuby ST | 141ops | -32.86% | 107ops | -18.94% |
| JRuby MT | 247ops | 17.62% | 179ops | 35.61% |
Conclusion
From my tests it appears that MRI is faster in single threaded mode, but JRuby makes up for the loss big time in the multi-threaded tests. It's also interesting to see that the multi-threaded mode gives MRI(green threads) a performance boost, but it's nowhere close to the boost that JRuby(native threads) can squeeze out from using multiple threads.During the tests I noticed that rails was reporting more times spent in the db when using JRuby (2-80ms) compared to MRI (1-3ms). I don't know how reliable this data is but I wonder if this is the bottleneck that is holding JRuby back in the single threaded mode.
