Well, how slow? During my quest to find out, I collected some interesting data, and even more importantly put JRuby and MRI Ruby face to face.
Disclaimer: the benchmarks were not done on a well isolated and specially configured test harness, but I did my best to gather data with informational value. All the components were used with OOB settings.
- ruby 1.8.6 (2008-03-03 patchlevel 114) [universal-darwin9.0] + Mongrel Web Server 1.1.4
- jruby 1.1.6 (ruby 1.8.6 patchlevel 114) (2008-12-17 rev 8388) [x86_64-java] + GlassFish gem version: 0.9.2
- common backend: mysql5 5.0.75 Source distribution (InnoDB table engine, Rails pool set to 30)
BenchmarksI used an excellent high quality benchmarking framework Faban for my tests. I was lazy, so I only used fhb (very similar to ab, but without its flaws) to invoke simple benchmarks:
- simple request benchmark: bin/fhb -r 60/120/5 -c 10 http://localhost:3000/buckets/1
- component request benchmark: bin/fhb -r 60/120/5 -c 10 http://localhost:3000/bucket1/object1
ruby 1.8.6 + mongrel --------------------------------- simple action + single-threaded: ops/sec: 210.900 % errors: 0.0 avg. time: 0.047 max time: 0.382 90th %: 0.095 simple action + multi-threaded: ops/sec: 226.483 % errors: 0.0 avg. time: 0.044 max time: 0.180 90th %: 0.095 component action + single-threaded: ops/sec: 132.950 % errors: 0.0 avg. time: 0.075 max time: 0.214 90th %: 0.130 component action + multi-threaded: ops/sec: 131.775 % errors: 0.0 avg. time: 0.076 max time: 0.279 90th %: 0.125 jruby 1.2.6 + glassfish gem 0.9.2 ---------------------------------- simple action + single-threaded: ops/sec: 141.417 % errors: 0.0 avg. time: 0.070 max time: 0.259 90th %: 0.115 simple action + multi-threaded: ops/sec: 247.333 % errors: 0.0 avg. time: 0.040 max time: 0.318 90th %: 0.065 component action + single-threaded: ops/sec: 107.858 % errors: 0.0 avg. time: 0.092 max time: 0.595 90th %: 0.145 component action + multi-threaded: ops/sec: 179.042 % errors: 0.0 avg. time: 0.055 max time: 0.357 90th %: 0.085
ConclusionFrom my tests it appears that MRI is faster in single threaded mode, but JRuby makes up for the loss big time in the multi-threaded tests. It's also interesting to see that the multi-threaded mode gives MRI(green threads) a performance boost, but it's nowhere close to the boost that JRuby(native threads) can squeeze out from using multiple threads.
During the tests I noticed that rails was reporting more times spent in the db when using JRuby (2-80ms) compared to MRI (1-3ms). I don't know how reliable this data is but I wonder if this is the bottleneck that is holding JRuby back in the single threaded mode.
I benchmarked jruby-1.1.6, glassfish3 and rails-2.2 running on threadsafe mode.
The page I tested had several db lookups (perhaps 20), so it was rather rough on the implementation. I noticed that jruby+glassfish with several threads (i.e. stressing the CPU at 400%) was as fast as ruby+mongrel in single thread (i.e. stressing the CPU at 100%). So, jruby+glassfish was 4 times slower than ruby+mongrel. The DB used was mysql, and on jruby, I used the jdbcmysql adapter.
Was the number of requests per second the same in both cases? The difference in cpu usage only shows that jruby can use all four cpu cores you have, while c-ruby can use only one.
Post a Comment