Monday, February 01, 2010

Benchmarking Voldemort on SSD with production data

Its not a secret kaChing is using Voldemort in production. As our data set and grows larger we wish to figure out what is the best bang for the buck in terms of hardware investment. Would it be migrating our 32bit machines to 64bit ones and have more memory available for them or using SSD? There are good reasons why one may be better then the other but the only way to know is to test. Yes, we believe that Test Driven Development (TDD) applies to building system as well as coding.

The time John invested the time in creating the tool (written in Scala btw) to parse the logs and get the graphs had quickly payed itself. In John's post Voldemort in the Wild he wrote about the initial settings he created to test the changes before we actually did them and some of the initial result. The results are unique to our own system, which is the best thing about them. You may get different results for different load, usage patterns, data size, cluster setup.

There are more tunings we played with, waiting to John's next post.

Creative Commons License This work by Eishay Smith is licensed under a Creative Commons Attribution 3.0 Unported License.