Sunday, November 16, 2008

Serialization: Protobuf vs Thrift vs Java

Here is a new project on the google code site named thrift-protobuf-compare that tries to compare Protobuf, Thrift and Java POJO serialization.
The results turned out to be very interesting.

The method is: creating three sets of objects and run the tests on them using the respective tools.
In each test the code first run a 200k iteration of the test without taking time (let the JIT warm up), asking the system to invoke the GC, and then running another 200k iteration with time measuring.

Note that the project does not do a full compare yet, and there are few ways to do each of the actions. Moreover, there is much more then the numbers (features like versioning, object merging and the RPC mechanizem). More investigation will follow.

The numeric results are in the project page, here are the graphs:
Milliseconds to create an object, smaller is better. The Protobuf results is not a mistake! It was created by the builder pattern.

Milliseconds to serialize an object to a byte array, smaller is better.
Milliseconds to deserialize an object from byte array, smaller is better.
Size of the byte array of a serialized object, smaller is better.

Results as you see are mixed, and maybe with a different implementation they might look different. But it seems that for now, Thrift performs much better in terms of CPU speed and protobuf buffer size is about 30% smaller then Thrift.

Please look at the code and post suggestions to add/change test cases.

4 comments:

Jon Skeet November 18, 2008 at 1:17 AM  

The protobuf you're using isn't optimised for speed. When you add "option optimize_for = SPEED" I expect you'll see a big difference.

Eishay Smith November 18, 2008 at 8:09 AM  

Ok, will do.

aantono November 20, 2008 at 7:12 AM  

The difference would be, indeed, huge. I would expect for protobuf with speed optimization to perform better than thrift.

Eishay Smith November 20, 2008 at 9:09 AM  

@aantono yes, it definitely is. See the sequel post Protobuf with option optimize for SPEED

Creative Commons License This work by Eishay Smith is licensed under a Creative Commons Attribution 3.0 Unported License.