Monday, February 16, 2009

Fun with Voldemort, Scala and Protobuf

Here is an idea for an alternative for the Voldemort shell. I added one Class and use the Scala interactive shell.
Below is an example of how you can use it to manipulate Java and Protobuf objects (of course it can be applied to other stuff), do scripting/grandfathering and enjoy the interactive shell.

Before we go on, you can use this shell without protobuf or scala, they are in the example only for proof of concept.
For the example I created two 'person' classes, one POJO and the other Java generated form protobuf.
The example simulates put/get/copy/grandfathering of a list of objects.
This is a copy from the shell:

scala>//this is a comment (like in C).
scala>//creating pojo store client. note how it prints some info
about itself after construction.
scala> var pojoStore = new VoldemortClient[String,Person]
Established connection to pojo-store via tcp://localhost:6666
pojoStore: voldemort.VoldemortClient[String,test.pojo.Person] =
store name : pojo-store
bootstrap url : tcp://localhost:6666
key serializer: StringSerializer
val serializer: ObjectSerializer

scala>// creates a person, load it to pojoStore,
download it back and prints its first name
scala> var me = new Person("Lord", "Voldemort", 666)
me: test.pojo.Person = test.pojo.Person@d5ea2c
scala> pojoStore put ("t1", me)
scala> pojoStore get "t1" getFirstName
res31: java.lang.String = Lord

scala>//The line below creates a list of three person objects
scala> val pojos = List(new Person("Harry", "Potter", 1),
new Person("Rubeus", "Hagrid", 2),
new Person("Ron", "Weasley", 3))

scala>// closure that loads a person object with its ID as key
scala> val upload = (p: Person) => pojoStore put (p.getId.toString, p)
upload: (test.pojo.Person) => Unit =

scala>// loads the person objects to the store
scala> pojos foreach upload

scala>// closure that retrieves the loaded persons from store
and prints their first name
scala> val getNames = (p: Person) =>
println(pojoStore get (p.getId.toString) getFirstName)
scala> pojos foreach getNames

scala>// creating protobuf store client. note the value serializer.
PersonPBO.Person is a protobuf Message class.
scala> var protoStore = new VoldemortClient[String,PersonPBO.Person]
Established connection to proto-store via tcp://localhost:6666
protoStore: voldemort.VoldemortClient[int,test.proto.PersonPBO.Person] =
store name : proto-store
bootstrap url : tcp://localhost:6666
key serializer: StringSerializer
val serializer: ProtoBufSerializer

scala>// converts a person to protobuf
scala> val toProto = (p: Person) =>
PersonPBO.Person newBuilder() setId(p getId)
setFirstName(p getFirstName) setLastName(p getLastName) build
toProto: (test.pojo.Person) => test.proto.PersonPBO.Person =

scala>// loads the protobuf object to the protoStore
scala> val copyToProto = (p: PersonPBO.Person) =>
protoStore put (p.getId.toString, p)
copyToProto: (test.proto.PersonPBO.Person) => Unit =

scala>// downloads the pojo object from the pojoStore
scala> val get = (id: Int) => pojoStore get (id.toString)
get: (Int) => test.pojo.Person =

scala>// download pojos from pojoStore, create protos and
upload them to protoStore
scala> List(1, 2, 3) map get map toProto foreach copyToProto

scala>// closure that downloads and prints objects from protoStore
scala> val getProto = (id: Int) => println(protoStore get (id.toString))
getProto: (Int) => Unit =

scala>// the printout is the protobufs the next line found on the protoStore
scala> List(1, 2, 3) foreach getProto
firstName: "Harry"
lastName: "Potter"
id: 1

firstName: "Rubeus"
lastName: "Hagrid"
id: 2

firstName: "Ron"
lastName: "Weasley"
id: 3

The example above is basic, it can be enhanced much more.


Creative Commons License This work by Eishay Smith is licensed under a Creative Commons Attribution 3.0 Unported License.