Continuous Deployment at kaChing

In my last visit to Israel I visited IBM Research, Outbrain and Answers.com. Over there we discussed kaChing's culture, working methodologies and tools. The talk focused on continuous deployment and lean startup and was based on Pascal's and David's posts at kaChing Eng Blog.
Here is the presentation, using the fantastic Prezi tool.

Continuous Deployment on Prezi

Scala on Eclipse without the plugin

The Scala supported IDE is one of the few pain points of developers who want to start using Scala in their Java project. On existing long term project developed by a team its hard to step in and introduce a new language that is not supported by the existing IDE. On way to go about it is to hid the fact that you use Scala from the Java world by using one way dependency injection. Still, if you wish to truly absorb Scala into your existing java environment then you'll soon introduced cross language dependencies.

Most of our team is using Eclipse as the main IDE, its incrimental compilation in Java with its tight JUnit integration are great for fast TDD programming. Unfortunately the Eclipse Scala plugin is not there yet, it may hangs the IDE and messes up Java compilation - especially in large (more then 1000 source files) Java/Scala projects. Though the plugin is getting better over time some developers would find the plugin as a majore drag on their productivity.
For developers who do not write Scala at all or rather edit Scala with other editors, you can use this alternate path which lets them work on their Java or Scala code without messing with the plugin.

Compile Scala Eclipse project without Scala Plugin 

1. Add a compilation script to your project.
The Following script is using the Fast Scala Compiler (fsc). The fsc is a compilation server which always run in the background, as in a warm scalac always ready to receive new work. Is will reduce compilation time dramatically.
The classpath for compilation is taken from the Eclipse project .classpath file. You may take the source directory from there as well if you wish (exercise to the reader).
The params are not passed to the fsc in the command line since in my project's case the line is too long for the OS to handle. The alternative is to put it into a file and let fsc handle it for you.

#!/bin/sh
# Create a classpath from the eclipse .classpath file
lib=`grep classpathentry .classpath | grep "kind=\"lib\"" | tr -d '\t' | sed 's/.*path=\"\(.*[^src]\.jar\)\".*/\1/' | grep -v classpathentry`
CLASSPATH=`echo ${lib} | sed 's/ /:/g' `  

# point SCALA_HOME to Scala home (might want to add it to your project as well)  
export SCALA_HOME=lib-tools/scala-2.8.0

# java opts for your compilation server
export JAVA_OPTS="-client -Xmx1024M -Xms256M -XX:PermSize=128m -Xss2M -XX:MaxPermSize=256m -Xverify:none -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled"

DEST=target/bin-scala
mkdir -p $DEST

SCALAC_PROPERTIES=scalac.properties
echo "-classpath $CLASSPATH:target/bin" > $SCALAC_PROPERTIES
echo "-d $DEST -deprecation" >> $SCALAC_PROPERTIES
find src srctest -name *.java -o -name *.scala >> scalac.properties

time $SCALA_HOME/bin/fsc @$SCALAC_PROPERTIES &

2. Eclipse classpath
Add the scalac destination directory ($DEST) to the .eclipse classpath

3. Add the fsc builder to your project
Open the project properties and add the script above to the list of builders.
Set the working directory to be your project root directory.

5. Set the builder properties
Run it in the background when needed:

Add Scala syntax highlighting editor for Eclipse 

Even without the Scala plugin refactoring and debugging capabilities you can write some decent Scala code with the help of syntax highlighting for better readability. There are two options:

• The color editor which use jEdit modes. Daniel Spiewak added a Scala mode to it, unfortunately the plugin doesn't seem to work with Eclipse 3.6.
• The Colorer take5 project which works fine on my Eclipse on the Mac. It does not have a Scala mode yet but you can grab one at http://github.com/eishay/eclipsecolorer_scala 

It gives you a lightweight way (with its pros and cons) to write Scala in Eclipse.

InfoQ talk "Absorbing Scala in the Java Ecosystem"

About six months after, InfoQ released the video of the "Absorbing Scala in the Java Ecosystem" talk I gave at QCon San Francisco 2009.
The event was great though it feels funny looking at myself talking.

Findbugs, Husdon and Pizza Driven Development (PDD)

Original post at the kaChing Eng Blog

As you may know, kaChing is an test driven engineering organization. Test driven is not an option, its a must. We move fast and push code to production few dozens of times a day in a five minutes release cycle, so we must have high confidence in our code.
In complex systems there is no end to testings, each test system is an another line of defense which eventually gets broken but the more you have, the less chances bugs will reach production. We do not have QA team and do not want to have one, the reasoning is that if a human is involved in testing then there is a higher chance of missing things and you simply can't test all the site dozens of times a day.

Lately we decided to add a yet another line of defense: Code Static Analysis (e.g. Findbugs, PMD and CPD). We decided to start with Findbugs which has a great ANT task and Hudson plugin (both of these we use). The problem with these tools is that they're producing tons of warnings and most organizations ignore them after since they're too noisy to deal with.

David V. from testdriven.com recommended: "Pizza Driven Development" (aka PDD) which works as following:

Step one: Order Pizza. Most engineers would commit on doing something if it will get them Pizza. For the minority that would not be seduced by Pizza, good old fashion violence would do.
Step two: Give each member of the team two cards. Go over the list of rules with the team and have them vote on them. Voting is done using the cards where:

• No cards: I think the rule is stupid and we should filter it out in the findbugsExclude.xml
• One card: The rule is important but not critical.
• Two cards: The rule is super important and we should fix it right away.

The test sherif them creates three lists using the voting and the team works on fixing the "must fix" list which should have no more then few dozens of issues so it can be all fixed in couple of hours. Once you have that done the findbugs build is green and we're ready to go.

Next step is having hudson run findbugs on every post commit so the build is considered to be broken if a new findbugs issue is introduced. The engineer who introduced the issue must either filter the class from that rule in the xml file or fix the bug as a first priority. Since the engineers get a notification few minutes after the commit then they are probably still messing with the code and its easy for them to fix it on the spot.

In the next few weeks we are adding a new rule from the "not critical" list every few days. The goal is to have all the rules we think are important without the common "its to noisy, lets ignore it" approche. Only after we're done with that we're going to add the next static analysis tool to build. The good thing about these tools and hudson is that you can run them in parallel to the unit/integration tests, on another machine, so they won't slow down the overall release cycle.

Testing with Hibernate

Original post at the kaChing Eng Blog

Some of the unit tests I wrote lately involved setting up a data setup in the DB and testing the code using and mutating it. To make it clear, the DB is an in memory DB so the setup is super fast without IO slowdowns. Its very easy to do the setup using hibernate but the problem comes when you set a large collection of objects and a "java.sql.BatchUpdateException: failed batch" is being thrown. The frustrating point there is that hibernate won't let you know what exactly went wrong, even if you set your logger to "trace" level.

In order to solve it you can add the following to the system properties of that test:

-Dhibernate.statement_cache.size=0 
-Dhibernate.jdbc.batch_size=0 
-Dhibernate.jdbc.use_scrollable_resultset=false 
-Dhibernate.bytecode.use_reflection_optimizer=false 
-Dhibernate.show_sql=true

It will execute the statements one by one and will give more precise details of what went wrong.
Note: you do NOT want to use these properties in production or in the continuance integration (CI) environment.