Thursday, July 16, 2009

Scala and EasyMock

Writing tests is an integral part of coding. If you're writing unit tests without using a Mock library like EasyMock or jMock you better check one of them out.

Didn't test EasyMock to its extreme but until now I didn't see any problem. Using partial functions as follows looks like a nice pattern to use EasyMock. It takes care of the EZMock in the useMock utility method and creates a nice separation between the "expected" block and "executor" one:

* 1. gets a mockable class
* 2. creates a mock object out of it
* 3. executes the "expected" block
* 4. replays the mock
* 5. runs the execution code
* 6. verify the mock
private def useMock[S](mockable : Class[S])(run : S => Unit)(expected : S => Unit ) {
val mock = createMock(mockable)
Using it goes like this:
    val mockRunner = useMock (classOf[ToMock]) { mock =>
val toTest = new ToTest(mock)
toTest.doSomethingWithToMock()//should call method1() and method2() on mocked
} _
//change some conditions
mockRunner {mock =>

Wednesday, July 15, 2009

Speaking at the SiliconValley CodeCamp '09

The two talks at the SiliconValley CodeCamp 2009 (October 3rd & 4th, 2009).
Naturally the talks are Scala and Google App Engine centric. Please post suggestions and comments. Here are the abstracts:

A First Look at Scala on Google App Engine
GAE is a great Scala environment, especially since its coding patterns are pushing the programmer to do functional oriented programming.

In this technical talk we will discuss using Scala In Google App Engine, we’ll go through using Scala along with:

  • GAE’s basic services
  • Java Data Objects (JDO) – GAE’s interface to the BigTable based Datastore. How to use Scala syntax for JDO annotation and class declaration
  • Google Web Toolkit (GWT) – interfacing with GWT Java only garden
  • ANT – loosing IDE dependency, compiling/running your Scala app from the command line without Eclipse’s Plugins
  • Discuss what you cannot do with Scala

Absorbing Scala into Java Ecosystem
Scala runs on the JVM, can use and be used by Java code almost transparently. Its Java speed and focus on concurrency well position it for demanding server side applications.
This session is for those who consider using Scala in their existing Java projects. We’ll discuss how to smoothly integrate Scala into an existing Java build, testing, development and runtime systems.

In this session we will talk about how to deal with the learning curve, IDE integrations and the peopleware aspects of introducing Scala to your organization.

The session will include examples and anecdotes from the LinkedIn teams who currently use Scala in production.

Tuesday, July 14, 2009

Scala @ LinkedIn

I should actually do a full post on Scala at LinkedIn, but today I only wish to report that we had a very nice Scala BASE meeting at LinkedIn. We actually had a much better place today (a full building to ourselves). See you again in two months !

Monday, July 13, 2009

Social goes vertical

Since I found comments on few posts on other social I'm starting to use DISQUS for comments. I like the way social activity goes vertical across different sites as a large set of mashups having every solution excel in a very focused part of the overall social network we live in. No doubt, Google about to focus on this trend with Wave and Friend Connect.

Saturday, July 11, 2009

Scala on Google App Engine playing it nice with Java GWT and JDO

Few days ago I gave a short review on the SF Bay Area Google App Engine Developers meetup about Scala on the Google App Engine. If you would like to dive into it, here are some of the details with examples taken from the code of the newspipes app available at GitHub.
Scala playes out very nicely with GAE with the same pros and cons a Java application would have. There is one thing Scala can't do which is to take part in any GWT related code. It has nothing to do with GAE since GWT is designed to work only with Java code. If you wish to use GWT for the front end and still use Scala for the back end - no problems. You write your GWT code with Java, write the GWT service interface with Java so the GWT compiler will be happy, and implement the service with Scala. The service implementation is hooked up with GWT in the web.xml as the http end point.

Next step will be using JDO. If your persistent class is used by GWT then you must use Java. Else you may use Scala which, typical to Scala, makes the code much smaller. For example:

import javax.jdo.annotations.{Extension, Persistent, IdGeneratorStrategy, PrimaryKey, PersistenceCapable, IdentityType}

@PersistenceCapable{val identityType = IdentityType.APPLICATION}
class SearchKeyword(
@Persistent{val valueStrategy = IdGeneratorStrategy.IDENTITY}
@Extension{val vendorName="datanucleus", val key="gae.encoded-pk", val value="true"}
var key: String,
@Persistent var value: String,
@Persistent var count: Int)
Now we need to compile it all. Using the Eclipse plugins might be nice but you should have a proper build file if you want to do more then "Hello World". This build will run test, deploy in production and test environment and handle Java, Scala, GWT and JDO compilations. On top of it, I use IntelliJ IDEA which has the best Scala support at the moment (competition is great!) and you really don't want to be strained into an IDE.
To do the Scala part first run the Scalac compiler to the same place you'll compile the Java code a step after:
  <target name="compile" depends="copyjars" description="Compiles Java source and copies other source files to the WAR.">
<mkdir dir="${dstdir}" />
<copy todir="${dstdir}">
<fileset dir="${srcdir}">
<exclude name="${javafiles}" />
<exclude name="${scalafiles}" />
<src path="${srcdir}"/>
<classpath refid="project.classpath"/>
<javac srcdir="${srcdir}"
debug="on" />
Then comes the JDO enhancement part which takes the compiled source code and does its stuff. Note that it does not care at this point where the classes came from (Java or Scala) since it works on the *.class files the compilers placed in ${dstdir} (war/WEB-INF/classes).
  <target name="datanucleusenhance" depends="compile"
description="Performs JDO enhancement on compiled data classes.">
<enhance_war war="war" />
That's about it.

Thursday, July 02, 2009

Microbenchmarking Scala vs Java

Following Nick's post I came up to check the numbers and investigate a possible improvement using Scala 2.8 @specialized annotation.
Final numbers are at the end of the post.
I added few changes:
First I thought it would be better if both Scala and Java would sort the same size of array ;-) In Nick's example Java sorted 10,000 elements and Scala sorted 100,000.
The other was to do few iterations int he same JVM run to let JIT kick in. So now the Scala code looks like this:

package quicksort
import java.lang.Long.MAX_VALUE

object QuicksortScala {

def quicksort(xs: Array[Int]) {

def swap(i: Int, j: Int) {
val t = xs(i); xs(i) = xs(j); xs(j) = t

def sort1(l: Int, r: Int) {
val pivot = xs((l + r) / 2)
var i = l;
var j = r
while (i <= j) {
while (xs(i) < pivot) i += 1
while (xs(j) > pivot) j -= 1
if (i <= j) {
swap(i, j)
i += 1
j -= 1
if (l < j) sort1(l, j)
if (j < r) sort1(i, r)
sort1(0, xs.length - 1)

def main(args : Array[String]) {
var time = MAX_VALUE
for(i <- 0 to 100) (time = Math.min(time, doSort))
println("Scala time = " + time)

def doSort() = {
var a : Array[Int] = new Array[Int](10000000)
var i : Int = 0
for (e <- a) {
a(i) = i*3/2+1;
if (i%3==0) a(i) = -a(i);
i = i+1
val t1 = System.currentTimeMillis();
quicksort (a)
val t2 = System.currentTimeMillis();
t2 - t1
And here is the Java code:
package quicksort;

public class QuicksortJava {

public void swap(int[] a, int i, int j) {
int temp = a[i];
a[i] = a[j];
a[j] = temp;

public void quicksort(int[] a, int L, int R) {
int m = a[(L + R) / 2];
int i = L;
int j = R;
while (i <= j) {
while (a[i] < m)
while (a[j] > m)
if (i <= j) {
swap(a, i, j);
if (L < j)
quicksort(a, L, j);
if (R > i)
quicksort(a, i, R);

public void quicksort(int[] a) {
quicksort(a, 0, a.length - 1);

public static void main(String[] args) {
QuicksortJava sorter = new QuicksortJava();
long time = Long.MAX_VALUE;
for(int i = 0; i < 100; i++)
time = Math.min(time, sorter.doSort());

System.out.println("java time = " + time);

private long doSort(){
// Sample data
int[] a = new int[10000000];
for (int i = 0; i < a.length; i++) {
a[i] = i * 3 / 2 + 1;
if (i % 3 == 0)
a[i] = -a[i];

long t1 = System.currentTimeMillis();
long t2 = System.currentTimeMillis();
return t2 - t1;
Decompiling the Scala code shows that there is no benefit of using the @specialized annotation since the Scala compiler compiles all the ints to primitives. Here is the Scala class decompiled code:
package quicksort;
import java.rmi.RemoteException;
import scala.*;
import scala.runtime.*;
public final class QuicksortScala$ implements ScalaObject{
public QuicksortScala$(){}
private final void sort1$1(int l, int r, int ai[]) {
int pivot = ai[(l + r) / 2];
int i = l;
int j = r;
if(i > j) break;
for(; ai[i] < pivot; i++);
for(; ai[j] > pivot; j--);
if(i <= j) {
swap$1(i, j, ai);
} while(true);
if(l < j) sort1$1(l, j, ai);
if(j < r)l = i;
else return;
} while(true);
private final void swap$1(int i, int j, int ai[]){
int t = ai[i];
ai[i] = ai[j];
ai[j] = t;
public long doSort(){
ObjectRef a$1 = new ObjectRef(new int[0x989680]);
IntRef i$1 = new IntRef(0);
(new BoxedIntArray((int[])a$1.elem)).foreach(new anonfun.doSort._cls1(a$1, i$1));
long t1 = System.currentTimeMillis();
long t2 = System.currentTimeMillis();
return t2 - t1;
public void main(String args[]){
LongRef time$1 = new LongRef(0xffffffffL);
Predef$.MODULE$.intWrapper(0).to(100).foreach(new anonfun.main._cls1(time$1));
Predef$.MODULE$.println((new StringBuilder()).append("Scala time = ").append(BoxesRunTime.boxToLong(time$1.elem)).toString());
public void quicksort(int xs$1[]){
sort1$1(0, xs$1.length - 1, xs$1);
public int $tag() throws RemoteException {
return scala.ScalaObject.class.$tag(this);
public static final QuicksortScala$ MODULE$ = this;
static { new QuicksortScala$(); }
I tried to compile and run the code in Scala 2.8 anyway and the results where exactly the same as with running it under Scala 2.7.5.
The numbers:
Scala: 1355ms
Java: 1265ms

==> Java was 7% faster then Scala (Closer gap then the 33% in the previous benchmark).

Ismael found a bug in the Scala Code, I updated Nick as well. The line
      if (j < r) sort1(i, r)
should have been
      if (i < r) sort1(i, r)
Changing the line made Java and Scala be equal in performance.
==> The Scala code above is no slower or faster then Java (as should be in this specific case).

Creative Commons License This work by Eishay Smith is licensed under a Creative Commons Attribution 3.0 Unported License.