Skip to content
Print

Interop Between Java and Scala

9
Feb
2009

Sometimes, the simplest things are the most difficult to explain.  Scala’s interoperability with Java is completely unparalleled, even including languages like Groovy which tout their tight integration with the JVM’s venerable standard-bearer.  However, despite this fact, there is almost no documentation (aside from chapter 29 in Programming in Scala) which shows how this Scala/Java integration works and where it can be used.  So while it may not be the most exciting or theoretically interesting topic, I have taken it upon myself to fill the gap.

Classes are Classes

The first piece of knowledge you need about Scala is that Scala classes are real JVM classes.  Consider the following snippets, the first in Java:

public class Person {
    public String getName() {
        return "Daniel Spiewak";
    }
}

…and the second in Scala:

class Person {
  def getName() = "Daniel Spiewak"
}

Despite the very different syntax, both of these snippets will produce almost identical bytecode when compiled.  Both will result in a single file, Person.class, which contains a default, no-args constructor and a public method, getName(), with return type java.lang.String.  Both classes may be used from Scala:

val p = new Person()
p.getName()       // => "Daniel Spiewak"

…and from Java:

Person p = new Person();
p.getName();      // => "Daniel Spiewak"

In the case of either language, we can easily swap implementations of the Person class without making any changes to the call-site.  In short, you can use Scala classes from Java (as well as Java classes from Scala) without ever even knowing that they were defined within another language.

This single property is the very cornerstone of Scala’s philosophy of bytecode translation.  Wherever possible — and that being more often than not — Scala elements are translated into bytecode which directly corresponds to the equivalent feature in Java.  Scala classes equate to Java classes, methods and fields within those classes become Java methods and fields.

This allows some pretty amazing cross-language techniques.  For example, I can extend a Java class within Scala, overriding some methods.  I can in turn extend this Scala class from within Java once again with everything working exactly as anticipated:

class MyAbstractButton extends JComponent {
  private var pushed = false
 
  def setPushed(p: Boolean) {
    pushed = p
  }
 
  def getPushed = pushed
 
  override def paintComponent(g: Graphics) {
    super.paintComponent(g)
 
    // draw a button
  }
}
public class ProKitButton extends MyAbstractButton {
    // do something uniquely Apple-esque
}

Traits are Interfaces

This is probably the one interoperability note which is the least well-known.  Scala’s traits are vastly more powerful than Java’s interfaces, often leading developers to the erroneous conclusion that they are incompatible.  Specifically, traits allow method definitions, while interfaces must be purely-abstract.  Yet, despite this significant distinction, Scala is still able to compile traits into interfaces at the bytecode level…with some minor enhancements.

The simplest case is when the trait only contains abstract members.  For example:

trait Model {
  def value: Any
}

If we look at the bytecode generated by compiling this trait, we will see that it is actually equivalent to the following Java definition:

public interface Model {
    public Object value();
}

Thus, we can declare traits in Scala and implement them as interfaces in Java classes:

public class StringModel implements Model {
    public Object value() {
        return "Hello, World!";
    }
}

This is precisely equivalent to a Scala class which mixes-in the Model trait:

class StringModel extends Model {
  def value = "Hello, World!"
}

Things start to get a little sticky when we have method definitions within our traits.  For example, we could add a printValue() method to our Model trait:

trait Model {
  def value: Any
 
  def printValue() {
    println(value)
  }
}

Obviously, we can’t directly translate this into just an interface; something else will be required.  Scala solves this problem by introducing an ancillary class which contains all of the method definitions for a given trait.  Thus, when we look at the translation for our modified Model trait, the result looks something like this:

public interface Model extends ScalaObject {
    public Object value();
 
    public void printValue();
}
 
public class Model$class {
    public static void printValue(Model self) {
        System.out.println(self.value());
    }
}

Thus, we can get the effect of Scala’s powerful mixin inheritance within Java by implementing the Model trait and delegating from the printValue() method to the Model$class implementation:

public class StringModel implements Model {
    public Object value() {
        return "Hello, World!";
    }
 
    public void printValue() {
        Model$class.printValue(this);
    }
 
    // method missing here (see below)
}

It’s not perfect, but it allows us to use some of Scala’s more advanced trait-based functionality from within Java.  Incidentally, the above code does compile without a problem.  I wasn’t actually aware of this fact, but “$” is a legal character in Java identifiers, allowing interaction with some of Scala’s more interesting features.

There is, however, one little wrinkle that I’m conveniently side-stepping: the $tag method.  This is a method defined within the ScalaObject trait designed to help optimize pattern matching.  Unfortunately, it also means yet another abstract method which must be defined when implementing Scala traits which contain method definitions.  The correct version of the StringModel class from above actually looks like the following:

public class StringModel implements Model {
    public Object value() {
        return "Hello, World!";
    }
 
    public void printValue() {
        Model$class.printValue(this);
    }
 
    public int $tag() {
        return 0;
    }
}

To be honest, I’m not sure what is the “correct” value to return from $tag.  In this case, 0 is just a stub, and I’m guessing a safe one since StringModel is the only subtype of Model.  Can anyone who knows more about the Scala compiler shed some light on this issue?

Generics are, well…Generics

Generics are (I think) probably the coolest and most well-done part of Scala’s Java interop.  Anyone who has more than a passing familiarity with Scala will know that its type system is significantly more powerful than Java’s.  Some of this power comes in the form of its type parameterization, which is vastly superior to Java’s generics.  For example, type variance can be handled at declaration-site, rather than only call-site (as in Java):

abstract class List[+A] {
  ...
}

The + notation prefixing the A type parameter on the List class means that List will vary covariantly with its parameter.  In English, this means that List[String] is a subtype of List[Any] (because String is a subtype of Any).  This is a very intuitive relationship, but one which Java is incapable of expressing.

Fortunately, Scala is able to exploit one of the JVM’s most maligned features to support things like variance and higher-kinds without sacrificing perfect Java interop.  Thanks to type erasure, Scala generics can be compiled to Java generics without any loss of functionality on the Scala side.  Thus, the Java translation of the List definition above would be as follows:

public abstract class List<A> {
    ...
}

The variance annotation is gone, but Java wouldn’t be able to make anything of it anyway.  The huge advantage to this translation scheme is it means that Java’s generics and Scala’s generics are one and the same at the bytecode level.  Thus, Java can use generic Scala classes without a second thought:

import scala.Tuple2;
 
...
Tuple2<String, String> me = new Tuple2<String, String>("Daniel", "Spiewak");

Obviously, this is a lot more verbose than the Scala equivalent, “("Daniel", "Spiewak")“, but at least it works.

Operators are Methods

One of the most obvious differences between Java and Scala is that Scala supports operator overloading.  In fact, Scala supports a variant of operator overloading which is far stronger than anything offered by C++, C# or even Ruby.  With very few exceptions, any symbol may be used to define a custom operator.  This provides tremendous flexibility in DSLs and even your average, every-day API (such as List and Map).

Obviously, this particular language feature is not going to translate into Java quite so nicely.  Java doesn’t support operator overloading of any variety, much less the über-powerful form defined by Scala.  Thus, Scala operators must be compiled into an entirely non-symbolic form at the bytecode level, otherwise Java interop would be irreparably broken, and the JVM itself would be unable to swallow the result.

A good starting place for deciding on this translation is the way in which operators are declared in Scala: as methods.  Every Scala operator (including unary operators like !) is defined as a method within a class:

abstract class List[+A] {
  def ::[B >: A](e: B) = ...
 
  def +[B >: A](e: B) = ...
}

Since Scala classes become Java classes and Scala methods become Java methods, the most obvious translation would be to take each operator method and produce a corresponding Java method with a heavily-translated name.  In fact, this is exactly what Scala does.  The above class will compile into the equivalent of this Java code:

public abstract class List<A> {
    public <B super A> List<B> $colon$colon(B e) { ... }
 
    public <B super A> List<B> $plus(B e) { ... }
}

Every allowable symbol in Scala’s method syntax has a corresponding translation of the form “$trans“.  A list of supported translations is one of those pieces of documentation that you would expect to find on the Scala website.  However, alas, it is absent.  The following is a table of all of the translations of which I am aware:

Scala Operator Compiles To
= $eq
> $greater
< $less
+ $plus
- $minus
* $times
/ div
! $bang
@ $at
# $hash
% $percent
^ $up
& $amp
~ $tilde
? $qmark
| $bar
\ $bslash
: $colon

Using this table, you should be able to derive the “real name” of any Scala operator, allowing its use from within Java.  Of course, the idea solution would be if Java actually supported operator overloading and could use Scala’s operators directly, but somehow I doubt that will happen any time soon.

Odds and Ends

One final tidbit which might be useful: @BeanProperty.  This is a special annotation which is essentially read by the Scala compiler to mean “generate a getter and setter for this field”:

import scala.reflect.BeanProperty
 
class Person {
  @BeanProperty
  var name = "Daniel Spiewak"
}

The need for this annotation comes from the fact that Scala’s ever-convenient var and val declarations actually generate code which looks like the following (assuming no @BeanProperty annotation):

// *without* @BeanProperty
public class Person {
    private String name = "Daniel Spiewak";
 
    public String name() {
        return name;
    }
 
    public void name_$eq(String name) {
        this.name = name;
    }
}

This works well from Scala, but as you can see, Java-land is not quite paradise.  While it is certainly feasible to use the _$eq syntax instead of the familiar set/get/is triumvirate, it is not an ideal situation.

Adding the @BeanProperty annotation (as we have done in the earlier Scala snippet) solves this problem by causing the Scala compiler to auto-generate more than one pair of methods for that particular field.  Rather than just value and value_$eq, it will also generate the familiar getValue and setValue combination that all Java developers will know and love.  Thus, the actual translation resulting from the Person class in Scala will be as follows:

public class Person {
    private String name = "Daniel Spiewak";
 
    public String name() {
        return name;
    }
 
    public String getName() {
        return name();
    }
 
    public void name_$eq(String name) {
        this.name = name;
    }
 
    public void setName(String name) {
        name_$eq(name);
    }
}

This merely provides a pair of delegates, but it does suffice to smooth out the mismatch between Java Bean-based frameworks and Scala’s elegant instance fields.

Conclusion

This has been a whirlwind, disjoint tour covering a fairly large slice of information on how to use Scala code from within Java.  For the most part, things are all roses and fairy tales.  Scala classes map precisely onto Java classes, generics work perfectly, and pure-abstract traits correspond directly to Java interfaces.  Other areas where Scala is decidedly more powerful than Java (like operators) do tend to be a bit sticky, but there is always a way to make things work.

If you’re considering mixing Scala and Java sources within your project, I hope that this article has smoothed over some of the doubts you may have had regarding its feasibility.  As David Pollack says, Scala is really “just another Java library”.  Just stick scala-library.jar on your classpath and all of your Scala classes should be readily available within your Java application.  And given how well Scala integrates with Java at the language level, what could be simpler?

Comments

  1. Daniel,

    Nice write up. There’s one typo, though. The example
    “def ::[B >: A](e: B) = …”
    translates to Java
    “public <A> List<B> $colon$colon(B e) { … }”
    That is, the A & B are switched in the generics bit.

    Dean Wampler Monday, February 9, 2009 at 11:09 am
  2. @Dean

    Nice catch! I’ll fix the article asap.

    Daniel Spiewak Monday, February 9, 2009 at 11:57 am
  3. Fun stuff! What about closures? Just instances of Function in Java?

    Jevgeni Kabanov Monday, February 9, 2009 at 4:02 pm
  4. @Jevgeni

    Exactly. Specifically, Scala defines the following translations:

    ()=>String       ---->     Function0<String>
    (Int)=>String           ---->      Function1<Integer, String>
    (Boolean, Int)=>String     ---->     Function2<Boolean, Int, String>
    ...

    And so on. Function0, Function1 and Function2 are all traits. This means that function literals are almost trivial to *use* within Java (just call the apply() method to invoke), but much more difficult to create. For example:

    Function1<Integer, String> f = new Function1<Integer, String>() {
        public int $tag() {
            return Function1$class.$tag(this);
        }
    
        public <A> Function1<A, String> compose(Function1<A, Integer> f) {
            return Function1$class.compose(this, f);
        }
    
        public String apply(Integer i) {
            return i.toString();
        }
    };

    Compared to the precisely equivalent Scala:

    val f = { i: Int => i.toString }

    Quite a difference, isn’t it? :-)

    Daniel Spiewak Monday, February 9, 2009 at 4:11 pm
  5. Minor correction: the Function1 anonymous inner-class would have to implement the `andThen` method as well.

    Daniel Spiewak Monday, February 9, 2009 at 4:12 pm
  6. I was afraid that this’d be the breaking point of the smooth integration. Considering how much Scala API is based on closures it could get quite painful to use from Java. I guess loss of the context supplied by the advanced type system might also get quite painful with a lot of casts involved. Still better than the next best thing, though :)

    BTW I wouldn’t say that Groovy is harder it use from Java. It’s just that it loses most of it dynamic features if you don’t treat it as interpreted text.

    Jevgeni Kabanov Monday, February 9, 2009 at 4:40 pm
  7. @Jevgeni

    Yeah, Scala function values in Java are nightmarish.

    One thing that can be done to improve that is define utility methods and Java interfaces to abstract out most of the boilerplate. Like so:

    public interface JFunction0<R> {
        public R apply();
    }
    
    public class FunUtils {
        public static <R> Function0<R> fun(final JFunction0<R> f) {
            return new Function0<R>() {
                public int $tag() {
                    return Function0$class.$tag();
                }
    
                public R apply() {
                    f.apply();
                }
            };
        }
    
        // etc...
    }

    With this, you can define Scala functions from Java like so:

    Function0<String> f = fun(new JFunction0<String>() {
        public String apply() {
            return "Hello, World!";
        }
    });

    Better, but still pretty ugly.

    Actually, I think that a lot of the pain comes from the fact that Scala requires the $tag method. The good news is that this method is in candidacy for removal. It was an optimization back in the day, but it has been discovered that the JVM is pretty efficient with a large string of instanceof checks, so it may not really be necessary anymore.

    Daniel Spiewak Monday, February 9, 2009 at 4:54 pm
  8. I was actually considering at some point to make a small bytecode postprocessing library for Scala integration. You could definitely make implementing traits and closures easier (e.g. convert single-method classes to closures on the fly and autoimplement delegated trait methods).

    Jevgeni Kabanov Monday, February 9, 2009 at 5:28 pm
  9. Yeah, the $tag thing doesn’t seem too useful. The fastest/coolest implementation of instanceof is on Azul, they have 64bit pointers to objects and use ~18bits of that pointer to encode the class information. Instance of is just an AND test against the class bitmask. It’s quite fast on the rest of JVMs as well.

    Jevgeni Kabanov Monday, February 9, 2009 at 5:32 pm
  10. @Jevgeni

    Post-processing would definitely smooth a lot of these integration issues, particularly with boiler-plate stuff like traits. I’m always a bit leery of bytecode processing (except when absolutely necessary), but this seems like one of those times where it would be extremely useful.

    Interesting stuff re: Azul. The unfortunate consequence of that implementation would be a limitation to 64bit machines, but that’s not as much of an issue as it used to be — particularly on the server side, where 64bit is truly ubiquitous.

    I built a compiler once (targeting MIPS) where every class in the language had a unique integer ID. Every object in the heap had (in addition to its dispatch table, generation flag, etc) an integer space which registered its type. The real beauty of the scheme was the way in which type integers were computed based on their supertypes. Each type also knew how many subtypes it had. Thus, instanceof was a simple matter of “typeSup < typeSub < (typeSup + numSub)”. Not as good as an AND operation, but not too horrible either. It also had the added disadvantage of obviating separate compilation, but it got the job done.

    Daniel Spiewak Monday, February 9, 2009 at 7:32 pm
  11. When do you need to add parentheses to a method definition, and when not? Eg. in your first Scala example you have them, in the second (def getPushed = …) not.

    Jörn Zaefferer Tuesday, February 10, 2009 at 6:47 am
  12. @Jörn

    It’s a matter of style. By convention, methods with side-effects are given parentheses, while methods without side-effects leave them off. Thus, pure-functional methods look more like public fields when you’re reading through the code.

    Whether or not you apply parentheses, the compilation result will be the same except for some bytecode attributes that Java ignores. Thus, it makes no difference for Java interop.

    Daniel Spiewak Tuesday, February 10, 2009 at 9:35 am
  13. Great write-up.

    There are issues with the Eclipse plugin currently though, when using both Scala and java…some deficiencies in handling dependencies.

    Razvan Cojocaru Tuesday, February 10, 2009 at 2:05 pm
  14. >> Other areas where Scala is decidedly more powerful than Java (like operators) do tend to be a bit sticky, but there is always a way to make things work.

    That is the same as saying all languages are interoperable in machine code.

    In your examples I do not consider over loaded operator (and to a lesser extends BeanProperty annotations) to support interop in any reasonable manner.

    Bhaskar Maddala Tuesday, February 10, 2009 at 3:31 pm
  15. Having Odersky gives an unfair advantage to Scala in JVM integration :)

    Gabriel C Thursday, February 12, 2009 at 9:22 am
  16. @Gabriel

    :-) I could agree more!

    Daniel Spiewak Thursday, February 12, 2009 at 9:38 am
  17. Excellent post.

    I’m not a fan of the dynamic language trend where all of the magic is done at runtime and nothing maps to real method calls, just lots of metaclass lookup.

    Scala doing magic at compile time and then mapping to regular Java constructs under the hood is a welcome change.

    I’m currently pining for a language with the same compile-time, “still Java” magic of Scala, but a bit simpler. boojay is really close, I think, but needs to mature beyond a cool hack.

    Stephen Haberman Friday, February 13, 2009 at 9:10 pm
  18. may i know in wat all possible ways scala is advantageous over java?

    prithvi Sunday, February 15, 2009 at 1:14 am
  19. @prithvi

    * Syntax (much more concise than Java)
    * Function values (sometimes called “closures”)
    * Concise immutability
    * Symbolic methods (operator overloading)
    * No statics (pure object-orientation)
    * A type system which actually works
    * Sane generics

    I could go on and on, but those are the biggies.

    Daniel Spiewak Monday, February 16, 2009 at 8:37 am
  20. Unfortunately though, lists aren’t lists:
    http://stubbisms.wordpress.com/2009/02/18/fighting-scala-scala-to-java-list-conversion/

    Antony Stubbs Friday, February 20, 2009 at 6:48 pm
  21. It looks like the ‘FunctionN$class.$tag’ methods aren’t defined in v2.7.1 through v2.7.3. They don’t show up using javap and Java code like the examples above in the comments don’t compile. Are the examples based on earlier Scala versions?

    Dean Wampler Sunday, March 22, 2009 at 1:49 pm
  22. @Dean

    You’re quite right:

    daniel@lal ~ $ javap -classpath /usr/local/scala-2.7.3.final/lib/scala-library.jar scala.Function1$class
    Compiled from "Function1.scala"
    public interface scala.Function1 extends scala.ScalaObject{
        public abstract scala.Function1 andThen(scala.Function1);
        public abstract scala.Function1 compose(scala.Function1);
        public abstract java.lang.String toString();
        public abstract java.lang.Object apply(java.lang.Object);
    }
    
    daniel@lal ~ $ javap -classpath /usr/local/scala-2.7.3.final/lib/scala-library.jar scala.ScalaObject
    Compiled from "ScalaObject.scala"
    public interface scala.ScalaObject{
        public abstract int $tag()       throws java.rmi.RemoteException;
    }
    
    daniel@lal ~ $ javap -classpath /usr/local/scala-2.7.3.final/lib/scala-library.jar 'scala.Function1$class'
    Compiled from "Function1.scala"
    public abstract class scala.Function1$class extends java.lang.Object{
        public static void $init$(scala.Function1);
        public static scala.Function1 andThen(scala.Function1, scala.Function1);
        public static scala.Function1 compose(scala.Function1, scala.Function1);
        public static java.lang.String toString(scala.Function1);
    }

    I didn’t actually try to compile any of my examples in the comments, which would explain how I missed this. As a general work-around, you can probably just return 0 from $tag and I don’t think that it will cause too many problems. Alternatively, you can compile a real function value in Scala, decompile it and then look at the bytecode source for $tag, see what Scala does in this case.

    Daniel Spiewak Sunday, March 22, 2009 at 1:56 pm
  23. Martin suppressed $tag methods in pure interfaces about a year ago:

    https://lampsvn.epfl.ch/trac/scala/ticket/722

    I got rid of $tag entirely a couple months ago.

    Paul Phillips Sunday, April 12, 2009 at 6:27 am
  24. I think a lot of us need the more basic questions answered, such as how do you compile the two languages together? How do we change an existing (eclipse) ant build.xml to include scala classes? If we have an existing Java project, is there a quick scala script or skeleton we can adapt to call our classes and see if there are compatibility problems? Advice, such as “If you see this error, then you have a compatibility problem” vs “If you see this error, then you just haven’t laid out your file tree correctly”.

    Gary Saturday, April 25, 2009 at 9:50 am
  25. Hi sir,

    How to implement the scala List with Java code?
    Such as:
    def archs (e:Edge) : List[Point] = Nil

    How can I transfer it to java code?

    Jiang Thursday, December 3, 2009 at 12:17 am
  26. Daniel, thank you for all the Scala content. It is very, very useful for a C# refugee like me. My interest in Scala may yet drag me to the Java platform.

    This post just makes me wish that we had a real Scala implementation on .NET/Mono. In a lot of ways, Scala seems like a better fit for the CLR than it does for the JVM.

    Method definitions in traits would map well to extension methods on interfaces in .NET. You could translate trait methods right back to C# code and never know it was not natively written. No need for ‘$’ in the names. Tools like Visual Studio would show the extension methods as valid methods for any classes deriving from the interface. Very seamless.

    It is great that Scala generics are Java generics but both are smoke and mirrors at the JVM level. The CLR supports generics natively at the bytecode level.

    Properties are native to the CLR as well so they translate naturally from Scala. Of course, getters and setters seem just as natural in Java.

    Of course, operator overloading is allowed on the CLR so you would not need as much of this brilliant but still somewhat ugly operator to alphabetical name translation stuff.

    Having interoperability in reverse would be the real treat. It would be great to see my C# interfaces and extension methods map to nice clean Scala traits for example.

    My point is not to slam the JVM. It is the real-use in the real world leader after all. Rather, I am simply expressing my Scala envy and thinking out loud what a great language it would be on .NET/Mono.

    There is a real overlap in capability between C# and Scala (even the mix of OO and functional) but Scala takes it a little further and seems to suit me better.

    Maybe someday Martin will reward me (he threatens to sometimes).

    Thanks for the great content.

    jmalcolm Tuesday, July 27, 2010 at 11:42 am
  27. Glad to see scala’s classes can be used in java. An example of say using a scala immutable map might be nice, too…

    roger Friday, June 17, 2011 at 11:20 am
  28. I’ve found one big inter-op problem. If I have a abstract class in Scala with a protected method and I extend this class in Java and wish to keep this method protected – I CANNOT.

    I get a compiler error that you must use a public modifier. If I look into the bytecode created from the Scala source I see the originally protected method is now public in the .class file.

    This mandates me to expose my API in a public form – WHICH IS NOT WHAT I WANT!!!

    Yair Ogen Sunday, July 17, 2011 at 5:57 am
  29. Great article, Daniel. What’s the story on tuples? Suppose I have a Scala method that returns a tuple, and it is called from Java. Can Java figure it out? Thanks.

    Russ P.

    Russ P. Wednesday, November 9, 2011 at 12:00 am

Post a Comment

Comments are automatically formatted. Markup are either stripped or will cause large blocks of text to be eaten, depending on the phase of the moon. Code snippets should be wrapped in <pre>...</pre> tags. Indentation within pre tags will be preserved, and most instances of "<" and ">" will work without a problem.

Please note that first-time commenters are moderated, so don't panic if your comment doesn't appear immediately.

*
*