Skip to content
Print

The Brilliance of BDD

9
Jun
2008

As I have previously written, I have recently been spending some time experimenting with various aspects of Scala, including some of the frameworks which have become available.  One of the frameworks I have had the privilege of using is the somewhat unassumingly-titled Specs, and implementation of the behavior-driven development methodology in Scala.

Specs takes full advantage of Scala’s flexible syntax, offering a very natural format for structuring tests.  For example, we could write a simple specification for a hypothetical add method in the following way:

object AddSpec extends Specification {
  "add method" should {
    "handle simple positives" in {
      add(1, 2) mustEqual 3
    }
 
    "handle simple negatives" in {
      add(-1, -2) mustEqual -3
    }
 
    "handle mixed signs" in {
      add(1, -2) mustEqual -1
      add(-1, 2) mustEqual 1
    }
  }
}

We could go on, of course, but you get the picture.  This code will lead to the execution of four separate assertions in three tests (to put things into JUnit terminology).  Fundamentally, this isn’t too much different than a standard series of unit tests, just with a slightly nicer syntax.

Specs defines a domain-specific language for structuring test assertions in a simple and intuitive way.  However, this is hardly the only framework for BDD.  Perhaps the most well-known such framework is RSpec, which answers a similar use-case in the Ruby programming language.  Our previous specification could be rewritten using RSpec as follows:

describe AddLib do
  it 'should handle simple positives' do
    add(1, 2).should == 3
  end
 
  it 'should handle simple negatives' do
    add(-1, -2).should == -3
  end
 
  it 'should handle mixed signs' do
    add(1, -2).should == -1
    add(-1, 2).should == 1
  end
end

The end-result is basically the same: the add method will be tested against the given assertions (all four of them) and the results printed in some sort of report form.  In this area, RSpec is significantly more mature than Specs, generating very slick HTML reports and nicely formatted console output.  This isn’t really a fundamental weakness of the Specs framework however, just indicative of the fact that RSpec has been around for a lot longer.

These two frameworks are interesting of course, but they are merely implementations of a much larger concept: behavior-driven development.  I’ve never been much of a fan of unit testing.  It’s always seemed to be incredibly dull and a very nearly fruitless waste of effort.  As much as I hate it though, I have to bow to the benefits of a self-contained test suite; and so I press on, cursing JUnit every step of the way.

BDD provides a nice alternative to unit testing.  At its core, it is not much different in that test groupings and primitive assertions are used to check all aspects of a test unit against predefined data.  However, there is something about the “flow” of a behavioral spec that is considerably easier to deal with.  For some reason, it is far less painful to devise a comprehensive test suite using BDD principles than conventional unit testing.  It seems a little far-fetched, but BDD actually makes it easier to write (and more importantly, formulate) exactly the same tests.

It’s an odd phenomenon, one which can only be caused by the storyboard flow of the code itself.  It is very natural to think of distinct requirements for a test unit when each of these requirements are being labeled and entered in a logical sequence.  Moreover, the syntax of both Specs and RSpec is such that there is very little boiler-plate required to setup an additional test.  Compare the previous BDD specs with the following JUnit4 example:

public class MathTest {
    @Test
    public void testSimplePositives() {
        assertEquals(3, add(1, 2));
    }
 
    @Test
    public void testSimpleNegatives() {
        assertEquals(-3, add(-1, -2);
    }
 
    @Test
    public void testMixedSigns() {
        assertEquals(-1, add(1, -2));
        assertEquals(1, add(-1, 2));
    }
}

JUnit just requires that much more syntax.  It breaks up the logical flow of the tests and (more importantly) the developer train of thought.  What can be worse is this syntax bloat makes it very tempting to just group all of the assertions into a single test – to save typing if nothing else.  This is problematic because one assertion may shadow all the others in the case of a failure, preventing them from ever being executed.  This can make certain problems much more difficult to isolate.

Logical flow is extremely important to test structure.  BDD frameworks provide a very nice syntax for painlessly defining comprehensive test suites.  The really wonderful thing about all of this is that BDD is available on the JVM, right now.  There’s nothing stopping you from writing your code in Java as you normally would, then creating your test suite in Scala using Specs rather than JUnit.  Alternatively, you could use RSpec on top of JRuby, or Gspec with Groovy.  All of these are seamless replacements for a test framework like JUnit, and requiring of far less syntactic overhead.

The growing move toward polyglot programming encourages the use of a separate language when it is best suited to a particular task.  In this case, several languages are available which offer far more powerful test frameworks than those which can be found in Java.  Why not take advantage of them?

Comments

  1. Very cool article ;-)

    Just one thing I want to mention. If you want to have the exact equivalent of the rspec example, you can remove a “should” level and write:

    object AddSpec extends Specification {
    “it should handle simple positives” in {
    add(1, 2) mustEqual 3
    }

    “it should handle simple negatives” in {
    add(-1, -2) mustEqual -3
    }

    “it should handle mixed signs” in {
    add(1, -2) mustEqual -1
    add(-1, 2) mustEqual 1
    }
    }

    And if you want purely anonymous examples, because your code reads like no one, you can write:

    object AddSpec extends Specification {
    add(1, 2) mustEqual 3
    add(-1, -2) mustEqual -3
    add(1, -2) mustEqual -1
    add(-1, 2) mustEqual 1
    }

    (this works in the current release but I enhanced the naming in the latest SNAPSHOT release: 1.3.0-SNAPSHOT).

    Cheers,

    Eric (specs developer).

    Eric Torreborre Monday, June 9, 2008 at 12:48 am
  2. I wasn’t aware of that syntax. Thanks for the heads-up, Eric!

    As an aside, it’s worth mentioning that the should/in syntax is one of the things that I really liked about Specs as opposed to RSpec and similar. I’m sure there’s a way to replicate the syntax somehow with rspec, but it just falls out naturally with Specs. Very nice job!

    Daniel Spiewak Monday, June 9, 2008 at 1:02 am
  3. G’day Daniel,
    Have you seen Reductio at http://reductiotest.org/ ?
    In particular, the Scala examples? http://reductiotest.org/examples/scala

    Tony Morris Monday, June 9, 2008 at 7:23 pm
  4. I’m a bit skeptical. Is it that much of an improvement to group your tests this way?

    Eventually, you want to be pointed to a Java method so you can inspect it and debug it, so the information ’should handle mixed signs’ seems to be one extra layer of noise over “shouldHandleMixedSigns()”…


    Cedric

    Cedric Monday, June 9, 2008 at 7:46 pm
  5. @Tony

    Yes, I’ve looked Reductio, but only briefly. It’s about on the same level as ScalaCheck: a testing solution which is quite obviously superior in coverage and minimalism, but also way beyond my limited intellect. :-) I’m planning on really digging into combinitorial testing one of these days, but at the moment it’s easier for me to stick with the conventional canned-data methodology.

    @Cedric

    To be honest, I was quite skeptical too when I first started playing with Specs. It’s a very difficult phenomenon to describe, one which almost requires you to experience it for yourself. Obviously the reduced syntax cruft is one benefit of DSLs like Specs, but the overall flow is really the big win. Somehow – and I’m honestly not sure how – BDD specifications are psychologically easier to formulate, easier to code and (ultimately) easier to verify correct.

    Oh, and for the record: I have looked at TestNG, and I do find it to be a nicer framework than JUnit. However, it’s still a straight-up unit test framework. TestNG is to JUnit what SVN is to CVS: vastly improved, but still fundamentally the same.

    Daniel Spiewak Monday, June 9, 2008 at 8:12 pm
  6. You’re too hard on yourself mate :) Feel free to ask any questions.

    Tony Morris Monday, June 9, 2008 at 8:14 pm
  7. While I agree completely with your example on here, I think that the Java example is perhaps a little harsh; in JUnit 4 you now have the Hamcrest matchers which do a surprisingly good job (not nearly as good as Specs or RSpec, of course) considering that Java is the language.

    Written in this style the Java example would be:
    public class MathTest {
    @Test
    public void simplePositives() {
    assertThat( add(1, 2), is( 3 ) );
    }

    @Test
    public void simpleNegatives() {
    assertThat( add( -1, -2 ), is( 3 ) );
    }

    @Test
    public void testMixedSigns() {
    assertThat( add( 1, -2 ), is( -1 ) );
    assertThat( add( -1, 2 ), is( 1 ) );
    }
    }

    It at least reads a little more sensibly, and you can specify more complex Matchers which can be used like “is”. I love the stuff that you can do with the simple-but-powerful syntax within Scala; Specs is just one example but there are many more.

    Calum Leslie Tuesday, June 10, 2008 at 1:26 am
  8. Looks a lot like Hecl’s test suite in terms of how it flows, and the lack of extraneous crud getting in your way. test intro-3 { proc bar {} { return puts } proc foo {} { [bar] xyz } foo intro proccode foo } {[bar] xyz}

    website design Tuesday, June 10, 2008 at 7:24 am
  9. JTestr: http://jtestr.codehaus.org/ is also an alternative to consider.

    Torbjørn Tuesday, June 10, 2008 at 7:57 am
  10. Daniel, don’t forget Easyb (http://easyb.org) on Groovy’s options.
    Great post!

    Andres Almiray Wednesday, June 11, 2008 at 12:32 pm
  11. As previously pointed out, the Java example is a poor straw man. JUnit 4.4 has BDD extensions (Hamcrest) and has been shipping for almost a year.

    Andrew Binstock Thursday, June 19, 2008 at 6:40 pm
  12. Hamcrest doesn’t really solve everything. It is a more “fluid” syntax, but it’s also significantly more verbose. It’s not too bad, but it really feels like strapping a saddle onto a dinosaur: you just can’t “fix” JUnit. It’s a nice tool because it’s so ubiquitous, but you can’t say much more for it.

    Daniel Spiewak Thursday, June 19, 2008 at 6:58 pm
  13. BDD and TDD is not about Java or Scala, or verbose Syntax. People will write Specs/Tests or won’t.

    The difference is a state of mind, BDD is different than TDD (and I was doing BDD sucessfully with JUnit and TestNG)

    Peace
    -stephan

    Stephan Schmidt Sunday, September 21, 2008 at 10:34 pm
  14. Theres also JBehave ( http://jbehave.org/ ) and JDave ( http://www.jdave.org/ ) to consider..

    I really like JBehave’s scenarios..

    Really cool to write tests like this:

    Given a stock of prices 0.5,1.0 and a threshold of 15.0
    When the stock is traded at 5.0
    Then the alert status should be OFF
    When the stock is traded at 11.0
    Then the alert status should be OFF

    Nino Martinez Wednesday, October 29, 2008 at 1:56 pm

Post a Comment

Comments are automatically formatted. Markup are either stripped or will cause large blocks of text to be eaten, depending on the phase of the moon. Code snippets should be wrapped in <pre>...</pre> tags. Indentation within pre tags will be preserved, and most instances of "<" and ">" will work without a problem.

Please note that first-time commenters are moderated, so don't panic if your comment doesn't appear immediately.

*
*