Skip to content
Print

Buildr Still Not Ready for Prime Time

12
May
2008

As some of you may know, I’ve been following the Buildr project with a fair degree of interest of late.  Just last week, Assaf announced their first release as an Apache project: Buildr 1.3.0.  Being of the inquisitive bend, I decided that this would be an excellent time to reevaluate its suitability for use in my projects, both commercial and personal.

A while back, I evaluated Buildr as a potential replacement for Ant and Maven2 and eventually came to the unfortunate conclusion that it just wasn’t ready.  At the time, Buildr was still hampered by a number of annoying limitations (such as being unable to reconfigure the default directory structure).  I’m happy to say that many of these problems have been fixed in the 1.3 release, as well as a large number of feature additions which I hadn’t thought to request.  Despite all that, I’m still forced to conclude that Buildr simply isn’t (yet) suitable as my build tool of choice.

Now before you stone me for utter heresy, try to hear me out.  I’m not dissing Buildr in any way, I really want to use it, but I just can’t justify moving over to it in the face of all of its current issues.  What’s really unfortunate is that all of these issues can be summed up with a single word: transitivity.

One of Maven’s killer features is the ability to resolve the entire transitive dependency graph.  Now I’ll grant that it does this with varying degrees of success, but for the most part it’s pretty smart about how it fixes up your CLASSPATH.  As of 1.3, Buildr claims to have experimental support for transitive dependency resolution, but judging from the problems I encountered in my experimentation, it’s barely even deserving of mention as an experiment, much less a full-fledged feature.

To understand why transitive dependencies are such a problem, it is of course necessary to first understand the definition of such.  In a word (well, several anyway), a transitive dependency is an inherited dependency, an artifact which is not depended upon directly by the project, but rather by one of its dependencies.  This definitions carries recursively to dependent parents, grand-parents and so on, thus defining a transitive dependency graph.  Consider it this way:

image

In the diagram, MyCoolProject is the artifact we are trying to compile.  The only dependencies we have actually specified for this artifact are the DBPool and Databinder artifacts.  However, the Databinder artifact has declared that it depends upon both the Wicket and the ActiveObjects artifacts.  ActiveObjects doesn’t depend upon anything, but Wicket has dependencies SLF4J and on the Servlets API.  Thus, our original goal, MyCoolProject, has transitive dependencies Wicket, SLF4J, Servlets and ActiveObjects.  Quite a bit more than we thought we were asking for when we declared our dependency on Databinder.

In general, this sort of transitive resolution is a good thing.  It means that instead of specifying six dependencies, we only had to specify two.  Furthermore, we observed the DRY principle by not re-specifying dependency information already contained within the various packages.  As with any “good thing”, there’s definitely a valid argument regarding its down sides, but on the whole I’m quite fond of this feature.

In Maven, this sort of thing happens by default, which can lead to some very confusing and subtle CLASSPATH problems (conflicting packages, etc).  Buildr takes a slightly different approach in 1.3 by forcing the use of the transitive method:

repositories.remote << 'http://www.ibiblio.org/maven2'
 
define 'MyCoolProject' do
  project.version = '1.0'
 
  compile.with transitive('xmlrpc:xmlrpc-server:jar:3.0')
 
  package :jar
end

It’s easy to see why Buildr is raising such a ruckus in the build genre.  It’s syntax is elegance itself, and since it’s actually a proper scripting language (Ruby), there’s really nothing you can’t do with it.  Having to remember to specify the default repository is a bit of a pain, but it’s certainly something I can live with.  The real problem with this example is a little less subtle: It doesn’t work.

If you create a buildfile with the above contents and then run the buildr command, the results will be something like the following:

Downloading org.apache.xmlrpc:xmlrpc:jar:3.0
rake aborted!
Failed to download org.apache.xmlrpc:xmlrpc:jar:3.0, tried the following repositories:

http://www.ibiblio.org/maven2/

(See full trace by running task with --trace)

After a fairly significant amount of digging, I managed to discover that this problem is caused by the fact that Buildr attempts to download a corresponding JAR file for every POM it resolves.  This seems logical until you consider that many large projects (including Databinder, Wicket, and Hibernate) define POM-only projects which exist for the sole purpose of creating transitive dependencies.  They’re organizational units, designed to allow projects to depend upon one super-artifact, rather than a dozen sub-projects.  It’s a very common practice, and one which Buildr completely fails to handle appropriately.

After some prodding on the Buildr dev mailing-list, Assaf admitted that this is an issue worth looking at and provided a temporary workaround (pending a rework of the functionality in 1.4):

def pom_only(a)
  artifact a do |task|
    mkpath File.dirname(task.to_s)
    Zip::ZipOutputStream.open task.to_s
  end
end

This was about the point that I started wondering if maybe Ant/Ivy would be a better bet, at least for the time being.  To use this workaround, you must call the pom_only method once for every POM-only project in the dependency graph.  Usually, this means you must invoke Buildr repeatedly and find the troublesome artifacts by trial and error.  Not exactly a “just works” solution.

Pressing forward however, I unearthed a deeper, even more insidious issue: an intermittent failure to generate Eclipse project meta.  I’m not sure if this is due to the POM-only dependencies or just bad juju, but whatever the reason, it’s annoying.  I’ve raised the issue on the Buildr mailing lists, but so far no response.  Basically, what happens is something like this:

C:\\Users\\Daniel Spiewak\\Desktop\\MyCoolProject> buildr eclipse
(in C:/Users/Daniel Spiewak/Desktop/MyCoolProject, development)
Completed in 0.499s
C:\\Users\\Daniel Spiewak\\Desktop\\MyCoolProject>

Not exactly the most helpful output.  In case you were wondering, this process did not create the Eclipse metadata.  It’s interesting to note that calling buildr idea (to create project metadata for IntelliJ) seems to work just fine.  Whatever causes the bug, it seems to be specific to just the Eclipse project generator.

Buildr is a remarkable project.  It shows potential to someday become the de-facto build system, possibly even unseating Ant.  Unfortunately, that day is not today.  There are too many odd wrinkles and unpredictable errors to really call it a “finished product”.  Hopefully, the Buildr dev team will continue their excellent work, eventually producing a tool worthy of serious consideration.  Until then, I guess that I’m (still) stuck with Ant.

Update: It seems that the issues with transitive dependencies have been resolved in the latest Buildr versions in their SVN.  I’m looking forward to when this all becomes stable and publicly consumable!

Comments

  1. I like Schmant (http://www.schmant.org/) as a build tool.

    Erik Tjernlund Monday, May 12, 2008 at 12:26 am
  2. What about stuff like SWF and SWC artifacts. Would I have to write a workaround that handles all the different types of dependencies I might want to retrieve from the remote repository?

    Tim O'Brien Monday, May 12, 2008 at 8:09 am
  3. Hmm, SWF artifacts might be a problem. You could actually reuse the same `pom_only` method since actually it’s doing a generic mock of a JAR file. In fact, you would have to use this method on any artifact which does not contain a JAR, not just artifacts which are POM only.

    As an aside, I’m not sure that Buildr knows how to properly download SWF artifacts. It’s fairly hacky right now, the way it parses the POM and figures out what to download. You may have to write some special logic for this above and beyond the workaround for JAR-less artifacts.

    Daniel Spiewak Monday, May 12, 2008 at 8:49 am
  4. From what I could tell it would deal with SWF and SWC correctly, the problem is only with master POMs that a few (but pretty damn useful) projects use. Hopefully we’ll fix that for the transitive method soon enough.

    Separately, watch the buildr-dev list. Looks like we have agreement on doing transitive dependencies rfor 1.4.

    Assaf Monday, May 12, 2008 at 12:23 pm
  5. @Assaf

    I’m looking forward to it! 1.4 is going to be a great release.

    Daniel Spiewak Monday, May 12, 2008 at 12:30 pm
  6. I can definitely see where your criticism of the feature comes from, seeing as it doesn’t currently work well, but it’s ironic because transitive dependencies are in many cases undesirable. The cleanliness and correctness of the POMs buried deep beneath the other POMs that you rely on can have undesirable effects, e.g., putting JUnit or three different versions of commons-logging in your web application…

    Paul Brown Monday, May 12, 2008 at 2:57 pm
  7. While I agree that there are some cases were transitive dependencies can be a major pain / totally crippling, I still think that they are a *critical* feature for any dependency management system. They eliminate a lot of redundancy and make it much easier to work with major packages like Hibernate and Wicket.

    Daniel Spiewak Monday, May 12, 2008 at 3:02 pm
  8. Gant and Gradle are two possibilities. They both use Groovy as the scripting language and they eschew XML. Gant is the build engine used by Grails, so it clearly works on large complex projects. Just a thought.

    Andrew Binstock Monday, May 12, 2008 at 3:04 pm
  9. Do you mind sharing your thoughts on Gradle?

    Ittay Dror Thursday, June 5, 2008 at 3:53 am
  10. To be honest, I know next-to-nothing about it. I’ve looked at their project page briefly, and it seemed interesting, but the syntax looked almost too heavy-weight for a proper build system. Even Buildr can be a bit too much at times. That was just my impression though, I’m sure there’s a lot about it that I’m missing.

    Daniel Spiewak Thursday, June 5, 2008 at 11:35 am

Post a Comment

Comments are automatically formatted. Markup are either stripped or will cause large blocks of text to be eaten, depending on the phase of the moon. Code snippets should be wrapped in <pre>...</pre> tags. Indentation within pre tags will be preserved, and most instances of "<" and ">" will work without a problem.

Please note that first-time commenters are moderated, so don't panic if your comment doesn't appear immediately.

*
*