Happy Easter everyone, we’ve got a new release!  ActiveObjects 0.8 is simmering at low heat (and footprint) on the servers even as I type.  This is probably the most significant milestone we’ve released thus far in that the 1.0 stream is now basically feature-complete.  I won’t be adding any new features to this release, just polishing and testing the heck out of the old ones.  We’ve already got a suite of 91 tests and I’ve got lots of ideas for more.  Expect this framework to get very, very stable in the next few months.

Features

Despite an annoyingly cramped schedule in the last couple months, I have managed to implement a few long-requested features.  The big ticket item for this release is pluggable cache implementations.  For those of you not “in the know”, ActiveObjects has a multi-tiered cache system composed of several layers and delegates.  This isn’t entirely unusual for an ORM, which is why I hadn’t mentioned it before.  The three basic layers:

• Entity Cache - A SoftReference map from id/type pairs to actual entity values.  This means that if you run a query which returns an entity for people {id=1} and then run a second query which returns an entity for the same row, the two instances will be identical.  This is what enables the additional cache layers to function properly (and it saves on memory).
• Value Cache - This is where all the field values for a given row are stored.  This will store both field values from the database as well as entity values (corresponding to foreign key fields).  Each entity has a separate instance of this cache.  Most of the memory required by ActiveObjects is taken up here.  With a long-running application, quite a bit of the database can actually get paged into memory.  This is a really good thing, since it drastically reduces round-trips to the database.  Thanks to the SoftReference(s) in the entity cache, running out of memory due to stale cache entries isn’t a problem.
• Relations Cache - This is probably the most complex of all of the cache layers.  This cache literally caches any one-to-one, one-to-many or many-to-many result sets and spits them out on demand.  This reduces round-trips still more to the point where certain applications can be running completely from memory, even querying on complex relations.  This in and of itself isn’t that complex; the hard part is knowing when to expire specific cache entries.  This cache layer is essentially three different indexes to the same data, allowing cache expiry for a number of different circumstances.  Naturally, these structures don’t represent well as a single map of key / value pairs.  (more on this in a bit)

As of the 0.8 release, the value cache and relations cache layers have been unified under a single controller.  They’re still separate caches, but this unification allows for something I’ve been wanting to implement since 0.2: Memcached support.

High-volume applications often run into the problem of limited memory across the various server nodes.  Solutions like Terracotta can help tremendously, but unfortunately this doesn’t solve everything.  One answer is to provide every node in the cluster with a shared, distributed, in-memory map of key / value pairs.  This allows the application to page data into cluster memory and retrieve it extremely quickly.  Memcached is effectively this.  Naturally, it would be very useful if an ORM could automatically make use of just such a distributed memory cache and thus share cached rows between nodes.  Hibernate has had this for a long time, and now ActiveObjects does as well.

By making use of the MemcachedCache class in the activeobjects-memcached module (separate from the main distribution), it is possible to point ActiveObjects at an existing Memcached cluster and cause it to store the value cache there, rather than in local memory.  Once the relevant call has been made to EntityManager (setting up the cache), all cached rows will be stored in Memcached and shared between every instance of ActiveObjects running in your cluster.

Caveats

Of course, nothing is perfect.  For the moment, the major issues is that the relations cache doesn’t work properly against Memcached.  I had originally planned to just release it running against the local RAM, but this causes issue with proper cache expiry.  As such, the relations cache is disabled when running ActiveObjects against Memcached.  This doesn’t really impair functionality other than forcing ActiveObjects to hit the database every time a relationship is queried.  Since this is what most ORMs do anyway, it’s not too horrible.

I plan to have this issue resolved in time for the 0.9 release.  The biggest problem is figuring out how to flatten the multi-index structure into a single map.  Once I can do that, rewriting things for Memcached should be a snap.  (btw, if anyone wants to help out with patches in this area, you’re more than welcome!)

Databases Galore

One of the major focuses of this release was testing on different platforms.  This process uncovered an embarrassing number of bugs and glitches in some of the supposedly “supported” databases.  With the exception of Oracle, every bug I’ve been able to identify has been fixed.  So if you’ve tried ActiveObjects in the past and run into problems on non-MySQL platforms, now would be the time to give it another shot!

Speaking of Oracle, we’ve made some tremendous progress toward full support of this ornery database.  To be fair, I’m not the one responsible.  One of our community members has been tirelessly submitting patches and running tests.  We didn’t have time to get things fully sorted for this release (so don’t try AO 0.8 with Oracle unless you’re willing to risk server meltdown), but you can expect dramatic improvements in 0.9.  Once these fixes are in place, we should be able to safely claim that we support most of the database market (in terms of product adoption).

Final Thoughts

This really is an amazing release, well worth the download.  Most of the core API has remained the same, so things should be basically plug-n-play for existing applications.  The user-facing API is now in freeze and (unless some serious bug arises) will experience no changes between now and 1.0.  If you’ve been considering trying ActiveObjects for your project, now would be an excellent time.  I can’t promise no bugs, but if you point out my idiocy, I’ll certainly work to fix it!

Oh, as an aside, ActiveObjects is now in a Maven2 repository.  (thanks to some POM reworking by Nathan Hamblen)  To make use of this somewhat dubious feature, add the java.net Maven2 repository to your POM and then insert the following dependency:

<dependency>
    <groupId>net.java.dev.activeobjects</groupId>
    <artifactId>activeobjects</artifactId>
    <version>0.8</version>
</dependency>

After that, the magic of Maven will take over.  Of course, you’ll need to add the dependency for the appropriate database driver and (optionally) connection pool.  Those dependencies are left as an exercise to the reader.  Note, activeobjects-memcached isn’t in Maven yet, but it will be for the 0.9 release (either that or integrated into the core, I haven’t decided yet).

I certainly hope you enjoy this latest release.  As always, feel free to drop by the users list if you have any questions or (more likely) uncover any problems.

• activeobjects-0.8.tar.gz
• activeobjects-0.8.zip

Just in time for the Christmas holidays, I give you ActiveObjects version 0.7!  This release sports a whole slew of minor performance tweaks and bugfixes, providing better reliability.  Also, we’ve greatly increased the number and scope of the unit tests running against the core.  This gives us more confidence that the code is indeed as stable as we hope.

Actually, the big ticket item for 0.7 is the introduction of polymorphic relations.  This feature is fully tested and stable, so I don’t anticipate it introducing any issues in the coming days as we narrow our focus to 1.0.  However, no feature is so well tested as to be certainly bug free under all circumstances.  I look forward to fixing any problems you may encounter with the library!

A few minor feature additions:

• Support for persisting enum values
• One-to-one relations
• EntityManager#flush(…) now also flushes the relations cache

Looking toward the road ahead, things are really starting to settle down a bit.  The main feature I’m looking to implement for 0.8 is extensible value caching, which should allow you to determine how and where the cached field values are stored.  The main use-case behind this is support for memcached, which has been requested a few times and has actually been on the agenda since 0.3.  With memcached driving the value cache, ActiveObjects should be almost perfectly scalable.

To that end, it’s also worth mentioning that I did some profiling (using the NetBeans Profiler) using a Wicket-based web application I’m developing on the side.  I’m happy to report that the overhead imposed by ActiveObjects is extremely minimal.  In each test, the bottleneck was in the PreparedStatement#execute method (in JDBC).  Since ActiveObjects generates very streamlined and natural queries, it seems that this is really the minimal execution time possible for such database access.  Extrapolating further from this data, and given that the JDBC exec time was avg 40ms, while the execution time in the ActiveObjects API was on the order of 1ms-5ms, we can safely say that ActiveObjects is less than 10% slower than using JDBC directly (compare that to the 50% overhead claimed by ActiveRecord).  This “estimate” doesn’t even take into account the many performance advantages offered by ORMs over hand-coded database access (such as relations caching, optimal query construction, etc).

Anyway, enough shameless boasting.  As Hibernate has often pointed out, ORM benchmarks are usually worse than useless.  The only benchmark which really matters is how well the ORM performs in your application.  Try it out, see what you think.

• activeobjects-0.7.tar.gz
• activeobjects-0.7.zip

The main obstacle in building an ORM is deciding how to map between a class hierarchy and a database table set.  While this may seem fairly clear cut in many situations, it’s unfortunately not so easy once you get into more complex issues like inheritance.

At a basic level, tables are classes.  It makes sense, right?  A class defines a structure which is instantiated and populated with data.  A table defines a structure into which data is inserted, one set per row.  Superficially, these two concepts sound more-or-less identical.  The differences creep in when you look at things like a class hierarchy.  Classes have the ability to inherit attributes from a superclass.  Thus, if class A inherits from class B, class A has everything that class B has (simplistically put). 

Unfortunately, database tables have no such capability.  You can’t define a table which has everything a “supertable” has.  Nor can you select data out of a “subtable” as if it were a supertable (polymorphism).  This lack of capability creates a disconnect between database structure and object-oriented class structure.

As with all of these “little differences” (between languages and databases), many solutions have been tried, none of them very successfully.  One of the most popular ways to solve the problem is to use separate tables for the supertable and subtable.  Then, whenever one SELECTs from the subtable, a JOIN is used to include the inherited row from the supertable in the result set.  In principle, it sounds right.  After all, this is how C++ handles inheritance.  However, in practice it becomes rather unwieldy.

To start with, JOINing every time you perform a simple SELECT is both inefficient and annoying.  Maintaining two separate rows is also a difficult problem to keep up with.  Inevitably, the data gets a bit out of sync and the whole thing falls apart.  Granted a good ORM will prevent this from happening in the first place, but ORMs are not the exclusive means for accessing database schemata.  Often times your schema will be in use not just by your application, but also by a web site, a demo utility and random DBAs who refuse to enter data in any way other than hand-written SQL.  In short, table inheritance using JOINs is clunky, unmaintainable and really messes you up down the line.

Another, more conservative way to map inheritance is inclining, where every subtable contains all of the fields which would be in a supertable.  Thus, if table A and B inherit from table C, table A and B are created with all of the fields that would have been in table C, and table C isn’t created at all.  Technically speaking, it’s not inheritance, just a slightly more centralized way to specify fields.  However, this technique is much more in line with how a database schema would normally work if designed by hand, and that’s what ORMs are supposed to simplify, right?

The ActiveObjects Approach

As you have have guessed from my comments, I really lean toward the inline strategy.  I think this works best in a practical environment and is far more maintainable down the line.  Thankfully, this strategy is considerably easier to implement than JOINing.  Syntax-wise, inheritance is used like this:

public interface Person extends Entity {
    public String getFirstName();
    public void setFirstName(String firstName);
 
    public String getLastName();
    public void setLastName(String lastName);
}
 
public interface Employee extends Person {
    public String getTitle();
    public void setTitle(String title);
 
    public short getHourlyRate();
    public void setHourlyRate(short rate);
 
    public Manager getManager();
    public void setManager(Manager manager);
}
 
public interface Manager extends Person {
    @OneToMany
    public Employee[] getPeons();
 
    public long getSalary();
    public void setSalary(long salary);
}

A fairly standard, object-oriented interface hierarchy, right?  Logically it makes sense.  This is how ActiveObjects would represent such a hierarchy in the database:

So basically inheritance in ActiveObjects is just a mechanism which allows the definition of common fields in a single place.  There’s no real polymorphism (that is to say, you can’t INSERT an employee where a person is required, since there is no “people” table).  The idea is: keep it simple stupid.  In fact, ActiveObjects would happily generate a table to correspond to Person if we asked it to, since as far as it’s concerned it’s just an entity like any other.

Of course, there are quite a few serious disadvantages to this approach.  For one thing, without polymorphism or actually centralizing the fields in a common table, inheritance is just an illusion.  From a database standpoint, the tables aren’t really related in any interesting way.  You’re still duplicating fields in the database (though not duplicating data), though since the duplication is all handled by the ORM, it’s not too much of an issue.

For me though, it all comes back to the decision I made to not make the schema over-complicated.  To ensure that no matter what the object hierarchy, it could be represented in the database (assuming it’s valid) without undue weirdness.  Keep the schema simple, make it easy to do stuff outside of the ORM.  The ORM should be a liberating too, not a constraining one.

I realize not everyone agrees with this particular style of table inheritance, so maybe at some point in the future ActiveObjects will allow configuration in this area.  But for the moment, the uncomplicated schema is the way to go.  :-)  Enjoy!