In late 2002, Javier Soltero, Doug MacEachern, Ryan Morgan, Jon Travis, and I (the eventual co-founders of Hyperic) began designing and architecting an application management system that was to become Hyperic HQ. We wanted it to be the management system that bridged the gap between open source and commercial applications and, furthermore, we wanted it to use, be built on, and deployed to the applications and operating systems that we managed.

To achieve that goal, we set out to implement on standards using a cross-platform language. Thus the decision to use Java was pretty clear, as the abstraction of the system-specific runtime freed us from having to figure out native APIs and implement different paths to achieve the desired functionality. We chose JBoss (then mildly popular) as our J2EE application server. When it came to our data persistence strategy, we knew we wanted to stay away from developing our own (Javier and I had already done that before for an Apache configuration management project). Database portability was important, as we had experimented with and implemented PointBase, Cloudscape, and InstantDB for that Apache configuration management project, and knew we would be moving on from databases like bad relationships. Much like bad relationships, you only knew they were over after you had put a lot of effort into them. By 2002, EJB2 had become a standard, and Hibernate was very young and unproven. Compared to EJB 1.0, EJB2 was, in our minds, ridiculously easy to implement. We would use XDoclet to annotate the code, and the whole thing would practically write itself!

We set out to implement to EJB2. True, we were able to quickly map out our object model, and the amount of database-specific SQL was kept to a minimum. We wrote our own tools that functioned as Ant tasks and allowed us to create and populate database schemas for different databases that we supported. However, the runtime was handled by container-managed persistence (CMP); we had no need for bean-managed persistence (BMP). Pretty quickly we ran into some issues such as needing to specify an "order by" clause. While EJBQL did not support "order by," JBossQL did. We thought, "We are only deploying in JBoss at the moment and we will continue to maintain compatibility by defining both EJBQL and JBossQL." Everywhere an "order by" was needed, we had defined both "@ejb:finder" and "@jboss:query" XDoclet tags. We were diligent about it for a while, but then pretty soon we'd forget to fix queries in "@ejb:finder" when we fixed them in "@jboss:query". Then, after more time had passed, we would just plain leave the "@ejb:finder" empty sometimes, especially when we needed outer joins, grouping, or other SQL queries. Since EJBQL did not support the full set of standard SQL syntax, we began to add "@jboss:declared-sql" tags, again a JBoss-specific declaration.

At least we had the CMPs working as we needed them. We were getting caching of CMPs, so we were gaining some optimization. However, it was clear that the performance of CMPs would not be acceptable in some areas. In our case, we collected a voluminous amount of metric data. No matter how much we tried to configure JBoss to not lock pessimistically, we could not get HQ to work with metric data as CMPs without locking up everywhere. We were resigned to using SQL statements and JDBC to access the metric data directly. Once we went down that route, it was easy to decide to follow the same pattern elsewhere in the product. However, since we eventually settled on supporting PostgreSQL and Oracle, these code paths would sometimes require database-specific SQL to get the right behavior. Speaking of pessimistic locks, half of our issues became embroiled in transaction demarcation to avoid the dreaded TransactionRolledbackLocalException (the telltale sign of transaction deadlocks). We had to mark every API as transaction REQUIRED, REQUIRESNEW, SUPPORTS, and NOTSUPPORTED through trial and error. We had been pulling our hair out for the past few years over transaction issues. We implemented our own caching to get around some of these problems, even forking our own version of XDoclet to inject caching behavior into EJB2. There were other issues such as paging of the result set, for which there's absolutely no support in EJB2.

Don't get me wrong, we were able to get this far with EJB2. We are pretty proud of what we have put together in Hyperic HQ, and we accomplished what we set out to do. However, it's 2006 and the landscape is very different. Hibernate has emerged as the runaway preferred object relationship mapping (ORM) tool, and for good reasons, too. With the problems that we had faced with EJB2 and the difficulties with new feature implementation, we knew that the right thing to do was to move forward to Hibernate. However, the task seemed daunting, especially since we had over 80 entities and many tweaks in our application to make it work just right. Frankly, I just didn't think we could ever do it in a reasonable amount of time. However, in the fall of 2006 we bit the bullet and just did it.

It was not the gradual migration that we thought it would be. No, we converted wholesale to Hibernate, and we did it fairly efficiently as well. The change occurred between versions 2.7.6 and 3.0 of Hyperic HQ, with the bulk of the work occurring in just about three weeks. The whole team definitely pulled together for this effort. In my forthcoming articles, I will detail how we went about the conversion and the patterns that we adopted. Hopefully it will benefit others who are in the same boat as we were. We knew that we were in a bad relationship with EJB2, and that it was time to move on.