Java IoT Authors: Zakia Bouachraoui, Pat Romanski, Elizabeth White, Liz McMillan, Yeshim Deniz

Related Topics: Java IoT

Java IoT: Article

Coming Out of the JDO Closet

Coming Out of the JDO Closet

As part of building the infrastructures for a large J2EE project, we've spent the last few months designing and implementing a JDO-based O/R persistency framework. This framework provides our business logic programmers with the following features: an interface-based abstract view of the data-layer with full OO semantics, zero-need knowledge of the object-to-database mapping details, "delta" support, and more. This article presents the persistency framework that we've built on top of JDO, and offers a commentary on the current advantages and shortcomings based on our experience with the JDO specification.

Choosing an Object Persistency Layer
Choosing the right object persistency layer is a major milestone in almost every J2EE project. Given the state of today's market, you should follow the "buy, don't reinvent" rule (assuming your object persistency demands haven't reached the "you must be dreaming" line). But though today's component server-side frameworks surely make the architect/developer's life simpler, there are still some issues to consider when designing your data layer, namely performance, sound API toward the business logic, and standardization. Limiting ourselves to the object-relational mapping market per se, the choice is between:

  • Using a proprietary yet powerful and proven tool from one of the veteran O/R mapping vendors such as TopLink (www.webgain.com), CocoBase (www.thoughtinc.com), or Castor (http://castor.exolab.org)
  • Relying on one of the newer yet standard object persistence Java specifications: EJB entity beans (ThoughtInc., WebGain, and others are offering entity beans-based solutions as well) or Java Data Objects (JDO)

    While all the above vendors offer (more or less) the features listed in Table 1, since we're especially zealous about standardization and vendor independence, we decided on JDO, a strategic decision. Left with choosing between entity beans and Java Data Objects (it was about a year ago that we made this decision), we went again for the second choice. Entity beans seemed rather too bulky and heavyweight for our purposes, obscuring the simple Java model while providing (unnecessarily in our case) factory-wrapped network interfaces and limited inheritance capabilities. We felt that JDO provided a simpler, cleaner API with pure OO syntax. Still another advantage of JDO lies in its relative lightness. (Note that by saying that we're certainly not trying to slight EJB entity beans. Indeed, the initial doubts about their true usefulness seem to have died out, especially after the improvements introduced in the EJB 2.0 spec.)

    Short Overview of JDO
    In the remainder of this article we'll share the experience gained while building an O/R persistency framework on top of JDO, designed and implemented in a large organization. Let's start by briefly summarizing the main features of JDO.

    The JDO specification provides a flexible method for handling persistent objects. It originates from JSR-000012, approved in July 1999 - those not yet familiar with the spec should consult the "Resources" section of this article for the appropriate URL - and was designed to integrate with existing frameworks (most notably J2EE), offering more complete transactional facilities. Indeed, session beans can directly manipulate JDO persistent instances, while entity beans can delegate business logic and persistence management to JDO classes. In our project we took the first approach.

    Working with JDO, classes that can be made persistent implement the PersistenceCapable interface, but the class author need not explicitly declare this in the code. All a developer needs to do is to provide a companion XML file, which contains metadata about the class (e.g., the names of the fields that are to be made persistent). This XML metadata is used by a "class enhancer" to generate a persistence-capable copy of the original class. (JDO ensures binary compatibility between persistence-capable classes, even if generated by different vendors' enhancers.)

    As a result, all the details of the class's actual persistence management happen behind the scenes. Each JDO instance of a persistent class has its own unique identity in the datastore and hence can be shared between different applications concurrently. When a reference is followed from one persistent class to another, the JDO implementation transparently instantiates it as necessary into the JVM. When fields of a persistent object are accessed or modified, the JDO implementation transparently marks them for change in the datastore. Almost all user-defined classes can be made persistent. While this excludes most system classes (like those in java.io or java.net), some are handled specially by a JDO implementation. For instance, the java.util.Collection interface is supported, as are immutable classes like Integer, Float, String, and Date - in addition to arrays.

    The PersistenceManager is the primary interface for JDO-aware applications. It enables users to store and fetch persistent objects from the datastore and allows users to perform synchronized transactions and queries on those objects. JDO uses a neutral query syntax similar to OQL (Object Query Language).

    Pros and Cons of JDO
    The JDO specification has honestly won its territory, being a compact, smart, and truly pure object-oriented persistency specification. However, being a newcomer also means being young, suffering from childhood fallacies, and, sadly enough, having older enemies.

    JDO touts as an advantage its back-end and datastore neutrality. This is indeed a virtue of the specification. But on the other hand it entirely ignores various important O/R issues such as object locking paradigm and O/R mapping instructions (e.g., schema structure, referential integrity, constraints, and so on). That is, JDO can be categorized as a general object persistence framework, with no special interest in being "baptized" as a full-blown O/R persistence specification. Furnishing no O/R mapping guidelines (let alone standards) and leaving this important role to the different product vendors makes the application developer's life somewhat confusing. For instance, Kodo (www.solarmetric.com), ObjectFusion (www.prismtech.com), and IntelliBO (www.signsoft.com) don't use database-referential constraints. But other vendors may choose to do so. And while IntelliBO lets you define code mappings (translation of pre-defined values into more compact database representations), it requires you to use proprietary XML tags to do it.

    Aside from its current "O/R uncertainties," there's still some uncertainty surrounding JDO. Sun, by not making a very good job of telling us when to use EJB entity beans and when to use Java Data Objects, has added to the overall confusion. Furthermore, the current JDO vendors tend to be "niche vendors" in the Java industry. A major step toward the acceptance of JDO would happen when the "big guns" like IBM, Oracle, and BEA begin to follow suit and join them - something that's not yet happened. Finally, JDO has also managed to win itself some sour enemies. The infamous duel on TheServerSide.com between JDO specification lead Craig Russel and Thought Inc. CTO Ward Mullins is just one example (see "Resources").

    Balancing the pros and cons of JDO, we believe that in spite of its current shortcomings JDO will mature into a pervasive first-class object persistency solution. An up-to-date portrait of the JDO market can be found on the JDO official site (see "Resources").

    A Simple Approach to Working with JDO
    As can be seen in our UML class diagram (see Figure 1), the superclass of the entities in our example is LifeForm. Man, Dog, and Flea are all LifeForms, where Dog also references a collection of Fleas and holds a reference to his best friend, Man. Name and Age are aggregated types in a LifeForm where an Age is always positive and a Name consists entirely of English letters. Note that Name and Age are "primitive wrappers" (of a String and Integer, respectively) supplying us with better abstraction, type safety, and basic validation in the type itself (validated during construction).

    The Java code pertaining to the UML specification is presented in Listing 1. Examining the syntax of class Age, a question arises: Is the use of an Integer necessary? Can't we simply use an int? Using the latter would, curiously enough, make it impossible to create an ageless life form, since there's no Java null value for int, thus no datastore NULL value for age. Of course, you can allocate a prespecified int value representing null (e.g., -1) but this would make writing your queries a tad peculiar.

    Writing the Java code, our next step is to run the vendor's JDO enhancer tool on the classes. Using the XML metadata file supplied in Listing 2, the enhancer will make these classes PersistenceCapable by applying direct bytecode manipulation. Some enhancers will automatically generate DD SQL statements (or directly create the database schema given a JDBC connection) as part of the enhancing process, while other vendors will politely ask you to run their schema-building tool. Notwithstanding, some vendors let you capture a preexisting schema and declare the O/R mappings yourself.

    Robust OO models should follow the real world. Thus, when our dog passes away, its fleas should gracefully pass along with it. The dog's best friend, however, won't (we hope). In JDO terms (see Listing 2) Dog and Man are First Class Objects (FCOs) - i.e., they possess an independent life cycle. On the other hand, Fleas are embedded in Dog as Second Class Objects (SCOs). SCOs have no JDO identity of their own and are stored/deleted from the datastore as part of their owning FCO. To complete the picture, note that Age and Name are embedded as SCOs in LifeForm.

    Finally, some business logic code using our data objects can be found in Listing 3. Observe that in deleteDog(Dog) it's enough to explicitly delete the Dog object. Its fleas are implicitly deleted from the datastore along with its Name and Age objects.

    Performance Analysis (Round 1)
    Albeit simple, the naive implementation results in poor runtime performance. The database tables generated by a typical schema tool are depicted in Figure 2. The exact table and field names are, not surprisingly, vendor-proprietary as are the referential constraints imposed on the tables. Nevertheless, they would follow the general structure presented here. Looking at Figure 2 we can see that fetching a Dog object results in a minimum of four fetches and three joins (assuming no fleas, no friend, but proper identification). Fetching a LifeForm translates to three fetches and two joins. This is highly inefficient.

    You're probably curious why Name and Age aren't represented as columns in LifeFormT, even though they've been declared as "embedded" SCOs. Well, in order to be consistent with the JDO specification, an O/R vendor must make the life cycle of Name and Age dependent on their owner, but there are no restrictions regarding the actual O/R mapping. All the vendors we've tested have failed to exploit the "embed" hint into a smart datastore embedding.

    It should be clear that the issue that causes a problem isn't the creation of the NameT and AgeT tables. Since a vendor's knowledge of the application semantics is limited to that defined in the metadata XML file, it would rightfully choose to generate the NameT and AgeT tables. Indeed, Name and Age could be used as an FCO or as elements of Collection in another part of the application. It's the lack of O/R embedding capabilities in cases analogous to LifeFormT that causes the problem.

    Introducing Persistency Framework Based on JDO
    Looking back at the naive approach above, you can spot a three-layered persistency architecture consisting of the JDO enhanced classes, the JDO implementation, and the datastore

    Our JDO-based persistency framework introduces two additional layers, namely the interface layer and the wrapper layer, thereby attaining the following added-value goals:

  • A: Interface-based abstract view of the data layer
  • B: Tuned O/R mapping providing superior performance
  • C: "Delta" support and Delta-driven semantics (described below)
  • D: Change tracking during the transaction

    Figure 3 shows the architecture of our framework. A vertical view of the framework reveals two major components: data entities (generated from the repository) and persistency management (an adapter, delegating the application persistency instructions to the concrete JDO implementation - e.g., persistence by reachability, JDOQL query engine, integration with EJB container, and so on). The horizontal view unveils the framework's layers: interface layer, wrapper layer, JDO layer, JDO implementation, and datastore.

    We won't go any further into the persistency management component, since it's mainly just an adapter. Readers eager to learn about persistency features are wholeheartedly directed to the JDO specification.

    Figure 4 presents the life form example, refactored to fit our framework. Relations connecting different layers are, as you can see, framework driven, while relations within a layer are OO-model driven. In fact, each layer encapsulates a "replica" of the life form model, using its own abstraction level to do so. Note that whereas Dog, Flea, and Man are explicitly represented throughout the layers, Name and Age are missing from the JDO layer. The reason for this will be made clear later.

    Interface Layer
    It's hard to overstate the importance of using a metadata repository within any large J2EE project. Our project uses one ("home made") in which we define our data entities, their fields, properties, and much more. The metadata repository is the source for our active code-generation tools, which generate our entities' interfaces and implementations and also much of the framework's generic code.

    The interface layer (see Listing 4) provides a façade toward the business logic, allowing it to work in terms of abstract interfaces (Goal A). The clear separation between interfaces and implementing classes is not just a good programming practice, but also allows for a more loosely coupled compilation model. Moreover, as our data entities can implement multiple interfaces, we're able to provide different business logic modules with personalized local views of the data and enable the creation of generic mechanisms/algorithms operating on interfaces. For instance, one could declare Dog and Man to implement a Trainable interface. Doing that, it's possible to write a generic algorithm working in terms of trainable life forms.

    Wrapper Layer
    The wrapper layer is the middleware between the abstract interfaces presented to the business logic and the persisted objects actually stored in the database. It provides the concrete implementation of the interfaces defined in the interface layer through a symbiotic relationship with the JDO-layer classes (the enhanced classes), which act as the actual value holders. The wrapper layer's chief responsibilities focus on attaining the remaining goals, namely:

  • Concrete implementation of the interface layer, employing smart object embedding (Goal B)
  • Delta support (Goal C)
  • Change tracking during the transaction (Goal D)

    Wrapper objects make use of OO mechanisms such as reflection, dynamic invocation, dynamic proxy, lazy wrappings, and more. Their concrete code, being somewhat intricate (yet generic and actively generated), is not listed.

    Object Embedding
    In Figure 5 (a subdiagram of Figure 4) you can see the symbiotic relationship between the wrapper instances and the JDO-layer instances. The business model views a Man instance as an entity composed of two wrapper fields, Name and Age, that respectively reference a String object and an Integer object. The JDO model, being less abstract - in order to be efficiently persisted (Goal B) - has no explicit Name or Age objects. Instead, a ManJdo instance shares the same String/Integer instances with its ManImpl wrapper. The following code fragment (line numbers correspond to tag numbers in Figure 6) illustrates this sharing. The code creates a man and assigns it a name. Figure 6 illustrates the procedure, marking each element with the code line that created it. Note that, when updated, the ManImpl wrapper internally synchronizes its ManJdo accordingly:

    1 public static void main(String[] args)
    2 Man man = new ManImpl();
    // implicit creation of ManJdo
    3 Name name = new NameImpl("John Smith");
    4 man.setName(name);
    // implicit update of ManJdo
    5 }
    Not surprisingly, we call this feature embedding, since the persisted ManJdo class actually embeds fields pertaining to several wrapper objects (e.g., NameImpl, AgeImpl). In fact, we use our metadata repository to define which fields in an entity are embedded and which are simply referenced. Our framework also supports (to some extent) the embedding of "complex" types, themselves containing multiple types, though the concrete implementation details would be out of scope. For instance, an Address composed of street, city, zip code, state, and country can be declared embedded in a Person. And while the business logic would work in terms of person and address interfaces, the only class to be actually persisted would be PersonJdo.

    Delta Support and Change Tracking
    Our project's functional requirements turn delta support (Goal C) and change tracking (Goal D) into important technical mechanisms. However, given that they're not the principal focal point of this article, we offer the following concise review. (A thorough investigation of these mechanisms would require an article of its own.)

    Delta Support
    Supporting remote clients that work on local copies of data entities, we let our client-side code build and transmit an "entity-delta" (the entity subgraph defined only by those fields that have actually been made dirty) to the server. The server-side business logic can now pursue "delta-driven" (i.e., "change-driven") semantics. So, for example, a typical session bean method would fetch the matching persistent entity, apply the delta, or perform some additional logic depending on the delta values, and commit (see Listing 5 for an example of this).

    As Listing 5 shows, each entity interface (implementation) has a matching - actively generated - delta interface (implementation) exposing only the getter (accessor) methods. This is both type-safe and logically firm.

    Our entities (in fact, the wrapper objects) support the creation and application of deltas by continuously keeping track of the changes made on them during the transaction. The delta-build and delta-apply code is entity-specific - i.e., it's actively generated along with each data entity. We chose this method over a general mechanism that uses reflection due to complexity, maintainability, and speed issues.

    Change Tracking
    Our server-side framework often needs to be handed the entities changed so far (created, updated, or deleted) during the transaction. It needs these in order to perform certain generic functional mechanisms. The persistency framework helps out by supplying a ChangeTracker component, which cooperates with the wrapper layer in tracking these entities (JDO provides instance callbacks that help maintain this information).

    JDO and Database Layers
    The JDO layer is made up of the classes that are actually persisted (enhanced), i.e., the JDO-suffixed classes (see Figure 4). JDO regards dates, strings, and Java primitive wrappers like Integer and Float as fundamental SCOs. Consequently, the typical vendor will choose to embed such fields in their owning entity's table. In our case, since the enhanced LifeFormJDO contains fields referencing a String and an Integer, the correspondent LifeFormT table directly contains the Name and Age columns. We can thus see that, due to the embeddings performed by their wrappers, the JDO layer classes - and so too the conforming database tables - are better suited to efficient O/R mappings. The database schema depicted in Figure 7 conforms to the JDO layer classes of the life form example.

    Performance Analysis (Round 2)
    If you compare Figure 7 with Figure 2, you can see that the table structure has improved since Round 1. It has become more compact, resulting in faster, more efficient runtime behavior. Fetching a LifeForm requires only a single fetch as opposed to three fetches plus two joins, as required by the naive implementation. When looking at the toy example, the improvements might seem minor. But the framework's smart-embedding capabilities become much more important when you're confronted with large data sets and a variety of entity and field types, as is the case with e-businesses.

    Summing up, the JDO specification is a promising newcomer in the object persistency arena, even in light of its current shortcomings. Its pure semantics and light model seamlessly integrate within the J2EE environment, making it a viable candidate for small as well as large J2EE projects. However, on projects that involve a rich set of interfaces, classes, and types, in addition to large runtime data sets, developers should take care when building their persistency layer around a JDO product to ensure abstraction and performance.

    In this article we've tried to contribute from our experience of doing just that by presenting the JDO-based framework we've built for our own project. We demonstrated how our framework attains the important added-value goals of abstraction and performance, while providing additional capabilities such as code generation, delta-oriented business logics, and more.


  • Java Data Objects Public Access (JDO official site contains JDO specification): http://access1.sun.com/jdo
  • Rettig, M.J., with Fowler, M. (2001). Reflection vs Code Generation: www.javaworld.com/javaworld/jw-11-2001/jw-1102-codegen_p.html
  • J2EE specification: http://java.sun.com/j2ee/
  • The infamous "Java Data Objects vs Entity Beans" duel: www.theserverside.com/discussion/thread.jsp?thread_id=771
  • More Stories By Yaron Telem

    Yaron Telem and Shay Litvak are IT/J2EE architects, working for "Mamdas", involved in the construction of large enterprise systems in the last 5 years. Yaron received his B.Sc. and M.Sc. in Computer Science and Mathematics from Tel-Aviv University. Shay received his B.Sc. in Computer Science from the Israel Institute of Technology.

    Comments (0)

    Share your thoughts on this story.

    Add your comment
    You must be signed in to add a comment. Sign-in | Register

    In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.

    IoT & Smart Cities Stories
    Moroccanoil®, the global leader in oil-infused beauty, is thrilled to announce the NEW Moroccanoil Color Depositing Masks, a collection of dual-benefit hair masks that deposit pure pigments while providing the treatment benefits of a deep conditioning mask. The collection consists of seven curated shades for commitment-free, beautifully-colored hair that looks and feels healthy.
    The textured-hair category is inarguably the hottest in the haircare space today. This has been driven by the proliferation of founder brands started by curly and coily consumers and savvy consumers who increasingly want products specifically for their texture type. This trend is underscored by the latest insights from NaturallyCurly's 2018 TextureTrends report, released today. According to the 2018 TextureTrends Report, more than 80 percent of women with curly and coily hair say they purcha...
    The textured-hair category is inarguably the hottest in the haircare space today. This has been driven by the proliferation of founder brands started by curly and coily consumers and savvy consumers who increasingly want products specifically for their texture type. This trend is underscored by the latest insights from NaturallyCurly's 2018 TextureTrends report, released today. According to the 2018 TextureTrends Report, more than 80 percent of women with curly and coily hair say they purcha...
    We all love the many benefits of natural plant oils, used as a deap treatment before shampooing, at home or at the beach, but is there an all-in-one solution for everyday intensive nutrition and modern styling?I am passionate about the benefits of natural extracts with tried-and-tested results, which I have used to develop my own brand (lemon for its acid ph, wheat germ for its fortifying action…). I wanted a product which combined caring and styling effects, and which could be used after shampo...
    The platform combines the strengths of Singtel's extensive, intelligent network capabilities with Microsoft's cloud expertise to create a unique solution that sets new standards for IoT applications," said Mr Diomedes Kastanis, Head of IoT at Singtel. "Our solution provides speed, transparency and flexibility, paving the way for a more pervasive use of IoT to accelerate enterprises' digitalisation efforts. AI-powered intelligent connectivity over Microsoft Azure will be the fastest connected pat...
    There are many examples of disruption in consumer space – Uber disrupting the cab industry, Airbnb disrupting the hospitality industry and so on; but have you wondered who is disrupting support and operations? AISERA helps make businesses and customers successful by offering consumer-like user experience for support and operations. We have built the world’s first AI-driven IT / HR / Cloud / Customer Support and Operations solution.
    Codete accelerates their clients growth through technological expertise and experience. Codite team works with organizations to meet the challenges that digitalization presents. Their clients include digital start-ups as well as established enterprises in the IT industry. To stay competitive in a highly innovative IT industry, strong R&D departments and bold spin-off initiatives is a must. Codete Data Science and Software Architects teams help corporate clients to stay up to date with the mod...
    At CloudEXPO Silicon Valley, June 24-26, 2019, Digital Transformation (DX) is a major focus with expanded DevOpsSUMMIT and FinTechEXPO programs within the DXWorldEXPO agenda. Successful transformation requires a laser focus on being data-driven and on using all the tools available that enable transformation if they plan to survive over the long term. A total of 88% of Fortune 500 companies from a generation ago are now out of business. Only 12% still survive. Similar percentages are found throug...
    Druva is the global leader in Cloud Data Protection and Management, delivering the industry's first data management-as-a-service solution that aggregates data from endpoints, servers and cloud applications and leverages the public cloud to offer a single pane of glass to enable data protection, governance and intelligence-dramatically increasing the availability and visibility of business critical information, while reducing the risk, cost and complexity of managing and protecting it. Druva's...
    BMC has unmatched experience in IT management, supporting 92 of the Forbes Global 100, and earning recognition as an ITSM Gartner Magic Quadrant Leader for five years running. Our solutions offer speed, agility, and efficiency to tackle business challenges in the areas of service management, automation, operations, and the mainframe.