Welcome!

Java IoT Authors: Derek Weeks, Elizabeth White, Jyoti Bansal, Dana Gardner, Liz McMillan

Related Topics: Java IoT

Java IoT: Article

Optimizing Database Performance in J2EE Applications

Optimizing Database Performance in J2EE Applications

The Java 2 Platform, Enterprise Edition (J2EE), is the platform of choice for implementing scalable and reliable enterprise applications from reusable components. But Java developers building enterprise-class J2EE applications face a quandary.

The object paradigm has proven ideal for modeling a wide variety of real-world scenarios. However, finding a Java-compatible data repository optimized for such applications has become a stumbling block. While object database management systems (ODBMSs) provide the convenience of transparent persistence of Java objects, their client-centric architecture has not scaled well in enterprise environments. Relational database management systems (RDBMSs) do scale well, but map each object to a two-dimensional relational table. The increased overhead can reduce application performance to a crawl.

This article discusses the limits of using these two types of databases with Java and suggests a better alternative for J2EE - a hybrid database that combines the best features of both. Hybrid databases share with ODBMSs the ability to map data stored in back-end databases directly into an implementation-neutral Java representation. As with relational systems, hybrid databases can scale to meet the performance requirements of an enterprise-class J2EE application.

ODBMSs: The Hidden Headache of Transparent Persistence
Over the years, finding a database that's both Java-compatible and scalable enough for enterprise-class J2EE applications has not been easy. Ideally, a Java-compatible database should store Java objects whose classes have been declared "persistent-capable" and can be manipulated seamlessly by the Java language.

That has been the promise of ODBMSs, which made their appearance in the mid-1990s as a solution designed specifically for objects and thus better suited for object development. With ODBMSs, Java developers can define persistent Java classes in the same way transient Java classes are defined in the application.

An apparent advantage of pure object databases is the implementation of transparent persistence that automates the process of mapping persistent data objects into the data repository. With transparent persistence, you don't even have to alter your existing Java classes to describe the persistent data that's permanently stored in the database (see Listing 1). That means you don't have to decide ahead of time, usually during the design phase, which objects to include and exclude from the database.

Adding a new customer order into the database is as simple as creating a new object in Java. Persistent-capable objects are transient until attached to a persistent manager or to other persistent objects.

This convenience quickly becomes a nightmare, however, when developing scalable enterprise-class applications. In a typical application, objects are highly interconnected, and it's very important to know precisely which objects have been stored with the database and which have not. Consider an e-commerce application in which products, customers, and orders are all linked together (see Figure 1). The object model naturally captures the interrelationships of real-world applications. With transparent persistence, you wind up loading an entire closure of objects even though you want to access only a single object (see Figure 2). While the programmer wants to load only one customer, the closure of instances reachable from this object recursively loads a large portion of the database. Loading unneeded data in the Java VM limits concurrency and scalability.

A simple customer query, for example, could also lock pending orders and products purchased, even though this data was not requested and will remain unchanged. Such "overloading" is not a noticeable problem within a standalone environment that manipulates a small amount of data. However, in an enterprise-class, multiuser, transaction-intensive application, large portions of data get locked and instantiated, limiting concurrency and scalability.

During the pilot phase of development, performance is usually acceptable since the system is not running under heavy computational loads. But with wider deployment and more users, transaction rates can slow unacceptably as a massive amount of data - much of it unneeded - fills the pipeline. In the end, transparent persistence leads to a performance black hole, requiring substantial work to improve scalability, increase concurrency, and reduce network traffic. To gain sufficient control over which objects stay persistent and which do not, the ODBMS's transparent persistence mechanism must be bypassed and the ODBMS's proprietary API used instead. Developers must master the ODBMS's proprietary API and then invest the many hours required for the complex, trial-and-error process, which has no guarantee of success.

The hard lesson, often learned at company expense, is that the ODBMS used to validate a pilot application must be replaced by a relational database when the system goes into production. That's the programming equivalent of a heart transplant, setting development schedules back by months. As we will see, relational databases bring their own set of problems in terms of overhead, and can require 25-50% more Java code.

RDBMSs: The Frustration of Object-Relational Mapping
Java developers are hindered by relational databases; however, RDBMSs do have two major advantages: a long, successful track record of deployment in scalable, transaction-processing systems and a standard language, SQL. While the relational model works well enough in banking applications where the row-and-column model reflects the two-dimensional world of ledgers and spreadsheets, it has proven more limited in tracking highly interconnected information. Relationship navigation commonly used in J2EE applications requires extensive use of multitable joins. But joins are computationally intensive, and each join is computed at runtime to link information on-the-fly (see Listing 2). Reconstructing an order object with its line items from row-and-column tables requires two SQL queries and much coding. The same operation in an object database would require only one call. Moreover, relational systems require the rebuilding of relationships between objects each time they're accessed, substantially impacting performance.

In today's economy where business intelligence is key, the Java object model provides a more powerful mechanism for capturing real-world relationships and concept commonalities. In the relational model the relationships disappear and are replaced by primary keys; foreign keys, columns, and indexes; and often by intermediate tables (see Figure 3).

In response to the demands from object developers, relational vendors have extended the relational model to support objects, much the way C++ was an object extension of C. But just as C programmers did not fully embrace C++, Java programmers have remained skeptical of object extensions to what is clearly not an object-oriented environment.

The underlying model of object-relational databases remains the same: rows and columns. As a result, the simplicity of the object model vanishes because classes, inheritance, and relationships must be mapped into tables - a structure ill-suited to the task. Even a simple many-to-many relationship between two classes must be expressed using intermediate tables, with two associated indexes. Therefore, a cleanly designed Java application translated through the normalization process results in a thicket of tables that must be recombined whenever an object is called by the application. The process adds significant load, especially when executing extensive table joins.

To solve the problem of mapping objects into relational databases, a number of OR mapping tools have been created. While these tools do make it easier to develop Java applications that use relational databases, they don't eliminate the underlying RDBMS problems of code complexity and poor performance.

Both database technologies have limitations for Java programming. A pure object database makes sense in a standalone environment in which concurrency and network traffic are not issues. Relational databases, while accommodating transaction-processing loads, merely simulate a true object environment.

Hybrid Databases: The Best of Both Worlds
Hybrid databases represent the best of both worlds: the ability to map objects from Java directly to the database with the support of a standard query language (SQL-99) and the scalable, enterprise capabilities implemented in relational database products. Designed from the ground up as a database server for objects, hybrid databases directly map the object model of Java as well as other object programming languages. Because the database object model matches perfectly with Java, you can freely and easily define the database classes that describe real-world scenarios.

Unlike an RDBMS, a hybrid database preserves the original Java data model. For example, a single class and two subclasses represent customers, consumers, and business customers, respectively. No tables are mapped back into Java objects; no translation of any kind is needed. Unlike an ODBMS, a hybrid database enforces a layered design of the persistent classes. The operations to manipulate objects are explicit, enabling you to keep tight control over the data that's locked and instantiated in the JVM, seamlessly improving the application's scalability.

Hybrid databases eliminate the mismatch between the Java and database environments, while still maintaining the scalability of server-side processing, such as relational systems. Within the J2EE environment, you manipulate Java objects representing a proxy to the object in the database by means of object-to-object mapping. The proxy objects are pure Java classes that map to those of the database schema (see Listing 3). With a hybrid database, the code stays compact and object-based (as in Listing 1), providing the same benefit as a first-generation ODBMS. Hybrid databases don't require any of the special compilation tricks or postprocessing byte code manipulations of ODBMSs - both of which make it hard to identify the root cause of performance degradation.

In a typical application, classes are highly interconnected, and the graph of instances can include large portions of the database. Therefore, controlling object-locking effectively, always a challenge in enterprise-class J2EE applications, is crucial to controlling the instantiation of Java objects in the JVM. To build scalable applications, data-intensive processing needs to take place where the data sits on the server, not on the client, further reducing locking contention as well as network traffic and taking advantage of the faster processing speeds of many server architectures.

Like RDBMSs, hybrid databases support the SQL-99 syntax. While SQL queries are relational in their syntax, they take advantage of the object paradigm by supporting inheritance, polymorphism, and true navigation. Furthermore, the query processing takes place on the server to enforce security and achieve performance. Consider a broad query of two classes of customers: business and consumer. The query is issued from the client, executed on the server, with selected objects from each class retrieved to the client.

This approach gives developers full access to Java objects through JDBC without having to learn a proprietary API (see Listing 4). In this listing, two customer subclasses, Consumer and Business, share properties from the parent Customer class while maintaining properties of their own. A query to locate "good customers" can combine criteria - bonus miles for home consumers, a high credit line for businesses - pulling the information simultaneously from both subclasses. Unlike an RDBMS, a hybrid database returns Java objects through JDBC and natively supports inheritance.

While developers still benefit from the power of expression and performance of SQL queries, these queries eliminate the object-relational mapping layer to reduce source code by 25-50% and improve application performance.

Unlike first-generation ODBMSs, hybrid databases can be accessed through JDBC and ODBC drivers, both of which support the SQL-99 language, thereby taking advantage of in-house SQL expertise. Support for ODBC and JDBC drivers also allows IT staff to use off-the-shelf database tools without having to master SQL.

First Major Optimization: Keep It Simple
Building enterprise-class J2EE applications with a hybrid database is straightforward. Here are some considerations to make the process even smoother:

  • Carefully define the object model of your persistent classes, reflecting the business model as closely as possible. That's common sense in an object environment, but is even more crucial in database applications because the way you define the model greatly impacts system performance.

    Defining the right level of granularity for your objects has a big payoff in terms of transaction rate because only the specific queried data gets locked.

  • Avoid cross-referencing persistent and transient objects as transient information can access persistent information, but not the other way around. Doing so makes the application much more complex to manage since the persistent objects loaded from the database may need to be linked to transient information that's not yet available. While a callback can also be used, it unnecessarily complicates program flow and can usually be avoided with more ordered layering of the application.
  • Keep transactions as short as possible. Long transactions will unnecessarily lock data for long periods of time, making it unavailable to other business transactions.
  • In some cases, data is cached by the middleware, reducing contention, but it requires "dirty reads" (reading data without locking) from the database. A way around this is to use a versioning facility, which allows a consistent view of the database any time, even while users are modifying the current version.

    Conclusion
    Hybrid databases give developers a new and important option when selecting a database for their J2EE application. Until now, Java developers have really had just one viable option: an RDBMS. Despite the drawbacks of the relational model, only RDBMSs solved the performance requirements intrinsic to enterprise applications. As for ODBMSs, they haven't even begun to meet these requirements. Without that, an adaptable object model is irrelevant to large-scale J2EE development.

    With hybrid databases, J2EE developers can demand both: a database that meets the intrinsic requirements of scalability, high transaction volumes, high-volume data transfer, and the need for fast throughput, together with an object data model that more accurately represents business processes, now and in the future.

    As the number of J2EE applications grows, the limitations of RDBMSs and ODBMSs will become more and more apparent. Hybrid databases represent the missing ingredient for broader J2EE implementation, providing scalability without compromising Java's object environment.

  • More Stories By Didier Cabannes

    Didier Cabannes, chief technology officer at Fresher Information, is the chief architect of the Matisse database, a hybrid database for object developers. For the past 15 years, he has been focused on object and database technology, and developing and deploying mission-critical object-based applications in a variety of environments. He holds a master degree in engineering and has conducted post-graduate research in computer science.

    Comments (16) View Comments

    Share your thoughts on this story.

    Add your comment
    You must be signed in to add a comment. Sign-in | Register

    In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


    Most Recent Comments
    kasiazaki 07/25/08 06:06:47 AM EDT

    dfdf

    prakash 09/05/03 02:40:50 AM EDT

    Dear Sir/Madam

    This is Prakash.I am finished my M.C.A this year(Fresher).I am looking an opportunity in any esteemed organization.I know the software are as follows Java,J2EE,JSP,EJB,HTML,DHTML,JVSCRIPT,VBSCRIPT,ASP,VB,ORACLE.

    But my familiar in my line is JavaTechnologies.So could you help me send any concern opportunities e-address and help me to get.

    Kind Regards.

    prakash 09/05/03 02:40:10 AM EDT

    Dear Sir/Madam

    This is Prakash.I am finished my M.C.A this year(Fresher).I am looking an opportunity in any esteemed organization.I know the software are as follows Java,J2EE,JSP,EJB,HTML,DHTML,JVSCRIPT,VBSCRIPT,ASP,VB,ORACLE.

    But my familiar in my line is JavaTechnologies.So could you help me send any concern opportunities e-address and help me to get.

    Kind Regards.

    john santhosh 07/06/03 08:25:00 AM EDT

    dear sir/madam,
    Iam john santhosh and i have completed MCA by the year 2002 and iam in search of job and i know JAVA,C,C++,COBOL,ASP,XML,MS-SQL SERVER..ETC..
    SO please find me aplace.

    MY QUALIFICATION AND PERCENTAGE

    Bsc comuter science-60%
    MCA-65%

    THANKING YOU

    YOURS FAITHFULLY
    JOHN SANTHOSH

    Jyotish Talukdar 05/29/03 04:21:00 AM EDT

    Hi my name is Jyotish Talukdar and i am doing my final year MCA project in J2EE including JSP,JDBC,JAVA BEAN. I have done Java Script also. I have done Oracle including PL/SQl. So I am waiting eagerly for any kindly call for interviewing.
    My MCA % until last sem: 73%.
    Thanks.

    lalit 05/02/03 05:11:00 AM EDT

    I am a mca student continuing
    my training in java for sixth
    semestar.so any one can help
    me ragarding the above matter
    then plz mail me in above Email
    Adds. If u can then ,u also test
    my skill in java

    MCA: - 80%
    B.Sc. 62%

    piyush vijay vergia 04/20/03 01:32:00 PM EDT

    I am a mca student continuing
    my training in java for sixth
    semestar.so any one can help
    me ragarding the above matter
    then plz mail me in above Email
    Adds. If u can then ,u also test
    my skill in java

    MCA: - 74%
    B.Sc. 64%

    Basudeva Kar 04/05/03 09:10:00 AM EST

    Hi,
    I am a mca student continuing
    my training in java for sixth
    semestar.so any one can help
    me ragarding the above matter
    then plz mail me in above Email
    Adds. If u can then ,u also test
    my skill in java

    rini 02/03/03 04:41:00 AM EST

    Tolong dikirim artikel ke alamat di atas

    vaibhav kulkarni 01/23/03 06:00:00 AM EST

    hi i am vaibhav from pune.
    right now i am studing in Pune Institute of Computer Tech. (PICT).
    i need help regarding OODBMS design in JAVA.
    so if anybody is intrested plz mail me
    on [email protected]

    vaibhav kulkarni 01/23/03 04:44:00 AM EST

    hi i am vaibhav from pune.
    right now i am studing in Pune Institute of Computer Tech. (PICT).
    i need help regarding OODBMS design in JAVA.
    so if anybody is intrested plz mail me
    on [email protected]

    Avishek Sachan 11/10/02 12:21:00 PM EST

    Response As Soon As
    Record
    MCA up to 4th sem 72.3%
    B.Sc. ......... 68.0%
    Intermediate..... 71.4%
    High School 70.1%

    10/21/02 06:49:00 PM EDT
    onkar gramopadhye 06/02/02 02:18:00 PM EDT

    I have done master of computer management course from pune university. I am looking for job for almost 1 year now but not getting any positive response from anywhere I am desperately in need of job so if it is possible please tell me any suggestion or give help, any thing from your end is appreciated

    dezo 05/13/02 04:31:00 AM EDT

    thank you

    Iftach Ragoler 05/13/02 01:19:00 AM EDT

    Do you know of vendors that implements Hybrid Databases? or this is just theoretical issue?

    Regards

    @ThingsExpo Stories
    What sort of WebRTC based applications can we expect to see over the next year and beyond? One way to predict development trends is to see what sorts of applications startups are building. In his session at @ThingsExpo, Arin Sime, founder of WebRTC.ventures, will discuss the current and likely future trends in WebRTC application development based on real requests for custom applications from real customers, as well as other public sources of information,
    China Unicom exhibit at the 19th International Cloud Expo, which took place at the Santa Clara Convention Center in Santa Clara, CA, in November 2016. China United Network Communications Group Co. Ltd ("China Unicom") was officially established in 2009 on the basis of the merger of former China Netcom and former China Unicom. China Unicom mainly operates a full range of telecommunications services including mobile broadband (GSM, WCDMA, LTE FDD, TD-LTE), fixed-line broadband, ICT, data communica...
    My team embarked on building a data lake for our sales and marketing data to better understand customer journeys. This required building a hybrid data pipeline to connect our cloud CRM with the new Hadoop Data Lake. One challenge is that IT was not in a position to provide support until we proved value and marketing did not have the experience, so we embarked on the journey ourselves within the product marketing team for our line of business within Progress. In his session at @BigDataExpo, Sum...
    With the introduction of IoT and Smart Living in every aspect of our lives, one question has become relevant: What are the security implications? To answer this, first we have to look and explore the security models of the technologies that IoT is founded upon. In his session at @ThingsExpo, Nevi Kaja, a Research Engineer at Ford Motor Company, will discuss some of the security challenges of the IoT infrastructure and relate how these aspects impact Smart Living. The material will be delivered i...
    Apache Hadoop is emerging as a distributed platform for handling large and fast incoming streams of data. Predictive maintenance, supply chain optimization, and Internet-of-Things analysis are examples where Hadoop provides the scalable storage, processing, and analytics platform to gain meaningful insights from granular data that is typically only valuable from a large-scale, aggregate view. One architecture useful for capturing and analyzing streaming data is the Lambda Architecture, represent...
    As organizations realize the scope of the Internet of Things, gaining key insights from Big Data, through the use of advanced analytics, becomes crucial. However, IoT also creates the need for petabyte scale storage of data from millions of devices. A new type of Storage is required which seamlessly integrates robust data analytics with massive scale. These storage systems will act as “smart systems” provide in-place analytics that speed discovery and enable businesses to quickly derive meaningf...
    Your homes and cars can be automated and self-serviced. Why can't your storage? From simply asking questions to analyze and troubleshoot your infrastructure, to provisioning storage with snapshots, recovery and replication, your wildest sci-fi dream has come true. In his session at @DevOpsSummit at 20th Cloud Expo, Dan Florea, Director of Product Management at Tintri, will provide a ChatOps demo where you can talk to your storage and manage it from anywhere, through Slack and similar services ...
    SYS-CON Events announced today that Ocean9will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Ocean9 provides cloud services for Backup, Disaster Recovery (DRaaS) and instant Innovation, and redefines enterprise infrastructure with its cloud native subscription offerings for mission critical SAP workloads.
    The taxi industry never saw Uber coming. Startups are a threat to incumbents like never before, and a major enabler for startups is that they are instantly “cloud ready.” If innovation moves at the pace of IT, then your company is in trouble. Why? Because your data center will not keep up with frenetic pace AWS, Microsoft and Google are rolling out new capabilities In his session at 20th Cloud Expo, Don Browning, VP of Cloud Architecture at Turner, will posit that disruption is inevitable for c...
    SYS-CON Events announced today that SoftLayer, an IBM Company, has been named “Gold Sponsor” of SYS-CON's 18th Cloud Expo, which will take place on June 7-9, 2016, at the Javits Center in New York, New York. SoftLayer, an IBM Company, provides cloud infrastructure as a service from a growing number of data centers and network points of presence around the world. SoftLayer’s customers range from Web startups to global enterprises.
    SYS-CON Events announced today that Conference Guru has been named “Media Sponsor” of SYS-CON's 20th International Cloud Expo, which will take place on June 6–8, 2017, at the Javits Center in New York City, NY. A valuable conference experience generates new contacts, sales leads, potential strategic partners and potential investors; helps gather competitive intelligence and even provides inspiration for new products and services. Conference Guru works with conference organizers to pass great dea...
    SYS-CON Events announced today that Technologic Systems Inc., an embedded systems solutions company, will exhibit at SYS-CON's @ThingsExpo, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Technologic Systems is an embedded systems company with headquarters in Fountain Hills, Arizona. They have been in business for 32 years, helping more than 8,000 OEM customers and building over a hundred COTS products that have never been discontinued. Technologic Systems’ pr...
    SYS-CON Events announced today that CA Technologies has been named “Platinum Sponsor” of SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY, and the 21st International Cloud Expo®, which will take place October 31-November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. CA Technologies helps customers succeed in a future where every business – from apparel to energy – is being rewritten by software. From ...
    With major technology companies and startups seriously embracing Cloud strategies, now is the perfect time to attend @CloudExpo | @ThingsExpo, June 6-8, 2017, at the Javits Center in New York City, NY and October 31 - November 2, 2017, Santa Clara Convention Center, CA. Learn what is going on, contribute to the discussions, and ensure that your enterprise is on the right path to Digital Transformation.
    SYS-CON Events announced today that Telecom Reseller has been named “Media Sponsor” of SYS-CON's 20th International Cloud Expo, which will take place on June 6–8, 2017, at the Javits Center in New York City, NY. Telecom Reseller reports on Unified Communications, UCaaS, BPaaS for enterprise and SMBs. They report extensively on both customer premises based solutions such as IP-PBX as well as cloud based and hosted platforms.
    SYS-CON Events announced today that Loom Systems will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Founded in 2015, Loom Systems delivers an advanced AI solution to predict and prevent problems in the digital business. Loom stands alone in the industry as an AI analysis platform requiring no prior math knowledge from operators, leveraging the existing staff to succeed in the digital era. With offices in S...
    SYS-CON Events announced today that Interoute, owner-operator of one of Europe's largest networks and a global cloud services platform, has been named “Bronze Sponsor” of SYS-CON's 20th Cloud Expo, which will take place on June 6-8, 2017 at the Javits Center in New York, New York. Interoute is the owner-operator of one of Europe's largest networks and a global cloud services platform which encompasses 12 data centers, 14 virtual data centers and 31 colocation centers, with connections to 195 add...
    SYS-CON Events announced today that T-Mobile will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. As America's Un-carrier, T-Mobile US, Inc., is redefining the way consumers and businesses buy wireless services through leading product and service innovation. The Company's advanced nationwide 4G LTE network delivers outstanding wireless experiences to 67.4 million customers who are unwilling to compromise on ...
    In his session at @ThingsExpo, Eric Lachapelle, CEO of the Professional Evaluation and Certification Board (PECB), will provide an overview of various initiatives to certifiy the security of connected devices and future trends in ensuring public trust of IoT. Eric Lachapelle is the Chief Executive Officer of the Professional Evaluation and Certification Board (PECB), an international certification body. His role is to help companies and individuals to achieve professional, accredited and worldw...
    SYS-CON Events announced today that Infranics will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Since 2000, Infranics has developed SysMaster Suite, which is required for the stable and efficient management of ICT infrastructure. The ICT management solution developed and provided by Infranics continues to add intelligence to the ICT infrastructure through the IMC (Infra Management Cycle) based on mathemat...