Java IoT Authors: Zakia Bouachraoui, Pat Romanski, Elizabeth White, Liz McMillan, Yeshim Deniz

Related Topics: Java IoT

Java IoT: Article

Service Brokers

Service Brokers

The Challenge of Java Integration with Legacy Assets
Organizations are developing a new class of electronic business applications, using Java to capture new business logic. In some cases these e-business applications ,have gone beyond just successful deployment to redefining the competitive landscape. Examples are package tracking, home banking, travel services, automobile purchasing, home shopping, customer support, billing, global securities trading and publishing. The driving forces behind these applications are increased competition in growing markets and the need to reduce costs in services and support. Because e-business is a significant part of the business image in competitive markets, demand for higher performance and reliability is continually increasing.

Successful e-business applications must meet demanding performance, scalability and reliability requirements that previously have been found only on the mainframe. A short list of required Java e-business server capabilities includes multithreading, load balancing, resource management, state persistence, connection pooling and fault-failover. Achieving any one of these capabilities is a large development cost.

JavaSoft, along with leading partners like IBM, are meeting the challenge of Java e-business requirements with Enterprise JavaBeans (EJB). The key capabilities of the EJB framework are:

  • E-business logic is packaged as a Java-Bean component that executes within an EJB server. The EJB server platform supplies thread management, state persistence and other enterprise capabilities to the JavaBean. The e-business application developer gains performance and scalability from the EJB server platform facilities.

  • The EJB server platform supplies data access (JDBC), transactions (JTS), messaging (JMS), directory (JNDI) and other fundamental services as standard Java interfaces. The e-business application developer gains Plug and Play component software development productivity by using tools that support EJB service specifications. Component software developer productivity, previously available only to the visual JavaBean developer, can now be achieved by the server-side Java developer.

    A complete, well-defined set of interfaces is a major strength of Enterprise JavaBeans. The service interface limits the complexity that's visible to the application implementation while enabling the application to exploit the full power of the distributed object system. Server-side component services are fundamentally new because in the past, the integration of service-based capabilities always had to be custom crafted. Without a component framework such as EJB, there was no way of predicting - other than empirically - how the capabilities of services from different vendors could be integrated. The EJB server standardizes how the behavior of one Java service interface will change when another Java service interface is also being used. Business logic and different service interfaces become Plug and Play application components (beans!).

    Enterprise JavaBeans is a complete platform for Java e-business applications. But what about the world's existing business logic? Can we afford to build new e-business applications that ignore existing production systems? The answer, of course, is "NO!"

    The most critical challenge facing Java developers today is with integrating legacy data and applications. The world's information technology (IT) investment (sometimes referred to as legacy) is realized in mission-critical business logic that controls such critical systems as fund transfers, stock exchanges, transportation scheduling, payroll and the worldwide distribution of goods and services. It's estimated that there's over $5 trillion in IT assets that conduct the day-to-day business of the world's economy. In 1997 the U.S., Europe and Japan spent over $600 billion on the maintenance and upgrade of these IT assets.

    The roadmap for legacy integration is inherent in the specification of the EJB platform. Access to legacy assets should be accomplished using Java platform services such as Java Database Connectivity for legacy data and Java Transaction Service for legacy transactional logic. By integrating through Java platform services, legacy business logic and data appear as native JavaBeans to the e-business application developer.

    Let's explore the technical challenge of transforming legacy IT assets into Java services.

    The Challenge of Transforming IT Assets into Java Services
    E-business applications are inherently distributed and multitier. In a multitier distributed application, the logical architecture is typically three-tier architecture, with business logic separate from presentation and data access logic. Mid-tier Java business logic requires access to back-tier IT assets that:

  • Provide high performance throughout for both small (one to hundreds of rows) and large (thousands or millions of rows) amounts of data
  • Sustain high performance throughout under peak concurrent user loads as well as trivial user loads
  • Maintain high availability greater than 99.9%
  • Support centralized management and administration using standard tools
  • Use standard interfaces to reduce the cost of development and maintenance

    Java developers are discovering that existing middleware is not designed to support hundreds or thousands of connections from distributed business logic servers to databases. For example, client/server middleware, such as ODBC, was designed to connect a single resource-rich desktop client to a database server. An entirely new kind of middleware is required that can manage large concurrent sessions loads of an e-business application server.

    A new breed of middleware must meet the performance requirements of e-business and it must also meet the rigid integration limitations of legacy IT assets. By introducing a new tier, a middleware tier between the e-business logic server and the legacy asset tier, we have a platform to add all the facilities that we need. A middleware tier can enhance the legacy asset by adding such facilities as load balancing, connection pooling, multithreading and distributed failover. I-Kinetics calls this middleware architectural approach the Service Broker.

    A good Service Broker isn't just a pass through proxy for the Java service interface. What looks like a simple implementation of a Java service interface may require the Service Broker to coordinate the implementation of several features within the limitations of a legacy interface. For example, obtaining a connection to a legacy application may appear straightforward. However, if that connection is to be pooled, load-balanced and encrypted, and to support a distributed two-phase commit, then the Service Broker will have to implement most of the distributed connectivity management facilities for the legacy asset.

    Adding new capabilities is a significant advantage of a Service Broker architecture. For example, I-Kinetics DataBroker implements the JDBC service with additional capabilities such as multithreaded data streaming, load balancing and connection pooling. You can expect Service Broker architectures to be able to offer the best JDBC 2.0 extension implementations for transaction resource management, batch updates and backward and forward scrolling result sets. Additionally, SNMP management, query caching, messaging and object persistence are advanced capabilities a Service Broker can offer across JDBC, JTS and JMS services.

    The impact of using a Service Broker on the full life-cycle cost of the e-business application can be enormous. For legacy system integration projects, 40 to 70% or more of the development cost may be eliminated because:

  • A Service Broker transforms the legacy data structures into standard Java data formats.
  • A Service Broker offers standard application service interfaces that are widely supported by tool vendors. A Java development tool can be used for the complete development life-cycle -- from prototyping to full deployment of the multitier e-business application.
  • Complex capabilities such as multithreading, opportunistic concurrency, load balancing, connection pooling, partitioning, recoverability, failure isolation and administration are introduced by the Service Broker without any implmentation cost or complexity in the business logic.

    Let's explore a practical example of accessing mainframe data from Java with several different approaches, including the Service Broker approach.

    Comparison of Different Mainframe Data Access Solutions
    One of the most significant data access challenges facing e-business application developers today is accessing mainframe data. Access to mainframe data can create significant business opportunities, but only if a sufficient business case can be made.

  • Lower life-cycle costs - You need to retain mainframe business logic while moving new e-business application load off the mainframe to meet growing demand.
  • Lower operational costs - Equipment support and staff training costs of terminal user interfaces (i.e., green screens) isn't competitive with Web-based browsers.
  • Increase revenue - You can deploy e-business services that directly increase your market share at the expense of your competitors.

    There seems to be a befuddling array of choices when it comes to which solution to use to access mainframe assets. However, your requirements are timeless as they exist for any system deployed for a business revenue-critical enterprise environment. These requirements are reliability, performance, scalability, security, legacy support, multiplatform support, management and low development and maintenance costs. There are a multitude of solutions for integrating mainframe assets, most of which can be identified as being in one of three general categories:

  • Terminal emulation middleware
  • Indirect access gateway middleware
  • Direct access Service Broker middleware

    IBM's Information Management System (IMS) provides On-Line Transaction Processing (OLTP) for much of the world's critical business and commercial service applications. In existence for over 25 years, only CICS surpasses IMS in managing the world's business-critical transactional data. IMS is an ideal testbed for comparing the capabilities of terminal emulation, gateway middleware and Service Broker middleware.

    The business logic for the IMS testbed consists of a common Java application that invokes each different middleware solution to extract 100 rows of IMS data. The Java application and middleware is hosted on a Solaris 2.6 Ultrasparc platform (e-business server). The IMS application and data are hosted on an IBM OS/390 mainframe. The Java code differs only in the call syntax required by the specific middleware interface. JDK 1.1.6 with the JIT enabled was used for all configurations.

    The performance data shown in Table 1 consists of the average of a sample size of five measurement runs. Column 2 (Process Launch), is the amount of time needed to launch and initialize the business logic process. Column 3 (Connect) is the amount of time needed to establish a connection to IMS. Column 4 (Execute) is the amount of time needed to execute the transaction through the middleware and Column 5 (Extract) is the amount of time needed to extract the IMS data from the data block returned by the transaction query. Column 6 (Total Time) is the total clock time needed to execute all aspects of the Java application and middleware on the e-business server. The individual times of Process Launch, Connect, Execute and Extract account for over 90% of the Total Time. Column 7 is the total number of bytes transferred from the OS/390 mainframe tier to the e-business server. Column 8 is the amount of OS/390 processor CPU consumed to execute the transaction.

    If your mainframe applications are mature and you just want to overhaul the interface, not add any new business logic, then you may want to consider using terminal emulation middleware. The terminal emulation (or screen scraper) approach doesn't require any installation of new software on the mainframe. However, terminal emulation has distinct disabilities when used as e-business middleware. The mainframe host must manage a virtual terminal session as well as add terminal management control code data in the data stream for each transaction. All this terminal emulation processing and control is wasteful as it goes unused by the mid-tier business logic.

    For the IMS testbed results given in Table 1, the terminal emulation middleware consumed 85 msec - 280% more than the Gateway and 400% more than the Service Broker solution. In terms of capacity planning, terminal emulation will need 300% or more equivalent MIPS to service the same user load as a Service Broker solution. Additionally, any 3270 screen format change will require code maintenance of each affected screen-scraper script. You can expect high maintenance costs, low performance and increased load on the mainframe with a terminal emulation solution. At a current cost of $10,000 per mainframe MIP, terminal emulation is impractical for the high concurrent user loads sustained by e-business systems.

    The Gateway middleware for accessing IMS was the IBM DB2 Connect ODBC driver on the Java platform connecting to DB2 on the mainframe. DB2 acts as a gateway by enabling a DB2 procedure to access IMS. Two different Java platform configurations were used:
    1. The Java application invoked a CGI/Perl script which in turned invoked the ODBC driver. The Perl script "extracted" the returned IMS data from the ODBC driver as an array of strings. The array wasn't passed back to the Java application and if it had, it would have resulted in a larger Extract time.
    2. The Java application used the Javasoft JDBC/ODBC bridge. The JDBC/ODBC bridge was invoked directly and data returned directly to the Java application using the same extraction logic as the Terminal Emulation and Service Broker middleware configuration.

    The performance of the CGI/Gateway configuration is at a large disadvantage because of the process launch time of 200 msec. However, most of the process launch time can be almost completely eliminated by using FastCGI, which keeps the CGI process in memory.

    The JDBC/ODBC Connect and Extract performance was better than CGI/Perl for Gateway middleware. In this case, Java string manipulation and foreign function call facilities matched Perl. Perl's string manipulation facilities are famous for being able to match or sometimes exceed C/C++ code performance. Both were equal in their use of mainframe resources as they both shared the ODBC/DB2 gateway to IMS. The developer productivity advantages of Common Gateway Interface (CGI) using Perl and other scripting languages has been matched and exceeded by the developer productivity found with Java servlets. JDK 1.2 performance improvements in string manipulation and JIT technology will further increase Java development and runtime advantages. Given the performance advantages of Java servlets over CGI, it becomes easy to understand why server-side Java is so quickly replacing CGI as a business logic platform.

    The Service Broker configuration shown in Figure 3 consists of the I-Kinetics DataBroker and the Neon Systems Shadow Direct for IMS. DataBroker features a complete JDBC driver. The DataBroker is the Service Broker that directly interfaces to Shadow Direct - an installed service running under OS/390 that integrates directly with the IMS Transaction Manager.

    The first immediate performance advantage realized with the Service Broker approach is the Connect time of 12 ms for the IMS testbed. The DataBroker automatically organizes all database connections into a connection pool. When a requested database connection is already available in a connection pool, it will be reused.

    For e-business transactions with small Execute and Extract costs, eliminating fixed overhead Process Launch and Connect costs are essential to achieving high performance. For Insert/Update/Delete transactions with an Execute time of 10 ms or less, a Connect time of 300 ms will heavily tax the peak transactional throughput. Reducing Connect time with connection pooling can achieve significant increases in performance and scalability.

    Other performance advantages realized with the Service Broker were with Execute and Extract performance. Because DataBroker is directly integrated with Shadow Direct and Shadow Direct directly accesses the IMS transaction manager data queues, the resulting Execute performance is very high. Additionally, with Shadow Direct compressing the data stream the amount of data transmitted (3,500 bytes) is dramatically smaller than with Terminal Emulation (11,700 bytes) and Gateway (7,600 bytes) middleware configurations.

    The Java code for accessing IMS using the Service Broker is shown in Figure 4. IMS applications are accessed using a SQL-92-compliant stored procedure call, broker_ims, invoked using the JDBC statement interface method executeQuery. The broker_ims stored procedure call includes parameters that specify the IMS application to execute and result set data formatting. From this example, it is easy to grasp the power of using Java service interfaces, such as JDBC, to access legacy assets, such as IMS.

    The Right Way: Transforming Legacy Assets into Java Services
    The Service Broker architecture provides a path for the enterprise to scale up existing client/server applications to high-performance, high-capacity multitier e-business applications. The enterprise is protected against being locked into a single platform or vendor middleware solution. The enterprise IT infrastructure decision maker can depend on Java services, such as JDBC and JTS, to be essential standard elements of the Java and Enterprise JavaBean platform. The Service Broker provides a complete migration from current IT systems to future IT systems, overcoming the drawbacks of other integration strategies.

    An important part of the Java service legacy integration strategy is for the software industry to create Java Service Brokers for the world's $5 trillion of IT assets. You can expect the Java software vendor community to supply Service Brokers for a wide range of legacy assets. Currently, you can access almost any mainframe asset, such as ADABAS, CICS, DB2, IMS, IDMS and VSAM, as a JDBC data source with the I-Kinetics DataBroker. IBM has announced the access of their popular messaging middleware, MQSeries, as a JMS service. Of course, the leading transaction processing vendors, such as BEA and IBM, will be offering JTS for their transaction monitors.

    A good Service Broker will offer high-performance and scalability features to complement a high-capacity Enterprise Java-Bean platform. Building your own Service Broker is a difficult and expensive experience. Seek out and research existing Service Brokers for your current legacy integration needs. With Plug and Play Service Brokers delivering JDBC, JTS and JMS services, you can focus on the critical tasks of developing new e-business applications.

    References and Resources:

  • "Solving the Data Inter-Operability Problem Using a Universal Data Access Broker," Mike Higgs & Bruce Cottman. IEEE Data Engineering Bulletin, September 1998.
  • "Introducing DataBroker: Standards Based Data Access for the Enterprise," Mike Higgs, Netscape ViewSource 1997.
  • "DataBroker Version 6: A Technical White Paper," I-Kinetics, September 1998.
  • "Universal Data Access," Bruce Cottman. Java Developer's Journal, April, 1997.
  • The Essential Distributed Objects Survival Guide, Robert Orfali, Dan Harkey and Jeri Edwards. John Wiley & Sons, Inc., 1996.
  • Client/Server Programming with JAVA and CORBA, 2nd ed. Robert Orfali and Dan Harkey. John Wiley & Sons, Inc., 1998.
  • More Stories By Dr. Bruce H. Cottman

    Dr. Cottman is founder and CEO of I-Kinetics and is founder and Chairman of the BomponentWare Consortium. As M.I.T. Lincoln Laboratory principal research staff, he designed and deployed one of the first distributed object management frameworks. As President and then later V.P. of R&D, he led the ACCESS and SOCIAL distributing computing projects, resulting in the successful CORBA-like commercial products, MetaCourier and ObjectExpress. In 1991, Dr. Cottman becamse a CORBA convert, sold his Symbiotics holdings and formed I-Kinetics, which is a leading provider of enterprise component software and tools.

    Comments (0)

    Share your thoughts on this story.

    Add your comment
    You must be signed in to add a comment. Sign-in | Register

    In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.

    IoT & Smart Cities Stories
    Moroccanoil®, the global leader in oil-infused beauty, is thrilled to announce the NEW Moroccanoil Color Depositing Masks, a collection of dual-benefit hair masks that deposit pure pigments while providing the treatment benefits of a deep conditioning mask. The collection consists of seven curated shades for commitment-free, beautifully-colored hair that looks and feels healthy.
    The textured-hair category is inarguably the hottest in the haircare space today. This has been driven by the proliferation of founder brands started by curly and coily consumers and savvy consumers who increasingly want products specifically for their texture type. This trend is underscored by the latest insights from NaturallyCurly's 2018 TextureTrends report, released today. According to the 2018 TextureTrends Report, more than 80 percent of women with curly and coily hair say they purcha...
    The textured-hair category is inarguably the hottest in the haircare space today. This has been driven by the proliferation of founder brands started by curly and coily consumers and savvy consumers who increasingly want products specifically for their texture type. This trend is underscored by the latest insights from NaturallyCurly's 2018 TextureTrends report, released today. According to the 2018 TextureTrends Report, more than 80 percent of women with curly and coily hair say they purcha...
    We all love the many benefits of natural plant oils, used as a deap treatment before shampooing, at home or at the beach, but is there an all-in-one solution for everyday intensive nutrition and modern styling?I am passionate about the benefits of natural extracts with tried-and-tested results, which I have used to develop my own brand (lemon for its acid ph, wheat germ for its fortifying action…). I wanted a product which combined caring and styling effects, and which could be used after shampo...
    The platform combines the strengths of Singtel's extensive, intelligent network capabilities with Microsoft's cloud expertise to create a unique solution that sets new standards for IoT applications," said Mr Diomedes Kastanis, Head of IoT at Singtel. "Our solution provides speed, transparency and flexibility, paving the way for a more pervasive use of IoT to accelerate enterprises' digitalisation efforts. AI-powered intelligent connectivity over Microsoft Azure will be the fastest connected pat...
    There are many examples of disruption in consumer space – Uber disrupting the cab industry, Airbnb disrupting the hospitality industry and so on; but have you wondered who is disrupting support and operations? AISERA helps make businesses and customers successful by offering consumer-like user experience for support and operations. We have built the world’s first AI-driven IT / HR / Cloud / Customer Support and Operations solution.
    Codete accelerates their clients growth through technological expertise and experience. Codite team works with organizations to meet the challenges that digitalization presents. Their clients include digital start-ups as well as established enterprises in the IT industry. To stay competitive in a highly innovative IT industry, strong R&D departments and bold spin-off initiatives is a must. Codete Data Science and Software Architects teams help corporate clients to stay up to date with the mod...
    At CloudEXPO Silicon Valley, June 24-26, 2019, Digital Transformation (DX) is a major focus with expanded DevOpsSUMMIT and FinTechEXPO programs within the DXWorldEXPO agenda. Successful transformation requires a laser focus on being data-driven and on using all the tools available that enable transformation if they plan to survive over the long term. A total of 88% of Fortune 500 companies from a generation ago are now out of business. Only 12% still survive. Similar percentages are found throug...
    Druva is the global leader in Cloud Data Protection and Management, delivering the industry's first data management-as-a-service solution that aggregates data from endpoints, servers and cloud applications and leverages the public cloud to offer a single pane of glass to enable data protection, governance and intelligence-dramatically increasing the availability and visibility of business critical information, while reducing the risk, cost and complexity of managing and protecting it. Druva's...
    BMC has unmatched experience in IT management, supporting 92 of the Forbes Global 100, and earning recognition as an ITSM Gartner Magic Quadrant Leader for five years running. Our solutions offer speed, agility, and efficiency to tackle business challenges in the areas of service management, automation, operations, and the mainframe.