Java IoT Authors: Pat Romanski, Elizabeth White, Liz McMillan, Yeshim Deniz, Mehdi Daoudi

Related Topics: Java IoT

Java IoT: Article

Guaranteed Messaging With JMS

Guaranteed Messaging With JMS

The notion of guaranteed delivery of Java Message Service messages has been lightly touched on in other recently published articles on JMS. But what really makes a JMS message "guaranteed"? Should you just take it on faith, or would you like to know what's behind it?

This article answers these questions via a detailed discussion of message persistence, internal acknowledgment rules, and message redelivery. Using excerpts condensed from the book we coauthored, Java Message Service, we'll explain how JMS guaranteed messaging works - including once-and-only-once delivery semantics, durable subscriptions, failure and recovery scenarios, and transacted messages.

JMS Guaranteed Messaging
There are three key parts to guaranteed messaging: message autonomy, store-and-forward, and the underlying message acknowledgment semantics. Before we discuss these parts, we need to review and define some new terms. A JMS client application uses the JMS API. Each JMS vendor provides an implementation of the JMS API on the client, which we call the client runtime. In addition to the client runtime, the JMS vendor provides some kind of message "server" that implements the routing and delivery of messages. The client runtime and the message server are collectively referred to as the JMS provider.

"Provider failure" refers to any failure condition that is outside the domain of the application code. It could mean a hardware failure that occurs while the provider is entrusted with the processing of a message, an unexpected exception, the abnormal end of a process due to a software defect, or network failures.

Message Autonomy
Messages are self-contained autonomous entities. A message may be sent and re-sent many times across multiple processes throughout its lifetime. Each JMS client along the way will consume the message, examine it, execute business logic, modify it, or create new messages in order to accomplish the task at hand.

Once a JMS client sends a message, its role is completed. The JMS provider guarantees that any other interested parties will receive the message. This contract between a sending JMS client and the JMS provider is much like the contract between a JDBC client and a database. Once the data is delivered, it is considered "safe" and out of the hands of the client.

Store-and-Forward Messaging
When messages are marked persistent, it is the responsibility of the JMS provider to utilize a store-and-forward mechanism to fulfill its contract with the sender. The storage mechanism is used for persisting messages to disk to ensure that the message can be recovered in the event of a provider failure or a failure of the consuming client. The forwarding mechanism is responsible for retrieving messages from storage, and subsequently routing and delivering them.

Message Acknowledgments
The message acknowledgment protocol is key to guaranteed messaging, and support for acknowledgment is required by the semantics of the JMS API. The acknowledgment protocol allows the JMS provider to monitor the progress of a message so that it knows whether the message was successfully produced and consumed. With this knowledge the JMS provider can manage the distribution of messages and guarantee their delivery.

The acknowledgment mode is set on a JMS session when it is created, as indicated below:

tSession = tConnect.createTopicSession(false, Session.CLIENT_ACKNOWLEDGE);

qSession = qConnect.createQueueSession(false, Session.DUPS_OK_ACKNOWLEDGE);

Session bean state is not transactional.

Let's start by examining the AUTO_ACKNOWLEDGE mode. In the loosely coupled asynchronous environment of JMS, senders and receivers (producers and consumers) are intentionally decoupled from each other. Hence, the roles and responsibilities are divided among the message producer, the message server, and the message consumer.

The Producer's Perspective
Under the covers, the TopicPublisher.publish() or Queue- Sender.send() methods are synchronous. These methods are responsible for sending the message and blocking until an acknowledgment is received from the message server. If a failure condition occurs during this operation, an exception is thrown to the sending application and the message is considered undelivered.

The Server's Perspective
The acknowledgment sent to the producer (sender) from the server means that the server has received the message and has accepted responsibility for delivering it. From the JMS server's perspective, the acknowledgment sent to the producer is not tied directly to the delivery of the message. They are logically two separate steps. For persistent messages, the server writes the message out to disk (the store part of store-and-forward), then acknowledges to the producer that the message was received (see Figure 1). For nonpersistent messages this means the server may acknowledge to the sender as soon as it has received the message and has the message in memory. If there are no subscribers for the message's topic, the message may be discarded.

In a publish/subscribe model, the message server delivers a copy of a message to each of the subscribers. For durable subscriber, the message server doesn't consider a message fully delivered until it has received an acknowledgment from all of the message's intended recipients. It knows on a per-consumer basis which clients have received each message and which have not.

Once the message server has delivered the message to all of its known subscribers and has received acknowledgments from each of them, the message is removed from its persistent store (see Figure 2).

If the subscriptions are durable and the subscribers aren't currently connected, the message will be held by the message server until either the subscriber becomes available or the message expires. This is true even for nonpersistent messages. If a nonpersistent message is intended for a disconnected durable subscriber, the message server saves the message to disk as though it were a persistent message. In this case the difference between persistent and nonpersistent messages is subtle, but very important. For nonpersistent messages the JMS provider could fail before it's had a chance to write the message out to disk on behalf of the disconnected durable subscribers. Messages may be lost (see Figure 3).

With persistent messages a provider may fail and recover gracefully, as illustrated in Figures 4 and 5. Since the messages are held in persistent storage, they're not lost and will be delivered to consumers when the provider starts up again. If the messages are sent using a p2p queue, they're guaranteed to be delivered. If the messages were sent via publish/subscribe, they're guaranteed to be delivered only if the consumers' subscriptions are durable.

The Consumer's Perspective
There are also rules governing acknowledgments and failure conditions from the consumer's perspective. If the session is in AUTO_ACKNOWLEDGE mode, the JMS provider's client runtime must automatically send an acknowledgment to the server as each consumer gets the message. If the server doesn't receive this acknowledgment, it considers the message undelivered and may attempt redelivery.

Message Redelivery
The message may be lost if the provider fails while delivering a message to a consumer with a nondurable subscription. If a durable subscriber receives a message, and a failure occurs before the acknowledgment is returned to the provider (see Figure 6), the JMS provider considers the message undelivered and will attempt to redeliver it (see Figure 7). In this case the "once-and-only-once" requirement is in doubt. The consumer may receive the message again, because when delivery is guaranteed, it's better to risk delivering a message twice than to risk losing it entirely. A redelivered message will have the JMSRedelivered flag set. A client application can check this flag by calling the getJMSRedelivered() method on the Message object. Only the most recent message received is subject to this ambiguity.

Point-to-Point Queues
For point-to-point queues, messages are marked by the producer as either persistent or nonpersistent. If the former, they are written to disk and subject to the same acknowledgment rules, failure conditions, and recovery as persistent messages in the publish/subscribe model.

From the receiver's perspective the rules are somewhat simpler since only one consumer can receive a particular instance of a message. A message stays in a queue until it is delivered to a consumer or expires. This is somewhat analogous to a durable subscriber in that a receiver can be disconnected while the message is being produced without losing the message. If the messages are nonpersistent they aren't guaranteed to survive a provider failure.

Specifying the DUPS_OK_ACKNOWLEDGE mode on a session instructs the JMS provider that it is okay to send a message more than once to the same destination. This is different from the once-and-only-once or the at-most-once delivery semantics of AUTO_ACKNOWLEDGE. The DUPS _OK_ACKNOWLEDGE delivery mode is based on the assumption that the processing necessary to ensure once-and-only-once delivery incurs extra overhead and hinders performance and throughput of messages at the provider level. An application that is tolerant of receiving duplicate messages can use the DUPS_OK_ACKNOWLEDGE mode to avoid incurring this overhead.

In practice, the performance improvement you gain from DUPS_OK_ACKNOWLEDGE may be something you want to measure before designing your application around it.

With AUTO_ACKNOWLEDGE mode the acknowledgment is always the last thing to happen implicitly after the onMessage() handler returns. The client receiving the messages can get finer-grained control over the delivery of guaranteed messages by specifying the CLIENT_ACKNOWLEDGE mode on the consuming session.

The use of CLIENT_ACKNOWLEDGE allows the application to control when the acknowledgment is sent. For example, an application can acknowledge a message - thereby relieving the JMS provider of its duty - and perform further processing of the data represented by the message. The key to this is the acknowledge() method on the Message object, as shown in Listing 1.

The acknowledge() method informs the JMS provider that the message has been successfully received by the consumer. This method throws an exception to the client if a provider failure occurs during the acknowledgment process. The provider failure results in the message being retained by the JMS server for redelivery.

Transacted Messages
JMS transactions follow the convention of separating the send operations from the receive operations. Figure 8 shows a transactional send in which a group of messages are guaranteed to get to the message server, or none of them will. From the sender's perspective the messages are cached by the JMS provider until a commit() is issued. If a failure occurs or a rollback() is issued, the messages are discarded. Messages delivered to the message server in a transaction are not forwarded to the consumers until the producer commits the transaction.

The JMS provider won't start delivery of the messages to its consumers until the producer has issued a commit() on the session. The scope of a JMS transaction can include any number of messages.

JMS also supports transactional receives, in which a group of transacted messages are received by the consumer on an all-or-nothing basis (see Figure 9). From the transacted receiver's perspective the messages are delivered to it as expeditiously as possible, yet they are held by the JMS provider until the receiver issues a commit() on the session object. If a failure occurs or a rollback() is issued, the provider will attempt to redeliver the messages, in which case the messages will have the redelivered flag set.

Transacted producers and transacted consumers can be grouped together in a single transaction if they are created from the same session object, as shown in Figure 10. This allows a JMS client to produce and consume messages as a single unit of work. If the transaction is rolled back, the messages produced within the transaction won't be delivered by the JMS provider. The messages consumed within the same transaction won't be acknowledged and will be redelivered.

Unless you're doing a synchronous request/reply, you should avoid grouping a send followed by an asynchronous receive within a transaction. There could be a long interval between the time a message is sent and the related message is asynchronously received, depending on failures or downtime of other processes involved. It's more practical to group the receipt of a message followed by the send of another message.

Creating a JMS Transaction
Enabling a JMS transacted session happens as part of creating a Session object, as shown in Listing 2.

The first parameter of createTopicSession() or createQueueSession() method is a Boolean indicating whether this is a transacted session. That is all we need to create a transactional session. There is no explicit begin() method. When a session is transacted, all messages sent or received using that session are automatically grouped in a transaction. The transaction remains open until either a session.rollback() or a session.commit() happens, at which point a new transaction is started. An additional Session method, isTransacted(), returns a Boolean true or false indicating whether the current session is transactional.

Distributed Transactions
Having all producers and all consumers participate in one global transaction would defeat the purpose of using a loosely coupled asynchronous messaging environment.

Sometimes it's necessary to coordinate the send or receipt of a JMS transaction with the update of another non-JMS resource, like a database or an EJB entity bean. This typically involves an underlying transaction manager that takes care of coordinating the prepare, commit, or rollback of each resource participating in the transaction. JMS provides JTA transaction interfaces for accomplishing this.

JMS providers that implement the JTA XA APIs can participate as a resource in a two-phase commit. The JMS specification provides XA versions of the following JMS objects : XAConnectionFactory, XAQueueConnection, XAQueueConnectionFactory, XAQueueSession, XASession, XATopicConnection, XATopicConnectionFactory, and XATopicSesion.

Our discussion of message acknowledgment shows that producers and consumers have different perspectives on the messages they exchange. The producer has a contract with the message server that ensures that the message will be delivered as far as the server. The server has a contract with the consumer that the message will be delivered to it. The two operations are separate, which is a key benefit of asynchronous messaging. It is the role of the JMS provider to ensure that messages get to where they are supposed to go.

Through message acknowledgment, message persistence, and transactions, JMS provides a strict set of rules that guarantees that your business critical data will travel reliably throughout your global enterprise. Note: This material is condensed from our book, which contains full working examples with detailed explanations of the concepts presented here.

More Stories By Dave Chappell

David Chappell is vice president and chief technologist for SOA at Oracle Corporation, and is driving the vision for Oracle’s SOA on App Grid initiative.

More Stories By Richard Monson-Haefel

Richard Monson-Haefel, an award-winning author and technical analyst, owns Richard Monson-Haefel Consulting. Formerly he was VP of Developer Relations at Curl Inc. and before that a Senior Analyst at The Burton Group. He was the lead architect of OpenEJB, an open source EJB container used in Apache Geronimo, a member of the JCP Executive Committee, member of JCP EJB expert groups, and an industry analyst for Burton Group researching enterprise computing, open source, and Rich Internet Application (RIA) development.

Comments (1)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.

IoT & Smart Cities Stories
The platform combines the strengths of Singtel's extensive, intelligent network capabilities with Microsoft's cloud expertise to create a unique solution that sets new standards for IoT applications," said Mr Diomedes Kastanis, Head of IoT at Singtel. "Our solution provides speed, transparency and flexibility, paving the way for a more pervasive use of IoT to accelerate enterprises' digitalisation efforts. AI-powered intelligent connectivity over Microsoft Azure will be the fastest connected pat...
There are many examples of disruption in consumer space – Uber disrupting the cab industry, Airbnb disrupting the hospitality industry and so on; but have you wondered who is disrupting support and operations? AISERA helps make businesses and customers successful by offering consumer-like user experience for support and operations. We have built the world’s first AI-driven IT / HR / Cloud / Customer Support and Operations solution.
Codete accelerates their clients growth through technological expertise and experience. Codite team works with organizations to meet the challenges that digitalization presents. Their clients include digital start-ups as well as established enterprises in the IT industry. To stay competitive in a highly innovative IT industry, strong R&D departments and bold spin-off initiatives is a must. Codete Data Science and Software Architects teams help corporate clients to stay up to date with the mod...
At CloudEXPO Silicon Valley, June 24-26, 2019, Digital Transformation (DX) is a major focus with expanded DevOpsSUMMIT and FinTechEXPO programs within the DXWorldEXPO agenda. Successful transformation requires a laser focus on being data-driven and on using all the tools available that enable transformation if they plan to survive over the long term. A total of 88% of Fortune 500 companies from a generation ago are now out of business. Only 12% still survive. Similar percentages are found throug...
Druva is the global leader in Cloud Data Protection and Management, delivering the industry's first data management-as-a-service solution that aggregates data from endpoints, servers and cloud applications and leverages the public cloud to offer a single pane of glass to enable data protection, governance and intelligence-dramatically increasing the availability and visibility of business critical information, while reducing the risk, cost and complexity of managing and protecting it. Druva's...
BMC has unmatched experience in IT management, supporting 92 of the Forbes Global 100, and earning recognition as an ITSM Gartner Magic Quadrant Leader for five years running. Our solutions offer speed, agility, and efficiency to tackle business challenges in the areas of service management, automation, operations, and the mainframe.
The Jevons Paradox suggests that when technological advances increase efficiency of a resource, it results in an overall increase in consumption. Writing on the increased use of coal as a result of technological improvements, 19th-century economist William Stanley Jevons found that these improvements led to the development of new ways to utilize coal. In his session at 19th Cloud Expo, Mark Thiele, Chief Strategy Officer for Apcera, compared the Jevons Paradox to modern-day enterprise IT, examin...
With 10 simultaneous tracks, keynotes, general sessions and targeted breakout classes, @CloudEXPO and DXWorldEXPO are two of the most important technology events of the year. Since its launch over eight years ago, @CloudEXPO and DXWorldEXPO have presented a rock star faculty as well as showcased hundreds of sponsors and exhibitors! In this blog post, we provide 7 tips on how, as part of our world-class faculty, you can deliver one of the most popular sessions at our events. But before reading...
DSR is a supplier of project management, consultancy services and IT solutions that increase effectiveness of a company's operations in the production sector. The company combines in-depth knowledge of international companies with expert knowledge utilising IT tools that support manufacturing and distribution processes. DSR ensures optimization and integration of internal processes which is necessary for companies to grow rapidly. The rapid growth is possible thanks, to specialized services an...
At CloudEXPO Silicon Valley, June 24-26, 2019, Digital Transformation (DX) is a major focus with expanded DevOpsSUMMIT and FinTechEXPO programs within the DXWorldEXPO agenda. Successful transformation requires a laser focus on being data-driven and on using all the tools available that enable transformation if they plan to survive over the long term. A total of 88% of Fortune 500 companies from a generation ago are now out of business. Only 12% still survive. Similar percentages are found throug...