Welcome!

Java IoT Authors: Liz McMillan, Elizabeth White, Sematext Blog, William Schmarzo, AppNeta Blog

Related Topics: Java IoT, Open Source Cloud, Machine Learning

Java IoT: Article

Asynchronous Logging Using Spring

Create applications that require high-volume logging without impacting an application’s performance

Each application developer faces the problem of logging usage information. On the one hand, the more logging that's done the easier it is to detect and locate the source of problems. On the other hand, large volume logging might impair an application's performance.

This problem is typically solved by defining various log levels dependent on a program's maturity. For example, a program in developmental stages would have a higher logging requirements; logging requirements would be relatively lower in the production phase. If an application requires a lot of logging for audit purposes, then special measures are required to protect performance.

This article provides a possible solution for this problem by using Spring asynchronous support.

Rationale for High-Volume Logging
Logging is used extensively to help find problems within applications. A developer who finds a problem can investigate it by enabling debug logging. He may then reproduce the problem, or create additional logging if needed. Programmers usually require extensive logging to locate problems.

However, the aforementioned process is not a universal solution. If a developer comes across a one-time event that does not happen again, then he has to uncover and fix the problem using only existing log data. For instance, a security problem should not be resolved by enabling full logging and waiting for another intrusion. Such cases require a lot of logging upfront.

Above and beyond the use of logging for application development, regulatory compliance requirements also demand a minimum amount of logging.

Possible Solutions
The simplest solution would be to log everything synchronously, right inside business logic methods. Of course this solution would reduce processing speed, but if an application does not have strict performance requirements, it might not be a problem.

A developer may also use some existing solutions, such as synchronous logging. Examples include log4j, AsyncAppender, and JMS. The latter is especially useful if none of the logging events should be lost.

Very often logging operations are not very fast because, for reliability purposes, these logs are often written either to a database or to a remote server. In addition, data may be subject to operations before they can be logged.

Consider a simple case when logging is needed, but performance should not be affected.

Please note that threads may not necessarily execute in the same order as they were called, so you'll have to sort the logged events by timestamp in order to have them in chronological order.

Spring 3.0 Asynchronous Support
Overview

Spring 2.0 provided a TaskExecutor abstraction that could be used for running asynchronous tasks. Evolution to version 3.0 enabled annotations to be used for this purpose. Refer to the Task Execution and Scheduling section in the Spring 3.0 documentation.

There are several built-in TaskExecutor implementations, and not all of them are asynchronous. For example, the SyncTaskExecutor invokes a task in the calling thread.

This article will concentrate on the ThreadPoolTaskExecutor - the most commonly used implementation in the Java 5 environment.

The ThreadPoolTaskExecutor provides the following configurable parameters:

pool-size: Defines the size of a pool; it also accepts a "core-max" range. The "core" is the initial number of threads in a pool; the "max" is the maximum number of threads. If only one number is defined, then the core and max are the same size.

Please note that the executor will first use the core number of threads, and then add threads to the pool only when the queue is full.

queue-capacity: Refers to the size of the task queue when all threads are busy. By default the queue is unlimited. That might lead to an OutOfMemory exception and disrupt the application, therefore, a specific queue size should be defined.

rejection-policy: A parameter that determines what should happen if both the thread pool and queue are full. In this case the task is rejected, and then the rejection policy determines what should happen to it.

The following policies are supported:

  • AbortPolicy: Executor will throw a TaskRejectedException
  • DiscardPolicy: Tasks will be skipped
  • DiscardOldestPolicy: Oldest tasks will be skipped
  • CallerRunsPolicy: Task will be executed in caller thread (synchronously)

Spring 3.0 allows bean configuration using both Annotations and XML. We will consider Annotation and XML configuration of TaskExecutors only.

Here is a simple business class that requires logging.

@Service
public class BankOperator {

@Autowired

private BusinessLogger businessLogger;

public void transferMoney() {

//... business logic for transferring money
// writing log
businessLogger.logMoneyTransfer(new Date(), "John", new BigDecimal(100));
}
}

As you can see the businessLogger has a specific method for a specific business operation. Developers always pass the current timestamp and a number of parameters. The timestamp is needed because the logging method will be invoked asynchronously at an arbitrary time in the future. The list of parameters will depend on what kind of logging is done. Usually this kind of logging is saved to a database, so developers need a number of parameters that will be mapped to the fields of the database table.

The maximum possible amount of operations should utilize asynchronous logging.

For example, if logging information depends on an account number, years of service and the balance of a bank account, then it's better to pass these three parameters to a logging method. The logging logic would be implemented, thereby, lessening the influence of logging on execution of the primary service.

XML - configuration
The configuration of ThreadPoolTaskExecutor looks like this:

<bean id="businessLogExecutor" class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor">
<property name="corePoolSize" value="1" />
<property name="maxPoolSize" value="1" />
<property name="queueCapacity" value="1" />
<property name="rejectedExecutionHandler" ref="callerRuns" />
</bean>

<bean id="callerRuns" class="java.util.concurrent.ThreadPoolExecutor.CallerRunsPolicy"/>

Spring 3.0 enables developers to use the ‘executor' element to create a ThreadPoolTaskExecutor instance:

<task:executor id="businessLogExecutor"
pool-size="5-25"
queue-capacity="100"
rejection-policy="CALLER_RUNS"/>

The logger class might look like the following:

public class AsyncBusinessLogger implements BusinessLogger {

private TaskExecutor taskExecutor;


public void setTaskExecutor(TaskExecutor taskExecutor) {

this.taskExecutor = taskExecutor;
}

public void logMoneyTransfer(final Date timestamp, final String customerName, final BigDecimal sum) {

taskExecutor.execute(new Runnable() {
public void run() {
System.out.println("Customer '" + customerName + "' transfered $" + sum + " on " + timestamp);
}
});
}
}

A BusinessLogger may resemble the following:

<bean id="businessLogger" class="com.axmor.async.logging.AsyncBusinessLogger">
<property name="taskExecutor" ref="businessLogExecutor" />
</bean>

Using XML configuration is a more flexible approach. A developer can tell many Business Loggers to use the same TaskExecutor, or tell a single Business Logger to use multiple TaskExecutors.

@Async annotation
To use the @Async annotation, turn on task annotation support by including the following in config.xml:

<task:annotation-driven />

In this case, the ThreadPoolTaskExecutor with default settings will be used.

To use the aforementioned executor, insert the following instead:

<task:annotation-driven executor="businessLogExecutor"/>

The logger will look like the following:

@Service
public class AsyncBusinessLogger implements BusinessLogger {

@Async

public void logMoneyTransfer(Date timestamp, String customerName, BigDecimal sum) {
System.out.println("Customer '" + customerName + "' transfered $" + sum + " on " + timestamp);
}
}

The result is simpler and cleaner code. But a less flexible configuration comes at a price: all methods annotated with @Async will use the same TaskExecutor throughout the application.

@Async vs TaskExecutor: when to use
Using the @Async annotation is much simpler than having to wire the task executor to call tasks manually.

Usually the @Async annotation is the preferred method.

In cases where a developer doesn't want to use a single TaskExecutor for all asynchronous operations, which is the case for @Async annotation, he will have to use TaskExecutors.

Conclusion
Developers should strongly consider using asynchronous logging when creating applications that require high-volume logging without impacting an application's performance.

The article shows how easy it is to create your own custom asynchronous logging using Spring 3.0 Async support.

More Stories By Mylnikov Sergey

Mylnikov Sergey is a senior software engineer in the IBM Solution Group at Axmor Software. He has over seven years of experience in software development, as well as strong knowledge in Java SE/EE and web application development. He has been a technical lead and software architect in several successfully completed enterprise projects for US, UK, and Australian customers.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


@ThingsExpo Stories
SYS-CON Events announced today that Auditwerx will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Auditwerx specializes in SOC 1, SOC 2, and SOC 3 attestation services throughout the U.S. and Canada. As a division of Carr, Riggs & Ingram (CRI), one of the top 20 largest CPA firms nationally, you can expect the resources, skills, and experience of a much larger firm combined with the accessibility and attent...
In his session at @ThingsExpo, Eric Lachapelle, CEO of the Professional Evaluation and Certification Board (PECB), will provide an overview of various initiatives to certifiy the security of connected devices and future trends in ensuring public trust of IoT. Eric Lachapelle is the Chief Executive Officer of the Professional Evaluation and Certification Board (PECB), an international certification body. His role is to help companies and individuals to achieve professional, accredited and worldw...
In his General Session at 16th Cloud Expo, David Shacochis, host of The Hybrid IT Files podcast and Vice President at CenturyLink, investigated three key trends of the “gigabit economy" though the story of a Fortune 500 communications company in transformation. Narrating how multi-modal hybrid IT, service automation, and agile delivery all intersect, he will cover the role of storytelling and empathy in achieving strategic alignment between the enterprise and its information technology.
Microservices are a very exciting architectural approach that many organizations are looking to as a way to accelerate innovation. Microservices promise to allow teams to move away from monolithic "ball of mud" systems, but the reality is that, in the vast majority of organizations, different projects and technologies will continue to be developed at different speeds. How to handle the dependencies between these disparate systems with different iteration cycles? Consider the "canoncial problem" ...
The Internet of Things is clearly many things: data collection and analytics, wearables, Smart Grids and Smart Cities, the Industrial Internet, and more. Cool platforms like Arduino, Raspberry Pi, Intel's Galileo and Edison, and a diverse world of sensors are making the IoT a great toy box for developers in all these areas. In this Power Panel at @ThingsExpo, moderated by Conference Chair Roger Strukhoff, panelists discussed what things are the most important, which will have the most profound e...
Keeping pace with advancements in software delivery processes and tooling is taxing even for the most proficient organizations. Point tools, platforms, open source and the increasing adoption of private and public cloud services requires strong engineering rigor - all in the face of developer demands to use the tools of choice. As Agile has settled in as a mainstream practice, now DevOps has emerged as the next wave to improve software delivery speed and output. To make DevOps work, organization...
My team embarked on building a data lake for our sales and marketing data to better understand customer journeys. This required building a hybrid data pipeline to connect our cloud CRM with the new Hadoop Data Lake. One challenge is that IT was not in a position to provide support until we proved value and marketing did not have the experience, so we embarked on the journey ourselves within the product marketing team for our line of business within Progress. In his session at @BigDataExpo, Sum...
Web Real-Time Communication APIs have quickly revolutionized what browsers are capable of. In addition to video and audio streams, we can now bi-directionally send arbitrary data over WebRTC's PeerConnection Data Channels. With the advent of Progressive Web Apps and new hardware APIs such as WebBluetooh and WebUSB, we can finally enable users to stitch together the Internet of Things directly from their browsers while communicating privately and securely in a decentralized way.
DevOps is often described as a combination of technology and culture. Without both, DevOps isn't complete. However, applying the culture to outdated technology is a recipe for disaster; as response times grow and connections between teams are delayed by technology, the culture will die. A Nutanix Enterprise Cloud has many benefits that provide the needed base for a true DevOps paradigm.
What sort of WebRTC based applications can we expect to see over the next year and beyond? One way to predict development trends is to see what sorts of applications startups are building. In his session at @ThingsExpo, Arin Sime, founder of WebRTC.ventures, will discuss the current and likely future trends in WebRTC application development based on real requests for custom applications from real customers, as well as other public sources of information,
In his General Session at 17th Cloud Expo, Bruce Swann, Senior Product Marketing Manager for Adobe Campaign, explored the key ingredients of cross-channel marketing in a digital world. Learn how the Adobe Marketing Cloud can help marketers embrace opportunities for personalized, relevant and real-time customer engagement across offline (direct mail, point of sale, call center) and digital (email, website, SMS, mobile apps, social networks, connected objects).
With the introduction of IoT and Smart Living in every aspect of our lives, one question has become relevant: What are the security implications? To answer this, first we have to look and explore the security models of the technologies that IoT is founded upon. In his session at @ThingsExpo, Nevi Kaja, a Research Engineer at Ford Motor Company, will discuss some of the security challenges of the IoT infrastructure and relate how these aspects impact Smart Living. The material will be delivered i...
"My role is working with customers, helping them go through this digital transformation. I spend a lot of time talking to banks, big industries, manufacturers working through how they are integrating and transforming their IT platforms and moving them forward," explained William Morrish, General Manager Product Sales at Interoute, in this SYS-CON.tv interview at 18th Cloud Expo, held June 7-9, 2016, at the Javits Center in New York City, NY.
SYS-CON Events announced today that Ocean9will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Ocean9 provides cloud services for Backup, Disaster Recovery (DRaaS) and instant Innovation, and redefines enterprise infrastructure with its cloud native subscription offerings for mission critical SAP workloads.
Your homes and cars can be automated and self-serviced. Why can't your storage? From simply asking questions to analyze and troubleshoot your infrastructure, to provisioning storage with snapshots, recovery and replication, your wildest sci-fi dream has come true. In his session at @DevOpsSummit at 20th Cloud Expo, Dan Florea, Director of Product Management at Tintri, will provide a ChatOps demo where you can talk to your storage and manage it from anywhere, through Slack and similar services ...
SYS-CON Events announced today that Linux Academy, the foremost online Linux and cloud training platform and community, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Linux Academy was founded on the belief that providing high-quality, in-depth training should be available at an affordable price. Industry leaders in quality training, provided services, and student certification passes, its goal is to c...
"delaPlex is a software development company. We do team-based outsourcing development," explained Mark Rivers, COO and Co-founder of delaPlex Software, in this SYS-CON.tv interview at 18th Cloud Expo, held June 7-9, 2016, at the Javits Center in New York City, NY.
SYS-CON Events announced today that SoftLayer, an IBM Company, has been named “Gold Sponsor” of SYS-CON's 18th Cloud Expo, which will take place on June 7-9, 2016, at the Javits Center in New York, New York. SoftLayer, an IBM Company, provides cloud infrastructure as a service from a growing number of data centers and network points of presence around the world. SoftLayer’s customers range from Web startups to global enterprises.
SYS-CON Events announced today that Technologic Systems Inc., an embedded systems solutions company, will exhibit at SYS-CON's @ThingsExpo, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Technologic Systems is an embedded systems company with headquarters in Fountain Hills, Arizona. They have been in business for 32 years, helping more than 8,000 OEM customers and building over a hundred COTS products that have never been discontinued. Technologic Systems’ pr...
SYS-CON Events announced today that CA Technologies has been named “Platinum Sponsor” of SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY, and the 21st International Cloud Expo®, which will take place October 31-November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. CA Technologies helps customers succeed in a future where every business – from apparel to energy – is being rewritten by software. From ...