Welcome!

Java IoT Authors: Liz McMillan, Pat Romanski, Elizabeth White, Yeshim Deniz, Frank Lupo

Related Topics: @DevOpsSummit, Java IoT, @CloudExpo

@DevOpsSummit: Blog Post

Beyond the Box: Redis Transaction Correlation for Node.js By @PHolditch | @DevOpsSummit #DevOps

How flexible the Node agent is in terms of being able to accommodate scenarios that don't work out of the box

Beyond the Box: Redis Transaction Correlation for Node.js
By Peter Holditch

Looking at your nodes
The AppDynamics Pro product has boasted an agent for instrumenting Node.js application components for several releases now, which is providing many customers with end to end transaction visibility across their applications incorporating Node.js tiers, as well as - uniquely - method level drill-down into the javascript code itself.

In common with the other AppDynamics application agents, the Node.js agent recognises the arrival of http traffic as starting a transaction and also auto-detects many common exit points (places where the code makes requests of downstream systems) out of the box, including outbound http calls, queries on various databases and accesses to the memcached and redis caches.

As caches go, however, redis is a slightly unusual beast in that in addition to storing name.value pairs of data in memory for fast access, it also provides message broker functionality, allowing application processes to interact via publish/subscribe messaging.  I recently came across a customer who was using redis to pass requests (and return responses) between services implemented in Node.  To illustrate this, I have a sample app showing this communication between 2 simple simple services.

The app either runs as the http entry point for requests ("node example.js -entry") or as a service ("node example.js -service")  (for those readers wanting to follow along with the javascript action at home, the sources are available here)

When these two commands are run, the out of the box instrumentation yields the following flowmap:

Unfortunately, this does not look anything like the request/response interaction between 2 application tiers that we know is really going on under the hood. We only see the entry tier of the application, apparently accesing a Redis cache.

The service tier is missing entirely since only tiers executing transactions appear on the flowmap and we do not recognise the arrival of a message on the Redis bus as constituting a transaction entry point and the sending of a message over the Redis bus appears as a cache access since that is the only Redis use-case supported by the out of the box instrumentation.

Fortunately, the Node.js agent provides a java script API we can use to customise the agent's behavior.

Fear not, we can customize it
In order to follow the transaction flow through this Node application, we are going to need to do a couple of things:

  • Recognize calls to the redis message bus as exits
  • tag-and-follow the pub/sub messages that flow through Redis, to facilitate end to end correlation of the transaction flow
  • Recognize the arrival of messages arriving from the Redis bus as starting a transaction

Given that I wrote, and you are reading, this article it will not surprise you to know that the AppDynamics Node agent API provides for all three of these eventualities.

Recognizing the calls to the Redis message bus

To do this, we will need to suppress the out of the box identification of the Redis exit, and then create our own custom exit point.

Suppressing the out of the box handling requires us to implement the beforeExitCall callback.  If our callback does not return anything, the default handling of the exit is suppressed.  We do this conditionally for Redis exits to avoid impacting other exits which we may want:

Note we also had to add a call to the startTransaction API in order that we could interact with the agent's view of the http transaction, which the out of the box HTTP entry support started for us.

Now we have suppressed the out of the box exit handling for Redis, we need to add the code to establish our custom redis exit:

Where the original code simply includes a call to its publish function, the AppDynamics instrumented version surrounds this code with calls to startExitCall and endExitCall to delimit the call itself.

Tagging and following transactions
AppDynamics tags and follows transactions by adding a correlation string to the payload.  Depending on the protocol, this can be added in the message header or the message payload itself.

Since Redis publish / subscribe simply delivers a payload with no bus-level headers we can use, we will transport the correlation string as a field in the JSON payload.

The correlation string, obtained via a call to createCorrelationInfo, is added to the JSON message payload in the ci field.

Note that since this example only contains one client tier and one service tier we can directly correlate the tiers in the call, which means the identifying properties we passed to the startExitCall call will be mapped by ApDynamics to a single destination tier.

If, as is entirely possible in a pub/sub messaging model, the message could arrive at multiple destinations the optional doNotResolve flag should be set on the createCorrelationInfo call, which tells AppDynamics to expect a 1:many interaction between the single client (producer) and multiple consumers and suppress this mapping.

Recognizing Redis pub/sub message arrival
The final piece of the jigsaw is detecting the arrival of messages across Redis and causing this to start a continuing segment of the transaction (the sample does this twice, once for the arrival of the request at the service and again to handle correlation of the reply from the service before the http response is written)

Unsurprisingly, the startTransaction API is used to start the transaction.  Since it is a continuing transaction segment, this needs to process the incoming correlation string, which is obtained through the parseCorrelationInfo API, which is simply passed the string from the incoming payload.

Of course, you also see the reappearance of the code to suppress the default Redis exit handling and the dispatch of the service reply message through the Redis bus wrapped as an exit.

Putting it all together
Putting all the above into practise in the application code yields the following flowmap, which looks like we would expect, given our understanding of the workload:

Job done!
I hope I have shown you how flexible the Node agent is in terms of being able to accommodate scenarios that don't work out of the box.

In conclusion... Rest assured; even if your application flowmap doesn't appear out of the box "by magic", never fear...  AppDynamics has the flexibility and simplicity to get the job done easily!

The post Beyond the Box: Redis transaction correlation for Node.js appeared first on Application Performance Monitoring Blog | AppDynamics.

Read the original blog entry...

More Stories By AppDynamics Blog

In high-production environments where release cycles are measured in hours or minutes — not days or weeks — there's little room for mistakes and no room for confusion. Everyone has to understand what's happening, in real time, and have the means to do whatever is necessary to keep applications up and running optimally.

DevOps is a high-stakes world, but done well, it delivers the agility and performance to significantly impact business competitiveness.

@ThingsExpo Stories
SYS-CON Events announced today that IBM has been named “Diamond Sponsor” of SYS-CON's 21st Cloud Expo, which will take place on October 31 through November 2nd 2017 at the Santa Clara Convention Center in Santa Clara, California.
In his Opening Keynote at 21st Cloud Expo, John Considine, General Manager of IBM Cloud Infrastructure, will lead you through the exciting evolution of the cloud. He'll look at this major disruption from the perspective of technology, business models, and what this means for enterprises of all sizes. John Considine is General Manager of Cloud Infrastructure Services at IBM. In that role he is responsible for leading IBM’s public cloud infrastructure including strategy, development, and offering ...
Coca-Cola’s Google powered digital signage system lays the groundwork for a more valuable connection between Coke and its customers. Digital signs pair software with high-resolution displays so that a message can be changed instantly based on what the operator wants to communicate or sell. In their Day 3 Keynote at 21st Cloud Expo, Greg Chambers, Global Group Director, Digital Innovation, Coca-Cola, and Vidya Nagarajan, a Senior Product Manager at Google, will discuss how from store operations...
In a recent survey, Sumo Logic surveyed 1,500 customers who employ cloud services such as Amazon Web Services (AWS), Microsoft Azure, and Google Cloud Platform (GCP). According to the survey, a quarter of the respondents have already deployed Docker containers and nearly as many (23 percent) are employing the AWS Lambda serverless computing framework. It’s clear: serverless is here to stay. The adoption does come with some needed changes, within both application development and operations. Tha...
Infoblox delivers Actionable Network Intelligence to enterprise, government, and service provider customers around the world. They are the industry leader in DNS, DHCP, and IP address management, the category known as DDI. We empower thousands of organizations to control and secure their networks from the core-enabling them to increase efficiency and visibility, improve customer service, and meet compliance requirements.
Join IBM November 1 at 21st Cloud Expo at the Santa Clara Convention Center in Santa Clara, CA, and learn how IBM Watson can bring cognitive services and AI to intelligent, unmanned systems. Cognitive analysis impacts today’s systems with unparalleled ability that were previously available only to manned, back-end operations. Thanks to cloud processing, IBM Watson can bring cognitive services and AI to intelligent, unmanned systems. Imagine a robot vacuum that becomes your personal assistant tha...
With major technology companies and startups seriously embracing Cloud strategies, now is the perfect time to attend 21st Cloud Expo October 31 - November 2, 2017, at the Santa Clara Convention Center, CA, and June 12-14, 2018, at the Javits Center in New York City, NY, and learn what is going on, contribute to the discussions, and ensure that your enterprise is on the right path to Digital Transformation.
Recently, REAN Cloud built a digital concierge for a North Carolina hospital that had observed that most patient call button questions were repetitive. In addition, the paper-based process used to measure patient health metrics was laborious, not in real-time and sometimes error-prone. In their session at 21st Cloud Expo, Sean Finnerty, Executive Director, Practice Lead, Health Care & Life Science at REAN Cloud, and Dr. S.P.T. Krishnan, Principal Architect at REAN Cloud, will discuss how they b...
SYS-CON Events announced today that mruby Forum will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. mruby is the lightweight implementation of the Ruby language. We introduce mruby and the mruby IoT framework that enhances development productivity. For more information, visit http://forum.mruby.org/.
Digital transformation is changing the face of business. The IDC predicts that enterprises will commit to a massive new scale of digital transformation, to stake out leadership positions in the "digital transformation economy." Accordingly, attendees at the upcoming Cloud Expo | @ThingsExpo at the Santa Clara Convention Center in Santa Clara, CA, Oct 31-Nov 2, will find fresh new content in a new track called Enterprise Cloud & Digital Transformation.
SYS-CON Events announced today that NetApp has been named “Bronze Sponsor” of SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. NetApp is the data authority for hybrid cloud. NetApp provides a full range of hybrid cloud data services that simplify management of applications and data across cloud and on-premises environments to accelerate digital transformation. Together with their partners, NetApp emp...
SYS-CON Events announced today that Avere Systems, a leading provider of enterprise storage for the hybrid cloud, will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 - Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Avere delivers a more modern architectural approach to storage that doesn't require the overprovisioning of storage capacity to achieve performance, overspending on expensive storage media for inactive data or the overbui...
Smart cities have the potential to change our lives at so many levels for citizens: less pollution, reduced parking obstacles, better health, education and more energy savings. Real-time data streaming and the Internet of Things (IoT) possess the power to turn this vision into a reality. However, most organizations today are building their data infrastructure to focus solely on addressing immediate business needs vs. a platform capable of quickly adapting emerging technologies to address future ...
Most technology leaders, contemporary and from the hardware era, are reshaping their businesses to do software. They hope to capture value from emerging technologies such as IoT, SDN, and AI. Ultimately, irrespective of the vertical, it is about deriving value from independent software applications participating in an ecosystem as one comprehensive solution. In his session at @ThingsExpo, Kausik Sridhar, founder and CTO of Pulzze Systems, will discuss how given the magnitude of today's applicati...
SYS-CON Events announced today that Taica will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. TAZMO technology and development capabilities in the semiconductor and LCD-related manufacturing fields are among the best worldwide. For more information, visit https://www.tazmo.co.jp/en/.
SYS-CON Events announced today that Avere Systems, a leading provider of hybrid cloud enablement solutions, will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Avere Systems was created by file systems experts determined to reinvent storage by changing the way enterprises thought about and bought storage resources. With decades of experience behind the company’s founders, Avere got its ...
SYS-CON Events announced today that TidalScale will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. TidalScale is the leading provider of Software-Defined Servers that bring flexibility to modern data centers by right-sizing servers on the fly to fit any data set or workload. TidalScale’s award-winning inverse hypervisor technology combines multiple commodity servers (including their ass...
As hybrid cloud becomes the de-facto standard mode of operation for most enterprises, new challenges arise on how to efficiently and economically share data across environments. In his session at 21st Cloud Expo, Dr. Allon Cohen, VP of Product at Elastifile, will explore new techniques and best practices that help enterprise IT benefit from the advantages of hybrid cloud environments by enabling data availability for both legacy enterprise and cloud-native mission critical applications. By rev...
SYS-CON Events announced today that Ryobi Systems will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Ryobi Systems Co., Ltd., as an information service company, specialized in business support for local governments and medical industry. We are challenging to achive the precision farming with AI. For more information, visit http:...
Amazon is pursuing new markets and disrupting industries at an incredible pace. Almost every industry seems to be in its crosshairs. Companies and industries that once thought they were safe are now worried about being “Amazoned.”. The new watch word should be “Be afraid. Be very afraid.” In his session 21st Cloud Expo, Chris Kocher, a co-founder of Grey Heron, will address questions such as: What new areas is Amazon disrupting? How are they doing this? Where are they likely to go? What are th...