Welcome!

Java IoT Authors: Pat Romanski, Elizabeth White, Yeshim Deniz, Carmen Gonzalez, Liz McMillan

Related Topics: Machine Learning , Java IoT

Machine Learning : Article

Selenium Tests for Automated JavaScript/AJAX Performance Analysis

A step-by-step guide

With more than 3 million downloads, Selenium is a functional testing framework. It allows web developers and QA professionals to automatically test how an application functions on multiple browsers, such as Firefox, Internet Explorer and Chrome and across operating systems, such as different versions of Windows and Mac OS.

The FREE dynaTrace AJAX Edition on the other side is probably the best web site performance analysis tool for Internet Explorer 6, 7 and 8.  Here is what John Resig, Creator of jQuery has to say about it: “I’m very impressed with dynaTrace AJAX Edition’s ability to get at the underlying “magic” that happens inside a browser: page rendering, DOM method execution, browser events, and page layout calculation. Much of this information is completely hidden from developers and I’ve never seen it so easily collected into a single tool. Huge kudos to dynaTrace for revealing this information and especially so for making it happen in Internet Explorer

Combine best of breed tools for Automated Performance Analysis
If we look at these two tools it almost seems logical to use them in combination in order to automatically capture performance data from within the browser for every Selenium Test you execute.

Why would you want to do that?

In case your Selenium Tests identify a problem – whether it is a functional issue (page not working correctly) or a performance problem (page are much slower in the latest build) – the data that is automatically captured by dynaTrace AJAX contains all information necessary for your web engineers to figure out the root cause of the problem. Without this information, your engineers probably go ahead and manually reproduce the error by executing and debugging through the test case manually. Manual work means additional time and effort – and that’s what can be avoided. Best of all – BOTH TOOLS ARE FREE!!

How it works – Example on testing Google Search
Selenium has multiple options to execute tests. Either use your HTML Test Suites that define the individual test cases in HTML Tables or use a programming language such as Java and “code” your tests. Let’s have a look at these two scenarios and how you can get Selenium to work with the dynaTrace AJAX Edition.

The integration basics are explained in Automation with dynaTrace AJAX Edition and  in Advanced Timing and Argument Capturing. The dynaTrace AJAX Agent that runs as an Internet Explorer Add-On gets activated and is configurable via several environment variables. DT_IE_AGENT_ACTIVE=true activates the Add-On so that it collects data from the current browser session. DT_IE_SESSION_NAME can be used to specify the recorded session name, e.g.: MySeleniumTest123. DT_IE_SERVER_HOST and DT_IE_SERVER_PORT can be used to connect to a dynaTrace AJAX Edition client that runs on a different machine – this is useful when you want to centralize data capturing. DT_IE_CLEAR_CACHE=true causes the Add-On to clear the browser cache before any actions are executed – this is very useful when you want to test your web sites cache behaviour.

These environment variables need to be set for the browser instance that runs the tested website. The browser is launched by the selenium server process. Therefore we have to set these environment variables for that server process as they are propagated to all processes that get launched from it.

Requirement: The dynaTrace AJAX Edition must be running on the local machine in order for the launched browser instances to connect to it. You can also run the AJAX Edition on a different machine – in this case you need to specify DT_IE_SERVER_NAME with the name of the machine where the AJAX Edition runs on.

Running an HTML Test Suite
The following batch file sets the necessary environment variables to a) activate the dynaTrace AJAX Add-On in IE and b) specifiy a custom session name:

@echo off
set SELENIUM_RC=C:\selenium\selenium-server.jar
set SCRIPT_PATH=%~dp0

rem dynaTrace AJAX IE agent settings
set DT_IE_AGENT_ACTIVE=true
set DT_IE_SESSION_NAME=Google_SeleniumTest

java -jar %SELENIUM_RC% -log exec.log -htmlSuite "*iexplore" "http://www.google.com"
"%SCRIPT_PATH%\GoogleSuite.html" "%SCRIPT_PATH%\GoogleResults.html"

Before I run this script I make sure to launch the dynaTrace AJAX Edition. When I execute the script Selenium will launch the HtmlRunner in a separate browser instance and will then launch the browser that will execute your test steps. The second browser instance will have the active dynaTrace AJAX agent. You can tell this by the changed IE icon as well as an increasing number in the Captured Events displayed in the dynaTrace AJAX Edition IE Toolbar:

Executing an HTML Test Suite with dynaTrace AJAX capturing  performance data

Executing an HTML Test Suite with dynaTrace AJAX capturing performance data

Note: Even though both browser instances use the DT_IE_XXX environment variables, the AJAX Edition will not collect information from the Selenium Test Runner. The Test Runner runs as an HTA Application which prevents the AJAX Edition Add-On from being loaded. In this case it is an advantage as we are not interested in the performance of the Selenium Runner.

Once the test is done and the browser is closed we can explore the results captured in the dynaTrace AJAX Edition:

dynaTrace AJAX Edition captured performance data from the Selenium  Test Run using the configured Session Name

dynaTrace AJAX Edition captured performance data from the Selenium Test Run using the configured Session Name

The session name is the one we passed in via DT_IE_SESSION_NAME – Google_SeleniumTest in this case. You will see a URL to a local file. That is a file that Selenium opens which then actually opens the URL of the test application. Simply ignore this line and focus on those URLs of your test scenario.

Running Test Case from Java (or JUnit)
Many folks out there use the option to write their Selenium tests in Java – either using JUnit as executing framework or just using the DefaultSelenium proxy in a console app or custom test framework. The approach here is the same as explained in the Html Test Case Use Case. We have to start the selenium server with the DT_IE_XXX environment variables. Here is the batch file I use:

@echo off
set SELENIUM_RC=C:\selenium\selenium-server.jar
set SCRIPT_PATH=%~dp0

rem IE agent settings
set DT_IE_AGENT_ACTIVE=true
set DT_IE_SESSION_NAME=dynaTrace_Selenium

java -jar %SELENIUM_RC%

Here is my Java application that executes the same Google Search test steps as my HTML Test Suite:

public static void main(String args[]) {
Selenium selenium
= new DefaultSelenium("localhost", 4444, "*iexplore", "http://www.google.com");

selenium.
start();
selenium.
open("http://www.google.com");
selenium.
type("q", "dynatrace");
selenium.
click("btnG");

selenium.
stop();
}

The result is the same. The Selenium server launches two browser instances. The first loads the Selenium Runner which won’t be analyzed by dynaTrace AJAX as it is an HTA Application. The second launches google.com and searches for dynaTrace. The following screenshot shows this captured session with the name dynaTrace_Selenium:

dynaTrace AJAX Edition captured the data from the test steps  executed by my Java application

dynaTrace AJAX Edition captured the data from the test steps executed by my Java application

Where to go from here?
The feedback I get from our user community is that this type of integration is a huge time saver. Whenever their Selenium tests fail, the in-depth performance data from the test run is available for analysis. A key point to remember is that this is not just about performance. As the AJAX Edition captures full JavaScript traces, DOM Access, Network and Rendering activities, it makes it very easy to identify all functional and deployment-related problems. Analyzing the dynaTrace AJAX data allows one to identify whether the problem is caused by files that didn’t get deployed on the web server, whether a new JavaScript library was introduced that wasn’t fully tested yet, or whether the problem actually originates on the server-side code. Check out the following additional resources to learn more about how to analyze the data that is captured: Getting Started with dynaTrace AJAX Edition (includes links to real-life web site analysis examples), Why Web 2.0 requires full End-to-End visibility and Learn how Monster.com uses the AJAX Edition for their web-site

In case you have questions on this topic, either comment on the blog or use the comment feature on our dynaTrace AJAX Community Portal.

Related reading:

  1. 5 Steps to Automate Browser Performance Analysis with Watir and dynaTrace AJAX Edition I’ve recently been working with several clients to analyze their...
  2. Ensuring Web Site Performance – Why, What and How to Measure Automated and Accurately It is a fact that end user response time is...
  3. Selenium/BrowserMob integration with dynaTrace Selenium is a free Web Application Testing System. It gained...
  4. Automated Performance Analysis: What’s going on in my ASP.NET or ASP.NET MVC Application? I’ve spent some time in the last weeks playing with different...
  5. Week 6 – How to Make Developers Write Performance Tests I had an interesting conversation with our Test Automation team...

More Stories By Andreas Grabner

Andreas Grabner has been helping companies improve their application performance for 15+ years. He is a regular contributor within Web Performance and DevOps communities and a prolific speaker at user groups and conferences around the world. Reach him at @grabnerandi

@ThingsExpo Stories
SYS-CON Events announced today that Outscale, a global pure play Infrastructure as a Service provider and strategic partner of Dassault Systèmes, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Founded in 2010, Outscale simplifies infrastructure complexities and boosts the business agility of its customers. Outscale delivers a secure, reliable and industrial strength solution for its customers, which in...
SYS-CON Events announced today that Cloudistics, an on-premises cloud computing company, has been named “Bronze Sponsor” of SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Cloudistics delivers a complete public cloud experience with composable on-premises infrastructures to medium and large enterprises. Its software-defined technology natively converges network, storage, compute, virtualization, and management into a ...
New competitors, disruptive technologies, and growing expectations are pushing every business to both adopt and deliver new digital services. This ‘Digital Transformation’ demands rapid delivery and continuous iteration of new competitive services via multiple channels, which in turn demands new service delivery techniques – including DevOps. In this power panel at @DevOpsSummit 20th Cloud Expo, moderated by DevOps Conference Co-Chair Andi Mann, panelists will examine how DevOps helps to meet th...
SYS-CON Events announced today that A&I Solutions has been named “Bronze Sponsor” of SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Founded in 1999, A&I Solutions is a leading information technology (IT) software and services provider focusing on best-in-class enterprise solutions. By partnering with industry leaders in technology, A&I assures customers high performance levels across all IT environments including: mai...
Every successful software product evolves from an idea to an enterprise system. Notably, the same way is passed by the product owner's company. In his session at 20th Cloud Expo, Oleg Lola, CEO of MobiDev, will provide a generalized overview of the evolution of a software product, the product owner, the needs that arise at various stages of this process, and the value brought by a software development partner to the product owner as a response to these needs.
Most technology leaders, contemporary and from the hardware era, are reshaping their businesses to do software in the hope of capturing value in IoT. Although IoT is relatively new in the market, it has already gone through many promotional terms such as IoE, IoX, SDX, Edge/Fog, Mist Compute, etc. Ultimately, irrespective of the name, it is about deriving value from independent software assets participating in an ecosystem as one comprehensive solution.
SYS-CON Events announced today that EARP will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. "We are a software house, so we perfectly understand challenges that other software houses face in their projects. We can augment a team, that will work with the same standards and processes as our partners' internal teams. Our teams will deliver the same quality within the required time and budget just as our partn...
SYS-CON Events announced today that delaPlex will exhibit at SYS-CON's @ThingsExpo, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. delaPlex pioneered Software Development as a Service (SDaaS), which provides scalable resources to build, test, and deploy software. It’s a fast and more reliable way to develop a new product or expand your in-house team.
SYS-CON Events announced today that Tappest will exhibit MooseFS at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. MooseFS is a breakthrough concept in the storage industry. It allows you to secure stored data with either duplication or erasure coding using any server. The newest – 4.0 version of the software enables users to maintain the redundancy level with even 50% less hard drive space required. The software func...
In his keynote at @ThingsExpo, Chris Matthieu, Director of IoT Engineering at Citrix and co-founder and CTO of Octoblu, focused on building an IoT platform and company. He provided a behind-the-scenes look at Octoblu’s platform, business, and pivots along the way (including the Citrix acquisition of Octoblu).
SYS-CON Events announced today that Systena America will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Systena Group has been in business for various software development and verification in Japan, US, ASEAN, and China by utilizing the knowledge we gained from all types of device development for various industries including smartphones (Android/iOS), wireless communication, security technology and IoT serv...
SYS-CON Events announced today that Outscale will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Outscale's technology makes an automated and adaptable Cloud available to businesses, supporting them in the most complex IT projects while controlling their operational aspects. You boost your IT infrastructure's reactivity, with request responses that only take a few seconds.
DevOps at Cloud Expo – being held October 31 - November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA – announces that its Call for Papers is open. Born out of proven success in agile development, cloud computing, and process automation, DevOps is a macro trend you cannot afford to miss. From showcase success stories from early adopters and web-scale businesses, DevOps is expanding to organizations of all sizes, including the world's largest enterprises – and delivering real r...
SYS-CON Events announced today that delaPlex will exhibit at SYS-CON's @CloudExpo, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. delaPlex pioneered Software Development as a Service (SDaaS), which provides scalable resources to build, test, and deploy software. It’s a fast and more reliable way to develop a new product or expand your in-house team.
Five years ago development was seen as a dead-end career, now it’s anything but – with an explosion in mobile and IoT initiatives increasing the demand for skilled engineers. But apart from having a ready supply of great coders, what constitutes true ‘DevOps Royalty’? It’ll be the ability to craft resilient architectures, supportability, security everywhere across the software lifecycle. In his keynote at @DevOpsSummit at 20th Cloud Expo, Jeffrey Scheaffer, GM and SVP, Continuous Delivery Busine...
SYS-CON Events announced today that CollabNet, a global leader in enterprise software development, release automation and DevOps solutions, will be a Bronze Sponsor of SYS-CON's 20th International Cloud Expo®, taking place from June 6-8, 2017, at the Javits Center in New York City, NY. CollabNet offers a broad range of solutions with the mission of helping modern organizations deliver quality software at speed. The company’s latest innovation, the DevOps Lifecycle Manager (DLM), supports Value S...
In order to meet the rapidly changing demands of today’s customers, companies are continually forced to redefine their business strategies in order to meet these needs, stay relevant and continue to see profitable growth. IoT deployment and development is integral in this transformation, and today businesses are increasingly seeing the value of investing their resources into IoT deployments. These technologies are able increase ROI through projects such as connecting supply chains or enabling sm...
A strange thing is happening along the way to the Internet of Things, namely far too many devices to work with and manage. It has become clear that we'll need much higher efficiency user experiences that can allow us to more easily and scalably work with the thousands of devices that will soon be in each of our lives. Enter the conversational interface revolution, combining bots we can literally talk with, gesture to, and even direct with our thoughts, with embedded artificial intelligence, whic...
SYS-CON Events announced today that Peak 10, Inc., a national IT infrastructure and cloud services provider, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Peak 10 provides reliable, tailored data center and network services, cloud and managed services. Its solutions are designed to scale and adapt to customers’ changing business needs, enabling them to lower costs, improve performance and focus intern...
Everywhere we turn in our industry we can find strong opinions about the direction, type and nature of cloud’s impact on computing and business. Another word that is used in every context in our industry is “hybrid.” In his session at 20th Cloud Expo, Alvaro Gonzalez, Director of Technical, Partner and Field Marketing at Peak 10, will use a combination of a few conceptual props and some research recently commissioned by Peak 10 to offer a real-world consideration of how the various categories of...