Click here to close now.




















Welcome!

Java IoT Authors: Carmen Gonzalez, Pat Romanski, Liz McMillan, Elizabeth White, Dave Burlington

Related Topics: IoT User Interface, Java IoT

IoT User Interface: Article

Selenium Tests for Automated JavaScript/AJAX Performance Analysis

A step-by-step guide

With more than 3 million downloads, Selenium is a functional testing framework. It allows web developers and QA professionals to automatically test how an application functions on multiple browsers, such as Firefox, Internet Explorer and Chrome and across operating systems, such as different versions of Windows and Mac OS.

The FREE dynaTrace AJAX Edition on the other side is probably the best web site performance analysis tool for Internet Explorer 6, 7 and 8.  Here is what John Resig, Creator of jQuery has to say about it: “I’m very impressed with dynaTrace AJAX Edition’s ability to get at the underlying “magic” that happens inside a browser: page rendering, DOM method execution, browser events, and page layout calculation. Much of this information is completely hidden from developers and I’ve never seen it so easily collected into a single tool. Huge kudos to dynaTrace for revealing this information and especially so for making it happen in Internet Explorer

Combine best of breed tools for Automated Performance Analysis
If we look at these two tools it almost seems logical to use them in combination in order to automatically capture performance data from within the browser for every Selenium Test you execute.

Why would you want to do that?

In case your Selenium Tests identify a problem – whether it is a functional issue (page not working correctly) or a performance problem (page are much slower in the latest build) – the data that is automatically captured by dynaTrace AJAX contains all information necessary for your web engineers to figure out the root cause of the problem. Without this information, your engineers probably go ahead and manually reproduce the error by executing and debugging through the test case manually. Manual work means additional time and effort – and that’s what can be avoided. Best of all – BOTH TOOLS ARE FREE!!

How it works – Example on testing Google Search
Selenium has multiple options to execute tests. Either use your HTML Test Suites that define the individual test cases in HTML Tables or use a programming language such as Java and “code” your tests. Let’s have a look at these two scenarios and how you can get Selenium to work with the dynaTrace AJAX Edition.

The integration basics are explained in Automation with dynaTrace AJAX Edition and  in Advanced Timing and Argument Capturing. The dynaTrace AJAX Agent that runs as an Internet Explorer Add-On gets activated and is configurable via several environment variables. DT_IE_AGENT_ACTIVE=true activates the Add-On so that it collects data from the current browser session. DT_IE_SESSION_NAME can be used to specify the recorded session name, e.g.: MySeleniumTest123. DT_IE_SERVER_HOST and DT_IE_SERVER_PORT can be used to connect to a dynaTrace AJAX Edition client that runs on a different machine – this is useful when you want to centralize data capturing. DT_IE_CLEAR_CACHE=true causes the Add-On to clear the browser cache before any actions are executed – this is very useful when you want to test your web sites cache behaviour.

These environment variables need to be set for the browser instance that runs the tested website. The browser is launched by the selenium server process. Therefore we have to set these environment variables for that server process as they are propagated to all processes that get launched from it.

Requirement: The dynaTrace AJAX Edition must be running on the local machine in order for the launched browser instances to connect to it. You can also run the AJAX Edition on a different machine – in this case you need to specify DT_IE_SERVER_NAME with the name of the machine where the AJAX Edition runs on.

Running an HTML Test Suite
The following batch file sets the necessary environment variables to a) activate the dynaTrace AJAX Add-On in IE and b) specifiy a custom session name:

@echo off
set SELENIUM_RC=C:\selenium\selenium-server.jar
set SCRIPT_PATH=%~dp0

rem dynaTrace AJAX IE agent settings
set DT_IE_AGENT_ACTIVE=true
set DT_IE_SESSION_NAME=Google_SeleniumTest

java -jar %SELENIUM_RC% -log exec.log -htmlSuite "*iexplore" "http://www.google.com"
"%SCRIPT_PATH%\GoogleSuite.html" "%SCRIPT_PATH%\GoogleResults.html"

Before I run this script I make sure to launch the dynaTrace AJAX Edition. When I execute the script Selenium will launch the HtmlRunner in a separate browser instance and will then launch the browser that will execute your test steps. The second browser instance will have the active dynaTrace AJAX agent. You can tell this by the changed IE icon as well as an increasing number in the Captured Events displayed in the dynaTrace AJAX Edition IE Toolbar:

Executing an HTML Test Suite with dynaTrace AJAX capturing  performance data

Executing an HTML Test Suite with dynaTrace AJAX capturing performance data

Note: Even though both browser instances use the DT_IE_XXX environment variables, the AJAX Edition will not collect information from the Selenium Test Runner. The Test Runner runs as an HTA Application which prevents the AJAX Edition Add-On from being loaded. In this case it is an advantage as we are not interested in the performance of the Selenium Runner.

Once the test is done and the browser is closed we can explore the results captured in the dynaTrace AJAX Edition:

dynaTrace AJAX Edition captured performance data from the Selenium  Test Run using the configured Session Name

dynaTrace AJAX Edition captured performance data from the Selenium Test Run using the configured Session Name

The session name is the one we passed in via DT_IE_SESSION_NAME – Google_SeleniumTest in this case. You will see a URL to a local file. That is a file that Selenium opens which then actually opens the URL of the test application. Simply ignore this line and focus on those URLs of your test scenario.

Running Test Case from Java (or JUnit)
Many folks out there use the option to write their Selenium tests in Java – either using JUnit as executing framework or just using the DefaultSelenium proxy in a console app or custom test framework. The approach here is the same as explained in the Html Test Case Use Case. We have to start the selenium server with the DT_IE_XXX environment variables. Here is the batch file I use:

@echo off
set SELENIUM_RC=C:\selenium\selenium-server.jar
set SCRIPT_PATH=%~dp0

rem IE agent settings
set DT_IE_AGENT_ACTIVE=true
set DT_IE_SESSION_NAME=dynaTrace_Selenium

java -jar %SELENIUM_RC%

Here is my Java application that executes the same Google Search test steps as my HTML Test Suite:

public static void main(String args[]) {
Selenium selenium
= new DefaultSelenium("localhost", 4444, "*iexplore", "http://www.google.com");

selenium.
start();
selenium.
open("http://www.google.com");
selenium.
type("q", "dynatrace");
selenium.
click("btnG");

selenium.
stop();
}

The result is the same. The Selenium server launches two browser instances. The first loads the Selenium Runner which won’t be analyzed by dynaTrace AJAX as it is an HTA Application. The second launches google.com and searches for dynaTrace. The following screenshot shows this captured session with the name dynaTrace_Selenium:

dynaTrace AJAX Edition captured the data from the test steps  executed by my Java application

dynaTrace AJAX Edition captured the data from the test steps executed by my Java application

Where to go from here?
The feedback I get from our user community is that this type of integration is a huge time saver. Whenever their Selenium tests fail, the in-depth performance data from the test run is available for analysis. A key point to remember is that this is not just about performance. As the AJAX Edition captures full JavaScript traces, DOM Access, Network and Rendering activities, it makes it very easy to identify all functional and deployment-related problems. Analyzing the dynaTrace AJAX data allows one to identify whether the problem is caused by files that didn’t get deployed on the web server, whether a new JavaScript library was introduced that wasn’t fully tested yet, or whether the problem actually originates on the server-side code. Check out the following additional resources to learn more about how to analyze the data that is captured: Getting Started with dynaTrace AJAX Edition (includes links to real-life web site analysis examples), Why Web 2.0 requires full End-to-End visibility and Learn how Monster.com uses the AJAX Edition for their web-site

In case you have questions on this topic, either comment on the blog or use the comment feature on our dynaTrace AJAX Community Portal.

Related reading:

  1. 5 Steps to Automate Browser Performance Analysis with Watir and dynaTrace AJAX Edition I’ve recently been working with several clients to analyze their...
  2. Ensuring Web Site Performance – Why, What and How to Measure Automated and Accurately It is a fact that end user response time is...
  3. Selenium/BrowserMob integration with dynaTrace Selenium is a free Web Application Testing System. It gained...
  4. Automated Performance Analysis: What’s going on in my ASP.NET or ASP.NET MVC Application? I’ve spent some time in the last weeks playing with different...
  5. Week 6 – How to Make Developers Write Performance Tests I had an interesting conversation with our Test Automation team...

More Stories By Andreas Grabner

Andreas Grabner has been helping companies improve their application performance for 15+ years. He is a regular contributor within Web Performance and DevOps communities and a prolific speaker at user groups and conferences around the world. Reach him at @grabnerandi

@ThingsExpo Stories
17th Cloud Expo, taking place Nov 3-5, 2015, at the Santa Clara Convention Center in Santa Clara, CA, will feature technical sessions from a rock star conference faculty and the leading industry players in the world. Cloud computing is now being embraced by a majority of enterprises of all sizes. Yesterday's debate about public vs. private has transformed into the reality of hybrid cloud: a recent survey shows that 74% of enterprises have a hybrid cloud strategy. Meanwhile, 94% of enterprises are using some form of XaaS – software, platform, and infrastructure as a service.
While many app developers are comfortable building apps for the smartphone, there is a whole new world out there. In his session at @ThingsExpo, Narayan Sainaney, Co-founder and CTO of Mojio, will discuss how the business case for connected car apps is growing and, with open platform companies having already done the heavy lifting, there really is no barrier to entry.
SYS-CON Events announced today that the "Second Containers & Microservices Expo" will take place November 3-5, 2015, at the Santa Clara Convention Center in Santa Clara, CA. Containers and microservices have become topics of intense interest throughout the cloud developer and enterprise IT communities.
Manufacturing connected IoT versions of traditional products requires more than multiple deep technology skills. It also requires a shift in mindset, to realize that connected, sensor-enabled “things” act more like services than what we usually think of as products. In his session at @ThingsExpo, David Friedman, CEO and co-founder of Ayla Networks, will discuss how when sensors start generating detailed real-world data about products and how they’re being used, smart manufacturers can use the data to create additional revenue streams, such as improved warranties or premium features. Or slash...
The Internet of Things is in the early stages of mainstream deployment but it promises to unlock value and rapidly transform how organizations manage, operationalize, and monetize their assets. IoT is a complex structure of hardware, sensors, applications, analytics and devices that need to be able to communicate geographically and across all functions. Once the data is collected from numerous endpoints, the challenge then becomes converting it into actionable insight.
Contrary to mainstream media attention, the multiple possibilities of how consumer IoT will transform our everyday lives aren’t the only angle of this headline-gaining trend. There’s a huge opportunity for “industrial IoT” and “Smart Cities” to impact the world in the same capacity – especially during critical situations. For example, a community water dam that needs to release water can leverage embedded critical communications logic to alert the appropriate individuals, on the right device, as soon as they are needed to take action.
With the Apple Watch making its way onto wrists all over the world, it’s only a matter of time before it becomes a staple in the workplace. In fact, Forrester reported that 68 percent of technology and business decision-makers characterize wearables as a top priority for 2015. Recognizing their business value early on, FinancialForce.com was the first to bring ERP to wearables, helping streamline communication across front and back office functions. In his session at @ThingsExpo, Kevin Roberts, GM of Platform at FinancialForce.com, will discuss the value of business applications on wearable ...
Akana has announced the availability of the new Akana Healthcare Solution. The API-driven solution helps healthcare organizations accelerate their transition to being secure, digitally interoperable businesses. It leverages the Health Level Seven International Fast Healthcare Interoperability Resources (HL7 FHIR) standard to enable broader business use of medical data. Akana developed the Healthcare Solution in response to healthcare businesses that want to increase electronic, multi-device access to health records while reducing operating costs and complying with government regulations.
With major technology companies and startups seriously embracing IoT strategies, now is the perfect time to attend @ThingsExpo in Silicon Valley. Learn what is going on, contribute to the discussions, and ensure that your enterprise is as "IoT-Ready" as it can be! Internet of @ThingsExpo, taking place Nov 3-5, 2015, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with 17th Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The Internet of Things (IoT) is the most profound change in personal an...
Containers are not new, but renewed commitments to performance, flexibility, and agility have propelled them to the top of the agenda today. By working without the need for virtualization and its overhead, containers are seen as the perfect way to deploy apps and services across multiple clouds. Containers can handle anything from file types to operating systems and services, including microservices. What are microservices? Unlike what the name implies, microservices are not necessarily small, but are focused on specific tasks. The ability for developers to deploy multiple containers – thous...
As more intelligent IoT applications shift into gear, they’re merging into the ever-increasing traffic flow of the Internet. It won’t be long before we experience bottlenecks, as IoT traffic peaks during rush hours. Organizations that are unprepared will find themselves by the side of the road unable to cross back into the fast lane. As billions of new devices begin to communicate and exchange data – will your infrastructure be scalable enough to handle this new interconnected world?
Consumer IoT applications provide data about the user that just doesn’t exist in traditional PC or mobile web applications. This rich data, or “context,” enables the highly personalized consumer experiences that characterize many consumer IoT apps. This same data is also providing brands with unprecedented insight into how their connected products are being used, while, at the same time, powering highly targeted engagement and marketing opportunities. In his session at @ThingsExpo, Nathan Treloar, President and COO of Bebaio, will explore examples of brands transforming their businesses by t...
Through WebRTC, audio and video communications are being embedded more easily than ever into applications, helping carriers, enterprises and independent software vendors deliver greater functionality to their end users. With today’s business world increasingly focused on outcomes, users’ growing calls for ease of use, and businesses craving smarter, tighter integration, what’s the next step in delivering a richer, more immersive experience? That richer, more fully integrated experience comes about through a Communications Platform as a Service which allows for messaging, screen sharing, video...
The 3rd International WebRTC Summit, to be held Nov. 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA, announces that its Call for Papers is now open. Topics include all aspects of improving IT delivery by eliminating waste through automated business models leveraging cloud technologies. WebRTC Summit is co-located with 15th International Cloud Expo, 6th International Big Data Expo, 3rd International DevOps Summit and 2nd Internet of @ThingsExpo. WebRTC (Web-based Real-Time Communication) is an open source project supported by Google, Mozilla and Opera that aims to enable bro...
SYS-CON Events announced today the Containers & Microservices Bootcamp, being held November 3-4, 2015, in conjunction with 17th Cloud Expo, @ThingsExpo, and @DevOpsSummit at the Santa Clara Convention Center in Santa Clara, CA. This is your chance to get started with the latest technology in the industry. Combined with real-world scenarios and use cases, the Containers and Microservices Bootcamp, led by Janakiram MSV, a Microsoft Regional Director, will include presentations as well as hands-on demos and comprehensive walkthroughs.
WebRTC has had a real tough three or four years, and so have those working with it. Only a few short years ago, the development world were excited about WebRTC and proclaiming how awesome it was. You might have played with the technology a couple of years ago, only to find the extra infrastructure requirements were painful to implement and poorly documented. This probably left a bitter taste in your mouth, especially when things went wrong.
The 17th International Cloud Expo has announced that its Call for Papers is open. 17th International Cloud Expo, to be held November 3-5, 2015, at the Santa Clara Convention Center in Santa Clara, CA, brings together Cloud Computing, APM, APIs, Microservices, Security, Big Data, Internet of Things, DevOps and WebRTC to one location. With cloud computing driving a higher percentage of enterprise IT budgets every year, it becomes increasingly important to plant your flag in this fast-expanding business opportunity. Submit your speaking proposal today!
In his session at @ThingsExpo, Lee Williams, a producer of the first smartphones and tablets, will talk about how he is now applying his experience in mobile technology to the design and development of the next generation of Environmental and Sustainability Services at ETwater. He will explain how M2M controllers work through wirelessly connected remote controls; and specifically delve into a retrofit option that reverse-engineers control codes of existing conventional controller systems so they don't have to be replaced and are instantly converted to become smart, connected devices.
SYS-CON Events announced today that HPM Networks will exhibit at the 17th International Cloud Expo®, which will take place on November 3–5, 2015, at the Santa Clara Convention Center in Santa Clara, CA. For 20 years, HPM Networks has been integrating technology solutions that solve complex business challenges. HPM Networks has designed solutions for both SMB and enterprise customers throughout the San Francisco Bay Area.
All major researchers estimate there will be tens of billions devices - computers, smartphones, tablets, and sensors - connected to the Internet by 2020. This number will continue to grow at a rapid pace for the next several decades. With major technology companies and startups seriously embracing IoT strategies, now is the perfect time to attend @ThingsExpo, November 3-5, 2015, at the Santa Clara Convention Center in Santa Clara, CA. Learn what is going on, contribute to the discussions, and ensure that your enterprise is as "IoT-Ready" as it can be.