Welcome!

Java IoT Authors: Stackify Blog, XebiaLabs Blog, Liz McMillan, William Schmarzo, Elizabeth White

Related Topics: @CloudExpo, Java IoT, @DevOpsSummit

@CloudExpo: Blog Feed Post

New to Cloud-Based Performance Testing? By @DanBoutinSOASTA | @CloudExpo #Cloud

You want to get with the times, but you can’t. Why? Because you don’t speak the new language

New to Cloud-Based Performance Testing?  Here's a Quick Language Lesson
by Dan Boutin

OK, go ahead… admit it. You listen to Sirius/XM Radio channels 8 and 9 only. You haven’t noticed who the president is since Bill Clinton left office. If you haven’t had your mid-life crisis, you’re thinking about having it soon. And you may still call your test tool Mercury LoadRunner, refusing to acknowledge HP’s purchase of Mercury since it happened eight years ago.

You want to get with the times, but you can’t. Why? Because you don’t speak the new language. And that language barrier is stopping you from entering the new world order in performance testing: cloud-based performance testing.

So, I’m here to help you feel young again with a language lesson. Today, we’re going to translate your circa-1989 client/server tool terminology to today’s web and mobile cloud-based performance testing language.

(To paraphrase a former University of Florida coach, Urban Meyer, who refused to ever mention Florida State University (FSU) during his entire tenure at the University of Florida — instead calling them “that school out west” — I’ll do the same in this blog post. I will henceforth refer to that tool that had the lion’s share of the marketplace and was named after a planet as “that tool out west”.)

This blog post is a translator for anyone using “that tool out west”. The goal of this post is to map familiar concepts within, say, a client/server test tool developed in 1989 to corresponding similar concepts in SOASTA CloudTest.

Cloud testing terminology and concepts
Both CloudTest and “that tool out west” require planning before creating performance test cases. The performance engineer needs to understand the system, user experience, and business drivers for load testing the system under test. The load test plan needs to identify test cases, Entry and Exit criteria, metrics to be collected, timelines, and include a final report.

Terminology Differences

  1. An environment in CloudTest is very similar to an environment in “that tool out west”: they are both web applications. Both of them perform the job of controlling virtual users for a test scenario, and both require a username and password to access the Performance Center manages and maintains scenarios. “That tool out west” controller, the stand-alone solution, also does this. It is done by SOASTA’s repository in CloudTest. One important difference is that the SOASTA CloudTest environment incorporates load test creation, browser-based functional test creation, and can serve as a Load Generator, all in one platform.
  1. A maestro in CloudTest is the same as a load generator in “that tool out west”. In the case of SOASTA CloudTest, a load server can be local machine or it can be in thecloud. Deploying load generators in the cloud is cost effective and can simulate traffic from different geographic locations on the Internet. With “that tool out west”, load generators are usually deployed locally. A recently announced version of “that tool out west” provides the ability to deploy both controller and load generator in the cloud, but this storm is still evolving and does not yet qualify for a name from the National Hurricane Center.
  1. Analytics dashboards in CloudTest are similar to performance analysis in “that tool out west”.
  1. Virtual users in CloudTest are similar to Vusers in “that tool out west”.
  1. Clips in CloudTest are similar to Vuser scripts in “that tool out west”.
  1. A composition in CloudTest is similar to a scenario in “that tool out west”. Both composition and scenario determine what should happen in a testing session with details about what test cases should be

Building scripts
“That tool out west” uses VuGen. VuGen captures all the recorded traffic based upon the type of protocol selected for the test.

With SOASTA CloudTest, recording is integrated in the product and uses an agent called the SOASTA Conductor on the machine from which recording is being done. The Conductor acts as a web proxy for capturing all the HTTP/HTTPS traffic when recording.

SOASTA CloudTest has the option of viewing recording in list and icon views.

Correlation
SOASTA CloudTest has a “Session Template Package Wizard” which is somewhat similar to “that tool out west’s” “Scan Script for Correlations”. However, SOASTA CloudTest scans for name/value pairs and not for differences between recording and playback. The Session Template Package Wizard identifies all dynamic values in the clip and required values can be selected from the UI.

LR-1

Seed data
SOASTA CloudTest has a seed data repository, which is similar to “that tool out west’s” parameterization functionality, but covering only the parameter type File. Test data can be stored within CloudTest or it can be hosted anywhere accessible by the load generators and used within a clip. Once test data is available, it can be used in a random fashion, as single shot data or unique data for each virtual user. This functionality can be achieved either from the UI or by using JavaScript.

LR-2LR-3

Validation
CloudTest has validation functionality similar to “that tool out west’s” “Content Check”. The validations can be done within a clip/script and added for text, HTML, JSON, XML and SOAP. Validations can be done for data within the header or body. The validations can be added for each and every message within a clip. Similar to the content check functionality in “that tool out west”, success and failure messages can be added as part of validations.

LR-3

Collections
CloudTest can group multiple messages or pages as transactions (also known as collections) and metrics can be tracked in the analytics dashboard for a particular transaction. This is similar to the concept of transaction in “that tool out west”.

LR-4

Test compositions
For creating a test in CloudTest, one or more clips are added to a composition. This is similar to adding vu scripts in a scenario. The composition specifies load servers, number of virtual users, number of iterations, ramp time and duration of the test. This is like adding load generators, building a schedule, specifying duration and ramp up and ramp down for the test in “that tool out west”.

LR-5

Analytics
SOASTA CloudTest has extensive metrics in the analytic dashboards. These dashboards show real-time data, independent of the scale of the test. They are similar to “that tool out west” performance analysis. SOASTA CloudTest results can be viewed during and after the test execution and results can be exported in .csv, .xml and .doc formats.

LR-6

Repository
The SOASTA repository stores results from the test. This is similar to the result directory in “that tool out west”. The SOASTA repository can be maintained on premise or in the cloud and can accessed over the internet. In both cases, the client is an Ajax- based browser.

Monitoring
Just like “that tool out west”, SOASTA CloudTest has integrated monitors for monitoring system utilization of systems under load. These monitors can be created as web monitors, database monitors, application server monitors, etc.

CloudTest can also integrate monitored data from third-party monitoring tools, such as those from New Relic, CA, Amazon, AppDynamics, Correlsense and more, displaying the results in the same real-time dashboards.

LR-7

LR-8

Takeaways
Now that you’re up to speed on the terminology, let’s keep the momentum going. Go ahead and download our free version of CloudTest, the performance testing solution used by every one of the top twenty online retailers.

And now that you speak the language, take the extra mile by getting trained and certified. Find out how here.

Once you are done with that, feel free to check out the other 200+ channels on Sirius/XM radio. And kiss 1989 goodbye. You deserve it.

performance testing

Read the original blog entry...

More Stories By SOASTA Blog

The SOASTA platform enables digital business owners to gain unprecedented and continuous performance insights into their real user experience on mobile and web devices in real time and at scale.

@ThingsExpo Stories
"There's plenty of bandwidth out there but it's never in the right place. So what Cedexis does is uses data to work out the best pathways to get data from the origin to the person who wants to get it," explained Simon Jones, Evangelist and Head of Marketing at Cedexis, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
"Cloud Academy is an enterprise training platform for the cloud, specifically public clouds. We offer guided learning experiences on AWS, Azure, Google Cloud and all the surrounding methodologies and technologies that you need to know and your teams need to know in order to leverage the full benefits of the cloud," explained Alex Brower, VP of Marketing at Cloud Academy, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clar...
Large industrial manufacturing organizations are adopting the agile principles of cloud software companies. The industrial manufacturing development process has not scaled over time. Now that design CAD teams are geographically distributed, centralizing their work is key. With large multi-gigabyte projects, outdated tools have stifled industrial team agility, time-to-market milestones, and impacted P&L stakeholders.
Gemini is Yahoo’s native and search advertising platform. To ensure the quality of a complex distributed system that spans multiple products and components and across various desktop websites and mobile app and web experiences – both Yahoo owned and operated and third-party syndication (supply), with complex interaction with more than a billion users and numerous advertisers globally (demand) – it becomes imperative to automate a set of end-to-end tests 24x7 to detect bugs and regression. In th...
"Akvelon is a software development company and we also provide consultancy services to folks who are looking to scale or accelerate their engineering roadmaps," explained Jeremiah Mothersell, Marketing Manager at Akvelon, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
"MobiDev is a software development company and we do complex, custom software development for everybody from entrepreneurs to large enterprises," explained Alan Winters, U.S. Head of Business Development at MobiDev, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
SYS-CON Events announced today that CrowdReviews.com has been named “Media Sponsor” of SYS-CON's 22nd International Cloud Expo, which will take place on June 5–7, 2018, at the Javits Center in New York City, NY. CrowdReviews.com is a transparent online platform for determining which products and services are the best based on the opinion of the crowd. The crowd consists of Internet users that have experienced products and services first-hand and have an interest in letting other potential buye...
"IBM is really all in on blockchain. We take a look at sort of the history of blockchain ledger technologies. It started out with bitcoin, Ethereum, and IBM evaluated these particular blockchain technologies and found they were anonymous and permissionless and that many companies were looking for permissioned blockchain," stated René Bostic, Technical VP of the IBM Cloud Unit in North America, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Conventi...
SYS-CON Events announced today that Telecom Reseller has been named “Media Sponsor” of SYS-CON's 22nd International Cloud Expo, which will take place on June 5-7, 2018, at the Javits Center in New York, NY. Telecom Reseller reports on Unified Communications, UCaaS, BPaaS for enterprise and SMBs. They report extensively on both customer premises based solutions such as IP-PBX as well as cloud based and hosted platforms.
"Space Monkey by Vivent Smart Home is a product that is a distributed cloud-based edge storage network. Vivent Smart Home, our parent company, is a smart home provider that places a lot of hard drives across homes in North America," explained JT Olds, Director of Engineering, and Brandon Crowfeather, Product Manager, at Vivint Smart Home, in this SYS-CON.tv interview at @ThingsExpo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
Coca-Cola’s Google powered digital signage system lays the groundwork for a more valuable connection between Coke and its customers. Digital signs pair software with high-resolution displays so that a message can be changed instantly based on what the operator wants to communicate or sell. In their Day 3 Keynote at 21st Cloud Expo, Greg Chambers, Global Group Director, Digital Innovation, Coca-Cola, and Vidya Nagarajan, a Senior Product Manager at Google, discussed how from store operations and ...
In his session at 21st Cloud Expo, Carl J. Levine, Senior Technical Evangelist for NS1, will objectively discuss how DNS is used to solve Digital Transformation challenges in large SaaS applications, CDNs, AdTech platforms, and other demanding use cases. Carl J. Levine is the Senior Technical Evangelist for NS1. A veteran of the Internet Infrastructure space, he has over a decade of experience with startups, networking protocols and Internet infrastructure, combined with the unique ability to it...
It is of utmost importance for the future success of WebRTC to ensure that interoperability is operational between web browsers and any WebRTC-compliant client. To be guaranteed as operational and effective, interoperability must be tested extensively by establishing WebRTC data and media connections between different web browsers running on different devices and operating systems. In his session at WebRTC Summit at @ThingsExpo, Dr. Alex Gouaillard, CEO and Founder of CoSMo Software, presented ...
WebRTC is great technology to build your own communication tools. It will be even more exciting experience it with advanced devices, such as a 360 Camera, 360 microphone, and a depth sensor camera. In his session at @ThingsExpo, Masashi Ganeko, a manager at INFOCOM Corporation, introduced two experimental projects from his team and what they learned from them. "Shotoku Tamago" uses the robot audition software HARK to track speakers in 360 video of a remote party. "Virtual Teleport" uses a multip...
A strange thing is happening along the way to the Internet of Things, namely far too many devices to work with and manage. It has become clear that we'll need much higher efficiency user experiences that can allow us to more easily and scalably work with the thousands of devices that will soon be in each of our lives. Enter the conversational interface revolution, combining bots we can literally talk with, gesture to, and even direct with our thoughts, with embedded artificial intelligence, whic...
SYS-CON Events announced today that Evatronix will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Evatronix SA offers comprehensive solutions in the design and implementation of electronic systems, in CAD / CAM deployment, and also is a designer and manufacturer of advanced 3D scanners for professional applications.
Leading companies, from the Global Fortune 500 to the smallest companies, are adopting hybrid cloud as the path to business advantage. Hybrid cloud depends on cloud services and on-premises infrastructure working in unison. Successful implementations require new levels of data mobility, enabled by an automated and seamless flow across on-premises and cloud resources. In his general session at 21st Cloud Expo, Greg Tevis, an IBM Storage Software Technical Strategist and Customer Solution Architec...
To get the most out of their data, successful companies are not focusing on queries and data lakes, they are actively integrating analytics into their operations with a data-first application development approach. Real-time adjustments to improve revenues, reduce costs, or mitigate risk rely on applications that minimize latency on a variety of data sources. In his session at @BigDataExpo, Jack Norris, Senior Vice President, Data and Applications at MapR Technologies, reviewed best practices to ...
An increasing number of companies are creating products that combine data with analytical capabilities. Running interactive queries on Big Data requires complex architectures to store and query data effectively, typically involving data streams, an choosing efficient file format/database and multiple independent systems that are tied together through custom-engineered pipelines. In his session at @BigDataExpo at @ThingsExpo, Tomer Levi, a senior software engineer at Intel’s Advanced Analytics gr...
When talking IoT we often focus on the devices, the sensors, the hardware itself. The new smart appliances, the new smart or self-driving cars (which are amalgamations of many ‘things’). When we are looking at the world of IoT, we should take a step back, look at the big picture. What value are these devices providing? IoT is not about the devices, it’s about the data consumed and generated. The devices are tools, mechanisms, conduits. In his session at Internet of Things at Cloud Expo | DXWor...