Welcome!

Java IoT Authors: Elizabeth White, Pat Romanski, Liz McMillan, Yeshim Deniz, Zakia Bouachraoui

Related Topics: @DevOpsSummit, Mobile IoT, Agile Computing

@DevOpsSummit: Blog Post

The Ultimate Mobile Performance Testing Checklist | @DevOpsSummit #DevOps

Align mobile performance testing objectives with corporate and business requirements

To conduct complete mobile performance testing, a process that will allow you to consistently deliver an exceptional application experience to mobile users, make sure you hit every point on our ultimate mobile performance testing checklist.

By now, hopefully you're well aware of the importance of mobile performance testing. It's essential for your organization to test for mobile users whether or not you have a dedicated mobile application. In case you aren't aware of its necessity, here are a couple statistics to cement the idea:

  • 29% of smartphone users will immediately switch to another site or app if it doesn't satisfy their needs (e.g., they can't find information or it's too slow)
  • Of the aforementioned users that switch, 70% do so because of lagging load times (Source)

To conduct complete mobile performance testing, a process that will allow you to consistently deliver an exceptional application experience to mobile users, make sure you hit every point on our ultimate mobile performance testing checklist:

✔ Align mobile performance testing objectives with corporate and business requirements

If you ignore the needs of the business, you risk testing the wrong aspects of your application. Identify which aspects of the app experience drive meaningful metrics and then determine the user experience requirements relative to these metrics.

✔ Identify KPIs for application performance

Your key performance indicators (KPIs) will depend on your business objectives. Clarify your goals early on and consider these important KPIs when initiating mobile performance testing: average requests per second, average request response time, maximum response time, average throughput, and error rate.

✔ Work directly with the codebase

Building a mobile performance testing suite without any knowledge of how the code works may cause you to overlook significant problems and bottlenecks. Work with your developers to identify parts of the application that should be tested based on the way the application is built. Also, design your tests in conjunction with Dev to gain a comprehensive understanding of the application areas that need to be tested.

✔ Prioritize test cases

Executing all of your tests across every combination of variants is both time and cost intensive. Initially, you'll want to use your KPIs and behavioral analytics to determine the 10-20 combinations you wish to run in the first round of testing. By focusing on these selected combinations, you'll have a better idea of the types of scenarios your application can and cannot handle. Obviously, you should fix any bottleneck or bug before moving on to the next mobile performance testing scenario.

✔ Load testing

Mobile traffic spikes can affect everyone on your application. Consult your analytics platform to ensure your load testsaccurately reflect realistic proportions of mobile users and web users-this is especially important if your mobile and web sites share a backend.

✔ Device simulation

To make the testing environment and user experience as realistic as possible, utilize emulators. These allow you to test various phones, tablets, etc. without having to physically own and/or manage those devices. While emulating devices, be sure to take into account parallel connections as mobile networks, devices and applications all enforce varying policies in terms of how many are allowed at a given time. You'll also need to customize the user agent header in your scripts to ensure the web or mobile server is sending the correct content to all of your users' different devices.

✔ Network virtualization

Mobile devices will access your servers over 3rd-party carrier networks that vary greatly in speed, latency, bandwidth, etc. By employing network virtualization, you can mimic the network characteristics that mobile devices actually experience. Prioritize the characteristics you see most often, but don't ignore how your application performs for users on the extreme ends of the spectrum. You'll want to make sure that every mobile user's experience is the best it can be, given their specific network limitations.

✔ Geographic location

As you well know, your mobile users won't be accessing your application from the same place. Depending on their location, mobile users will experience different levels of performance, and as a tester, this is something you'll need to account for. Create a geographic profile of your users based on historical data and run geographically-dispersed load tests from the cloud.

✔ Real-device recording

It can be easy to overlook the importance of recording from real devices when conducting mobile performance testing, especially when you're testing a web app that can be recorded from a browser. The problem comes when any call from the mobile interface is different from a call from the web app, or if there is not a corresponding web app. Recording your test scenarios directly from a mobile device will give you the most accurate test results in either situation.

✔ End-user experience testing on real devices

Quickly identify the root cause of mobile performance issues by engaging in end-user experience testing on real devices. Load testing paired with functional testing on a real device will allow you to measure rendering times on the front end of your application that you can then correlate with backend times.

✔ Testing in production

No matter how closely your test environment resembles your production environment, it will never be 100% the same. Sometimes bugs will only reveal themselves in the real world, under actual operation. As such, you'll need to test for mobile users in your real-world production environment. While testing in production poses some risk, it can be riskier not to invest in the practice as you'll end up blind to the levels of performance actually experienced by your mobile users.

Don't Skimp on Mobile Performance Testing
People are becoming more dependent on smartphones and other connected mobile devices for quick access to applications and websites. If you aren't testing for mobile users, you will fall short in terms of application performance.

If you complete this mobile performance testing checklist, however, you will not only appease managers and stakeholders, but you will also be able to successfully deliver exceptional application performance to all of your mobile users, regardless of device, network conditions and geographic location.

To learn more about mobile performance testing strategies and tools that will delight your users, check out our recent webinar on the subject here.

More Stories By Tim Hinds

Tim Hinds is the Product Marketing Manager for NeoLoad at Neotys. He has a background in Agile software development, Scrum, Kanban, Continuous Integration, Continuous Delivery, and Continuous Testing practices.

Previously, Tim was Product Marketing Manager at AccuRev, a company acquired by Micro Focus, where he worked with software configuration management, issue tracking, Agile project management, continuous integration, workflow automation, and distributed version control systems.

IoT & Smart Cities Stories
Founded in 2000, Chetu Inc. is a global provider of customized software development solutions and IT staff augmentation services for software technology providers. By providing clients with unparalleled niche technology expertise and industry experience, Chetu has become the premiere long-term, back-end software development partner for start-ups, SMBs, and Fortune 500 companies. Chetu is headquartered in Plantation, Florida, with thirteen offices throughout the U.S. and abroad.
René Bostic is the Technical VP of the IBM Cloud Unit in North America. Enjoying her career with IBM during the modern millennial technological era, she is an expert in cloud computing, DevOps and emerging cloud technologies such as Blockchain. Her strengths and core competencies include a proven record of accomplishments in consensus building at all levels to assess, plan, and implement enterprise and cloud computing solutions. René is a member of the Society of Women Engineers (SWE) and a m...
SYS-CON Events announced today that DatacenterDynamics has been named “Media Sponsor” of SYS-CON's 18th International Cloud Expo, which will take place on June 7–9, 2016, at the Javits Center in New York City, NY. DatacenterDynamics is a brand of DCD Group, a global B2B media and publishing company that develops products to help senior professionals in the world's most ICT dependent organizations make risk-based infrastructure and capacity decisions.
A valuable conference experience generates new contacts, sales leads, potential strategic partners and potential investors; helps gather competitive intelligence and even provides inspiration for new products and services. Conference Guru works with conference organizers to pass great deals to great conferences, helping you discover new conferences and increase your return on investment.
DXWorldEXPO LLC announced today that ICOHOLDER named "Media Sponsor" of Miami Blockchain Event by FinTechEXPO. ICOHOLDER gives detailed information and help the community to invest in the trusty projects. Miami Blockchain Event by FinTechEXPO has opened its Call for Papers. The two-day event will present 20 top Blockchain experts. All speaking inquiries which covers the following information can be submitted by email to [email protected] Miami Blockchain Event by FinTechEXPOalso offers sp...
Headquartered in Plainsboro, NJ, Synametrics Technologies has provided IT professionals and computer systems developers since 1997. Based on the success of their initial product offerings (WinSQL and DeltaCopy), the company continues to create and hone innovative products that help its customers get more from their computer applications, databases and infrastructure. To date, over one million users around the world have chosen Synametrics solutions to help power their accelerated business or per...
Poor data quality and analytics drive down business value. In fact, Gartner estimated that the average financial impact of poor data quality on organizations is $9.7 million per year. But bad data is much more than a cost center. By eroding trust in information, analytics and the business decisions based on these, it is a serious impediment to digital transformation.
@DevOpsSummit at Cloud Expo, taking place November 12-13 in New York City, NY, is co-located with 22nd international CloudEXPO | first international DXWorldEXPO and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The widespread success of cloud computing is driving the DevOps revolution in enterprise IT. Now as never before, development teams must communicate and collaborate in a dynamic, 24/7/365 environment. There is no time t...
When talking IoT we often focus on the devices, the sensors, the hardware itself. The new smart appliances, the new smart or self-driving cars (which are amalgamations of many ‘things'). When we are looking at the world of IoT, we should take a step back, look at the big picture. What value are these devices providing. IoT is not about the devices, its about the data consumed and generated. The devices are tools, mechanisms, conduits. This paper discusses the considerations when dealing with the...
SYS-CON Events announced today that IoT Global Network has been named “Media Sponsor” of SYS-CON's @ThingsExpo, which will take place on June 6–8, 2017, at the Javits Center in New York City, NY. The IoT Global Network is a platform where you can connect with industry experts and network across the IoT community to build the successful IoT business of the future.