The test results should answer the following questions:

• How many users can the application handle while maintaining an acceptable response time?
• What is the load threshold above which the server begins to generate errors and/or refuse connections?
• Does the server remain functional under high load or does it crash?

To do a stress test, testers should ramp up the load, starting from normal and up to the maximum predicted limit, and monitor the response times and error rates. A sudden change indicates that a threshold has been passed.

A stress test can also be used to validate performance variations after an application or infrastructure update. In this case, the stress test should answer the following questions:

• Have the implemented upgrades resulted in real performance gains?
• Which pages have seen a loss in performance?

Make sure that the load testing tool you use can compare a performance between the two sets of test results.

What Is Your Criteria for Success or Failure?
Before you test an application, it's essential to define the success or failure criteria. So first set the acceptable levels for robustness and performance.

In most cases, you can define the criteria in terms of response time, error rate, or number of requests per second. This includes defining:

• Average response time per page (this may be different from one page to another)
• Maximum response time per page (may also be different from one page to another)
• Maximum error rate

How to Test Your Application
To produce a realistic test, it's essential to pay special attention to certain key items.

Number of Virtual Users
The number of virtual users must be close to the number of real users once the application is in production, with a realistic think time applied between pages. Think time is the time it takes a user to click through to the next page from the current one. Avoid testing with too few virtual users and a reduced think time. Many assume that the results would be the same, since the number of requests played per second is identical. However, this is not the case, for the following reasons:

• The memory burden on the server is different: Each user session uses a certain amount of memory. If the number of user sessions is underestimated, the server will be running under more favorable conditions than in real life and the results will be distorted.
• The number of sockets open on the server simultaneously is different. An underestimation of user numbers means the maximum threshold for open server sockets cannot be tested.
• The resource pools (JDBC connection pools) will not be operating under realistic conditions. An inappropriate pool-size setting might not be detected during the test.

Type of Virtual Users
Not all users employ a Web application in the same way. Therefore it is important to define a virtual user for each user profile: simple browsing, browsing with modifications, system administrator, etc.

Group these user types into a population. Populations allow a user ratio to be maintained when the load varies. For example, a ratio of 90% standard users and 10% system administrators may be selected.

User Accounts and Values
Testers should use variables to dynamically modify key values such as user account logins or certain form parameters (such as productID in an e-business application). The main idea of this is to bypass the use of the various server caches, for the following reasons:

• Playing the same requests with the same values produces an unrealistically high performance due to the use of various caches: preloading into memory cache, connection pools, system swap, etc.
• Yet, completely disabling the caches (when available) will produce an unrealistically poor performance.

Load Balancers
Load balancers can use the client IP address to balance the load over several servers. If all virtual users are using the same IP address (default setting), the total load will be sent to a single server.

To obtain a realistic load, configure your scenario to have several IP addresses used to play the load.

Simulating Actual User Bandwidth
A user connecting at 100 Mbps through a local network and a user connecting with a dial-up modem at 56 Kbps do not load the server and bandwidth in the same way. Therefore, it is important to configure the bandwidth setting to realistically simulate the users' connection speeds. This can be done in the following steps:

• After a restart, don't hesitate to "warm up" the server with a few calls before generating a sudden high load that, in addition to being unrealistic, can cause the server to crash. Sending a short light load beforehand lets certain resources, such as connection pools or thread pools, be pre-allocated.
• Use your load testing tool to gradually spread the initial load using a start-up policy. For example, you can choose to start your 1,000 virtual users over a two-minute period.
• Run the test for a significant length of time to iron out any outliers.
• Make sure the load generators are not overloaded; CPU and memory usage are displayed in real-time throughout the test.

The Test Results - How to Use Them?
To effectively use your test results, go back to your testing objectives.

Performance
If your goal was to test performance, then it is important to find useful information to eliminate bottlenecks and establish a baseline for future testing.

Look at your test results; where did the bottlenecks occur? At the application level? The database level? The operating system level? Analyze the results and modify the application and the database if necessary. Make sure to modify one variable at a time and redo the tests, otherwise multiple changes can have subtle interactions that are hard to qualify and repeat.

Load - Crash Testing
If your goal was to determine the greatest number of users who can use your application without it crashing, then it's important that your application meets the performance baseline established during performance testing. This can be done by running regression tests against the application at a specified maximum load.

Also make note to identify bugs that do not surface in cursory testing, such as memory management bugs.

Stress Testing
If your main purpose in testing was to make sure that the system fails and recovers gracefully, make sure your stress tests take unpredictability into account. For example, how does the system react when you double the baseline number for concurrent users/HTTP connections? Or run processes that consume resources (CPU, memory, disk, network) on the Web and database servers?

Conclusion
Poor application performance, unexpected deployment issues, and server malfunction can compromise your Web application. But by following the best practices and techniques explained in this article, testers can improve reliability and performance, reduce development costs and most importantly help their companies deliver what they promised in terms of application functionality and performance.