How To: Load Test Web Applications

J.D. Meier, Carlos Farre, Prashant Bansode, Scott Barber

Applies To

Summary

This How To explains how to load-test a Web application. Load testing helps to identify the maximum operating capacity of the application and any bottlenecks that might be degrading performance. The basic approach to performing load testing on a Web application is:
  1. Identify the performance-critical scenarios.
  2. Identify the workload profile for distributing the entire load among the key scenarios.
  3. Identify the metrics that you want to collect in order to verify them against your performance objectives.
  4. Design tests to simulate the load.
  5. Use tools to implement the load according to the designed tests, and capture the metrics.
  6. Analyze the metric data captured during the tests.

By using an iterative testing process, these steps should help you achieve your performance objectives.

Contents

Objectives

Overview

The most basic type of load testing is used to determine the Web application’s behavior under both normal and anticipated peak load conditions. As you begin load testing, it is recommended that you start with a small number of virtual users and then incrementally increase the load from normal to peak. You can then observe how your application performs during this gradually increasing load condition. Eventually, you will cross a threshold limit for your performance objectives. For example, you might continue to increase the load until the server processor utilization reaches 75 percent, or when end-user response times exceed 8 seconds.

Input

The following are useful inputs for load-testing a Web application:

Output

The main outcomes that load testing helps you to accomplish are:

Summary of Steps

The following steps are involved in load-testing a Web application:

The sections that follow discuss each step in detail.

Step 1 - Identify Performance Acceptance Criteria

Identifying performance acceptance criteria is most valuable when initiated early in the application’s development life cycle. It is frequently valuable to record the acceptance criteria for your application and store them in a place and format that is available to the entire team for review and comment. Criteria are typically determined by balancing your business, industry, technology, competitive, and user requirements.

Test objectives frequently include the following:

Step 2 - Identify Key Scenarios

Scenarios are anticipated user paths that generally incorporate multiple application activities. Key scenarios are those for which you have specific performance goals, those considered to be high-risk, those that are most commonly used, or those with a significant performance impact. The basic steps for identifying key scenarios are.
  1. Identify all the scenarios for a Web application. For example, even the most basic e-commerce application must support the following user scenarios:
    • Browse catalog
    • Search for a product
    • Place an order
  2. Identify the activities involved in each of the scenarios. For example, a “Place an Order” scenario will include the following activities:
    • Log on to the application.
    • Browse the product catalog.
    • Search for a specific product.
    • Add items to the shopping cart.
    • Validate credit card details and place an order.
  3. Identify the scenarios that are most commonly executed or most resource-intensive; these will be the key scenarios used for load testing. For example, in an e-commerce application, browsing a catalog may be the most commonly executed scenario, whereas placing an order may be the most resource-intensive scenario because it accesses the database.
    • The most commonly executed scenarios for an existing Web application can be determined by examining the log files.
    • The most commonly executed scenarios for a new Web application can be obtained from market research, historical data, market trends, and so on.
    • Resource-intensive scenarios can be identified by using design documents or the actual code implementation. The primary resources are:
      • Processor
      • Memory
      • Disk I/O
      • Network I/O
Once they have been identified, you will use these key scenarios to create workload profiles and to design load tests.

Step 3 - Create a Workload Model

When defining workload distribution, consider the following key points for determining the characteristics for user scenarios:

Step 4 - Identify Target Load Levels

Identify the load levels to be applied to the workload distribution(s) identified during the previous step. The purpose of identifying target load levels is to ensure that your tests can be used to predict or compare a variety of production load conditions. The following are common inputs used for determining target load levels:

By combining the items above, you can determine the remaining details necessary to implement the workload model under a particular target load.

Step 5 - Identify Metrics

There is a virtually unlimited number of metrics that can be collected during a performance test execution. However, collecting too many metrics can make analysis unwieldy as well as negatively impact the application’s actual performance. For these reasons, it is important to identify the metrics that are most relevant to your performance objectives and those that you anticipate will help you to identify bottlenecks. Only well-selected metrics that are analyzed correctly and contextually provide information of value.

The following are a few suggestions for identifying the metrics that will provide the most valuable information to your project:

Additionally, to evaluate the performance of your application in more detail and to identify potential bottlenecks, it is frequently useful to monitor metrics in the following categories:

Step 6 - Design Specific Tests

Using your scenarios, key metrics, and workload analysis, you can now design specific tests to be conducted. Each test will generally have a different purpose, collect different data, include different scenarios, and have different target load levels. The key is to design tests that will help the team collect the information it needs in order to understand, evaluate, or tune the application.

Points to consider when designing tests include:

Step 7 - Run Tests

Poor load simulations can render all of the work in the previous activities useless. To understand the data collected from a test execution, the load simulation must reflect the test design. When the simulation does not reflect the test design, the results are prone to misinterpretation. Consider the following steps when preparing to simulate load:
  1. Configure the test environment in such a way that it mirrors your production environment as closely as possible, noting and accounting for all differences between the two.
  2. Ensure that performance counters relevant for identified metrics and resource utilization are being measured and are not interfering with the accuracy of the simulation.
  3. Use appropriate load-generation tools to create a load with the characteristics specified in your test design.
  4. Using the load-generation tool(s), execute tests by first building up to the target load specified in your test design, in order to validate the correctness of the simulation. Some things to consider during test execution include:
    • Begin load testing with a small number of users distributed against your user profile, and then incrementally increase the load. It is important to allow time for the system to stabilize between increases in load while evaluating the correctness of the simulation.
    • Consider continuing to increase the load and record the behavior until you reach the threshold for the resources identified in your performance objectives, even if that load is beyond the target load specified in the test design. Information about when the system crosses identified thresholds is just as important as the value of the metrics at the target load of the test.
    • Similarly, it is frequently valuable to continue to increase the number of users until you run up against the service-level limits beyond which you would be violating your SLAs for throughput, response time, and resource utilization.

Note: Make sure that the client computers (agents) you use to generate load are not overly stressed. Resource utilization such as processor and memory must remain well below the utilization threshold values to ensure accurate test results.

Step 8 - Analyze the Results

You can analyze the test results to find performance bottlenecks between each test run or after all testing has been completed. Analyzing the results correctly requires training and experience with graphing correlated response time and system data.

The following are the steps for analyzing the data:
  1. Analyze the captured data and compare the results against the metric’s accepted level to determine whether the performance of the application being tested shows a trend toward or away from the performance objectives.
  2. Analyze the measured metrics to diagnose potential bottlenecks. Based on the analysis, if required, capture additional metrics in subsequent test cycles. For example, suppose that during the first iteration of load tests, the process shows a marked increase in memory consumption, indicating a possible memory leak. In the subsequent iterations, additional memory counters related to generations can be captured to study the memory allocation pattern for the application.

Resources