pnp.gif

How To: Load Test Web Applications Using Visual Studio Team System

J. D. Meier, Prashant Bansode, Carlos Farre, Scott Barber

Applies to


Summary

This How To topic will show you how to create a load test using Visual Studio Team System to simulate workload with a mix of user scenarios and validate if the load test is simulating the workload realistically. The load test then will produce performance results, to validate if performance goals are being met with regards of the requirements and performance metrics.

Contents

Objectives

Overview

Realistic load testing is the process of simulating workload with user scenarios, ensuring realism of test design, so it can provide accurate performance data that yield results that will reproduce performance characteristics of production or projected business volume.

Scenario

A Web based application has the following characteristics extracted from production in IIS logs or with marketing projections:
User Scenarios% of Work distribution
Browse50
Search30
Place Order 20

Workload

Given the characteristics of the scenario, the following are the number of concurrent users needed to simulate the scenario.

Total Hourly Number of Users (One user is counted as the user performing a complete scenario one of the below, Browse, Search, Place order 10000
Session Average time - This is the average time among scenarios a user completes them.7.5 Minutes
Number of Sessions Per Hour - This is the number of scenarios completed in one hour per concurrent user60/7.5=8
Number of concurrent Users10000/8=1250

Performance Goals

The performance goals mandated by business are listed in table below:

ObjectivesPerformance goals
# Users Processed Hourly - This is the total number of users scenarios completed ( user sessions )10000 total completed scenarios, 50% Browse. 50000 Browse scenarios, 30% Search. 30000 Search scenarios, 20% Place Orders: 20000 Placed Orders scenarios
Orders TransactionOrder transaction should complete in less than 8 seconds
Failed TransactionsNo failed transactions will occurs
ProcessorProcessor time should be less than 75%

Before You Begin

Summary of Steps


Step 1 Create a Test Project

Create a test project that will have the 3 scenarios (Orders, Browse, Search) simulating the workload for the scenario above. Name your project; by default it will have an empty unit test. We will add Web Test and Load Test to this project in later steps.

Creating a Test Project

Step 2 Create a Web Test

In this step, you create 3 Web tests for 3 user scenarios (Browse, Search and Orders). The Web tests will be used in your Load Test. The Web test will simulate the workload characterization for the scenario.

Creating a Web Test

  1. In Solution Explorer, right-click TestProject1, select Add, and then select Web Test.

Step 3 Record Your Web Test

  1. The Web Test Recorder opens inside a new instance of Internet Explorer.

For this How to we are creating Web test by recording HTTP requests using the Web Test Recorder in a browser session, but you can also build Web tests manually using the Web Test Editor.

For this how to we will be recording a Web Test for all the scenarios identified in scenarios section:

Recording Your Web Test

  1. In the Address bar, replace about:blank with the URL address for the three scenarios ( Browse, Search and Orders) individually to create the 3 separate Web tests.
  2. When you press Enter, the page is processed and rendered in the browser window. The actions are recorded for subsequent playback when you run the test.
  3. Go through various steps to create your scenarios.

Step 4 Configure Your Web Test

Once your Web tests are created, configure them for the following items:

Data binding for data parameters

Set data binding in post or query string parameters to be replaced during runtime. For more information on data binding refer to How to data binding in Visual Studio Team System <<Add url>>

Think times

Setting the correct think times is important to replicate the user behavior. The average user session lasts 7.5 minutes, including the time to process the transactions in the server.
Make sure think time is correct adjusting the value to meet the user’s behavior as identified in scenario section. After the Web test is recorder right click on the Web request and enter the think times. Repeat this step for all Web requests of the Web tests (Orders, Browse, and Search). Leave all remaining values of the properties of the Web request as default.

LoadTestVS1.GIF

Create the Transactions on the Web requests

Create the transaction in the Web request or group of Web requests as identified in the performance goals section. In below example it needs to be created transactions around Orders processing with three Web requests: add to cart, shopping cart and Check out.
Right click on the Web request you want to start the transaction, and click insert transaction. In this case the start transaction will be Add to car page.

LoadTestVS2.GIF

Choose the first page and last page as the transaction range. In the example below three web requests are included in the transaction: AddToCart.aspx, ShoppingCart.aspx and Checkout.aspx. You could include one or several Web requests in one transaction.

LoadTestVS3.GIF

After the transaction is created, the tree view of the script will show the transaction around the Web request(s).

LoadTestVS3a.GIF

Step 5 Replay Your Web Test

In this step you replay the web test to ensure that the web tests are all passing.
  1. In the main window of Web test click the Run Test button in the left hand corner. Or you can right click the Web test and choose the option Run Web Test.
  2. The Web test will be run for a single user, and result displayed in the main window as shown in the figure.
  3. Analyze and make sure there are no errors in any of the requests.
LoadTestVS4.GIF

More Information

The purpose of replaying your Web test is not merely to ascertain if the Web Test passes or fails. Here are the things you might want to check before adding the web test to a Load Test.

Step 6 Add Your Load Test

In this step, you add the Load Test to the TestProject1 project. Subsequently we will add the Web Test to the Load Test, which will simulate the load while executing the Web test. You will then set various properties of your Load Test in subsequent steps.

Adding Your Load Test

  1. In Solution Explorer, right-click the TestProject1 project node. Select Add, and then click Load Test.
  2. The New Load Test Wizard starts.
  3. The Welcome page of the New Load Test Wizard is the first page.
  4. Click Next.

Step 7 Specify Your Scenario Settings

Specify name of your scenario. We are performing Load Test for Orders, Browse and Search.
  1. Set the Time Profile Think to Use normal distribution centered on recorded think times. This will use the recorded times from Step 3. Think times represent the time that a user would ponder a Web page before going on to the next page.

LoadTestVS5.GIF

More Information

The choice of Think Time Profiles depends upon intend of usage.
For more information on Think Time, please see "About Think Times" at http://msdn2.microsoft.com/en-us/library/ms184790(VS.80).aspx

Step 8 Specify Your Load Pattern Settings

In this step, you will set the load pattern, with the number of concurrent users as identified in the workload .section. The number of concurrent users should simulate the business volume for the scenario (total number of users – sessions in a given time; one hour for our scenario). They do not represent concurrent users, since they will be randomly executing the 3 user scenarios (Browse, Search, Place Orders), with the variance of session time duration.

Using a Constant Load Pattern

  1. Select the Constant Load radio button. Select 1250 users
LoadTestVS6a.GIF

Step 9 Add Your Web Test to Your Load Test

Add all the web tests identified in scenario section with profile workload distribution,(Place Orders 20%, Browse 30%, Search 50%).

Adding Your Web Test to Your Load Test

  1. In the Add tests to a load test scenario and edit test mix screen, Click the Add button.
  2. In the Add Tests dialog box that opens, in the Available tests list box, select the Web test created in previous steps and then click the right arrow to move it to the Selected tests list box.
  3. In the Add Tests dialog box, click OK
LoadTestVS8.GIF

Step 10 Specify Your Browser Mix

In this step, you add the browser mix by specifying types of browsers being used and distribution ratio for each type of browser.

Specifying Your Browser Mix

  1. From the drop-down list of browser types, select the IE6 browser to add to the mix.
  2. As our web site is internet facing web site we would be adding Netscape 6.0 as well, Click Add button from the drop down select Netscape 6.0.
  3. Adjust the distribution of IE user and Netscape as 88% IE and 12% Netscape (based on industry trends).
  4. Then click Next.
LoadTestVS9.GIF

Step 11 Add Your Network Mix

In this step, you add the network mix by specifying types of network, on which the application is accessed and distribution ratio for each type of network.

Specifying Your Network Mix

  1. As our Web Site is internet facing and users from various location are expected to use the Web site, it will be mix of DSL and dial-up connection.
  2. From the Network Type drop down, select the Cable / DSL 384k connection type.
  3. Click Add button and select the Dial-up 56K.
  4. Let the distribution be 70% and 30% each.
  5. Then click Next.
LoadTestVS10.GIF

Step 12 Specify Your Computers and Counter Settings

In this step, you will specify the performance counters to be used for capturing load test results, in order to validate your tests against the objectives identified in scenario section LoadTestVS11.GIF LoadTestVS12.GIF

Step 13 Specify Your Run Settings

In this step, specify set of properties which determine how the Load test runs. The run settings determine the length of the test, warm-up duration, maximum number of error details reported, sampling rate, validation level etc.

Specifying Your Settings

  1. On the Run Settings page choose your initial settings.
  2. Set the Warm-up duration as 5 minutes.
  3. Set Run duration 1 hour.
  4. Set Sampling rate as 5 seconds
  5. In the Description edit box specify the description of the Run Settings.
  6. Accept the other default settings.
  7. Click Finish. Your Load test is opened in the Load Test Editor.
LoadTestVS13.GIF

Step 14 Set The Threshold Values For The Metrics.

In this step you will set the threshold values for the metrics identified in the Performance Goals section. During the runtime of a load test the threshold violations will be flagged in the UI for test pass criteria analysis.

Failed Orders Transactions

Click on LoadTest under counter sets then right click on Failed Tests and select add threshold value.
LoadTestVS14.GIF
In the Threshold value window set alert if over to true and warning and critical threshold values to 1. True setting will flag a failure if value goes higher than critical values.
LoadTestVS15.GIF

Processor Time

Do the same procedure for Counter Sets->application->processor->%Processor time.
Set the warning threshold to 70% and critical value to 75%.
LoadTestVS16.GIF

Step 15 Run Your Load Test

In this step you run the Load test to see, how the Web application responds to the web test, under the load simulation.

Running Your Load Test

  1. In the Load Test Editor right click on the Load Test and select Run Test option.
  2. Click View menu in main window then select Full Screen, this will maximize the viewable area.
  3. Once the Load test is complete, it displays a message, "Load Test 'LoadTest1' is complete. The test data currently displayed only represents a portion of the available results. Would you like to view the detailed results from the load test result store?"
  4. Click Yes on the message.

LoadTestVS23.GIF

Step 16 Analyze Load Test Results

The number of browse completed scenarios is missing 37 to complete 5000 and the number or search completed scenarios is missing 7 to complete 3000. Also the number of failed tests is one. One Placed Order failed to complete.

LoadTestVS24.GIF
LoadTestVS28.GIF

Considerations

Additional Resources