Web Stress Testing

Matt Odhner
Program Manager
Application Center
Microsoft
 Topics to be Covered

Important web hosting statistics

What stress testing solves & common stress
testing issues

Web Application Stress tool demonstration

Understanding the report data

What a commercial stress tool needs

Application Center Test (ACT) demonstration

Visual Studio integration

The importance of bandwidth testing

Other ACT features

Capacity planning
 Web Hosting Statistics

Popular web sites receive over 40,000,000

requests per day

Web hosting is expected to be a $14.6 billion
industry by 2003

Newport Group study revealed 52% of web
applications failed to scale, 60% had no
performance testing, only 25% used a testing tool

Zona Research found that $4 billion a year is lost
due to slow and failed web pages
 Stress Testing Solves …

Stability issues - unexpected downtime and poorly

written COM objects

Performance problems – locate bottlenecks and
whether the application will handle peak loads

Capacity planning - how many machines are
needed to support usage

Plus, it avoids financial losses and ensures
customer satisfaction
 Stress Testing Locates Bottlenecks

Memory

Processor

Network

Hard disk

COM component
 Common Stress Test Issues

Invalid test platform

Invalid script

Thread safety issues with unstable server COM
components

Active Server Page script errors and GLOBAL.ASA
issues

Insufficient processor power and/or scripts are
too complex for the hardware
 Traditional Test Bed

Web server

Member Member
Stress Stress
Client Client
Controller
Stress
Client
 Approaches to Stress Testing

Confirm that the application functions under load

Find the maximum requests per second my
application can handle

Determine the maximum number of concurrent
connections my application can handle

Test the application with x number of unique
users
 Creating the Script

Scripts view - create, edit, delete, and run stress

scripts.
 Creating the Script (cont’d)

Record using your browser – this method is

recommended because all page elements are
captured.
Script Configuration - Settings

Concurrent
browser
connections
 Script configuration - Users

Users view - manage user names and passwords.

 Script Configuration - Clients

Clients view - add and remove the client machines

used to stress the web server(s)
Script Configuration - Script Item Details

Script item details – modify query string, edit

POST data, use SSL, query RDS.
 Performance Monitor Data

Collect performance monitor

counters from within the tool
 Scriptable Object Model

Sub AddNewScript()
Set objWAS = CreateObject("WAS.EngControl.1")
Set objScripts = objWAS.Scripts
Set objScript = objScripts.Add
objScript.sName = "A New Script"
objScript.ScriptItems.sServer = "www.targetserver.com"
MsgBox "The new ScriptID is " & objScript.ScriptID
End Sub
 Session Trace Option

=h= Selected new user: User2, password,

ASPSESSIONIDGQQGQNYG=MPBBLMOAFBPBPDFAJLMCCFCO; Account=6001
=h= closed socket
=h= linger is on, linger timeout is 0
=h= connected to the server. 172.30.170.217:80
=h= sending request to server (470 bytes)
GET /fmstocks/_NewAccount.asp HTTP/1.1
Host: localhost
ASP Client
 Reports – Tabular Report Data

Reports view - analyze the results of a test.

Time to First and Last Byte

0 ms
Request

6 ms 9 ms
Response

Time To Time To Last

First Byte Byte
 Calculating ASP Performance

MHz Cost = N * S * avg (Pt) / avg(Rps)

Where:

N = Number of Processors

S = Speed of Processors

Pt = System: % Total Processor Time

Rps = ASP: Requests per secondReports view -
analyze the results of a test.
 Calculating ASP Performance

Example: a test using a 2 processor web server

achieved 350 requests per second, with the
processors 70% utilized.

So, this works out to …

2 processors * 400 MHz => 800 MHz

70% processor utilization => (800) * (0.70)
=> 560 MHz used

350 ASP requests per second 560/350
=> 1.6 million cycles per ASP request
 More About Web Application Stress Tool

Designedto meet internal test needs

http://webtool.rte.microsoft.com contains:
–Searchable knowledge base
–Tutorial of main features
–Latest download
–Object model sample scripts

Peer support on the discussion alias:
webtool@microsoft.com

Online help contains samples

The tool respects ROBOTS.TXT
 What is Needed in a
Commercial Stress Test Tool?

All of the features in Web application Stress, Plus:

Accurate simulation of client connection

speeds

Running a script a specific number of iterations

Testing of servers in multiple domains

Handle complex applications that require
conditional responses

Capable of changing subsequent requests based on
the response of a previous request

Functional and performance testing

In-depth reporting and troubleshooting
 Microsoft
Application Center Test
 Significant Features

1. Integration with Visual Studio

2. IP router technology
3. Data collector to replay actual web site traffic
4. Completely scriptable, even as the test is running
5. Graphical and tabular reporting
6. Capacity planning
7. Can import Web Application Stress scripts
1. Visual Studio Integration
Visual Studio Integration (cont’d)
 Bandwidth Statistics

58% of web site users said download speed was a

primary factor that influences them to return
to a site (Forrester Research)

Web surfers give a site about 10 to 15 seconds to
display before they hit the stop button and go
to a different site

Bandwidth is not expected to improve soon
Wireless
6%
Cable
16%
Chart shows forecast
for the year 2003 ISDN
5%
Analog
xDSL
60%
13%
2. IP Router Technology

Web farm

T1 T3
56 K 28.8

Client computers
 IP Router Technology (cont'd)

Web
ACT Local Area cluster
Network

Delay prior to sending each packet so that

1,588 bits per second is maintained
IP Router Technology (cont'd)
 3. Data Collector

Internet traffic

Replays exact traffic

Client computers
 4. Completely Scriptable

If IsObject(Session) = False Then

Set Session = CreateObject("ACT.Session")
End If

Set oRequest = Session.CreateRequest

oRequest.Path = "/samples/cookie.asp"
oRequest.Headers.Add "Connection", "Close"
oRequest.Headers.Add "Cookie", "FirstName=Matt&Lastname=Odhner"
Set oConnection = Session.CreateConnection("MATTOD15")
Set fs = CreateObject("Scripting.FileSystemObject")
Set writeToFile = fs.OpenTextFile("c:\test.txt", 8, TRUE)
thePath = oRequest.Verb & " " & oRequest.Path & " " & _
oConnection.Send(oRequest).ResultCode & chr(13) & chr(10)
writeToFile.Write thePath
writeToFile.Close
Reporting - Tabular Reports
Reporting - Connections vs. TTFB & TTLB
Reporting - Comparing Scripts
 6. Capacity Planning

1. Manual capacity
planning
 Capacity Planning (cont'd)

2. Historical capacity planning

Internet

Filter Filter Filter

Web cluster
 Capacity Planning (cont'd)

3. Calibrated capacity planning

Web farm

Client computers
 Summary

Web Stress tools give answers to the problems of

Web application stress, via:

Modeling

Prediction

Identification of bottlenecked resources

Provision of performance Metrics to help
developers decide what content to deploy

So - employ one today to improve your web

application's performance!
 Resources - Further Reading

http://www.microsoft.com/TechNet/iis/sol.asp

http://msdn.microsoft.com/library/techart/windnami
stakes.htm

http://msdn.microsoft.com/workshop/server/asp/server1
02599.asp

http://msdn.microsoft.com/workshop/server/asp/server0
92799.asp

http://msdn.microsoft.com/workshop/server/asp/server1
22799.asp

http://www.microsoft.com/siteserver/commerce/support/
highcapacity.htm
 Resources - Other Web Stress Tools
Available

Mercury Interactive, Loadrunner/Winrunner -

http://www.merc-int.com/products/

Radview, Webload - http://www.radview.com/webload/

RSW, eLoad - http://www.rswsoftware.com/

Segue, Silkperformer -
http://www.segue.com/html/s_solutions/silk/s_performe
r.htm

Bluecurve, Dynameasure -
http://www.bluecurve.com/products/products.htm

Rational, Rational Suite / PerformanceStudio -

http://www.rational.com/products/rs/pstudio/index.jtm
pl
Questions?

Matt Odhner
Program Manager
Application Center
Microsoft