Title

Author
Version
Date
Purpose

Description

References

Version History
0.1

Performance Testing Analysis template for SharePoint Server 2010

Sanjay Narang (sanjay.narang@microsoft.com)
0.1
6/5/2012

This workbook provides templates for performance testing reporting and analysis for SharePoint Serve

Many times, when IT Professionals do performance testing some application, they are not sure what m
- what performance counters to capture from servers,
- what key configurations/counters to capture from VSTS
- how to record or report test results
- how to create create test mix
- When they should consider a value as problem and what should they do to troubleshoot the problem

This workbook helps IT professionals in these tasks by providing template as well as examples. Two wo
be extended to cover other workloads

Some of the material available here is available in Technet or other resources also, but this document
performance tests for SharePoint Server 2010. Also, this document should ONLY be used as a refernce

The workbook contains the following worksheets:

1. Performance Targets - For validating success of failure of application in the performance testing, you
2. Test Results - template for how test results can be captured and compared in different runs. The wo
You can add/remove counters based on your requirement. The is written based on Visual Studio 2010.
create copies of this worksheet.
3. Performance Counters - This worksheet provides list important performance counter that you should
as problem and what should be resolution steps.
4. Test Mix Example 1 and 2 - The worksheets provide examples of how to define a test mix for a perfo
5. Server Configs - It is very important that server configuration is recorded completely for any perform
6. Global Settings - There are many settings (infrastructure or application or farm level) that remain co
performance test results greatly. Hence it very important to record these settings. This worksheet prov

You should also refer the following resources:

1. SharePoint 2010 Performance (Stress, Load) Testing http://blogs.msdn.com/b/sanjaynarang/archive/
2. SharePoint (Performance, Stress ) Load Testing http://blogs.msdn.com/b/sanjaynarang/archive/2009
3. Performance testing for SharePoint Server 2010 http://technet.microsoft.com/en-us/library/ff758659
4. Monitoring and maintaining SharePoint Server 2010 http://technet.microsoft.com/en-us/library/ff758
5. Performance Monitoring Getting Started Guide (Applied to Windows Server 2008) http://technet.mic
6. Performance Tuning Guidelines for Windows Server 2008 R2 http://www.microsoft.com/whdc/system
7. Visual Studio Performance Testing Quick Reference Guide http://vsptqrg.codeplex.com/

Initial Draft

5-Jun-12

DESCRIPTION: The worksheet provides typical performance targets. You can add
change them as per your requirements
Category

Parameter

Latency

Server responsiveness

Latency

Client side response time

Latency

Server spikiness

Througput

Request Per Second (RPS)

Througput

Concurrent users

Reliability

CPU Utilization

Reliability

Available Memory

provides typical performance targets. You can add/change/delete based on your requirements. The
uirements
Description

The time it takes the farm to serve requests is a good indicator to track the health of the
farm. This indicator is usually named server side latency, and it is common to use the
average or median (the 50th percentile) latency of the daily requests being served. The
targets are commonly described in sub seconds or seconds. Note that if your
organization has a target to serve pages from SharePoint Server 2010 in less than two
seconds, then the server side goal needs to be sub seconds to leave time for the page
to reach the client over the network and time to render in the browser. Also in general
longer server response times are an indication of an unhealthy farm, as this usually as
an impact on throughput and rarely can RPS keep up if you spend more than a second
on the server on most requests
This includes:
1. Server response time
2. time taken in network (request as well as response)
3. time taken by browser to render page (Targets can be based on TTFB or TTLB)

Another good server side latency indicator worth tracking is the behaviour of the
slowest 5% of all requests. Slower requests are usually the requests that hit the system
when it is under higher load or even more commonly, requests that are impacted by
less frequent activity that occur while users interact with the system; a healthy system
is one that has the slowest requests under control as well. The target here is similar to
Server Responsiveness, but to achieve sub-second response on server spikiness, you
will need to build the system with a lot of spare resources to handle the spikes in load.

RPS is a commonly used indicator used to describe the demand on the server farm
expressed in the number of requests processed by the farm per second, but with no
differentiation between the type or size of requests.
The number of requests received by a farm or server in one second. This is a common
measurement of server and farm load. The number of requests processed by a farm is
greater than the number of page loads and end-user interactions. This is because each
page contains several components, each of which creates one or more requests when
the page is loaded. Some requests are lighter than other requests with regard to
transaction costs. In our lab tests and case study documents, we remove 401 requests
and responses (authentication handshakes) from the requests that were used to
calculate RPS because they have insignificant impact on farm resources.

It is most common to measure the concurrency of operations executed on the server

farm as the number of distinct users generating requests in a given time frame. The key
metrics are the daily average and the concurrent users at peak load.
Measured through performance counter - % Processor Time:
This shows processor usage over a period of time. If this is consistently too high, you
may find performance is adversely affected. Remember to count "Total" in
multiprocessor systems. You can measure the utilization on each processor as well, to
ensure balanced performance between cores.
Measured through performance counter - Available Mbytes:
This shows the amount of physical memory available for allocation. Insufficient memory
will lead to excessive use of the page file and an increase in the number of page faults
per second.

ur requirements. The targets provided here are only example targets and you should

Example target

Remarks

< 0.5 sec

You can also have different response time target for different
usage scenarios. For example, the home page must load in
less than 0.5 seconds, whereas its okay for other pages to load
in less than 1 seconds

< 3 sec

You can also have different response time target for different
usage scenarios. For example, the home page must load in
less than 0.5 seconds, whereas its okay for other pages to load
in less than 1 seconds

< 1.5 sec

You can also have different response time target for different
usage scenarios. For example, the home page must load in
less than 0.5 seconds, whereas its okay for other pages to load
in less than 1 seconds

1. While keeping the target, clarify whether that includes

dependent request OR NOT. Generally, dependent requests
(images, css, js) are NOT included in the targets. In most
documentation available on Technet, RPS figures do NOT
include dependent requests
Depends on
2. You can also have different RPS target for different usage
organization user load scenarios. For example, the home page must have 100 RPS,
and application
whereas its okay for other pages to have 20 RPS

As it is not possible to find exact count of concurrent users in

HTTP web applications, this counter is generally NOT
considered as target, and RPS is used.
However, if you need to use, you need to define what you
mean by concurrent users. Some define it as total number of
Depends on
unique users that sent requests to system in a particular span
organization user load of time - that span can vary as 1 second or 1 minute or 10
and application
minutes or 1 hour

< 70 %

> 2 GB (for web servers)

DESCRIPTION: Provides template for how test results can be captured and compar
required for analysing performance results. You can add/remove counters based on
cover only 1 type of load test. For additional load tests, you need to create copies of
Test Usage Settings
Name
Site URL
Test Mix
Total Content
Content Type

Servers
General Test Settings
Warm-up duration
Duration
Number of agents
Parse Dependent Requests
Percentage of New Users (First Time Visitors)
Think Time
Test Results
Max User Load
Tests Failed
Pages/Sec
Avg. Page Time (sec)
Requests/Sec
Requests Failed
Requests Cached Percentage
Avg. Response Time (sec)
Avg. Content Length (bytes)
System Under Test Resources
Number of Web Servers
CPU - DB
Memory at End - DB
CPU - WFE
Memory at End
SharePoint and System Settings
Cachin Enabled at F5 load balancer
Output Cache Enabled?
Output Cache (Deviation from default)
Object Cache Size (in MB)
Cross List Query Cache Duration (in seconds)
BLOB Cache Enabled
Logging - Events level

Logging - Trace level

Logging - Events level (Project specific)
Event Log Flood Protection - Enabled
ASP.NET
RPS
Request Execution Time
Request Queued
Pipeline Instances Count
Cache API Entries
Cache API Hit Ratio
Cache API Turnover Rate
Cache API Trims
Cache Total Entries
Cache Total Hit Ratio
Cache Total Turnover Rate
Output Cache Entries
Output Cache Hit Ratio
Output Cache Turnover Rate
SharePoint Publishing Cache
Publishing Cache Hits / sec
Publishing Cache Flushes / sec
Publishing Cache Hit Count
Publishing Cache Hit Ratio
Total Number of cache compactions
Total Number of cache flushes
Total Objects
NumberOfCaches
SharePoint Disk Based Cache
Blob Cache fill ratio
Blob cache flushes / second
Blob cache hit count
Blob cache hit ratio
Blob cache hits / sec
Blob cache misses / sec
Total blob disk size
Total number of cache compactions
Total objects in Blob Cache
Controllers and Agents
Test Agent 1 CPU
Test Agent 2 CPU
Test Agent 3 CPU
Test Agent 4 (Controller) CPU
Test Agent 5
Test Agent 6
Test Agent 7

Test Agent
Test Agent
Test Agent
Test Agent
Test Agent
Memory At End
Test Agent
Test Agent
Test Agent
Test Agent
Test Agent
Test Agent
Test Agent
Test Agent
Test Agent
Test Agent
Test Agent
Test Agent
RPS
Test Agent
Test Agent
Test Agent
Test Agent
Test Agent
Test Agent
Test Agent
Test Agent
Test Agent
Test Agent
Test Agent
Test Agent

8
9
10
11
12
1 Memory At end
2 Memory At end
3 Memory At end
4 (Controller) CPU
5 Memory At end
6 Memory At end
7 Memory At end
8 Memory At end
9 Memory At end
10 Memory At end
11 Memory At end
12 Memory At end
1 RPS
2 RPS
3 RPS
4 (Controller) RPS
5 RPS
6 RPS
7 RPS
8 RPS
9 RPS
10 RPS
11 RPS
12 RPS

results can be captured and compared in different runs. The worksheet also have most common counters that
u can add/remove counters based on your requirement. The is written based on Visual Studio 2010. This worksh
ad tests, you need to create copies of this worksheet.

Sanjay Narang:

Sanjay Narang:
HM0_Homepage
See worksheet Test Mix
Add a number as per y
Example 1 or 2 for how to
http://sitename/Pages/Home.aspx
usage scenario
define test mix for load tests
HM0_Homepage=100%
41 pages
Article Pages with average 2 images on SharePoint 2010 OOB Publishing Portal Site, average size 3100
Run 1: 6:48 PM 17th Dec
Run 2: 5:48 PM 31 Dec
<Add description of Run 2, what were key ch
<Add description of Run 1>
after Run 1>
Value
WFE1
WFE2
WFE3
Value
WFE1

2
10
10
0
100
0

2
10
9
0
100
0

50
0
150
0.33
150
0
0
0.33
33727

55
0
159
0.34
159
0
0
0.34
34571

2
34.6
28865
75.8
12340

2
41.4
13517
78.35
12225

76.4
12334

75.4 Not Used

12346

NO
Yes
None

NO
Yes
None
100
60

NO
Information

79.3
12239

100
60
NO
Information

Medium
Warning
Yes

Medium
Warning
Yes
1230
548.5
0
45
8.5
100
0
#DIV/0! NA
599
98.7
0.01
2
49.85
0

150
920
0
50
11
100
0

ASP .NET (Sparsh Instance)

159.5
159
173.5
180
0
0
36.5
35
7.5
6
100
100
0.00165
0.0033
0
0
572
579
99.85
99.8
0.095
0.17
1
2
99.75
99.7
0
0

150 Not Used

177
0
40
6
100
0
NA

783
98.7
0
3
99.7
0

415
98.7
0.02
1
0
0

1796
0
5200516.5
100
0
0
0
2

1791
0
5157009
100
0
0
0
2

1801
0
5244024
100
0
0
0
2

0
0
0
0
0
0
0
0
0

0
0
0
0
0
0
0
0
0

39366540
42944200
1309898.5
87
0
0
0
2

0
1910
1713050
100
0
0
0
2

Disabled
0
0
0
0
0
0
0
0
0

15.2
6.23 gent
6.34
14.8
NotUsed

8.17
7.3
7.5
14.5
NotUsed

6.48
6.54

8.16
8.35

6.62
7.8
7.87
6.79
7.76

6.5
7.46
7.25
7.33
7.45

3,069
3,318
3,347
396

3,063
3,098
3,106
246

NotUsed

NotUsed
1,977
3,006
2,697
3,287
3,286
3,226
3,240

1,513
2,623
2,275
3,311
3,319
3,328
3,359

15
15
15
0

20
20
20
0

NotUsed

NotUsed
15
15
15
15
15
15
15

20
21
17
17
17
17
17

common counters that are

tudio 2010. This worksheet can

Sanjay Narang:
Add a number as per your
usage scenario

Sanjay Narang:
Describe the type of pages
that you are testing

l Site, average size 3100

5:48 PM 31 Dec
Run 2, what were key changes
fter Run 1>
WFE2
WFE3

Sanjay Narang:
Value column is calculated differently
for different counters. For example, for VSTS counters, it's exact vallue
reported by VSTS,
- for ASP.Net RPS, it is sum total of RPS
on all web servers,
- for CPU, you should take average of
value on all web or app servers

Sanjay Narang:
You will add more columns if you have
more than 1 app servers or more than
1 DB server

77.4
12211

Not Used

P .NET (Sparsh Instance)

160 Not Used
167
0
38
9
100
0
0
565
99.9
0.02
0
99.8
0

Sanjay Narang:
In integrated mode, server
performance counters show
double number of the
actual RPS. Hence you need
to half it to get actual value

78733080
85886490
906747
74
0
0
0
2
0
0
0
0
0
0
0
0
0
Sanjay Narang:
It is important to monitor test
agents, as sometime, problem can
be at agent side and NOT at server
side