com
Architecture Overview
LoadRunner works by creating virtual users who take
the place of real users operating client software,
such as Internet Explorer sending requests using the
HTTP protocol to IIS or Apache web servers.
With Java clients, VuGen captures calls by hooking within the client JVM.
At the end of each run, the Controller combines its monitoring logs with logs obtained from load generators, and makes
them available to the "Analysis" program, which can then create run result reports and graphs for Microsoft Word,
Crystal Reports, or an HTML webpage browser.
Each HTML report page generated by Analysis includes a link to results in a text file which Microsoft Excel can open to
perform additional analysis.
Errors during each run are stored in a database which can be read using Microsoft Access
No additional license is needed to monitor standard web (HTTP) servers (Apache, IIS, and Netscape).
To use Web Services Monitors for SOAP and XML, a separate license is needed, and vUsers require the Web
Services add-in installed with Feature Pack (FP1)
Product Versions
Note: Links to documents that used to be
Version 9.10 installer, available Feb. 2008, is 2.31 GB after expansion. However, here were removed after Mercury Interactive,
the folder created after an English language install is 931MB. Inc. lawyers demanded their removal.
Page numbers in online pdf files are different
Version 8.1 Feature Pack 4 patch installer LR81FP4P136.exe at 7,786,800 bytes, (have more pages) than page numbers in the
paper document of the same title.
was signed on January 2, 2007 as file 8.1.4.0 (Build: 1735) is Recorder Version:
Although version 9.10 is now installed under
1290
1 of 22 9/23/2008 8:53 AM
LoadRunner Architecture http://onestopsoftwaretesting.com
signed on December 15, 2006 as file version 8.1.4.0 (Build: 2249) is Recorder These, the hidden folder C:\Config.Msi, the
Version: 1289. This requires an upgrade to MS.NET 2.0 clients. MacroVision folder (within Documents and
Settings\All Users.WINDWS\Application
Data), and many other files remain after
Version 8.1 Feature Pack 3 installer LR81FP3.exe, at 116,601,240 bytes, was uninstall.
signed on June 18, 2006 as file version 8.1.3.0 (Build 2085). It installs (as an item Over three thousand entries also remain
on your Start > Program Files) Microsoft WSE (Web Services Enhancements) 2.0 within the Windows Registry after uninstall.
SP3 to deploy security policies for sytems running .NET Framework 1.1.
Version 8.1 became available October 2005. In VuGen it adds a "Workflow View",
"Workflow Wizard", and a memory leak which is fixed with a patch downloadable
since Dec. 2005. It renames the VuGen "Execution Log" the "Replay Log".
Version 6.0 used a separate thread per user, which required almost 10 times more
i/o and CPU cycles than 6.5.
V8.0
Application Process File
V9.0 Img
Product Image Name Size
KB
Console programs
perl5.8.0.exe Interpreter 20,535
regtlb.exe registers the batch automation type library 30,720
sed.exe GNU sed (gsed) version 2.05 55,296
wdiff.exe Compares text files 197,632
Alex Arbitman's LR 7.8 Footprints.xls reports that to run Web requires __ per process and __ per thread.
2 of 22 9/23/2008 8:53 AM
LoadRunner Architecture http://onestopsoftwaretesting.com
system registry; however, the user session now active on the RDP-Tcp expert.
connection will not be changed."
Make sure that when you do this you're not violating one of your corporation's
security policies.
The LR box comes with two CD's and this installation manual. Separate installation manuals are available for the
Controller and Analysis modules.
The Windows CD autostarts to this initial screen for v7.8 and this initial screen for v8.0
You can install just a single component (such as VuGen) by (ironically) selecting "Full install" and then the
"Custom" option to check the specific components to install. However,
due to a strange bug with v8.0, before you do that, first install the Load Generator, then return to install
"custom" components.
The UNIX CD installs only the Load Generator (not the Controller or VuGen) on UNIX machines because the Controller
and VuGen only run on Windows machines.
Zero fill machine names to t001, ... t010, etc. The LR Controller sorts machines named t1, t2, ... t10 as t1, t10, t2.
If you get a "License violation" message, you need to get from HP support a one-day license key to install with.
Program Files (x86) is the default folder if you install LoadRunner on a 64 bit machine.
LoadRunner 8.1 and 9.0 executable files are installed to file path
"C:\Program Files\Mercury\LoadRunner\bin"
Note: Even though 8.0 uses a different folder, folders created by previous versions still need to be removed before
its installation.
When working with Java, instead of overridding these default installation folder to a path without spaces (such as
C:\LR78)
as recommended by KB article 11878, just use the equivalent DOS 8.3 file names:
To quickly get at this LoadRunner installation folder, create an environment variable named "LR91" so you can use
a quick command such as
cd %lr81%
To get at this folder quickly, I created a batch file named "L.bat" in the cmd's default C:\ root folder containing this:
cd \Program Files\Mercury\LoadRunner\bin
pause
3 of 22 9/23/2008 8:53 AM
LoadRunner Architecture http://onestopsoftwaretesting.co
pause
I created a shortcut to this file on my desktop and dragged it over Windows so that I can click into that folder
from anywhere. The pause command ensures that the command window does not disappear automatically.
Alternately, from within a command window I can just type "L" and press Enter.
The C:\WINNT (or C:\Windows) folder also holds the Maintenance Number (MPN) specified during installation, stored
as a parameter named "LoadRunner_SerialNumber" (such as 1234-1234567890) in the mercury.ini file.
Start Menu
Since LR 9.0, installers added links to the most used programs in
However, some programs are installed which are not conveniently listed there, such as
WDiff.exe v1.49 to compare differences between two ASCII text files. It has an accompanying help file
Beginning with v7, LoadRunner prevents software piracy (much like Microsoft began doing with Windows XP) by
requiring that a license key be provided within 10 days of installation. Mercury generates its license key based on a host
ID generated on each computer. CPT12784.doc
With v7.x, to generate Generate a HostID key (such as "XCCWJU-APBE-BYDS") click down
The key can be obtained before installation from program licidgenerator.exe and (after registration) its lm70.dll from
the installation CD folder \lrunner\lm70.nt\bin or \lrunner\setup\lm70.nt\bin.
Copy the link location below into the "Program to record:" field:
Protocol Server Client Program Parameter Notes
Web WebTours\StartServer.bat http://localhost:1080/mercuryWebTours
COM/DCOM (Operating System) samples\bin\frsui.exe
Winsock sockfrs.exe samples\bin\flights.exe Winsock
WinSockWeb
ODBC (MS Access) samples\bin\flights.exe ODBC_Access
CORBA samples\CorbaSamples\server.cmd samples\CorbaSamples\client.cmd & Stuart
& samples\Corbasamples\clientrecord.cmd Moncrieff's
samples\CorbaSamples\server.bat article on
CORBA
RMI samples\RMISamples\server.cmd & samples\RMISamples\client.cmd &
samples\RMISamples\server.bat samples\RMISamples\clientrecord.cmd
According to CPT11877.doc, JDK 1.5 users need to contact Mercury Support for a patch to each specific
LoadRunner version (7.6, 7.8 FP1 or 8.0). Otherwise, you'll get these messages:
Error: Failed to find javac.exe Java Compiler in Path and JDK installation folder in registry. [MsgId: MERR-22981]
Error: Failed to compile the Actions.java file. Please add the \bin to the path and try again. [MsgId: MERR-22996]
Warning: Extension java_int.dll reports error -1 on call to function ExtPerProcessInitialize [MsgId: MWAR-10485]
Error: Thread Context: Call to service of the driver failed, reason - thread context wasn't initialized on this thread. [MsgId:
MERR-10176]
The Java sample apps use the "flight32lr" User Data Source with Microsoft Access driver(*.mdb) in the USER DNS table
in Data Sources(ODBC) of the VuGen's local machine.
Additionally, the sample Java servers must be operational prior to starting the client. This is done with the
"samples\RMISamples\server.cmd":
set lrpath=C:\PROGRA~1\Java\jre1.5.0_02\bin;C:\PROGRA~1\MERCUR~1\MERCUR~1\classes
set
lrclasspath=C:\PROGRA~1\MERCUR~1\MERCUR~1\classes;C:\PROGRA~1\MERCUR~1\MERCUR~1\classes\srv;C:\PROGRA~1\Java\jre1.5.0_02\lib\rt.jar
set flightRmi=%~dp0;%~dp0RmiSamples.zip
set classpath=%lrclasspath%;%flightRmi%;C:\PROGRA~1\Java\lib\rt.jar;.;%classpath%
set path=%lrpath%;.;%path%
cd %~dp0
start java -Djava.security.policy="%~dp0RmiFlights.policy" RmiFlights.Server
Note the location of loadrunner class files I added to the default sample. They are pre-pended to the existing
classpath.
N t th t th i th fil th
4 of 22 9/23/2008 8:53 AM
LoadRunner Architecture http://onestopsoftwaretesting.com
Do not delete the black command window because the Java server runs within it.
CORBA and RMI Java clients are invoked with a command for Windows to start the java.exe program. This
"samples\RMISamples\client.cmd" file contains:
set lrpath=C:\PROGRA~1\Java\jdk1.5.0_02\bin;C:\PROGRA~1\MERCUR~1\MERCUR~1\classes
set
lrclasspath=C:\PROGRA~1\MERCUR~1\MERCUR~1\classes;C:\PROGRA~1\MERCUR~1\MERCUR~1\classes\srv;C:\PROGRA~1\Java\jdk1.5.0_02\lib\rt.jar
set flightRmi=%~dp0;%~dp0RmiSamples.zip
set classpath=%lrclasspath%;%flightRmi%;C:\PROGRA~1\Java\jdk1.5.0_02\lib\rt.jar;.;%classpath%
set path=%lrpath%;.;%path%
cd %~dp0
Note that the RmiFlights.main class file name is passed into java for it to load.
When recording Java with VuGen, a different command — such as the sample clientRecord.cmd — needs to be
invoked because VuGen needs to be invoked within the JVM sandbox:
set flightRmi=%~dp0;%~dp0RmiSamples.zip
set classpath=%flightRmi%;%classpath%
cd %~dp0
start InvokeVugen.exe
The location of the JDK needs to be specified in the Windows PATH environment variable PATH to avoid this
message:
Error: Failed to find javac.exe Java Compiler in Path and JDK installation folder in registry. [MsgId: MERR-22981]
VuGen "Java Vusers" can only operate as Single Vuser mode (not multi-vuser).
Instead of web "Start recording", Java VuGen scripts invoke Java functions within the Actions section.
"vuser_init" and "vuser_end" actions are not relevant within Java VuScripts.
Internally, the cjhook.ini file specifies which Java classes can hook in its [EXC_SYSTEM_CL] section. Java classes
specified in the [SYSTEM_CL] section are not hooked.
The user.hooks file in LR \bin folder is a general format and cannot be used as-in. It needs to be copied.
Uninstall
To uninstall LoadRunner, you must be logged in with the same Windows userID as was used during original
installation! If you use a different userid, the uninstall will delete only the dat folder which contains the "miuninst" file.
Unlike Microsoft Office applications, Mercury has not programmed invididual components to be selectively uninstalled on
its own.
5 of 22 9/23/2008 8:53 AM
LoadRunner Architecture http://onestopsoftwaretesting.com
Runs can be invoked to run "silently" by invoking Mdrv.exe from a Windows batch script.
Mdrv can automatically stop loading Vusers because they communicate with Vusers and monitor CPU usage on
Windows Load Generator machines.
A separate JVM is instantiated by each Java-based Vuser on Windows-based machines. Java Vusers are not
supported on Unix platforms.
3. virtual Vusers are invoked as groups (logical collection of virtual users running the same script on a specific load
generator machine)
4. by agents (3,900K magentproc.exe) running as a service or as a process
5. on load generator client machines.
9. Upon a pre-set delay, the Scheduler running on a Controller machine instructs agents (via Windows port 54345 or
dynamic Unix port) to initiate test session scenarios. The Controller (wlrun.exe) sends a copy of scenario files along
with the request.
10. Agents are launched by the Remote Agent Dispatcher process (formerly called Remote Command Launcher (RCL)) on
each load generator machine.
11. Each agent refer to scenario (.lrs) definition files to determine which Vuser groups and scripts to run on host
machines.
This means the Controller can be started from a DOS batch (.bat) file (preferrably with a short name on a root
drive):
Including the -Run parameter is the same as manually pressing the "Start Scenario" automatically upon
invocation. This is not a good idea because you may have to decide about collating the file from a previous run or
want to change the output folder.
This assumes that the system's environment PATH variable was updated to include where LoadRunner is
installed.
12. The Controller is invoked using parameter values within files in the Windows OS folder (WINNT for Windows 2000 and
WINDOWS for Windows XP). The Windows folder is used because LoadRunner is designed to have only one instance of
Controller running at a time on a machine.
To quickly switch among several applications, save a copy of LoadRunner's ini files after working on it within
the Controller, then use Notepad to craft a batch file to copy application-specific versions of ini files before
executing wlrun. An example of file copy actions for application XXX:
copy %LRDir%/config/wlrun7-XXX.ini %LRDir%/wlrun7.ini
copy %LRDir%/config/wlrun7-XXX.dft %LRDir%/wlrun7.dft
Prior to v9.0:
copy %WinDir%/wlrun7-XXX.ini %WinDir%/wlrun7.ini
copy %WinDir%/wlrun7-XXX.dft %WinDir%/wlrun7.dft
In the wlrun7.ini file file [output] section, MaxNumberOfOutputMessages= from 10000 to 100000 for long runs.
This limits the number of output messages stored in the database.
MaxOutputUIRowsToShow limits the amount of messages/errors (lines) displayed in the Controller's Output
window.
In the QTWeb.lrp file within the LoadRunner Program Files dat\protocols folder section [Vugen], add entry
MaxThreadPerProcess=5 to limit the number of threads managed by each load generator mdrv.exe process.
Values for DefaultScenarioDir, DefaultScriptDir, DefaultResultDir, and [Recent File List] stored in the wlrun5.ini
and wlrun7.dft files updated whenever values are changed within the Controller.
VuGen stores and retrieves a vugen.ini file in the Windows folder. When using Java, enable additional debug
options:
[DynaDlg]
6 of 22 9/23/2008 8:53 AM
LoadRunner Architecture http://onestopsoftwaretesting.com
JavaLevel=3
Vu scripts can be coded to use variable values obtained from parameter files external to the script.
16. Within each results folder, a "Log" folder is automatically created to contain a log file for each group. After a run, to
view a log file from within the Controller, click then right-click on a group to select "Show Vuser Log".
18. After a run, the "collate" process takes .eve and .lrr result files and creates in the results folder a temporary .mdb
(MS-Access ) database.
To prevent errors when processing large result files, use MSDE (Microsoft SQL Desktop Engine). Don't install it from the
Add-in folder on the LoadRunner 7.8 CD, which is obsolete SQL7. Download MSDE 2000 Release A which includes MSDE
2000 Service Pack 3a and MDAC 2.7 SP1a for use by Analysis on any Windows machine. Extract the file and share that
folder. Open a command window to run a command such as:
setup SAPWD="StrongPassword" INSTANCENAME="LR" SECURITYMODE=SQL DISABLENETWORKPROTOCOLS=0 /L*v path to log file
Using Windows Explorer, share the Data folder.
Then in Analysis Options > Database tab, use 8.3 names without spaces (indentified with DOS command DIR /X):
25. Optionally, Mercury's Remote Performance Monitoring (RPM) MS-IIS/ASP web server for LoadRunner 7.8 can be
installed on a Windows 2000 server ( but not on a Windows 2003 server) so that
26. load test results to be viewed using a web browser.
Not pictured is the LoadRunner Tuning Module (a separate $50,000 product). Mercury's "Get Ready" white paper
selecting "Allow virtual users to run on this machine without user login"
means the LoadRunner agent will be run as a SYSTEM service named
"LoadRunner Agent Service". From within Window's Perfmon, this image is
named magentservice.exe.
For better security, specify a separate service account userid and
password so that permissions for it can be limited.
After installation, to tell if it's running , bring up the Windows Services list:
selecting "Manual log in to the Load Generator " means the LoadRunner agent
will be run as a process named magenproc.exe. This approach means you
need to invoke the Load Generator from LoadRunner's \launch_service\bin folder
every time you boot-up that machine:
magentproc.exe
You can tell it's running by the "satellite dish" icon in the Windows task bar at
the lower right corner of your screen.
7 of 22 9/23/2008 8:53 AM
LoadRunner Architecture http://onestopsoftwaretesting.com
As a process, the Windows operating system constrains the agent service from
running GUI (WinRunner, QuickTest Professional, etc.) or GUI-like scripts (Citrix,
SAPGUI, etc.). Therefore, GUI and SAP Vusers cannot run if the Remote Agent
Dispatcher is installed on the Load Generator machine as a service rather than as a
process.
cd \PROGRA~1\MERCUR~1\MERCUR~1\ LAUNCH~1\bin
magentservice.exe -remove
magentservice.exe -install
These commands do not result in response messages. But they do put the
m_agent_attribs.cfg file in the load generator's C:\ root folder.
If you do not have Administrator rights and try to change the UserID: Admin and
Password: Admin, you will see message ERROR: "29972:- Failed to reset launcher status
call back function reason:no monikor was passed.
Repeat these commands if you get message "Failed to connect to the agent. Load
Generator not responding after timout Command line that was executed:"
If this still doesn't result in "Ready" Status, go to Windows Services on the load
generator client machine and kill, then remove the "LoadRunner Agent Service"
before repeating the above commands.
1. Logon the load generator as the user the load generator will use
2. Open Windows Explorer and under Tools select Map a Network Drive and
create a drive.
It saves time and hassle to have consistent drive letters across load
generators, so some organizations reserver certain drive letters for specific
locations.
3. Open the LoadRunner service within Services (accessed from Control Panel,
Administrative Tasks),
4. Click the "Login" tab.
5. Specify the username and password the load generator service will use. (A
dot appears in front of the username if the userid is for the local domain).
6. Stop and start the service again.
By default, the LoadRunner Controller uses TCP port 50500 to send data to TCP
port 54345 on the Windows Load Generator.
The Load Generator sends information back via a dynamic port. through the MI
Listener.
8 of 22 9/23/2008 8:53 AM
LoadRunner Architecture http://onestopsoftwaretesting.com
the dat folder where LoadRunner is installed. The [FireWall] section should
be changed to contain "FireWallServiceActive=1". This should cause the tiny
stoplight to turn green in the agent icon on the toolbar.
br_lnch_server.cfg
MoFW communicates with the MI Listener through port 443, so you can't have any
web servers (like Apache WebTours, IIS, or Oracle HTTP servers) running on both
the machines.
Verify whether port 443 actually allows communication by running command and
substituting the ip address in:
When defining a "remote" load generator from within the Controller, click "Details"
for the "Load Generator Information" dialog, where you can click the "Firewall" tab
and check "Enable Firewall".
File Locations
I recommend that you store all files to a drive letter you created (by mapping to
a folder). This makes it easier to override LoadRunner's defaults initially and also
easier to remain consistent when you change machines and drives over time,
especially when several testers use LoadRunner.
By default, .htm and resource files captured during a script's development are
stored in a data folder created for each script under folder X:\Program Files \ Mercury
Interactive \ LoadRunner \ scripts .
Each script's run-time settings are stored in that script's .cfg file within each Vuser
script's directory for use by both VuGen.exe and the LR Controller.
VuGen also stores and retrieves Windows Registry key Test Results \ Recent File
List the location of every .qtp file generated.
The maximum number of scripts displayed in the Available Scripts list is stored in
Registry key HKEY_CURRENT_USER \ Software \ Mercury Interactive \ RecentScripts \
max_num_of_scripts
Each Vuser can apply a different weighting to iterate some action files more
frequently than others.
9 of 22 9/23/2008 8:53 AM
LoadRunner Architecture http://onestopsoftwaretesting.com
y , ( g , , ) p
reside in that script's folder. I like leaving it there rather than moving it to the
common root shared by all scripts because VuGen automatically copies those files
when you save or import a script. Plus, changing the data structure of a common
file disables previous versions of the script.
I like random access to data files because it's a easy way to test multiple values
from within VuGen — I just run the script again. However, since VuGen always uses
the first value when Sequential access is specified, to exercise multiple values I
have to either change the data file or run the script again within the Controller.
Browsers
Browser Type Browser Version Platform
Microsoft Internet 7.0, 6.0, 5.5, 5.01, 5.0, 4.0 HP-UX, Unix, SunOS, Win32, Windows, WinNT,
Explorer WinXP
Mozilla Firefox 1.5.0.6, 1.5.0.4, 1.5, 1.0.7, 1.0.6,
1.0
Netscape Navigator 6.2, 6.0, 4.76, 4.6, 4.5, 4.0
KB article 11367 notes that the _t_rep.eve events file contains binary Vuser
and rendezvous info. There is a localhost_1.eve host event file for each
agent host. Their first line contains:
28 11 1018300521 2 4627103 22735576 5481115"
The 11 in the example above is the event code to start the scenario.
The 1018300521 is the start time.
KB article 22538 notes that if this is not the same as the start time in the .lrr
file, a "Fatal error" will occur when opening an Analysis file.
If the lr_end_transaction command does not use exactly the same name as its
corresponding lr_start_transaction name, all subsequent actions will be under that
transaction.
If an Web SSL script works in the Controller but not vuGen, try changing the
controller's Runtime Settings Preferences to "WININET replay instead of Sockets".
10 of 22 9/23/2008 8:53 AM
LoadRunner Architecture http://onestopsoftwaretesting.com
Controller Scenarios
Scenarios are specified within the Controller
Manual scenarios can distribute the total number of Vusers among scripts based
on the analyst-specified percentage (evenly among load generators).
C.
Menu Setting A. Speed B. Contention D. Longevity
Overload
When scheduled to Run until completion, the Quantity for a Scenario is the number of vusers running one at a
time.
When scheduled to Run for a period of time, the Quantity for a Scenario is the number of vusers running
simultaneously.
The time specified begins after the Ramp Up period, when all vusers have entered Run state. This specified time plus
the time it took to get all vusers into Run state becomes the total Elapsed Time of a time-limited run.
To run a specific number of vusers simultaneously, set the Parameter file in the script (in VuGen) to Abort after
reaching the end of file.
Scenario Settings
Scenario Run Time Settings make use of Run logic scripts plus:
Pacing of how soon to start a new iteration (__ seconds after the previous iteration ends or at certain intervals).
Logging - AdvancedTrace
Think Time
Miscellaneous Continue on error? Fail open transactions on lr_error_message
Under Network Speed Simulation of bandwidth, selection of anything other than "Maximum available" requires a
separate WAN Emulator license purchase to mimic WAN/Internet by slowing, dropping, and changing packets
between client and server. With the license, you can choose custom, or these rated speeds:
14.4 Kbps (Analog modem)
28.8 Kbps (Analog modem)
56 Kbps (Analog modem)
64 Kbps (ISDN)
128 Kbps (Dual ISDN)
512 Kbps (DSL)
Browser Emulation: Simulate browser cache?
Internet Protocol of Proxy or use of ContentCheck application
11 of 22 9/23/2008 8:53 AM
LoadRunner Architecture http://onestopsoftwaretesting.com
The Controller saves these settings in the .lrr file specified in the Results Settings dialog.
Advanced Trace
If Run-Time Settings has the "Advanced Trace" checkbox selected before the script is run, these lines will appear in the
output log:
UTC (GMT) start date/time : 2005-08-02 02:31:30 [MsgId: MMSG-26000]
LOCAL start date/time : 2005-08-01 21:31:30 [MsgId: MMSG-26000]
Local daylight-Savings-Time: Yes [MsgId: MMSG-26000]
Some of the Run-Time Settings: [MsgId: MMSG-27142]
Run Mode: HTML [MsgId: MMSG-26845]
Download non-HTML resources: Yes [MsgId: MMSG-26845]
Verification checks: No [MsgId: MMSG-26845]
Simulate a new user each iteration: Yes [MsgId: MMSG-26845]
Non-critical item errors as warnings: Yes [MsgId: MMSG-26845]
WinInet replay instead of Sockets: No [MsgId: MMSG-26845]
HTTP version: 1.1 [MsgId: MMSG-26845]
Keep-Alive HTTP connections: Yes [MsgId: MMSG-26845]
Max self Meta refresh updates: 2 [MsgId: MMSG-26844]
No proxy is used (direct connection to the Internet) [MsgId: MMSG-27171]
DNS caching: Yes [MsgId: MMSG-26845]
Simulate browser cache: Yes [MsgId: MMSG-26845]
Cache URLs requiring content (e.g., HTMLs): Yes [MsgId: MMSG-26845]
Additional URLs requiring content: None [MsgId: MMSG-26845]
Check for newer versions every visit to the page: No [MsgId: MMSG-26845]
Page download timeout (sec): 120 [MsgId: MMSG-26844]
Resource Page Timeout is a Warning: No [MsgId: MMSG-26845]
ContentCheck enabled: Yes [MsgId: MMSG-26845]
ContentCheck script-level file: "C:\...\LrwiAedScript.xml" [MsgId: MMSG-26842]
Enable Web Page Breakdown: No [MsgId: MMSG-26845]
Enable connection data points: Yes [MsgId: MMSG-26845]
Process socket after reschedule: Yes [MsgId: MMSG-26845]
Snapshot on error: No [MsgId: MMSG-26845]
Define each step as a transaction: No [MsgId: MMSG-26845]
Read beyond Content-Length: No [MsgId: MMSG-26845]
Parse HTML Content-Type: TEXT [MsgId: MMSG-26845]
Graph hits per second and HTTP status codes: Yes [MsgId: MMSG-26845]
Graph response bytes per second: Yes [MsgId: MMSG-26845]
Graph pages per second: No [MsgId: MMSG-26845]
Web recorder version ID: 5 [MsgId: MMSG-26844]
Log Files
One of the most common headaches with load testing is running of hard disk space
during a long run. When log files grow beyond a megabyte,
Notepad cannot open the file. So you need
a text editor program that does not try to
If a transaction fails, search the load generator machine for log files containing load the entire file into memory before
text such as displaying it, such as Freeware Edit Pad
Lite from gsoft or HexEdit
xxx" ended with "Fail"
Efficient Search from Sowsoft works
where xxx is the last few characters of the transaction name. faster than Microsoft's search, and
retrieves your specifications it stores in the
Windows Registry.
LR vuGen stores logs (for a "null" user) in the "output.txt" file.
Controller Schedules
One minor irritation with the Schedule UI is that different words are used for selections are summarized. Ramp
down is not shown at all on the summary page.
Groups exists with a scenario. So "Scenario Scheduling" involves changing settings for all groups.
One way to determine appropriate ramp up time to specify is to set vusers to start simultaneously, then look at the
resulting rate Running Vusers drop off after processing (such as 10 users within a 15 second span).
12 of 22 9/23/2008 8:53 AM
LoadRunner Architecture http://onestopsoftwaretesting.com
Merging Graphs
In both the Run chart and Analysis graphs, I prefer to merge into a single graph "Response Time" and "Running
Vusers" and/or "Number of Connections".
"Hits per Second" — the number of hits on the Web server (y-axis) as a function of the elapsed time in the
scenario (x-axis). This graph can display the whole scenario, or the last 60, 180, 600 or 3600 seconds. You can
compare this graph to the Transaction Response Time graph to see how the number of hits affects transaction
performance.
"HTTP Responses per Second" — the number of HTTP status codes, which indicate the status of HTTP requests, for
example, "the request was successful," "the page was not found" returned from the Web server during each
second of the scenario run (x-axis), grouped by status code.
"Pages Downloaded per Second" from the server during each second of the scenario run. This graph helps you
evaluate the amount of load Vusers generate, in terms of the number of pages downloaded. Like throughput,
downloaded pages per second is a representation of the amount of data that the Vusers received from the server
at any given second.
"Pages Downloaded per Second" from the server during each second of the scenario run. This graph helps you
evaluate the amount of load Vusers generate, in terms of the number of pages downloaded. Like throughput,
downloaded pages per second is a representation of the amount of data that the Vusers received from the server
at any given second.
Total Transaction per Second (Passed)
Total Transaction per Second (Failed)
Connections + SSL
LR does not remember most scenario graph settings (4 graphs is the hard coded default). So instead of building
graphs from scratch, I start from opening and then changing my custom but standard scenario file.
Delete graph definitions you don't need to ever see. LR collects data for graphs even if it is not displayed.
KB 26817: By default, the Controller online monitor shows a maximum of 20 measurements for each graph. To increase
it, go to the LoadRunner\dat\online_graphs directory to modify the value of MaxDispMeasurments= in the file
controlling each type of graph:
Default counters for the System Resource, Microsoft IIS, Microsoft ASP, or SQL Server monitors are defined in the
res_mon.dft file within the LoadRunner/dat folder. Its values can be pasted from the [MonItemPlus] section within
scenario .lrs files.
UNIX Resources and some other graphs are continuously updated even after the test is done. So immediately
after the scenario runs, right-click on the graph to freeze the values displayed to lock in values associated with other
graphs.
Installation tip: If the Web Resource Graph is blank, try re-registering .dll files by running MS-DOS Batch Files
resister_controller.bat and set_mon.bat in the LoadRunner\bin folder.
Actions are executed sequentially in the order shown in Run-time Settings. However, transactions that start and
stop between two observations will appear to be running simultaneously even if they were actually executed
sequentially. An observation interval of 4 seconds is the most often that you can set for Controller on-line graph (to
prevent too much CPU-intensive graphic refresh time from consuming the Controller machine). But Analysis reports will
show more granularity than on-line graphs — down to 1 second.
13 of 22 9/23/2008 8:53 AM
LoadRunner Architecture http://onestopsoftwaretesting.com
Called
Template
Service Protocol/Product Header Notes
folder
[Graph]
Java General-Java -
Baan baan -
Distributed J2EE, - -
Components [Java CORBA-Java
Performance]
[Application Microsoft COM/DCOM, COM+ com -
Component]
.NET CLR dotNet -
14 of 22 9/23/2008 8:53 AM
LoadRunner Architecture http://onestopsoftwaretesting.com
g , ,
VoiceXML voiceXML -
WAP wap -
Legacy Terminal 327x (IBM mainframe) UNIX RTE (Remote Terminal rte lrrte.h
Service Emulation)
Application F5 BIG-IP - -
Traffic Management
The free Mindcraft WebStone 2.5 benchmark improves on the 1995 version originally from Silicon Graphics by also
simulating the activity of 100's of web clients on a computer making GET calls to CGI and server API as well as static
HTML pages. Its run rules currently does not support POST, SSL, Authentication, HTTP 1.1, HTTP 1.0 keep-alives,
Cookies, dynamic workloads with database access.
Sample Test Results of runs using Web Server Stress Tool from network monitoring company Paessler
Web Performance
The TPC (Transaction Performance Council's) TPC-W Web eCommerce benchmark (first announced July 2000 & with
v1.8 published Feb 2002) measures the number of Web Interactions processed Per Second (WIPS) from a "Web
Interaction Mix" of Shopping (WIPS), browsing (WIPSb) and ordering (WIPSo) transactions simulating a retail bookstore
with 14 web pages, including shopping cart functionality. The top audited price/performance result on 01/28/02 is a
TCO-based range of US$24.50 - $277.08/WIPS using IIS5 & SQL2000 within Windows 20003AS on a Dell server.
Audited performance characteristics are detailed by web server, web cache, database server, and image server. The
benchmark measures scalability by providing a Remote Browser Emulator (RBE) executable that (without client caching)
simulates 2880 different users accessing databases at various scales (10,000 or 100,000, etc. unique product items
within a schema of 8 tables) on database servers with 2 to 4 gigabytes of memory (with a 30 second non-SSL cache
time-out). So TPC-W requires a network topology that supports several 100 Mbytes/sec of data. User think time is
based upon a distribution with an average of 7 seconds and a maximum of 70 seconds. Article by Wayne D. Smith,
Intel Corporation
Loadrunner obtains values for these counters from issuing http://server /server-status?auto
The number and description of Apache counters are specified in LoadRunner's \dat\monitors\apache.cfg file.
15 of 22 9/23/2008 8:53 AM
LoadRunner Architecture http://onestopsoftwaretesting.com
p p p \ \ \ p g
This file should be replaced with apache_2.0.x.cfg if you are using Apache v2.
Network and Server response times are measured by running non-GUI Vusers on the client machine.
GUI file download response times are calculated by comparing GUI end-to-end times vs. Non-GUI total times.
GUI client display (paint) times are obtained manually (with a stopwatch).
Middleware to Server response times are obtained only by running Vusers issuing the separately licensed
middleware API to the app server.
Individual server response time are obtained by running Vusers on a machine directly connected to the app
server (not just on the same subnet together).
Difference in Transaction Time from several runs allow different loads to be compared in a Transaction Analysis
graph.
Each transaction measure performance of one or more action steps. [Workbook 3-17]
In Transaction Performance graphs, "Top time" transaction results point to bottlenecks in the system in that they,
irrespective of system load -- require significantly more than the average time to complete.
16 of 22 9/23/2008 8:53 AM
LoadRunner Architecture http://onestopsoftwaretesting.com
Web Page Breakdown graphs are available in Analysis only if before the run, in LR8 Controller Diagnostics menu
Distribution... dialog, BOTH the "Enable the following diagnostics" and the "Web page Diagnostics" checkbox are
checked.
AND they are added in the Analysis program.
This explains the default order of Page Download Time Breakdown graph measurements:
Infrastructure
Protocol Metric Measurement Description
Technologies
HTTP/S [Client Time] the average amount of time that passes while a request is delayed on the -
client machine due to browser think time or other client-related delays. This
does not include time for Flash to paint graphics (which takes many seconds).
[Connection is the time needed to establish an initial connection with the Web server -
Time] hosting the specified URL. This gives a good indicator of problems along the
network. It also indicates whether the server is responsive to requests.
[DNS time needed to resolve the DNS name to an IP address. If the hosts file -
Resolution contains the IP/host name pair under test, this should be very quick.
Time] Otherwise, the DNS server specified for the TCP/IP Properties is used. DNS
Lookup measurement is a good indicator of problems in DNS resolution, or
problems with the DNS server.
[Error Time] the average amount of time that passes from the moment an HTTP request is -
sent until the moment an error message (HTTP errors only) is returned
[First Buffer time that passes between when the initial HTTP GET/PUT request until the -
Time] first buffer (8K large) is successfully received back from the Web server. This
measurement is a good indicator of Web server delay as well as network
latency. (Time to First Buffer)
FTP [FTP time taken by the FTP server to authenticate the client before it starts
Authentication processing the client's commands. This measurement is applicable only to
time] communications using FTP (not HTTP/S) protocol. So HTTP transactions would
always show zero (0) for this metric.
HTTP/S [Receive Time] time that passes between when the first byte to when the last byte arrives
from the server -- when downloading is considered complete. The Receive
measurement is a good indicator of network quality look at the time/size ratio
to calculate receive rate). This is the metric reported by LoadRunner function
longLastByteMSecs=web_get_int_property( HTTP_INFO_DOWNLOAD_TIME );
HTTPS [SSL time taken to establish a Secure Socket Layer connection (includes the client
Handshaking hello, server hello, client public key transfer, server certificate transfer, and
Time] other stages).
If this value is high during high load, the web server may benefit from
additional technologies such as an encryption daughter board on the web
server or a load balancer device that also performs SSL encryption.
Each column represents a different hit (resource). The Average and Maximum
LRAnalysis70.ini
The Analysis module reads and saves its setting in the LRAnalysis70.ini file with in the Windows directory (C:\Winnt or
C:\Windows on Windows XP). To enable LoadRunner 7.8 Webpage Breakdown graphs to display URLs with a length
greater than 25 characters, add a SURLSize entry in the [WPB] section to specify 255 (the maximum):
17 of 22 9/23/2008 8:53 AM
LoadRunner Architecture http://onestopsoftwaretesting.com
g , y [ ] p y ( )
[WPB]
SURLSize=255
Also, in the LoadRunner\bin\dat directory, use MS Access 2000 to open loader2.mdb or use MS Access 97 to open
loader.mdb. Highlight the Breakdown_map table, select design view, and change the "Event Name" column size to
255.
The Throughput graph shows the amount of throughput on the Web server (y axis) during each second of the scenario
run (x axis).
The Hits Per Second graph shows the number of hits on the Web server (y axis) as a function of the elapsed time in
the scenario (x axis).
Measurement Calculations
The less time per transaction, the greater a system's throughput, as expressed by
the formula derived from Little's Law: Ivan Maclaine (from
Qualtech Australia)
Server Thruput = # of real simultaneous users / ( Average Response Time +
offers a concise
Think Time )
Introduction to
So when no think time is used, the number of virtual users can be calculated by LoadRunner which
this formula suggested by Menasce & Almeida: includes a definition of
Black & White-Box
# of real simultaneous users * [ Response Time / ( Response Time + Think testing, a description
Time ) ] of LR product
components, and a
For example, to simulate 1,000 real users using 2 second response time and 20 summary of load
second think times: testing methodology.
Peak number per hour or per second is typically used for system sizing.
Normal load levels are used to plan on-going support requirements.
Low load levels are used to plan the percentage of resources which can be
taken off-line for repairs and upgrades.
A Risk Profile may also include an estimation of risk of malfunction and likelihood of
impact to the business.
Measures of Variation
The average measures the central tendency in a group of numbers. For example,
the average of 10, 20, and 30 is 20 . This is calculated by adding all three numbers "The "User Experience,
together (10 + 20 + 30 = 60) and dividing this sum by the number of items added Not Metrics" 9 part white
together (in this example, three). paper on Performance
Engineering [Summary] by
Scott Barber of Noblestar is
Standard Deviation based on Rational's
Standard deviation is a statistical measure of the extent of variation in a set of
Performance Studio, but the
data. The standard deviation tells you the extent of the spread of individual items
concepts are relevant to
around the average.
LoadRunner as well.
The number in the 90% column is shown after a run is completed in the Analysis
"Beyond Performance
program, not in the Controller) because it is calculated by sorting all items by
Testing" addresses the
value obtained during the run, then identifying the value associated the item at
iterative process what
the top 10% of all items.
happens after initial test
results are collected, the
The CV (Coefficient of Variation) makes the standard deviation number
18 of 22 9/23/2008 8:53 AM
LoadRunner Architecture http://onestopsoftwaretesting.com
If one average is larger than the 90% number of another average, there is a good
chance are that there is a significant difference between the two averages.
By default, the Analysis Summary is filtered to include think times. So after The output.mdb output
each run, click the global filter tunnel icon, scroll down to the "Think Time" Filter MS-Access database file
condition and uncheck the box. created by Analysis
Merged Graphs defined in the Controller are not carried over to Analysis. So
you have to recreate them.
Popular reasons for duplicating a [Whole Run] graph are listed below.
I like to erase the default name "Copy of ..." and at the end of the graph name
add a marker such as the words in brackets above. This is so that similar graphs
remain vertically aligned.
Are there others?
The Analysis Module can cross-result native Windows NT/200X Perfmon statistics
within graphs of LoadRunner output.
Since you have to manually assign colors anyway, consider using a standard
set of basic colors for major metrics:
Major Metric color
Running Vusers beige upper left corner
Average Transaction Time green
Data Throughput purple
Transactions Per Second blue
Excel files generated with the HTML report are scaled. So either change all scale
to 1 before creating the HTML report, or multiply the numbers in the spreadsheet.
19 of 22 9/23/2008 8:53 AM
LoadRunner Architecture http://onestopsoftwaretesting.com
6. Click Tools > "Export Custom Graphs Settings..." to store the format in a
*.axm file so that the format can be reused after the current session. They
contain the name of the monitor and information about the X-axis and Y-axis.
IP Spoofing
When testing using Web and WinSocket protocols, by default Vusers on each host
machine use the host's (agent's) IP address. Since a particular server keeps serving
a specific IP address, load tests would reflect network router caching and other
optimizations rather than the real-life situation where different users arrive to the
server with different IP addresses. This is especially needed when the hardware
configuration under test balances load across a "farm" of several (web or database)
servers. "IP spoofing" enables each host machine to "spoof" the server into
thinking it is using many different IP addresses.
Note: IP addresses include two components, a netid and hostid. The submask
determines where the netid portion of the address stops and where the hostid
begins.
When a host machine is re-started, routing tables are updated with the new
addresses.
The IP Wizard program creates a batch file (nt_routing.bat for Windows and
unix_routing.bat for UNIX machines) and .ips file which updates IP tables.
After turning IP spoofing on or off, for the setting to take effect, the Load
Generator needs to be disconnected and reconnected again.
Certification in LoadRunner
As of July 15, 2008: HP abandoned use of the "HP Certified Professional"
designation. CPC Exam: any advice?
To become an ASE (Accredited Systems Engineer) of HP's Performance Center Interview Questions
v9.0, take two exams each costing $750. In Aug. 08, it is conducted once a month
(on a Friday) only at HP education vendor SkyIT in New York City, NY 10001 (2
Blocks S. of PENN Station) 330 7th Avenue, 12th Floor. Phone: 866.641.6017,
(919) 544-2354 Email: training@skyitgroup.com, tcook@skyitgroup.com
20 of 22 9/23/2008 8:53 AM
LoadRunner Architecture http://onestopsoftwaretesting.com
User Hangouts
LoadRunner competes with several other load
Before allowing you into their product support pages or receive your calls at testing products and services
408.822-5400 (press 1 for support, then 3 for LoadRunner) Mercury requires that
LoadRunner jobs aggregated by
you provide your “Maintenance Number” (MPN) from your product box. This indeed.com
number is stored in the "LoadRunner_SerialNumber" entry in the mercury.ini file
created during installation within C:\WINNT. This is, in effect, is the extreme
opposite of the open-source paradigm.
On the Mercury site, don't use a password you want to keep secret (such as the
one you use to enter bank bill paying sites) because Mercury.com stores your
password in plain text (unencrypted) within a internet browser cookie — named
"password" no less!
Loadrunner Yahoo Group has amassed over 10,000 messages since 1999. Top
contributors include founder Mark McWhinney (Portata.com), Tim Nichols, James Pully, Kevin Quinn, Steve Cheney, Michael
Foley, Vishnu Reddi
The TCQAA Mercury Forum has had few entries, but worth an occassional look.
An additional license is not needed for LR 7.8, but is needed for LR 7.6.
Mercury Knowledge Base article 22664 notes that RPM requires IIS Server 4.0 or
5.0 on Windows 2000, but NOT IIS 6.0 installed with Windows 2003 servers.
If the Controller was installed on a Windows XP machine and the RPM server is
installed on a Window 2000 Server, or Windows NT machine with IIS 4.0, the XP
Controller machine's Local Security Policy "Network Security: Sharing and Security
model for local accounts" should be changed from "Guest Only" to "Classic". During
RPM installation:
1. Specify a root-level folder (such as "RPM") on a data drive rather than allow
RPM to be installed in the default "RPM Server" folder on the same level as
LoadRunner under Program Files > Mercury Interactive.
2. Specify the UserID and Password of the Controller machine. If you take the
default MI_Viewer, another new user will be created on the RPM machine.
The default password for user MI_Viewer is MIOrchid#1
3. Go to Start -> Programs -> RPM Server -> Remote Performance Monitor
User Configuration, in the "Test Configuration" section of the dialog, type in
the machine name of the Controller machine and then click the "Test Machine"
button.
If you do not have Administrator rights and try to change the UserID: Admin
and Password: Admin, you will see this message:
Run-time error -2147467259 (80004005)':
Method '~' of object '~' failed
After the RPM Server is installed on a machine, the Controller cannot be installed
on top of it. The RPM must be uninstalled before the Controller can be installed on
that machine.
In Control Panel > Administrative Tools > Internet Services Manager right-click on
your IIS server for New, Web Site. In the Web Site Creation Wizard, specify name
"RemoteView", then Browse to associate that name with the "WWW Pages" folder.
To start the newly defined site you need to restart the IIS server then start a
21 of 22 9/23/2008 8:53 AM
LoadRunner Architecture http://onestopsoftwaretesting.com
To start the newly defined site, you need to restart the IIS server, then start a
scenario in the Controller so that it displays monitored information. If you try to
view the monitor in Internet Explorer when the Controller is not running a scenario,
you will be brought back to the Login page.
It features a not very helpful "one size fits all" error message:
Each remote viewer can select and customize the graphs to monitor.
RPM times out if there is no browser activity for 20 minutes. This then requires
another user login.
22 of 22 9/23/2008 8:53 AM