Load Testing Oracle Business Intelligence Enterprise Edition

(OBIEE) Using Oracle Load Testing (OLT) 12.x

Author @ Ahmed Awan

Customer Engineering & Advocacy Lab (CEAL) https://blogs.oracle.com/pa

December 2014
Version 3.0

2014 Oracle Corporation

 Introduction
This document is intended as a guide for creating load/performance test
scripts against Oracle Business Intelligence Enterprise Edition (OBIEE)
using Oracle Load Testing. This guide will assist during the script creation
process and enable the OBIEE tester to create scripts faster and more
reliably. It assumes that the person using this document has experience
working with Oracle Application Testing Suite. This document does not
necessarily cover all OBIEE transactions that have to be tested, nor does it
guarantee that the parameters mentioned in this document will perfectly
match your particular environment.

Table of Content:
- 1: Oracle Application Testing Suite (OATS) Overview.

- 2: OBIEE Load / Performance Testing Process Using OATS.

2014 Oracle Corporation

 1: Overview
About Oracle Application Testing Suite (OATS 12.x)

Functional Testing for Web Apps

Test Manager for Web Applications

Load Testing for Web Apps

For more information on this OATS, please visit

> http://www.oracle.com/technetwork/oem/app-test/etest-101273.html

2014 Oracle Corporation

 1: Overview
About OATS - Oracle Load Testing (OLT) Architecture

Oracle Load Testing Agents

(OATS Agent Service)

Agent 1 Virtual Users

Oracle Load Testing Server
(OATS Application Service)

Virtual Users
Agent 2
OLT
OLT Controller
ServerStats
Module
Module
Virtual Users
Agent 3

WebLogic Server

Virtual Users
Agent n

OLT Database

Oracle DB

2014 Oracle Corporation

 1: Overview
About OATS - Oracle Load Testing (OLT) Components

OLT Server: Main interface for configuring load test scenarios and
server monitors, running load tests and viewing results
- OATS Application Service is used to run the OLT Server
- Typically one Server/Controller component per deployment
- WebLogic 11gR1
- Web-based user interface accessible through IE or Firefox

OLT Agents: Agent is used to run Virtual Users which execute load
test scripts to generate load against application under test
- OATS Agent Service in is used to run the OLT Agent
- Controller will start Virtual Users on configured Agent systems at run time

2014 Oracle Corporation

 1: Overview
OATS Learning Resources Links

Oracle e-Learning Library for OATS:

- http://apex.oracle.com/pls/apex/f?p=44785:2:0:FORCE_QUERY::2,R
IR,CIR:P2_TAGS:ATS

All About Oracle Application Testing Suite (OATS)

- http://www.oracle.com/technetwork/oem/app-test/etest-101273.html

About Oracle Load Testing (OLT)

- http://download.oracle.com/technology/products/oem/screenwatches/ats_loa
d_testing/index.html

Download latest OATS from

- http://www.oracle.com/technetwork/oem/downloads/index-084446.html

2014 Oracle Corporation

 Load / Performance Testing OBIEE

Design / Build Test Scripts

Perform Basic OBIEE Tuning

System Requirements: Oracle Load Testing (OLT)

Perform Load Test

Load Test Results & Analysis

2014 Oracle Corporation

 Performance Test Process - OBIEE
Using Oracle Application Testing Suite (OATS)

2014 Oracle Corporation

 <Insert Picture Here>

Design / Build Test Scripts

2014 Oracle Corporation

 Design & Build Load Test Scripts

Define User Scenarios.

Record the Load Test Script.
Set the Thinktime in the Load Test Script.
Correlate the Load Test Script.
Parameterize the Load Test Script.
Tune and Validate Load Test Script.

2014 Oracle Corporation

 Define User Scenarios

Define the user scenarios for the load test.

- For example,
- Go to the website URL
(http://<hostname>.oracle.com:9704/analytics)
- Enter username/password & click Login button
- Click Dashboards menu
- Select dashboard Competitors
- Click Sign Out

2014 Oracle Corporation

 Record the Load Test Script
Note: you need to have Oracle Application Testing Suite (OATS)
installed to be able to record load test scripts.

Here are the steps to record the script:

- Open the browser and go through the scenario once to warm up
the browser and server
- Close the browser
- Launch Open Script to record the script
- Create new script & select Web/HTTP protocol.
- Click the record button to start recording, go through the user
scenario and save the script.

2014 Oracle Corporation

 Record the Load Test Script
How to Use OpenScript to record the script:
- Create new script & select Web/HTTP protocol.

2014 Oracle Corporation

 Record the Load Test Script
How to Use OpenScript to record the script:
- Click Record button* to begin recording script

2014 Oracle Corporation

 Set the Thinktime in the Load Test Script
Setting the right Thinktime is very critical to get the correct results
because it controls the load the load generator is going to put into
the OBIEE system.
- The Thinktime (in milliseconds) is the first parameter for the function
in the script. For example in java code:
- beginStep("[3] Oracle BIEE Home", 60000);

- For example from UI:

2014 Oracle Corporation

 Set the Thinktime in the Load Test Script

In OpenScript - How to define Thinktime between each steps. In

Tree View, double click on each Step Group defined:

2014 Oracle Corporation

 Correlate the Load Test Script
There are dynamic contents returned from the server, and the script
needs to be able to pick up the changes accordingly , such as _scid,
ViewStateetc. The process is called script correlation. What
needs to be correlated is application specific.
Tip: How to define Correlation rules
- The following rule defines the variable

2014 Oracle Corporation

 Correlate the Load Test Script

Where the value is captured in the script

- http.solve("Scid_2", "var obips_scid=\"(.+?)\";", "", true,Source.Html, 0, EncodeOptions.None);

Where the value is substituted in the script

- http.post(47, "http://pa.us.oracle.com:9704/analytics/saw.dll",
http.querystring(http.param("CatalogTreeModel")), http.postdata(http.param("action", "ii"),
http.param("path", "/users/{{db.Login_Users.Login_User,psr1004.biconsumeruser4}}"),
http.param("followLinks", "f"), http.param("_scid","{{Scid_2,vXRfc3pu3ZmpTBa0OHOG}}"),
http.param("icharset", "utf-8"), http.param("urlGenerator","qualified")), null, true, "UTF8", "UTF8");

2014 Oracle Corporation

 Correlate the Load Test Script

Import Correlation Library for OBIEE:

- For automatic correlation, it is recommended to import the rules xml file.

In OpenScript, run Menu: Script->

Script Properties and Edit button.

2014 Oracle Corporation

 Correlate the Load Test Script
Import Correlation Library for OBIEE
- Click on Import button and use below embedded XML
(OBIEE_Correlation_Library.xml attached in this pdf)

2014 Oracle Corporation

 Correlate the Load Test Script
After you import the correlation xml file you should see below
known correlation rules for OBIEE. You might need to define your
own correlation rules when needed (as per slide 19-20).

Apply correlation rules in the

recorded script:
In OpenScript, run Menu: Script->
Correlate Script to apply the
correlation rules

2014 Oracle Corporation

 Parameterize the Load Test Script
We would like each virtual user to use a different user id. Basically,
this is done by define a parameter in the databanks (called
db.Login_Users.Login_User in this example), and then use it in the
script where the user id is needed.
- How to use Databanks (refer to OpenScript Help for details):

2014 Oracle Corporation

 Parameterize the Load Test Script

In the Code View, the parameters appear as below:

http.post(19,"http://pa.us.oracle.com:7001/analytics/saw.dll",
http.querystring(http.param("bieehome")),
http.postdata(http.param("NQUser","{{db.Login_Users.Login_User,ceal1004.biconsumeruser4}}"),
http.param("NQPassword", "welcome1"),
http.param("Locale", ""),
http.param("startPage", "1"),
http.param("icharset", "utf-8")),
null,
true,
"UTF8", "UTF8");

2014 Oracle Corporation

 Tune and Validate Load Test Script.
In OpenScript, playback the final parameterized / correlated script

Note: Playback script with Iterations. Select Iterate from the Script menu and select the
Run no more than [ ] iterations option and set the iteration count to 10.

2014 Oracle Corporation

 <Insert Picture Here>

Perform Basic OBIEE Tuning

2014 Oracle Corporation

 Perform Basic OBIEE Tuning

Tuning and Optimization.

Specific Configurations for Load Test.

2014 Oracle Corporation

 Tuning and Optimization
Prior to load tests execution, it is recommended to implement best
practices tuning settings stated in below document:

- OBIEE Tuning Guide @

1333049.1 (OBIEE 11g Infrastructure Performance Tuning Guide) can be found at:
https://support.oracle.com/epmos/faces/DocumentDisplay?id=1333049.1

2014 Oracle Corporation

 Specific Configurations for Load Test
OBIPS Cursor Wait Time
- There are two parameters that control how long OBI PS should wait before
replying refreshing page to the browser. NewCursorWaitSeconds is the
time to wait on the initial callback from the UI for a query to finish.
OldCursorWaitSeconds is for subsequent callbacks for the query to finish.
Note: It is helpful to increase the NewCursorWaitSeconds for
performance load testing, so the load test scripts wont error out because
of not seeing the expected results:
<Cursors>
<NewCursorWaitSeconds>1000</NewCursorWaitSeconds>
<OldCursorWaitSeconds>1000</OldCursorWaitSeconds>
</Cursors>

Important Note: Revert the above setting to default once the load
test is completed.
Avoid Excessive Logging
- Avoid excessive writing to log files in following BI / WebLogic Server
components because this can quickly create a disk bottleneck under high
users load test.

2014 Oracle Corporation

 Specific Configurations for Load Test
Set BIEE log level to Error
- In EM, set default log level for all BI System components to Error.
See below:

2014 Oracle Corporation

 Specific Configurations for Load Test
Set WebLogic Server debug log to disabled
- Make sure debug log is set to disabled in WebLogic Server for each
bi_server_n managed server. See below:

Lower the OBIS Query Loglevel

- Loglevel 2 provided the logical query, physical query and the response times
- It has reasonable low overhead in low to moderately busy system and can be
kept at level 2 in production

2014 Oracle Corporation

 <Insert Picture Here>

System Requirements: Oracle Load Testing

2014 Oracle Corporation

 System Requirements: Oracle Load Testing (OLT)

Number of Load Testing Server machine will vary depending on the size and
configuration of your load test. If you are running larger load tests, of greater
than 1000 Virtual Users, you should consider deploying Oracle Load Testing
(OLT) on a faster server class machine with additional RAM recommended.
- Can typically run up-to 1500 Virtual Users (VUs) per OLT Agent (64bit)
system machine with following hardware specifications:
- Windows 2003, Windows 7, Windows 2008
- System: x86 64-bit processor, 2.6 GHz or faster (4 Cores)
- Memory: 8 GB.
So for example if you are planning to run 5000 Virtual Users (VUs), you
would need three machines running OLT Agent 64-bit version with above
hardware specifications and one machine (with 2.0 GHz, 4 GB Memory)
to host OLT Server (used to direct entire load test & results) .

Note: It is not recommended to run OLT Agent and OLT Server on same
system.

2014 Oracle Corporation

 <Insert Picture Here>

Perform Load Test

2014 Oracle Corporation

 Perform Load Test
Launch Oracle Load Testing Controller.
- First step is to create a load test scenario by selecting the script or scripts
you want to run in your load test. Select the script(s) from the appropriate
Workspace

2014 Oracle Corporation

 Perform Load Test
Configure all the parameters:
- Click the button to go into the configure all parameters page.
- Set the Thinktime range 50% - 150%

2014 Oracle Corporation

 Perform Load Test
Run the test with Oracle Load Testing
- Set Virtual user (VU) ramp up: Ramp up 2 users every 15 seconds

- Click the Run Test button to start the run

2014 Oracle Corporation

 Perform Load Test
View Real Time Results
OLTs Watch VU Grid and View Run Graphs tabs allows you to
see real-time results as the load test session is running:

2014 Oracle Corporation

 Perform Load Test
System Monitoring Here are various monitoring options,
you can use to monitor the system
usage in real-time during load test run:

1. When running a load test, Oracle Load

Testing also allows you to monitor key
performance diagnostics for OBIEE
infrastructure using ServerStats.

2. For diagnostics and monitoring OBIEE

you can use OEM 12c BI
Management Pack.
(http://www.oracle.com/technetwork/oem/pdf/
511863.pdf).
OATS
3. Live monitoring using dtrace in Oracle
Linux / Solaris or Perfmon in
Windows.

4. Using sar command you can monitor

Tuning and Optimization
performance of various in Linux /
Solaris subsystems (CPU, Memory,
I/O..) in real time.

2014 Oracle Corporation

 <Insert Picture Here>

Load Test Results & Analysis

2014 Oracle Corporation

 Post Run Reports
Create Reports tab allows users to access results from
previously run load test Sessions
- Select previous test result from Sessions pulldown
- Access the default graphs and reports.

Following are the main types of results you can create in Oracle
Load Testing (OTL) to include in your final load test report:
- Session Performance Report.
- Performance vs. Users.
- Statistics (Throughput) vs. Users.

2014 Oracle Corporation

 Session Performance Report
In Report sub-tab select / generate session report that
includes following useful information:
- Amount of load being generated (# of VUs, hits, page requests, data
rec., etc.). Performance of individual scripts (script response times,
page response times)

2014 Oracle Corporation

 Session Performance Report
Performance of individual scripts (script response times, page response
times)

2014 Oracle Corporation

 Performance vs. Users
Shows script completion times (y-axis) plotted against the # of
running Virtual Users (x-axis)
- Bottlenecks will cause script completion times to increase as you add more
VUs; otherwise graph should remain flat
- Invalid server responses may cause scripts to fail/terminate early and a
decrease in script completion times

2014 Oracle Corporation

 Statistics (Throughput) vs. Users
Shows amount of load being generated (y-axis) plotted
against the # of running Virtual Users (x-axis)
- Load generated is represented in either Kilobytes/Second
(received), Transactions/Second, Pages/Second, and Hits/Second
- Expect to see load generated increasing as you add more Virtual
Users

2014 Oracle Corporation

 Throughput Notes

Kilobytes/Second is the easiest throughput graph to

look at:
- As Virtual Users continue to increase, KB/Sec should increase.
- If KB/Sec remains DOES NOT increase with increasing VUs, there
may be a queuing issue.
- If KB/Sec decreases given increasing users, a bottleneck may
exist.
- Heavy KB/Sec fluctuations may indicate unstable servers given
the current load.

2014 Oracle Corporation

 <Insert Picture Here>

Tips & Tricks

2014 Oracle Corporation

 Tip 1: Creating Load Test Script in OpenScript
for SmartView.
Some points to note:
During SmartView recording, you need to keep the browser opened.
In OATS 12.x release, SmartView compression is now supported.

1. Create new script & select

the Load Testing >
Web/HTTP

2014 Oracle Corporation

 Tip 1: Creating Load Test Script in OpenScript
for SmartView.
2. Click Record button* to
begin recording script.
Note the SV correlation rule
is already defined in OBIEE
correlation library xml file
(imported in OpenScript as
stated previous slide)

3. Keep the browser opened and

step through the transaction
in the Excel to capture.
Test case used for recording:
select Oracle BI EE, then type in the
OBIEE SmartView URL in the format
http://[machine_name:port]/analytics/jbips
, enter login / password > browse the BI
Catalog and select a SmartView report >
Disconnect All

2014 Oracle Corporation

 Tip 1: Creating Load Test Script in OpenScript
for SmartView.
3. Keep the browser opened
and step through the
transaction in the Excel
to capture.

Test case used for recording:

select Oracle BI EE, then type
in the OBIEE SmartView URL
in the format
http://[machine_name:port]/ana
lytics/jbips, enter login /
password > browse the BI
Catalog and select a
SmartView report > Disconnect
All

2014 Oracle Corporation

 Tip 1: Creating Load Test Script in OpenScript
for SmartView.
4. Once recording is completed,
make sure to add Step Group
for each step e.g. Connect,
Refresh etc.

5. We would like each virtual user

to use a different user id using
DataBank. Create CSV file, set
user variable and substitute the
value in <usr></usr> in Java
code view.

Playback script with Iterations.

Select Iterate from the Script
menu and Select the Run no 6. Run the script to execute the
more than [ ] iterations option transaction & analyze results to
and set the iteration count to 10. verify it worked

2014 Oracle Corporation

 2014 Oracle Corporation