Robin Moffatt, WM Morrisons plc

http://rnm1978.wordpress.com

Oracle BI specialist at Morrisons plc

Big IT development programme at its early stages implementing OBIEE, OBIA, ORDM, all on Oracle 11g & HP-UX

A Performance Tuning Methodology

OBIEE techie stuff Learn from my mistakes!

Response times
Report ETL batch

OLTP transaction

System impact
Resource usage Scalability

Check that your system performs

Are the users going to be happy?

Baseline
How fast is fast? How slow is slow?

Validate system design

Do it right, first time

Capacity planning

Its never too late

Youll never catch all your problems in pre-

production testing. Thats why you need a reliable and efficient method for solving the problems that leak through your pre-production testing processes.
Cary Millsap - Thinking Clearly About Performance

Because it makes you better at your job

At the very least, your performance test plan will

make you a more competent diagnostician (and clearer thinker) when it comes time to fix the performance problems that will inevitably occur during production operation.
Cary Millsap - Thinking Clearly About Performance

Quantifying response times

System impact

User expectations Problem diagnosis Design validation

Define Do it right Timebox! Evaluate design / config options Measure

Dont fudge it

Analyse Redefine test

Do more testing

Review

Do more testing

Implement

Define Measure

Analyse
Review Implement

Define what are you going to test

Aim of the test Scope Assumptions Specifics Data, environment, etc

E.g. : Check that the system performs Baseline performance Prove system capacity Validate system design

Build how are you going to test it

OBIEE specific

Define Measure

Analyse
Review Implement

More components = more complex = more=variables = larger margin of error Fewer components = easier to manage more precise = more efficient

Define Measure

Analyse
Review Implement

Report / Dashboard

Rendered report

Presentation Services

Logical SQL Physical SQL statement(s)

Data set

BI Server

Data set(s)

Database

Excludes App/Web server & presentation services plug-in

Define Measure

Analyse
Review Implement

User & Stopwatch

Load Testing tool (eg. LoadRunner, OATS)

Presentation Services

nqcmd

LSQL
SQL Client

BI Server

Physical SQL

Physical SQL

Database

Define Measure

Analyse
Review Implement

nqcmd

LSQL

BI Server

Physical SQL

Database

Define Measure

Analyse
Review Implement

Usage Tracking or NQQuery.log

Logical SQL Test script

Logical SQL

Logical SQL

BI Server

Data

nqcmd

Define Measure

Analyse
Review Implement

Test script

nqcmd nqcmd nqcmd nqcmd BI Server Data

Test script

Test script

Logical SQL

Master test script

Test script

Define Measure

Analyse
Review Implement

Simulates user interaction HTTP traffic

Powerful, but can be difficult to set up
Ajax complicates things

Do you really need to use it? Tools

Fiddler2 FireBug

Reference:
My Oracle Support Doc ID 496417.1 http://rnm1978.wordpress.com/category/loadrunner

Define Measure

Analyse
Review Implement

Be very clear what the aim of your test is You probably need to define multiple tests Different points on the OBIEE stack to interface
Pick the most appropriate one

Write everything down!

Define

Measure

Analyse

Review

Implement

Define Measure

Analyse
Review Implement

Enterprise Manager (Oracle)

PerfMon (Windows)

Apache log OAS log Analytics log

Web Server App Server

Presentation Services plug-in

Server metrics e.g. : IO, CPU, Memory

Oracle OS Watcher (unix)

sawserver.log

Presentation Services

NQServer.log NQQuery.log

Usage Tracking

Enterprise Manager ASH, AWR, SQL Monitor

PerfMon (windows only) jConsole etc Enterprise Manager

BI Management Pack

BI Server
systems management

Database
Presentation services

Define Measure

Analyse
Review Implement

Define Measure

Analyse
Review Implement

Lots of different ways to measure

Build measurement into your test plan
Automate where possible Easier Less error

Define

Measure

Analyse

Review

Implement

Define Measure

Analyse
Review Implement

Define Measure

Analyse
Review Implement

Define Measure

Analyse
Review Implement

Response time 1 1 9 1

Average (mean) 3.4

3 2
2 10 2 2 3 1 3 2 9 3 10

50th percentile (Median) 2 90th percentile 9.1

Define Measure

Analyse
Review Implement

Dashboard

Requests
Logical SQL
ORA_HASH(QUERY_TEXT) SQL IDs

Physical SQL

Execution plan Execution plan hash id

Define Measure

Analyse
Review Implement

S_NQ_ACCT START_TS ROW_COUNT TOTAL_TIME_SEC NUM_DB_QUERY QUERY_TEXT QUERY_SRC_CD SAW_SRC_PATH SAW_DASHBOARD

OBIEE_REPLAY_STATEMENTS qt_ora_hash query_text saw_path dashboard

OBIEE_REPLAY_STATS testid testenv qt_ora_hash start_ts response_time row_count db_query_cnt

Define Do it right Measure Evaluate design / config options Analyse

Dont fudge it

Timebox!

Review

Implement

Define

Measure Redefine test Continue testing Analyse

Review Implement Implement

Define Measure

Analyse
Review Implement

Define

Measure

Analyse

Review

Implement

You wont get your testing right first time

Theres no shame in that Dont cook the books
Better to redefine your test than invalidate its results

Stick to the methodology

Dont move the goalposts Very tempting to pick off the low-hanging fruit
If you do, make sure you dont get indigestion

Timebox Test your implementation!

Define Do it right Measure Evaluate design / config options Analyse Redefine test Do more testing
Dont fudge it

Review

Do more testing

Implement
robin@moffatt.me http://rnm1978.wordpress.com http://twitter.com/rnm1978