Start Here
Contact Us
510.818.9480 | www.kpipartners.com
Agenda
Agenda
Introducing The Performance Layer Building The Performance Layer Mapping Into Oracle BI Implementation Considerations Q&A
Wide tables carry more data than you need Dashboards only need a few fields
Smaller is faster
Performance Tuning Oracles BI Applications
7
Great performance requires perfect design for how it is used A Dashboarding environment is an Application Use a top-down design approach to support that application
Pre-built logic Clean star models Reduced data weight Tables which match usage by Oracle BI
10
Introducing The Performance Layer >> Oracles View On Data Warehouse Architecture
11
*Optimize To Usage*
Performance Tuning Oracles BI Applications
12
Performance Mini-Fridge
Build a new data model Copy data from BI Apps/DW tables Bring only what you need Denormalize & pre-calculate
ETL
Keep the BI Apps/DW model mostly as-is & add a Performance Layer
The Performance Layer is an industry standard architecture Design is driven only by report performance improvement Travel light No need to alter BI Apps or DW
Performance Tuning Oracles BI Applications
13
14
1. Identify a priority area (select a Fact table) 2. Identify common use cases (reports w/ prompts & data
security)
3. Analyze resulting physical SQL 4. Try to tune the BI Apps model first! (Indexes, etc)
15
5. Prototype a new data model to match those needs (SQL handcrafting of new tables) 6. Adjust SQL & benchmark (SQL handcrafting needed) 7. Map into Oracle BI & test (Unit & Regression) 8. Benchmark the Oracle BI report using prototyped tables (Reports have many SQLsn)
16
9. Build the tables using INFA & DAC - Complete Oracle BI RPD mapping 10. Formal Regression Test 11. Deploy 12. Enjoy praise from users
17
Let the Performance Layer do the work, not the report query Follow the KISS principle: Use a simple and clean Star. No Snowflakes! Ensure OBI is mapped properly and uses correct tables with perfect SQL Favor a general approach as opposed to a case-by-case approach
BI Apps or DW
Perf. Layer
Built directly from the base BI Apps or DW tables Goal: use these tables in as many reports as possible Guiding principles and performance influences:
1. 2. 3. 4. 5. 6. Application use cases drive the layers design Use minimal data for the job at hand Aggregate Fact data when needed Denormalize dimensions to eliminate extra joins Pre-Build calculations to eliminate extra joins Pre-Split data sets based on logical usage
19
Table Partitioning
For all Fact tables of a reasonable size (e.g., > 5M rows) Usually partition on Month (Range
or Interval)
The Database can easily eliminate the majority of the table Allows for smaller, local indexes
Single column bitmap indexes on all dimensional fields used in any sort of prompt or report filter Special composite B-Tree indexes to assist Snowflaked areas Composite B-Tree indexes on large dimensions for join backs (for list reports)
20
Both Dimensions and Facts Very easy to build and use Goal: Reduce Avg. Row Length to 1/5th - 1/20th original size Include only the columns you will need for top-down reporting analysis
If you dont need Customer Address, dont include it Ignore Meta Data columns (e.g., INTEGRATION_ID, etc.)
21
Build using:
1. 2. 3. Create Table as Select (CTAS) Insert /*+ APPEND */ Materialized Views
Compress the table Use Parallel hints & options Dont forget partitions Enhance the tables with calculation logic Database is very fast at these operations expect only a few minutes for 100M rows
Performance Tuning Oracles BI Applications
22
CREATE TABLE WC_ACCT_BUDGET_SF COMPRESS NOLOGGING PARALLEL (DEGREE 8) PARTITION BY RANGE(PERIOD_END_DT_WID) INTERVAL(NUMTOYMINTERVAL(1, 'MONTH')) (PARTITION Part_01 VALUES LESS THAN (20100101)) AS SELECT /*+ PARALLEL(F,8) */ F.PERIOD_END_DT_WID, F.X_PERIOD_END_DT_WID, F.COMPANY_ORG_WID, F.GL_ACCOUNT_WID, F.X_POSTED_TOTAL_AMT, case when GL_D."GL_ACCOUNT_NUM" = 'S250' then F."X_POSTED_TOTAL_AMT" end as PLAN_CASES, FROM W_ACCT_BUDGET_F F, W_GL_ACCOUNT_D GL_D WHERE F.GL_ACCOUNT_WID = GL_D.ROW_WID;
Enhance _SF tables with logic Identify CASE WHEN statements which require other dimensions
Potential great benefit if the join can be eliminated Dont over do it table will get less skinny with each column
case when GL_D."GL_ACCOUNT_NUM" = 'S250' then F."X_POSTED_TOTAL_AMT" end as PLAN_CASES, FROM W_ACCT_BUDGET_F F, W_GL_ACCOUNT_D GL_D WHERE F.GL_ACCOUNT_WID = GL_D.ROW_WID;
Create table WC_WRKFC_EVT_EVENTS_SF WHERE SNAPSHOT_IND = 0 Create table WC_WRKFC_EVT_MONTH_SNP_SF WHERE SNAPSHOT_IND = 1
23
Typical to get a 10X to 20X and even 50X I/O benefit in the _SF vs. the base _F in size
Reduced AVG_ROW_LEN COMPRESSION Record Set Splitting All without any aggregation
Real Examples Sub Ledger (custom) Workforce Snap Workforce Events GL Balance Acct Budget
24
Pre-build major pieces of commonly used but complex logic into the Data Model
Over-relying on the RPD or Reports for logic can harm performance Let the ETL for the Performance Layer do the work not the query
Example #2: Date format conversions dynamically building a new column with a string concatenation statement:
substring(T66755."PER_NAME_MONTH" , 1, 4) , '') + '-' + isnull(right(T66755."PER_NAME_MONTH" , 2) , '')
25
Create a new ROW_WID Compress and index as normal, Parallel if needed for creation Easy to build and map
Performance Tuning Oracles BI Applications
26
Create table WC_EMPLOYEE_MD COMPRESS as select ROWNUM AS ROW_WID, W_ETHNIC_GRP_DESC, WC_RACE_ETHNIC_DIVRSE_GRP_DESC, W_SEX_MF_CODE, W_SEX_MF_DESC, WC_NON_EMPLOYEE_VENDOR_NAME from ( select distinct W_ETHNIC_GRP_DESC, WC_RACE_ETHNIC_DIVRSE_GRP_DESC, W_SEX_MF_CODE, W_SEX_MF_DESC, WC_NON_EMPLOYEE_VENDOR_NAME from W_EMPLOYEE_D);
This real world example created 5,400 records from a W_EMPLOYEE_D of 9+ Million rows.
The OBI RPD can select which one is best for each query Benefits of linking into the _SF
1. 2. 3. 4. The same set of fact rows are selected no benefit Reduced dimension I/O, CPU and buffer space Faster join-back on list reports Very fast prompts, especially when constrained
_SD
_D
27
A good Mini Dimension and Skinny Fact/Aggregate can serve a large % of dashboard queries MDs & Fact Aggregates offer extreme performance:
1. 2. Real World Ex #1: From time-out after 10 minutes to 4 seconds Real World Ex #2: GL Account MD: 131X I/O benefit
28
Aggregates are used when summary reports exist Pre-aggregate the dataset to make it smaller & faster Sometimes they are the only solution Typically a minimum of a 10:1 ratio is used Use all database tools as with any fact table
Partitioning, Indexing, Star Transformations, Compression
Advanced implementations use partition management to build only changed data for faster load times
29
Building the underlying tables is relatively simple But strong SQL & Tuning expertise is needed Take cues from dashboard, report & RPD configuration Any savings in I/O helps the overall system Use all of the available database performance tools
30
31
Link as much as possible to allow for the best performance across all scenarios
Best
Performance Layer
Better
BI Apps Base
32
The 3 Fact tables are mapped like any aggregate The Skinny Fact (_SF) will have fewer dimensions and fewer metrics mapped to it
Along the Employee dimension however it is the same as the base _F
_A
_SF
_F
Raise the priority group on the base _D to have OBI prefer the _SD
As both LTS grains are identical, OBI needs more info to make a choice
_D
_SD
34
Create a dummy hierarchy level and map the LTSs for the Mini Dimension and Fact Aggregate to it The grain of the Mini Dimension is arbitrary As long as OBI knows it is higher than the other LTSs it will be preferred (Priority groups not needed)
_MD
_A
35
Table Mapping
The mapping of tables is straightforward
36
Implementation Considerations
37
Implementation Considerations
The whole prototyping process can be done on a simple star in roughly two weeks Allow for more time if you have:
Large data volumes Difficult performance targets More complex models and logic Many disparate report patterns or lots of reports to consider More stars are needed (e.g., Actuals and Budgets together)
38
Implementation Considerations
Use Production data volumes for accurate analysis Use Production DDL, ETL code, OBI RPD and OBI Webcat Quiet, unused machine for accurate benchmarking Use hardware that is as similar to Prod as possible
KPI uses a database benchmarking tool to compare environments
39
Implementation Considerations
Additional Testing Regression test is easy Additional ETL Run Time may be critical Additional Database size - minor Customization Propagation / Impact Analysis
True of any aggregate E.g. #1: Financial Analytics uses snowflakes with multiple segment hierarchies E.g. #2: HR Workforce Event & Snapshot logic uses effective dates for future dated events
40
Q&A
41
www.kpipartners.com
Staff built from Oracle/Siebel/Hyperion engineering teams On-site, off-shore and blended shore delivery models Exclusive pre-built solutions for Oracle BI & E-Business Suite
Depot Repair Analytics Fixed Asset Analytics Manufacturing Analytics Salesforce.com Analytics Student Info Analytics Subledger (SLA) Analytics and more
42
Contact Us
Global Offices
Bangalore, India Hyderabad, India
43
www.kpipartners.com
44