Version 8.6
Ben Lee
Informatica Professional Services
Senior Consultant
Mid-Atlantic IUG – October 8, 2009
1
Agenda
• Memory optimization
• Performance tuning methodology
• Tuning source, target, & mapping bottlenecks
• Pipeline partitioning
• Server Grid
• Q&A
• Course evaluation
2
Anatomy of a Session
Integration Service
DTM Buffer
Source Target
WRITER data
data READER
Transformation
caches
TRANSFORMER
3
DTM Buffer
4
Reader Bottleneck
DTM Buffer
waiting
READER WRITER
Slow reader
waiting waiting
TRANSFORMER
5
Transformer Bottleneck
DTM Buffer
waiting waiting
READER WRITER
TRANSFORMER
Slow transformer
6
Writer Bottleneck
DTM Buffer
waiting
READER WRITER
Slow writer
waiting waiting
TRANSFORMER
7
Large Commit Interval
DTM Buffer
waiting
READER WRITER
TRANSFORMER
Target rows remain in the buffers until the DTM reaches the
commit point
8
Tuning the DTM Buffer
9
Transformation Caches
10
Max Memory for Transformation Caches
11
Max Memory for Transformation Caches
12
Tuning the Transformation Caches
• Options to tune:
• Increase the maximum memory allowed for Auto
transformation cache sizes
• Set the cache sizes for individual transformations manually
13
Performance Counters
14
Tuning the Transformation Caches
15
Aggregator Caches
• Unsorted Input
• Must read all input before releasing any output rows
• Index cache contains group keys
• Data cache contains non-group-by ports
• Sorted Input
• Releases output row as each input group is processed
• Does not require data or index cache
(both =0)
• May run much faster than unsorted BUT
must consider the expense of sorting
16
Joiner Caches: Unsorted Input
MASTER
Staging algorithm:
All master data loaded
into cache
DETAIL
17
Joiner Caches: Sorted Input
18
Lookup Caches
19
Rank Caches
20
Sorter Cache
• Sorter Transformation
• May be faster than a DB sort or 3rd party sorter
• Index read from RDB = pre-sorted data
• SQL SELECT DISTINCT may reduce the volume of data
across the network versus sorter with “Distinct” property set
• Single cache
(no separation of index & data)
21
64 bit vs. 32 bit OS
22
Performance Tuning Methodology
• It is an iterative process
• Establish benchmark
• Optimize memory
• Isolate bottleneck
• Tune bottleneck
• Take advantage of under-utilized CPU & memory
23
The Production Environment
Disk Disk
24
Preliminary Steps
25
Preliminary Steps
26
Benchmarking
27
Identifying Bottlenecks
28
Thread Statistics
29
Thread Statistics - Terminology
30
Thread Statistics - Terminology
DETAIL
PIPELINE 2
31
Using Thread Statistics
32
Target Bottleneck
33
Transformation Bottleneck
34
Integration Service Monitor in WFMonitor
35
Other Methods of Bottleneck Isolation
36
Session Statistics in WFMonitor
37
Target Optimization
38
Target Optimization
39
Target Optimization
40
Source Bottlenecks
41
Source Bottlenecks
42
Reduce Data Set
43
Expressions Language Tips
44
Expressions Language Tips
instead of:
IIF(condition1,result1,IIF(condition2,
result2,IIF… ))))))))))))
try:
DECODE (TRUE,
condition1, result1,
:
conditionn, resultn)
45
General Guidelines
46
Transformation Specific
47
Other Transformations
• Normalizer
• This transformation INCREASES the number of rows
• Place as far downstream as possible
48
Iterative Process
49
Partitioning
50
Rules for Adding Partition Points
51
Guidelines for Adding Partition Points
52
Partition Points & Partitions
Threads - partition 1
Threads – partition 2
Threads – partition 3
53
Rules for Adding Partitions
54
Rules for Adding Partitions
55
Cache Partitioning
56
Monitoring Partitions
57
Dynamic Partitioning
58
Concurrent Workflow Execution (8.5)
• Prior to 8.5
59
Concurrent Workflow Execution
60
Workflow on Grid (WonG)
61
Load Balancer Modes
• Round Robin
• Honors Max Number of Processes per Node
• Metric-based
• Evaluates nodes in round-robin
• Honors resource provision thresholds
• Uses stats from last 3 runs - if no statistics is collected yet,
defaults used (40 MB memory, 15% CPU)
62
Session on Grid (SonG)
63
Configuring Session on Grid
64
SonG Partitioning Guidelines
65
SonG Partitioning Guidelines
66
File Placement Best Practices
67
File Placement Best Practices
68
Data Integration Certification Path
Level Certification Title Recommended Training Required Exams
Additional Training:
» PowerCenter 8.5 New Features » PowerCenter 8 Team-Based Development
» PowerCenter 8.6 New Features » PowerCenter 8.5 Unified Security `
» PowerCenter 8 Upgrade
69
Q&A
Thomas Bennett
Informatica Professional Services
Senior Consultant
70
Appendix
Informatica Services by Solution
71
B2B Data Exchange
Recommended Services
B2B
83
Informatica Global Education Services
84
Informatica Contact Information
http://www.informatica.com
85
Title: Times Bold 32 pt.; Title Case
86
Bullet Format
87
Chart Example
70
60
50
40
30
20
10
0
1st Qtr 2nd Qtr 3rd Qtr 4th Qtr
88
Pie Chart Example
20%
25%
1st Qtr
2nd Qtr
3rd Qtr
4th Qtr
20%
35%
89
Table Example
90
Text with Graphic on Right
91
Sample Quote
Anonymous
Title
Company
92
How To Apply Different Styles To Objects
Use this way if you see a color you want to use from the “Graphics Elements” slide
Select the object with Click on the object you Depending on the
the style and color you Click on the “Format Painter”. want to apply the new object the 3D effect
want to use If you do not see it then select style and color may look different
“View/Toolbars/Standard”
from the top menu bar
93
How To Apply Different Colors To Objects
Use this way if you don’t see a color you want to use from the “Graphics Elements” slide
Right mouse click on Click on the “Color” Select Select the new
the object you want pop-up and select “More Colors” color you want and
to change and select “Fill Effects” click “OK”
“Format AutoShape”
from the pop-up
94
Color Palette
95
PowerPoint Object Parts
96
2008 PowerPoint Icon Library
97
Informatica Logo
98
Informatica Logo
99
100
Slide Transitions
101
Out-of-Bounds
Graphics and text should not extend into this red area
102