Oracle Data Integrator

Agenda

Overview
Architecture
Developing interfaces
Creating packages
Generating scenarios
Variables declaration
User functions
Creating sequences
Knowledge modules
Provisioning security
Migration Techniques
Overview
An integration platform
To move and transform information across
the information system
Based on metadata
Centralized repository
Enables simply faster integration
ELT architecture provides high
performance.
Knowledge modules provide flexibility
and extensibility.

EL-T
Extract: Extracting the data from
various sources
Load: Loading the data into the
destination target
Transform: Transforming the data
according to set of business rules
Architecture
ODI graphical modules:
ODI Designer
ODI Operator
ODI Topology Manager
ODI Security Manager
ODI agents - A run-time
component of ODI that
orchestrates the integration
process.
ODI Console - The
application server links the
ODI repository to any Web
browser such as Firefox or
Internet Explorer.
ODI repositories - All ODI
modules store their
information in the
centralized ODI repository.
ODI Interfaces Key Concepts
An interface is an ODI object that loads one target data store with data
from one or more source data stores.
An interface implements business rules as mappings, joins, filters, and
constraints.

Business Rules for Interfaces

Business rules are implemented in interfaces as:
Mappings
Filters
Joins
Constraints
They are implemented within Designer in:
Data models
Interfaces
They are stored in the Work repository.
Implementing Business rules
Mapping Clauses
CLAUSE DESCRIPTION
Value String values should be enclosed with quotation marks:
USA,1 Jan 2000. Numbers need not: 10
Source Column Drag-and-drop a column or use the Expression Editor. It is
prefixed with the data store alias.
DBMS Function Use the Expression Editor for the list of allowed functions
and operators.
DBMS MAX(), MIN() and so on. ODI automatically generates the
Aggregate GROUP BY clause.
Combination Any combination of clauses is allowed:
SRC_SALES_PERSON.FIRST_NAME || ||
UCASE(SRC_SALES_PERSON.LAST_NAME)

Other ODI Specific Elements like Variables, Sequences and User functions.
 Sequence of Execution

Join these two tables to combine the

Extract the data from the corrections file Before integrating the data intoFinally,
the integrate the remaining data into
Extract the data from the source tables, corrections into the orders data and
and place that in another temporary destination table, run some checkstheandtarget table, with UPDATE or INSERT
join and place it into a temporary table. store it in a temporary table that closely
table. isolate the errors into a separate table. statements.
resemble the target table.
Staging Area
The staging area is a separate, dedicated area in an RDBMS where ODI
creates its temporary objects and executes some of your transformation
rules.
The staging area is usually located on the target data server.

Execution Location
In an ODI Interface, each rule may be executed on the:
Source
Target
Staging area
The execution location is specified at design time.
How a rule gets implemented depends on the technology of its
execution location.
Case Studies

Staging Area

Alternate
Target Server Source Server
Server Both
LKM Only IKM Only
LKM and IKM
 Configuring Filters, Joins, and Mappings

Active Mapping: When deselected, the

filter, join, and mapping are disabled for
this interface.
Enable mapping for update and/or insert.
It allows mappings to apply only to updates
or inserts.
By default, both insert and update are
enabled.
Select the update key by selecting the Key
check box.
Change the execution location of the filter,
join, or mapping.
KM Selection
When processing happens between two data servers, a data transfer KM is
required.
Before integration (Source > Staging Area) - Requires a Loading Knowledge
Module (LKM), which is always multi-technology
At integration (Staging Area > Target) - Requires a multi-technology Integration
Knowledge Module (IKM)
When processing happens within a data server, it is performed entirely by
the server.
A single-technology IKM is required.
No data transfer is performed.
Naming conventions:
LKM <source tech> to <target tech> [(<method>)]
IKM <tech1> [to <tech2>] <strategy> [(<method>)]
- [to <tech2>] indicates a multiple-technology IKM.
The technology name SQL stands for any SQL-enabled technology.
It can be used for most technologies supporting the SQL-89 syntax.
 Most Common KM Options

KM OPTIONS Description
INSERT UPDATE Should data be inserted/updated in the target?
COMMIT Should the interface commit the insert/updates?
If no, a transaction can span several interfaces.
FLOW CONTROL Should data in the flow be checked?
STATIC CONTROL Should data in the target be checked after the interface?
TRUNCATE / DELETE ALL Should the target data be truncated or deleted before
integration?
DELETE TEMPORARY OBJECTS Should temporary tables and views be deleted or kept for
debugging purposes?
Monitoring Interface Executions
Generating Scenarios
A scenario is designed to put a source component (interface, package,
procedure, variable) into production.
A scenario results from the generation of code (SQL, shell, etc.) for this
component.
The scenario code (the language generated) is frozen, and all subsequent
modifications of the components which contributed to creating it will not
change it in any way.

Re-generating Scenarios
An existing scenario can be regenerated with the same name and version
number.
This lets you replace the existing scenario by a scenario generated from the
source object contents.
ODI Procedures
A procedure is a sequence of commands executed by database engines,
the operating system, or using the ODI tools.
A procedure can have options that control its behavior.
Procedures are reusable components that can be inserted into packages.

Examples

Email Administrator procedure

Initialize Drive procedure
Create and populate RDBMS table
Valid Types of Commands
Commands Description
SQL statement Executed on any DBMS technology.
OS commands Executed on the OS technology.
In OS-specific syntax, using shell commands or
binary programs.
Oracle Data Integrator Tools Executed on the Sunopsis API technology.
Any ODI tool command call.
Jython programs Executed on the Jython technology.
Can call Java code or precompiled objects.

ODI Specific Elements

Elements Description
Variables They may be specified either in substitution mode, or in
bind mode.
Sequences They may be specified either in substitution mode, or in
bind mode.
User functions Used in the same way as DBMS functions; replaced at the
time of code generation by their implementation
Variables

Using variables is highly recommended for:

Creating reusable packages or packages with a complex
conditional logic
Interfaces
Procedures
Variables values can be stored:
Persistently in the ODI Repository if their Keep History parameter is set to All
values or Latest value
Only in the memory of the agent during the execution of the current session
if their Keep History parameter is set to No History
Binding Versus Substitution
Substitution:
The variable is replaced by its value before execution.
You need to use single quotation marks for nonnumeric values
SELECT * FROM MYTABLE WHERE NAME LIKE #GLOBAL.PATTERN

Binding:
Works only for SQL clauses by prefixing it with a colon : rather than a hash
# (binding).
The statement is prepared in the DBMS without the value, then the value is
sent for processing.
It may be used for DBMSs that optimize repeated statements with different
parameter values.
User Functions
ODI user functions are macros in ODI that are replaced at run time by code
that is specific to the technology where they are used.
It is used to create an alias for a recurrent piece of code or encapsulate a
customized transformation.
Use of User Functions:
Simple formula:
If <param1> is null then <param2> else <param1> end if
Can be implemented differently in different technologies:
Oracle:
nvl (<param1>, <param2>)
Other technologies:
case when <param1> is null
then <param2>
else <param1>
end
Properties of User functions

A user function always has:

One syntax: Defines how the function is called by other objects in ODI
Several implementations specific to different technologies
A scope:
Global functions can be used anywhere.
Project functions can be used within the project.
Sequences
They function in the same way as a variable that is automatically
incremented each time it is accessed.
In code, the sequence name is followed by _NEXTVAL.
This increments the sequence, and then retrieves the new value.
You do not need to specify the scope.
Substitution versus binding:
Substitution: #<sequence_name>_NEXTVAL
Binding: :<sequence_name>_NEXTVAL
Native sequences (Oracle and so on)
<sequence_name>_NEXTVAL
An ODI sequence is incremented by the agent each time it processes a
row affected by this sequence.
When a rule is entirely processed within a DBMS, Oracle Data Integrator
sequences in this rule are not incremented.
Using Sequences in Mappings

To generate a new sequence value for each row, your mapping should meet
the following conditions:
Mapping must be executed on the target.
The column must be set for insert but not update.
Unique keys must be checked in the flow not containing the column.
The Not Null check box must be deselected for the column.
The column must not be part of the update key.
Modify the flow to force the integration phase to pass all data through the
agent
Use a multiple-technology append or control append IKM.
There should be a SELECT / INSERT command, rather than an INSERTSELECT.
ODI Packages
A package is a pre-defined sequence of steps, designed to be executed in
order.
There are many types of steps, such as:
Interfaces
Procedures
Actions/evaluations on variables
Actions on models, sub-models or data stores
OS Commands
Oracle Data Integrator Tools
Actions/Evaluations On Variables
Declare Variable: When a variable is used in a Package (or in elements of the topology
which are used in the Package), it is strongly recommended that you insert a Declare
Variable step in the Package. This step explicitly declares the variable in the Package.

Refresh Variable: This step refreshes the variable by running the query specified in the
variable definition.

Set Variable: There are two functions for this step:

Assign sets the current value of a variable.

Increment increases or decreases a numeric value by the specified amount.

Evaluate Variable: This step compares the value of the variable with a given value
according to an operator. If the condition is met, then the evaluation step is true,
otherwise it is false. This step allows for branching in Packages.

Actions on Models
Journalizing
static check
Reverse-engineering
Knowledge Modules
A Knowledge Module is a code template containing the sequence of
commands necessary to implement a data integration task.
There are different predefined Knowledge Modules for loading, integration,
checking, reverse engineering, journalizing, and deploying data services.
All Knowledge Modules work by generating code to be executed at run time.

Scope of Knowledge Module

Global
Project
Types of KM
KM Run time
 Developing KMs
you do not create a Knowledge Module from the beginning but rather develop a KM
starting from some existing KM.
To develop a Knowledge Module:
1. Import a Knowledge Module of appropriate type.
2. Modify imported KM to add new functionality based on your project requirements.

General Guidelines
Very few KMs are ever created from scratch. They usually are extensions or
modifications of existing KMs.
Duplicate existing steps and modify them. This prevents typos in the syntax of the odiRef
methods.
All interfaces using the KM inherit the new behavior. Remember to make a copy of the
KM if you do not want to alter existing interfaces. Modify the copy, not the original.
Modifying a KM that is already used is a very efficient way to implement modifications
in the data flow and affect all the existing developments.
ODI Security Profiles
Security Cases
Setting up LDAP security
Setting external authentication
Creating users using generic profiles
Creating users using Non-generic profiles
Creating new custom profiles

Migration of objects
Smart Export
Smart Import

Import Modes
Duplication
Synonym mode Insert
Synonym mode Update
Synonym mode Insert/Update
Import Replace