Oracle
CONTENTS
1 INTRODUCTION................................................................................................................................................3 2 OBJECTIVE.........................................................................................................................................................3 3 ABOUT EIM.........................................................................................................................................................3 4 ABOUT IFB..........................................................................................................................................................4 5 IFB FILE STRUCTURE......................................................................................................................................5 5.1 HEADER SECTION.............................................................................................................................................6 5.2 PROCESS SECTION............................................................................................................................................6 5.2.1 IMPORT...................................................................................................................................................7 5.2.2 EXPORT...................................................................................................................................................9 5.2.3 DELETE.................................................................................................................................................10 5.2.4 MERGE..................................................................................................................................................13 6 RUNNING EIM..................................................................................................................................................14 6.1 USING THE GRAPHICAL USER INTERFACE (GUI)..........................................................................................15 6.2 USING THE COMMAND-LINE INTERFACE.......................................................................................................18 6.3 IMPORT ORDER OF ENTITIES..........................................................................................................................19 7 CHOOSING THE CORRECT INTERFACE TABLE...................................................................................20 7.1 DRIVING FACTORS HILE CHOOSING EIM TA!LE.........................................................................................2" 7.2 HANDLING !ASE TA!LES ITH NO USER #E$S ............................................................................................22 7.3 USER #E$ UPDATE USING EIM ....................................................................................................................22 7.% POPULATION OF USER #E$ AND RE&UIRED COLUMNS.................................................................................22 7.5 SIE!EL RO 'ID CREATION THROUGH EIM.................................................................................................23 8 TRACKING AND HANDLING ERRORS......................................................................................................25 8.1 COMMON ERRORS AND RESOLUTION.............................................................................................................25 8.2 ERROR TRAC#ING PROCESS ...........................................................................................................................29 8.3 EIM ERROR AND TRACE FLAGS....................................................................................................................31 DATA DE!ENDENCIES IN IFB FILES.........................................................................................................34 10 ISSUES "HILE MIGRATING ORGANI#ATION DATA.........................................................................35 1".1 POPULATING THE '!U COLUMN..................................................................................................................35 1".2 CHOOSE !ET EEN DEFAULT ORGANI(ATION AND OTHER ORGANI(ATIONS..............................................37 1".3 LOADING OF S$NDICATED DATA.................................................................................................................37 11 EIM !ERFORMANCE TUNING...................................................................................................................38 12 REFERENCE....................................................................................................................................................41 13 A!!ENDI$ A....................................................................................................................................................42 14 A!!ENDI$ B....................................................................................................................................................45
EIM Handbook
Page: 2 of 47
Oracle
1 Introduction
This document serves as a reference or handbook that would provide guidelines for writing Configuration files in order to migrate data to or from Siebel 7 base tables using EIM, in an optimal fashion. AUDIENCE The uide is intended for the following users. Siebel Application Administrators Siebel Application Developers Siebel System Administrators !eople responsible for planning, setting up, and maintaining Siebel applications !eople responsible for planning, implementing, and configuring Siebel applications !eople responsible for implementation of the entire application, including installing, maintaining, and upgrading Siebel products
PRE-REQUISITE The user should have prior functional knowledge of Enterprise Integration Manager "EIM# including running EIM batch for IM!$%T&E'!$%T and&or (E)ETE&ME% E process. *dditionall+, the user should have working knowledge of the Siebel "both version , and 7# data models.
2 Objective
The ob-ective of this document is to serve as a guide that could be referred to, when migrating data to or from Siebel .ase tables, using the Enterprise Integration Manager. It provides the reader with information as to how the EIM Configuration /ile "I/.# should be written so as to optimi0e the performance of the EIM process and minimi0e the occurrence of errors. The document la+s a lot of emphasis on the Error handling and !erformance tuning aspects of a t+pical EIM process, which would prove beneficial to individuals involved in (ata Migration using EIM.
bout EIM
The Siebel Enterprise Integration anager "EIM# manages the bi1directional e2change of data between the Siebel databases and other corporate databases. This e2change of information is accomplished through intermediar+ tables called EIM tables. The EIM tables act as a staging area between the Siebel application database and other databases. In order to perform import, e2port, update or delete of data in Siebel base tables, Siebel S+stems recommends the use of EIM. Siebel S+stems does not support the use of native S3) for loading data into the base tables. EIM reads a special configuration file that specifies the EIM process to perform import, merge, delete, or e2port with the appropriate parameters. The EIM configuration file "the default file is default.ifb# is an *SCII te2t file of e2tension t+pe .ifb that resides in the admin subdirector+ under the Siebel server
EIM Handbook
Page: 3 of 47
Oracle
director+. .efore running an EIM process, the EIM configuration file must be edited to define the processes that EIM will perform. The following diagram illustrates the flow of data between Siebel databases and other corporate databases4
Non$Siebel %ataba&e
Siebel EI
Siebel %ataba&e
EI
tables is a staging area bet!een t"e Siebel database and ot"er databases
Siebel recommends #se o$ EI Siebel does not s#pport native SQ% to load data into t"e base tables
bout I"#
5hat is I/. file6 *n I/. file is nothing but a EIM configuration file which is an *SCII or 7nicode ".inar+# te2t file of e2tension t+pe .ifb. I/. stands for Interface .atch. It resides in the *dmin director+ of Siebel Server and allows the database administrator to define the t+pe of EIM process like Import, E2port, Merge and (elete to be performed. 5h+ I/. is needed6 *n I/. file contains specifications for batch -obs for the data element to be operated based on the process t+pe mentioned in it.
EIM Handbook
Page: 4 of 47
Oracle
Header Section
Proce&& Section
EIM Handbook
Page: 5 of 47
Oracle
)*1
Header Section
The first nonblank, uncommented line of the header section of an+ .ifb file must contain the e2act information4 ;Siebel Interface Manager< The other parameters that are common to all process t+pes "Import, E2port, (elete = Merge# in the header section are4 PR(CESS' Identifies the specific process to run during this invocation of EIM. The named process must be defined in the process section of this file. C(NNECT' The $(.C source name for connecting to the database server. USERNA E' Specifies the database logon name for this process. This parameter is inherited for the EIM component from the atewa+ server, so it should alread+ be set. >owever, +ou can specif+ this in the .ifb file if +ou are running EIM from the Siebel application "not the command line# and if +ou have not alread+ set this value in the EIM Server Component parameters. PASS)(RD4 Specifies the database password for this process. This parameter is inherited for the EIM component from the atewa+ server, so it should alread+ be set. >owever, +ou can specif+ this in the .ifb file if +ou are running EIM from the Siebel application "not the command line# and if +ou have not alread+ set this value in the EIM Server Component parameters. !*SS5$%( and 7SE%:*ME values are not used for access authentication or as a securit+ measure? EIM ac@uires this value from the component parameters !*SS5$%( and 7SE%:*ME are used onl+ if the parameters are not set at the enterprise or component level
)*2
Proce&& Section
EIM Handbook
Page: 6 of 47
Oracle
7nlike header section parameters, the process section parameters are not generic for all the processes "Import, E2port, (elete = Merge#. .ut some parameters are common for all. The first nonblank, uncommented line of each process section is a bracketed string ";<# that specifies the name of the process. This is the name used in the !%$CESS parameter in the header section.
5.2.1 IMPORT
T*PE' This parameter specifies the t+pe of process being defined "IM!$%T, E'!$%T, (E)ETE, ME% E, or S>E))#. * shell process uses the I:C)7(E statement to invoke a se@uence of processes in a single run. +ATC&' Specifies a re@uired batch number for this process. This number is predefined as the user sets this number at the time of EIM table load. It is used to identif+ the set of rows to load from the interface tables for this specific process. It corresponds to the value in the interface table column I/A%$5A.*TC>A:7M and must be a positive integer between B and CCCCCCCCCCCCCCC "up to DE digits, no commas#. To specif+ multiple batches, use a range or list of batch numbers. .*TC>FDBB1DGB To list batches, use the comma1delimited format as shown in this e2ample4 .*TC>FDBB,DBH,DBI It is recommended to use batch ranges ".*TC> F 2J+#. This allows the user to run with smaller batch si0es and avoid the startup overhead on each batch. The ma2imum number of batches that one can run in an EIM process is D,BBB. TA+%E' Specifies the name of an EIM table used in this process. /rom performance aspect the selection of EIM table is vital. The things that should be kept in mind while selecting the EIM table are discussed earlier.
TYPE = IMPORT BATCH = 10000-10100 TABLE = EIM_ACCOUNT
(N%* +ASE TA+%E' Specifies and restricts selected base tables for the import process. 7se commas to separate table names. Target table for interface tables must be included. This parameter is used to improve performance when loading some specific base tables. /or e2ample4
ONLY BASE TABLES = S_PARTY,S_ORG_EXT,S_ACCNT_POSTN,S_ORG_BU
I,N(RE +ASE TA+%ES' Specifies base tables to be ignored b+ the import process. 7se commas to separate table names. Target tables for interface tables cannot be ignored. The default is to not ignore an+ base tables. 7se this parameter to improve performance when loading all but a few tables. (N%* +ASE C(%U NS' Specifies and restricts base table columns for the import process. 7se commas to separate column names, which can be @ualified with base table names. Include all user ke+
EIM Handbook
Page: 7 of 47
Oracle
columns and re@uired columns. 7se this parameter to improve performance when updating man+ rows but few columns. The default is to process all interface columns mapped to the base table.
ONLY BASE COLUMNS= S_ORG_EXT.NAME, S_ORG_EXT.LOC, S_ORG_EXT.BU_ID
I,N(RE +ASE C(%U NS' Specifies base table columns to be ignored b+ the import process. This is also written with the same manner as $:)K .*SE C$)7M:. %e@uired columns and user ke+ columns cannot be ignored. 7se this parameter to improve performance when loading all but a few columns. The default is to not ignore an+ interface columns.
IGNORE BASE COLUMNS = S_PROD_INT_BU.PR_FULFL_INVLOC_ID,S_PROD_INT_BU.PR_PROD_LN_ID
DE-AU%T C(%U N' Specifies a default value for an interface table column. The s+nta2 is C$)7M: :*ME, L*)7E, as in the following e2ample4
DEFAULT COLUMN = CURCY_CD, "USD"
The given value will be used onl+ if the column is :7)) in the interface table. -I.ED C(%U N' Specifies the value for a column from the interface table. The s+nta2 is the same as for (E/*7)T C$)7M:. The given value will be loaded into the Siebel base table, overriding the value in the interface table column.
FIXED COLUMN = PARTY_TYPE_CD, "Organization"
INSERT R()S' Specif+ing I:SE%T %$5S as /*)SE indicates that rows from EIM table are not to be inserted into the specified Siebel base table. The default is I:SE%T %$5S F T%7E. To change this for all tables, use this s+nta24
INSERT ROWS = FALSE
/or a Siebel base table, the setting is applied when data is imported from an+ interface table as in the following e2ample4
INSERT ROWS = S_ORG_BU, FALSE INSERT ROWS = S_ACCNT_POSTN, FALSE
If the named table is an interface table, as in the e2ample below, the setting applies to all Siebel base tables imported from this interface table.
INSERT ROWS = EIM_ACCOUNT, FALSE
UPDATE R()S' This parameter specifies if foreign ke+ references can be updated. The default value is 7!(*TE %$5S F T%7E which affects all tables. To change onl+ for specific table"s#, specif+ the table name as follows4
UPDATE ROWS = S_CONTACT, FALSE
EIM Handbook
Page: 8 of 47
Oracle
If it is re@uired to set the parameter as /*)SE for most tables, and T%7E for onl+ a few, use this method4
UPDATE ROWS = FALSE UPDATE ROWS = S_CONTACT, TRUE UPDATE ROWS = S_ADD_ORG, TRUE
.+ default, when importing information, EIM performs both inserts and updates based on the content of the batch set. If the named table is an interface table, as in the e2ample below, the setting applies to all Siebel base tables imported from this interface table.
UPDATE ROWS = EIM_ACCOUNT, FALSE
Note4 If neither I:SE%T %$5S nor 7!(*TE %$5S are set to /*)SE, EIM has to perform additional processing "via S3) statements# to determine whether to update or insert. .asicall+ when importing data via EIM, EIM will first look up user ke+ columns in a base table. If it finds the matching user ke+s, EIM will continue to compare all non1user ke+ columns in order to determine whether it needs to update the record or re-ect it as duplicate. 0 ISC SQ%' This parameter is used to set specific e2plicit or implicit primaries. ME2plicitN is when it is re@uired to set specific values as primaries. MImplicitN is when an+ of a group of values is acceptable. /or e2ample, +ou are importing one account with nine addresses. If an+ of the addresses is acceptable as being the primar+, then set primar+ to implicit. EIM then selects one of the addresses as primar+. If a specific address should be the primar+, then set primar+ to e2plicit and indicate the primar+ account b+ setting its flag column. /or e2ample, set EIMA*CC$7:T.*CCA!%A*((% to MKN for e2plicit !rimar+. /or e2ample,
MISC SQL = EXPR_S_ORG_EXT_PR_ADDR_ID
MISC S3) is intended for initial data loading onl+ "with ($COI: T%*:S*CTI$:S F /*)SE#. Sometimes the intention is to appl+ e2plicit primar+ when specified and implicit primar+ otherwise. In that case, we mention the following in the ifb file4
MISC SQL = EXPR_S_ORG_EXT_PR_ADDR_ID, IMPR_ S_ORG_EXT_PR_ADDR_ID
USE INDE. &INTS' The default value for this parameter is T%7E. If it is set to T%7E, EIM generates hints during processing which helps in achieving performance gain. It is recommended for EIM processes to be tested with both T%7E and /*)SE settings in order to determine which provides the better performance for each of the respective EIM -ob.
5.2.2 EXPORT
EIM Handbook
Page: 9 of 47
Oracle
Some process parameters like TK!E, .*TC>, and T*.)E are similar to an Import process. E.P(RT A%% R()S' It implies that all rows in the target base table and data from the related child tables in a specific batch section are to be e2ported. Lalid values are T%7E and /*)SE "the default#. 5hen it is set to T%7E, the E'!$%T M*TC>ES parameter is ignored, as it is selective e2port process.
[Export Accounts] TYPE = EXPORT BATCH = 2 TABLE = EIM_ACCOUNT EXPORT ALL ROWS = TRUE
E.P(RT ATC&ES' This is used for filtering the base table rows to be e2ported. The value is in two parts4 the Siebel interface table name and the filter e2pression that goes against the target base table.
[Export Accounts] TYPE = EXPORT BATCH = 2 TABLE = EIM_ACCOUNT EXPORT MATCHES = S_ORG_EXT, (LAST_UPD > 2003-01-01)
5.2.3 DELETE
Some process parameters like TK!E, .*TC> and T*.)E are much like the other above1mentioned processes. DE%ETE R()S' /or (elete process the TK!E parameter should be defined as TK!E F (E)ETE. *s a result the (E)ETE %$5S parameter will be automaticall+ set to T%7E. This parameter can prevent deletions from one table while allowing them in others. /or e2ample, the following parameter prevents deletion of rows from the SA*((%A!E% table4
DELETE ROWS = S_ADDR_PER, FALSE
UPDATE R()S4 This parameter specifies if foreign ke+ references can be updated. The default is 7!(*TE %$5S F T%7E, which affects all tables. The 7!(*TE %$5S parameter also prevents updates in one table while allowing them in others. If this parameter is set to /*)SE, EIM does not update rows in the specified base table. If +ou need to specif+ multiple tables, use one 7!(*TE %$5S statement for each table.
UPDATE ROWS = S_CONTACT, FALSE UPDATE ROWS = S_ADDR_PER, FALSE
DE%ETE A%% R()S4 It will delete all rows in the named base table including an+ seed data if it is set to T%7E. The default value is /*)SE.
[Delete Accounts] TYPE = DELETE
EIM Handbook
Page: 10 of 47
Oracle
C%EAR INTER-ACE TA+%E' Lalid values are T%7E and /*)SE. If it is set to T%7E, the e2isting rows in the interface table for a given batch number will be deleted. The default setting of this parameter depends on the (E)ETE E'*CT parameter setting. The C)E*% I:TE%/*CE T*.)E default is T%7E if (E)ETE E'*CT is set to /*)SE and vice versa. CASCADE DE%ETE (N%*' "(efault F /*)SE#. Set this parameter to T%7E to delete child records with nullable foreign ke+s when the parent record is deleted. If /*)SE, then when EIM deletes a parent record, it sets the foreign ke+s of the child records to :7)). Sometimes cascade delete takes place during delete process even if it is not mentioned specificall+ in the .ifb file. 5hen a foreign ke+ column that references the deleted record is a re@uired one, the record with the foreign ke+ is deleted. $therwise, the foreign ke+ column is cleared. EIM deletion of a parent row causes cascade deletion of child rows onl+ if the foreign ke+ column in the child table is a mandator+ column. $therwise a cascade clear is performed. DE%ETE E.ACT' This parameter specifies the base table rows to delete b+ using user ke+ values specified in the interface table. It is recommended to use the (E)ETE E'*CT parameter to delete non1target base tables that contain user ke+s onl+. .+ default, (E)ETE E'*CT F /*)SE. If (E)ETE E'*CT is set to T%7E, the $:)K .*SE T*.)ES parameter in con-unction with this parameter needs to be mentioned to identif+ the base tables. It should be ensured that an+ columns that are not part of the user ke+ are :7)) in the interface table.
TYPE = DELETE BATCH NUMBER = 100 TABLE = EIM_ACCOUNT ONLY BASE TABLES = S_ORG_PROD DELETE EXACT=TRUE
DE%ETE ATC&ES' This parameter is used for filtering or more precisel+ to choose selective base table rows to be deleted. The value is in two parts4 the Siebel interface table name and the filter e2pression that goes against the target base table. *n e2ample would be4
DELETE MATCHES = S_ORG_EXT, (LAST_UPD > 2004-07-15 AND LAST_UPD < 2004-07-17)
.+ default, (E)ETE M*TC>ES e2pressions are not used. DE%ETES1IPPRI AR*' This is a special parameter that has been included Siebel 7.2 onwards, and is set to either T%7E or /*)SE, with T%7E being the default value. The default value results in EIM not performing a cascade update on the primar+ child column. Setting the parameter to /*)SE indicates that EIM will perform the cascade update. Setting DeleteS2ipPrimary to /*)SE ma+ affect performance, as additional @ueries will be generated b+ EIM. /or e2ample,
EIM Handbook
Page: 11 of 47
Oracle
TYPE = DELETE BATCH = 20 TABLE = EIM_FN_ASSET1 ONLY BASE TABLES = S_ASSET_POSTN DELETE EXACT = TRUE
In the above process, EIM will delete records from SA*SSETA!$ST: but the !%A!$ST:AI( column in SA*SSET will not be ad-usted to reflect the changed relationship. That is, !%A!$ST:AI( ma+ still contain the %$5AI( of the old SA!$ST: record rather than 8:o Match %ow Id9 as is desired. The !%A!$ST:AI( field in SA*SSET can be reset to reflect the change, b+ adding the following line in the process4
DELETESKIPPRIMARY = FALSE3
EIM Handbook
Page: 12 of 47
Oracle
5.2.4 MERGE
* merge process deletes one or more e2isting rows from the base table and ensure that intersecting table rows are ad-usted to refer to the remaining rows. Some process parameters like TK!E, .*TC> and T*.)E are similar to that in case of the above mentioned processes. 7pon completion of the merge process, the some row survives and the remaining rows are deleted. /or surviving records, the I/A%$5AME% EAI( column in the EIM table is set to :7)). /or rows to be merged "and subse@uentl+ deleted#, this column is set to the value of the base table %$5AI( where this row will be merged. *ll child and intersection table rows that previousl+ pointed to %$5AI(s of the deleted record now point to the merged "survivor# record. /or deleted rows, EIM sets TAME% E(A%$5AI( to the %$5AI( of the row that was merged into "the surviving row# and TA(E)ETE(A%$5AI( to the %$5AI( of the deleted base table row. UPDATE R()S4 This parameter is necessaril+ to be set to T%7E in case of Merge process. It specifies whether the foreign ke+ "or ke+s#, which references the merged rows, in the named table need to be ad-usted. Lalid values are T%7E "the default# and /*)SE.
[Merge Accounts] TYPE = MERGE BATCH = 1 TABLE = EIM_ACCOUNT UPDATE ROWS = TRUE
To set the value to /*)SE, the table name on which the parameter setting will act should be mentioned.
[Merge Accounts] TYPE = MERGE BATCH = 1 TABLE = EIM_ACCOUNT UPDATE ROWS = S_ADDR_PER, FALSE
EI can only be #sed to merge ro!s $rom target base tables and not secondary base tables4 -or e5ample/ EI 6ASSET can only be #sed to merge ro!s in target base table S6ASSET and not secondary base table S6ASSET6C(N4
EIM Handbook
Page: 13 of 47
Oracle
+ ,unning EIM
The EIM !rocess consists of e2ecuting the following se@uence of steps4 Populate the column !n the Inte"#ace$ EIM ta%le that a"e "e&u!"e' to loa' the S!e%el %a e ta%le
("!te$E'!t the EIM Con#!)u"at!on *IFB+ F!le to 'e#!ne the EIM P"oce to pe"#o"m
RUN the EIM , Su%m!t the EIM a a S!e%el Se"-e" Batch component ta .
R#nning t"e EI ' The EIM !rocess can be run once the EIM Tables have been prepared, i.e. suitabl+ populated, and the EIM configuration file has been accordingl+ written. The EIM process can be initiated b+ running a server task for the Enterprise Integration Manager component. $n each pass, EIM processes one interface table and performs a particular action "IM!$%T, (E)ETE, E'!$%T, ME% E# on all rows in that table for that batch. There are two methods for running an EIM process4 ()) %unning an EIM !rocess 7sing the raphical 7ser Interface " 7I# ())) %unning an EIM !rocess 7sing the Command1)ine Interface
Note4 Transaction will be logged during EIM to s+nchroni0e mobile clients. (uring initial data loads the Transaction )ogging parameter should be set to /*)SE in the S+stem !references settings under *pplication *dministration, in order reduce transaction activit+ to the Siebel docking tables. $nce the initial loads are complete it can be set back to T%7E if there are Siebel Mobile 5eb Clients.
EIM Handbook
Page: 14 of 47
Oracle
+*1
In the Component$3 o% '!alo) %o41 elect the Ente"p"! e Inte)"at!on M)" component an' cl!c. OK2
In the Component$3 o% #!el'1 cl!c. the ell!p ! %utton2 The Component$3 o% '!alo) %o4 appea" 2
In the Component Re&ue t Pa"amete" l! t1 a'' o" chan)e an5 component pa"amete" #o" the EIM p"oce
In the Component Re&ue t #o"m1 cl!c. the menu %utton1 an' then cl!c. Su%m!t "e&ue t
EIM ta"te'
The following screens shows how EIM can be run from 7I4
EIM Handbook
Page: 15 of 47
Oracle
EIM Handbook
Page: 16 of 47
Oracle 3. -,, $ * ''a+. #a+a" t +' i$ !o"#o$ $t R %& 't Pa+a" t +' -##/ t.
4. 0&1"it t2 + %& 't f+o" !o"#o$ $t R %& 't fo+" " $&
EIM Handbook
Page: 17 of 47
Oracle
process. Lalue B specifies the EIM process to use the batch number as specified in the EIM configuration file. 2. Con$ig#ration $ile4 This parameter names the configuration file to be used b+ the EIM process. This file should be placed in the *(MI: subdirector+ of the Siebel server. 3. Error -lags/ SQ% Trace -lags and Trace -lags4 These parameters are used to specif+ whether logging of Error, S3) Statements and other EIM operations would be done in detail or in brief. * detailed discussion on the settings of these flags, and their effect on EIM performance can be found in the Error >andling Section of this guide.
+*2
To start the srvrmgr program /or 5indows servers onl+4 at the ($S prompt, change to the Pbin subdirector+ within the Siebel Server root director+4
cd \siebel_server_root\bin
Start the svrmgr program along with the parameters mentioned below4
4.at it &0eci(ie& S5nta6 (or 4indo7& S5nta6 (or -NI8 ,e9uired
Para3eter
gateway_server
Net!or2 address o$ t"e Siebel ,ate!ay mac"ine Siebel Enterprise Server name Siebel Server name 7t"e de$a#lt is all servers8 Siebel Server administrator #sername Siebel Server administrator pass!ord
:g
$g
;e&
entrpr_server
:e
$e
;e&
siebel_server username
:&
$&
No
:u
$u
;e&
password
:0
$0
;e&
EIM Handbook
Page: 18 of 47
Oracle
G. E2ecute a start task command or a run task command on the EnterpriseIntegrationMgr component. The Configuration file name needs to be specified here using the C(N-I, parameter. The default path for the Configuration file would be in the *(MI: sub1director+ within the Siebel Server root director+. If the Configuration /ile is paced in some other director+, the entire path of the .ifb file needs to be specified when starting the EIM component. If the .ifb file is not specified at all, then the de$a#lt4i$b configuration file will be used. Sample synta5 $or speci$ying t"e R#n Tas2 command'
r)& "!s+ ,-r .-m'-&#&" #(m $("/ .-&,(g=(m'-r"0(,1
EI is a m#lti-step process4 When the EIM process is running, the task should not be interrupted9 ot"er!ise/ some steps may not roll bac2 correctly :eri$y t"at Siebel ; database/ Siebel server7s8 are #p and r#nning and t"e Server Components li2e Server anager/ Server Re<#est +ro2er/ Server Re<#est Processor and Enterprise Integration anager are also #p and r#nning
+*3
The order in which legac+ data is imported is critical to make sure that relationships between dependent data elements are established correctl+. Siebel interface tables do not map one1to1one with Siebel target database tables. To make sure that the necessar+ data is present to establish relationships between data entities, import data in the following recommended order4 1. 2. 3. %. 5. 6. 7. 8. 9. 1". 11. 12. 13. 1%. 15. 16. 17. 18. 19. 2". 21. *dministrative .usiness 7nit !ositions *ccounts Contacts Emplo+ees !roducts $pportunities !ersonal *ccounts 3uotes (ocuments /orecasts /ulfillment Marketing Campaigns C! !romotion Management C! !roduct Movement Service %e@uests !roduct (efects *ctivities and *ppointments :otes /ile *ttachments
Page: 19 of 47
EIM Handbook
Oracle
This import order reflects most import processes. In some cases, the import order ma+ var+ slightl+ depending on the re@uirements. * sample recommended order of importing could be found in the Siebel provided default ifb file. N(TE4 *n e2ample of administrative data would be a )ist of Lalues for Currenc+ or Qip Code.
<*1
>ere is a list of all factors that should be kept in mind while choosing the EIM Table. +ase table 1 Identif+ all the .ase tables that need to be populated. This is driven b+ re@uirement. EI table 1 Identif+ all the EIM tables that contain mappings to the .ase Table. The @uer+ that will help to determine all the EIM tables that populate a particular .ase table is4
SELECT DISTINCT FROM 3HERE AND AND AND AND AND ORDER BY T20NAME BASE2 T10NAME EIM S_TABLE T12 S_TABLE T22 S_EIM_TBL_MAP MAP T20NAME=44g(v# B!s# T!15# N!m# /#r#66 T10RO3_ID=MAP0IF_TBL_ID T20RO3_ID=MAP0DEST_TBL_ID T10INACTI7E_FL8=9N9 T20INACTI7E_FL8=9N9 MAP0INACTI7E_FL8=9N9 T10NAME:
Target Table 1 In the I/. file, while specif+ing the base tables that a particular EIM table will load, it is mandator+ to specif+ the target base table for the EIM table. If the target table is not mentioned, EIM will generate an error. The following @uer+ determines the target base table for an EIM table4
SELECT DISTINCT FROM T10NAME T20NAME S_TABLE S_TABLE SOURCE2 TAR8ET T12 T2
Page: 20 of 47
EIM Handbook
Oracle 3HERE AND ORDER BY T10TAR8ET_TBL_ID = T20RO3_ID T10NAME =44g(v# EIM T!15# N!m# /#r#66 T10NAME:
EI Table Selection to load ma5im#m base tables 1 5hen more than one related .ase tables need to be populated, instead of selecting different EIM tables for each .ase table, the optimal choice would be an EIM table that would contain mapping to all the related .ase tables that need to be populated. The following @uer+ determines the .ase tables that can be loaded from a particular EIM table4
SELECT FROM 3HERE AND AND AND AND AND ORDER BY T10NAME EIM2 T20NAME BASE S_TABLE T12 S_TABLE T22 S_EIM_TBL_MAP MAP T10RO3_ID=MAP0IF_TBL_ID T20RO3_ID=MAP0DEST_TBL_ID T10INACTI7E_FL8=9N9 T20INACTI7E_FL8=9N9 MAP0INACTI7E_FL8=9N9 T10NAME =44g(v# EIM T!15# N!m# /#r#66 T10NAME:
/or e2ample, consider a situation where both the tables SA!*%TKA!E% and SA!$ST:AC$: need to be populated for storing relationships between an Emplo+ee and a position. Since emplo+ee related data are usuall+ migrated using EIMAEM!)$KEE, this interface table would seem like an obvious choice. .ut EIMAEM!)$KEE does not have mapping with SA!$ST:AC$:. So in this case we could i. ii. )oad SA!*%TKA!E% using EIMAEM!)$KEE and SA!$ST:AC$: using EIMAC$:T*CT. *lternativel+, we could use onl+ EIMAC$:T*CT, since this interface table loads both SA!$ST:AC$: as well as SA!*%TKA!E%.
The choice ii would be optimal since our purpose is getting solved using a single EIM table. EI Table Selection to load ma5im#m col#mns 1 *mong these Interface tables, select the Interface table that contains mappings to the ma2imum number of .ase table columns. The ideal choice would be the interface table that would have mapping to all the base table columns that have been identified for population. /or e2ample, the base table SAC$:T*CT can be loaded using both EIMAEM!)$KEE as well as EIMAC$:T*CT. .ut if our re@uirement indicates that a specific column, sa+, !%A>E)(A!$ST:AI( of SAC$:T*CT needs to be populated then we would have to choose EIMAEM!)$KEE because EIMAC$:T*CT does not have mapping to !%A>E)(A!$ST:AI( column of SAC$:T*CT. >owever, if our re@uirement were to populate !%A!$ST:AI( of SAC$:T*CT, then we would have to choose EIMAC$:T*CT and not EIMAEM!)$KEE because EIMAEM!)$KEE does not have mapping to !%A!$ST:AI( column of SAC$:T*CT.
EIM Handbook
Page: 21 of 47
Oracle
<*2
Some base tables like :ote tables do not have user ke+s. Since user ke+s pla+ a vital role in migration of data using EIM, the EIM behavior is significantl+ different in case of .ase Tables without user ke+s. /or such tables, although IM!$%T and E'!$%T processes work, but Merge process does not work. The (E)ETE process, when using (E)ETE M*TC>ES and (E)ETE *)) %$5S parameters, works fine, but does not work with (E)ETE E'*CT parameter. This is because the EIM process cannot identif+ the rows to delete, in the absence of user ke+s. >owever, Siebel 7.E.2 versions have made deletion of data from :otes tables, using a special EIM, EI 6N(TE6DE%. The EIMA:$TEA(E) interface table should onl+ be used to delete notes from the SA:$TEAR tables. To use it, users need to populate %$5AI(s of records to be deleted into corresponding columns in the interface table and then use (E)ETE E'*CT to delete them. /or further details, please refer the Ot.er I&&ue& ,elevant to Note& Migration ' *tio$ of t2
Not 'DataMig+atio$G&i, .
<*3
3'i$g EIM ta'4 &' + *a$$ot $o+"a//. &#,at 3' + 4 . att+i1&t ' of t2 1a' ta1/ . 5o( v +6 t2 + a+ 'o" '# *ia/ i$t +fa* ta1/ ' EIM7ORG7EXT738 a$, EIM7PROD7INT738 t2at *a$ 1 &' , to &#,at &' + 4 . *o/&"$' fo+ 'o" 4 . ta1/ ' 07ORG7EXT a$, 07PROD7INT.
-0dating u&er ke5 colu3n& u&ing EIM ta&k 7it. t.e&e &0ecial table& 7ill not u0date denor3ali>ed colu3n&* "or e6a30le? u0dating S@O,/@E8T*'OC 7ill not u0date S@ CCNT@POSTN* CCT@'OC N% S@O,/@#-*O,/@'OC*
<*!
It is mandator+ to load the re@uired columns of the base tables. If one or more re@uired fields are missed, then the EIM process will fail to load the base table and generate the %E37I%E(AC$)S error status. To identif+ which are the re@uired columns for a base table, the following @uer+ can be used4
SELECT FROM 3HERE AND AND TBL0NAME2 COL0NAME S_COLUMN COL2 S_TABLE TBL COL0TBL_ID=TBL0RO3_ID COL0RE;UIRED=9Y9 TBL0NAME = 44g(v# B!s# T!15# N!m# /#r#66:
Similarl+ to determine the user ke+s for a base table, the following @uer+ can be used4
EIM Handbook
Page: 22 of 47
Oracle SELECT *(s"(&." FROM 3HERE AND AND TBL0NAME2 COL0NAME S_COLUMN COL2 S_TABLE TBL COL0TBL_ID=TBL0RO3_ID TBL0NAME = 44g(v# B!s# T!15# N!m# /#r#66 COL0USR_KEY_SE;UENCE IS NOT NULL:
If the above @uer+ does not return an+ row, then that implies that the corresponding base table does not have an+ user ke+.
<*)
The uni@ue identifier associated with ever+ record in Siebel Enterprise databases is known as a %ow I(. The column in which this value is found is %$5AI( and it is present on ever+ table. The %$5AI( is uni@ue for an entit+. /or e2ample, the %$5AI(s for the same person in SA!*%TK, SAC$:T*CT, and SAC$:T*CTA' are the same because the+ each refer to the same person. %ow I(s are used e2tensivel+ throughout Siebel Enterprises to access specific records. *lthough users access records b+ a 7ser !rimar+ Oe+ "such as $pportunit+ :ame#, it is more efficient for the Siebel Enterprise to store and access related data via the %ow I(. = The %ow I( is a base1H, se@uence number generated using a confidential, proprietar+ algorithm that ensures no duplication, thus protecting the referential integrit+ of the database. The %$5AI( column is L*%C>*%"DE#, which ma+ contain one or two non1numeric s+mbols "plus signs or h+phens, or both#. The format of the %$5AI( is one of the following4 C!1:S C!S:!S:S C!1:!1:S where4 C! :! :S F Corporate !refi2, up to G alphanumeric characters F :e2t !refi2, up to , alphanumeric characters F :e2t Suffi2, up to 7 alphanumeric characters %ecords created b+ the user interface %ecords created b+ Interface Manager "EIM# %ecords created b+ EIM "Starting in Siebel versions ,.G and higher, and Siebel version 7#
The ma2imum length of the %$5AI( is DE alphanumeric characters. The corporate prefi2 will alwa+s be uni@ue for an+ database "main or local#. The server maintains its original value, and mobile databases created against that particular server database are alwa+s assigned a new, uni@ue value. The Siebel %$5AI( is a combination of the SASE37E:CEAS and information from SASS*AI( table. *ll connected users share the same prefi2, which is obtained from the table SASS*AI( on the server
EIM Handbook
Page: 23 of 47
Oracle
database. %emote 7sers are assigned a uni@ue prefi2 when the+ are db1e2tracted. This value is also stored in the SASS*AI( table on the local database. Suffi2 is generated using an internal algorithm handled b+ the source code. 5hen a new record is created through the user interface, the Siebel application reads the value of the current :S column from SASS*AI( table and increments this value b+ a value more than D J for performance reasons, generall+ EB. The client caches these fift+ potential %$5AI(s for future inserts. * new record entered from the user interface ma+ result in man+ inserts to the underl+ing tables, depending on the business components used. 5hen the client disconnects, cached %$5AI(s are lost. The combination of the prefi2 and suffi2 generates a uni@ue rowAid. Since Siebel does not e2pose the algorithm behind generation of row ids, the %$5AI( generation is internal to the EIM process. Therefore Siebel provides user ke+s to map to a uni@ue record in the base table. (uring EIM, the EIM table columns, which map to the user ke+s of the base table, are populated with values so as to map to a uni@ue record. If that record is to be updated b+ EIM later, the same user ke+ values have to be populated in the corresponding EIM table columns. $therwise, it will not resolve into the correct base table record. /or e2ample, !E%S$:AI( and .7AI( constitute the user ke+s for SAC$:T*CT. The corresponding EIMAC$:T*CT columns are C$:A!E%S$:A7I( and C$:A.7. 5hen a new record is imported in SAC$:T*CT using EIMAC$:T*CT, the C$:A!E%S$:A7I( and C$:A.7 should uni@uel+ identif+ a Contact record. :ow, for updating some fields for this Contact using EIMAC$:T*CT, C$:A!E%S$:A7I( and C$:A.7 have to be populated with the same set of values that were used during the initial load in order to map to the same Contact record in the base table. Similarl+ while loading a child table, which references the row id of the master table, the EIM table columns that map to the foreign1ke+ column of the child table must be populated in the same wa+ the user ke+s of the master table were populated. /or e2ample, SAC$:T*CTA'M is a child table of SAC$:T*CT. The column !*%A%$5AI( of SAC$:T*CTA'M references the %$5AI( of SAC$:T*CT. SAC$:T*CTA'M is loaded b+ EIMAC$:A(T). The columns of EIMAC$:A(T) that need to be populated to generate the !*%A%$5AI( of SAC$:T*CTA'M are C$:A.7, C$:A!E%S$:A7I( and C$:A!%ILA/) . These EIM columns should be populated with the same set of values that were used to populate its parent SAC$:T*CT record through EIMAC$:T*CT. :ote that though !%ILA/) is not a part of the user ke+ of SAC$:T*CT, but it is a re@uired column and is part of the foreign1ke+ mapping of its child table.
EIM Handbook
Page: 24 of 47
Oracle
A*1
The following chart lists the possible values of I/A%$5AST*T, what the+ indicate and some possible resolutions. I-6R()6STAT I:A!%$ %ESS Indicates Integration Manager sets I/A%$5AST*T to this initial value for all rows in the batch. If rows still have this status value after Integration Manager e2its, a failure occurred that aborted processing for this table. * S3) error occurred during an attempt to import this row. This error occurs for rows processed when transaction logging is T%7E. Resol#tion $ne possible reason ma+ be that an e2tension column in the .ase table that the EIM table maps to does not e2ist. *nother reason ma+ be that there is insufficient space in the Siebel Server for the generation of )$ files. In this case, the S3) statement that failed e2ecution is logged in the EIM log file. This statement can be anal+0ed to deduce the specific error.
S3)AE%%$%
EIM Handbook
Page: 25 of 47
Oracle
I-6R()6STAT IM!$%TA%ETECTE(
Resol#tion The @uer+ specified using the /I)TE% 37E%K parameter needs to be anal+0ed and corrected. The @uer+ e2pression should be a self1contained 5>E%E clause e2pression "without the 5>E%E ke+word# and should use onl+ un@ualified column names from the interface table or literal values "such as :*ME IS :$T :7))#. $ne reason could be that all the columns in the EIM table that map to the foreign ke+ in the base table have not been populated. *nother reason could be that the EIM table columns mapping to the foreign ke+ column, resolve into a %$5AI( that is non1e2istent in the /oreign Oe+ table. The error could be resolved b+ ensuring that a picklist e2ists for the column that is )$L bounded. The value being imported for this column corresponds to one of the values in the picklist.
/$%EI :AOEK
* re@uired foreign ke+ column in the target table could not be resolved.
!ICO)ISTAL*)7ES
* re@uired picklist value in the target table could not be resolved. This error occurs for :7)) or invalid bounded picklist values.
EIM Handbook
Page: 26 of 47
Oracle
I-6R()6STAT %E37I%E(AC$)S
Indicates $ne or more re@uired columns for the target table were :7)).
Resol#tion .efore running the EIM, it should be ensured that the re@uired columns in the base table"s#are being populated. Even if these columns are not specified in the EIM table, the+ could be defaulted to specific values using the (E/*7)T or /I'E( C$)7M: parameters in the I/. file. It should be ensured that in the EIM table, records with the same batch number have distinct uni@ue ke+ combinations.
(7!A%EC$%(AE'ISTS
The row e2actl+ matches rows that alread+ e2ist in the destination tables. This error can also occur when the same record "same user ke+# e2ists with the same EIM batch number with a lower %$5AI(. In this case, MI: "%$5AI(# is the record processed, and the other records with the same user ke+ are marked as (7!A%EC$%(AE'ISTS. The row did not fail for the target table "although it ma+ have been a duplicate#, but did fail during processing of a secondar+ base table. This status is set after the import has completed.
!*%TI*))KAIM!$%TE(
In case, a parent1child relationship e2ists between the target table and the secondar+ base table, the user ke+s mapping to the /oreign Oe+ column in the Child table referencing the !arent table must be populated in the EIM table in e2actl+ the same wa+ in which the user ke+ columns resolving to %$5AI( of the parent table are populated.
EIM Handbook
Page: 27 of 47
Oracle
I-6R()6STAT (7!A%EC$%(AI:AEIMAT.)
Indicates Indicates that there are G or more interface records having the same user ke+ values.
Resol#tion To avoid this situation, anal+0e the input records before beginning the EIM task. If +ou find duplicate records, +ou can either combine them into one record, or specif+ a different batch number for the duplicate record. > It is necessar+ to identif+ the particular 7SE%OEK that the EIM table uses and populate columns mapping to that 7SE%OEK uni@uel+.?
:$:A7:I3EA7OEKS
*M.I 7$7S
This error occurs if there is more than one 7SE%OEK, sa+ UU.asetable nameA7DVV, UU.asetable nameA7GVV, etc. defined on a base table. The particular EIM table being used to populate the base table uses the second 7SE%OEK but the user has populated EIM columns corresponding to the columns of the first 7SE%OEK This error is usuall+ encountered when there are some columns in the .ase table, other than the uni@ue ke+ columns, which also need to be uni@uel+ populated for each record. This is because of uni@ue inde2 database constraint defined on these columns. This error is also encountered when attempting to run (E)ETE or ME% E processes on records in the base table with same uni@ue ke+s but different conflictAids.
The workaround is to identif+ all the column combinations in the base table, which participate in the different uni@ue inde2 constraints defined on it, and populate each combination uni@uel+. @
EIM Handbook
Page: 28 of 47
Oracle
A*2
Sta"t
1. Query the appropriate interface tables for rows whose IF_ROW_BATC _!"# e$uals the batch nu%ber for the i%port.
&. Chec' the (alue in IF_ROW_)TAT colu%n for all these rows
:o
Kes
.. "sin/ the chart of possible IF_ROW_)TAT (alues0 analy1e the possible cause of error
3. )earch for the (alue in the +I# lo/ file for the current run
4. This (alue correspon5s to a particular pass nu%ber in the +I# process0 an5 is lo//e5 in the +I# lo/ file in case of +rror
6. "sin/ the 5escription in the +I# lo/ file correspon5in/ to the IF_ROW_)TAT_!"# (alue0 5e5uce the cause of the error
En'
EIM Handbook
Page: 29 of 47
Oracle
The Step , in the above illustration needs to be modified in case the particular EIM table is being used to migrate data to more than one base table. /or each base table, SA'KQ that the EIM table maps to there is a special temporar+ column in the EIM table, called TA'KQAAST*. If the migration from the EIM table to the base table is successful then the TA'KQAAST* column is populated with B. $therwise, this column is populated with the pass number in which the error occurred. So, if an EIM table maps to multiple base tables, in a particular batch, it is advisable to check the individual TARAAST* column for each base table, instead of the I/A%$5AST*TA:7M column which contains the result of the last migration "i.e. migration to the last .ase table mentioned in the I/. file for that EIM table for that particular batch#. /or e2ample, the table EIMA%ES! loads the base tables namel+ SA%ES!, SA*!!ALIE5A%ES!, SA!E%A%ES! and SA%ES!A.7. The four corresponding temporar+ columns in EIMA%ES! are TA%ES!AAST*, TA*!!LIE5%EAAST*, TA!E%A%ES!AAST* and TA%ES!A.7AAST* respectivel+. *fter running EIM to load an+ or all of these base tables, the temporar+ columns ma+ assume an+ of the following values4 "i# B4 The corresponding base table is successfull+ loaded "ii# :7))4 )oading of the corresponding base table was not desired "iii# :umerical Lalue4 Error occurred while loading the corresponding base table /or e2ample, as the screenshot below suggests, while loading SA%ES!, SA*!!ALIE5A%ES! and SA!E%A%ES! "SA%ES!A.7 is not being loaded# through EIMA%ES!, the status of TA%ES!AAST* and TA*!!LIE5%EAAST* are 8B9 that is SA%ES! and SA*!!ALIE5A%ES! tables have been successfull+ loaded. Since SA%ES!A.7 is not loaded, the corresponding EIM column TA%ES!A.7AAST* is :7)). *n error "!*%TI*))KAIM!$%TE(# has occurred while loading SA!E%A%ES! and the corresponding EIM column TA!E%A%ES!AAST* contains the particular pass number at which the error occurred.
Note4 The numerical value corresponds to a particular pass number in the EIM process. The EIM log can then be scanned to check what went wrong in that particular pass while loading the corresponding base table. The )$ file "referenced in Step 7# for the current run can be found under the log sub director+ of the root director+ of the Siebel Server. The detail1level of the description found in the )og /ile, depends
EIM Handbook
Page: 30 of 47
Oracle
on the parameters set for Trace /lags, and Error /lags, when running the EIM task. The settings for the various flags will be discussed in details in the following section. The log can also be viewed from the 7I b+ following these steps4 :avigate to the Tasks screen. Click Task Info )og view tab. In the Tasks list, select the task for the EIM process. The log is displa+ed in the Task Info )og list. A*3
The settings for the T%*CE /)* S, S3) T%*CE /)* S and E%%$% /)* S can be specified either from the 7I or from the Command !rompt when starting an EIM task. If the EIM process is being run from the 7I, then the flags can be set in the C$M!$:E:T %E37EST !*%*METE%S list in the C$M!$:E:T %E37ESTS Liew of the Enterprise $perations Screen.
EIM Handbook
Page: 31 of 47
Oracle
If the EIM !rocess is being run from the command prompt, then the flags can be set within the command for running the EIM task, as shown in the sample below4 srvrmgr /g siebelserver /e AspireEnt7 ! /s "IE#E$"E%&E% /u sadmin /p sadmin /c 'run task (or component eim with con(ig)**(ilename+i(b,,,Error(lags)-,".l(lags)/,0race(lags)!'
Note4 Task parameters can override component1level event logging to create additional information in the log. In Siebel, parameters can be set, from highest to lowest, at Enterprise, Server, Component (efinition, :amed Subs+stem, Sever Component and Task level. * parameter set at a lower level can override the same parameter set at a higher level. Error -lags' .+ setting the Error /lags to appropriate levels, a detailed e2planation of rows that were not successfull+ processed, can be obtained. Setting the task parameter WError /lagsW F D can be used to write a detailed description of the cause of each error to the task log during processing. SQ% Trace -lags' The S3) Trace flags parameter is used for logging S3) statements that make up the EIM task. Setting the S3) Trace /lags parameter to X creates a log of the summar+ S3) statements that make up the EIM task. Since it dramaticall+ impedes EIM performance and created a large log file, hence this option is recommended onl+ for test environments. The lower values for S3) (ebug /lags "D, G, and I# are used for logging at the $(.C level.
EIM Handbook
Page: 32 of 47
Oracle
Trace -lags' Trace flags contain logs of various EIM processing steps. Siebel 7.2 onwards / Event %ogging needs to be set for the EIM component, in order to activate Trace /lags. /or setting Event %ogging/ the following se@uence needs to be followed4 Click the Server *dministration screen tab. /rom the Show drop1down list, select Component. Select Enterprise Integration Manager as the component. Click Component Event Configuration. !erform a @uer+ and enter the )og )evel values for the following4 E:ENT T*PE S3) Tracing S3) Summar+ Task Configuration Component Tracing %(, %E:E% :A%UE I I I H
Trace /lags are bit1based. *vailable Trace /lags include D, G, I, X, and HG. To activate multiple trace flags, the Trace /lags parameter has to be set to the sum of individual trace flag numbers. /or e2ample, to log trace flags G and I, the Trace /lags parameter needs to be set to ,. Setting Trace -lags to 0' Setting the Trace /lags parameter to D creates a step1oriented log of the task. This can be used to determine the amount of time EIM spends on each step of the EIM task, or for each interface table processed. Setting Trace -lags to 3' Setting the Trace /lags parameter to G creates a file log that traces all substitutions of user parameters. Setting Trace -lags to >' Setting the Trace /lags parameter to I creates a file log that traces all user1ke+ overrides. Setting Trace -lags to A' Setting the Trace /lags parameter to X creates a file log that traces all Interface Mapping warnings. Setting Trace -lags to =3' Setting the Trace /lags parameter to HG creates a file log that traces all file attachment status. The trace file contains four labels, three of which are used to trace file attachment processes as described. %A+E% *ttachment Imported *ttachment "$ld# (eleted *ttachment :ot /ound
EIM Handbook
DESCRIPTI(N Indicates whether the file attachment was encoded, compressed, and copied to the Siebel file server with the new name This label applies onl+ to updates and indicates whether an e2isting file was replaced and deleted Indicates that the file attachment cannot be found in the input director+
Page: 33 of 47
Oracle
Activating $lags !ill "ave a direct negative e$$ect on per$ormance since/ since a lot o$ in$ormation !ill be recorded in t"e log $ile4 Typically/ activating $lags s"o#ld only be done !"en testing EI processes/ and s"o#ld be avoided in a prod#ction environment #nless absol#tely necessary4
Oracle
I:C)7(E F WImport !osition $rgani0ationW I:C)7(E F WEmplo+ee !osition %elW Suppose, the logic for migrating !osition $rgani0ation relationship has undergone some change. >ence the MImport !osition $rgani0ationN !rocess needs to be tested. So, in the I/. file we comment out all other processes and keep onl+ INC%UDE B CImport Position (rganiDationC uncommented. 5e delete the records in the SA*CC:TA!$ST: table, which stores the !osition $rgani0ation %elationship, and then run the EIM process. This works fine. >owever, suppose the logic for migrating !osition data has undergone some change, which is wh+ the MImport !ositionN !rocess needs to be tested. .ut in this case, if we comment out all other processes and run the EIM onl+ for EImport PositionF, after deleting records from SA!*%TK and SA!$ST:, then the !osition $rgani0ation relationship data records would become orphans. Similarl+, records in all other tables, which have a foreign ke+ relationship with SA!$ST:, would become orphan records. This is because ever+ time we delete a .ase table "either manuall+ or through EIM#, and then run the EIM process to migrate records into it, SIE.E) generates new %$5AI(s for each of these records. >ence the foreign ke+ reference that other tables have with this record would be lost. >ence the thumb rule that should be followed is4 5hen testing data migration for a Master table "which has foreign ke+ relationship with one or more child tables#, run the EIM process not onl+ to import the Master table records, but also to import records to all tables that have foreign ke+ relationship with the !arent table. In our e2ample, therefore if we need to test the migration of !osition Master records into SA!$ST:, we would need to run the EIM process for EImport PositionF, WImport Position (rganiDationC and CEmployee Position RelC to ensure that foreign ke+ references are not lost.
EIM Handbook
Page: 35 of 47
Oracle
T&E ISSUE' *lthough the above scenario generall+ holds true, however it has been noticed that the $rgani0ation fields in some of the EIM tables are mapped directl+ to the relevant RA.I column rather than the relevant RA.7 column. In such cases, the $rgani0ation columns in the Siebel .ase tables cannot be resolved b+ populating the RA.7 columns onl+. /or some columns, the RA.7 column e2ists but is not used. Instead, EIM erroneousl+ maps directl+ the RA.I column. This means that for relevant RA.7 columns, the appropriate organi0ation is not set. Appendi5 A contains a list of all EIM tables, which contain the above1mentioned erroneous mapping. T&E )(R1AR(UND' To work around the issue the following steps need to be followed. ; 1. Manuall+ populate the relevant RA.I column within the EIM Interface Table with correct value using the following S3) @uer+4
%&*+TE SIE'EL,<EIM Tab ! "am!# SET (<EIM 'I fi! -#) . (SELECT MI"(C+SE /0E" OI,"+ME . <EIM '% fi! -# T0E" OI,RO/_I* E"*) FROM SIE'EL,S_OR$_E1T OI /0ERE OI,"+ME . <EIM '% fi! -#) /0ERE (IF_RO/_'+TC0_"%M . <'atch numb!r#)
%eplacing4 UEIM Table :ameV b+ the relevant EIM Interface Table UEIM .I fieldV b+ the relevant RA.I column UEIM .7 fieldV b+ the associated RA.7 column U.atch numberV b+ the relevant EIM .atch :umber /or e2ample, if using the EIMA/:A*SSET Interface Table in order to populate the target table SA*SSET, the EIMA/:A*SSET.*STA!%$(A.I column would need to be updated in order to reflect the correct %$5AI( in order to resolve the correct organi0ation. iven an EIM batch number of DBBD, the following S3) can be used4
%&*+TE SIE'EL,EIM_F"_+SSET SET (+ST_&RO*_'I) . (SELECT MI"(C+SE /0E" OI,"+ME . +ST_&RO*_'% T0E" OI,RO/_I* E"*) FROM SIE'EL,S_OR$_I"T OI /0ERE OI,"+ME . +ST_&RO*_'%) /0ERE (IF_RO/_'+TC0_"%M . 2332))
EIM Handbook
Page: 36 of 47
Oracle
2. $nce the RA.I column has been populated, the EIM process can be run as usual.
In Siebel 7.E.2, if we have .S*/ files in /ile S+stem of Siebel Installation Environment, we can run EIM e2port and the original attachment files will be available in the $7T!7T folder of Siebel Server. .ut for Siebel ,.2, this does not happen. *nd this is a Siebel !roduct defect. /or Siebel ,.2, we have to uncompress the .S*/ files with the help of one utilit+ program, called SSE7:QI!.E'E, which resides in .I: director+ of Siebel Server. /or migration of attachment record and to generate new S*/ files, we need to keep all the files "with actual e2tension# in I:!7T folder of the Siebel 7 Server. Then we have to load EIM table with proper sets of values. $nce EIM is loaded, we have to run EIM batch. 5hen the EIM batch is run for loading base tables, the EIM process itself will search for the attachment files in the I:!7T director+ and create one S*/ for each attachment file in the /ile Server *ttachment director+ PPsea7EGPsiebfile7EHPatt. /or further details, please refer to the *ttachmentMigration uide.
EIM Handbook
Page: 37 of 47
Oracle
EIM Handbook
Page: 38 of 47
Oracle
Sometimes it is recommended to create additional inde2es to improve performance of time1consuming S3) statements. The EIM columns used in the 5>E%E clause of the S3) statements, in con-unction with the %$5AI( is a good option to induce the optimi0er to choose this inde2. The inde2 can be dropped after EIM e2ecution. Dropping +ase table inde5es d#ring initial r#n /or each row inserted in the base table, the inde2 pages of all the affected inde2es are modified. .ut most of these inde2es are never used during EIM. /rom past e2periences, it has been anal+0ed that the !rimar+ and 7ni@ue Inde2es can be retained for target base tables "like SA$% AE'T# while the remaining inde2es can be dropped. /or non1target tables "like SA*((%A!E%#, the !rimar+, 7ni@ue and /oreign ke+ Inde2es onl+ need to be retained. The database administrator can create these inde2es later in batch mode b+ utili0ing parallel e2ecution strategies available for the respective database platform. Controlling SiDe o$ batc"es $ptimal batch ranges have been observed to range an+where between EBB and DE, BBB rows. /or an initial load, one can use HB,BBB rows for a large batch. /or ongoing loads, GB,BBB rows for a large batch can be tried. $ne should not e2ceed DBB,BBB rows in a large batch. >owever, Siebel S+stems recommends that customers use a batch si0e no more than EBBB rows, preferabl+ less than GBBB rows in a single batch. G 7sing batch ranges ".*TC> F 2J+# allows us to run with smaller batch si0es and avoid the startup overhead on each batch. The ma2imum number of batches that can be run in an EIM process is D,BBB. USIN, S*N(N* S parameter #sage This parameter, 7SI: SK:$:KMS, controls the @ueries of account s+non+ms during import processing. 5hen set to /*)SE, this parameter saves processing time because @ueries that look up s+non+ms are not used. The default setting is T%7E. This parameter should be set to /*)SE when account s+non+ms are not needed. It should not be set to /*)SE if multiple addresses are used for accounts. $therwise, EIM will not attach addresses to the appropriate accounts. Transaction %ogging (uring initial data loads this parameter should be set to /*)SE in the S+stem !references settings within the Siebel application, in order reduce transaction activit+ to the Siebel docking tables. $nce the initial loads are complete it can be set back to T%7E if there are Siebel Mobile 5eb Clients. Disable Triggers (isabling database triggers from Server *dministration, can also improve EIM performance. It should be kept in mind that components such as 5orkflow Manager and *ssignment Manager will not
EIM Handbook
Page: 39 of 47
Oracle
function for the new or updated data. *lso the triggers should be reapplied after completing the EIM load. Parallel E5ec#tion o$ EI *fter appl+ing all the possible optimi0ation strategies, if the EIM throughput is to be increased, running EIM concurrentl+ can be considered. It should be ensured that the concurrentl+ running EIM processes should be mutuall+ e2clusive. The Ma2imum Tasks parameter should be checked before running parallel EIM processes. (ptimiDing SQ% %unning an EIM -ob with the following flag settings4 Error flag F D, S3) flag F X, and Trace flag F H will produce a log file containing S3) statements and how long it took for each statement. The S3) statements that are taking too long "on a run of EBBB rows in a batch, look for statements that took longer than one minute#, tuning measures can be taken to optimi0e the @ueries. UPDATE PRI AR* 1E*S and PRI AR* 1E*S (N%* parameter #sage The !%IM*%K OEKS $:)K parameter should be eliminated and 7!(*TE !%IM*%K OEKS parameter should be avoided in the I/. configuration file. Reg#lar Table aintenance
/re@uent insert or delete operations on interface tables can cause fragmentation in the table. %egular table maintenance b+ the database administrator on the interface tables will correct fragmentation problems in the EIM interface tables. *lso, if the data dictionar+, metadata information and statistics are updated for the database, EIM takes shorter time to e2ecute. /or e2ample,
ANALY<E TABLE 0000 COMPUTE STATISTICS
for the Siebel base and EIM tables improves the EIM performance. Clear EI tables
)eaving old batches in the EIM interface table will waste space and could adversel+ affect performance. It is a good practice to delete batches from interface tables on completion. Error %ogging parameters The EIM log file can contain different levels of detail information depending on the values of three flagsYthe Error flag, the S3) flag, and the Trace flag. Some of the recommended settings are described in the following list.
EIM Handbook
Page: 40 of 47
Oracle
*s a starting point, it is recommended to set the Error /lagFD, the S3) flag FD, and the Trace /lagFD. This setting will show errors and unused foreign ke+s. Setting the Trace /lagsFD will provide a summar+ "after each batch# of the elapsed time in step numbers DB and DD. Set Error flag F D, the S3) flag F X, and the Trace flag F H. These settings will produce a log file with S3) statements that include how long each statement took, which is ver+ useful for optimi0ing S3) performance. Set the Error flag F B, the S3) flag F B, and the Trace flag F D. These settings will produce a log file showing how long each EIM step took, which is useful when figuring out the optimal batch si0e as well as monitoring for deterioration of performance in a particular step.
*ctivating flags will have a direct negative effect on performance since, depending on the amount of data, a lot of information will be recorded in the log file. T+picall+, activating flags should onl+ be done when testing EIM processes. *ctivating flags in a production environment should be avoided unless absolutel+ necessar+. %efer Appendi5 + for a sample performance metrics chart of a tuned and optimi0ed EIM process.
12 ,e(erence
*) +) ,) -)
0
!lease refer to Service %e@uest Z4 12=12D6EF for further details. !lease refer to Service %e@uest Z4 HX17,CBC,IED. !lease refer to Service %e@uest Z4 12=19DX5N and /*3 DII, for further details.
>
The difference between (7!A%EC$%(AE'ISTS and (7!A%EC$%(AI:AEIMAT.) is following4 M(7!A%EC$%(AE'ISTSN status indicates that the same record alread+ e2ists in the base table and M(7!A%EC$%(AI:AEIMAT.)N status indicates that there are G or more interface records having the same user ke+ values. /or reference see Service %e@uest Z4 HX1CICBC,IED
?
!lease refer to Service %e@uest Z4 HX1DGDCCHIEX7 for more specific details. !lease refer to Service %e@uest Z4 HX17DEXBCXED for more specific details. !lease refer to Siebel *lert GC7 for further details.
!lease refer to Service %e@uest Z4 HX1,BDC7ECED, HX1CGBGGEGED, HX1DGIHBDEECD and HX1 7,,XGBEEDfor further details for further details.
G
EIM Handbook
Page: 41 of 47
Oracle
13
00endi6
EIM Table ((ected Colu3n PO07PROD7;I TERR7PROD7ID -0T7PROD7;I P-R7PROD7;I POLI!G7PROD7;I
EIM7>N7-00ET1
EIM7>N7-00ET710
EIM7>N7-00ET3
-0T7PROD7;I
EIM Handbook
Page: 42 of 47
Oracle
EIM7>N7-00ET4
EIM7>N7-00ET5
EIM7>N7!ONT-!T3 EIM7>N75LDNG
EIM7>N75LT5N!TR EIM7>N7IN0!LM1
EIM Handbook
Page: 43 of 47
Oracle
EIM7>N7OPTGPRD1
EIM7>N7REPL
ITM7PROD7;I P-G7PROD7;I
EIM70R<REF1
0R7PROD7;I 0R7PROD7;I1
EIM Handbook
Page: 44 of 47
Oracle
1!
00endi6 #
Ti3e E&ti3ate (or Siebel Initial %ata 'oad (ro3 Siebel EIM Table& to Siebel #a&e Table&
Ste0& Involved 'OD Li't of <a/& ' /rou0 Pa+ $t G+o&# EIM Table #a&e Table1&2 E&ti3ated Dolu3e E&ti3ated ,ecord& Per Minute 74 47 E&ti3ated Ti3e 1Min2 1.8 0.5 E&ti3ated Ti3e 1%a5&2 0.00 0.00
EIM7L0T7O>7<-L EIM7-!!O3NT
07L0T7O>7<-L 07P-RTG6 07ORG7EXT6 07-!!NT7PO0TN6 07ORG7;3 07-DDR7PER 07P-RTG6 07ORG7EXT6 07-!!NT7PO0TN6 07ORG7;3 07-DDR7PER 07P-RTG6 07ORG7EXT6 07-!!NT7PO0TN6 07ORG7;3 07-DDR7PER 07PROD7INT 07PROD7INT7;3 07PROD7INT7XM
135 23
EIM7-DDR7PER EIM7-!!O3NT
23 826715
210 654
0.1 126.5
0.00 0.09
EIM7-DDR7PER EIM7-!!O3NT
826715 896267
176092 622
4.8 143.5
0.00 0.10
0G -,,+ '' Product P+o,&*t P+o,&*t O+ga$i:atio$ P+o,&*t E9t $'io$' Plan P/a$ !/a'' Net7ork /rou0& N t(o+4 G+o&#
07-00ET 07>N7GRP7!L-00 07P-RTG6 07ORG7EXT6 07-!!NT7PO0TN6 07ORG7;3 07P-RTG6 07ORG7EXT6 07-!!NT7PO0TN6 07ORG7;3 07-DDR7PER 07ORG7PROD 07P-RTG6 07ORG7EXT6 07-!!NT7PO0TN6 07ORG7;3 07-DDR7PER 07ORG7EXT7XM 07ORG7EXT7>NXM
5096526 160526212 1
36617 56160 40
Net7ork N t(o+4
EIM7-!!O3NT
51
600
0.1
0.00
51 16479 296990
>a*i/iti ' -,,+ '' ' >a*i/iti ' !a#itatio$ !+ , $tia/' >a*i/iti ' 0# *ia/ti '
EIM Handbook
Page: 45 of 47
Oracle
Ste0& Involved Practitioner P+a*titio$ +' Aa4a 0i 1 / P+ovi, +B P+a*titio$ + -,,+ '' ' Aa4a 0i 1 / P+ovi, +B P+a*titio$ + 0# *ia/t. Aa4a 0i 1 / P+ovi, +B P+a*titio$ + Lo*atio$ 0 +vi* ' Aa4a 0i 1 / P+ovi, +B P+a*titio$ + !+ , $tia/' Aa4a 0i 1 / P+ovi, +B !o""o$ P+a*titio$ +' Aa4a 0i 1 / P+a*titio$ +B >a*i/it. -ffi/iatio$' N t(o+4 -ffi/iatio$' P+a*titio$ + -ffi/iatio$' Me3ber& M "1 +' M "1 +' -'' t' M "1 +' D tai/'
EIM Table
#a&e Table1&2
E&t Dolu3e
EIM7!ONT-!T
1456951
26308
0.04
EIM7!ONT-!T7XM EIM7!ONT-!T
1176926 1126297
216427 16070
5.5 105.0
0.00 0.07
56165 9 116755
M "1 +H0&1'*+i 1 +' M "1 + P!PH0t&, $tH5a $,i*a# M "1 +'=!O; M "1 +'=!O; !o$ta*t' M "1 +' -,,+ '' ' Clai3& !/ai"' !/ai"' P+ovi, +
07P-RTG 07-00ET7!ON 07P-RTG6 07!ONT-!T6 07PO0TN7!ON6 07!ONT-!T7;3 07P-RTG7REL 07!ONT-!T7XM 07>N7IN07!O; 07>N7IN0!O;7!ON 07-DDR7PER
9696500 966761
16164 16989 14 15
9696499
116930
EIM7>N7IN0!LM1 EIM7>N7IN0!LM1
EIM7>N7IN0!LM7 EIM7>N70R!P-G
07IN07!L-IM6 07IN0!LM7PO0TN6 07IN0!LM7;3 07IN07!L-IM6 07IN0!LM7ORG6 07IN0!LM7PO0TN6 07IN0!LM7;3 07IN0!LM7!ON 070R!7P-GMENT6 070R!P-G7IN0!LM 07IN07!L-IM7XM
165336128 360666256
584 16200
26625.2 26555.2
1.82 1.77
360666256 161616230
26274 26059
16348.4 564.0
0.94 0.39
EIM7>N7IN0!LM3
461346315
16800
26296.8
1.60
EIM Handbook
Page: 46 of 47
Oracle
Ste0& Involved Note& D / t Not ' -**o&$t Not !o$ta*t Not !/ai" Not
EIM Table
#a&e Table1&2
E&t Dolu3e
EIM Handbook
Page: 47 of 47