Informatica
Unix
Project
Functionality
Devlopment of Project
Simple Dimension Loading
Complex Dimension Loading
Simple Fact table loading
Complex Fact table Loading
Aggregates
cdc
src stg wh
100 1 100
1
date parameters
CDC Tools
power exchange
Dev
citrix dwhserver
unix applebigdata target olap
File
Dev_OLTP
Integration Servie
oltpserver Dev_OLAP
oltp
Repository
oltp Service
Putty
RepDev
OPB* winftp/wins
cp
Client(de.....)
applebigdata Client(de.....)
Client(de.....)
sIT
Integration Servie
target
Repository
Service
sit_OLTP
sit_OLAP
OPB*
oltp
Integration Servie
target
Repository
Service
OPB*
oltp
dwhserver
applebigdata
RepDev
Client(de.....)
..)
.....)
CreditLimitAmount
Requirements Gathering
Scope
work Assignment
Development Issue Tracker/closer
Prepare Unit test cases
Execute Unit testcases
Peer Review
SIT
(System Integration Testing)
Offshore
TL
SITSAT
(System Acceptance Testing)
(System Integration Testing) Support Testing
(defect/bug)
SATUAT PAT
(User Acceptance Testing)
(System Acceptance Testing)
Production Support
Production Monitoring jobs
Production
verify logs
correct data issues
re -run jobs
communicate to support team about producti
CreditLimitAmount
Delivery Manager
Sr Project Manager/Manager 8>
Team Leads(TL) 5>
Module Lead
Sr ETL Developer ETL dev Trainees
3-6 2-3 0
SE
client
Onsite US
meeting
TL
pport team about production issue
SURR_CUST_ID
CUST_ID
NAME
GENDER
INCOME
1) SDLC
2) Unix
3) Un known Inf
4) Project Overiview
Simple
5 DimensionMedium
Complex
Simple
6 Fact Medium
Complex
ETL Archi
Unix
Inform
AIX
SCO
Solaris
Linux
Rhel
putty
/ (root)
next 3 others
r read
w write
x execute
ls -ltr
t -time based(order by time)
r - reverse order
wildcard characters
* any character match
? Single character match
touch zero.txt
create zero byte file
also changes the file modifications time
vi editor
i insert mode
Esc takes to normal mode
when we are Esz(normal) mode
we can perform file operations like
:w save file
:w filename save filewith filename
:q quit file
:wq save and quit
! force quit or force write by iusing
:w!
:q!
Redirction operator
head
tail c1 a
c2 x
c1 a c1 a
c1 a c2 x
c2 x c2 y
c2 y
c2 y
head -2 filename
display first 2 lines
tail -2 filesname
display bottom 2 lines
H file name
n line number
v negative earch
du
display file sizes of given path
h human readable format
s summarises the output
df
filesystem utilisation
h human readable format
simple scripts
execute shell
sh a.sh
$# no of input params
$0 file name
$1 first param
1)
awk -F'|' '{print $2}' file_pipe.txt
awk -F'|' '{print $2 "|" $1 "|" $9}' file_pipe.txt|grep -e "A|GBR" -e "C|GBR" -e "9|GBR"
A | GBR | G 1901-01-01
C | GBR | G 1901-01-01
9 | GBR | J 1901-01-01
2)
Repository Service is a service process to run the repository using repository database
Integration Service Is a service to run /execute code(workflows/sessions)
Grid Grid group of nodes to utilise more hardware processing
Domain
node1
Repository Service
Inf Client
Cleint
Integraton
node2 Service
Repdatabas
e
src1 tgt1
src2 tgt2
lkp
lkp
0
exp tgt
src
Source Qualifier
FF
1MILL
sitory database
nf Client
Cleint
Filter
in(deptno,10,20)=1
Router
empno ename sal
e1 xyz 10 cond1
cond
Exp
def
Sort
trn3
(passive)
trn1 trn2
trn4
(Active)
default
1m
filt
(deptno=10)
1m SQ
filt
(deptno=20)
1m
filt
(deptno=20)
1m
filt
SQ (deptno=10)
deptno=20
Exp
1 rec rec
EMPNO_Prev_v EMPNO_v 0 e1 e1
EMPNO_Prev_Out EMPNO_Prev_v 0 e1 e1
ENAME ENAME xyz xyz abc
EMPNO EMPNO e1 e1 e2
.
.
EMPNO_v EMPNO e1 e1 e2
Sort
will remove only row duplicates
e1
e1
src exp tgt
1m
e1
Joiner
SQ
Lookup
Joiner Lookup
inner/outer..
condition "="
Master
Detail
Cartesian/cross join
Sorted Input
seq
Lookup
Connectivity
Connected part of pipelines can return miltiple ports
UnConnected not part only one
Cache
No Cache
when lookup table is getting updated while mapping is running
not good from performance as it always cheks data on database
c1 xyz
c1 abc
static Cache
index data
condition column data other columns data
1 c1 xyz n
2 c1 abc y
1
2
c1 abc
Always use inner join of source and target SQL in Lookup Override to reduce the cache in Loo
c1
c1
c9 c2
c1 c3
c4
c5
c6
c7
c8
Dynamic
Cache is updated during mapping execution
New Lookup row port indicates the status of record(new/old/unchanged)
Associated port to compare columns from lookup to source
we can also ignore some columns in comparision
synchronise
along with updating data in cache this will also update data in lookup ta
Lookup as Active
this is available from 9.x
this can be used as join .
when there are maultiple matches it will match will all records
Target
updatestartegy
SP
trn2
trn4
(Active)
dept_t10
dept_t20
dept_t10
dept_t20
e1
tgt
tgt
ng is running
on database
c1 xyz n
c1 abc y
c1 xyz n
c1 abc y
of record(new/old/unchanged)
s from lookup to source
ome columns in comparision
will also update data in lookup table