DATA WAREHOUSING
SUSHIL
KULKARNI
I DATABASE VS DATA WAREHOUSE
PROBEMS OF DATABASES
N WHAT IS DATA WAREHOUSE?
T DATA ACCESS SYSTEMS AND DIFFERENCES
DATA WAREHOUSE SCOPE
E
TYPES OF DATA WAREHOUSES
N BIG PICTURE
END USERS
S
DATA MARTS
I DATA IN DATA WAREHOUSE
DATABASE STRUCTURE
O
OLTP AND OLAP
N ARCHITECTURE
ETL PROCESS
S SUSHIL KULKARNI
DATABASE
VS
DATA WAREHOUSE
SUSHIL KULKARNI
DATABASE
VS
DATA WAREHOUSE
To accelerate decision making:
1. Right information,
2. Right time,
3.Easily accessible
Problems with database
1. Fragments
2. Operational / Information
Processing
SUSHIL KULKARNI
PROBLEMS !
SUSHIL KULKARNI
PROBLEMS
SUSHIL KULKARNI
PROBLEMS !
IT business requires:
1. integrated
2. company-wide view of high quality
3. Fixed network with changing users
SUSHIL KULKARNI
DEFINITION
OF
DATA WAREHOUSE
SUSHIL KULKARNI
SO WHAT IS
DATA WAREHOUSE ?
Subject-oriented:
customers, patients, students,
products, time.
Time - variant:
Non - updatable:
SUSHIL KULKARNI
DATA SYSTEMS
SUSHIL KULKARNI
DATA SYSTEMS AVAILABLE
SUSHIL KULKARNI
OPERATIONAL SYSTEMS
Used to run a business in real time based
on current data and process large volumes
of relatively simple read/write transactions,
while providing fast response.
Examples
1. Historical data
2. Prediction data.
Examples:
SUSHIL KULKARNI
DATA WAREHOUSE
SCOPE
SUSHIL KULKARNI
DATA WAREHOUSE SCOPE
Broad :
Narrow:
SUSHIL KULKARNI
TYPES OF DATA WAREHOUSE
Point – Point
SUSHIL KULKARNI
TYPES OF DATA WAREHOUSE
Central Data Warehouses
SUSHIL KULKARNI
TYPES OF DATA WAREHOUSE
1. EIS : Executive Information System
2. DSS: Decision Support System
3. Reporting
SUSHIL KULKARNI
BIG PICTURE
SUSHIL KULKARNI
BIG PICTURE
SUSHIL KULKARNI
END USERS
SUSHIL KULKARNI
END USERS
SUSHIL KULKARNI
DATA MART
SUSHIL KULKARNI
DATA MARTS
SUSHIL KULKARNI
DATA MARTS
Create many DM’s
Limited scope
Independent ETL process or derived from
DW
Examples:
1. Financial DM
2. Marketing DM
3. Supply chain DM SUSHIL KULKARNI
D.M. PICTURE
SUSHIL KULKARNI
DATA
IN
DATA WAREHOUSE
SUSHIL KULKARNI
DATA IN
DATA WARE HOUSE
one version of the truth across the
enterprise with meaning full recordes
SUSHIL KULKARNI
DATA IN
DATA WARE HOUSE
SUSHIL KULKARNI
DATA IN
DATA WARE HOUSE
SUSHIL KULKARNI
DATA IN
DATA WARE HOUSE
SUSHIL KULKARNI
DATA BASE
STRUCTURE
SUSHIL KULKARNI
DATA BASE STRUCTURE
SUSHIL KULKARNI
OLTP
AND
OLAP
SUSHIL KULKARNI
OLTP
On line transaction processing
Delete
SUSHIL KULKARNI
OLAP
On line analytical processing , Star
Schema [See Table]
Read Only
Historical data
Aggregated data
SUSHIL KULKARNI
ARCHETECTURE
SUSHIL KULKARNI
ARCHITECTURE
AND
END-TO - PROCESS
SUSHIL KULKARNI
BACK END TOOLS
AND
UTILITIES
Tools are used to extract & loading data
SUSHIL KULKARNI
PROCESS OF BRINGING DATA
TO DATA WAREHOUSE
ETL PROCESS
SUSHIL KULKARNI
CLEANING
Large volumes of data from multiple
sources are involved
SUSHIL KULKARNI
CLEANING
Examples where data cleaning becomes
necessary are:
1. Data Migration
2. Data scrubbing
3. Data Auditing tools SUSHIL KULKARNI
DATA MIGRATION
Data migration tools allow simple
transformation rules to be specified
SUSHIL KULKARNI
DATA SCRUBBING
Data scrubbing tools use domain-specific
knowledge
SUSHIL KULKARNI
LOADING
Additional preprocessing required:
1.Checking integrity constraints
2. Sorting; summarization, aggregation
3.Other computation to build the derived tables
stored in the warehouse
SUSHIL KULKARNI
REFRESH
SUSHIL KULKARNI
SUMMARIZATION
SUSHIL KULKARNI
METADATA
Administrative metadata
SUSHIL KULKARNI
The ETL Process
Capture
Transform
Record-level: Field-level:
Selection – data partitioning single-field – from one field to
Joining – data combining one field
Aggregation – data multi-field – from many fields to
summarization one, or one field to many
Steps in data reconciliation
SUSHIL KULKARNI