Data Warehouse/Data Mart

Components Concepts Characteristics

Overview
Operational vs Informational Systems Data Warehouse components Data Marts

Basic Data Warehouse Architecture

Source OLTP Systems

One Version of the Truth

Subset Data Marts

Enterprise Data Warehouse

Copyright 1997, Enterprise Group, Ltd.

Operational vs. Informational Systems

Order Entry Manf.

Operational Systems

Information Access Today

Operational vs. Informational Systems

Operational Systems

Informational Systems

Information Access Today

Operational vs. Informational Systems

Most of the advances in end-user programming have run into difficulty in actually accessing data that exists in backbone, operational data bases.
Operational data bases have a very, very long life. Large operational systems are converted from one technology to a more advanced one very infrequently (typically every eight to twenty years). Therefore, why not create specific DBs whose role was to make large scale end user access easy to isolate the operational DBs, i.e. a Data Warehouse

Operational vs. Informational Systems

Operational Systems Information Delivery System Informational Systems

Operational vs. Informational Systems

Operational Systems Data Information Warehouse Delivery System Informational Systems

Operational vs. Informational Systems

Operational Systems Data Information Warehouse Delivery System Informational Systems

Operational vs. Informational Systems

Operational Systems Data Information Warehouse Delivery System Informational Systems

Operational vs. Informational Systems

Notice that one of the big impacts of Operational Data Warehousing is to eliminate large Systems numbers of existing DSS systems! Data Information Y2000 will make this essential!!!
Warehouse Delivery System Informational Systems

Operational vs. Informational Systems

Operational Systems Data Information Warehouse Delivery System Informational Systems

Data Marts

Data Marts vs Data Warehouses

Internet/Intranet Layer 11

direct queries virtual queries ad hoc queries

Virtual DW

Coarse DW Operational Data Layer 2a

Central DW

North Americ a
Uni ed St at es t
$11, 000

Distributed DW Core DW Layer 3

Sa l s e
Un i e d t
b y

External Data Layer

2b

St a t e s
Sa l s e

$ 1 0 , 3 4 0 , 3 5 0 ) t$ o 0 1 ( 1 $ 8 , 7 t$ o0 0 , 3 4 0 ) 31 ( 2 $ 4 , 3 t 2 o0 , 7 3 ( 0 2 ) $ 8 $ 1 , 1 t 0 o0 , 3 2 ( 0 1 ) $ 4 $ 7 3 t0 $ 1 , 1 0 ( 0 3 ) o

Presentation/ Desktop Access Layer 1

Data Mart Layer 4

Data Feed/ Data Mining/ Indexing Layer 6 Meta-data Repository Layer 8 Warehouse Management Layer 9

Data Staging and Quality Layer

Data Access Layer 7

Non-operational Data Layer 2c

Application Messaging (Transport) Layer 10

Central Data Warehouse

Internet/Intranet Layer 11

direct queries virtual queries ad hoc queries

Tracking DB

Central DW

Lawson DB Operational Data Layer 2a

North Americ a
Uni ed St at es t
$11, 000

Core DW Layer 3
Sa l s e
Un i e d t
b y

External Data Layer

2b

St a t e s
Sa l s e

$ 1 0 , 3 4 0 , 3 5 0 ) t$ o 0 1 ( 1 $ 8 , 7 t$ o0 0 , 3 4 0 ) 31 ( 2 $ 4 , 3 t 2 o0 , 7 3 ( 0 2 ) $ 8 $ 1 , 1 t 0 o0 , 3 2 ( 0 1 ) $ 4 $ 7 3 t0 $ 1 , 1 0 ( 0 3 ) o

Presentation/ Desktop Access Layer 1

Data Mart Layer 4

Data Feed/ Data Mining/ Indexing Layer 6 Meta-data Repository Layer 8 Warehouse Management Layer 9

Data Staging and Quality Layer

Data Access Layer 7

Non-operational Data Layer 2c

Application Messaging (Transport) Layer 10

Virtual Date Warehouse

A Virtual Data Warehouse approach is often chosen when there are infrequent demands for data and management wants to determine if/how users will use operational data. One of the weaknesses of a Virtual Data Warehouse approach is that user queries a made against operational DBs. One way to minimize this problem is to build a Query Monitor to check the performance characteristics of a query before executing it.

 A Coarse Data Warehouse is often chosen when the organization has a relatively clean/new operational system and management wants to make the operational data more easily available for just that system. A Central Data Warehouse is often chosen when the organization has a clear understanding about it Information Access needs and wants to provide quality, integrated , information to its knowledge workers A Distributed Data Warehouse is similar in most respects to a Central Data Warehouse, except that the data is distributed to separate mini-Data Warehouses (Data Marts )on local or specialized servers

Central Data Warehouse

Internet/Intranet Layer 11

direct queries virtual queries ad hoc queries

Virtual DW

Coarse DW Operational Data Layer 2a

Central DW

North Americ a
Uni ed St at es t
$11, 000

Distributed DW Core DW Layer 3

Sa l s e
Un i e d t
b y

External Data Layer

2b

St a t e s
Sa l s e

$ 1 0 , 3 4 0 , 3 5 0 ) t$ o 0 1 ( 1 $ 8 , 7 t$ o0 0 , 3 4 0 ) 31 ( 2 $ 4 , 3 t 2 o0 , 7 3 ( 0 2 ) $ 8 $ 1 , 1 t 0 o0 , 3 2 ( 0 1 ) $ 4 $ 7 3 t0 $ 1 , 1 0 ( 0 3 ) o

Presentation/ Desktop Access Layer 1

Data Mart Layer 4

Data Feed/ Data Mining/ Indexing Layer 6 Meta-data Repository Layer 8 Warehouse Management Layer 9

Data Staging and Quality Layer

Data Access Layer 7

Non-operational Data Layer 2c

Application Messaging (Transport) Layer 10

Data Marts Only

Internet/Intranet Layer 11

direct queries virtual queries ad hoc queries

Virtual DW

Coarse DW Operational Data Layer 2a

Central DW

North Americ a
Uni ed St at es t
$11, 000

Distributed DW Core DW Layer 3

Sa l s e
Un i e d t
b y

External Data Layer

2b

St a t e s
Sa l s e

$ 1 0 , 3 4 0 , 3 5 0 ) t$ o 0 1 ( 1 $ 8 , 7 t$ o0 0 , 3 4 0 ) 31 ( 2 $ 4 , 3 t 2 o0 , 7 3 ( 0 2 ) $ 8 $ 1 , 1 t 0 o0 , 3 2 ( 0 1 ) $ 4 $ 7 3 t0 $ 1 , 1 0 ( 0 3 ) o

Presentation/ Desktop Access Layer 1

Data Mart Layer 4

Data Feed/ Data Mining/ Indexing Layer 6 Meta-data Repository Layer 8 Warehouse Management Layer 9

Data Staging and Quality Layer

Data Access Layer 7

Non-operational Data Layer 2c

Application Messaging (Transport) Layer 10

Heterogeneity - The Reality

i2 Supply Chain Oracle Financials Siebel CRM 3rd Party Data

Packaged I2 Supply Chain Non- Architected Data Mart

Packaged Oracle Financial Data Warehouse Subset Data Marts

Custom Marketing Data Warehouse

Federated BI Architecture
i2 Supply Chain Oracle Financials Siebel CRM 3rd Party e-commerce

Common Staging Area

Real Time ODS

Federated Financial Data Warehouse Federated Packaged I2 Supply Chain Data Marts

Federated Marketing Data Warehouse

Real Time Data Mining and Analytics Real Time Segmentation, Classification, Qualification, Offerings, etc.

Subset Data Marts Analytical Applications

Benefits of Data Warehouse Architecture

Provides organizing framework Gives flexibility for changes and allows simplified maintenance Speeds up future development by aiding understanding of dw Communication tool for roles and requirements Coordinate data marts

Primary Technical Challenge Axis

Slow
Dirty Data Large Co. Parallel Near ERP DW Real Custom VLDB Time ERP DW Finance Multi-Source

Monthly Freq Turnkey ERP DW Mid-Size Co. Small DB Marketing Single Source Clean Data

Fast
Easy

Hard

Prerequisites for Success

Pain driven Sponsorship at the highest levels Sustainable political will Iterative methodology Manageable scope User driven design Service business mindset Sustainability