Chapter 5:

Foundations for a New

Enterprise Application
Development Era

Learning Map
The Future of
Enterprise
Compu'ng

In-Memory
Database
Operators

Advanced
Database
Storage
Tech-
niques

Founda'ons of
Database Storage
Techniques

Founda'ons for a
New Enterprise
Applica'on
Development Era

Implications on Application
Development

Dr.-Ing. Jrgen Mller

Simplified Application
Development
Tradi&onal
Applica&on
cache

Column-oriented

No caches needed

No redundant data/
objects

No maintenance of
indexes or aggregates

Data movements are

minimized

Database
cache

Prebuilt
aggregates

Raw data

111

SAP ERP Financials on

In-Memory Technology
In-memory column database for an enterprise system

Combined workload (parallel OLTP/OLAP queries)

Leverage in-memory capabili&es to

Reduce amount of data

Aggregate on-the-y

Run analy&c-style queries (to replace materialized views)

Execute stored procedures

112

Use Case: SAP ERP Financials solu&on

Post and change documents

Display open items

Run dunning job

Analy&cal queries, such as balance sheet

Current Financials
Solutions

113

The Target
Financials Solution
Only base tables, algorithms, and some indexes

114

In-Memory Financials on
SAP ERP
accounting documents

BKPF

BSEG

secondary indices

BSAD

BSAK

BSAS

BSID

BSIK

BSIS

115

dunning data

sum tables

MHNK

MHND

GLT0

change documents

CDHDR

CDPOS

LFC1

KNC1

In-Memory Financials on
SAP ERP (prototype)
accounting documents

BKPF

BSEG

116

Feasibility of Financials on
In-Memory Technology

Modica&ons on SAP Financials

Removed secondary indices, sum tables and pre-calculated and materialized

tables

Reduce code complexity and simplify locks

Insert Only to enable history (change document replacement)

Added stored procedures with business func&onality

European division of a retailer

ERP 2005 ECC 6.0 EhP3

5.5 TB system database size

Financials:

23 million headers / 8 GB in main memory

252 million items / 50 GB in main memory

(including inverted indices for join a_ributes and insert only extension)

117

Reduction by a Factor 10
DBMS

IMDB

BKPF

8.7 GB

1.5 GB

BSEG

255 GB

50 GB

263.7 GB

51.5 GB

Secondary Indices

255 GB

Sum Tables

0.55 GB

Complete

519.25 GB

51.5 GB

118

Dunning Run

Dunning run determines all open and due invoices

Customer dened queries on 250M records
Current system: 20 min
New logic: 1.5 sec

119

In-memory column store

Parallelized stored procedures
Simplied Financials

Why?
Being able to perform the dunning run in such a short &me
lowers TCO
Add more func&onality!
Run other jobs in the mean&me! - in a mul&-tenancy cloud
setup hardware must be used wisely

120

Bring Application Logic Closer

to the Storage Layer

Select accounts to be dunned, for each:

Create and write dunning item tables

121

Select open account items from BSID, for each:

Calculate due date
Select dunning procedure, level and area
Create MHNK entries

Bring Application Logic Closer

to the Storage Layer

Select accounts to be dunned, for each:

Select open account items from BSID, for each:

Calculate due date
Select dunning procedure, level and area
Create MHNK entries

Create and write dunning item tables

1 SELECT
10,000 SELECTs
10,000 SELECTs
31,000 Entries

122

Bring Application Logic Closer

to the Storage Layer

Select accounts to be dunned, for each:

1 SELECT
One
single
Select open account items from BSID, for each: stored procedure
10,000 SELECTs
Calculate due date
Select dunning procedure, level and area
Create MHNK entries

Create and write dunning item tables

123

10,000 SELECTs
31,000 Entries

Bring Application Logic Closer

to the Storage Layer

Select accounts to be dunned, for each:

1 SELECT
One
single
Select open account items from BSID, for each: stored procedure
10,000 SELECTs
Calculate due date
Select dunning procedure, level and area
Create MHNK entries

10,000 SELECTs

Create and write dunning item tables

31,000 Entries

Calculated on-the-y

124

Results
Original Version needed about 20 minutes
Factor 800x accelera&on achieved
#
Opera'on

1

Select Open Items

Due date, dunning level

0.63s

Variant 2

Variant 3

1.01s
0.6s
(incl. T047 & KNB5 Join)
(incl. T047 & KNB5 Join)

27s

deferred to aggrega&on

0.5s

Filter 1 (Verify Dunning levels)

19s

1.1s

0.5s

Filter 2 (Check Last Dunning)

15s

0.8s

0.4s

Generate MHNK (Aggregate)

done in #1

1.2s

done in #1

Generate MHND (Execute Filters)

done in #1

140ms

done in #1

1 Minute

3.0s

Total

125

HANA2 Version

(#3, #4 exec. in parallel)

1.5s

(#3, #4 exec. in parallel)

Hardware: 4 CPUs x 6 Cores (Intel Dunnington), 256GB RAM

Customer Data: 250mio line items, 380k open, 200k due

Dunning Application

126

Dunning Application

127

Available-to-Promise
Check

Can I get enough quan&&es of a requested product on a

desired delivery date?
Goal: Analyze and validate the poten&al of in-memory and
highly parallel data processing for Available-to-Promise (ATP)
Challenges

Dynamic aggrega&on
Instant rescheduling in minutes vs. nightly batch runs
Real-&me and historical analy&cs

Outcome

Real-&me ATP checks without materialized views

Ad-hoc rescheduling
No materialized aggregates

128

In-Memory Available-to-Promise

129

Demand Planning

Flexible analysis of demand

planning data
Zooming to choose granularity
Filter by certain products or
customers
Browse through &me spans
Combina&on of loca&on-based
geo data with planning data in an
in-memory database
External factors such as the
temperature, or the level of
cloudiness can be overlaid to
incorporate them in planning
decisions

130

GORFID

HANA for Streaming Data Processing

Use Case: In-Memory RFID Data Management
Evalua&on of SAP OER
Prototypical implementa&on of:

RFID Read Event Repository on HANA

Discovery Service on HANA (10 billion
data records with ca. 3 seconds response &me)
Frontends for iPhone, iPad2

Key Findings:

131

HANA is suited for streaming data

(using bulk inserts)
Analy&cs on streaming data is now possible

Near Real-Time as a
Concept

ReadEvent
Event
Read
Read
Event
Repositories
Repositories
Repositories

Verification
Verification
Verification
Services
Services
Services

8,000read
read
upupto
toto8.000
8.000
up
read
eventnotifications
notifications
event
eventper
notifications
second
per second
second
per

Bulk load every

2-3 seconds:
> 50,000 inserts/s

upto
to2.000
2,000
up
up
to 2.000
requests
requests
requests
per
second
per
second
per
second
DiscoveryService
Service
Discovery
Discovery
Service

SAP HANA
HANA
SAP
SAP HANA

PPA

A
A

132

Learning Map
The Future of
Enterprise
Compu'ng

In-Memory
Database
Operators

Advanced
Database
Storage
Tech-
niques

Founda'ons of
Database Storage
Techniques

Founda'ons for a
New Enterprise
Applica'on
Development Era

Views

Dr.-Ing. Jrgen Mller

Dynamic Views
Presentation Layer

Excel

Business
Objects
Explorer

SAP

View

View Layer
(Calculations, Filter, ...)

View

View

View

...

View

View

Node Tables

Persistency Layer

Any Software

View

Other DB

Node Tables

Node Tables

Node Tables

Store

(Main Memory)

DB
Persistence

Restart

Object Hierarchy
Write Complete
Objects

Recovery

135

Logical
Log

Learning Map
The Future of
Enterprise
Compu'ng
Founda'ons of
Database Storage
Techniques

In-Memory
Database
Operators

Founda'ons for a
New Enterprise
Applica'on
Development Era

Advanced
Database
Storage
Tech-
niques

Bypass Solution

Dr.-Ing. Jrgen Mller

Bypass Solution Allows a Smooth

Transition to In-Memory Technology
Millions of old un-op&mized lines of code at the
customers site Transi&on required
Row-store replacement
Part-for-part replacement with bypass
Transform row-store to column-store on the y
Change of applica&on code

140

Bypass Solution for Transition

Todays System

Traditional BI
SAP

ERP

ETL

OLAP
Engine

BIA

Business
Objects

...

Traditional DB
w/ Cubes

Traditional DB

141

Excel

Bypass Solution for Transition

STEP 1: Install and run the in-memory database in parallel

Todays System with IMDB

Traditional BI
SAP

ERP

ETL

OLAP
Engine

BIA

Business
Objects

...

Traditional DB

Excel

w/ Cubes

Traditional DB

IMDB

SSD

Bypass Solution for Transition

STEP 1: Install and run the in-memory database in parallel
STEP 2: Generate business value from the first day on

Todays System with IMDB

Traditional BI
SAP

ERP

ETL

OLAP
Engine

BIA

Business
Objects

...

Traditional DB
w/ Cubes

Traditional DB

IMDB

SSD

New
Applications

Excel

Bypass Solution for Transition

STEP 1: Install and run the in-memory database in parallel
STEP 2: Generate business value from the first day on
STEP 3: Re-create traditional BI w/o cubes in IMDB

Todays System with IMDB

Traditional BI with IMDB

SAP

ERP

EL

OLAP
Engine

BIA

Business
Objects

...

IMDB

Excel

w/o cubes

Traditional DB

IMDB

SSD

New
Applications

Bypass Solution for Transition

STEP 1: Install and run the in-memory database in parallel
STEP 2: Generate business value from the first day on
STEP 3: Re-create traditional BI w/o cubes in IMDB
STEP 4: Introduce next-gen BI running in parallel without materialized views

Todays System with IMDB

Traditional BI with IMDB

SAP

ERP

EL

OLAP
Engine

BIA

Business
Objects

...

IMDB
w/o Cubes

Traditional DB

IMDB
OLAP

Engine

SAP

New
Applications

Business
Objects

...

SSD

Excel

BI 2.0

Excel

Bypass Solution for Transition

STEP 1: Install and run the in-memory database in parallel
STEP 2: Generate business value from the first day on
STEP 3: Re-create traditional BI w/o cubes in IMDB
STEP 4: Introduce next-gen BI running in parallel without materialized views
STEP 5: Eliminate all the traditional BI, virtualize all in-memory BI, using non-materialized views

Todays System with IMDB

ERP

Traditional DB

BI 2.0

IMDB
OLAP

Engine

Business
Objects

New
Applications

...

SSD

SAP

Excel

Bypass Solution for Transition

STEP 1: Install and run the in-memory database in parallel
STEP 2: Generate business value from the first day on
STEP 3: Re-create traditional BI w/o cubes in IMDB
STEP 4: Introduce next-gen BI running in parallel without materialized views
STEP 5: Eliminate all the traditional BI, virtualize all in-memory BI, using non-materialized views
STEP 6: Run ERP and BI on IMDB

Future System with IMDB

ERP Future
Releases
OLTP & OLAP
BI 2.0

IMDB
OLAP

Engine

New
Applications

Business
Objects

...

SSD

SAP

Excel