Disclaimer
The information in this presentation is confidential and proprietary to SAP and may not be disclosed without the permission of SAP. This presentation is not subject to your license agreement or any other service or subscription agreement with SAP. SAP has no obligation to pursue any course of business outlined in this document or any related presentation, or to develop or release any functionality mentioned therein. This document, or any related presentation and SAP's strategy and possible future developments, products and or platforms directions and functionality are all subject to change and may be changed by SAP at any time for any reason without notice. The information on this document is not a commitment, promise or legal obligation to deliver any material, code or functionality. This document is provided without a warranty of any kind, either express or implied, including but not limited to, the implied warranties of merchantability, fitness for a particular purpose, or non-infringement. This document is for informational purposes and may not be incorporated into a contract. SAP assumes no responsibility for errors or omissions in this document, except if such damages were caused by SAP intentionally or grossly negligent. All forward-looking statements are subject to various risks and uncertainties that could cause actual results to differ materially from expectations. Readers are cautioned not to place undue reliance on these forward-looking statements, which speak only as of their dates, and they should not be relied upon in making purchasing decisions.
Agenda
1. Overview
What is an Incremental Load? b) Identifying Changed Records c) Types of Incremental Load in Rapid Marts
a)
2. Financial Documents (Several Methods) 3. Standard & Special Ledgers (Partitioned Tables) 4. Material Documents (Date of Last Change) 5. Purchase Documents (Change History)
1a. Overview: What is an Incremental (or Delta) Load? The terms Incremental or Delta Load are interchangeable They are designed to minimize load time for Rapid Marts They load only records that have changed in the SAP source system since last update of the Rapid Mart
This is referred to as the Delta between source (SAP) and target (Rapid Mart) systems
They have a Prerequisite: They need a way of identifying changed records in the source (SAP) system
1b. Overview: Identifying Changed Records Rapid Marts use 2 basic ways of identifying changed records in the SAP source system:
Date of Last Change a date column in the source table Change History Entries in SAP table CDHDR
However:
There are many variations on how these 2 methods are implemented in the SAP source system Many SAP source tables use no method of identifying a changed record at all
Purchasing Documents
Purchasing Rapid Mart Change History Method
Agenda
3. Standard & Special Ledgers (Partitioned Tables) 4. Material Documents (Date of Last Change) 5. Purchase Documents (Change History)
FY 2008 - 2010
Note: in First Run, $G_SDATE and $G_EDATE typically load many years of data
2. On Incremental Loads, this value is retrieved and stored in Global Variable $G_ORIG_SDATE $G_ORIG_SDATE = 01-Jan-2000
4. This is stored in Global Variable $G_ABSOLUTE_START_DATE Example: $G_ABSOLUTE_START_DATE = 04-May-2005 Note: BKPF.CPUDT (Entry Date) is stored as FINANCIAL_DOCUMENT_FACT.ENTRY_DATE in the Rapid Mart
This ensures records that were open but have closed since last load can be reloaded as closed items
Load Closed
Load Open
Note: BSEG.AUGDT is the Clearing Date. It is 0000000 for Open Records and set to the clearing (or closing) date for Closed Records.
Partition by Fiscal Year Drop & Re-add Partitions is much faster than deleting range of Fiscal Years
Example: As computed, $G_ABSOLUTE_START_DATE = 14-Jan-2003. This is because you have some very old bad debts that cannot be collected. These are open financial documents. Without these bad debts, $G_ABSOLUTE_START_DATE would be set to 22-Feb-2009. Override the setting to 22-Feb-2009 in your job initialization script in Data Services. The Data Services job will use your override instead of the value in the Financial Document Fact table.
Data Services
BSAK
Data Services
Delete Closed Load Closed: where BSAK.CPUDT >= $G_ABSOLUTE_START_DATE Where CLEARED_FLAG = C and ENTRY_DATE >= $G_ABSOLUTE_START_DATE Delete Open Where CLEARED_FLAG = O
SAP AG 2009. All rights reserved. / Page 21
Data Services
Data Services
BSEG
Data Services
BSAD
Data Services
Delete Closed Load Closed: where BSAD.CPUDT >= $G_ABSOLUTE_START_DATE Where CLEARED_FLAG = C and ENTRY_DATE >= $G_ABSOLUTE_START_DATE Delete Open Where CLEARED_FLAG = O
SAP AG 2009. All rights reserved. / Page 23
Disadvantages
Substantial cost for 3rd party software Need to write database triggers to capture date of last change for SAP base table (BKPF, BSEG, BSID, BSAD, etc) on mirror DB Extensive modification of Rapid Marts to use trigger derived data
Disadvantages
Substantial cost for 3rd party software Extensive modification of Rapid Marts to use history derived data
Agenda
1. Overview 2. Financial Documents (Several Methods) 3. Standard & Special Ledgers (Partitioned Tables)
Overview b) Table Partition Example c) Additional Suggestions
a)
This presentation will focus on Partitioned Tables used in Classic GL to accommodate the Incremental Load for these tables Partitioned tables technique is used in:
Cost Center Rapid Mart General Ledger Rapid Mart Plant Maintenance Rapid Mart Project Systems Rapid Mart
The 4th DBMS (Teradata) optimizes the SQL Delete statement so no partitions are needed.
FISC YEAR 2005 FISC YEAR 2006 FISC YEAR 2007 FISC YEAR 2008
SAP AG 2009. All rights reserved. / Page 31
FISC YEAR 2005 FISC YEAR 2006 FISC YEAR 2007 FISC YEAR 2008
The incremental load starts by dropping or truncating the partition for the year specified
GLPCT FY 2007
PROFIT CENTER SUMMARY FACT VR
EC-PCA Ledger
FISC YEAR 2005
Data Services
FISC YEAR 2006 FISC YEAR 2007 (Reloaded) (Empty) FISC YEAR 2008
Agenda
1. Overview 2. Financial Documents (Several Methods) 3. Standard & Special Ledgers (Partitioned Tables) 4. Material Documents (Date of Last Change)
Overview b) Date of Last Change Example
a)
4. Material Documents
a. b.
Both are compared to global variables $G_SDATE and $G_EDATE in the Data Services job
Selection Logic: (MKPF.CPUDT >= $G_SDATE AND MKPF.CPUDT <= $G_EDATE) OR (MKPF.AEDAT >= $G_SDATE AND MKPF.AEDAT <= $G_EDATE) MKPF MATERIAL MVMT FACT
Data Services
MSEG
Agenda
1. Overview 2. Financial Documents (Several Methods) 3. Standard & Special Ledgers (Partitioned Tables) 4. Material Documents (Date of Last Change) 5. Purchase Documents (Change History)
Overview b) Change History Example c) Additional Information
a)
To identify new documents, the item change date on the Purchase Document Line Item is used:
EKPO.AEDAT
Note: Although called change date in the SAP Data Dictionary, EKPO.AEDAT is actually set to the current (system) date when a new Purchase Document is created. Thus it serves as the creation date.
The Rapid Mart uses a text file (flat file) on the SAP server to store the list of changed or new Purchase Documents The text file contains only a list of Purchase Document IDs A 3rd pass reads the text file as input, and collects the remaining fields needed to load Purchase Documents to the Rapid Mart target table.
PurchDoc.dat EKPO
Data Services
(Delete)
Any existing data (from previous loads) is deleted before new data is added.
PurchDoc.dat CDHDR
Data Services
(Append)
Purch. Doc. IDs Appended to results of 1st Data Flow
Where: CDHDR.OBJECTCLAS = 'EINKBELEG' and CDHDR.UDATE >= $G_SDATE and CDHDR.UDATE <= $G_EDATE
Data Services
PurchDoc.dat (Input)
Table CDHDR can be used to identify changes to other documents by using different values for OBJECTCLAS column:
EINKBELEG Purchase Document VERKBELEG Sales Document (used in Sales Rapid Mart) MELDUNG Notifications (Plant Maintenance) MATERIAL Material Master LIEFERUNG Vendor Master