Applies to:
This article talks about safety limits that we can use in Generic Data Source in SAP BI. For more information, visit the EDW homepage.
Summary
Safety Intervals are used in Generic Data Source to make sure that no data records are missed, even if those are not stored in the DB table when the extraction took place. Author: Gaurav Namdeo and Debjit Singha Company: L & T Infotech. Created on: 13 December 2010
Author Bio
Gaurav Namdeo working in L&T Infotech has an experience 4 years in SAP BI/BO. He has been a part of Development and Implementation Projects.
Debjit Kumar Singha is working in L & T Infotech has an experience of more than 1 year in SAP BI/BO. He has been a part of Development and Implementation Projects.
Table of Contents
Step By Step Explanation: .................................................................................................................................. 3 Scenario: ............................................................................................................................................................. 6 Example 1: ...................................................................................................................................................... 6 Example 2: ...................................................................................................................................................... 7 Example 3: ...................................................................................................................................................... 7 Example 4: ...................................................................................................................................................... 8 Example 5: ...................................................................................................................................................... 8 Example 6: ...................................................................................................................................................... 9 Related Content ................................................................................................................................................ 10 Disclaimer and Liability Notice .......................................................................................................................... 11
2. Specify the delta specific field (in our case we considered AEDAT) and select Calendar Day.
To find the time stamp, click on the respective icon on the Stat tab.
Scenario:
Safety Intervals are used in Generic Data Source to make sure that no data records are missed, even if those are not stored in the DB table when the extraction took place. Time stamps got created during extraction and which is in turn used for delta extraction. Time stamp holds the time (day) of the last data extraction. If in our case, where no safety limits is defined. Suppose Time Stamp holds 11.02.2010 (in mm.dd.yyyy) and the next delta extraction begins on 11.03.2010. In this case after extraction, the pointer is set to 11.03.2010. A document is created on 11:59 PM, 11.02.2010 and not saved until 01:05 AM, 11.03.2010. It is not considered during Initialization (or delta) because it is not saved and it wont be extracted the next time ether (delta).So the safety limit is always picked in such a way that is greater than the maximum time that it takes to create a record (document).
Note: Convention followed
Last Extraction
(Today-3) (Today-2) (Today-1)
Current day
(Today)
Data loaded from last day extraction Data yet to load as of current extraction. Data that is going to be loaded again.
Example 1:
When Upper and lower limit are blank. Upper limit = 0 & Lower Limit = 0.
Last Extraction
Data Loaded in BI (Init) Yet to load
Today
In case of blank there is always a risk of missing out records. Below image shows the Time Stamp (marker) after delta load.
Example 2:
When Upper and lower limit are blank. Upper limit = 0 & Lower Limit = 1.
Last Extraction
Data Loaded in BI (Init) Overlap Yet to load
Today
Delta will load data from previous day as well (data will overlap). Below image shows the Time Stamp (marker) after delta load.
Example 3:
When Upper and lower limit are blank. Upper limit = 1 & Lower Limit = 0.
Last Extraction
Data Loaded in BI (Init) Yet to load
For Init as well as Delta both extract one day before data.
Today
Example 4: When Upper and lower limit are blank. Upper limit = 1 & Lower Limit = 1.
Last Extraction
Data Loaded in BI (Init) (Overlap) Yet to load
Today
Example 5:
When Upper and lower limit are blank. Upper limit = 1 & Lower Limit = 2.
Last Extraction
Data Loaded in BI (Init) (Overlap) Yet to load
Today
Example 6:
When Upper and lower limit are blank. Upper limit = 2 & Lower Limit = 1.
Last Extraction
Data Loaded in BI (Init) (Overlap) Yet to load
Today
Note: It is always preferred to use upper limit and lower limit together (for system running 24/7), where upper limit describes, how old the data is. Otherwise it is suggested to use upper limit of calendar day. When we are using lower limit, some records may get processed twice, so we have to make sure that those data records are feed in DSO (overwrites the repeated records.
Related Content
Delta Load Management Framework Overview Generic Extraction Maintaining Generic DataSources