Transfer of Archived SAP ERP Data to SAP NetWeaver BW

Using PBS archive add ons

15 November 2011

Transfer of Archived SAP ERP Data to SAP NetWeaver BW

2003 - 2011 PBS Software GmbH

Schwanheimer Strasse 144a 64625 Bensheim, Germany Phone: Fax: Email: +49 - 6251 - 174 0 +49 - 6251 - 174 174 info@pbs-software.com

Internet: www.pbs-software.com

PBS archive add ons Bensheim.

is a registered trademark of PBS Software GmbH,

SAP, SAP ERP, R/3, R/2, ABAP/4 are registered trademarks of SAP AG, Walldorf/Baden. Acrobat Reader is a registered trademark of Adobe Systems Incorporated.

Transfer of Archived SAP ERP Data to SAP NetWeaver BW

Table of Contents
1 2 3
3.1 3.2 3.3

Introduction ......................................................................... 4 Standard Extractors from BI Content ................................ 5 Data Transfer through Flat Files ........................................ 9
Extract Archived Data from SAP R/3 and SAP ERP .......................... 10 Transfer to SAP NetWeaver BW......................................................... 13 Import into SAP NetWeaver BW ......................................................... 14

4 5
5.1 5.2 5.3

Creation of a generic DataSource .................................... 21 Module-specific Features ................................................. 24

PBS archive add on CFI ..................................................................... 24 PBS archive add on CCO ................................................................... 27 PBS archive add on CCOPA .............................................................. 28

Transfer of Archived SAP ERP Data to SAP NetWeaver BW

Introduction

Data Warehouse systems enable efficient access to data from heterogeneous information sources, suitable storage and end user-specific representation of information that has been gathered in this way. Data from almost any source can be used for data procurement and provision in the SAP BW system. Main sources are SAP systems that are implemented in the company. The data procurement layer (also called the ETL process layer) is used to extract, transfer and load this data into the BW system. A growing number of customers need to extract archived data from an ERP source system and load it into a BW system. This documentation describes two methods to fulfill this task. In the first method extractors from BI Content are used. Chapter 2 lists all DataSources from SAP BI Content which are able to select archived data, together with DataSources from PBS. The second method, described in chapter 3, is based on data transfer through flat files. It is a generic method which allows the user to transfer any structured data to SAP NetWeaver BW.

Transfer of Archived SAP ERP Data to SAP NetWeaver BW

Standard Extractors from BI Content

The SAP BI Content provides a set of standardized DataSources which are widely used by customers. However, only a small percentage supports the extraction of archived data. In order to allow our customers the extraction of indexed data from archive, many PBS archive add ons now contain a selected number of archiveenabled DataSources and load reports.

Table 1 lists all DataSources from SAP BI Content which are able to select archived data. In addition, all DataSources that are provided by a specific PBS archive add on are added to the list. In general, DataSources use two different methods to select archived data: either direct access to archive in the DataSource itself, or indirect access through selecting archived data via initial load. The second method is especially used in all DataSources from Logistics by using communication structures from the Logistics Information System (LIS). Now, the second method will be described more in detail. In order to extract data from Logistics, all DataSources from BI Content use an indirect access method. First the data is stored in LIS tables as a buffer. Then the extractor reads from these LIS tables. To build initial data loads for SAP NetWeaver BW, first you need to populate the corresponding LIS tables with the related transaction. Hence, for archive access these transactions must be able to select archived data. Table 2 lists all DataSources from Logistics with their corresponding transactions for an initial load. In addition, the transactions and related programs are listed in Table 3. In case of PBS versions, the supported SAP release is added to the program name together with a release date. More module-specific details about using PBS extractors and load programs can be found in chapter 5.

Transfer of Archived SAP ERP Data to SAP NetWeaver BW

Data Source Technical Name Data Source Description

0CO_OM_OPA_6 0CO_OM_CCA_9 0CO_OM_ABC_7 0CO_OM_WBS_6 0CO_OM_NAE_1 0CO_OM_NTW_1 0CO_OM_NWA_1 0CO_OM_WBS_1 0FI_AP_3 0FI_AP_4 0FI_AR_3 0FI_AR_4 0FI_GL_4 0FI_TX_4 0UC_SALES_STATS_01 0UC_SALES_STATS_02 Customer-specific DataSources 2LIS_02_xx 2LIS_03_BF 2LIS_03_UM 2LIS_04_xx 2LIS_05_xx 2LIS_08_xx 2LIS_11_xx 2LIS_12_xx 2LIS_13_xx 2LIS_17_xx 2LIS_18_xx 2LIS_45_xx Table 1:

Orders: Actual Costs with Delta Extraction Cost Centers: Actual Costs with Delta Extraction Processes: Actual Costs with Delta Extraction WBS Elements: Actual Costs with Delta Extraction Network Activity Element: Costs Network Plan: Costs Network Activity: Costs WBS Elements: Costs Vendors: Line Items Vendors: Line Items with Delta Extraction Customers: Line Items Customers: Line Items with Delta Extraction General Ledger Line Items with Delta Extraction Taxes: Line Items with Delta Extraction IS-U Sales Statistics IS-U Sales Statistics Profitability Analysis Purchasing Documents Material movements Revaluations Production Orders Quality Notifications Shipments and Shipment Costs Sales Orders Delivery Notes Billing Documents Plant Maintenance Orders/Notifications Customer Service Orders/Notifications Agency Documents

Archive Access possible in SAP BI PBS archive Content add on CCO (from 4.6C) CCO CCO CCO CPS (Q2/12) CPS (Q2/12) CPS (Q2/12) CPS (Q2/12) CFI CFI CFI CFI CFI (from 4.6C) CFI CISU CISU CCOPA MM CFI CFI CPP CQM CSD SD SD SD CPM CPM AB

DataSources with archive selection from SAP BI Content and PBS archive add ons

Transfer of Archived SAP ERP Data to SAP NetWeaver BW

Logistics DataSources 2LIS_02_xx 2LIS_03_BF 2LIS_03_UM 2LIS_04_xx 2LIS_04_xx 2LIS_05_xx 2LIS_08_xx 2LIS_08_xx 2LIS_11_xx 2LIS_12_xx 2LIS_13_xx 2LIS_17_xx 2LIS_18_xx 2LIS_45_xx Table 2: Initial Load Transaction OLI3BW OLI1BW OLIZBW OLI4BW OLIFBW OLIQBW VTBW VIFBW OLI7BW OLI8BW OLI9BW OLIIBW OLISBW OLIABW Description Archive Support Y N N N N N N N Y Y Y N N Y Logistics PBS Version --Y (CFI) Y (CFI) Y (CPP) Y (CPP) Y (CQM) Y (CSD) Y (CSD) ------Y (CPM) Y (CPM)

MM: Purchase MM: Material movements MM: Invoice Verification PP: Production Orders PP: Rep. Manuf. QM: Notifications/Inspect. SD: Shipment SD: Shipment Costs SD: Orders SD: Deliveries SD: Invoices PM: Maintenance Orders CS: Customer Services AB: Agency Business from

Initial load of extract data communication structures

with

LIS

Transfer of Archived SAP ERP Data to SAP NetWeaver BW

Transaction OLI3BW OLI1BW OLIZBW (no transaction) OLI4BW OLIFBW OLIQBW VTBW VIFBW OLI7BW OLI8BW OLI9BW OLIIBW OLISBW OLIABW Table 3: Description Purchase docs. Material movements Revaluations Material movemts per material*) Production Orders Rep. Manuf. Notifications / Inspec. Shipments Shipment Costs Orders Deliveries Invoices Maintenance orders Customer Services Agency Business Program RMCENEUA RMCBNEUA RMCBNERP RMCBMMAT RMCFNEUA RMCFNEUD RMCQNEBW VTRBWVTBWNEW VTRBWVIFBW RMCVNEUA RMCVNEUL RMCVNEUF RMCINEBW RMCSNEBW RMCENEUB PBS version

---/PBS/RMCBNEUA (CFI) /PBS/RMCBNERP (CFI) /PBS/RMCBMMAT (CFI) from release 4.7 /PBS/RMCFNEUA (CPP) (on demand) /PBS/RMCFNEUD (CPP) (on demand) /PBS/RMCQNEBW (on demand) (CQM) /PBS/VTRBWVTBWNEW (on demand) (CSD) /PBS/VTRBWVIFBW (on demand) (CSD) ---------/PBS/RMCINEBW (CPM) /PBS/RMCSNEBW (on demand) (CPM) ----

Transactions and program names for initial load

*) This function is only used for making corrections to initial stocks and should only be used in exceptional cases. For further details, see SAP Notes 79083 and 111691.

Transfer of Archived SAP ERP Data to SAP NetWeaver BW

Data Transfer through Flat Files

The following chapter describes a generic method which is based on data transfer through flat files. These flat files contain data in AIS (Audit Information System) format. Since DART (Data Access Retention Tool) extractions use the same format, they can also be used as a data source. The transfer of archived data is performed in three steps. In the first step the archived R/3 or ERP data is extracted in text format and written into flat files. Then all these files are transferred to the destination, whereas in the third step the data in the flat files is imported into the SAP BW system. Up to now, extraction reports are available for the PBS archive add ons CAB, CCO, CFI, CFICA, CISU, CMM, CPCA, CPP (Q2/2012), CSD, CSL and CWM. Note: Please keep in mind that the method described in this chapter is only a workaround solution. The BI Content extractors use extended business rules for extraction which cannot be realized by this method. As long as the respective extractor from BI Content is not able to read archived data (see chapter 2), extraction before archiving should be the preferred way. As an alternative, SAP describes a method which is based on the information structures of the Archiving Information System (SAP AS). For detailed information refer to http://www.sdn.sap.com/irj/sdn/howtoguides SAP NetWeaver 2004 Business Intelligence How to Extract Archived R/3 Data via the Archive Information System.

In the following pages the extraction of purchasing document data (archiving object MM_EKKO) will be used as an example. Direct access to the archived data is provided by the PBS archive add on CMM. The method described can be transferred to any other archiving object which is supported by a PBS archive add on.

Transfer of Archived SAP ERP Data to SAP NetWeaver BW 3.1 Extract Archived Data from SAP R/3 and SAP ERP
Call CMM extract report /PBS/CMM_LIS_ITEM_EKKO (via SE38). From ECC 5.0 or higher, you can also call transaction /PBS/CMME_EXPORT. In order to extract only archived data, use 3 for the data source (Archive only). Use Item list as list layout, and use Export to file as output option. Specify file name and destination for the extract file.

10

Diagram 1:

CMM transaction /PBS/CMME_EXPORT

Before you start the extraction it is recommended to view some records as an example by using the Display/Print option. You will get a list as shown in the following diagram. The layout is defined by the display variant which you have chosen in the selection screen. By changing the display variant or creating a new one it is possible to change the layout structure of the list. Starting from the list the screen menu provides the function to change or define display variants.

Transfer of Archived SAP ERP Data to SAP NetWeaver BW

11

Diagram 2:

List of purchase orders selected by the CMM extract report

Use Settings Layout Change to change the layout, i.e. the containing fields and the desired order. Save the layout structure as a new display variant, which you can choose in the selection screen of the extract report.

Transfer of Archived SAP ERP Data to SAP NetWeaver BW

12

Diagram 3:

Change of the list layout

Transfer of Archived SAP ERP Data to SAP NetWeaver BW 3.2 Transfer to SAP NetWeaver BW
After extraction you will get an extract file in the following format. The first 8 lines describe the field structure (technical name, long/short description, type, length and decimal places). After this the extracted data is listed whereas each row is separated by tabs. Each line is terminated by CR LF.

13

Diagram 4:

Extracted purchase documents in SAP AIS format

In the next step the data has to be transferred from the source system to the BW system or at least to the client workstation (front end). The FTP transfer should be done in two steps. First, use binary mode from source to front end. Then use text mode from front end to destination to ensure that all lines are correctly terminated (CR LF in Windows servers, LF in UNIX servers).

Transfer of Archived SAP ERP Data to SAP NetWeaver BW 3.3 Import into SAP NetWeaver BW
Importing data from flat files requires the definition of a flat file source system in SAP NetWeaver BW. Make sure you have defined such a source system. If not, please proceed as follows. From the Data Warehousing Workbench, click the Source Systems tabstrip. Select the appearing Source Systems node, right-click, and choose Create. From the generated list, select File System as shown in Diagram 5. Now a DataSource has to be created. In BW release 7.x two types of DataSources are available: NetWeaver 7 DataSources and 3.x DataSources. In the following we will refer to the new type of DataSources. From the source systems tabstrip in the DW Workbench, double-click on the generated source system. Select or create an application component, right-click, and select Create DataSource. From the appearing window, choose the data type of the DataSource: transaction data, master data text or master data attributes. In this example we are using the transaction data type. After that, you have to set the options as shown in Diagram 6. The most important options are as follows: Data Format: Separated with separator (for example, CSV) Data separator: 009 Hex Number of header rows: 08 After setting the options the DataSource must be activated. It is recommended to check the parameters with the Preview function in the DataSource definition. Possible problems concerning the data layout (e.g. incompatible format) can be detected that way in an early stage.

14

Transfer of Archived SAP ERP Data to SAP NetWeaver BW

15

Diagram 5:

Definition of a source system with system type File

In the next step you have to connect the DataSource to a destination by defining a transformation. In SAP NetWeaver BW the following BW object types can be used as destination: InfoSources, InfoCubes and DataStore Objects. In this example we are using an InfoSource as a destination. If there is no adequate InfoSource it has to be created. From the Data Warehousing Workbench, click the InfoSources tabstrip. Select an InfoArea, right-click, and choose Create InfoSource. In the appearing window an InfoSource structure with appropriate InfoObjects has to be constructed, corresponding to the flat file field definitions (see Diagram 7). If necessary, the related InfoObjects must be created as well. After defining DataSource and InfoSource the transfer rules have to be set. These rules assign the InfoObjects of the InfoSource to the corresponding fields in the DataSource. In SAP NetWeaver BW a so-called transformation is used for that purpose.

Transfer of Archived SAP ERP Data to SAP NetWeaver BW

16

Diagram 6:

Definition of a DataSource in the File source system

Diagram 7:

Definition of an InfoSource

Diagram 8 shows the definition of a transformation between DataSource and InfoSource. Unlike the modeling of 3.x transfer rules, the mapping is not done by combining the corresponding fields in a list, but uses a graphical design method with arrows. After completing the mappings, the transformation must be activated as well.

Transfer of Archived SAP ERP Data to SAP NetWeaver BW

17

Diagram 8:

Definition of transformation rules between DataSource and InfoSource

Transfer of Archived SAP ERP Data to SAP NetWeaver BW

18

Diagram 9:

Complete data flow from the DataSource through the InfoSource to an InfoProvider

After connecting the DataSource to an InfoSource, the InfoSource will be assigned to its data target. In this example we are using an InfoCube to serve that purpose. Diagram 9 shows the complete data flow from DataSource to InfoCube. When using a NetWeaver 7 DataSource, the load process must be done in two steps: from source system to DataSource, and from DataSource to InfoCube. In the first step the data is extracted from the source system and stored in the PSA table which is assigned to the DataSource. After creating an InfoPackage to the DataSource the extraction can be scheduled (see Diagram 10). After scheduling the data loads, you can monitor the progress of each data load job request. Often differences between the flat file data structure and the DataSource structure cause a failure of the load job. Make sure that the file definitions map correctly with the metadata defined in SAP NetWeaver BW. The second step consists of creating a Data Transfer Process (see Diagram 11) and loading the data target with it (see Diagram 12). If the transformations are not properly designed, it will result in a failure of the load process. If the process succeeds without errors, the data import has been successfully completed.

Transfer of Archived SAP ERP Data to SAP NetWeaver BW

19

Diagram 10:

Scheduling the data load in the InfoPackage definition

Diagram 11:

Creation of a Data Transfer Process DataSource to InfoProvider/InfoObject

(DTP)

from

Transfer of Archived SAP ERP Data to SAP NetWeaver BW

20

Diagram 12:

Scheduling the Data Transfer Process from DataSource to InfoProvider/InfoObject

Transfer of Archived SAP ERP Data to SAP NetWeaver BW

21

Creation of a generic DataSource

When defining the DataSource transfer structure in the Administrator Workbench (see chapter 3), an existing DataSource can be very helpful. For that reason, the extract report creates an additional table in the Data Dictionary. This table is overwritten by the report during each extraction. Therefore, it always contains the current extract structure. The structures use the name /PBS/xxx_EXTRACT where xxx represents the respective PBS archive add on. By defining a generic DataSource in the ERP system and moving it to the BW system, the transfer structure in this DataSource can be used as a template for the flat file DataSource in chapter 3. The procedure is as follows: In the source system, call transaction RSO2. Choose a name for the new transaction data DataSource, and click the Create button. Specify application component, description and extraction table. In our example, we are using /PBS/CMM_EXTRACT.

Diagram 13:

Creating a generic DataSource in transaction RSO2

Transfer of Archived SAP ERP Data to SAP NetWeaver BW

When saving your DataSource, you will be asked to specify which fields are used for selection or should be hidden (see diagram below). In order to complete the DataSource definition, you have to press the Save button again.

22

Diagram 14:

Specifying the fields for data selection

Replicate the DataSource in the BW system as shown below. From the Administrators Workbench, use the Source Systems tabstrip, select the source system which corresponds to your ERP system, and select the application component which you have specified above. Right-click and select Replicate DataSources. After that the DataSource structure should be available in your BW system.

Transfer of Archived SAP ERP Data to SAP NetWeaver BW

23

Diagram 15:

Replicating the defined DataSource via transaction RSA1

Transfer of Archived SAP ERP Data to SAP NetWeaver BW

24

Module-specific Features

5.1 PBS archive add on CFI

The support of selected DataSources for archive access is available for SAP Release 4.6C and above. For SAP Release 4.6C the extractors and programs are delivered in a separate transport.

Activating PBS functionality Archive data access is activated by exchanging the function module definition in table ROOSOURCE. To do this, PBS provides Customizing program /PBS/CFI_SETUP_BW which can be started via transaction SA38 or SE38. Each DataSource can be activated individually for PBS archive access. To do this, you merely select the corresponding checkbox and start the program. For productive use, deactivate the button Test Run. It is possible to reset to the SAP standard settings at any time. The PBS BW extractors for DataSources 0FI_AR_4, 0FI_AP_4, 0FI_GL_4 and 0FI_TX_4 only support the update modes F (full) and C (initialization). Recommendation: 1. Activation of PBS BW extractors 2. Execution of extraction in the modes F or C 3. Resetting of BW extractors to the SAP standard 4. Execution of delta extraction before the SAP data archiving The BW DataSources extracts which you have changed to PBS extractors need to be replicated again in the BW system.

Transfer of Archived SAP ERP Data to SAP NetWeaver BW

25

Diagram 16:

Activation/Deactivation of PBS versions for BW extraction

Transfer of Archived SAP ERP Data to SAP NetWeaver BW

Customer-Specific Enhancements of Extractors Many DataSources provided by SAP within Business Content can be used for data procurement from SAP source systems. These extract structures, however, often need to be changed, because all customers have their own requirements which data an extractor should provide. SAP supports this customer requirement in the form of append structures. The individual fields of the append structures that have been appended to an extract structure can be filled with data via function enhancements (Customer Exits). You can access the function enhancements via transaction code CMOD. First, a project must be created to which SAP enhancement RSAP0001 has been added. These Customer Exits now contain function modules that must be programmed via ABAP: EXIT_SAPLRSAP_001 for the enhancement of transaction data EXIT_SAPLRSAP_002 for the enhancement of attributes and texts EXIT_SAPLRSAP_004 for the enhancement of hierarchies. If a function enhancement has been performed in the function module for transaction data EXIT_SAPLRSAP_001, this needs to be checked in order to determine whether it is necessary to adjust it for PBS archive access. This is not done automatically via the activation program described above!

26

Support for the adjustment can be provided by the PBS Conversion Tool (see the separate manual in the software delivery or in the PBS menu under 'Documentation Guide') or for specific adjustments, the PBS Archive Access Tutorial via transaction /PBS/UTIL_ACCESS_ARC.

Transfer of Archived SAP ERP Data to SAP NetWeaver BW 5.2 PBS archive add on CCO

27

The support of selected DataSources for archive access is available in the PBS archive add on CCO from SAP Release 4.6C.

Activating PBS functionality Archive data access is activated by exchanging the function module definition in table ROOSOURCE. For this purpose, PBS provides the transaction /PBS/CCO_SETUP_BW.

Diagram 17:

Activation/Deactivation of PBS versions for BW extraction

Transfer of Archived SAP ERP Data to SAP NetWeaver BW 5.3 PBS archive add on CCOPA
The support of selected DataSources for archive access is available in the PBS archive add on CCOPA from SAP Release 4.6C.

28

Activating PBS functionality Archive data access is activated by exchanging the function module definition in table ROOSOURCE. For this purpose PBS provides the transaction /PBS/CCOPA_SETUP_BW.