Anda di halaman 1dari 12

www.Fullinterview.

com

www.Fullinterview.com

Page<- 1 - >

www.Fullinterview.com

TABLE OF CONTENT

1.Introduction...................................................................................................................................3 2.Purpose..........................................................................................................................................3 3.Approach Points............................................................................................................................3 4.Conclusion ...................................................................................................................................3 5.Procedures and Processes.............................................................................................................4 5.1Converting MV into Table..........................................................................................................4 5.2Creating Datastage jobs to populate the MV..............................................................................5 6.Sample code..................................................................................................................................7 6.1MV Creation Script: ..................................................................................................................7 6.2Query Used in Job: ....................................................................................................................7 7. Assumptions and Limitations....................................................................................................11 7.1Assumptions..............................................................................................................................11 7.2Limitations................................................................................................................................11 8.Appendix .................................................................................................................................11 .......................................................................................................................................................12

www.Fullinterview.com

Page<- 2 - >

www.Fullinterview.com

1. Introduction In the problem scenario we have to convert Materialized View (MV) into table with minimum outage for data refresh. To achieve the requirement we have lots of options like using UNIX shell script, ETL tool etc. In our project we have used Datastage as an ETL tool for data refresh.

2. Purpose In this project we had some materialized views which used to get refreshed after the completion of ETL jobs. As the previous approach was full refresh, so the downstream applications had to wait longer time to access the MVs and some of the MVs were taking several hours for this refresh. So we have a requirement to reduce the outage, so that the downstream application can access the data in minimum time. The purpose of this document is to explain the approach method used to convert materialized view into table with minimum outage for data refresh.

3. Approach Points Convert the MVs into tables through DB scripts. Create Datastgae jobs which will load data into new tables.

4. Conclusion We will create DB script which will convert MV into table and create datastage jobs for loading into the table, which will minimise outage for data refresh.

www.Fullinterview.com

Page<- 3 - >

www.Fullinterview.com

5. Procedures and Processes 5.1 Converting MV into Table For converting MV into tables the following activities have been performed: Creation of a temporary table MV_TB using the MV data and structure Dropping the original MV Renaming the temporary table with the MV name Creating indexes on the MV Table that originally existed on the MV Creating a table MV_TM using the MV Table and truncating it. Granting required privileges on each MV Table

www.Fullinterview.com

Page<- 4 - >

www.Fullinterview.com

5.2 Creating Datastage jobs to populate the MV The Materialized Views, converted into tables, will be refreshed by Datastage jobs. The job contains two Oracle Enterprise Stages. The source query is written in such a way that no rows actually are selected and as a result nothing is inserted into the target table.

Fig 1: Datastage Server Job

www.Fullinterview.com

Page<- 5 - >

www.Fullinterview.com

In the before SQL part of the source Oracle stage, the below mentioned steps are followed in the form of Oracle query which in turn minimises the outage for data refresh. A screenshot is also attached for reference. Data is loaded into the temporary table based on MV select query Truncate the MV The index is dropped from the MV if required MV is renamed to MV_S TM is renamed to MV MV_S is renamed to TM Index is created on the MV if required

Fig 2: Query Used in Server Job

www.Fullinterview.com

Page<- 6 - >

www.Fullinterview.com 6. Sample code 6.1 MV Creation Script: Execute the onetime script that converts the existing Materialized views into table--create table MV_TB as select * from MV; drop materialized view MV; alter table MV_TB rename to MV; CREATE INDEX SC1DM.ASSETS_MTM_SERIAL ON MV (MTM_SERIAL_NBR) NOLOGGING TABLESPACE SC1_DM_INDX_SML NOPARALLEL; create table MV_TM as select * from MV; truncate table MV_TM; 6.2 Query Used in Job: The query which loads data into newly created table, which in turn minimises the outage for data refresh, is given below TRUNCATE TABLE MV_TM; INSERT INTO MV_TM (SELECT DISTINCT b10.SIEBEL_ASSET_ID, b10.SERIAL_NBR AS SERIAL_Number, COALESCE (TO_CHAR(b11.START_DT,'mm/dd/yyyy'), (TO_CHAR(B10.INSTALL_DT,'mm/dd/yyyy'))) AS Entitlement_Start_Date, COALESCE (TO_CHAR(b11.END_DT,'mm/dd/yyyy'),TO_CHAR((B10.INSTALL_DT + ENT_T2.DURATION),'mm/dd/yyyy'))AS Entitlement_End_Date, Prod2.PRODUCT_MODEL_NM AS MTM, PROD2.PRODUCT_KEY, ctry2.LXK_ORG_COUNTRY_DESC AS Country, ast_p2.CURR_PROFILE_FLG, prod2.PRODUCT_PART_NBR, prod_ln2.PRODUCT_LINE_DESC AS Product_Line, (SUBSTR(prod2.PRODUCT_MODEL_NM,1,4) ||'-'|| b10.SERIAL_NBR ) AS mtm_Serial_nbr, (b10.SERIAL_NBR ||'-'||prod2.PRODUCT_PART_NBR ) AS Serial_nbr_part_nbr, ENT_T2.ENTLMT_TEMPL_NM AS Entitlement_Name, AC2.ACCOUNT_NM AS ACCOUNT_NAME, agr2.AGREEMENT_NBR AS Agreement_Number,

www.Fullinterview.com

Page<- 7 - >

www.Fullinterview.com adrs2.ADDRESS_LINE_1, ADRS2.CITY, ADRS2.STATE, ADRS2.POSTAL_CD, adrs2.COUNTRY AS Install_Country, adrs2.PROVINCE FROM SC1DM.SC1_DM_ASSET_D B10, SC1DM.SC1_DM_ASSET_ENTLMT_HIST_D T1, SC2dm.SC2_DM_ENTLMT_D B13, SC2DM.SC2_DM_ENTLMT_HIST_D B11 , SC1dm.SC1_DM_AGREEMENT_HIST_D agrh, SC1DM.SC1_DM_ASSET_PROFILE_D AST_P2, SC2DM.SC2_DM_ADDRESS_D ADRS2, SC2dm.SC2_DM_ACCOUNT_D ac2, SC1dm.SC1_DM_AGREE_ENTLMT_D agr_ent2, SC1dm.SC1_DM_AGREEMENT_D agr2, SC2dm.SC2_DM_ENTLMT_TEMPL_D ENT_T2, SC2dm.SC2_DM_PRODUCT_D prod2, SC2dm.SC2_DM_PRODUCT_LN_PROD_D prod_rel2, SC2dm.SC2_DM_PRODUCT_LINE_D prod_ln2, SC2dm.SC2_DM_LXK_ORG_D ctry2 WHERE B10.SIEBEL_ASSET_ID = T1.SIEBEL_ASSET_ID (+) AND T1.EFF_END_DT IS NULL AND b10.SIEBEL_ASSET_ID = ast_p2.SIEBEL_ASSET_ID (+) AND ast_p2.INSTALL_ADDRESS_KEY = ADRS2.ADDRESS_KEY (+) AND T1.ENTLMT_KEY = B13.ENTLMT_KEY (+) AND B13.ENTLMT_KEY = B11.ENTLMT_KEY (+) AND b11.EFF_END_DT IS NULL AND ((b13.ENTLMT_TEMPL_KEY = ent_t2.ENTLMT_TEMPL_KEY)) AND ast_p2.CURR_PROFILE_FLG = 'Y' AND ast_p2.INVOICE_ACCOUNT_KEY =AC2.ACCOUNT_KEY (+) AND b10.BASE_PRODUCT_KEY = prod2.PRODUCT_KEY AND prod2.PRODUCT_KEY = prod_rel2.PRODUCT_KEY AND prod_rel2.PRODUCT_LINE_KEY = prod_ln2.PRODUCT_LINE_KEY AND ast_p2.LXK_ORG_KEY = ctry2.LXK_ORG_KEY AND prod_ln2.PRODUCT_LINE_TYPE_KEY = 1 AND t1.ENTLMT_KEY = agr_ent2.ENTLMT_KEY (+) AND agr_ent2.SIEBEL_AGREEMENT_ID= agr2.SIEBEL_AGREEMENT_ID AND agr2.SIEBEL_AGREEMENT_ID=agrh.siebel_agreement_id (+) AND b13.deleted_dt IS NULL AND t1.delete_flg IS NULL AND agrh.eff_end_dt IS NULL AND agrh.AGREEMENT_STATUS NOT IN('Cancel','Cancelled') AND b11.START_DT != b11.END_DT ) UNION ( SELECT DISTINCT ast1.SIEBEL_ASSET_ID, ast1.SERIAL_NBR AS SERIAL_Number, TO_CHAR(AST1.INSTALL_DT,'mm/dd/yyyy') AS Entitlement_Start_Date,

www.Fullinterview.com

Page<- 8 - >

www.Fullinterview.com TO_CHAR((AST1.INSTALL_DT + ENT_T1.DURATION),'mm/dd/yyyy') AS Entitlement_End_Date, Prod1.PRODUCT_MODEL_NM AS MTM, PROD1.PRODUCT_KEY, ctry1.LXK_ORG_COUNTRY_DESC AS Country, ast_p1.CURR_PROFILE_FLG , prod1.PRODUCT_PART_NBR, prod_ln1.PRODUCT_LINE_DESC AS Product_Line, (SUBSTR(prod1.PRODUCT_MODEL_NM,1,4) ||'-'||ast1.SERIAL_NBR ) AS mtm_Serial_nbr, (ast1.SERIAL_NBR ||'-'||prod1.PRODUCT_PART_NBR ) AS Serial_nbr_part_nbr, ENT_T1.ENTLMT_TEMPL_NM AS Entitlement_Name, AC1.ACCOUNT_NM AS ACCOUNT_NAME, NULL AS Agreement_Number, adrs1.ADDRESS_LINE_1, ADRS1.CITY, ADRS1.STATE, ADRS1.POSTAL_CD, adrs1.COUNTRY AS Install_Country, adrs1.PROVINCE FROM SC1DM.SC1_DM_ASSET_D AST1, SC1DM.SC1_DM_ASSET_PROFILE_D AST_P1, SC2dm.SC2_DM_ENTLMT_TEMPL_D ENT_T1, SC2DM.SC2_DM_ADDRESS_D ADRS1, SC2dm.SC2_DM_ACCOUNT_D ac1, SC2dm.SC2_DM_PRODUCT_D prod1, SC2dm.SC2_DM_PRODUCT_LN_PROD_D prod_rel1, SC2dm.SC2_DM_PRODUCT_LINE_D prod_ln1, SC2dm.SC2_DM_LXK_ORG_D ctry1, SC2DM.SC2_DM_PROD_ENTLMT_TEMPL_D prod_ent1, SC2dm.SC2_DM_COUNTRY_D e_ctry WHERE AST1.SIEBEL_ASSET_ID = AST_P1.SIEBEL_ASSET_ID AND ast_p1.CURR_PROFILE_FLG = 'Y' AND ast_p1.INSTALL_ADDRESS_KEY = ADRS1.ADDRESS_KEY (+) AND ast_p1.INVOICE_ACCOUNT_KEY =AC1.ACCOUNT_KEY (+) AND ast1.BASE_PRODUCT_KEY = prod1.PRODUCT_KEY AND prod1.PRODUCT_KEY = prod_rel1.PRODUCT_KEY AND prod_rel1.PRODUCT_LINE_KEY = prod_ln1.PRODUCT_LINE_KEY AND ast_p1.LXK_ORG_KEY = ctry1.LXK_ORG_KEY AND AST_P1.invoice_account_key IS NULL AND prod_ln1.PRODUCT_LINE_TYPE_KEY = 1 AND prod1.PRODUCT_KEY= prod_ent1.PRODUCT_KEY AND prod_ent1.ENTLMT_TEMPL_KEY = ent_t1.ENTLMT_TEMPL_KEY AND prod_ent1.DELETE_FLG is null AND ent_t1.COUNTRY_KEY = e_ctry.COUNTRY_KEY AND ctry1.lxk_org_country_desc= DECODE(LOWER(e_ctry.country_nm),'usa','United States',e_ctry.country_nm)

www.Fullinterview.com

Page<- 9 - >

www.Fullinterview.com AND TO_CHAR(AST1.INSTALL_DT,'mm/dd/yyyy') != TO_CHAR((AST1.INSTALL_DT + ENT_T1.DURATION),'mm/dd/yyyy')); commit; TRUNCATE TABLE MV; DROP INDEX SC1DM.ASSETS_MTM_SERIAL; ALTER TABLE MV RENAME TO MV_S; ALTER TABLE MV_TM RENAME TO MV; ALTER TABLE MV_S RENAME TO MV_TM; CREATE INDEX SC1DM.ASSETS_MTM_SERIAL ON MV (MTM_SERIAL_NBR) NOLOGGING TABLESPACE SC1_DM_INDX_SML NOPARALLEL

www.Fullinterview.com
>

Page<- 10 -

www.Fullinterview.com

7. Assumptions and Limitations 7.1 Assumptions The tables and indexes are in analyzed state before running the job for the first time. The user is having DBA privilege.

7.2 Limitations While creating any new index (via executing the MV creation script), ensure that no index with the same index name and same attribute combination exists in the Database.

8. Appendix Reference Material The Programming Language With Oracle by Ivan Bayross SQL: The Complete Reference by James R. Groff & Paul N. Weinberg

www.Fullinterview.com
>

Page<- 11 -

www.Fullinterview.com

About Wipro Technologies Wipro is the first PCMM Level 5 and SEI CMMi Level 5 certified IT Services Company globally. Wipro provides comprehensive IT solutions and services (including systems integration, IS outsourcing, package implementation, software application development and maintenance) and Research & Development services (hardware and software design, development and implementation) to corporations globally. Wipro's unique value proposition is further delivered through our pioneering Offshore Outsourcing Model and stringent Quality Processes of SEI and Six Sigma.

Wipro in Manufacturing & Healthcare


Wipro Technologies offers world class software and technology solutions for the Manufacturing & Healthcare industry. We address Sales and Distribution, Underwriting, Policy Administration, Accounting, Claims Processing and Backoffice. Wipros unique value proposition is delivered through our pioneering Offshore Development Model and stringent Quality Processes including ISO 9000, SEI CMM Level 5 and Six Sigma.

Copyright 2002. Wipro Technologies. All rights reserved. No part of this document may be reproduced, stored in a retrieval system, transmitted in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without express written permission from Wipro Technologies. Specifications subject to change without notice. All other trademarks mentioned herein are the property of their respective owners. Specifications subject to change without notice.

www.Fullinterview.com
>

Page<- 12 -