Conformed Dimensions (CD): these dimensions are something that is built once in your model
and can be reused multiple times with different fact tables. For example, consider a model
containing multiple fact tables, representing different data marts. Now look for a dimension
that is common to these facts tables. In this example let’s consider that the product dimension
is common and hence can be reused by creating short cuts and joining the different fact
tables.Some of the examples are time dimension, customer dimensions, product dimension.
Junked Dimensions (JD): When you consolidate lots of small dimensions and instead of having
100s of small dimensions, that will have few records in them, cluttering your database with
these mini ‘identifier’ tables, all records from all these small dimension tables are loaded into
ONE dimension table and we call this dimension table Junk dimension table. (Since we are
storing all the junk in this one table) For example: a company might have handful of
manufacture plants, handful of order types, and so on, so forth, and we can consolidate them
in one dimension table called junked dimension table.
Degenerated Dimension (DD): An item that is in the fact table but is stripped off of its
description, because the description belongs in dimension table, is referred to as Degenerated
Dimension. Since it looks like dimension, but is really in fact table and has been degenerated of
its description, hence is called degenerated dimension. Now coming to the slowly changing
dimensions (SCD) and Slowly Growing Dimensions (SGD): I would like to classify them to be
more of an attributes of dimensions its self.
Although other might disagree to this view but Slowly Changing Dimensions are basically those
dimensions whose key value will remain static but description might change over the period of
time. For example, the product id in a companies, product line might remain the same, but the
description might change from time to time, hence, product dimension is called slowly
changing dimension.
Lets consider a customer dimension, which will have a unique customer id but the customer
name (company name) might change periodically due to buy out / acquisitions, Hence, slowly
changing dimension, as customer number is static but customer name is changing, However, on
the other hand the company will add more customers to its existing list of customers and it is
highly unlikely that the company will acquire astronomical number of customer over night
(wouldn’t the company CEO love that) hence, the customer dimension is both a Slowly
changing as well as slowly growing dimension.
POSTED BY MANJULA AT 9:01 AM 41 COMMENTS
A. Targets The most common performance bottleneck occurs when the informatica server
writes to a target database. You can identify target bottleneck by configuring the session to
write to a flat file target. If the session performance increases significantly when you write to a
flat file, you have a target bottleneck.
Solution : Drop or Disable index or constraints Perform bulk load (Ignores Database log)
Increase commit interval (Recovery is compromised) Tune the database for RBS, Dynamic
Extension etc.,
B. Sources Set a filter transformation after each SQ and see the records are not through. If the
time taken is same then there is a problem. You can also identify the Source problem by Read
Test Session – where we copy the mapping with sources, SQ and remove all transformations and
connect to file target. If the performance is same then there is a Source bottleneck. Using
database query – Copy the read query directly from the log. Execute the query against the
source database with a query tool. If the time it takes to execute the query and the time to
fetch the first row are significantly different, then the query can be modified using optimizer
hints.
C. Mapping
If both Source and target are OK then problem could be in mapping. Add a filter transformation
before target and if the time is the same then there is a problem.
(OR) Look for the performance monitor in the Sessions property sheet and view the counters.
Solutions: If High error rows and rows in lookup cache indicate a mapping bottleneck. Optimize
Single Pass Reading:
2. Optimizing the lookup condition whenever multiple conditions are placed, the condition with
equality sign should take precedence.
You can improve the efficiency by filtering early in the data flow. Instead of using a filter
transformation halfway through the mapping to remove a sizable amount of data.
Use a source qualifier filter to remove those same rows at the source, If not possible to move
the filter into SQ, move the filter transformation as close to the source qualifier as possible to
remove unnecessary data early in the data flow.
1. Try creating a reusable Seq. Generator transformation and use it in multiple mappings 2.
The number of cached value property determines the number of values the informatica server
caches at one time.
If you do not have a source, target, or mapping bottleneck, you may have a session bottleneck.
You can identify a session bottleneck by using the performance details. The informatica server
creates performance details when you enable Collect Performance Data on the General Tab of
the session properties. Performance details display information about each Source Qualifier,
target definitions, and individual transformation. All transformations have some basic counters
that indicate the Number of input rows, output rows, and error rows. Any value other than zero
in the readfromdisk and writetodisk counters for Aggregate, Joiner, or Rank transformations
indicate a session bottleneck.
E. System (Networks)
Dataware House Basic Concepts
What is a Data Warehouse?
A Data Warehouse is the "corporate memory". Academics will say it is a subject oriented, point-
in-time, inquiry only collection of operational data.
Typical relational databases are designed for on-line transactional processing (OLTP) and do
not meet the requirements for effective on-line analytical processing (OLAP). As a result, data
warehouses are designed differently than traditional relational databases.
What is ETL?
ETL is the Data Warehouse acquisition processes of Extracting, Transforming (or Transporting)
and Loading (ETL) data from source systems into the data warehouse.
What is the difference between a data warehouse and a data mart?
There are inherent similarities between the basic constructs used to design a data warehouse
and a data mart. In general a Data Warehouse is used on an enterprise level, while Data Marts
is used on a business division/department level. A data mart only contains the required subject
specific data for local analysis.
A warehouse typically contains years of data (Time Referenced). Data warehouses group data
by subject rather than by activity (subject-oriented). Other properties are: Non-volatile (read
only) and Integrated.
When should one use an MD-database (multi-dimensional database) and not a relational one?
Data in a multi-dimensional database is stored as business people views it, allowing them to
slice and dice the data to answer business questions. When designed correctly, an OLAP
database will provide must faster response times for analytical queries.
Normal relational databases store data in two-dimensional tables and analytical queries against
them are normally very slow.
How can Oracle Materialized Views be used to speed up data warehouse queries?
With "Query Rewrite" (QUERY_REWRITE_ENABLED=TRUE in INIT.ORA) Oracle can direct queries
to use pre-aggregated tables instead of scanning large tables to answer complex queries.
Materialized views in a W/H environments is typically referred to as summaries, because they
store summarized data.