10101 11111 0101110

1111110 10101

Business Intelligence (BI)

Business Intelligence

The Real Danger is not that Computers will begin to think like Men/Women, but that men/women will begin to think like a Computer.

Growth Trends
Moores law
Computer Speed doubles every 18 months

Storage law
total storage doubles every 9 months

Consequence
very little data will ever be looked at by a human

Knowledge Discovery is NEEDED to make sense and use of data.

Technology Challenge:
Can we effectively process increasing volumes of data?
Sensors
Intelligent Agents

Multimedia

Data
Text

Image

Binary
11010010

One small step for man

1970

1980

1990

2000

2010

Time

Trends leading to Data Flood

More data is generated: Bank, telecom, other business transactions ... Scientific Data: astronomy, biology, etc Web, text, and ecommerce More data is captured: Storage technology faster and cheaper DBMS capable of handling bigger DB

Examples
Europe's Very Long Baseline Interferometry (VLBI) has 16 telescopes, each of which produces 1 Gigabit/second of astronomical data over a 25-day observation session storage and analysis a big problem Wal-Mart reported to have 24 Tera-byte DB

AT&T handles billions of calls per day data cannot be stored -- analysis is done on the fly

More data and data sources

What is the main WWW problem?

With an estimated 1000 million web pages finding the one you want is difficult!

Data, Information, Knowledge and Intelligence

Evolution from Static Report to BI

Past, Present and Future - Why Intelligence is important ?

1. 2. 3. 4. 5. 6. 7. 8. 9.

To convert Data into Information To support decision makers with information at all levels in organizations To manage organizations by facts To recognize if to take Action To analyse for answering Why? To control Risk To control Performance To make reliable predictions To Optimise for best outcome

12

Vision - BI: Right Stuff

Just the Right Content to Just the Right Person at Just the Right time on Just the Right Device in Just the Right Context and Just the Right way!

INTRODUCTION
Organizations need business intelligence Business intelligence (BI) knowledge about your customers, competitors, business partners, competitive environment, and internal operations to make effective, important, and strategic business decisions

15

Business Intelligence Definition

BI is neither a product nor a system. It is an architecture and a collection of integrated operational as well as decision-support applications and databases that provide the business community easy access to business data.

BI Defined
Business Intelligence (BI) encompasses the processes, tools, and technologies required to transform enterprise data into information, and information into knowledge that can be used to enhance decision-making and to create actionable plans that drive effective business activity. BI can be used to acquire Tactical insight to optimize business processes by identifying trends, anomalies, and behaviors that require management action. Strategic insight to align multiple business processes with key business objectives through integrated performance management and analysis.

Business Intelligence
Increasing potential to support business decisions End User

Making Decisions
Data Presentation Visualization Techniques Data Mining Information Discovery Data Exploration Statistical Analysis, Querying and Reporting

Business Analyst Data Analyst

Data Warehouses / Data Marts OLAP Data Sources Paper, Files, Information Providers, Database Systems, OLTP

DBA

OLAP

Online Analytical Processing, or OLAP is an approach to swiftly answer multidimensional analytical queries OLAP is part of the broader category of business intelligence, which also encompasses and data mining Databases configured for OLAP use a multidimensional data model, allowing for complex analytical and ad-hoc queries with a rapid execution time. They borrow aspects of navigational databases and hierarchical databases that are faster than relational databases. The core of any OLAP system is an OLAP cube (also called a 'multidimensional cube' or a hypercube). It consists of numeric facts called measures which are categorized by dimensions. The cube metadata is typically created from a star schema or snowflake schema of tables in a relational database. Measures are derived from the records in the fact table and dimensions are derived from the dimension tables. Fact Table: The fact table holds the main data. It includes a large amount of aggregated data, such as price and units sold. There may be multiple tables in a star schema. Dimension tables : which are usually smaller than fact tables, include the attributes that describe the facts. Often this is a separate table for each dimension. Dimension tables can be joined to the fact table(s) as needed.

OLAP
Multidimensional structure is defined as a variation of the relational model that uses multidimensional structures to organize data and express the relationships between data.The structure is broken into cubes and the cubes are able to store and access data within the confines of each cube. Each cell within a multidimensional structure contains aggregated data related to elements along each of its dimensions.Even when data is manipulated it is still easy to access as well as be a compact type of database. The data still remains interrelated. Multidimensional structure is quite popular for analytical databases that use online analytical processing (OLAP) applications Analytical databases use these databases because of their ability to deliver answers swiftly to complex business queries

Aggregations in OLAP
It has been claimed that for complex queries OLAP cubes can produce an answer in around 0.1% of the time for the same query on OLTP relational data.The most important mechanism in OLAP which allows it to achieve such performance is the use of aggregations. Aggregations are built from the fact table by changing the granularity on specific dimensions and aggregating up data along these dimensions. The number of possible aggregations is determined by every possible combination of dimension granularities. The combination of all possible aggregations and the base data contains the answers to every query which can be answered from the data. Because usually there are many aggregations that can be calculated, often only a predetermined number are fully calculated; the remainder are solved on demand. The problem of deciding which aggregations (views) to calculate is known as the problem. View selection can be constrained by the total size of the selected set of aggregations, the time to update them from changes in the base data, or both. The objective of view selection is typically to minimize the average time to answer OLAP queries,

Types of OLAP
Multidimensional ( MOLAP):MOLAP is the 'classic' form of OLAP and is sometimes referred to as just OLAP. MOLAP stores this data in an optimized multi-dimensional array storage, rather than in a relational database. Therefore it requires the pre-computation and storage of information in the cube - the operation known as processing. Relational ( ROLAP):ROLAP works directly with relational databases. The base data and the dimension tables are stored as relational tables and new tables are created to hold the aggregated information. Depends on a specialized schema design.This methodology relies on manipulating the data stored in the relational database to give the appearance of traditional OLAP's slicing and dicing functionality. In essence, each action of slicing and dicing is equivalent to adding a "WHERE" clause in the SQL statement. Hybrid : (HOLAP) : There is no clear agreement across the industry as to what constitutes "Hybrid OLAP", except that a database will divide data between relational and specialized storage. For example, for some vendors, a HOLAP database will use relational tables to hold the larger quantities of detailed data, and use specialized storage for at least some aspects of the smaller quantities of more-aggregate or less-detailed data.

Comparison
MOLAP implementations are prone to database explosion, a phenomenon causing vast amounts of storage space to be used by MOLAP databases when certain common conditions are met: high number of dimensions, precalculated results and sparse multidimensional data. MOLAP generally delivers better performance due to specialized indexing and storage optimizations. MOLAP also needs less storage space compared to ROLAP because the specialized storage typically includes compression techniques. ROLAP is generally more scalable. However, large volume pre-processing is difficult to implement efficiently so it is frequently skipped. ROLAP query performance can therefore suffer tremendously. Since ROLAP relies more on the database to perform calculations, it has more limitations in the specialized functions it can use. HOLAP encompasses a range of solutions that attempt to mix the best of ROLAP and MOLAP. It can generally pre-process swiftly, scale well, and offer good function support.

Why BI? The Five Questions

What happened?

Past

What is happening? Why did it happen? What will happen? What do I want to happen?

Present
Future

Data

ERP

CRM

SCM

3Pty
24

Business Case for BI

Competitive Advantage

Higher

Predictive Metrics

What will happen?

Business Value

Descriptive Metrics

Why has it happened?

Baseline Metrics

Lower

What has happened?

Data

Information

Analysis

Insight

Complexity of Analysis

Business Intelligence Solutions are...

"... solutions which "...the combination of enable an end-user to data warehouse and quickly and easily end-user data access analyze and analysis tools..." organizational data to "...an investment in make intelligent information decisions..." technology that can "... systems that put assist governments in information in the managing limited hands of end-users..." fiscal resources..."

Business Intelligence solutions start with data warehouses and data marts
Analysis Complexity & Value

Optimization
Discovery

Data Mining
Verification

Multidimensional

Statistical Data Mart Data Warehouse

Stage 1 Stage 2 Stage 3 Stage 4 Stage 5

And this is the Required Transformation

Business Intelligence Functions

Business Intelligence Architecture

Core Technologies
Core technologies
DBMS data warehouses data marts

Enabling Technologies
Enabling technologies
query processing data mining OLAP SQL XML Various integration tools

BI Solutions
BI Solutions
views querying modeling reporting

BI Applications
Supply Chain Management Remote Performance Monitoring Simulation Knowledge Management Customer Relationship Management

Page Effectiveness Report

Percentage of visits clicking on different links
14% 3% 2% 8% 2% 13% 9% 0.6% Top Menu 6%

3% 2% 2% 0.3% 2%

18% of visits exit at the welcome page

Any product link 7%

Customer Locations Relative to Retail Stores

Heavy purchasing areas away from retail stores can suggest new retail store locations
No stores in several hot areas: MEC is building a store in Montreal right now.

Map of Canada with store locations.

Black dots show store locations.

BI Implementation Issues
Planning Critical Success Factors Ease of use Scalability Performance Security Architecture Design Execution Planning needs to be very detailed and well through out Levels of detail required need to be considered carefully you cant drill down if there is no additional detail Needs of various departments to be considered

BI Tools and Skill Sets

Databases ETL Tools ROLAP Tools MOLAP Tools HOLAP Tools
Oracle, Sybase, SQL Server, Informix, Red Brick Informatica, Ascential Datastage

Microstrategy, Information Advantage, Informix Metacube

Cognos, Hyperion Essbase, Oracle express, Brio, Business Objects Seagate Info, Holos Cold Fusion 4.x, Microstrategy DSS Web, Brio On demand server, Cognos Powerplay Web, Business Objects- web

Web-enabling Tools

OLAP = online analytical processing applications ROLAP: data stored relational database; MOLAP: data stored in multi-dimen. database. HOLAP: Hybrid OLAP combination of both worlds.

Benefit of Business Intelligence

Improved performance based upon timely and accurate information Elimination of guesswork Expedited decision making Early visibility of changes: Customer buying patterns Supply chain activity Financial arrangements

Single Version of the truth Accurate, timely data available to all levels of the organization

Business Intelligence User

There are many different users who can benefit from business intelligence Executives Business Decision Makers Information Workers Line Workers Analysts

Business Intelligence Jobs

Business Intelligence Consultant Business Intelligence Developer Business Intelligence Analyst Business Intelligence Project Team Member of the fastest growing segments of IT

One

Less likely to be outsourced May exist in business units rather than IT Knowledge/understanding of the organization is key

Business Intelligence Jobs

Business Analyst Data Analyst Functional Analyst Marketing Analyst Report Developer Data Modeler ETL Developer Data Architect Data Warehouse Designer Data Warehouse Developer Data Warehouse Administrator Database Administrator

Next Generation BI
Is proactive rather than reactive Guides the future direction of business Leverages the modern Service Oriented architecture for its flexibility and adaptability Analyzes both structured data as well as bulk of unstructured content Produces results in real time Understands the context and hidden information