CON8279 Kent-2014.09.29 Dijcks Kent OOW Oracle Big Data Appliance OOW V3

Big Data Hands-On Labs:
Date Time Location

Tuesday 3:45pm 4:45pm Hotel Nikko - Peninsula
Wednesday 1:15pm 2:15pm Hotel Nikko - Peninsula
Thursday 11:30am 12:30pm Hotel Nikko - Peninsula
Or download: Big Data Lite Virtual Machine
Copyright 2014 Oracle and/or its affiliates. All rights reserved. |

Oracle Big Data Appliance
for
Customers and Partners
Jean-Pierre Dijcks
Oracle
Big Data Product Management
Paul Kent
SAS
VP Big Data

Oracle Big Data Appliance for Customers and Partners
1 Big Data Appliance Recap
2 Why You Should Consider Big Data Appliance
3 Driving Business Value with SAS on Big Data Appliance
4 Q&A
Copyright 2014 Oracle and/or its affiliates. All rights reserved. | 3

Oracle Big Data Management System
Oracle Big Data SQL

Oracle Database
Oracle Industry
Cloudera Hadoop Oracle
ModelsDatabase
Oracle Big Data
Oracle NoSQL Database Connectors Oracle
OracleAdvanced
AdvancedSecurity
Analytics
Oracle R Advanced Oracle Advanced
Analytics for Hadoop Oracle Data Analytics
Oracle Spatial & Graph
Integrator
Oracle R Distribution
B Oracle Spatial & Graph
Big Data Appliance Oracle Exadata

SOURCES

Recap: Big Data Appliance Overview
Big Data Appliance X4-2
Sun Oracle X4-2L Servers with per server:
2 * 8 Core Intel Xeon E5 Processors
64 GB Memory
48TB Disk space
Integrated Software:
Oracle Linux, Oracle Java VM
Oracle Big Data SQL*
Cloudera Distribution of Apache Hadoop EDH Edition
Cloudera Manager
Oracle R Distribution
Oracle NoSQL Database
* Oracle Big Data SQL is separately licensed

Recap: Standard and Modular
Starter Rack is a fully cabled and

configured for growth with 6 servers
In-Rack Expansion delivers 6 server
modular expansion block
Full Rack delivers optimal blend of
capacity and expansion options
Grow by adding rack up to 18 racks
without additional switches

Recap: Harness Rapid Evolution
BDA 4.0 Sept 2014
Big Data SQL
Node Migration
BDA 2.x April 2013
Starter Rack
In-Rack Expansion
EM Integration
BDA 3.x April 2014
CDH 5.0 (MR2 & YARN)
BDA
b b AAA Security
Encryption
4.0
b
BDA 1.0 Jan 2012
Initial BDA
Mammoth Install

Core Design Principles for Big Data Appliance
Operational Simplicity Simplify Access to ALL Data

Oracle Big Data SQL

Oracle SQL on ALL your data
All Native Oracle SQL Operators
SmartOperational Simplicity
Scan for Optimized Performance Simplify Access to ALL Data
Oracle Security
Govern all Data through a Single Set of
Security Policies

Oracle Big Data SQL A New Architecture
Powerful, high-performance SQL on Hadoop

Full Oracle SQL capabilities on Hadoop
SQL query processing local to Hadoop nodes
Simple data integration of Hadoop and Oracle Database
Single SQL point-of-entry to access all data
Scalable joins between Hadoop and RDBMS data
Optimized hardware
Balanced Configurations
No bottlenecks
Copyright 2014 Oracle and/or its affiliates. All rights reserved. | Oracle Confidential Internal/Restricted/Highly Restricted 10
Big Data SQL
SELECT w.sess_id, c.name
FROM web_logs w, customers c
WHERE w.source_country = Brazil
AND w.cust_id = c.customer_id;
Relevant SQL runs on BDA nodes
Big Data SQL

10s of Gigabytes of Data
WEB_LOGS
B B B Only columns and rows needed to
CUSTOMERS
answer query are returned
Hadoop Cluster Oracle Database

Big Data SQL
SELECT w.sess_id, c.name
FROM web_logs w, customers c
WHERE w.source_country = Brazil
SQL Push Down in
AND Big Data =SQL
w.cust_id c.customer_id;
Hadoop Scans on Unstructured Data

WHERE Clause Evaluation
Relevant SQL runs on BDA nodes
Column Projection

Big Data SQL Bloom Filters for Better Join Performance
10s of Gigabytes
JSON Parsing, Data Mining of Data
Model Evaluation
WEB_LOGS
B B B Only columns and rows needed to
CUSTOMERS
answer query are returned
Hadoop Cluster Oracle Database

Oracle Communications Data Model
Reference Architecture
Oracle Comms Apps

(BSS/OSS) ETL/ELT Big Data Customer
Adapters Platform Experience
(Hadoop/NoSQL)
Oracle Comms B
Ntwk Products
(Tekelec & Acme)
Real-Time
Operations
Adapters
Other Oracle Apps

Relational
(CRM, ERP, etc.) Data Warehouse
(OCDM)
Third
Monetization
Party
Third Party Sources
Adapters Data Management Analytic Apps
Data
Sources Feedback Loop To Other Apps

Operational Simplicity Simplify Access to ALL Data

No Bottlenecks
Full Stack Install and Upgrades
Simplified Management
Operational Simplicity Simplify
Cluster Access to ALL Data
Growth
Critical Node Migration
Always Highly Available
Always Secure
Very Competitive Price Point

Successful Big Data Systems Grow
From Cluster Install with HA to Large Clusters to Dealing with Operational Issues
12 node BDA for Production

Day 1 Hadoop HA and Security Set-up
Ready to Load Data
Full install with a single command:
RCK_1 ./mammoth i rck_1

Day 1
RCK_1
N Example Service: Hadoop Name Nodes

N

Add 12 New Nodes across two Racks

Day 90
Cluster expansion with a single command:
mammoth e newhost1,,newhostn
RCK_1 RCK_2
N
N

Cluster Expansion with a single command:
mammoth e newhost1,,newhostn
RCK_1 RCK_2
This expansion automatically optimizes HA
setup across multiple racks
N Because of uniform nodes and IB networking,
N
no data is moved
Critical Node Failure => Primary Name Node

Day n
RCK_1 RCK_2
N
N

Automatic Failover to other

NameNode
RCK_1 RCK_2 Automatic Service Request to
Oracle for HW Failure
N
N

Restore HA with a Single command

bdacli admin_cluster migrate N1
RCK_1 RCK_2
Reinstate the Repaired Node with a Single
Command:
N bdacli admin_cluster reprovision N1
N

Oracle Big Data Appliance is an excellent

choice for customers looking to work with
30% Quicker to Deploy
the full suite of ClouderasSimplicity
Operational leading
Hadoop-based technology. Its more cost-
effective and quicker to deploy than a DIY
cluster.
Mike Olson, Cloudera founder, Chief Strategy

21% Cheaper to Buy
Officer, and Chairman of the Board

Big Data Initiative @ Oracle Global Support Services
Real-time access to better data means better

insights, which means better decisions and better
business results
Integrate data associated with customer
telemetry, configurations, service history,
diagnostics, knowledge & support information
Anticipate Detect Predict Automate Delight

Core Design Principles Enable Success
Operational Simplicity
Simplify Access to ALL Data

There is one more thing
Business Value = Applications

Big Data Appliance powers instant Business Value
Customer Experience Communications Cyber Security

Management Data Model Solutions
B B B
Introducing
Paul Kent - SAS

Big Data and Big Analytics
So Much more Gunpowder!
Paul Kent
VP BigData, SAS Research and Development
Copyright 2014, SAS Institute Inc. All rights reserved.

1. Change
2. Safari Pics
[CON8279] Oracle Big Data Appliance:
Deep Dive and Roadmap for Customers and Partners
Oracle Big Data Appliance is the premier Hadoop appliance in the
market. This session describes the roadmap for customers in the
areas of high-performance SQL on Hadoop and securing big data,
plus overall performance improvements for Hadoop.
A special focus in the session is the roadmap and benefits Oracle
Big Data Appliance brings to Oracle partners.
To illustrate the benefits of running on a standardized and
optimized Hadoop platform, SAS presents the findings of its tests of
SAS In-Memory Analytics on Oracle Big Data Appliance.

Agenda
1. SAS & Oracle Partnership
2. Family Stories
1. Hadoop
2. Oracle Engineered Systems Family
3. SAS Software Family
3. Deployment Patterns
Reflection on a stronger partnership than ever
Both leaders in Big Data
Jointly solving the most difficult and demanding Big
Data Problems
Providing simplicity and agility to create flexible
configurations
Extensive engineering collaboration
2014 Can we answer:
How Does it Work?
How Does it Perform?

THE
TAMOXIFEN
DILEMMA
SOURCE: http://commons.wikimedia.org/wiki/File:Tamoxifen-3D-vdW.png
C op yr i g h t 2 0 1 2 , S A S I n s t i t u t e I n c . A l l r i g h t s r es er v e d .
Agenda
2. Family Stories
1. Hadoop
Elephant :: 3 Good Ideas !!
1. Never forgets
2. Is a good (hard) worker

3. Is a Social Animal (teamwork)

Hadoop Simplified View
Controller Worker Nodes
MPP (Massively Parallel) hardware running database-like software

data is stored in parts, across multiple worker nodes
work operates in parallel ,on the different parts of the table

Idea #1 - HDFS. Never forgets!
Head Node Data 1 Data 2 Data 3 Data 4

MYFILE.TXT
..block1 -> block1
..block2 -> block2
..block3 -> block3


MYFILE.TXT
..block1 -> block1 block1 copy2
..block3 -> block3 copy2 block3


MYFILE.TXT
..block1 -> block1 block1copy2
..block3 -> block3 copy2 block3

Redundancy Wins!

Idea #2 MapReduce Send the work to the Data
We Want the Youngest Person in the Room
Each Row in the audience is a data node
Ill be the coordinator
From outside to center, accumulate MIN

Sweep from back to front.
Youngest Advances

Agenda
2. Family Stories
1. Hadoop
Recap: Standard and Modular
Starter Rack is a fully cabled and

configured for growth with 6 servers
In-Rack Expansion delivers 6 server
modular expansion block
Full Rack delivers optimal blend of
capacity and expansion options
Grow by adding rack up to 18 racks
without additional switches

Oracle Big Data SQL A New Architecture
Powerful, high-performance SQL on Hadoop

Full Oracle SQL capabilities on Hadoop
SQL query processing local to Hadoop nodes
Simple data integration of Hadoop and Oracle Database
Single SQL point-of-entry to access all data
Scalable joins between Hadoop and RDBMS data
Optimized hardware
Balanced Configurations
No bottlenecks
Diversity. Its a good thing!
Impala Nyala
Agenda
2. Family Stories
1. Hadoop
4 Important Things
#1 Join the Family

SAS ACCESS to Hadoop
HADOOP
SAS
Hive QL
SERVER

#2 Be Familiar
SAS / High Performance Analytics
HADOOP
SAS SAS HPA
SERVER Procedures

#3 Use the Cluster!
Prepare Explore / Transform Model
HPDS2 HPSUMMARY HPLOGISTIC HPLSO
HPDMDB HPCORR HPREG HPSPLIT
HPSAMPLE HPREDUCE HPNEURAL HPTMINE

HPIMPUTE HPNLIN HPTMSCORE
HPBIN HPCOUNTREG
HPMIXED
HPSEVERITY
HPFOREST
HPSVM
HPDECIDE
HPQLIM

Controller
Client

#1 Join the Family
#2 Be Familiar
#3 Use the cluster
#4 Have a pretty face!

4 Important Things (for cluster friendly software)
1. Join the Family
2. Be Familiar
3. Performance
4. Have a pretty face

Agenda
2. Family Stories
1. Hadoop
SAS BIG DATA ON BIG DATA APPLIANCE
Flexible Architectural options for SAS deployments

Can run on Starter, Half and Full configurations
Optionally select nodes N, N-1, N-2, for additional SAS

Services such as SAS Compute Tier, SAS MidTier
Optionally select node subset N, N-1, N-2, N-3, ) for more

dedicated resources for SAS Analytic Compute Environment by
shifting Big Data Appliance roles
Option to selectively add more memory on a per node basis

depending on specific workload distribution
63
STARTER BDA
SAS Visual Analytics
SAS
HPA Metadata
Root Server SAS
Node SAS Midtier
Compute
SAS VISUAL ANALYTICS, HIGH-PERFORMANCE ANALYTIC COMPUTE

ENVIRONMENT CO-LOCATED WITH HADOOP
64
STARTER BDA
SAS
Metadata
HPA
Server SAS
Consider:
Root
Node SAS Midtier
Compute
Extra Memory for 5,6?

65
FULL RACK BDA
Metadata
SAS Server SAS
HPA SAS Midtier
Root Compute
Node
LASR LASR
Worker Worker
17 18

HDFS HDFS
Data Data
17 18

66
FULL RACK BDA ASSEMBLED IN OSC, SYDNEY AUSTRALIA
67
68
69
Basic Smoke Tests Confirmed:

Interoperate with Hadoop and Map Reduce
Read and Write text files to/from HDFS
Read and Write Tabular files to/from Hive (will confirm Oracle BIGSQL in OSC-SC)
Read and Write SAS binary format files to/from HDFS
High Degree Of Parallelism (DOP) reads via Map-Only jobs
SAS LASR server co-exists on/with datanodes

SAS HPA tasks scheduled on datanodes
70
SAS High-Performance Analytics Performance
SAS Format Data (SASHDAT)
1107 var 1107 var
11.795 Mobs 73.744 Mobs
97GB 608GB
5.7GB/node 35.7GB/node 6x
Create 208.79 sec 2284.29 sec 11
Scan/Count 24.60 sec 259.38 sec 10.5
HPCORR 295.20 1410.40 4.7
HPCNTREG
Table 1: Summation336.79 1547.59
of 5/20/100/200 columns; 4.6
Baseline:
HPREDUCE DOP=1
(u) (no236.55
parallelism) 2467.76 10.4
120M rows, 400 columns, reg_simtbl_400
HPREDUCE (s) 219.50 2037.74 9.3
71
OSC-AU FullRack BDA
408 Threads
600 GB dataset
17 servers
Your Problem solved ASAP
72
73
74
75
EXADATA SAS EMBEDDED PROCESSING (EP) TO EXADATA
INTEGRATION LEVERAGING BIG DATA SQL
SAS
HPA Metadata
Root Server SAS
Node SAS Midtier
Compute
LASR
Worker
18

HDFS
Data
18
SAS
EP
Big Data SQL
76
SAS EP Parallel Data Feeders
DOP=1 DOP=24 DOP=24
(flash cache)
Add(5) 1.25min 1.5min .5min
Add(20) 2.5min 1.5min .5min
Add(100) 13min 1.5min .6min
Add(200) 16min ~2min 1.25min (10x)
Table 1: Summation of 5/20/100/200 columns;

Baseline: DOP=1 (no parallelism)
120M rows, 400 columns, reg_simtbl_400
77
SAS EP Parallel Data Feeders
Access Access / SAS HPA

DBSlice Using EP
Reg_sim_200 1:01:12 0:28:37 0:08:00
Reg_sim_400 1:49:11 0:55:33 0:16:05 (7x!)
Table 2: Scan times for 2 tables

(200 columns, 400 columns, 120M rows);
Baseline: SAS/ACCESS vs. HPA EP feeder
78
SAS Format Data (SASHDAT) and Oracle EXADATA
1107 var 907 var 1107 var
11.795 Mobs 11.795 Mobs 73.744 Mobs
97GB 79.7GB 608GB
5.7GB/node 4.7GB/node 35.7GB/node
SASHDAT EXADATA SASHDAT
Create 208.79 sec 931.22 sec 2284.29 sec
Scan/Count 24.60 sec 956.16 sec 259.38 sec
HPCORR 295.20 833.24 1410.40
Table 1: Summation of 5/20/100/200 columns;
HPCNTREG
Baseline: DOP=1 336.79
(no parallelism) 756.97 1547.59
120M rows,
HPREDUCE (u) 400236.55
columns, reg_simtbl_400
1055.11 2467.76
HPREDUCE (s) 219.50 1051.93 2037.74
79
ORACLE ENGINEERED SYSTEMS FOR
ExaData ExaLogic SuperCluster Big Data ZFS Virtual Database
Appliance Storage Compute Backup, Recovery,
Appliance Appliance Logging Appliance
80
SAS AND ORACLE WORKING TOGETHER TO CREATE CUSTOMER VALUE
Joint R & D development and Template physical architectures Best Practice papers
Product Management teams developed based on use-cases SAS and Oracle Engineers
in Cary and Redwood Physically tested and provide joint "Sizing and
Shores benchmarked together Architecture Analysis and
Focus on driving SAS Reduction in physical effort Design"
technology components to Overall reduction in lifecycle
run natively in Oracle costs
database
Joint performance
engineering optimizations
Paul.Kent @ sas.com SAS AND ORACLE
BETTER TOGETHER
@hornpolish
paulmkent

CON8279 Kent-2014.09.29 Dijcks Kent OOW Oracle Big Data Appliance OOW V3

Diunggah oleh

Informasi Dokumen

Hak Cipta

Format Tersedia

Bagikan dokumen Ini

Bagikan atau Tanam Dokumen

Opsi Berbagi

Apakah menurut Anda dokumen ini bermanfaat?

Apakah konten ini tidak pantas?

Hak Cipta:

Format Tersedia

CON8279 Kent-2014.09.29 Dijcks Kent OOW Oracle Big Data Appliance OOW V3

Diunggah oleh

Hak Cipta:

Format Tersedia

Big Data Hands-On Labs:

Date Time Location

Or download: Big Data Lite Virtual Machine

Copyright 2014 Oracle and/or its affiliates. All rights reserved. |

Copyright 2014 Oracle and/or its affiliates. All rights reserved. |

3 Driving Business Value with SAS on Big Data Appliance

Copyright 2014 Oracle and/or its affiliates. All rights reserved. | 3

Oracle Big Data SQL

Big Data Appliance Oracle Exadata

Copyright 2014 Oracle and/or its affiliates. All rights reserved. |

* Oracle Big Data SQL is separately licensed

Starter Rack is a fully cabled and

Copyright 2014 Oracle and/or its affiliates. All rights reserved. | 6

Copyright 2014 Oracle and/or its affiliates. All rights reserved. | 7

Operational Simplicity Simplify Access to ALL Data

Copyright 2014 Oracle and/or its affiliates. All rights reserved. | 8

Oracle Big Data SQL

Copyright 2014 Oracle and/or its affiliates. All rights reserved. | 9

Powerful, high-performance SQL on Hadoop

Relevant SQL runs on BDA nodes

Big Data SQL

Hadoop Cluster Oracle Database

Copyright 2014 Oracle and/or its affiliates. All rights reserved. | 11

Hadoop Scans on Unstructured Data

Hadoop Cluster Oracle Database

Copyright 2014 Oracle and/or its affiliates. All rights reserved. | 12

Oracle Comms Apps

Other Oracle Apps

Copyright 2014 Oracle and/or its affiliates. All rights reserved. |

Operational Simplicity Simplify Access to ALL Data

Copyright 2014 Oracle and/or its affiliates. All rights reserved. | 14

Copyright 2014 Oracle and/or its affiliates. All rights reserved. | 15

12 node BDA for Production

Full install with a single command:

RCK_1 ./mammoth i rck_1

Copyright 2014 Oracle and/or its affiliates. All rights reserved. | 16

N Example Service: Hadoop Name Nodes

Copyright 2014 Oracle and/or its affiliates. All rights reserved. | 17

Add 12 New Nodes across two Racks

Cluster expansion with a single command:

Copyright 2014 Oracle and/or its affiliates. All rights reserved. | 18

Cluster Expansion with a single command:

Critical Node Failure => Primary Name Node

Copyright 2014 Oracle and/or its affiliates. All rights reserved. | 20

Automatic Failover to other

Copyright 2014 Oracle and/or its affiliates. All rights reserved. | 21

Restore HA with a Single command

Copyright 2014 Oracle and/or its affiliates. All rights reserved. | 22

Oracle Big Data Appliance is an excellent

Mike Olson, Cloudera founder, Chief Strategy

Copyright 2014 Oracle and/or its affiliates. All rights reserved. | 23

Real-time access to better data means better

Anticipate Detect Predict Automate Delight

Copyright 2014 Oracle and/or its affiliates. All rights reserved. |

Simplify Access to ALL Data

Copyright 2014 Oracle and/or its affiliates. All rights reserved. | 25

Business Value = Applications

Copyright 2014 Oracle and/or its affiliates. All rights reserved. | 26

Customer Experience Communications Cyber Security

Paul Kent - SAS

Copyright 2014 Oracle and/or its affiliates. All rights reserved. | 28

Copyright 2014, SAS Institute Inc. All rights reserved.

Copyright 2014, SAS Institute Inc. All rights reserved.

Copyright 2014, SAS Institute Inc. All rights reserved.