CON8279 - Kent-2014.09.29 - Dijcks - Kent - OOW - Oracle Big Data Appliance OOW V3

Big Data Hands-On Labs:
Date Time Location

Tuesday 3:45pm – 4:45pm Hotel Nikko - Peninsula
Wednesday 1:15pm – 2:15pm Hotel Nikko - Peninsula
Thursday 11:30am – 12:30pm Hotel Nikko - Peninsula
Or download: Big Data Lite Virtual Machine
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |

Oracle Big Data Appliance
for
Customers and Partners
Jean-Pierre Dijcks
Oracle
Big Data Product Management
Paul Kent
SAS
VP Big Data

Oracle Big Data Appliance for Customers and Partners
1 Big Data Appliance Recap
2 Why You Should Consider Big Data Appliance
3
Driving Business Value with SAS on Big Data Appliance
4
Q&A
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 3

Oracle Big Data Management System
Oracle Big Data SQL

Oracle Database
Oracle Industry
Cloudera Hadoop Oracle
ModelsDatabase
Oracle Big Data
Oracle NoSQL Database Connectors Oracle
OracleAdvanced
AdvancedSecurity
Analytics
Oracle R Advanced Oracle Advanced
Analytics for Hadoop Oracle Data Analytics
Oracle Spatial & Graph
Oracle R Distribution B Integrator
Oracle Spatial & Graph
Big Data Appliance Oracle Exadata

SOURCES

Recap: Big Data Appliance Overview
Big Data Appliance X4-2
Sun Oracle X4-2L Servers with per server:
• 2 * 8 Core Intel Xeon E5 Processors
• 64 GB Memory
• 48TB Disk space
Integrated Software:
• Oracle Linux, Oracle Java VM
• Oracle Big Data SQL*
• Cloudera Distribution of Apache Hadoop – EDH Edition
• Cloudera Manager
• Oracle R Distribution
• Oracle NoSQL Database
* Oracle Big Data SQL is separately licensed
Recap: Standard and Modular
 Starter Rack is a fully cabled and

configured for growth with 6 servers
 In-Rack Expansion delivers 6 server
modular expansion block
 Full Rack delivers optimal blend of
capacity and expansion options
 Grow by adding rack – up to 18 racks
without additional switches

Recap: Harness Rapid Evolution
BDA 4.0 – Sept 2014
• Big Data SQL
• Node Migration
BDA 2.x – April 2013
• Starter Rack
• In-Rack Expansion
• EM Integration
BDA 3.x – April 2014
BDA
b b
• CDH 5.0 (MR2 & YARN)
• AAA Security
• Encryption 4.0
b
BDA 1.0 – Jan 2012
• Initial BDA
• Mammoth Install

Core Design Principles for Big Data Appliance
Operational Simplicity Simplify Access to ALL Data

• Oracle Big Data SQL

– Oracle SQL on ALL your data
– All Native Oracle SQL Operators
– Smart Scan for Optimized Performance
• Oracle Security
– Govern all Data through a Single Set of
Security Policies

Oracle Big Data SQL – A New Architecture
• Powerful, high-performance SQL on Hadoop

– Full Oracle SQL capabilities on Hadoop
– SQL query processing local to Hadoop nodes
• Simple data integration of Hadoop and Oracle Database
– Single SQL point-of-entry to access all data
– Scalable joins between Hadoop and RDBMS data
• Optimized hardware
– Balanced Configurations
– No bottlenecks
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Oracle Confidential – Internal/Restricted/Highly Restricted 10
Big Data SQL
SELECT w.sess_id, c.name
FROM web_logs w, customers c
WHERE w.source_country = ‘Brazil’
AND w.cust_id = c.customer_id;
Relevant SQL runs on BDA nodes
Big Data SQL

10’s of Gigabytes of Data
WEB_LOGS B B B Only columns and rows needed to
CUSTOMERS
answer query are returned
Hadoop Cluster Oracle Database

Big Data SQL
SELECT w.sess_id, c.name
FROM web_logs w, customers c
WHERE w.source_country = ‘Brazil’
SQL PushAND
Down in Big Data SQL
w.cust_id = c.customer_id;
• Hadoop Scans on Unstructured Data

• WHERE ClauseRelevant
Evaluation
SQL runs on BDA nodes
• Column Projection
Big Data• SQLBloom Filters for Better Join Performance
• JSON Parsing,10’s Data
of Gigabytes of DataEvaluation
Mining Model
WEB_LOGS B B B Only columns and rows needed to
CUSTOMERS
answer query are returned
Hadoop Cluster Oracle Database

Oracle Communications Data Model
Reference Architecture
Oracle Comms Apps

(BSS/OSS) ETL/ELT Big Data Customer
Adapters Platform Experience
(Hadoop/NoSQL)
Oracle Comms B
Ntwk Products (Tekelec
& Acme)
Real-Time
Operations
Adapters
Other Oracle Apps

Relational
(CRM, ERP, etc.) Data Warehouse
(OCDM)
Third
Monetization
Party
Third Party Sources
Adapters Data Management Analytic Apps
Data
Sources Feedback Loop
To Other Apps


• No Bottlenecks
• Full Stack Install and Upgrades
• Simplified Management
– Cluster Growth
– Critical Node Migration
• Always Highly Available
• Always Secure
• Very Competitive Price Point
Successful Big Data Systems Grow
From Cluster Install with HA to Large Clusters to Dealing with Operational Issues
• 12 node BDA for Production
Day 1 HA and Security Set-up
• Hadoop
• Ready to Load Data
Full install with a single command:
RCK_1 ./mammoth –i rck_1

Day 1
RCK_1
N Example Service: Hadoop Name Nodes

N

Add 12 New Nodes across two Racks

Day 90
Cluster expansion with a single command:
mammoth –e newhost1,…,newhostn
RCK_1 RCK_2
N
N

Cluster Expansion with a single command:
mammoth –e newhost1,…,newhostn
RCK_1 RCK_2
This expansion automatically optimizes HA
setup across multiple racks
N Because of uniform nodes and IB networking,
N
no data is moved
Critical Node Failure => Primary Name Node

Day n
RCK_1 RCK_2
N
N

• Automatic Failover to other

NameNode
RCK_1 RCK_2 • Automatic Service Request to
Oracle for HW Failure
N
N

• Restore HA with a Single command

bdacli admin_cluster migrate N1
RCK_1 RCK_2
• Reinstate the Repaired Node with a Single
Command:
N
N bdacli admin_cluster reprovision N1

“Oracle Big Data Appliance is an excellent

choice for customers looking to work with
the full suite of Cloudera’sSimplicity
Operational leading
30% Quicker to Deploy
Hadoop-based technology. It’s more cost-
effective and quicker to deploy than a DIY
cluster.”
⁻ Mike Olson, Cloudera founder, Chief Strategy

Officer, and Chairman of the Board
21% Cheaper to Buy

Big Data Initiative @ Oracle Global Support Services
 Real-time access to better data means better

insights, which means better decisions and better
business results
 Integrate data associated with customer
telemetry, configurations, service history,
diagnostics, knowledge & support information
Anticipate
Anticipate Detect
Detect Predict
Predict Automate
Automate Delight
Delight

Core Design Principles Enable Success
Operational Simplicity
Simplify Access to ALL Data

There is one more thing…
Business Value = Applications

Big Data Appliance powers instant Business Value
Customer Experience Communications Cyber Security

Management Data Model Solutions
B B B
Introducing
Paul Kent - SAS

Big Data and Big Analytics
– So Much more
Gunpowder!
Paul Kent
VP BigData, SAS Research and Development
Copyright © 2014, SAS Institute Inc. All rights reserved.

1. Change
2. Safari Pics
[CON8279] Oracle Big Data Appliance:
Deep Dive and Roadmap for Customers and
Partners
Oracle Big Data Appliance is the premier Hadoop appliance in the
market. This session describes the roadmap for customers in the
areas of high-performance SQL on Hadoop and securing big data,
plus overall performance improvements for Hadoop.
A special focus in the session is the roadmap and benefits Oracle
Big Data Appliance brings to Oracle partners.
To illustrate the benefits of running on a standardized and optimized
Hadoop platform, SAS presents the findings of its tests of SAS In-
Memory Analytics on Oracle Big Data Appliance.

Agenda
1. SAS & Oracle Partnership
2. Family Stories
1. Hadoop
2. Oracle Engineered Systems Family
3. SAS Software Family
3. Deployment Patterns
 Reflection on a stronger partnership than ever
 Both leaders in Big Data –
 Jointly solving the most difficult and demanding Big
Data Problems
 Providing simplicity and agility to create flexible
configurations
 Extensive engineering collaboration
2014  Can we answer:
 How Does it Work?
 How Does it Perform?

THE
TAMOXIFEN
DILEMMA
SOURCE: http://commons.wikimedia.org/wiki/File:Tamoxifen-3D-vdW.png
C o py r i g h t © 2 0 1 2, S A S In s ti t u te I n c . A l l r i g ht s r e s e r v e d .
Agenda
2. Family Stories
1. Hadoop
Elephant :: 3 Good Ideas !!
1. Never forgets
2. Is a good (hard) worker
3. Is a Social Animal (teamwork)

Hadoop – Simplified View
Controller Worker Nodes
 MPP (Massively Parallel) hardware running database-like software

 “data” is stored in parts, across multiple worker nodes
 “work” operates in parallel ,on the different parts of the table

Idea #1 - HDFS. Never forgets!
Head Node Data 1 Data 2 Data 3 Data 4…

MYFILE.TXT
..block1 -> block1
..block2 -> block2
..block3 -> block3


MYFILE.TXT
..block1 -> block1 block1 copy2
..block3 -> block3 copy2 block3

X
MYFILE.TXT
..block1 -> block1 block1copy2
..block3 -> block3 copy2 block3
X
Redundancy Wins!

Idea #2 – MapReduce – Send the work to the Data
 We Want the Youngest Person in the Room
 Each Row in the audience is a data node
 I’ll be the coordinator
• From outside to center, accumulate MIN

• Sweep from back to front.
• Youngest Advances

Agenda
2. Family Stories
1. Hadoop
Recap: Standard and Modular
 Starter Rack is a fully cabled and

configured for growth with 6 servers
 In-Rack Expansion delivers 6 server
modular expansion block
 Full Rack delivers optimal blend of
capacity and expansion options
 Grow by adding rack – up to 18 racks
without additional switches

Oracle Big Data SQL – A New Architecture
• Powerful, high-performance SQL on Hadoop

– Full Oracle SQL capabilities on Hadoop
– SQL query processing local to Hadoop nodes
• Simple data integration of Hadoop and Oracle Database
– Single SQL point-of-entry to access all data
– Scalable joins between Hadoop and RDBMS data
• Optimized hardware
– Balanced Configurations
– No bottlenecks
Diversity. It’s a good thing!
Impala Nyala
Agenda
2. Family Stories
1. Hadoop
4 Important Things
#1 Join the Family

SAS ACCESS to Hadoop
HADOOP
SAS
Hive QL
SERVER

#2 Be Familiar
SAS / High Performance Analytics
HADOOP
SAS
SAS HPA
SERVER
Procedures

#3 Use the Cluster!
Prepare Explore / Transform Model
• HPDS2 • HPSUMMARY • HPLOGISTIC •HPLSO
• HPDMDB • HPCORR • HPREG •HPSPLIT
• HPSAMPLE • HPREDUCE • HPNEURAL •HPTMINE
• HPIMPUTE • HPNLIN •HPTMSCORE
• HPBIN • HPCOUNTREG
• HPMIXED
• HPSEVERITY
• HPFOREST
• HPSVM
• HPDECIDE
• HPQLIM

Controller
Client

#1 Join the Family
#2 Be Familiar
#3 Use the cluster
#4 Have a pretty face!
4 Important Things (for cluster friendly software)
1. Join the Family

2. Be Familiar
3. Performance
4. Have a pretty face

Agenda
2. Family Stories
1. Hadoop
SAS BIG DATA ON BIG DATA APPLIANCE
• Flexible Architectural options for SAS deployments

• Can run on Starter, Half and Full configurations
• Optionally select nodes “N, N-1, N-2, …” for additional SAS

Services such as SAS Compute Tier, SAS MidTier
• Optionally select node subset “N, N-1, N-2, N-3, …) for more
dedicated resources for SAS Analytic Compute Environment by
shifting Big Data Appliance roles
• Option to selectively add more memory on a per node basis

depending on specific workload distribution
63
C o py r i gh t © 2 0 1 3, S A S I ns t i t u t e I n c . A l l r i g ht s r e s e r v e d.
STARTER BDA
SAS Visual Analytics
SAS
HPA Metadata
SAS
Root Server
Midtie
Node SAS
r
Compute
SAS VISUAL ANALYTICS, HIGH-PERFORMANCE ANALYTIC COMPUTE

ENVIRONMENT CO-LOCATED WITH HADOOP
64
STARTER BDA
SAS
Metadata
HPA
Server
SAS Consider:
Root Midtie
Node SAS
r
Compute
Extra Memory for 5,6?

65
FULL RACK BDA
Metadata
SAS
SAS Server
Midtie
HPA SAS
r
Root Compute
Node
LASR LASR
Worker Worker
… 17 18
…
HDFS HDFS
Data Data
17 18

66
FULL RACK BDA
67
FULL RACK BDA
68
FULL RACK BDA
69
FULL RACK BDA
Basic Smoke Tests Confirmed:

Interoperate with Hadoop and Map Reduce
Read and Write text files to/from HDFS
Read and Write Tabular files to/from Hive (will confirm Oracle BIGSQL in OSC-SC)
Read and Write SAS binary format files to/from HDFS
High Degree Of Parallelism (DOP) reads via Map-Only jobs
SAS LASR server co-exists on/with datanodes

SAS HPA tasks scheduled on datanodes
70
SAS High-Performance Analytics Performance
SAS Format Data (SASHDAT)
1107 var 1107 var
11.795 Mobs 73.744 Mobs
97GB 608GB
5.7GB/node 35.7GB/node 6x
Create 208.79 sec 2284.29 sec 11
Scan/Count 24.60 sec 259.38 sec 10.5
HPCORR 295.20 1410.40 4.7
HPCNTREG
Table 1: Summation336.79 1547.59
of 5/20/100/200 columns; 4.6
Baseline:
HPREDUCE DOP=1
(u) (no236.55
parallelism) 2467.76 10.4
120M rows, 400
HPREDUCE (s) columns,
219.50reg_simtbl_400
2037.74 9.3
71
OSC-AU FullRack BDA
• 408 Threads
• 600 GB dataset
• 17 servers
Your Problem solved ASAP
72
73
74
75
EXADATA SAS EMBEDDED PROCESSING (EP) TO EXADATA
INTEGRATION LEVERAGING BIG DATA SQL
SAS
HPA Metadata
SAS
Root Server
Midtie
Node SAS
r
Compute
LASR
Worker
18
…
…
HDFS
Data
18
SAS
EP
Big Data SQL
76
SAS EP Parallel Data Feeders
DOP=1 DOP=24 DOP=24
(flash cache)
Add(5) 1.25min 1.5min .5min
Add(20) 2.5min 1.5min .5min
Add(100) 13min 1.5min .6min
Add(200) 16min ~2min 1.25min (10x)
Table 1: Summation of 5/20/100/200 columns;

Baseline: DOP=1 (no parallelism)
120M rows, 400 columns, reg_simtbl_400
77
SAS EP Parallel Data Feeders
Access Access / SAS HPA

DBSlice Using EP
Reg_sim_200 1:01:12 0:28:37 0:08:00
Reg_sim_400 1:49:11 0:55:33 0:16:05 (7x!)
Table 2: Scan times for 2 tables

(200 columns, 400 columns, 120M rows);
Baseline: SAS/ACCESS vs. HPA EP feeder
78
SAS Format Data (SASHDAT) and Oracle EXADATA
1107 var 907 var 1107 var
11.795 Mobs 11.795 Mobs 73.744 Mobs
97GB 79.7GB 608GB
5.7GB/node 4.7GB/node 35.7GB/node
SASHDAT EXADATA SASHDAT
Create 208.79 sec 931.22 sec 2284.29 sec
Scan/Count 24.60 sec 956.16 sec 259.38 sec
HPCORR 295.20 833.24 1410.40
Table 1: Summation
HPCNTREG 336.79of 5/20/100/200 columns;
756.97 1547.59
Baseline: DOP=1 (no parallelism)
HPREDUCE (u) 236.55 1055.11 2467.76
120M rows, 400 columns, reg_simtbl_400
HPREDUCE (s) 219.50 1051.93 2037.74
79
ORACLE ENGINEERED SYSTEMS FOR
ExaData ExaLogic SuperCluster Big Data ZFS Virtual Database
Appliance Storage Compute Backup, Recovery,

Appliance Appliance Logging Appliance
80
SAS AND ORACLE WOR
• Joint R & D development • Template physical architectures • Best Practice papers

and Product Management developed based on use-cases • SAS and Oracle Engineers
teams in Cary and Redwood • Physically tested and provide joint "Sizing and
Shores benchmarked together Architecture Analysis and
• Focus on driving SAS • Reduction in physical effort Design"
technology components to • Overall reduction in lifecycle
run natively in Oracle costs
database
• Joint performance
engineering optimizations
Paul.Kent @ sas.com SAS AND ORACLE
BETTER TOGETHER
@hornpolish
paulmkent

CON8279 - Kent-2014.09.29 - Dijcks - Kent - OOW - Oracle Big Data Appliance OOW V3

Diunggah oleh

Informasi Dokumen

Judul Asli

Hak Cipta

Format Tersedia

Bagikan dokumen Ini

Bagikan atau Tanam Dokumen

Opsi Berbagi

Apakah menurut Anda dokumen ini bermanfaat?

Apakah konten ini tidak pantas?

Hak Cipta:

Format Tersedia

CON8279 - Kent-2014.09.29 - Dijcks - Kent - OOW - Oracle Big Data Appliance OOW V3

Diunggah oleh

Hak Cipta:

Format Tersedia

Big Data Hands-On Labs:

Date Time Location

Or download: Big Data Lite Virtual Machine

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 3

Oracle Big Data SQL

Big Data Appliance Oracle Exadata

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |

 Starter Rack is a fully cabled and

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 6

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 7

Operational Simplicity Simplify Access to ALL Data

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 8

• Oracle Big Data SQL

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 9

• Powerful, high-performance SQL on Hadoop

Relevant SQL runs on BDA nodes

Big Data SQL

Hadoop Cluster Oracle Database

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 11

• Hadoop Scans on Unstructured Data

Hadoop Cluster Oracle Database

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 12

Oracle Comms Apps

Other Oracle Apps

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |

Operational Simplicity Simplify Access to ALL Data

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 14

Full install with a single command:

RCK_1 ./mammoth –i rck_1

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 16

N Example Service: Hadoop Name Nodes

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 17

Add 12 New Nodes across two Racks

Cluster expansion with a single command:

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 18

Cluster Expansion with a single command:

Critical Node Failure => Primary Name Node

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 20

• Automatic Failover to other

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 21

• Restore HA with a Single command

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 22

“Oracle Big Data Appliance is an excellent

⁻ Mike Olson, Cloudera founder, Chief Strategy

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 23

 Real-time access to better data means better

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |

Simplify Access to ALL Data

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 25

Business Value = Applications

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 26

Customer Experience Communications Cyber Security

Paul Kent - SAS

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 28

Copyright © 2014, SAS Institute Inc. All rights reserved.

Copyright © 2014, SAS Institute Inc. All rights reserved.

Copyright © 2014, SAS Institute Inc. All rights reserved.

Copyright © 2014, SAS Institute Inc. All rights reserved.

 MPP (Massively Parallel) hardware running database-like software

Copyright © 2014, SAS Institute Inc. All rights reserved.

Head Node Data 1 Data 2 Data 3 Data 4…