1
Agenda
1 Big Data & Tantangan Teknologi
2
Peluang Big Data dalam SJK
2
The History of Data Processing
22 PB (2012,
the Large Hadron
Collider)
Scientific Data,
Scientific Instruments
24 PB/day
(2009)
5
Velocity
6
Variety
7
Veracity
Timeliness
Completeness
Accuracy Uniqueness
Validity Consistency
Data Quality
8
Persoalan Big Data http://www.nature.com
https://www.codewars.com/
https://theentrepreneurfund.com
Descriptive:
• Summary Statistics
• Clustering
• Association Rules
Volume
Velocity
Big
Data Value!
Variety
Veracity
Predictive:
• Regression
• Classification
9
“Value” dalam Sektor Jasa Keuangan
Customer
Analytics
HR Fraud
Analytics Analytics
Value
Security Risk
Analytics Analytics
Operational
Analytics
10
“Value” dalam Sektor Jasa Keuangan
● Summary Statistics
○ Apakah ada hubungan (korelasi) antara jenis pekerjaan customer dengan
jenis dan frekwensi transkasi keuangannya
● Clustering
○ Lakukan clustering terhadap profile customer untuk kebutuhan promosi
produk keuangan yang baru.
● Association Rules/Market Bucket Analysis
○ Berdasarkan data transaksi melalui produk jasa keuangan dari seluruh
customer, tentukan adanya asosiasi antara berbagai jenis transaksi yang
ada.
● Regression
○ Berdasarkan data history dan variable-variable lain, prediksi berapa harga
saham sebuah perusahaan dalam beberapa hari ke depan
● Classification
○ Berdasarkan karakteristik sebuah transkasi credit card, prediksi apakah
transakasi yang sedang/baru saja berjalan adalah sebuah fraud
Source: https://www.hdfstutorial.com/blog/big-data-use-cases-in-
○ Berdasarkan data seorang calon custumer, tentukan apakah dia potensial banking-and-financial-services/
Source:
Shubham Sinha
https://www.edureka.co/blog/hadoop-ecosystem
12
Technical Challenge (2-Velocity): Kafka General Pipeline
13
Technical Challenge (3-Variety): No SQL
2 Non Relational P P P
3 Schema Free M M M
4 Simple API
Shared Nothing
Commodity Hardware
14
Paradigm Shift
1 Bringing the Data to the Analytics vs Bringing the Analytics to the Data
• Focus on mobilizing data for analysis
• Focus on structuring data for storage
• Immediate ingestion of new data sources
• Serial approach to mobilizing new • Continuous data discovery
data sources • Agile, self-service data visualization
• Episodic analytics • Data as a platform
• Pre-defined reports and dashboards • Derive insight from structured and
unstructured data
Business: IT/Data:
a platform to enable
Questions to ask
creative dicovery
IT/Data: Business:
Structures the data to explore what
answer the question question to ask
15
Source: UAE Views on Big Data
Data Sources
● Financial structured data: ● Financial semi-structured data:
○ Trading systems (transaction data) ○ Financial products Markup Language (FpML)
○ Account systems (data on account holdings and ○ Financial Information eXchange (FIX)
movements)
○ Interactive Financial eXchange (IFX)
○ Market data from external providers
○ Market Data Definition Language (MDDL)
○ Securities reference data
○ Financial Electronic Data Interchange (FEDI)
○ Price information
○ Technical indicators
● Financial unstructured data:
○ Daily stock feeds
○ Company announcements (ad-hoc news) Relational
16
Emerging Trend: Big Data in Finance
● Marketing partnerships to develop enhanced profile of customer
● Targeted offers to cross-sell and up-sell
● Performance marketing –improve promotion effectiveness, Personalized
Marketing
● Leverage multiple sources of unstructured data to improve 360 degree view of
customer
● Customer retention
● Banking-Customer Segmentation
● Manage credit risks
● Risk Modelling for Investment banks
● Credit and Payment Fraud Analytics for Credit cards
● Sales force productivity and effectiveness
● Trade portfolio performance and optimization
● Real time- Next Best Offer for Retail banking
Source:
UAE Views on Big Data (with modification)
17
Opportunity
• Increase credit score • Personalize offerings,
● Enterprise memiliki 80-85% accuracy based on new
data sources (e.g. social
identify new cross-sell
opportunities and control
meda) for improved risk churn based on deep
unstructured data, control and more
competitive pricing
insights on customer
behavior, drawn from
multiple sources of data
○ Financial sector/enterprise
memiliki banyak data Improve Risk Enhance
terstruktur/semi terstruktur and Pricing Customer
Management Experience
● Infrastructure/Technology
readiness
● New data sources: internet, web, Increase Identify New
Operational Business
open data, third party, etc Efficiency Models
● Opensource Analytic/ML Engine • Optimize data • Monetize raw
management cost using anonimmized customer
● Value creation based on bigdata Big Data Tech, improve
operational efficiency
data or behavioural
insights that enable
analytics/processing in the
through real time improved market
processing, performance, analysis
management
financial sector
Source:
https://www.linkedin.com/pulse/top-3-big-data-use-cases-banking-industry-converged-karan-sachdeva/
18
Use Case (1) - DSIB
Bank A
Bank B
Bank D
Bank C
Bank K Bank E
Bank F Bank G
Bank J
Bank I
19
Use Case (2) – Paypal, Kafka
400 Billion Messages a Day with
!
Kafka at Paypal
Source:
https://www.youtube.com/watch?v=1loifCT4gTo
20
Use Case (3): SberBank
Source:
Edwin Van der Ouderaa. The Digital Bank Powered by Analytic
21
Use Case (4): Real time- Next Best Offer for Retail banking
● Better profile the customer and use collaborative and content-based filtering
to offer the most appropriate product or bundle of products at any given time
Source:
https://www.linkedin.com/pulse/top-3-big-data-use-cases-banking-industry-converged-karan-sachdeva/ 22
Use Case (5): Proyeksi Kondisi Makro Ekonomi
23
Use Case (6): Sentimen Masyarakat, Dashboard
● Sentimen Positive/Negative
Masyarakat
● Ringkasan Analisis/Pendapat Ahli
Ekonomi
● Dampak Kebijakan Keuangan
terhadap Parameter Pertumbuhan
Ekonomi (Descriptive atau
Predictive)
24
Use Case (7): Graph Analytics to Fight Fraud
25
Challenges
26
Thank you.
27
Saiful Akbar
Perkuliahan:
Manajemen Informasi | Visualisasi Data
Analisis Data dan Bisnis | Teknologi Big Data
Interest:
Similarity Retrieval
Data Analytics & Visualization
Multimedia Database
28