Big Data Government Use Cases

Ellery Wee
Associate Director

This presentation, including any supporting materials, is owned by Gartner, Inc. and/or its affiliates and is for the sole use of the intended Gartner audience or other
authorized recipients. This presentation may contain information that is confidential, proprietary or otherwise legally protected, and it may not be further copied,
distributed or publicly displayed without the express written permission of Gartner, Inc. or its affiliates.
2013 Gartner, Inc. and/or its affiliates. All rights reserved.
Big data is #1
on the 2012 list of most ambiguous terms
- Global Language Monitor

most searched term among clients

- on Gartner.com
The Big Data Challenge
The Increasing Volume of And so is the Increasing Variety
Information is a Major Challenge and Velocity of Data

We now create as much Documents/Content

information every two days as
we did from the dawn of Rich Media
civilization to 2003." Operational Technology
Eric
EricSchmidt, Chairman
Schmidt, & &
Chairman Streaming Data
CEO Google
CEO Google
Source: Siegler, M.G. TechCrunch. 4 August 2010
Big Data Forecasts

Big Data IT spending will surpass $200 billion

through 2016.
Year Enterprise Social Media IT Services
Software Revenue ($M) Spending ($M)
Spending ($M)
2012 2,918 1,384 23,476

2013 3,516 1,812 28,578

2014 4,240 2,827 37,404

2015 5,207 3,615 36,189

2016 6,461 4,411 43,713

Source: Gartner (December 2012)

3
Key Issues

1. What is the definition of Big data? How do I

recognize Big data issues?
2. What are some business opportunities for
Big data?
3. What technology components constitute a Big
data architecture?
4. What are the primary challenges and best
practices for rolling out Big data projects?
Big Data Characteristics

Growing quantity of data

VOLUME
(e.g., social media, behavioral, video)

Quickening speed of data

(e.g., smart meters, VELOCITY
process monitoring)

Gartner, Feb 2001

Increase in types of data
(e.g., app data, unstructured data)
Big Data Opportunities
Business Amplification
Making better informed decisions
(e.g., strategies, recommendations)

Discovering hidden insights

(e.g., anomalies forensics,
patterns, trends)

Optimizing business processes

(e.g., complex events, translation)
 The Business Value Comes From
Three Sources
Big Data Capabilities

Ability to Store and Process Unstructured Data

Ability to move from historical to real-time and predictive analytics

Ability to Affordably Perform Comprehensive Analysis

Customer Product Operations Risk Fraud

Insights Management Management Detection
and Marketing

Primary Use Cases

Today's World
Unlimited
Analysis

Analytic
Omniscience
Factory

Limited Unlimited
Data Data

Today's Big
World Data

Constrained
Analysis
Today's World
Customer Centricity Is Central to Analytics

Customer-Centric Outcomes
Other Functional Objectives

Customer-Centric Outcomes
New Business Model
Operational Optimization
Employee Collaboration
Risk/Financial Management

Respondents were asked to rank their top functional objectives for big data
within their organizations. Responses were weighted and aggregated.
Total Respondents= 1067
Source: IBM
Analysis for External Use:
Analysis Is the Customer Experience

If You Enjoy then watch

BETA
Big Data What Does It Look Like?
Public Sector:
- Real-time monitoring of
traffic flows, vehicle
locations, resource use.
Public Safety:
- Forecasting the extent and
impact of natural and
man-made disasters.
- Surveillance.
Utilities:
- Understanding individualized
use patterns.
- Better demand
management.
Retail:
- Web and e-commerce:
Analyze clickstream,
customer feedback.
Provide faster and more
accurate customer
responses.
- Analyze video footage of
store activity in real time:
Monitor customer traffic.
Monitor sales conversations
in physical stores.
- Market basket analysis.
- Merchandising analytics.
The Big Data Opportunity: By Industry Vertical
Banking and Securities
Better understand customers, assess risk,
and improve customer experience and
business processes. Identify fraud and anti-
money-laundering. Help trading decisions.
Communications and Media
Customer churn and marketing. Location-
based services and context-aware
information. Sentiment analysis and
campaign effectiveness. Audience
engagement or target advertising
Education
Research programs (especially medical),
improving learning outcomes, adaptive
learning, "personalized learning, social
media data for marketing purposes
Government
Fighting fraud, waste, abuse and improper
payments. Benefits eligibility. justice and
public safety and actually target preventing
crime. intelligence, anti-terrorism, financial
fraud, anti-money-laundering, cross-border
tracking. Open data initiatives
Healthcare Providers
Discover patterns that yield new treatments
and new medicine. improve diagnosis,
delivery of care, operational efficiency and
compliance
Insurance
Fight claims fraud or assess risk. Actuarial
analyses and pricing. Leverage social
media, images, telematics and climate data
These Big Data Investments Are
Happening Across Multiple Industries
Has your organization already invested in technology specifically designed to
address the big data challenge?

4% 5%
12% 9% 10% 14% 13%
18% 16%
28% Dont know
18% 29%
25% 36%
30%
28%
30% 37% 35% No plans at this time
17%
29% 21% 15%
15% 11%
17% 21% No, but plan to within
15% 9%
11% 18% 18% two years
17%
12% 20%
18% 18% No, but plan to within
12% 8%
39% the next year
36% 36% 31%
29% 25% 21% 22% 23% 23% Yes
Communications,

Manufacturing
Banking
Education

Transportation

Energy, Utilities
Retail

Healthcare

Insurance

Government
Media, Services

Source: Gartner (July 2012)

The Different Characteristics of Big Data
by Industry Drive Different Use Cases

Media and Services

Manufacturing and
Natural Resources
Communications,

Wholesale Trade
Transportation
Banking and

Government

Healthcare
Education
Securities

Insurance
Providers

Utilities
Retail
Volume
of Data
Velocity
of Data
Variety
of Data
Underutilized
"Dark Data"

Potential big data opportunity on each dimension is:

Very hot (compared with other industries)
Hot
Moderate
Low
Very low (compared with other industries)

Based on Expertise of Gartner Vertical Industry Analysts

Source: Gartner (July 2012) 14
Top Business Problems Addressed By Big Data
Are Process, Cost and Customer Experience
What are the 'Big Data' business problems you are now addressing or will likely
address soon?
Comm,
Govern- Energy & Media & Trans- Health-
Mfg. Education Banking Insurance ment Utilities Svcs Retail portation care

Process efficiency/ cost

1 3 3 3 1 1 1 2 1 1
reduction

Enhanced customer
3 2 5 1 2 2 2 1 3 3
experience

Improved customer
4 6 6 4 4 4 4 3 2 2
service
Indentifying new
product/market 2 7 4 2 7 5 3 5 4 4
opportunities

Risk management 5 1 1 5 6 7 6 4 5 5

Regulatory compliance 6 4 2 6 5 3 5 7 7 6

Enhanced security
7 5 7 7 3 6 7 6 6 7
capabilities

Source: Gartner (July 2012)

Note: 1 = top ranked business problem within an industry, 2= second, etc.
Big Data Case Studies - HiTech
Started with Google, Facebook,
Yahoo using it every day
Have you ever wondered how
they know you so well and
prompt you so often?

What's your Klout score?

Take in data feeds from numerous
social networks and analyze it
Produce an influencer score and
make it available via an API
Big Data Case Studies Travel
and Media

Analyzing underutilized dark data

web traffic resulting from web traffic
Understanding customer behavior
better and make more targeted
offers

Optimizing new film launch Media Company

Understanding sentiment based on
trailers, advance screenings and
first weekend
Adjusting editing, budget, rollout,
marketing
6th Floor: Big Data Based Pricing
Opportunity
- Better react to changing market conditions with near-
real-time product pricing
Data and Analytics
- Sales, inventory data on 73M items; 50% annual data
growth
- Hadoop, R, Impala, SAS, Vertica, Tableau
- Data science organization Generates and tests
hundreds of thousands of analytic models on granular
data versus the dozens previously possible
Results
- Reduced time to price items from 27 hours to about
one hour
- 70% hardware cost reduction

18
Police Predict Predator's Position
Opportunity
- Increase the speed of Swedish police investigations
Data and Analytics
- Communication behaviour from phone calls in
combination with crime statistics, weather, day-of-
week and city events
- Analysed data from over 500,000 interrogations,
evidence and background info using QlikView
Results
Reduced nine months of manual analysis to three
minutes of automated analytics
Helped locate a serial killer in the city of Malm by
calculating the time and location of the next shooting
6.7M krone reallocated from administration to law
enforcement
19
NOAAs Ark of Weather Data
Opportunity National
Oceanic and
- Travel safety Atmospheric
Administration
- Community preparation for weather related events
Data and Analytics
- 30 petabytes of data per year from 3.5 billion daily
observations via satellites, ships, aircraft, buoys and other
sensors
- Sophisticated high-resolution predictive modeling
Results
- Generates millions of weather-related products per day
including weather warnings and guidance for public and
private sectors
- Saves lives and expense via severe weather alerts

20
Dial Algorith-M For Murder
Opportunity
- Reducing homicide rates
- Safer communities
Data and Analytics
- Dataset of two dozen variables on 60,000 crimes
- Predictive algorithm created by University of
Pennsylvania
Results
- Contradicts and corrects conventional wisdom
among law enforcement community
- Improves (optimizes) parolee supervision and
reduces dept of corrections expense
- Can predict 8 out of 100 homicides in MD, PA and
DC

21
Listening to united voices
Opportunity
- Evolving from reactive to proactive responses to
global issues
- Developing and guiding assistance programs
Data and Analytics
- Mining social networks to predict job losses,
spending reductions, or disease outbreaks within a
region
- Natural language deciphering
Result (TBD)
- Early warning signals to guide assistance
programs for preventing regions from slipping into
poverty, epidemics, or war

22
Gartner Analytic Ascendency Model
How can we
make it happen?
Prescriptive
What will Analytics
happen?
Predictive
Why did it
VALUE

Analytics
happen?
Diagnostic
What Analytics
happened?
Descriptive
Analytics

DIFFICULTY
Government Use Case #1

Law enforcement efficiency in relation to road

traffic can be achieved through analysis of traffic
data acquired from smartphones and other
location-aware devices, such as navigation units;
it includes analysis of average speed .
(descriptive, diagnostic, predictive, prescriptive
analytics).

24
Government Use Case #2

Targeted social media campaigns have been

used for elections, but can be used much more
broadly for local politics, referenda and other
means of communicating with citizens
(descriptive, predictive analytics).

25
Government Use Case #3

Adaptive services can provide a better citizen

experience. Sensors in trash containers, for
example, enable waste management to be
demand-based, instead of schedule-based; law
enforcement can be aided by CCTV analytics
(predictive analytics, prescriptive analytics).

26
Government Use Case #4

Parking optimization is under way in various

cities, based on sensor-driven parking meters.
Car drivers can see through a smartphone app
which parking spots are vacant in their vicinity,
and even which ones still have credit on them
(descriptive analytics).

27
Government Use Case #5

Microdata services can arise from various

government services by, for example, charging a
small fee for straightforward information queries,
such as those from the land registry office, or
providing access to decision models that help
with applications for subsidies, legal assistance,
environmental advice and so forth (general
management).

28
Government Use Case #6

Preventive policing, also known as algorithmic

law enforcement, is a new field that predicts
which areas are most susceptible to crime, based
on historical patterns, weather conditions, the
impact of particular events and so forth
(prescriptive analytics, predictive analytics).

29
Government Use Case #7
Identity matching at border controls is now
about more than checking photographic IDs. It
involves many factors, most of them unknown to
the general public
(descriptive analytics, predictive analytics).

30
Some Progress, Still Hyped
expectations
Logical Data
Warehouse
Internet of Things
Hadoop SQL Interfaces
Video Search
Information Semantic Services
Information Capabilities Framework
Table-Style Database Management Services
Intent-Driven Customer Systems
Search-Based Data Discovery Tools
Data Science
High-Performance Message Infrastructure
Big Data Analytics for E-Commerce
Graph Databases
Database Software
Quantified Self
Document Store Database Management Systems
Complex-Event Processing
Context-Enriched Services
Content Analytics
Dynamic Data Masking
Key-Value Database Management Systems
Cloud-Based Grid Computing
In-Memory Database Management Systems
Social Analytics
Entity Resolution and Analysis
Sales Analytics Predictive Analytics
Telematics
Hadoop Distributions Speech Recognition
In-Memory Data Grids
Social Media Monitors
Cloud Parallel Processing
as a Service Cloud Computing
(dbSaaS) Intelligent Electronic Devices

Operational Intelligence Platforms

Information Valuation and In-Memory Analytics
Infonomics Text Analytics

Big Data Analytics for Customer Service

As of July 2013
Peak of
Innovation Trough of Plateau of
Inflated Slope of Enlightenment
Trigger Disillusionment Productivity
Expectations
time
Plateau will be reached in: obsolete
less than 2 years 2 to 5 years 5 to 10 years more than 10 years before plateau

Source: Hype Cycle for Big Data, 2013, 31 July 2013 (G00252431)
Enabling Technologies
expectations
Logical Data Document Store DBMS
Warehouse Complex-event Processing

Hadoop SQL Interfaces Content Analytics

Video Search Dynamic Data Masking
Key-Value DBMS
Cloud-based Grid Computing
Table-Style DBMS In-memory Database Management Systems
Intent-driven Customer Systems
Search-based Data Discovery Tools

High-performance Message Infrastructure

Predictive Analytics

Hadoop Distributions Speech Recognition

Graph Databases In-memory Data Grids
Cloud Parallel Processing
Database Software
as a Service Cloud Computing
(dbSaaS) Intelligent Electronic Devices

Operational Intelligence Platforms

In-memory Analytics
Text Analytics

As of July 2013
Peak of
Innovation Trough of Plateau of
Inflated Slope of Enlightenment
Trigger Disillusionment Productivity
Expectations
time
Plateau will be reached in: obsolete
less than 2 years 2 to 5 years 5 to 10 years more than 10 years before plateau

Source: Hype Cycle for Big Data, 2013, 31 July 2013 (G00252431)
Skills and Use Cases
expectations

Context-Enriched Services

Video Search
Information Semantic Services

Intent-Driven Customer Systems

Social Analytics
Data Science
Entity Resolution and Analysis
Sales Analytics Predictive Analytics
Big Data Analytics for E-Commerce Telematics
Speech Recognition
Social Media Monitors

Quantified Self

Information Valuation and In-Memory Analytics

Infonomics Text Analytics

Big Data Analytics for Customer Service

As of July 2013
Peak of
Innovation Trough of Plateau of
Inflated Slope of Enlightenment
Trigger Disillusionment Productivity
Expectations
time
Plateau will be reached in: obsolete
less than 2 years 2 to 5 years 5 to 10 years more than 10 years before plateau

Source: Hype Cycle for Big Data, 2013, 31 July 2013 (G00252431)
 Evolving Big Data Technologies
In-memory DBMS
Business Intelligence
SAP HANA,
Datameer, Karmasphere
Oracle Exalytics

Data warehouse DBMS NoSQL Databases

IBM, HP, Teradata, HBase, Cassandra,
Oracle, EMC MongoDB, Riak

File Systems MapReduce

HDFS, GlusterFS, GPFS, Hadoop MapReduce
Isilon OneFS, Lustre .
State of Big Data Projects Today

Smaller-scale projects have been undertaken by

groups outside of IT:
- Skunkworks Hadoop/NoSQL projects built on unused
hardware, existing files, and open-source software.
Enterprise workloads have different
requirements:
- Requirements: Performance, stability, persistence, data
protection, management, open-source options,
analytics, manageable costs.
- Who holds the budget software, hardware
and services?
Message: It's time for adult supervision.
An Upcoming Shortage of Talent
Does today's workforce understand these analysis
techniques?
- A/B testing (split testing)
- Association rule learning
- Cluster analysis
- Crowdsourcing
- Data fusion/data integration
- Ensemble learning
- Machine learning
- Natural language processing (linguistics analysis)
How will IT support this?
- Hire data scientists or support those hired by LOBs
- Expert systems
Framework for Evaluating
Big Data Analytics Initiatives
Business
Technology Economic
Relevance POC POC
Filter
Filter Filter POC
Filter

Game Broad
Changers Adoption
Success Factors Go Go Go
Business Niche
Step 1 Step 2 Step 3 Step 4 Extenders Adoption

Issues & Barriers No Go No Go No Go No

Adoption
Recommendations

Balance the application of customer analysis for

internal and external (customer-facing) purposes.
Increase the volume of analysis through:
- The development of a model factory environment
- The adoption of packaged applications
- The use of analytic service providers
Deal with the growth of available data through a
focus on cross-domain insight and an emphasis
on real-time responsiveness.
Prepare for a world of omniscience by developing
a culture of questions and rapid execution.
Recommended Gartner Research
The Importance of 'Big Data': A Definition
Mark A. Beyer, Douglas Laney (G00235055)
Big Data Drives Rapid Changes in Infrastructure and
$232 Billion in IT Spending Through 2016
Mark A. Beyer, John-David Lovelock, Dan Sommer,
Merv Adrian (G00245237)
How to Choose the Right Apache Hadoop
Distribution
Merv Adrian (G00227159)
Six Best Practices for Apache Hadoop Pilot Projects
Arun Chandrasekaran, Merv Adrian (G00234972)

For more information, stop by Experience Gartner Research Zone.