Push straight to prod

API development with R and TensorFlow at T-Mobile

rstudio::conf
January 17th, 2019
Heather Nolis, Machine Learning Engineer, T-Mobile – @heatherklus
Jacqueline Nolis, Principal Data Scientist, Nolis LLC – @skyetetra

PAGE 1 | AI @ T-MOBILE
 What does it mean to put into production?

ANALYSIS BUILD
Running code once Writing code that is
to produce a result continuously running

Putting code into production is letting customers interact with it

PAGE 2 | AI @ T-MOBILE
 The project:

Goal Use machine learning to improve the customer experience

Scope customer care messaging

PAGE 3 | AI @ T-MOBILE
Scenario Customer sends message:
“This high bill shall not pass!”

Goal Prep customer care agent before first response:

Current bill status

Method
• Classify the message with machine learning: [bill breakdown]
• Improve the prediction with customer data:
• Recent account activity
• Signal strength
• Bill status [overdue]

PAGE 4 | AI @ T-MOBILE
 Model creation with R

PAGE 5 | AI @ T-MOBILE
 Model building workflow

1. rmarkdown for exploratory analysis

2. Save the model to flat files (and log the build with
rmarkdown)

3. Show model off with a shiny demo

PAGE 6 | AI @ T-MOBILE
 Neural networks with R, Keras, and TensorFlow
A Convolutional Neural Network (CNN) processes
initial customer message and customer data

“Unlock my phone.” Recent order: YES

CNN

ACCOUNT UNLOCK ORDER

0.05 0.80 0.15

PAGE 7 | AI @ T-MOBILE
PAGE 8 | AI @ T-MOBILE
 Model deployment to prod

PAGE 9 | AI @ T-MOBILE
 Choosing the language

“If you want machine learning in production you need Python” –literally everyone

Option Analysis Modeling Deployment Verdict

1 ❎

2 ❎

3 ✅

PAGE 10 | AI @ T-MOBILE
 Idea! Treat R like a full
programming language
(because it is one)

Commit is Jenkins builds the Marathon Mesos replicates

made to AI image orchestrates container and hosts the
repository deployment containers
PAGE 11 | AI @ T-MOBILE
 Turning R into an API: plumber

Create an R file for Start the plumber Now make HTTP

the API endpoints service requests to R!

rest_controller.R main.R
#* @get /sum library(plumber)
function(a, b){ r <- plumb("rest_contoller.R")
as.numeric(a) + as.numeric(b) r$run(host='0.0.0.0',port=80)
}

PAGE 12 | AI @ T-MOBILE
 Behold: containers!
Example Dockerfilae
# start from the rocker/r-ver:3.5.0 image
FROM rocker/r-ver:3.5.0

# install the linux libraries needed for plumber

RUN apt-get update -qq && apt-get install -y \
libssl-dev \
libcurl4-gnutls-dev

# install plumber
RUN R -e "install.packages('plumber')"

# copy everything from the current directory into

the container
COPY / /
• Dockerfile—build instructions # open port 80 to traffic
• Image—the result of the Dockerfile EXPOSE 80

• Container—Hitting run on an image # when the container starts, start the main.R
script
ENTRYPOINT ["Rscript", "main.R"]

PAGE 13 | AI @ T-MOBILE
 Container Struggles: Python

R Keras requires Python

• Added Python 3.6 (Anaconda) and
Python Keras installation to Dockerfile
• Configured Reticulate to use correct
Python

PAGE 14 | AI @ T-MOBILE
 Container Struggles: Security

Plumber does not support HTTPS

• Added an Apache 2 server to reroute

Text needs encryption Docker Container

• Sodium R library
HTTPS HTTP
Apache R
HTTPS 2 HTTP (plumber)

PAGE 15 | AI @ T-MOBILE
 Container Struggles: Size

Initial size (5+ GB)

• Switched Pythons
(Anaconda →Miniconda)
• Removed Rstudio
(rocker/tidyverse → rocker/r-
ver:3.5.0)
• Removed unnecessary Linux, R,
Python libraries

PAGE 16 | AI @ T-MOBILE
 Status: UP
R-native API container

TensorFlow enabled

Lightweight

Secure

PAGE 17 | AI @ T-MOBILE
 Lessons learned

PAGE 18 | AI @ T-MOBILE
 R vs Python

• R & Plumber close to parity with Python & Flask

• R advantageous for quick data exploration
• Language was never the project failpoint

PAGE 19 | AI @ T-MOBILE
 Working Together is Critical
Sr. Manager

ML Engineer

ML Engineer Data Scientist

Product

Java Engineer Data Scientist

Data Scientist

PAGE 20 | AI @ T-MOBILE
 Where are we today

PAGE 21 | AI @ T-MOBILE
 R in prod: Apple Business Chat

“Can I please pay my bill” Customer

PAGE 22 | AI @ T-MOBILE
 And you can be here too!

PAGE 23 | AI @ T-MOBILE
 Materials: nolisllc.com/rstudio19
twitter: @skyetetra @heatherklus

PAGE 24 | AI @ T-MOBILE