Best practice in data & scoring

Dr Paul Russell Director Analytical Solutions

Experian Limited 2007. All rights reserved. Experian and the marks used herein are service marks or registered trademarks of Experian Limited. Other product and company names mentioned herein may be the trademarks of their respective owners. No part of this copyrighted work may be reproduced, modified, or distributed in any form or manner without the prior written permission of Experian Limited. Confidential and proprietary.

Agenda
Some themes Analytics and the customer life cycle The role of scoring Building a scorecard Using scoring systems Risk management infrastructure

Experian Limited 2007. All rights reserved. Confidential and proprietary.

Themes

Best practice is often discussed but almost never seen

Do the simple things well

Risk management is more than just a scorecard

The same principles apply across the credit lifecycle

Experian Limited 2007. All rights reserved. Confidential and proprietary.

13 ways to grow bad debt

Credit process Target step population Customer acquisition

Description 1. Identifying potential customers; 2. Selling credit products to new customers; 3. Identifying the credit risk of the customer and the proposed transaction; 4. Identifying the risk of fraudulent application 5. Deciding whether to accept or decline the transaction; 6. Deciding, for accepted transactions, on the terms, e.g., credit amount, pricing. 7. Reviewing the customers facilities (e.g., credit limits, price, etc.); 8. Cross-selling new products to the customers; 9. Ensuring good customers are retained; 10. Identify fraudulent transactions. 11. Identifying self-cure customers; 12. Rehabilitation of potentially good customers; 13. Work-out customers where relationship is broken.
Experian Limited 2007. All rights reserved. Confidential and proprietary.

New customers

Customer management

Existing, nondelinquent customers Existing, delinquent customers

Collections

Why is credit risk management important?

2,986
European consumer finance business, Profit Before Tax and Impairment Charges ($m)

Get it right and it can support phenomenal value creation

1,522 1,520 1,382 1,230 1,094 802 676 478 288 340

2,196

1,374

Profits 836

924 740 804 764

Impairment charges

1998

1999

2000

2001

2002

2003

2004

2005

2006

Experian Limited 2007. All rights reserved. Confidential and proprietary.

Source: Annual Reports

5 core components
Component Data Description Application data (for new customers) Account behaviour data (for existing customers) External data (e.g., credit bureaux) Risk models (PD, LGD), fraud models (application and transaction fraud) and revenue models Business rules that translate the outcome of statistical models in credit decisions (accept/decline, price, credit limits, etc.) that maximise profit Software tools to automate the calculation of the above scores and credit strategies on-line on high volumes, with a high degree of flexibility to change credit strategies on the fly Software tools to evaluate the performance of statistical models and credit strategies, and accuracy of implementation
Experian Limited 2007. All rights reserved. Confidential and proprietary.

Statistical Models

Credit strategies

Implementation tools

Evaluation tools

Agenda
Some basic themes Analytics and the customer life cycle The role of scoring Building a scorecard Using scoring systems Risk management infrastructure

Experian Limited 2007. All rights reserved. Confidential and proprietary.

Analytics and the customer life cycle

Solicitation

Application

Customer management

Collections

Debt recovery

Population Information
Analytics touches every part of the customer lifecycle

Analytics touches every part of the customer life cycle Amount of information about the customer grows as the relationship advances through the customer life cycle
Experian Limited 2007. All rights reserved. Confidential and proprietary.

Analytics and the customer life cycle

Solicitation
Channel preference Contact history Demographics Location Bureau data Action outcomes Costs

Application
Channel Product holdings Demographics Bureau data Previous relationships Account performance Costs

Customer management
Product holdings Usage Delinquency Customer contacts Preferences Bureau data Actions taken Action outcomes Costs

Collections
Action history Promises to pay Promises fulfilled Action outcomes Bureau data Costs

Debt recovery
Action history Promises to pay Bureau data Agents used Promises fulfilled Litigation outcomes Costs

Experian Limited 2007. All rights reserved. Confidential and proprietary.

Analytics and the customer life cycle

Define Goals Agree objectives Understand results Assess Strategy Review Monitor Track progress against expectations Implement Design Build new strategy

Plan Review

Assess current challenger

Ensure operational deployment

Experian Limited 2007. All rights reserved. Confidential and proprietary.

10

Agenda
Some basic themes Analytics and the customer life cycle The role of scoring Building a scorecard Using scoring systems Risk management infrastructure

Experian Limited 2007. All rights reserved. Confidential and proprietary.

11

The role of scoring

Credit scoring is a technique for predicting the future This prediction can be anything of importance to the business
x x x x x x x

Arrears Fraud Profit Response Account closure Company failure Etc.

All scoring is based on one key assumption:

x

The past predicts the future

Experian Limited 2007. All rights reserved. Confidential and proprietary.

12

The role of scoring

How does scoring work?

Example Scorecard
Baseline Constant 800

Scorecards add and subtract points to a baseline constant according to individuals or accounts data Scorecards are easy to apply and simple to understand The resulting score gives a prediction of future behaviour Scores are used to rank individuals to assign the best actions

Applicant Age in Years < 22 22 - 25 26 40 41 55 > 55 -50 -20 0 +30 0

Worst Status L6M (on all Accounts) 0 1-2 3+ 0 -45 -100

Joint Applicant Present Y N +20 0 Etc.

Etc.

Experian Limited 2007. All rights reserved. Confidential and proprietary.

13

The role of scoring application scorecard

Consider a scorecard built to predict whether a new applicant for a credit product will default in the next 12 months This scorecard is used when a new customer applies

Application Form Data

Scorecard

Score

External Data (Bureau etc.)

Take most appropriate action for each individual

Experian Limited 2007. All rights reserved. Confidential and proprietary.

14

The role of scoring - scores can drive actions

Low Score / High Risk

Proportion of Applicants

A pplication Score

High Score / Low Risk Extremely Low Risk

High Risk Extremely High Risk Reject Reject or price to cover the high expected loss

Standard Risk Accept on standard terms

Consider for crosssell of other products

15

Experian Limited 2007. All rights reserved. Confidential and proprietary.

The role of scoring - benefits

Best use of data Objective Consistent Automation Control Reduced losses

Experian Limited 2007. All rights reserved. Confidential and proprietary.

16

Agenda
Some basic themes Analytics and the customer life cycle The role of scoring Building a scorecard Using scoring systems Risk management infrastructure

Experian Limited 2007. All rights reserved. Confidential and proprietary.

17

Building a scorecard 3 requirements

Development sample the historical data on which the scorecard will be built Outcome what we are trying to predict Modelling methodology the statistical tool that will help us form our scoring model

THEN

The recent past

Some time later

Development Sample Statistical Model

Outcome
Scorecard

Experian Limited 2007. All rights reserved. Confidential and proprietary.

NOW

18

Is my sample any good?

x x x

Products Business cycle The future

The recent past

Robust
x

Development Sample

Volumes

Mature Is the outcome reliable?

Experian Limited 2007. All rights reserved. Confidential and proprietary.

THEN
19

Representative

Building a scorecard the development sample

This data can come from a number of sources All relevant data should go into the development sample

The recent past

Development Sample
Application Form Credit Bureau Data Information on the individuals other credit commitments Historical Account Behaviour Information on the historical behaviour on the account Other Account Information Information on the historical behaviour on other accounts with the same lender

Information collected from the applicant at the application point

Experian Limited 2007. All rights reserved. Confidential and proprietary.

20

THEN

Building a scorecard the outcome

This is the behaviour that we are trying to predict

Can be a continuous variable (profit, revenue, loss given default, etc.) More commonly it is dichotomous - yes/no
x x x x

Will this applicant default? Is this transaction fraudulent? Will this company fail? Etc.

Outcome

Good
THE FUTURE

Bad
Observation - Now
Experian Limited 2007. All rights reserved. Confidential and proprietary.

Outcome - Prediction
21

NOW

What are we trying to predict?

Bad

Good

Consumer Limited business Non-limited business

3 payments in arrears Failed

Not 3 payments in arrears Still going

Outcome

Bankruptcy, court No bankruptcy, judgements or court judgements or defaults defaults

Experian Limited 2007. All rights reserved. Confidential and proprietary.

NOW
22

Building a scorecard the statistical model

Many statistical tools available Data is the most important factor

Statistical Model

Observation Data

Outcome

Statistical Model

Scorecard

Statistical tool needs to be: Powerful to get the best prediction from the data Flexible can handle varying data types and outcomes Interpretable easy to understand and to overlay business intelligence Transparent should be non-black box for regulatory reasons and to ensure understanding
Experian Limited 2007. All rights reserved. Confidential and proprietary.

23

Building a scorecard - the statistical model

Linear regression
Reality

Statistical Model

Logistic regression Artificial neural networks Etc Other things being equal the choice of algorithm has relatively little impact on the ultimate power of the model

x x x x x x x x x x x x x x x x x x x x x
Prediction

Experian Limited 2007. All rights reserved. Confidential and proprietary.

24

Building a scorecard assessing the model

Does the model solve the business problem?

Statistical Model

Discrimination the power to polarise individuals between good and bad - Gini statistic & Kolmogorov-Smirnov statistic Accuracy how much of the variability of the outcome is explained by the model Validation ensures that over-modelling has not occurred or that an anomalous sample has not been used Improvement the new model should outperforms the existing model

Experian Limited 2007. All rights reserved. Confidential and proprietary.

25

Agenda
Some basic themes Analytics and the customer life cycle The role of scoring Building a scorecard Using scoring systems Risk management infrastructure

Experian Limited 2007. All rights reserved. Confidential and proprietary.

26

Using scoring systems

DATA

SCORE CARD

DECISIONS

STRATEGY

The data feeds the scoring system, which is used to aid the decisioning The decisions a company makes determine its strategy It is the aims and strategy of the business that must be considered when deciding how to use a scoring system, e.g.

Growing the market share Reducing bad debt Increasing automation Maximising response for given marketing cost Combating fraud
Experian Limited 2007. All rights reserved. Confidential and proprietary.

27

The role of scoring - scores can drive actions

Low Score / High Risk

Proportion of Applicants

A pplication Score

High Score / Low Risk Extremely Low Risk

High Risk Extremely High Risk Reject Reject or price to cover the high expected loss

Standard Risk Accept on standard terms

Consider for crosssell of other products

28

Experian Limited 2007. All rights reserved. Confidential and proprietary.

Using scoring systems - the score distribution

Score Band 400 401 550 551 650 651 700 701 750 751 - 800 801 850 851 900 901 950 951 TOTAL

# Goods 500 700 815 1008 976 950 1000 1050 960 1000 8959

# Bads 500 350 163 84 61 38 25 21 16 10 1268

GB Odds 1 2 5 12 16 25 40 50 60 100 7.1

% Applicants 9.8 10.3 9.6 10.7 10.1 9.7 10.0 10.5 9.5 9.9 100

Score distribution is obtained by applying the score to the development sample Gives us a prediction for new applicants falling into a given score range
Experian Limited 2007. All rights reserved. Confidential and proprietary.

29

Building a scorecard - the score distribution

Score Band 400 401 550 551 650 651 700 701 750 751 - 800 801 850 851 900 901 950 951 TOTAL

# Goods 500 700 815 1008 976 950 1000 1050 960 1000 8959

# Bads 500 350 163 84 61 38 25 21 16 10 1268

GB Odds 1 2 5 12 16 25 40 50 60 100 7.1

% Applicants 9.8 10.3 9.6 10.7 10.1 9.7 10.0 10.5 9.5 9.9 100 ACCEPT WITH XSELL ACCEPT REFER REJECT

Score + Policy Rules + Terms of Business = Strategy

Experian Limited 2007. All rights reserved. Confidential and proprietary.

30

Agenda
Some basic themes Analytics and the customer life cycle The role of scoring Building a scorecard Using scoring systems Risk management infrastructure

Experian Limited 2007. All rights reserved. Confidential and proprietary.

31

Implementation the Business Rules Engine Data

Rules execution
(Decision Agent)

Rules Definition
(Strategy Design Studio)

Results
Experian Limited 2007. All rights reserved. Confidential and proprietary.

Deployed in: Origination Application processing Portfolio Management Customer level decisioning Collections Authorisations Intelligent Messaging Event Management Basel II Stress testing ..
32

The unsecured lending origination process

A full range of client options and interfaces for channel independence and data accuracy Online links to gather data about existing relationships and customer behaviour

Gather & validate application data

Invoke enrichment strategy

Credit bureau links

Gather existing customer information

Get decision & terms of business

Business-driven scoring & decisionmaking Comprehensive workflow capabilities and provision of relevant data for users

Detect application fraud

Application screening and data matching

Handle referrals and manual procedures

Get policy decision & enrichment strategy

Business-driven scoring and decisionmaking

Implement final decision

Automated account set-up. Provision of hand-off files. Letter and e-mail production

Experian Limited 2007. All rights reserved. Confidential and proprietary.

33

Implements

Business logic, Segmentation, Scorecards, Strategies and Champion Challenger

Defines

Business logic, Segmentation, Scorecards, Strategies and Champion Challenger Analytical ResultsData Mart

HOS T
Management System, Authorisation System etc

Active History

Extract

FeedbackVariables e.g. Account

Decision Engine

Evaluation Optimisation Reporting

Strategy Implementation

Rule Definition

Data Manager
Experian Limited 2007. All rights reserved. Confidential and proprietary.

Operational environment

Strategic business34 environment

Beyond scoring - strategy optimisation

There are disadvantages to traditional champion/challenger testing The time frame for observing results can be long It can be hard to design the next step The result can become a semi-random walk...
Value We want to get there with the first challenger !

Challeng er n

The challenger strategy proven in one time period, may no longer be appropriate for another time period things change
Champ ion

Decision strategy deploy-learndeploy process

Challeng er 1

Using performance data enables better decisions, but is also more complex to combine all the decision influences to maximise value

Challeng er 3 Challeng er 4

Challeng er 2

Time
Experian Limited 2007. All rights reserved. Confidential and proprietary.

Influence due to: Macro-economics? Use of intuition? Misunderstanding?

35

#35

Developments in analytics - strategy optimisation

The next step

Incremental benefit ROI Some organisations are still here

Most are here

Optimised Strategies

Elaborate Strategies

Scoring
Manual
X X X X X X X XX X X XX X X X X X X XX X XXX XX X XXX X XXX X X XXX X X X X

Allocates optimal action for each customer within constraints Objective, mathematical goal maximisation

single predictive model e.g. credit risk score Heuristic cut-offs assigned using good:bad odds

Segmentation based on predictive model dimensions: e.g. risk and revenue Subjective judgment used to manage trade-offs

Experience and intuition Trial and error

Underlying decision complexity

Experian Limited 2007. All rights reserved. Confidential and proprietary.

36

#36

Stage 1: build the infrastructure

Centralisation of credit decisioning Set-up of IT tools required to automate credit risk and market management processes and the interaction between front line and back office Development of decision support tools Development of credit / marketing databases
Automate the processes

Experian Limited 2007. All rights reserved. Confidential and proprietary.

37

Stage 2: fine-tune for performance

Fine customer segmentation based on customer profile, product holding and behavior data Advanced credit and marketing databases drive increased sophistication in statistical models development Customer interactions for risk and marketing are proactively initiated at all key points Strategies are designed at customer-level Automate the decisions

Experian Limited 2007. All rights reserved. Confidential and proprietary.

38

Stage 3: optimize for excellence

Infrastructure enables total proactive control of the business decision analytics becomes a way of life Risk and marketing strategies are centrally designed based on advanced statistical techniques and drive customer profitability Decision analytics is well structured and integrated across business functions including risk, marketing, sales, operations, finance Optimize the decisions

Experian Limited 2007. All rights reserved. Confidential and proprietary.

39

The Road Map

Build the infrastructure
Strategy Credit policy in place

Fine-tune for performance

Profit driven credit strategy in place and reviewed regularly

Optimize for excellence

Monthly review of credit strategies. Champion/challenger a way of life Processes fully support profit-driven strategy, and are integrated across functions Ongoing knowledge improvement Full suite of scorecards, ability to optimize credit strategies Ability to review and modify credit strategies on the fly a world-class consumer finance company

How are credit policies and strategies defined, reviewed and improved? Processes well defined and automated

Processes regularly reviewed and refined. Little manual intervention

Processes

How well defined and the processes, and what is the degree of automation?

Include all available data into the process. Focus underwriter on key review, not second scorecard

Education on strategy review process, fully understanding the use of MIS

Profit-driven organisation across functions

Create strategy review cross-functional team

How well do staff understand all profit drivers? What is the degree of expertise in credit scoring and decision science?

Knowledge

Tools

Scorecards in place for all critical segments, decision engine used to Ensure clear assignment of control terms of responsibilities for risk business. Generate management functions key KPIs What credit management tools are used? How How are credit risk, flexible are they? How marketing and finance easy is it for business user working together? How to change processes and Organisation are operational and strategies? strategic decisions taken?
Experian Limited 2007. All rights reserved. Confidential and proprietary.

40

Conclusions

It all starts with data Scorecards are important Strategy is more important Implementing the strategy properly is vital If you dont monitor youre wasting you time Risk management is a never-ending journey
Experian Limited 2007. All rights reserved. Confidential and proprietary.

41

Best practice in data & scoring

Dr Paul Russell Director Analytical Solutions

Experian Limited 2007. All rights reserved. Experian and the marks used herein are service marks or registered trademarks of Experian Limited. Other product and company names mentioned herein may be the trademarks of their respective owners. No part of this copyrighted work may be reproduced, modified, or distributed in any form or manner without the prior written permission of Experian Limited. Confidential and proprietary.