Lecture 5 Binary Choice Models

CIE 555: Discrete Choice Analysis

Instructor: Qian Wang

CIE 555: Discrete Choice Analysis L05

02/05&02/10, 2015

Outline
Binary Choices

Binary Choice Models

Probability of A Choice
Estimation of Binary Choice Models

An Example of Binary Choices

EZ-Pass payment methods
Choice problem: choose toll payment methods

given the time of day pricing

Decision makers: travelers
Choice alternatives
EZ-Pass vs. Cash

Choice rule: utility

maximization

Binary Choices
Choice set: Cn={alter. 1, alter. 2}
Choice tree

Travel decision

Alter. 1

Alter. 2

Examples
Travel decisions: travel or not
Payment types: paying tolls by E-ZPass or cash
4

Binary Choice Models

Utility function
Systematic utility
Alter. 1:

U1n V1n 1n
Error

Alter. 2:

U 2n V2 n 2 n

Systematic component
Alter. 1:
Alter. 2:

V1n 0 1x11n 2 x12 n ... K x1Kn

V2n

1 ' x21n 2 ' x22 n ... K ' x2 Kn

Where: k , k ' = the coefficient of independent variable k;

xikn = independent variable k for alternative i perceived by
decision maker n.

Notations of the Parameters

Conceptually: the parameters should vary by

alternatives and decision makers

Ways to deal with the variations of parameters:
Regarding the variations by alternatives:

Try the same parameters at first; if the variable prove to

be significant, vary the parameters by alternatives
Regarding the variations by decision makers:
Split the whole population to several homogeneous
groups, and then specify the choice models for each
group; OR
Treat the parameters as random variables that vary
across individuals (random parameter utility functions)

Error Terms
We assume the distributions of the error terms in

order to calculate the probability of a choice

Probability of A Binary Choice

The probability of choosing alternative i (i=1 or 2)

P(i | Cn ) Pr[Vin in V jn jn , j {1,2}]

Pr[ jn in Vin V jn , j {1,2}]
Let:

n jn in

We get:

P(i | Cn ) Pr[ n Vin V jn , j {1,2}]

8

Its distribution is the key to

calculate the probability

Linear Probability Model

Assuming the difference of errors follows an

uniform distribution
f ( n ) : Density function of the error
difference

0, n L or n L

f ( n ) 1
,
L n L

2L

-L

1
2L

n jn in

Linear Probability Model (Cont.)

The probability of choosing alter. i:
Vin V jn

Pn (i )

0, Vin V jn L

V V L
in
jn
f ( n )d n
, L Vin V jn L
2
L

1, Vin V jn L

pn (i) : Probability of choosing alter. i

1

0.5

450
-L
10

Vin V jn

Linear Probability Model (Cont.)

The value of L:
pn (i )
Let:

Vin V jn L
2L

Vin V jn L,

L Vin V jn L

1 1

We get:

pn (i )

L L
2L

1
,
2

1 1

The probability only depends on the ratio of the

utility difference to L but not L

We can arbitrarily set L=1/2
11

Binary Probit Model

In reality, few cases match the uniform

distributions
Since the error terms represent the combination
of various sources of randomness, by the central
limit theorem, their distributions would tend to be
normal
Note: central limit theorem states that the reaveraged sum of a sufficiently large number of
identically distributed independent random
variables, each with finite mean and variance, will
be approximately normally distributed
12

Binary Probit Model (Cont.)

Assuming the difference of errors follows an

normal distribution with mean as zero and

variance as 2

f ( n ) (Density function)

1
1
f ( n )
exp[ n ]
2
2
2

0
13

Binary Probit Model

The probability of choosing alter. i:
Vin V jn

Pn (i )

f ( n )d n

1
1

exp[ ]d n

2
2
1 (Vin V jn ) /
1 2

exp[ ]d

2
2
Vin V jn
(
)
2

Vin V jn

Standardized cumulative normal

distribution function
14

Binary Probit Model

The probability curve
pn (i) : Probability of choosing alter. i
1

0.5

Vin V jn

:
We choose 1

The value of
15

Binary Logit Model

Assuming the difference of errors follows an

logistic distribution
It is probitlike: it resembles the normal

distribution in shape but has heavier tails

It is analytically convenient

16

Binary Logit Model (Cont.)

The probability density function
f ( n )

e
(1 e

n 2

, 0, n

Where:
= a positive scale
parameter
s = set as 1 for the binary
logit case

17

Binary Logit Model (Cont.)

The probability of choosing alter. i:
Vin V jn

Pn (i )

f ( n )d n

Vin V jn

(1 e
1

n 2

(V V )

1 e in jn
e Vin
V
V
e in e jn
18

d n

Binary Logit Model (Cont.)

The probability curve
pn (i) : Probability of choosing alter. i
1

0.5

Vin V jn

19

Binary Logit Model (Cont.)

Replace V by the utility functions
Pn (i )

1
1 e

(Vin V jn )

1
1 e

( X in X jn )

This indicates that the scale parameter ( )

cannot be distinguished from the overall scales of

parameters ( )
For convenience, we assume that 1
2
Furthermore, this implies that var( n ) / 3
20

Extreme Cases of the Linear, Probit

and Logit Models
pn (i) : Probability of choosing alter. i
1

, 0, L 0

0.5

0, , L

Vin V jn

21

Other Binary Choice Models

Arctan probability model

Right-truncated exponential model

Left-truncated exponential model

Reference: Section 4.2 of the textbook (page 73)

22

Estimation of Binary Choice Models

Model estimation procedure
Calibration:
Specifying utility functional
forms

Estimating parameters

Validation

Application (Forecasting)
23

Methods to Estimate Parameters

Maximum likelihood estimation
Likelihood: the probability that the whole

population/sample of decision makers make certain

choices
Methodology: find the best parameter values that
maximize the logarithm likelihood

24

Maximum Likelihood Estimation

Likelihood function:
N

L(1 , 2, ..., K ) pni

yni=1 if decision maker n

chooses alternative i; 0,
otherwise

yni

n 1 i 1

Probability for decision

maker n to choose
alternative i

The log likelihood becomes

N

LL( 1 , 2, ..., K ) yni ln( pni )

n 1 i 1

ln( p

nN ( i )

25

ni

The group of decision makers

who chose alternative i

Maximum Likelihood Estimation

Step 2: Find the optimal solutions of the

parameters by maximizing the likelihood function

max

LL( 1 , 2, ..., K )

Given unknown variables:

0 , 1 ,..., K

Step 2.1: Find the POTENTIAL optimal solutions by

solving the first-order conditions (first-order

derivatives):
p (i) / k
LL N 2
yin n
0, k 0,...,K
k n1 i 1
pn (i)
26

Maximum Likelihood Estimation

Step 2.2: Pick the optimal solution by checking the

Hessian matrix of the likelihood function (secondorder condition)

2 LL
Hessian [
]
k l
If it is a negative semidefinite matrix: the likelihood function

is concave with a UNIQUE optimal solution

If not: calculate the likelihood value for each solution from
the first-order condition, and choose the solution resulting in
the maximum value of likelihood

27

Example: Estimation of Binary

Logit Models
In the binary Logit Models, the probabilities are:

pn (i)

' X in

e ' X in e

' X jn

, pn ( j )

' X jn

e ' X in e

' X jn

Step 1: the log likelihood function

LL( 1 , 2, ..., K )
N

[ yin log(
n 1

28

e
e

' X in

' X in

) (1 yin ) log(
' X jn

e
e

' X in

' X jn

' X jn

)]

Example: Estimation of Binary

Logit Models (Cont.)
Step 2: Maximum likelihood problem

max

LL( 1 , 2, ..., K )

Given unknown variables:

0 , 1 ,..., K

Step 2.1: Find the POTENTIAL optimal solutions by

solving the first-order conditions (first-order

derivatives):
LL N
[ yin pn (i)]xnk 0, k 1,...,K
k n1

29

Example: Estimation of Binary

Logit Models (Cont.)
Step 2.2: Pick the optimal solution by checking the

Hessian matrix of the likelihood function (secondorder derivatives)

N
2 LL
pn (i )(1 pn (i)) xnk xnl
k l
n 1

The Hessian matrix is negative semidefinite (check page 85

in the textbook to see why) the solution from the firstorder condition is the ONLY optimal solution

30

Properties Derived from the FirstOrder Conditions

For the derivatives corresponding to the constant

term:

=1

LL N
[ yin pn (i)]xn 0 0
0 n1
Which is equivalent to:
N

[ y

in

n 1

pn (i)] 0

n 1

n 1

yin pn (i ), i {alter.1, alter.2} Property 1

Where: N= the total number of the observations in the full sample
31

Properties Derived from the FirstOrder Conditions (Cont.)

For the derivatives corresponding to the

alternative-specific dummy variables:

LL N
[ yin pn (i)]xnk 0, k {K ' '}
k n 1

1
xnk
0

For a subset of the sample whose xnk equal 1

For the remainder of the sample

Which is equivalent to:

N'

[ y
n 1

in

N'

pn (i)] 0, k {K ' '}

N'

yin pn (i )
n 1

32

Property 2

n 1

Where: N= the number of observations in the subset of the sample

Computational Aspects of
Maximum Likelihood
The parameters are estimated by solving a group

of NOLINEAR equations from the first-order

conditions
LL N
[ yin pn (i)]xnk 0, k 1,...,K
k n1
Nonlinear functions of the unknown parameters

Iterative procedures are required to obtain the

solutions, e.g., the Newton-Raphson algorithm

(check the page 82 in the textbook)
33

Learning Outcomes
What is a binary (binomial) choice situation

What are the three typical choice models to deal

with the situations, and how they are derived

based on what assumptions
Binary-uniform
Binary-logit
Binary-probit

Be able to apply the appropriate method to deal

with a real-world binary choice problem

34

References
Chapter 4 of the text book

Holgun-Veras, J., & Wang, Q. (2011). Behavioral

investigation on the factors that determine

adoption of an electronic toll collection system:
Freight carriers. Transportation research part C:
Emerging technologies, 19(4), 593-605.

35

Next Class
Lab 1: use LIMDEP to estimate binary choice

models
Bring your laptop with the installed software to the

class
Lab 1 assignment will be given

36