Psychophysics 3

Research Methods
Fall 2010
Tams Bhm
Signal detection theory
Aka. sensory decision theory (SDT)
A model & a data analysis method for decision
problems with uncertainty (noise)
Originates from World
War II: aircraft detection
on radar signals
Today: widely used in
psychophysics, medicine,
radiology and
machine learning
Signal detection theory
Experiment setup:
In some trials a stimulus (signal) is presented, in
others there is no stimulus;
Observer reports if she/he saw a signal or not
Calculate how many times the observer detected a
signal when she/he was presented one (hit rate)
Is the hit rate all we want to know?
Two observers achieved the same hit rate.
Are they certainly behaving the same way?
NO, we also need to know how many times the
observer said I see when there was no signal
(false alarm rate)
Signal detection theory
Confusion matrix: contains all the information about the
observers performance
Signal detection theory
Confusion matrix: contains all the information about the
observers performance
As columns add up
to 100%, it is enough
to know one item
from each column
40 trials
20 20
18
2
6
14
= 100% = 100%
= 90%
= 10% = 70%
= 30%
Signal detection theory
Perfect detection:
100%
100% 0%
0%
Signal detection theory
No detection at all (1st example): always
reporting Seen
100%
0% 0%
100%
Signal detection theory
No detection (2nd example): always
reporting Not seen
0%
100% 100%
0%
Signal detection theory
No detection (3rd example): flipping a coin
50%
50% 50%
50%
Signal detection theory
No detection (4th example): reporting Seen in
30% of the trials (no matter what is presented)
30%
70% 70%
30% =
=
Rows equal
no detection
Signal detection theory
Receiver operating characteristic (ROC):
false alarm rate
h
i
t

r
a
t
e

100%
100%
Signal detection theory
Receiver operating characteristic (ROC):
false alarm rate
h
i
t

r
a
t
e

100%
100%
90% 30%
10% 70%
Signal detection theory
Receiver operating characteristic (ROC):
false alarm rate
h
i
t

r
a
t
e

100%
100%
100% 0%
0% 100%
Perfect detection
Signal detection theory
Receiver operating characteristic (ROC):
false alarm rate
h
i
t

r
a
t
e

100%
100%
100% 100%
0% 0%
No detection:
always yes
Signal detection theory
Receiver operating characteristic (ROC):
false alarm rate
h
i
t

r
a
t
e

100%
100%
0% 0%
100% 100%
No detection:
always no
Signal detection theory
Receiver operating characteristic (ROC):
false alarm rate
h
i
t

r
a
t
e

100%
100%
50% 50%
50% 50%
No detection:
reporting yes in
50% of the trials
(flipping a coin)
Signal detection theory
Receiver operating characteristic (ROC):
false alarm rate
h
i
t

r
a
t
e

100%
100%
40% 40%
60% 60%
No detection:
reporting yes in
40% of the trials
Signal detection theory
Receiver operating characteristic (ROC):
false alarm rate
h
i
t

r
a
t
e

100%
100%
30% 30%
70% 70%
No detection:
reporting yes in
30% of the trials
Signal detection theory
Receiver operating characteristic (ROC):
false alarm rate
h
i
t

r
a
t
e

100%
100%
60% 60%
40% 40%
No detection:
reporting yes in
60% of the trials
Signal detection theory
Receiver operating characteristic (ROC):
false alarm rate
h
i
t

r
a
t
e

100%
100%
Diagonal: no detection
Signal detection theory
SDT model:





No way to remove the noise
But sensation can be separated from decision by
using ROCs
Sensation
Noise
Decision
Signal
present
/absent
Sensation
level (SL)
SL
Criterion ()
SL <
YES
NO
Signal detection theory
Sensation
(Noise)
Decision
Signal
present
/absent
Sensation
level (SL)
SL
Criterion ()
SL <
YES
NO
sensation level
p
r
o
b
a
b
i
l
i
t
y

Without noise:
perfect detection is
possible
criterion
signal
present
signal
absent
Signal detection theory
Sensation
(Noise)
Decision
Signal
present
/absent
Sensation
level (SL)
SL
Criterion ()
SL <
YES
NO
sensation level
p
r
o
b
a
b
i
l
i
t
y

criterion
signal
present
signal
absent
100% 0%
0% 100%
Signal detection theory
Sensation
Noise
Decision
Signal
present
/absent
Sensation
level (SL)
SL
Criterion ()
SL <
YES
NO
sensation level
p
r
o
b
a
b
i
l
i
t
y

Noise: smears the
distributions
perfect detection
is impossible
(if the two distributions
overlap)
signal absent
(noise only)
signal present
(signal+noise)
criterion
online demo
Signal detection theory
Sensation level
Sensation level
http://www-psych.stanford.edu/~lera/psych115s/notes/signal/
Signal detection theory
Sensation level
Sensation level
false alarm rate
h
i
t

r
a
t
e

Signal detection theory
false alarm rate
h
i
t

r
a
t
e

ROC
curve
= 8
= 6
= 10
= 6
= 8
= 10
Signal detection theory
false alarm rate
h
i
t

r
a
t
e


sensation level
p
r
o
b
a
b
i
l
i
t
y

Criterion (): specifies
where we are on the
ROC curve
The ROC curve is
specified by sensory
capacities only
(discriminability)
Signal detection theory
Discriminability:
how well the
observer can
separate the
presence of signal
from its absence
~ overlap between
the two distributions
~ bowing out of the
ROC curve
Measured by d
(discriminability index,
also called sensitivity)
http://www-psych.stanford.edu/~lera/psych115s/notes/signal/
Signal detection theory
d: selects the ROC curve
: specifies a point on the
selected ROC curve
same information as hit rate
& false alarm rate, but:
hit rate, false alarm rate:
both reflect sensation &
decision characteristics;
cannot separate the two
d: depends only on sensation
: depends only on decision

The two processes
are separated
http://psych.hanover.edu/JavaTest/Media/Chapter2/MedFig.ROC.html
Signal detection theory
Fechners methods:
Is a stimulus detectable?
Yes or no?
Clear-cut threshold value
(with some variability)
that can be measured
Stimulus intensity >
threshold detectable
Stimulus intensity <
threshold not detectable
Dichotic outcome,
categorical model
Signal detection theory:
How well is it detectable?
How sensitive the
observer is to the
stimulus?
Measured by d
The higher d is, the more
the stimulus is detectable
d = 0
not detectable at all
Scalar outcome,
dimensional model

Signal detection theory
Sensation
(Noise)
Stimulus
Sensation
level (SL)
Different
task
Correct Incorrect
Forced-choice:
eliminates the
criterion
SDT: separates
the criterion
Decision
SL
Criterion ()
SL <
YES
NO
Problem with Fechners methods: criterion
Signal detection theory
Psychophysical measurements with SDT:
1. Create a stimulus set with a range of intensities (like
in the method of constant stimuli)
2. Test each stimulus many times with each observer
3. On each trial, either present a randomly selected
stimulus or do not present anything
4. Ask the observer if he/she detected the stimulus
5. Calculate the hit rate and false alarm rate for each
observer, for each stimulus intensity
6. Use the formula/table to calculate d for each case
7. Examine how d changes with intensity: the higher
d is for a stimulus intensity, the greater the
observers ability to detect this intensity
http://psych.hanover.edu/JavaTest/Media/Chapter2/MedFig.SignalDetection.html
Signal detection theory
Main results: changes in d values
CaudekRubin Vision Res. 2001
Signal detection theory
There is also a value for each d value
It can be informative about the decision
behavior:
Balanced: false alarm and
miss rates are equal
Liberal: the observer
says yes whenever
there may be a signal
Conservative:
decision is yes only
when it is almost
certain that there is
a signal
sensation level
p
r
o
b
a
b
i
l
i
t
y

balanced
conservative
liberal