017 PDF

Chapter 17
Attention and Conditioned

Reinforcement
Copyright American Psychological Association. Not for further distribution.
Timothy A. Shahan
The ability to appropriately attend to the important

features of a complex environment is a critical survival skill. Problems related to the allocation and
persistence of attending are associated with various
psychological disorders including attention-deficit/
hyperactivity disorder (American Psychiatric Association, 1994), autism and other developmental disabilities (Bryson, Wainwright-Sharp, & Smith, 1990;
Dube & McIlvane, 1997; Lovaas, Koegel, & Schreibman, 1979), schizophrenia (Nestor & ODonnell,
1998), and substance abuse (Ehrman et al., 2002;
Johnsen, Laberg, Cox, Vaksdal, & Hugdahl, 1994;
Lubman, Peters, Mogg, Bradley, & Deakin, 2000;
Townshend & Duka, 2001). Although many factors
surely contribute to the allocation and persistence of
attending, the relation between patterns of attending
and the resultant consequences may play an important role. I begin this chapter with a consideration of
attention and its potential relation to existing
accounts of behavior maintained by its consequences
(i.e., operant, or instrumental, behavior). Next, I
review research on how differential consequences
affect the allocation and persistence of attending to
important environmental stimuli. Finally, I examine
the relation between attending to stimuli associated
with differential consequences and the traditional
concept of conditioned, or secondary, reinforcement
the notion that stimuli associated with reinforcers
themselves acquire the ability to reinforce behavior.
In considering the concept of conditioned reinforcement, I explore the utility of an alternative framework based on the notion that attending to
important stimuli is instrumental in acquiring
reinforcers (i.e., obtaining goals), rather than such

stimuli becoming reinforcers themselves.
Attention and Behavior
Attention
An appropriate starting point for any treatment of
attention is a consideration of how to define attention and circumscribe what is to be discussed.
Unfortunately, clear technical definitions of attention are difficult to find. Consider that two booklength treatments of the topic (Pashler, 1998; Styles,
1997) and a chapter on attention in Stevenss Handbook of Experimental Psychology (Luck & Vecera,
2002) provide no technical definition of the term.
Instead, all three sources note William Jamess
(1890) famous suggestion that everyone knows
what attention is. All three then move on to suggest
that, despite Jamess claim, it may be more appropriate to say that no one knows what attention is.
Instead, they suggest that the term attention almost
certainly refers to more than one psychological phenomenon. The phenomena typically considered to
fall under the heading of attention involve limitations in the capacity of cognitive functioning and
selectivity of perception.
Even if attention is not a unitary psychological
phenomenon, the rest of Jamess (1890) definition is
instructive as a sort of summary of the set of phenomena captured under the heading. James
suggested,
It is the taking possession by the mind, in
clear and vivid form, of one out of what
DOI: 10.1037/13937-017
APA Handbook of Behavior Analysis: Vol. 1. Methods and Principles, G. J. Madden (Editor-in-Chief)
Copyright 2013 by the American Psychological Association. All rights reserved.
387
Timothy A. Shahan
seem several simultaneously possible

objects or trains of thought. Focalization,
concentration of consciousness are of its
essence. It implies withdrawal from some
things in order to deal effectively with
others. (pp. 403404)
Clearly, defining attention in terms of the even
more poorly defined concept of consciousness is less
than ideal. In addition, despite Jamess (1890) assertion that attention involves the focusing of consciousness, it is becoming increasingly clear that
conscious awareness is not necessary for even apparently goal-related shifts in attention (see Dijksterhuis & Aarts, 2010, for review).
Nonetheless, Jamess (1890) definition does generally capture the properties of capacity limitation
and selectivity noted earlier. Jamess assertion that
people all know what attention is probably stems
from the common personal experience that only a
small fraction of the stimuli impinging on peoples
senses seem to affect them at any given time. Jamess
definition also suggests that attention appears to be
something that organisms do. In this chapter, I
explore how attention might be thought of in terms
similar to those used to describe other activities in
which organisms engage.
A large empirical literature (see Yantis, 2000, for
review) has suggested that changes in what an
organism is attending to can be driven automatically
by properties of stimuli themselves (e.g., salience,
novelty, abrupt onset, innate biological significance)
or by goal-directed control. The phenomenon of
goal-directed control of attention goes by many
names, including endogenous, top-down, and voluntary control. A simple demonstration of such control
is to keep your gaze fixed on a word in the center of
this page and then pay attention to something in the
periphery to your right and then to your left.
Another example is to listen to a piece of music and
then shift your attention to different instruments at
different timesfirst the drums, then the bass, and
so on. In theoretical treatments of goal-directed control, such changes in the allocation of attention have
generally been hypothesized to be under the control
of what has been called a central administrator (Kahneman, 1973), a central executive (Baddeley, 1986),
388
or a supervisory attentional system (Norman & Shallice, 1986). As noted by Styles (1997), cognitive
psychologists have widely recognized that providing
a name for an entity in charge of decision making
does not escape the homunculus problem and that
the names serve only as temporary placeholders for
a problem remaining to be solved.
Experimental examinations of goal-directed control of attention almost always use instructions to
direct participants how to allocate their attention to
particular stimulus features, dimensions, or spatial
locations. For example, subjects might be instructed
to attend to only the color or shape of an object or to
devote 80% of their attention to the color and 20%
to the shape (e.g., Bonnel & Prinzmetal, 1998). As
another example, subjects in a cued detection task
might be asked to keep their gaze fixed at a central
location on a computer screen and then to report
targets presented to the left or right of the fixation
point. Arrows presented at the fixation location and
pointing to the left or the right instruct participants
where in space to allocate their attention, but on
some percentage of the trials the arrow is inaccurate
(see Posner, 1980). Detection accuracy or reaction
time data from such procedures are consistent with
what would be expected if subjects allocate their
attention as instructed (e.g., greater accuracy with
attended dimension, shorter reaction times with
accurate instruction).
Instructions about what to attend to might be
thought of as a way to change what Norman (1968)
called the pertinence of a stimulus. Norman suggested
that in addition to the physical aspects of a stimulus
(i.e., stimulus activation), the importance of a stimulus to the organism (i.e., pertinence) is involved in
the control of attention. The pertinence of a stimulus reflects the current goals of the organism and its
history with a stimulus. For example, your spoken
name is a stimulus that likely commands attention,
not because of the physical properties of the stimulus per se but because of your history with it.
Instructing subjects about how to allocate their
attention can be seen as a way of creating languagebased goals or quickly providing a relevant history
with respect to a stimulus. Although such instructional control of attending is important in demonstrating that attending can be goal directed, it does
Attention and Conditioned Reinforcement
not document what factors contribute to the pertinence of stimuli or the goal-directed control of
attending in the natural environment. Nonetheless,
the study of learning and behavior (i.e., conditioning
and learning) provides a reasonably well-developed
framework for understanding how experience
changes the importance of stimuli and how consequences affect the allocation and persistence of goaldirected behavior. This framework might also be
useful for understanding the control of attention.
Behavior
Although some may find it distressing that a wellarticulated definition of attention was not forthcoming
in the preceding section, one could argue that providing a definition of behavior is similarly problematic.
Consider the treatment of the terms learning and behavior in Catanias (1998) influential textbook, Learning:
This book is about learning, but from
the start we have to face the fact that we
wont be able to define it. There are no
satisfactory definitions of learning. Still,
we can study it. (p. 2)
Behavior is no easier to define than learning. We may say glibly that behavior
is anything an organism does, but this
definition is too global.... Lets not try
to resolve this problem. Our aim is to
examine some properties of behavior.
Although they sometimes share common names, the phenomena of behavior
are varied, so well probably do better by
considering examples than by attempting
definitions. (p. 7)
I hope the parallels between my treatment of
attention and Catanias (1998) treatment of learning
and behavior are obvious. As with attention, behavior may be difficult to define, but researchers have
nevertheless made progress in understanding some
basic principles of the phenomena captured under
that heading.
In the study of learning and behavior, it is customary to distinguish between classical (i.e., Pavlovian,
respondent) and operant (i.e., instrumental) conditioning. Classical conditioning involves learning the
relation between two environmental stimuli (see

Chapter 13, this volume). For example, if a tone (i.e.,
conditioned stimulus [CS]) reliably predicts food
delivery (i.e., unconditioned stimulus [US]) that
already elicits salivation (i.e., unconditioned response
[UR]), then the tone itself will come to elicit salivation (i.e., conditioned response [CR]). A large empirical literature since the time of Pavlov has provided a
wealth of information about how organisms learn the
predictive relations between stimuli and how initially
neutral stimuli acquire the ability to affect behavior
(see Mackintosh, 1974, and Rescorla, 1988, for
reviews). Such stimulusstimulus learning may be
contrasted with operant (i.e., instrumental) conditioning, which involves learning the relation between
a response and the consequences of that response.
For example, a hungry rat may learn to press a lever
that produces food or a child may learn to ask nicely
for a toy. In both cases, the behavior is instrumental
in the production of the consequence, and the future
probability of the behavior is changed as a result.
Consequences that increase the probability of the
behavior are called reinforcers and those that decrease
the probability of the behavior are called punishers.
Operant behavior roughly corresponds to what is
usually called voluntary or goal-directed behavior. A
vast literature on operant conditioning with humans
and nonhumans has demonstrated that variations in
such behavior are lawfully related to the type and
arrangement of consequences obtained (see Mazur,
2006, for review).
The fundamental unit of operant behavior is
widely believed to be the discriminated operant,
embodied by the three-term contingency between
(a) the discriminative stimulus context in which the
behavior occurs, (b) the behavior itself, and (c) the
consequence of the response. Schematically, the
three-term contingency is often described as SD:
B SR, where SD is a discriminative stimulus, B is
the behavior, and SR is a reinforcer for that behavior.
An SD comes to modulate the occurrence of the
behavior, or to set the occasion for the behavior, as a
result of the consequences experienced for the
behavior in its presence. For example, if the lever
pressing of a rat produces food when a light is on,
but not when it is off, lever pressing will come to
occur predominately in the presence of the light.
389
Timothy A. Shahan
Although distinguishing between classical and

operant conditioning may be customary, the two
types of learning clearly interact in most instances of
behavior. For example, not only do organisms learn
that an SD signals the contingent relation between
the behavior and the reinforcer, they also learn the
Pavlovian stimulusstimulus relation between the SD
and the reinforcer (see Rescorla, 1998, for a review).
Such stimulusstimulus learning within the discriminated operant plays an important role in the motivation and persistence of operant behavior (e.g.,
Nevin & Grace, 2000), a topic I return to later.
Another way in which classical and operant conditioning interact is through the process of conditioned (i.e., secondary) reinforcement. Stimuli
associated with the delivery of reinforcers for operant behavior are commonly believed to become reinforcers themselves as a result of the Pavlovian
relation between the stimulus and the existing reinforcer. For example, a click associated with the
delivery of a food reinforcer for a rat may also
appear to serve as a reinforcer for the lever pressing
that produces it (see Williams, 1994, for review).
Although both classical and operant conditioning
play a role in the treatment of attention in this chapter, I focus largely on operant conditioning because
my goal is to explore how consequences affect the
allocation and persistence of attending. Before proceeding to a discussion of how consequences affect
attending, I briefly review the matching law and
behavioral momentum theory, two influential theories of operant conditioning that play a critical role
in the discussion.
Operant behavior as choice: The matching law.
Much of the contemporary study of operant conditioning focuses on how differential consequences
affect the allocation of behavior (i.e., choice). This
focus on choice is largely the result of the influence of Herrnsteins (1961, 1970) matching law.
Herrnstein (1961) varied the rate at which two
concurrently available operant responses provided
food reinforcement for the key pecking of pigeons
in a standard operant conditioning preparation. He
found that the proportion of responses allocated to
one of two options roughly equaled (i.e., matched)
the proportion of reinforcers obtained from that
390
option. Herrnstein formalized this relation with the

matching law, which states that
B1
R1
=
,
B1 + B2 R1 + R 2
(1)
where B1 and B2 refer to behavior allocated to the

two options and R1 and R2 refer to the reinforcers
obtained by responding at those options (for a
primer on quantitative models of behavior, see
Chapter 10, this volume). Thus, when organisms are
confronted with a choice situation, they allocate
their behavior to the available options in proportion
to the reinforcers obtained from those options.
Later, Herrnstein (1970) extended the matching law
to behavior occurring in situations in which only
one response is explicitly defined. Although only a
single response may be defined, the organism could
also engage in other unmeasured behavior that may
provide other unmeasured sources of reinforcement.
This formulation has come to be known as Herrnsteins hyperbola, or the quantitative law of effect.
The quantitative law of effect states that
B=
kR
,
R + Re
(2)
where B is the rate of the measured behavior, R is the

rate of reinforcement for the measured behavior, the
parameter k defines asymptotic response rates when
all reinforcement is for the measured response, and
the parameter Re represents alternative sources of
reinforcement (i.e., reinforcement for all other
behavior). Thus, the frequency of an operant behavior is described as being determined by the rate of
reinforcement for that behavior in the context of
reinforcement for all other behavior. Response rates
increase with increases in reinforcement rate for the
measured behavior (i.e., R), but the degree to which
response rates increase with increases in reinforcement rate is determined by the availability of alternative reinforcers (i.e., Re). Within the framework
provided by the quantitative law of effect, all operant
behavior is choice and is governed by the relative
frequency of reinforcement obtained for the different
activities in which an organism might engage.
Baum (1974, 1979) extended the matching law
to account for common deviations from Equation 1.
Although Equation 1 suggests that the distribution
of behavior to two options strictly matches the distribution of reinforcers, real organisms are often less
than perfectly sensitive to the distribution of reinforcers. The generalized matching law suggests that
the ratio of behavior allocated to two options is a
power function of the ratio of reinforcers obtained at
the options. Thus,
a
R
B1
= b 1 ,
B2
R2
(3)
where B1 and B2 refer to behaviors 1 and 2, respectively, and R1 and R2 refer to reinforcers for B1 and
B2. The parameter b represents bias for one option or
the other unrelated to variations in relative reinforcement rate. The parameter a represents sensitivity of the allocation of behavior to changes in
reinforcement ratios. Values of a less than 1 reflect
relative insensitivity of the allocation of behavior to
the ratio of reinforcers (i.e., undermatching),
whereas values of a greater than 1 reflect hypersensitivity (i.e., overmatching). Equation 3 can also be
extended to more than two choice options (see Schneider & Davison, 2005), although the details of
such an extension are not of concern here.
Although Equation 3 suggests that choice is a function of relative reinforcement rate, the reinforcers
available for different options might differ with respect
to any number of parameters (e.g., amount, delay to
receipt, quality). Accordingly, Baum and Rachlin
(1969) proposed the concatenated matching law,
which suggests in its generalized matching law version
that choice is dependent on the multiplicative effects
of different reinforcement parameters, such that
a1
a2
a3
a4
R A 1 / d1 q1
B1
= b 1 1
, (4)
B2
R 2 A 2 1 / d 2 q2
with terms as in Equation 3 and additional terms for
relative reinforcement amount (A1 and A2), immediacy (1/d1 and 1/d2), quality (q1 and q2), and their
respective sensitivity parameters a2, a3, and a4. The
overall impact of a variety of parameters of reinforcement on choice can also be collapsed into a central
intervening variable called value (i.e., V) such that
a
B1 V1
.
=
B2 V2
(5)
Thus, in its most general form, the matching law

suggests that the allocation of behavior to the available choice options is governed by the relative value
of the reinforcers associated with those options.
The matching law has provided a quantitative
framework for describing how differential consequences govern the allocation of behavior and has
provided insights into how the value of reinforcers
and stimuli associated with reinforcers might be calculated. Although the matching law was developed
largely with data generated in operant conditioning
preparations with nonhumans, it has since been
shown to have widespread applicability to broader
issues in decision making, especially the apparently
irrational, suboptimal, or impulsive decision making
of humans (see Herrnstein, 1990; Herrnstein, Rachlin, & Laibson, 1997; and Madden & Bickel, 2010,
for reviews). Furthermore, the matching law and its
derivatives have provided a useful framework within
which to conduct analyses of the neuroscience of
decision making and reward valuation (e.g., Loewenstein & Seung, 2006; McClure, Laibson, Loewenstein, & Cohen, 2004; Sugrue, Corrado, &
Newsome, 2004). Perhaps the matching law may
also provide a useful framework for characterizing
how differential consequences affect decisions about
the allocation of attention.
Persistence of operant behavior: Behavioral momentum theory. Behavioral momentum theory (e.g.,
Nevin & Grace, 2000) addresses how the conditions
of reinforcement under which an operant behavior is
trained affect the persistence of that behavior when
it is disrupted. The theory suggests that there are
two important aspects of operant behavior: response
rates under steady-state conditions and resistance to
change under disruption conditions. The theory suggests that response rate is governed by the contingent
relation between the response and the reinforcer it
produces in a manner consistent with Herrnsteins
(1970) single-response version of the matching law
(i.e., Equation 2). Resistance to change is governed
by the Pavlovian relation between the discriminative
stimulus context in which the behavior occurs and
the reinforcers obtained in that context.
Resistance to change is typically studied by
arranging different conditions of reinforcement
391
Timothy A. Shahan
(e.g., reinforcement rates or magnitudes), presented

one at a time and each in the presence of a distinctive stimulus (i.e., a multiple schedule of reinforcement). Once steady-state responding is achieved in
the presence of both stimuli, a disruptor is introduced (e.g., satiation of the reinforcer, extinction).
The resultant decrease in behavior relative to the
predisruption baseline level provides the measure of
resistance to change. Relatively smaller decreases in
responding compared with baseline reflect greater
resistance to change. Although simple response rate
under steady-state conditions has traditionally been
used to measure the strengthening effects of reinforcers, behavioral momentum theory suggests that
resistance to change provides a better measure of
response strength. The reason is that response rates
are well known to be influenced by operations
(e.g., pacing contingencies such as differential reinforcement of low rate) that are generally not considered to affect response strength (e.g., Nevin, 1974;
Nevin & Grace, 2000).
Higher rates of reinforcement in a stimulus context would generally be expected to produce both
higher response rates and greater resistance to
changeand they do (see Nevin, 1992). However,
because response rates and resistance to change
are governed separately by the operant response
reinforcer and Pavlovian stimulusreinforcer relations, respectively, it is possible to simultaneously
decrease response rates and yet increase resistance
to change. In practice, this has been demonstrated
by adding reinforcers to the stimulus context that
are not contingent on the operant response. The
noncontingent reinforcers decrease baseline
response rates by degrading the contingent response
reinforcer relation but increase resistance to change
by improving the Pavlovian stimulusreinforcer relation signaled by the discriminative stimulus context
(e.g., Nevin, Tota, Torquato, & Shull, 1990).
Quantitatively, behavioral momentum theory
suggests that relative resistance to change in the
presence of two stimuli is a power function of the
relative rate of reinforcement obtained in the presence of those stimuli such that
b
R
m1
= b 1 ,
m2
R2
392
(6)
where m1 and m2 are resistance to change of responding in the presence of stimuli 1 and 2, R1 and R2 refer
to the rates of primary reinforcement obtained in the
presence of those stimuli, and the parameter b
reflects sensitivity of relative resistance to change to
variations in relative reinforcement rates (Nevin,
1992). Equation 6 and related quantitative models
making up behavioral momentum theory have provided a general framework within which to understand how reinforcement experienced in the
presence of stimuli govern the persistence of operant
behavior in the presence of that stimulus. The framework provided by the theory for characterizing the
persistence of operant behavior has been found to
have broad generality across reinforcer types, settings, and species ranging from fish to humans (e.g.,
Ahearn, Clark, Gardenier, Chung, & Dube, 2003;
Cohen, 1996; Grimes & Shull, 2001; Harper, 1999;
Igaki & Sakagami, 2004; Mace et al., 1990; Quick &
Shahan, 2009; Shahan & Burke, 2004).
Allocation and Persistence of Attention

and Operant Behavior: Shared
Mechanisms?
Operant behavior and at least part of what is called
attention appear to be goal-related activities.
Although the goal-directed aspects of attention have
generally been manipulated with instructions,
insights provided by quantitative theories of operant
behavior might be useful in understanding how
goals in the natural environment help to govern
attention. For example, both behavior and attention
appear to be limited resources requiring allocation
decisions. Thus, the quantitative framework for
choice provided by matching theory might also provide a first step toward replacing placeholder concepts, such as a central administrator, with a
quantitative theory of how experience translates into
the differential allocation of attending. In addition,
the framework provided by behavioral momentum
theory for understanding the persistence of operant
behavior might be useful for understanding the persistence of attending under conditions of disruption.
Nonetheless, the suggestion that consequences
might similarly affect what is typically called attention and what is typically called behavior would
require empirical evidence. Behavior analysts know
how consequences affect what traditionally falls

under the heading of operant behavior, but what
about phenomena more likely to be categorized
under the heading of attention?
Attention and Differential
Consequences
Human Attention and Differential

Consequences
Until recently, there was nearly no experimental
analysis of how variations in consequences affect
performance of humans in attention tasks. As noted
earlier, studies of goal-directed attention have
almost always relied on instructions to produce
changes in goal-directed attention. Nonetheless,
research by Gopher (1992) suggested that the successful allocation of attention appears to be a trainable skill that improves with practice and differential
consequences. For example, participants were
exposed to a demanding space-based game designed
to simulate the complexities of divided attention
encountered by fighter pilots. The simulation was
complex enough that the participants often initially
expressed panic and adopted suboptimal strategies.
Participants were instructed to focus their attention
on a subset of aspects of the situation and to periodically shift their attention to other aspects. The participants received feedback and point-based rewards
for successful performance, thus encountering differential consequences with changes in their allocation of attention. As a result, the participants
avoided adopting suboptimal strategies and became
proficient at the highly demanding tasks. Such
results are consistent with the suggestion that the
allocation of attention can be affected by experiencing differential consequences.
A more recent set of experiments examining how
differential consequences affect attention used a
negative priming procedure (Della Libera &
Chelazzi, 2006). Negative priming refers to a phenomenon whereby selectively attending to a target
stimulus accompanied by an irrelevant distractor
during a prime trial results in poorer performance
(i.e., increased reaction time) when the distractor
stimulus is used as a target shortly thereafter during
a probe test. For example, attending to a red shape
and ignoring a concurrently presented green distractor shape in a prime trial leads to an increased reaction time when participants are required to attend to
the previously ignored green shape in the next trial
(i.e., probe). The typical interpretation of negative
priming is that it results from a switching of selective visual attention to the target and away from the
distractor during the prime that persists until the
probe test (see Pashler, 1998, for a review). Pashler
(1998) suggested that negative priming might occur
as a result of an error-correction learning mechanism because a distractor item is unhelpful at one
time and therefore typically unlikely to be helpful
shortly after. If Pashlers suggestion is correct, it is
reasonable to expect that negative priming should
be affected by changing the consequences of attending to the target stimulus during the prime.
Della Libera and Chelazzi (2006) examined how
differential consequences affected negative priming
in a visual selective attention task. In one experiment, prime stimuli were global numbers consisting
of appropriately arranged local numbers (e.g., multiple instances of smaller 6s at the local level arranged
into a larger number 5 at the global level). Before
the presentation of a prime stimulus, the letter G or
the letter L instructed the subjects to attend to the
global or local level, respectively. A correct response
was defined as reporting the number at the
instructed level (either global or local). Thus, the
number at the other level became the to-be-ignored
distractor. For example, if the prime was a global 5
made up of local 6s, choosing the number 5 was correct after the instruction G and choosing the number 6 was correct after the instruction L. To
manipulate the consequences of attending to the target stimulus, correct responses to primes were followed by either a high (0.10) or a low (0.01)
payoff, as reported on the computer screen. The
probe stimulus was then presented 400 milliseconds
later. Subjects were to report the number in the
probe stimulus, regardless of whether it occurred at
the global level (e.g., a 5 made up of local Xs) or the
local level (i.e., an X made up of local 5s). In this
procedure, negative priming would be evidenced by
longer reaction times when the number in the probe
stimulus occurred at the same level of the distractor
stimulus in the prime (e.g., subject attended to
393
Timothy A. Shahan
local-level 6s and ignored a global-level 5 in the

prime, but in the probe trial, the target was at the
global level). Consistent with the hypothesis that
consequences can affect the allocation of attention,
negative priming occurred only when detecting the
target stimulus in the prime produced the larger
payoff. In other words, the highly reinforced allocation of attention to the target in the prime produced
a greater shift of attention to the target and away
from the distractor that was apparent in the subsequent probe test. A second experiment using the
same basic paradigm extended the result to formbased stimuli and samedifferent choice-based
responding. Thus, shifts in attention as measured by
a common selective attention paradigm appear to be
affected by differential consequences.
A subsequent set of experiments by Della Libera
and Chelazzi (2009) showed that differential consequences in selective visual attention tasks could
have relatively long-term effects on how attention is
allocated. In both experiments, subjects were
exposed to a 3-day training phase in which performance on a selective visual attention task produced
different magnitude reinforcers for two sets of stimuli. Specifically, trials started with a red or a green
cue that signaled the target in an immediately following display. The display consisted of three different nonsense shapes. One of the shapes was a black
comparison and was presented next to two overlapping shapes, one of which was red and the other of
which was green. The subjects task was to determine whether the shape of the previously cued color
(i.e., either red or green) matched the black comparison. Thus, the other colored shape served as a distractor stimulus to be ignored. Correct responses
were more likely to lead to a higher payoff (0.10)
for four shapes and a lower payoff (0.01) for four
different shapes. Two of the high-payoff stimuli
resulted in the higher payoff when they served as
the target and were correctly detected. The other
two high-payoff stimuli resulted in the higher payoff
when they served as the distractor stimulus and
were correctly ignored. The same was true for the
low-payoff stimulitwo stimuli resulted in the
lower payoff when they were correctly detected as
targets and the other two resulted in the lower payoff when they were correctly ignored as distractors.
394
An additional eight neutral shapes were equally as

likely to lead to the higher and lower payoffs as targets or distractors. Five days after the training phase
with the differential payoff, subjects were exposed to
either the same task (Experiment 1) or a different
visual search task (Experiment 2), both using the
stimuli from the training phase but in the absence
of reward.
In Della Libera and Chelazzis (2009) Experiment 1, stimuli associated with a high payoff when
serving as the target in the training phase were more
difficult to ignore when they appeared as the distractor during the subsequent test. However, stimuli
associated with a low payoff when serving as the target in training were more easily ignored when they
appeared as the distractor during the subsequent
test. Complementary results were obtained with
stimuli that had served as distractors during the
training phase. Stimuli that were associated with a
high payoff when they were ignored as distractors in
training produced less interference when they continued to serve as distractors in the test, but stimuli
previously associated with a low payoff when they
had been ignored in training continued to interfere
with test performance as distractors. In Experiment
2, the testing phase was a visual search task in
which a black sample shape was briefly presented
and followed by two black comparison shapes. The
subjects task was to report whether the sample
stimulus was present in the comparison display.
The results showed that detections in the comparison display were faster if the sample had previously
been associated with a higher payoff as a target in
the training phase. Together, the results from these
two experiments suggested that differential reinforcement for attending to or ignoring stimuli in a
selective attention task can have relatively longlasting effects on attending to or ignoring those
stimuli later.
An experiment by Raymond and OBrien (2009)
examined how differences in payoff for choosing
stimuli in a choice task affected subsequent recognition of those stimuli in an attentional blink procedure. Subjects were first given a series of choices
between sets of two computer-generated faces. Half
of the 24 face stimuli were associated with either a
gain (plus 5 pence) or a loss (minus 5 pence) at a
high probability (.8) or a low probability (.2). The

other half of the stimuli were neutral and associated
with no consequence. During exposure to this
choice condition, subjects learned to choose the
stimuli associated with a higher probability of a gain
and a lower probability of a loss. Next, the subjects
were exposed to a standard attentional blink procedure in the absence of explicit consequences. In the
attentional blink procedure, subjects were shown a
series of four brief stimuli in succession (85 milliseconds per stimulus). The first stimulus at Time 1
was an oval made up of either circles or squares followed immediately by a masking stimulus. The
masking stimulus occupied the same space as the
first stimulus and was used to stop processing of the
first stimulus and eliminate afterimages. The masking stimulus was followed by either a short
(200-millisecond) or long (800-millisecond) delay,
during which the screen was blank. At Time 2,
immediately after the delay, the face stimuli from
the earlier choice task or novel face stimuli were
presented and followed immediately by a masking
stimulus. Finally, the subjects were asked to report
by pressing buttons whether the stimulus at Time 1
had been made up of circles or squares and then
whether the face at Time 2 had been familiar or
novel. An attentional blink effect in this procedure
is evidenced by poorer performance on the face recognition task at Time 2 after a shorter delay between
Time 1 and Time 2. The usual interpretation of the
attentional blink effect is that it results from the fact
that attention is a limited resource that is temporarily taxed by performance at Time 1 and is therefore
not available again for some small amount of time
thereafter. Thus, long-delay (800-millisecond) and
short-delay (200-millisecond) conditions were chosen such that attention would typically be fully
available or taxed, respectively. The results showed
that regardless of whether attention was fully available or taxed, recognition of faces at Time 2 was better for faces previously producing a gain or a loss
than for previously neutral stimuli. Thus, stimuli
that needed to be chosen or avoided because of their
associated consequences in the choice task produced improved recognition performance at Time 2.
Most important, a typical attentional blink effect
was obtained when attention was taxed with the
shorter delay, but only for neutral stimuli or stimuli

previously associated with a loss. The attentional
blink effect was eliminated for stimuli that were previously associated with a gain. Thus, the availability
of attention for stimuli in a taxing situation appears
to depend on the consequences encountered as a
result of previous choices involving those stimuli
such that attention was more readily available for
stimuli previously associated with a gain.
The studies reviewed in this section generally
support the conclusion that consequences encountered as a result of attending to particular features of
the environment have an impact on the allocation of
attention in a manner similar to their impact on simple operant behavior. Thus, existing theoretical
accounts of operant behavior may be useful for
describing how differential consequences affect the
goal-directed control of attention.
Conditioning, Learning, and Attention

Unlike the relatively recent focus on the impact of
consequences on the allocation of human attention,
theories of conditioning and learning have a long
history of assuming reinforcement-related changes
in attention. For example, theories of discrimination
learning have commonly assumed that changes in
attending to relevant stimulus dimensions are based
on the experience of differential reinforcement in
the presence of stimuli (e.g., Blough, 1969; Lovejoy,
1965; Mackintosh, 1975; Mackintosh & Little, 1969;
Nevin, Davison, & Shahan, 2005; Sutherland &
Mackintosh, 1971; Zeaman & House, 1963). Many
such models have also been closely related to the
study of Pavlovian conditioning. Phenomena such
as blocking (Kamin, 1968), overshadowing (Pavlov,
1927), and the relative validity effect (Wagner,
Logan, Haberlandt, & Price, 1968) show that CSs
that are more salient or are better predictors of USs
reduce conditioning to jointly available stimuli that
are less salient or are poorer predictors of a US. The
RescorlaWagner (1972) model describes these cue
competition effects as being the result of competition between stimuli for a limited amount of associative value available from a US. Other theories,
however, describe cue competition effects in terms
of inferred changes in attending to stimuli with
experience rather than changes in US effectiveness.
395
Timothy A. Shahan
Mackintoshs (1975) attention theory is based on

the fact that stimuli that are better predictors of
reward attract increasing attention as conditioning
proceeds. Likewise, Lubows (1989) conditioned
attention theory suggests that attention to rewardpredictive stimuli increases in a manner consistent
with the principles of classical conditioning. Interestingly, an alternative attention-based model forwarded by Pearce and Hall (1980) predicts many
cue competition phenomena with the opposite
assumptionthat attention to a stimulus decreases
when it is a reliable predictor of reinforcement. More
recent accounts have allowed for both increases and
decreases in attention by proposing multiple attention systems for different types of learning situations
(Hogarth, Dickinson, & Duka, 2010).
Although an evaluation of the relative merits of
these attention-based theories of conditioning is
beyond the scope of this chapter, all the theories
share the assumptions that attention is critical for
learning associations between stimuli and that attention itself depends on previously experienced relations between stimuli and reinforcers. In a general
sense, such theories might be thought of as addressing how experience changes what Norman (1968)
referred to as the pertinence of stimuli. The
approach I explore in this chapter is clearly in the
same tradition as these other approaches in asserting
that at least part of what is referred to as attention is
modified by experience in accordance with existing
accounts of conditioning and learning. Specifically,
research in my laboratory has been focused on
extending existing quantitative accounts of operant
conditioning to attending. To do so, my lab has
relied on procedures with animals that are aimed at
more directly measuring attention.
To study attention more directly with animals,
Riley and colleagues (Riley & Leith, 1976; Riley &
Roitblat, 1978; Zentall & Riley, 2000) suggested
using procedures more similar to those used with
humans and likely to produce selective processing
as a result of capacity limitations (cf. Mackintosh,
1975). One such procedure is a modified delayed
matching-to-sample task used by Maki and Leuin
(1972) to study divided attention in pigeons. The
pigeons were presented with compound samples
consisting of a combination of two elements or
396
Figure 17.1. Schematic of a divided-attention

procedure for pigeons. B = blue; R = red.
samples made up of just one of the elements (see

Figure 17.1). In Maki and Leuin, line orientation
(i.e., vertical or horizontal) and key color (i.e., red
or blue) were used for the two sets of element stimuli. For single-element sample presentations, either
a simple color (i.e., red or blue) or line orientation
(i.e., white vertical or horizontal lines on a black
background) was presented. For compound sample
presentations, both a color and a line orientation
were presented simultaneously (e.g., a blue key with
superimposed white vertical lines). After both
single-element and compound sample stimuli, two
comparison stimuli were presented. The two comparison stimuli were always single-element stimuli
and were either the two line orientations or the two
colors. One of the comparison stimuli had been
present in the sample, and choice of that stimulus
was followed by a food reinforcer. Choice of the
other stimulus was not followed by a reinforcer.
Because the type of comparison stimuli to be presented is unpredictable, the pigeons must attend to
both elements in the compound sample to perform
accurately. Using this procedure, Maki and Leuin
found that to maintain accuracy at 80% for compound samples, the sample stimuli needed to be presented for longer durations than for element samples.
Subsequently, Maki and Leith (1973) found
superior accuracy for element-sample trials than for
compound-sample trials when the sample duration
was fixed at the value for both types of trials. Many
related experiments have obtained a similar result

(cf. Gottselig, Wasserman, & Young, 2001; Langley &
Riley, 1993; Leith & Maki, 1975; Santi, Grossi, &
Gibson, 1982; Zentall, Sherburne, & Zhang, 1997).
Nonattention-based hypotheses have been proposed to account for the poorer accuracy with compound samples than with element samples (e.g.,
generalization decrement, coding decrement), but
the data generally support the suggestion that this
difference results from the division of attention
required for compound sample stimuli (see Roberts,
1998, and Zentall & Riley, 2000, for reviews).
The Matching Law and Allocation of

Divided Attention
To assess the utility of the matching law for describing the effects of variations in relative reinforcement
on the allocation of attending, Shahan and Podlesnik (2006) used a divided-attention procedure with
pigeons similar to that of Maki and Leith (1973).
Figure 17.2 shows a schematic of the procedure.
Trials started with a white trial-ready stimulus, a
single peck on which produced a compound-sample
stimulus with one of two colors (i.e., green or blue)
and one of two line orientations (i.e., vertical or horizontal). The sample stimulus was presented for 5
seconds and was terminated response independently. Next, single-element comparison stimuli
that were of either two colors (hereinafter called

color trials) or two line orientations (hereinafter
called line trials) were presented, and a peck on the
comparison stimulus that had appeared in the sample produced a food reinforcer. Across conditions,
the probability of reinforcement for correct matches
for the two types of comparison stimuli was varied.
The overall probability of reinforcement remained at
.5, but the ratio of reinforcement probabilities for
the color and line dimensions varied across values of
1:9, 1:3, 1:1, 3:1, and 9:1 for the two color and two
line dimensions, respectively. The results showed
that variations in the ratio of reinforcement for the
two dimensions produced orderly changes in accuracy for that dimension. Specifically, accuracy
increased for the dimension associated with a higher
probability of reinforcement and decreased for the
dimension associated with a lower probability of
reinforcement.
Shahan and Podlesnik (2006) applied the matching law to changes in accuracy with changes in reinforcement allocation using log d as the measure of
accuracy. Log d is a common measure in behavioral
approaches to signal detection and conditional discrimination performance (e.g., Davison & Nevin,
1999; Davison & Tustin, 1978). Log d is particularly
useful because it is bias free and ranges from 0 at
chance performance to infinity at perfect performance. These metric properties make it similar to
response-rate measures typically used to characterize simple operant responding with the matching
law. Log d was calculated separately for color and
line trials such that
S1 S2
log d = 0.5 log corr corr ,
S1incorr S2incorr
(7)
where S1corr and S1incorr refer to correct and incorrect

responses after presentation of one sample (e.g.,
blue) and S2corr and S2incorr refer to correct and incorrect responses after the other sample (e.g., green).
To characterize changes in relative accuracy with
changes in relative reinforcement, Shahan and
Podlesnik used this version of the generalized
matching law:
Figure 17.2. Schematic of the divided-attention
procedure for pigeons used by Shahan and
Podlesnik (2006). G = green; B = blue.
R
log dC log d L = a log C + log b,
RL
(8)
397
Timothy A. Shahan
where log dC and log dL refer to log d for color and

line trials, respectively, and RC and RL refer to reinforcers obtained on color and line trials, respectively. The parameters a and log b refer to sensitivity
of relative accuracy to relative reinforcement and
bias in accuracy for one trial type unrelated to relative reinforcement, respectively. Figure 17.3 shows
the fits of Equation 8 to the mean data across
pigeons. Equation 8 did a good job describing the
effects of changes in relative reinforcement on relative accuracy for the two types of comparison trials.
With a (i.e., sensitivity to relative reinforcement)
equal to 0.57 and log b (i.e., bias) equal to 0.26,
Equation 8 accounted for 96% of the variance in the
averaged data. Assuming that the changes in relative
accuracy with the two types of comparison trials
reflect differences in attending to the elements of the
compound stimuli, the fact that sensitivity (i.e., the
a parameter) was greater than zero suggests that the
allocation of attention to the elements was sensitive
to the relative reinforcement associated with those
elements. The finding that log b was greater than
zero reflects the fact that overall accuracy also
tended to be greater for colors than for lines across
the range of relative reinforcement rates.
An alternative interpretation of the changes in
accuracy in the Shahan and Podlesnik (2006) experiment is that variations in relative reinforcement for
the two types of comparison stimuli affected behavior only at the choice point without changing
attending to the elements of the compound samples.
Even if the subjects had attended equally to the elements of the compound samples, variations in relative reinforcement at the comparison stimuli might
have resulted in changes in motivation to choose the
correct stimulus at the comparison choice point. To
assess this alternative account, Shahan and Podlesnik (2007) examined whether changes in the duration of the compound samples altered the effects of
differential reinforcement on performance in the
same divided-attention task. They compared performance with sample durations of 2.25 seconds and
0.75 seconds and examined sensitivity of both accuracy and choice-response speeds (i.e., l/latency) at
the comparison choice point. If differential reinforcement affected only motivation at the choice
point, sensitivity of accuracy to changes in allocation of reinforcement should not depend on the
sample duration.
Figure 17.4 shows mean sensitivity values for
both accuracy and choice-response speeds for the
short and long sample durations. Consistent with
the findings of Shahan and Podlesnik (2006), accuracy on the two types of comparison trials was sensitive to variations in relative reinforcement. In
addition, sensitivity of accuracy was greater for the
longer sample duration. However, choice-response
speeds were only weakly sensitive to changes in
relative reinforcement and did not differ for the
short and long sample durations. Although overall
Figure 17.3. Generalized matching

law analysis of the effects of differential reinforcement on pigeons accuracy in Shahan and Podlesniks (2006)
divided-attention experiment. The
fitted line and parameters values were
produced by least-squares regression
of Equation 8.
Figure 17.4. Sensitivity (i.e.,

a in Equation 8) of accuracy and
choice-response speeds to differential reinforcement when the
sample was either short or long in
the Shahan and Podlesnik (2007)
experiment on divided attention
of pigeons.
398
accuracy was lower with the short samples, Shahan

and Podlesnik (2007) showed that the lower sensitivity of accuracy with the shorter sample was not
likely an artifact of the lower accuracies because it
appeared to result from the pigeons attending too
much to the element associated with the lower probability of reinforcement. These findings led Shahan
and Podlesnik to conclude that changes in motivation at the comparison choice point were likely not
responsible for the effects of relative reinforcement
on relative accuracy.
Further support for the suggestion that differential reinforcement affects attending to the elements
of compound sample stimuli in the experiments
of Shahan and Podlesnik (2006, 2007) came from
Shahan and Quick (2009). Specifically, using an
adjusting sample duration procedure similar to that
of Maki and Leuin (1972), we found that sample
stimuli associated with a lower reinforcement rate
require a longer sample to maintain a constant accuracy. This outcome was consistent with the suggestion that effects of differential reinforcement in the
divided-attention task are mediated through changes
in attending to the sample stimuli.
These findings suggest that the allocation of
attention of both humans and pigeons is affected by
differential consequences in a manner consistent
with simple principles of operant conditioning. The
choice-based account provided by the matching law
appears to provide a useful framework within which
to understand how differential consequences affect
the allocation of attention. For example, Shahan and
Podlesnik (2006, 2007) suggested that the sensitivity and bias parameters of Equation 8 could be useful for providing quantitative descriptions of
goal-driven and stimulus-driven aspects of attentional control, respectively. They suggested that sensitivity (i.e., a) might be used to capture the effects
of variations in relative reinforcement on stimulus
pertinence or goal-directed control of attention and
that bias in accuracy independent of reinforcement
variations (i.e., log b) might reflect the effects of sensory features of the stimuli (i.e., sensory activation;
Norman, 1968). Thus, the matching law appears
useful as a first approximation to replace concepts
such as a central administrator with a quantitative
theory of how differential consequences translate
into differential attending. Nonetheless, it is important to note that the applicability of the matching
law to attending has only been directly assessed with
pigeons. Ultimately, assessing the utility of the
matching law for describing goal-directed attention
will require direct examinations with humans in
standard preparations such as those reviewed earlier. Regardless, the findings reviewed in the preceding section suggest that considering both behavior
and attention as limited resources requiring similar
allocation decisions based on differential consequences may indeed be useful.
Behavioral Momentum and the

Persistence of Attending
Although I am unaware of any experiments with
humans or nonhumans examining how differential
reinforcement affects the persistence of attending
using procedures such as those described earlier, my
laboratory has examined the issue using observing
responses of pigeons. Observing responses produce
sensory contact with stimuli to be discriminated and
have been used as an overt analog of attention for
more than 50 years (e.g., Wyckoff, 1952). In the
typical observing-response procedure, unsignaled
periods of primary reinforcement availability for
some response alternate unpredictably with periods
in which the response does not produce reinforcement (i.e., extinction). A second response (i.e., the
observing response) produces brief access to stimuli
signaling whether reinforcement is available. A stimulus signaling the availability of reinforcement is
referred to as an S+, and a stimulus signaling
extinction is referred to as an S. Figure 17.5 shows
an example of an observing-response procedure for
pigeons. Responses on the white key on the right
produce primary reinforcement (i.e., food) on a
variable-interval (VI) schedule of reinforcement. (A
VI schedule arranges reinforcement for the first
response after a variable amount of time around a
mean value.) The periods of VI food reinforcement
on the key alternate unpredictably with unsignaled
periods in which reinforcement is not available for
pecking the key (i.e., extinction is in effect). Pecks
on the key on the left (i.e., the observing response)
produce 15-second periods of stimuli correlated
with the conditions of reinforcement on the food
399
Timothy A. Shahan
Figure 17.5. Schematic of an observingresponse procedure for pigeons. W = white;

FR = fixed ratio; Ext = extinction; G = green;
R = red; VI = variable interval.
key. If the VI schedule of food reinforcement is

available on the food key, both keys are lighted
green for 15 seconds (i.e., S+). If extinction is in
effect on the food key, both keys are lighted red for
15 seconds (i.e., S). Pigeons readily learn to peck
the observing key and produce the stimuli differentially associated with the periods of primary reinforcement and extinction. In addition, they learn to
discriminate S+ and S and come to respond on
the food key predominately during S+. Responses
on the observing key are used as the measure of
looking at or attending to the stimuli to be discriminated (see Dinsmoor, 1985, for discussion).
Consistent with the research described earlier
using the divided-attention procedure, research in
my laboratory has shown that the matching law is
also applicable to attending as measured with an
observing response. For example, when the rate of
primary reinforcement associated with a stimulus
increases, rats observing of that stimulus increases
in a manner well described by the single-response
version of the matching law (i.e., Equation 2; see
Shahan & Podlesnik, 2008b). Furthermore, Shahan,
Podlesnik, and Jimenez-Gomez (2006) found that
when pigeons are given a choice between two
observing keys and the relative rate of S+ presentation is varied between the keys, pigeons allocate
their observing in accordance with the generalized
matching law (i.e., Equation 3). This result is consistent with the results obtained in the delayed
matching-to-sample procedure reported earlier
400
(i.e., Shahan & Podlesnik, 2006, 2007). Thus, in

terms of evaluating the applicability of quantitative
accounts of operant behavior to attending, the
observing-response procedure appears to yield conclusions similar to the procedures discussed earlier.
To examine how differential reinforcement
affects the persistence of attending, researchers have
used methods such as those typically used to study
the resistance to change of simple operant behavior.
For example, Shahan, Magee, and Dobberstein
(2003) arranged a multiple schedule in which
observing responses of pigeons produced different
stimuli in the two components of the multiple
schedule. In a rich component, observing responses
produced an S+ associated with a higher rate of primary reinforcement. In a lean component, observing
responses produced an S+ associated with a lower
rate of primary reinforcement. Consistent with the
matching law data presented earlier, Shahan et al.
found that observing occurred at a higher rate in the
rich than in the lean component during a predisruption baseline. When performance was disrupted by
satiating the pigeons with presession or intercomponent food, observing was more resistant to change in
the rich component than in the lean component.
Thus, using observing responses as a measure of
attention, it appears that stimuli associated with a
higher rate of reinforcement command more persistent attending. Furthermore, quantitative analyses
revealed that resistance to change of observing and
simple operant behavior maintained directly by the
food reinforcer were similarly sensitive to the reinforcement ratios arranged (i.e., b in Equation 6).
Thus, extending behavioral momentum theory from
simple operant behavior to attending appears to be
relatively straightforward.
Partly on the basis of the observing data, Nevin
et al. (2005) and Nevin, Davison, Odum, and Shahan (2007) suggested that the effects of reinforcement on attending are governed by behavioral
momentum theory. Specifically, the theory is based
on a version of behavioral momentum theory that
predicts rates of an operant response during baseline
and disruption conditions such that
B = k exp
(r
/ ra )
,
(9)
where B is response rate, rs is the rate of reinforcement in the presence of the stimulus in which the
behavior is occurring, and ra is the overall rate of
background reinforcement in the entire session. The
parameters k, x, and b correspond to asymptotic
response rate, background disruption that reduces
responding, and sensitivity to relative reinforcement, respectively. Under baseline conditions,
Equation 9 predicts functions relating response rates
and reinforcement rates that are nearly indistinguishable from Herrnsteins (1970) single-option
version of the matching law (i.e., Equation 2; see
Nevin et al., 2005). When additional terms are
included in the numerator for specific disruptors,
Equation 9 also captures the finding that responding
is more resistant to disruption in contexts associated
with higher rates of reinforcement. On the basis of
Shahan et al.s (2003) finding that the persistence
of observing (i.e., attending) appears to be affected
by reinforcement in a manner similar to simple
response rates, Nevin et al. (2005) suggested the
following equation:
p ( A s ) = exp
(r
/ ra )
,
(10)
where p(As) is the probability of attending to a sample stimulus and rs is now the rate of reinforcement
associated with attending to the stimulus and ra
remains the overall rate of background reinforcement in the session. The other parameters are as in
Equation 9. The scalar k is not required because the
asymptote of the probability of attending is at 1.0. In
effect, Equation 10 assumes that the probability and
persistence of attending is governed by reinforcement in the same way as is simple operant behavior.
Nevin et al. (2005, 2007) have shown that inferences about changes in attending based on Equation
10 can be used in a larger model addressing the
effects of reinforcement on discrimination performance in the steady state and during disruption.
The use of Equation 10 in this broader model of
stimulus control has provided an account of a wide
variety of reinforcement-related effects in the stimulus control literature, including the effects of reinforcement on the persistence of discriminative
performance when disrupted (e.g., Nevin, Milo,
Odum, & Shahan, 2003). These initial successes
suggest that behavioral momentum theory may

indeed be useful for understanding how differential
reinforcement contributes to the allocation and persistence of attending. Nonetheless, the research
effort directed at examining how reinforcement
affects the persistence of attending has just begun.
Data from a variety of typical paradigms for studying
attention will be required to more fully assess the
utility of the approach.
Attention, Conditioned
Reinforcement, and Signposts
The research reviewed in the preceding sections
suggests that differential reinforcement affects both
the allocation and the persistence of attending to
stimuli. One interpretation of how differential reinforcement affects attending is that stimuli associated
with reinforcers become conditioned reinforcers as a
result of Pavlovian conditioning (Mackintosh, 1974;
Williams, 1994). From this perspective, the effects
of differential reinforcement on attending result
from the strengthening of a connection between an
initially neutral stimulus and an existing reinforcer.
As a result, initially neutral stimuli may come to
reinforce behavior or attending that produces contact with them. Thus, quantitative models of simple
operant behavior may describe the effects of differential reinforcement on the allocation and persistence of attending because the stimuli being
attended to are themselves reinforcers. Although
such an account works fairly well when considered
only within the context of the effects of differential
reinforcement on the allocation of attending, the
account encounters some difficulties within the context of the persistence of attending. I consider these
issues in turn in the following sections.
Conditioned Reinforcement and

the Allocation of Attention
To understand how conditioned reinforcement
might provide an account of the effects of differential reinforcement on the allocation of attending,
consider Shahan and Podlesniks (2006) dividedattention experiment. In that experiment, each of
the two elements making up the compound sample
stimuli in the delayed matching-to-sample procedure
401
Timothy A. Shahan
was differentially associated with reinforcement.

When the color dimension was associated with a
higher probability of reinforcement than the line
dimension, the color appeared to maintain more
attending than the line. An account based on conditioned reinforcement would suggest that the color
served as a better conditioned reinforcer because it
was a better predictor of reinforcement than was the
line. As a result of being a more potent conditioned
reinforcer, the color would be expected to more
effectively strengthen attending to it. The resultant
data relating variations in relative reinforcement to
attending might be expected to conform to the
matching law because a variety of quantitative
extensions of the generalized matching law (i.e.,
Equation 3) to choice between conditioned reinforcers predict just this type of effect (see Mazur, 2001,
for review). The specific details of those models are
not important for the present purposes, but many of
the models take the general form
a
r V
B1
= b 1 1 ,
B2
r2 V2
(11)
where B1 and B2 refer to behavior that produces contact with two conditioned reinforcers, r1 and r2 refer
to frequencies of production of the conditioned
reinforcers, and V1 and V2 are summary terms
describing the value or strengthening effects of the
conditioned reinforcers. The models differ in how
value is calculated, but most of the approaches to
calculating value share important characteristics
with existing theoretical accounts of Pavlovian conditioning (see Chapters 13 and 14, this volume).
Assuming that such models can be extended to
attention, they predict relative changes in attending
to stimuli with changes in relative value such as
those obtained in Shahan and Podlesnik (2006,
2007) or even in the human attention experiments
reviewed earlier. Furthermore, the models accurately predict changes in the allocation of attending
when the relative frequency of reinforcementassociated stimuli is varied as in the concurrent
observing-response experiment of Shahan, Podlesnik, and Jimenez-Gomez (2006). Thus, such models of conditioned reinforcement may hold promise
as a means to understand how stimuli differentially
associated with primary reinforcers come to govern
402
the allocation of attention. Research evaluating the

different models in simple choice situations has documented several important phenomena contributing
to how conditioned reinforcers govern choice (see
Grace, 1994, and Mazur, 2001, for reviews). A
potentially fruitful research endeavor might be to
examine whether similar effects occur in tasks
designed specifically to study the effects of differential reinforcement on attention. Regardless of the
ultimate utility of the models for describing how
reinforcement affects the allocation of attention, an
interpretation of the effects of reinforcement on the
persistence of attending based on conditioned reinforcement encounters some difficulties, as described
in the next section.
Conditioned Reinforcement and

the Persistence of Attending
As I have noted, the only research I know of examining the effects of differential reinforcement on the
persistence of attending is Shahan et al.s (2003)
work with observing responses. Shahan et al.
showed that stimuli associated with higher rates of
primary reinforcement maintain more persistent
attending as measured with observing responses.
This outcome could suggest that the stimulus associated with the higher rate of primary reinforcement was a more effective conditioned reinforcer
and, hence, maintained more persistent attending.
In fact, observing responses are often considered
one of the best ways to study the conditioned reinforcing effects of reinforcement-associated stimuli
(e.g., Dinsmoor, 1983; Fantino, 1977; Williams,
1994). Regardless, it is important to note that Shahan et al. examined only the effects of differences
in primary reinforcement on the persistence of
observing. If stimuli associated with differential
reinforcement do affect the allocation and persistence of attention because they have themselves
become conditioned reinforcers, variations in the
rate or value of such stimuli would themselves be
expected to contribute to the persistence of attending in the same way as primary reinforcers. However, experiments from my laboratory have shown
that putative conditioned reinforcers do not contribute to the persistence of attending in the same
way as primary reinforcers.
Shahan and Podlesnik (2005) showed that an

observing response that produced more frequent
presentations of a food-associated stimulus (i.e.,
S+) occurred more frequently than an observing
response that produced less frequent S+ presentations. More important, the variation in frequency of
S+ presentation occurred in the absence of differences in frequency of primary reinforcement. Thus,
this result is consistent with the earlier choice
results and with what would be expected if S+ presentations strengthen responding in the same fashion as primary reinforcers. Despite the effect of S+
frequency on the rate of observing, however, differences in S+ frequency had no impact on the persistence of observing under conditions of disruption.
This result is not what would be expected if S+ presentations functioned as conditioned reinforcers.
Additional experiments by Shahan and Podlesnik
(2008a) examined the effects of variations in the
value of an S+ on observing rates and resistance to
change. The value of an S+ was varied by changing
the predictive relation between S + and the delivery
of primary reinforcement in variety of ways.
Although the details of the manipulations are not
important for present purposes, some of the methods pitted S+ value against the rate of primary reinforcement, and some varied S+ value while holding
constant rates of primary reinforcement. The results
showed that as with higher frequencies of S+ presentation in Shahan and Podlesnik (2005), higher
valued S+ presentations maintained higher rates of
observing. However, differences in the value of S+
presentations had no impact on the persistence of
observing under conditions of disruption. If higher
valued S+ presentations maintained higher observing rates because they served as more potent conditioned reinforcers, then higher valued S+
presentations should also have produced greater
resistance to disruption, but they did not.
Shahan and Podlesnik (2008b) provided a summary quantitative analysis of all these observing and
resistance-to-disruption experiments. Although
rates of observing were clearly affected by the frequency and value of S+ presentations, resistance to
disruption of observing was only an orderly function
of the rate of primary reinforcement in the context
in which observing was occurring. Resistance to
disruption of observing and frequency or value of

S+ presentations had no meaningful relationship.
Given that both frequency and value of S+ presentations would be expected to affect resistance to
disruption if S+ presentations were serving as conditioned reinforcers, Shahan and Podlesnik concluded that S+ presentations might affect the
frequency of observing through some other mechanism. Thus, at present, it appears that the effects of
differential reinforcement on the persistence of
attending might not be mediated through the process of conditioned reinforcement. In short, stimuli
that might be thought to command attention
because they have become conditioned reinforcers
do not appear to have an impact on the persistence
of attending as do other reinforcers.
Although the effects of differential reinforcement
on the allocation of attending discussed in the preceding section might still be mediated through the
process of conditioned reinforcement, it would not
be parsimonious to assert different mechanisms for
the allocation and persistence of attention. As I
show, the debate surrounding the utility of the concept of conditioned reinforcement is considerable.
As a result, the generalized matching lawbased theories of conditioned reinforcement might also be
interpreted in other terms.
Signposts as an Alternative to
Conditioned Reinforcement
A long history of skepticism surrounds the concept
of conditioned reinforcement. Data traditionally
interpreted as reflecting the acquired strengthening
effects of stimuli associated with reinforcers may be
interpreted in a variety of other ways. Although a
review of all these alternatives and the supporting
data is beyond the scope of this chapter, Shahan
(2010) provided a more thorough treatment of the
subject. Many commentators have noted that the
effects of stimuli associated with reinforcers on
response rates and choice might be more consistent
with a signaling or guidance process than with an
acquired reinforcement-strengthening process (e.g.,
Bolles, 1975; Davison & Baum, 2006; Longstreth,
1971; Rachlin, 1976; Staddon, 1983). Although a
variety of names have been used for reinforcementassociated stimuli in such an account, I have
403
Timothy A. Shahan
adopted signposts or means to an end as suggested by

Wolfe (1936), Longstreth (1971), and Bolles (1975).
According to a signpost account, stimuli associated
with reinforcers guide behavior because they predict
where reinforcers are located in space, time, or both.
Thus, an observing response that produces contact
with a stimulus associated with a primary reinforcer
serves to guide the organism to the reinforcer by
providing feedback about how or where the reinforcer is obtained. Behavior that produces signposts
occurs because the signpost is useful for getting to
the primary reinforcer, not because the signpost has
acquired the capacity to strengthen behavior in a
reinforcement-like manner. In this sense, signposts
might also be thought of as a means to an end for
acquiring reinforcers.
A set of two experiments by Bolles (1961) is useful for contrasting a signpost-based account and a
conditioned reinforcement account. In the first
experiment, rats responded on two levers that were
available at the same time and periodically produced a food pellet accompanied by a click of the
pellet dispenser. Receipt of a pellet plus click on
one of the levers was predictive of future pellets for
pressing that lever for a short time. During an
extinction test, the two levers were present, but
only one of them produced the click. Consistent
with what would be expected if the click had
acquired the capacity to function as a reinforcer,
the rats preferred to press the lever that produced
the click than the lever that did not. Nonetheless,
in the second experiment, receipt of a pellet plus
click on one lever predicted that subsequent pellets
were more likely on the other lever for a short
period. Unlike the first experiment, during the
extinction test the rats shifted their preference away
from the lever that produced the click. If the click
was a conditioned reinforcer, one would expect it
to further strengthen the response that produced it
during extinction. Bolles concluded that previous
demonstrations of apparent acquired conditionedreinforcing effects of stimuli might result not from
a reinforcement process, but because such stimuli
are signposts for how subsequent food is to be
obtained. Davison and Baum (2006) reached a similar conclusion on the basis of a related but more
extensive set of experiments with pigeons.
404
A traditional account of results such as those of

Bolles (1961) and Davison and Baum (2006) would
suggest that stimuli associated with the delivery of a
primary reinforcer might have multiple functions.
Specifically, in the Bolles experiments, one might
interpret the effect of the click in terms of both its
strengthening effects and its discriminative stimulus
effects. As I noted, the occurrence of operant behavior is known to be modulated by the presence of a
discriminative stimulus in the presence of which it
has been reinforced. In the Bolles experiment, the
click associated with food delivery might be interpreted to function as a discriminative stimulus for
pressing the appropriate lever (i.e., the same lever in
the first experiment and the other lever in the second experiment). A similar account might be
applied to the related findings of Davison and Baum:
When a food-paired stimulus signals that engaging
in a response is likely to be reinforced, the stimulus
sets the occasion for repeating the response, even if
that response is not the one that produced the stimulus. Thus, any conditioned reinforcementbased
strengthening effects of the food-paired stimulus
would be interpreted as being overridden by such discriminative stimulus effects of the stimulus. Although
this account seems plausible enough on its surface, it
is difficult to find any compelling data in the literature demonstrating strengthening effects of foodassociated stimuli above and beyond such signaling
effects (see Bolles, 1975; Davison & Baum, 2006;
Longstreth, 1971; Rachlin, 1976; Shahan, 2010; and
Staddon, 1983, for reviews). In short, it does not
appear that both a signaling and a strengthening function are required to account for effects typically
ascribed to conditioned reinforcement; a discriminative or signaling effect appears to be enough.
The suggestion that stimuli associated with reinforcers might serve as signals for behavior rather
than strengthen it is also consistent with some contemporary treatments of associative learning. For
example, Gallistel and Gibbon (2002; see also Balsam & Gallistel, 2009) suggested that conditioning
occurs not because of a reinforcement-driven backpropagation (i.e., strengthening) mechanism but
because organisms learn that some events in the
environment predict other events. Gallistel and
Gibbon suggested that organisms learn temporal,
spatial, and predictive regularities in the environment in the absence of any reinforcement-like backpropagation process. This suggestion is based on a
wealth of data showing that performance in both
Pavlovian and operant conditioning preparations is
often strikingly different from what would be
expected if learning resulted from a strengthening
process. As with Gallistel and Gibbon, but based
largely on data separate from operant choice experiments, Davison and Baum (2006) have also suggested that what are traditionally referred to as
reinforcers of operant behavior have their effects by
predicting additional such events and, thus, guiding
rather than strengthening behavior.
Although a full discussion of such a non
reinforcement-based approach to conditioning and
learning is not appropriate here (but see Shahan,
2010), its most important aspect is that it provides
mechanisms by which initially neutral stimuli might
come to guide behavior as signposts without acquiring conditioned-reinforcing effects. Because this
approach also asserts that the effect of primary reinforcers is to guide rather than strengthen behavior,
referring to initially neutral stimuli that come to
guide behavior as discriminative stimuli would not
be appropriate given the traditional meaning of that
term. For this reason, I have adopted the term signposts to describe a more general guidance and
means-to-an-end effect as an alternative to conditioned
reinforcement. Terminological issues aside, even if
one rejects the assertion that primary reinforcers do
not strengthen behavior, an account of conditioned
reinforcement in terms of a signposts or means-toan-end account is useful for understanding the
effects of differential reinforcement on the allocation
and persistence of attending.
From the perspective of a signpost-based
account, differential reinforcement might have its
impact on the allocation and persistence of attending because attending to stimuli that are more predictive of reinforcers is instrumental in guiding the
organism to those reinforcers. From this perspective, interpreting the data from the Shahan and
Podlesnik (2005, 2008a) experiments on the persistence of observing responses becomes easier.
Although more frequent production of a signpost or
production of a more informative signpost might be
expected to maintain more attending to it, such

signposts would not be expected to contribute to the
strength of attending (as would be predicted by a
back-propagation response-strengthening conditioned reinforcement process). In terms of the allocation of attention, discarding the notion that
stimuli associated with existing reinforcers themselves become reinforcers has little impact on the
utility of the generalized matching lawbased theories of conditioned reinforcement. Such theories
provide a quantitative description of how choice
between stimuli follows changes in the predictive
relations between those stimuli and reinforcers.
The notion of value can be interpreted to reflect the
utility of the stimuli in terms of their efficacy in
guiding organisms to reinforcers. The concept of
value need not carry any connotation of an acquired
reinforcement-like strengthening effect for the
models to be useful in describing how such stimuli
govern the allocation of behavior and attention.
Interpreting the models in such a way is consistent
with more general maximization-based economic or
regulatory approaches to instrumental action within
which the models might be included (e.g., Rachlin,
Battalio, Kagel, & Green, 1981; Rachlin, Green,
Kagel, & Battalio, 1976).
Summary and Conclusions
Both operant behavior and at least part of what is
usually referred to as attention can be characterized
as goal-directed activities. Although researchers
have only recently started examining the effects of
differential consequences on attention, such consequences appear to affect attention in much the same
way as they affect simple operant behavior. At this
early stage of the investigation, quantitative theories
describing how consequences affect the allocation
and persistence of operant behavior appear readily
applicable to attention. Such theories might provide
a starting point for replacing placeholder concepts
such as a central administrator responsible for the
goal-directed control of attention with a quantitative
framework for translating experience into differential attending. Finally, one interpretation of the
effects of differential reinforcement on attending
to stimuli is that stimuli predictive of existing
405
Timothy A. Shahan
reinforcers acquire the capacity to reinforce activity

that produces contact with them (i.e., they become
conditioned reinforcers). However, an alternative
approach suggests that such stimuli serve as signposts that guide behavior to reinforcers and serve as a
means to an end for acquiring reinforcers. Thus, differentially attending to stimuli predictive of reinforcers might be considered instrumental in acquiring
reinforcers rather than such stimuli strengthening
attending through a back-propagation mechanism.
Regardless of the ultimate mechanism responsible for
the effects of differential reinforcement on attention,
the fact remains that both operant behavior and attention are affected by consequences in a manner that is
usefully described by existing quantitative models.
Because the parameters known to affect behavior in
these models are easily manipulated, these theories
may lead to practical applications designed to affect
attention and other important behavior (see Volume
2, Chapters 5 and 7, this handbook).
References
Ahearn, W. H., Clark, K. M., Gardenier, N. C., Chung, B.
I., & Dube, W. V. (2003). Persistence of stereotyped
behavior: Examining the effects of external reinforcers. Journal of Applied Behavior Analysis, 36, 439448.
doi:10.1901/jaba.2003.36-439
American Psychiatric Association. (1994). Diagnostic
and statistical manual of mental disorders (4th ed.).
Washington, DC: Author.
Bolles, R. C. (1961). Is the click a token reward?

Psychological Record, 11, 163168.
Bolles, R. C. (1975). Theory of motivation (2nd ed.). New
York, NY: Harper & Row.
Bonnel, A.-M., & Prinzmetal, W. (1998). Dividing
attention between the color and the shape of
objects. Perception and Psychophysics, 60, 113124.
doi:10.3758/BF03211922
Bryson, S. E., Wainwright-Sharp, J. A., & Smith, L. M.
(1990). Autism: A developmental spatial neglect
syndrome? In J. Enns (Ed.), The development of attention: Research and theory (pp. 405427). Amsterdam,
the Netherlands: Elsevier. doi:10.1016/S01664115(08)60468-9
Catania, A. C. (1998). Learning (4th ed.). Upper Saddle
River, NJ: Prentice-Hall.
Cohen, S. L. (1996). Behavioral momentum of typing
behavior in college students. Journal of Behavior
Analysis and Therapy, 1, 3651.
Davison, M., & Baum, W. M. (2006). Do conditional
reinforcers count? Journal of the Experimental
Analysis of Behavior, 86, 269283. doi:10.1901/
jeab.2006.56-05
Davison, M., & Nevin, J. A. (1999). Stimuli, reinforcers, and behavior: An integration. Journal of the
Experimental Analysis of Behavior, 71, 439482.
doi:10.1901/jeab.1999.71-439
Davison, M., & Tustin, R. D. (1978). The relation between
the generalized matching law and signal-detection
theory. Journal of the Experimental Analysis of
Behavior, 29, 331336. doi:10.1901/jeab.1978.29-331
Baddeley, A. D. (1986). Working memory. Oxford,

England: Oxford University Press.
Della Libera, C., & Chelazzi, L. (2006). Visual selective

attention and the effects of momentary rewards.
Psychological Science, 17, 222227. doi:10.1111/
j.1467-9280.2006.01689.x
Balsam, P. D., & Gallistel, C. R. (2009). Temporal

maps and informativeness in associative learning.
Trends in Neurosciences, 32, 7378. doi:10.1016/j.
tins.2008.10.004
Della Libera, C., & Chelazzi, L. (2009). Learning to

attend and to ignore is a matter of gains and losses.
Psychological Science, 20, 778784. doi:10.1111/
j.1467-9280.2009.02360.x
Baum, W. M. (1974). On two types of deviation from the

matching law: Bias and undermatching. Journal of
the Experimental Analysis of Behavior, 22, 231242.
doi:10.1901/jeab.1974.22-231
Dijksterhuis, A., & Aarts, H. (2010). Goals, attention, and

(un)consciousness. Annual Review of Psychology, 61,
467490. doi:10.1146/annurev.psych.093008.100445
Baum, W. M. (1979). Matching, undermatching, and

overmatching in studies of choice. Journal of the
doi:10.1901/jeab.1979.32-269
Baum, W. M., & Rachlin, H. C. (1969). Choice as time
allocation. Journal of the Experimental Analysis of
Blough, D. S. (1969). Attention shifts in a maintained
discrimination. Science, 166, 125126. doi:10.1126/
science.166.3901.125
406
Dinsmoor, J. A. (1983). Observing and conditioned reinforcement. Behavioral and Brain Sciences, 6, 693728.
doi:10.1017/S0140525X00017969
Dinsmoor, J. A. (1985). The role of observing and attention in establishing stimulus control. Journal of the
doi:10.1901/jeab.1985.43-365
Dube, W. V., & McIlvane, W. J. (1997). Reinforcer frequency and restricted stimulus control. Journal of
doi:10.1901/jeab.1997.68-303
Ehrman, R. N., Robbins, S. J., Bromwell, M. A., Lankford,

M. E., Monterosso, J. R., & OBrien, C. P. (2002).
Comparing attentional bias to smoking cues in current smokers, former smokers, and non-smokers
using a dot-probe task. Drug and Alcohol Dependence,
67, 185191. doi:10.1016/S0376-8716(02)00065-0
Fantino, E. (1977). Conditioned reinforcement: Choice
and information. In W. K. Honig & J. E. R. Staddon
(Eds.), Handbook of operant behavior (pp. 313339).
Englewood Cliffs, NJ: Prentice-Hall.
Gallistel, C. R., & Gibbon, J. (2002). The symbolic foundations of conditioned behavior. Mahwah, NJ: Erlbaum.
Gopher, D. (1992). The skill of attention control:
Acquisition and execution of attention strategies.
In D. E. Meyer & S. Kornblum (Eds.), Attention and
performance XIV: Synergies in experimental psychology, artificial intelligence, and cognitive neuroscience
(pp. 299322). Cambridge, MA: MIT Press.
Gottselig, J. M., Wasserman, E. A., & Young, M. E.
(2001). Attentional trade-offs in pigeons learning to discriminate newly relevant visual stimulus
dimensions. Learning and Motivation, 32, 240253.
doi:10.1006/lmot.2000.1081
Grace, R. C. (1994). A contextual model of concurrentchains choice. Journal of the Experimental Analysis of
Grimes, J. A., & Shull, R. L. (2001). Responseindependent milk delivery enhances persistence of
pellet-reinforced lever pressing by rats. Journal of
doi:10.1901/jeab.2001.76-179
Harper, D. N. (1999). Drug-induced changes in responding are dependent on baseline stimulus-reinforcer
contingencies. Psychobiology, 27, 95104.
Herrnstein, R. J. (1961). Relative and absolute strength of
response as a function of frequency of reinforcement.
Journal of the Experimental Analysis of Behavior, 4,
267272. doi:10.1901/jeab.1961.4-267
Herrnstein, R. J. (1970). On the law of effect. Journal of
doi:10.1901/jeab.1970.13-243
Herrnstein, R. J. (1990). Rational choice theory:
Necessary but not sufficient. American Psychologist,
45, 356367. doi:10.1037/0003-066X.45.3.356
Herrnstein, R. J., Rachlin, H., & Laibson, D. I. (1997).
The matching law: Papers in psychology and economics.
Cambridge, MA: Harvard University Press.
Hogarth, L., Dickinson, A., & Duka, T. (2010). Selective
attention to conditioned stimuli in human discrimination learning: Untangling the effect of outcome
prediction, value, arousal and uncertainty. In C.
J. Mitchell & M. E. Le Pelley (Eds.), Attention and
associative learning (pp. 7197). Oxford, England:
Oxford University Press.
Igaki, T., & Sakagami, T. (2004). Resistance to change

in goldfish. Behavioural Processes, 66, 139152.
doi:10.1016/j.beproc.2004.01.009
James, W. (1890). The principles of psychology (Vol. 1).
New York, NY: Holt. doi:10.1037/11059-000
Johnsen, B. H., Laberg, J. C., Cox, W. M., Vaksdal, A., &
Hugdahl, K. (1994). Alcoholic subjects attentional
bias in the processing of alcohol-related words.
Psychology of Addictive Behaviors, 8, 111115.
doi:10.1037/0893-164X.8.2.111
Kahneman, D. (1973). Attention and effort. Englewood
Cliffs, NJ: Prentice Hall.
Kamin, L. J. (1968). Attention-like processes in classical conditioning. In M. R. Jones (Ed.), Miami
Symposium on the Prediction of Behavior: Aversive
stimulation (pp. 931). Miami, FL: University of
Miami Press.
Langley, C. M., & Riley, D. A. (1993). Limited capacity
information processing and pigeon matching-to-sample:
Testing alternative hypotheses. Animal Learning and
Behavior, 21, 226232. doi:10.3758/BF03197986
Leith, C. R., & Maki, W. S. (1975). Attention shifts during matching-to-sample performance in pigeons.
Animal Learning and Behavior, 3, 8589. doi:10.3758/
BF03209105
Loewenstein, Y., & Seung, H. S. (2006). Operant matching is a generic outcome of synaptic plasticity based
on the covariance between reward and neural activity. Proceedings of the National Academy of Sciences
of the United States of America, 103, 1522415229.
doi:10.1073/pnas.0505220103
Longstreth, L. E. (1971). A cognitive interpretation of secondary reinforcement. In J. K. Cole (Ed.), Nebraska
symposium on motivation (Vol. 19, pp. 3381).
Lincoln: University of Nebraska Press.
Lovaas, O. I., Koegel, R. L., & Schreibman, L. (1979).
Stimulus overselectivity in autism: A review of
research. Psychological Bulletin, 86, 12361254.
doi:10.1037/0033-2909.86.6.1236
Lovejoy, E. (1965). An attention theory of discrimination learning. Journal of Mathematical Psychology, 2,
342362. doi:10.1016/0022-2496(65)90009-X
Lubman, D. I., Peters, L. A., Mogg, K., Bradley, B. P., &
Deakin, J. F. W. (2000). Attentional bias for drug
cues in opiate dependence. Psychological Medicine,
30, 169175. doi:10.1017/S0033291799001269
Lubow, R. E. (1989). Latent inhibition and conditioned attention theory. New York, NY: Cambridge
University Press. doi:10.1017/CBO9780511529849
Luck, S. J., & Vecera, S. P. (2002). Attention. In H.
Pashler & S. Yantis (Eds.), Stevenss handbook of
experimental psychology: Vol. 1. Sensation and perception (3rd ed., pp. 235286). New York, NY: Wiley.
407
Timothy A. Shahan
Mace, F. C., Lalli, J. S., Shea, M. C., Lalli, E. P., West, B.

J., Roberts, M., & Nevin, J. A. (1990). The momentum of human behavior in a natural setting. Journal
of the Experimental Analysis of Behavior, 54, 163172.
doi:10.1901/jeab.1990.54-163
Mackintosh, N. J. (1974). The psychology of animal learning. London, England: Academic Press.
Mackintosh, N. J. (1975). A theory of attention: Variations
in the associability of stimuli with reinforcement.
Psychological Review, 82, 276298. doi:10.1037/
h0076778
Mackintosh, N. J., & Little, L. (1969). Intradimensional

and extradimensional shift learning by pigeons.
Psychonomic Science, 14, 56.
Madden, G. J., & Bickel, W. K. (Eds.). (2010). Impulsivity:
The behavioral and neurological science of discounting. Washington, DC: American Psychological
Association. doi:10.1037/12069-000
Maki, W. S., & Leith, C. R. (1973). Shared attention
in pigeons. Journal of the Experimental Analysis of
Maki, W. S., & Leuin, T. C. (1972). Informationprocessing by pigeons. Science, 176, 535536.
doi:10.1126/science.176.4034.535
Mazur, J. E. (2001). Hyperbolic value addition and general models of animal choice. Psychological Review,
108, 96112. doi:10.1037/0033-295X.108.1.96
Mazur, J. E. (2006). Learning and behavior (6th ed.).
Upper Saddle River, NJ: Prentice Hall.
McClure, S. M., Laibson, D. I., Loewenstein, G., &
Cohen, J. D. (2004). Separate neural systems value
immediate and delayed momentary rewards. Science,
306, 503507. doi:10.1126/science.1100907
Nevin, J. A., & Grace, R. C. (2000). Behavioral momentum

and the law of effect. Behavioral and Brain Sciences, 23,
7390. doi:10.1017/S0140525X00002405
Nevin, J. A., Milo, J., Odum, A. L., & Shahan, T. A.
(2003). Accuracy of discrimination, rate of
responding, and resistance to change. Journal of
doi:10.1901/jeab.2003.79-307
Nevin, J. A., Tota, M. E., Torquato, R. D., & Shull, R. L.
(1990). Alternative reinforcement increases resistance to change: Pavlovian or operant contingencies?
359379. doi:10.1901/jeab.1990.53-359
Norman, D. A. (1968). Toward a theory of memory
and attention. Psychological Review, 75, 522536.
doi:10.1037/h0026699
Norman, D. A., & Shallice, T. (1986). Attention to action:
Willed and automatic control of behavior. In R.
J. Davidson, G. E. Schwartz, & D. Shapiro (Eds.),
Consciousness and self-regulation (Vol. 4, pp. 118).
New York, NY: Plenum.
Pashler, H. E. (1998). The psychology of attention.
Cambridge, MA: MIT Press.
Pavlov, I. P. (1927). Conditioned reflexes. Oxford,
England: Oxford University Press.
Pearce, J. M., & Hall, G. (1980). A model for Pavlovian
learning: Variations in the effectiveness of conditioned but not of unconditioned stimuli.
Psychological Review, 87, 532552. doi:10.1037/
0033-295X.87.6.532
Posner, M. I. (1980). Orienting of attention. Quarterly
Journal of Experimental Psychology, 32, 325.
doi:10.1080/00335558008248231
Nestor, P. G., & ODonnell, B. F. (1998). The mind adrift:

Attentional dysregulation in schizophrenia. In R.
Parasuraman (Ed.), The attentive brain (pp. 527546).
Quick, S. L., & Shahan, T. A. (2009). Behavioral momentum of cocaine self-administration: Effects of frequency of reinforcement on resistance to change.
Behavioural Pharmacology, 20, 337345. doi:10.1097/
FBP.0b013e32832f01a8
Nevin, J. A. (1974). Response strength in multiple schedules. Journal of the Experimental Analysis of Behavior,
21, 389408. doi:10.1901/jeab.1974.21-389
Rachlin, H. (1976). Behavior and learning. San Francisco,

CA: Freeman.
Nevin, J. A. (1992). An integrative model for the study of

behavioral momentum. Journal of the Experimental
jeab.1992.57-301
Nevin, J. A., Davison, M., Odum, A. L., & Shahan, T.
A. (2007). A theory of attending, remembering,
and reinforcement in delayed matching to sample.
285317. doi:10.1901/jeab.2007.88-285
Nevin, J. A., Davison, M., & Shahan, T. A. (2005). A theory of attending and reinforcement in conditional discriminations. Journal of the Experimental Analysis of
408
Rachlin, H., Battalio, R., Kagel, J., & Green, L. (1981).

Maximization theory in behavioral psychology. Behavioral and Brain Sciences, 4, 371417.
doi:10.1017/S0140525X00009407
Rachlin, H., Green, L., Kagel, J. H., & Battalio, R. (1976).
Economic demand theory and psychological studies
of choice. In G. Bower (Ed.), The psychology of learning and motivation (Vol. 10, pp. 129154). New York,
NY: Academic Press.
Raymond, J. E., & OBrien, J. L. (2009). Selective visual
attention and motivation: The consequences of value
learning in an attentional blink task. Psychological
Science, 20, 981988. doi:10.1111/j.1467-9280.2009.
02391.x
Rescorla, R. A. (1988). Pavlovian conditioning: Its not

what you think it is. American Psychologist, 43,
151160. doi:10.1037/0003-066X.43.3.151
Rescorla, R. A. (1998). Instrumental learning: Nature and
persistence. In M. Sabourin, F. Craik, & M. Robert
(Eds.), Advances in psychological science: Vol. 2.
Biological and cognitive aspects (pp. 239257). Hove,
England: Psychology Press.
Rescorla, R. A., & Wagner, A. R. (1972). A theory of

Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In A.
H. Black & W. F. Prokasy (Eds.), Classical conditioning II: Current research and theory (pp. 6499). New
York, NY: Appleton-Century-Crofts.
Riley, D. A., & Leith, C. R. (1976). Multidimensional
psychophysics and selective attention in animals.
Psychological Bulletin, 83, 138160. doi:10.1037/
0033-2909.83.1.138
Riley, D. A., & Roitblat, H. L. (1978). Selective attention
and related cognitive processes in pigeons. In S. H.
Hulse, H. Fowler, & W. K. Honig (Eds.), Cognitive
processes in animal behavior (pp. 249276). Hillsdale,
NJ: Erlbaum.
Roberts, W. A. (1998). Principles of animal cognition.
Boston, MA: McGraw-Hill.
Santi, A., Grossi, V., & Gibson, M. (1982). Differences in
matching-to-sample performance with element and
compound sample stimuli in pigeons. Learning and
Motivation, 13, 240256. doi:10.1016/0023-9690
(82)90023-6
Schneider, S. M., & Davison, M. (2005). Demarcated
response sequences and generalized matching.
Behavioural Processes, 70, 5161. doi:10.1016/
j.beproc.2005.04.005
Shahan, T. A. (2010). Conditioned reinforcement and
response strength. Journal of the Experimental
jeab.2010.93-269
Shahan, T. A., & Burke, K. A. (2004). Ethanolmaintained responding of rats is more resistant to
change in a context with added non-drug reinforcement. Behavioural Pharmacology, 15, 279285.
doi:10.1097/01.fbp.0000135706.93950.1a
Shahan, T. A., Magee, A., & Dobberstein, A. (2003).
The resistance to change of observing. Journal of
doi:10.1901/jeab.2003.80-273
Shahan, T. A., & Podlesnik, C. A. (2005). Rate of conditioned reinforcement affects observing rate but
not resistance to change. Journal of the Experimental
Analysis of Behavior, 84, 117. doi:10.1901/jeab.
2005.83-04
Shahan, T. A., & Podlesnik, C. A. (2006). Divided attention performance and the matching law. Learning and
Behavior, 34, 255261. doi:10.3758/BF03192881
Shahan, T. A., & Podlesnik, C. A. (2007). Divided attention and the matching law: Sample duration affects
sensitivity to reinforcement allocation. Learning and
Behavior, 35, 141148. doi:10.3758/BF03193049
Shahan, T. A., & Podlesnik, C. A. (2008a). Conditioned
reinforcement value and resistance to change. Journal
of the Experimental Analysis of Behavior, 89, 263298.
doi:10.1901/jeab.2008-89-263
Shahan, T. A., & Podlesnik, C. A. (2008b). Quantitative
analyses of observing and attending. Behavioural
Processes, 78, 145157. doi:10.1016/j.beproc.
2008.01.012
Shahan, T. A., Podlesnik, C. A., & Jimenez-Gomez, C.
(2006). Matching and conditioned reinforcement
rate. Journal of the Experimental Analysis of Behavior,
85, 167180. doi:10.1901/jeab.2006.34-05
Shahan, T. A., & Quick, S. (2009, May). Reinforcement
probability affects adjusting sample duration in
a divided-attention task. In M. Davison (Chair),
Divided attention, divided stimulus control. Symposium
conducted at the 35th annual meeting of the
Association for Behavior Analysis International,
Phoenix, AZ.
Staddon, J. E. R. (1983). Adaptive behavior and learning.
New York, NY: Cambridge University Press.
Styles, E. A. (1997). The psychology of attention. East
Essex, England: Psychology Press. doi:10.4324/
9780203690697
Sugrue, L. P., Corrado, G. S., & Newsome, W. T. (2004).
Matching behavior and the representation of value
in the parietal cortex. Science, 304, 17821787.
doi:10.1126/science.1094765
Sutherland, N. S., & Mackintosh, H. J. (1971).
Mechanisms of animal discrimination learning. New
York, NY: Academic Press.
Townshend, J., & Duka, T. (2001). Attentional bias associated with alcohol cues: Differences between heavy
and occasional social drinkers. Psychopharmacology,
157, 6774. doi:10.1007/s002130100764
Wagner, A. R., Logan, F. A., Haberlandt, K., & Price, T.
(1968). Stimulus selection in animal discrimination learning. Journal of Experimental Psychology, 76,
171180. doi:10.1037/h0025414
Williams, B. A. (1994). Conditioned reinforcement:
Neglected or outmoded explanatory construct?
Psychonomic Bulletin and Review, 1, 457475.
doi:10.3758/BF03210950
Wolfe, J. B. (1936). Effectiveness of token-rewards for
chimpanzees. Comparative Psychology Monographs,
12, 172.
Wyckoff, L. B., Jr. (1952). The role of observing responses
in discrimination learning: Part 1. Psychological
Review, 59, 431442. doi:10.1037/h0053932
409
Timothy A. Shahan
Yantis, S. (2000). Goal-directed and stimulus-driven

determinants of attentional control. In S. Monsell &
J. Driver (Eds.), Attention and performance: Vol. 18:
Control of cognitive processes (pp. 73103).
Zeaman, D., & House, B. J. (1963). The role of attention in

retardate discrimination learning. In N. R. Ellis (Ed.),
Handbook of mental deficiency: Psychological theory and
research (pp. 159223). New York, NY: McGraw-Hill.
410
Zentall, T. R., & Riley, D. A. (2000). Selective attention

in animal discrimination learning. Journal of General
Psychology, 127, 4566. doi:10.1080/0022130000
9598570
Zentall, T. R., Sherburne, L. M., & Zhang, Z. (1997).
Shared attention in pigeons: Retrieval failure does
not account for the element superiority effect.
Learning and Motivation, 28, 248267. doi:10.1006/
lmot.1996.0965

017 PDF

Diunggah oleh

Informasi Dokumen

Deskripsi Asli:

Judul Asli

Hak Cipta

Format Tersedia

Bagikan dokumen Ini

Bagikan atau Tanam Dokumen

Opsi Berbagi

Apakah menurut Anda dokumen ini bermanfaat?

Apakah konten ini tidak pantas?

Hak Cipta:

Format Tersedia

017 PDF

Diunggah oleh

Hak Cipta:

Format Tersedia

Chapter 17

Attention and Conditioned

Copyright American Psychological Association. Not for further distribution.

The ability to appropriately attend to the important

reinforcers (i.e., obtaining goals), rather than such

Copyright American Psychological Association. Not for further distribution.

seem several simultaneously possible

Attention and Conditioned Reinforcement

Copyright American Psychological Association. Not for further distribution.

relation between two environmental stimuli (see

Copyright American Psychological Association. Not for further distribution.

Although distinguishing between classical and

option. Herrnstein formalized this relation with the

where B1 and B2 refer to behavior allocated to the

where B is the rate of the measured behavior, R is the

Attention and Conditioned Reinforcement

Copyright American Psychological Association. Not for further distribution.

Thus, in its most general form, the matching law

Copyright American Psychological Association. Not for further distribution.

(e.g., reinforcement rates or magnitudes), presented

Allocation and Persistence of Attention

Attention and Conditioned Reinforcement

how consequences affect what traditionally falls

Copyright American Psychological Association. Not for further distribution.

Human Attention and Differential

Copyright American Psychological Association. Not for further distribution.

local-level 6s and ignored a global-level 5 in the

An additional eight neutral shapes were equally as

Copyright American Psychological Association. Not for further distribution.

Attention and Conditioned Reinforcement

high probability (.8) or a low probability (.2). The

shorter delay, but only for neutral stimuli or stimuli

Conditioning, Learning, and Attention

Copyright American Psychological Association. Not for further distribution.

Mackintoshs (1975) attention theory is based on

Figure 17.1. Schematic of a divided-attention

samples made up of just one of the elements (see

Attention and Conditioned Reinforcement

Copyright American Psychological Association. Not for further distribution.

related experiments have obtained a similar result

The Matching Law and Allocation of

that were of either two colors (hereinafter called

where S1corr and S1incorr refer to correct and incorrect

Copyright American Psychological Association. Not for further distribution.

where log dC and log dL refer to log d for color and

Figure 17.3. Generalized matching

Figure 17.4. Sensitivity (i.e.,

Copyright American Psychological Association. Not for further distribution.

Attention and Conditioned Reinforcement

accuracy was lower with the short samples, Shahan

Behavioral Momentum and the

Copyright American Psychological Association. Not for further distribution.

Figure 17.5. Schematic of an observingresponse procedure for pigeons. W = white;

key. If the VI schedule of food reinforcement is

(i.e., Shahan & Podlesnik, 2006, 2007). Thus, in

Copyright American Psychological Association. Not for further distribution.

Attention and Conditioned Reinforcement

suggest that behavioral momentum theory may

Conditioned Reinforcement and

Copyright American Psychological Association. Not for further distribution.

was differentially associated with reinforcement.

the allocation of attention. Research evaluating the