Observing Sniffy -

Shaping and Extinction under an

FR1 Schedule and Its Application to
Current Research

Student: Alan Cummins 1165236 Course: PSY286 Lecturer: Dr. Sinead Eccles
Contents

Account of Cumulative Records for Shaping and Extinction .......................................................... 3

Observation Sheet 1 – Shaping Sniffy Using FR1 Schedule ............................................................ 6
Observation Sheet 2 – Extinction.................................................................................................... 7
Observational Record Behavioral Repertoire Key Guide................................................................ 8
Cumulative Records ........................................................................................................................ 9
Shaping and Extinction Using an FR1 Schedule – Application in Current Research ..................... 11
References .................................................................................................................................... 18

Student: Alan Cummins 1165236 Course: PSY286 2 of 20

Account of Cumulative Records for Shaping and Extinction

Sniffy had previously been shaped to associate the sound of the foot-dispenser with the

dropping of a food pellet. That is Sniffy had already been magazine trained. There were two

reinforcers in use: Food as the primary reinforcer and sound as the secondary reinforcer. The

operant-level or base-line level behaviour for Sniffy was that of mixed exploration, sniffing and

rearing up against the cage walls. This operant level had its rate and pattern of emitted

responses noted. Based on Skinner, 1938 operant conditioning was carried out, where the

animal was presented with a reinforcing stimulus, that of food, immediately following the

occurrence of the given response, namely button-pressing.

The cumulative records (See Figure 1 and Figure 2) consist of sections split into 5 minute

segments with 75 responses till he market returns to its base position. Reinforced responses

are marked by short oblique lines in the records.

Shaping (See Figure 1)

Conditioning occurs if and only if the response rate increases in rate of occurrence, magnitude

or relative frequency or decreases in latency as a consequence of the conditioning. The shaping

of Sniffy was carried out using reinforcement of successive approximations of the desired target

behaviour. As indicated in Table 1 actions such as rearing up, paw lifting and so on were

reinforced to train Sniffy to move closer to the food dispenser and finally to associate the action

of bar pressing with the reward of food. Looking at Figure 1 it can be seen that Sniffy began

with occasional spontaneous bar presses causing food dispensation but spent most time

Student: Alan Cummins 1165236 Course: PSY286 3 of 20

exploring and grooming. As Sniffy began to move towards behaving in a desirable manner he

was reinforced with food pellets. The action strength increased thereby strengthening the

linking of behaviour to obtaining food. Four major behavioural changes can be seen on the

cumulative chart. There was an increase in response frequency, noted by the number of

reinforced responses as the training progressed and also by the steepness of the cumulative

line. Other behaviour such as locomotion and grooming decreased as paw lifting and bar

pressing increased. There was sequential change in the responding. That is there was a steady

rate of advancement to the desired behaviour. Finally there was a change in response

variability. Sniffy began carrying out many different behaviours but via shaping decreased these

down till eventually he was consumed with bar pressing alone by the end of the cumulative

report.

Extinction (See Figure 2)

Extinction is behaviour changes that occur when a previously reinforced behaviour no longer

produces reinforcement. Looking at Figure 2 the association of bar pressing and sound was

strongly associated with food dispensation. Once the bar press triggering release of food pellets

and the sound of the bar press itself were turned of, extinction began. There was an extinction

burst at the start where Sniffy repeatedly hit the bar. He doesn’t step down to examine the bar

or the cage because there is still a strong association with food and bar pressing. Extinction

occurs quickly as the steepness of the cumulative graph decreases rapidly. This is due to

continuous reinforcement having previously been used to shape Sniffy. A fixed ratio of one bar

press to one food pellet release means Sniffy immediately begins to notice when no food is no

Student: Alan Cummins 1165236 Course: PSY286 4 of 20

longer being delivered. There is a decline in response rate marked by progressive increases in

the frequency of relatively long periods of non-responding. In doing so there is an increase in

variability of the form and magnitude of the response. As indicated in Table 2 there is an

increase in exploration and grooming. This is due t the disruption in the loop of behaviour that

characterised the reinforced operant. By the end of the cumulative graph extinction has

completely occurred and Sniffy makes no bar press and spends his time exploring and

grooming. In this software based experiment it means that Sniffy has returned to his base-line

operant level.

Student: Alan Cummins 1165236 Course: PSY286 5 of 20

Observation Sheet 1 – Shaping Sniffy Using FR1 Schedule

Name: … Alan Cummins………………….. I.D. No: …1165236……………………Time of Session: 50 min.

Record of Behaviour from Operant Conditioning Training Session (See Table 3 for Key)

Reports are given for successive 5-minute periods

Schedule Description of Behaviour

0-5 min 1 BPE, General Behav Sequence: L, S, FT, HL, Predominant Behav: S, Comments: S
and L occurring, most actions with a single HSE
5-10 1 BPE, General Behav Sequence: L, S, FT, HL, PL Predominant Behav: L, Comments:
S and L occurring, most actions but increased PL
10-15 22 BPE, General Behav Sequence: S, FT, PL, Predominant Behav: PL, Comments:
Subject beginning to spend more time near food dispenser, raising paws and rearing
up but not always on the bar press area
15-20 31 BPE, General Behav Sequence: S, L, HL, FT, PL,BPE Predominant Behav: BPE, S,
Comments: More consistent actions mainly compromising S and BPE but occasional
wandering L, more focus on Food Dispenser
20-25 42 BPE, General Behav Sequence: S,HL, FT, PL,BPE,BPE Predominant Behav: BPE,
Comments: Mainly BPE now, occasional but rare S
25-30 47 BPE, General Behav Sequence: S,HL, FT, PL,BPE,BPE Predominant Behav: BPE,
Comments: Mainly BPE now, occasional but rare S
30-35 48 BPE, General Behav Sequence: S,HL, FT, PL,BPE,BPE Predominant Behav: BPE,
Comments: Mainly BPE now, occasional but rare S
35-40 47 BPE, General Behav Sequence: S,HL, FT, PL,BPE,BPE Predominant Behav: BPE,
Comments: Mainly BPE now, occasional but rare S
40-45 62 BPE, General Behav Sequence: BPE, Predominant Behav: BPE, Comments: As
expected, shaping has occurred
45-50 57 BPE, General Behav Sequence: BPE, Predominant Behav: BPE, Comments: As
expected, shaping has occurred, consistent pressing of bar press for food pellet
Table 1 - Observation Sheet 1 - Shaping

Student: Alan Cummins 1165236 Course: PSY286 6 of 20

Observation Sheet 2 – Extinction

Name: … Alan Cummins………………….. I.D. No: …1165236……………………Time of Session: 50 min.

Record of Behaviour from Operant Conditioning Training Session (See Table 3 for Key)

Reports are given for successive 5-minute periods

Schedule Description of Behaviour

0-5 min 40 BPE, General Behav Sequence:, Predominant Behav:, Comments:
5-10 Extinction Started, 55 BP, General Behav Sequence:, Predominant Behav: BP,
Comments: Extinction burst, rapid BP, as expected
10-15 18 BP, General Behav Sequence: PL, BP, S, FT, L, S, Predominant Behav: S, BP,
Comments: Beginning to lose interest in BP, more time spent S
15-20 4 BP, General Behav Sequence: S, FT, HL, Predominant Behav: S, Comments: A lot
of S, FT but still near food dispenser
20-25 2 BP, General Behav Sequence: S, FT, HL, L, S, L, Predominant Behav: S, Comments:
S and FT, HL and L away from food dispenser, starting to lose interest in back area of
cage where bar press is
25-30 2 BP, General Behav Sequence: L, S, FT, L, Predominant Behav: L, S, Comments:
Increased distance from food dispensing area
30-35 2 BP, General Behav Sequence: L, S, FT, L, Predominant Behav: L, S, Comments:
Increased distance from food dispensing area
35-40 2 BP, General Behav Sequence: L, S, FT, L, Predominant Behav: L, S, Comments:
Increased distance from food dispensing area
40-45 2 BP, General Behav Sequence:, Predominant Behav: L, Comments: L much more
frequent, as expected L and S has become more frequent due to no BPE or BP
45-50 0 BP, General Behav Sequence:, Predominant Behav:, Comments: No BP at this
stage extinction has fully occurred
Table 2 - Observation Sheet 2 – Extinction

Student: Alan Cummins 1165236 Course: PSY286 7 of 20

Observational Record Behavioral Repertoire Key Guide

Key Description
PL Paw Lift
FT Face Touch
S Sniff
L Locomotion
HSE Hopper Sniff, Eat
D Drinking
BP Bar Press
BPE Bar Press Eat
HL Head Lower
Table 3 - Sniffy Behavioral Repertoire Key

Student: Alan Cummins 1165236 Course: PSY286 8 of 20

Cumulative Records

Figure 1- Sniffy Cumulative Record Shaping

Student: Alan Cummins 1165236 Course: PSY286 9 of 20

 Figure 2 - Sniffy Cumulative Record Extinction

Student: Alan Cummins 1165236 Course: PSY286 10 of 20

Shaping and Extinction Using an FR1 Schedule – Application in Current
Research

The Sniffy experiment of Shaping and extinction does agree with previous research and in

applied situations. The following are criticisms and strengths of the experimentation carried out

and its application in the real world via previous research studies carried out.

Relation of Operant Conditioning to the Real World

The Sniffy experiment for shaping uses operant conditioning. Operant behaviour

constitutes a large proportion of everyday activities of humans. We are constantly emitting

operant behaviour. There is a high degree of similarity between humans and other species. The

Sniffy experiment can be related to previous research in animals and then in applied

behavioural analysis with human participants. Operant conditioning has important adaptive

mechanism that has been favoured by natural selection. While the Sniffy experiment makes use

of simple operant conditioning, it can be applied to simple operant conditioning of complex

human behaviour. Hall, Lund and Jackson, 1968 have carried out research on disruptive

children in school and now operant conditioning can be used to increase wanted behaviour.

Although many species in the studies such as pigeons, rats and dogs generate similar and

characteristic patterns of behaviour on simple reinforcement schedules humans are more

difficult to observe and evaluate. Humans have different verbal rules for describing

reinforcement contingencies that may be operating, therefore applying operant conditioning to

one or many humans becomes one of determining what the exact reinforcement stimulus is.

This can be overcome by shaping successive guesses which change both the verbal rule being

Student: Alan Cummins 1165236 Course: PSY286 11 of 20

formulated and the operant behaviour of the participants. See Catania, Matthews and Shimoff,

1990 for example of such. Bentall, Lowe and Beasty, 1985 have noted that young children

produce patterns of behaviour typical of other species. In applied settings such as classroom,

home, hospitals, institutions and society behavioural programs rely heavily on principles of

operant conditioning. It should be noted that Behaviourists, in the main, are interested in

‘learning’ rather than in instinct, the behavioural predispositions that an individual organism

possesses by virtue of its membership of a particular species. Research has begun to examine

ontogenetic influences on behaviour and this work should be considered in relation to operant

conditioning as a whole. Reinforcement as applied in Sniffy has been extended out into society

in many research studies. Garcia, Guess and Byrnes, 1973 looked at improving simple language

skills of retarded children by giving sweets as reinforcement when correct responses where

elicited. Cartwright and d’Orso, 1993 took an inner city school and transformed it into a

dynamic learning centre via behavioural shifts with operant conditioning. Tokenized economies

were put in place by Ayllon and Azrin, 1965 to promote positive and active engagement in

behaviour with institutionalised groups which had been suffering from lethargy, inaction and

depression. Hall et al, 1968 focused on disruptive children of 6-8 year old and made attention

and praise contingent on studying thereby sharply increasing the amount of study in class and

reducing the disruptive behaviour of the children.

Student: Alan Cummins 1165236 Course: PSY286 12 of 20

Schedule in Use

There are several different schedules of reinforcement that may be used within

behavioural analysis. The Sniffy experiment makes use of a continuous fixed ratio of one

reinforcement per desired behaviour. Other schedules include those such as a ratio scheduling

where reinforcement if given at fixed or intermittent points once a certain number of desired

behaviours have been noted, or interval scheduling where reinforcement is given after a certain

period of time has occurred. Intermittent reinforcement is generally of more benefit for

increasing positive behaviour than pure continuous reinforcement as used in Sniffy.

Intermittent reinforcement resembles many situations in every day life where behaviour is

maintained by a reinforcer that occurs only occasionally and is highly resistant to extinction.

While Sniffy had a steady fixed rate of reinforcement Cohen, Chelland, Ball and LeMura, 2002

have found that responses become faster as you increase the ratio requirement. This means

Sniffy could have been modified to use intermittent reinforcement where the amount of

desired behaviour before reinforcement would be increased till an optimum level at which

reinforcement rates at maximised. Stephens, Pear, Wray and Jackson, 1975 found that while

working with a child, Sidney that his learning rate increased as they increased the interval ratio

and peaked at a level of twenty desired behaviours per reinforcement. Continuous

reinforcement, however does have a place within applied behavioural analysis. Continuous

reinforcement if administered after every desired or correct response is the most effective

method of conditioning a new response. Continuous reinforcement can be used at the

beginning of a case to engage children in the desired behaviour. Partial reinforcement, a

pattern of reinforcement in which some portion of correct responses are reinforced can then

Student: Alan Cummins 1165236 Course: PSY286 13 of 20

be applied at a later date. Goldstein and Krasner, 1987 have carried out work on such. However

intermittent reinforcement have great utility for generating stable long term baselines of

learned behaviour against which effect of drugs, physiological manipulations, emotional stimuli

and motivational factors can be studied. Intermittent reinforcement produces reliable and

distinctive patterns of behaviour which are extremely resistant to extinction.

Extinction Alone

Extinction in the Sniffy experiment simply worked in isolation. The pairing of behaviour

and reward was removed immediately. The base-line operant level returned to its original level

as if no shaping had occurred. This is not the case in research using live animals or human

participants. There is never a full return to the base-level. Extinction is much more effective

when used in conjunction with reinforcement. Extinction alongside positive reinforcement was

used by Pinkston, Reese, LeBlanc and Baer, 1973 while observing preschool children and the

aggressive behaviour of one student called Cain. They ignored the aggressive behaviour,

essentially extinguishing the reinforcement of attention that he had previously been getting

and reinforced only whenever good behaviour was noted. Sniffy is an isolated and simple case

and it should be noted that resistance to extinction is high when a large number of responses

have been reinforced and is low when responses require great effort. Extinction can be

confused and obfuscated by behavioural processes such as fatigue, habituation, satiation and

punishment. Extinction brings with it increased behaviour in other areas once the reinforced

behaviour has been extinguished. Eckerman and Lanson, 1969 noted that response rates

decreased but that there was a change in behaviour as well as their experimental subjects,

Student: Alan Cummins 1165236 Course: PSY286 14 of 20

pigeons began to peck over a wider area upon extinction. Extinction as in Sniffy has been

applied in applied settings such Allyon, Michael, 1959 where there was a need to reduce visits

of a psychiatric patient to nurses, in France and Hudson, 1990 where children had difficulties

sleeping and parents where asked to extinguish attention to reduce awakening in middle of

night. That is extinction useful in situations in which reinforcer maintains behaviour can be

readily identified and controlled. However, extinction doesn’t develop new pro-social

behaviours to replace responses that have been extinguished. There can be increased

aggression if extinction occurs in an immediate and absolute way such as in Sniffy. Aggression

upon extinction of a shaped response within pigeons was noted by Azrin, Hutchinson and Hake,

1966 with previously unseen aggressive pecking taking the place of the extinguished behaviour.

Extinction is not recommended for behaviours that are dangerous or highly disruptive, such as

in self-harm. Sniffy gave example of an extinction burst. With self-harm it is too dangerous to

the client to simply ignore behaviours to try to extinguish them. Furthermore the amount of

responding in extinction is affected by number of reinforcers and the effortful ness of the

response during the previous period of reinforcement. Even more powerful influence on the

resistance to extinction is the schedule on which reinforcers were previously delivered. Partial

reinforcement extinction effect has been used to overcome persistent problem of treatment

gains that have not been maintained after behaviour intervention has been withdrawn. Kazdin,

1994, Nation and Woods, 1978 and Tierney and Smith, 1988 have all researched training at a

continuous schedule until it occurs at a high rate and then intermittent reinforcement is

invoked, which is where the schedule is incrementally gradually till the client is responding on a

Student: Alan Cummins 1165236 Course: PSY286 15 of 20

very thin reinforcement schedule. Kazdin and Polster, 1973 have used this with adults with

learning difficulties.

Behavioural Repertoire of Sniffy

Sniffy is a simplified version of a rat with a much reduced behavioural repertoire. This

makes it simpler to identify and classify actions and successive steps towards or from a desired

behaviour. The behaviour to be reinforced is close to an action that would be carried out

anyway. It becomes more difficult in applied situations. In such situations there is greater

difficulty in identifying rewards and punishments. Particularly in the case of humans whom

have different ideational concepts of what constitutes a reward for one as compared to

another. Primary reinforcers such as food in the Sniffy experiment are generally not used and

secondary reinforcers have to be identified instead. Reinforcers are identified by their function.

Primary or unconditioned reinforcers and conditioned reinforcers come into play with a

stimulus acquiring conditioned reinforcing value if it signals a reduction in the delay of

reinforcement, Fantino 1977. Sniffy suffers none of the emotional and physical limits that

hinder human behavioural analysis. Sniffy is never satiated, tired, bored or frustrated. The

reinforcer of food never grows old for Sniffy whereas in applied situations reinforcers do lose

value over time and need to be modified and extended over a period of analysis. Difficulties

arise in trying to make internal changes with human clients. The cycle of When, do, get must be

broken so as the behavioural analyst and in turn the primary care-giver do not become

Student: Alan Cummins 1165236 Course: PSY286 16 of 20

automaton reinforcers. Reinforcement is difficult within groups and it is important to identify

all possible sources of reinforcement.

Observational Report Ties to Research Data

Much research has been carried out into response shaping. Sniffy is shaped by

successively reinforcing closer and closer approximations to a desired behaviour, namely hat of

bar pressing. Pear and Legris, 1987 carried out response shaping by reinforcing successive

closer approximations of pigeons head to a spatial position. Similarly Midgley, Lea and Kirby,

1989 looked at shaping rats to deposit ball bearings by successive approximations of the

desired behaviour. This mimics what occurred within the Sniffy experiment carried out.

Response shaping is a highly important technique in the application of behavioural analysis to

human problems. Many such problems can be characterised as behavioural deficits. The

individual concerned does not succeed in making normal responses and thus his or her

behaviour is not maintained by the social consequences that influence the behaviour of others.

Galbicka, 1994 used response shaping in applied settings with the use percentile reinforcement

schedules while Howie and Woods, 1982 used shaping to increase fluency in stuttering. It

should be noted that a non-zero operant level is essential for simple operant conditioning. That

is there must be a base-line operant behaviour or set of behaviours that can be shaped. An

example of such is the many studies regarding speech development in children and adults as

compared to studies such as Hayes, 1951 which tried to shape apes to increase speech skills.

Student: Alan Cummins 1165236 Course: PSY286 17 of 20

References

Ayllon, T., and Michael, J. (1959). The psychiatric nurse as a behavioral engineer. Journal of the
Experimental Analysis of Behavior, 2, 323-334.

Azrin, N.H., Hutchinson, R.R., and Hake, D.F. (1966). Extinction-induced aggression. Journal of
the Experimental Analysis of Behavior, 9, 191-204.

Bentall, R.P., Lowe, C.F., and Beasty, A. (1985). The role of verbal behaviour in human learning.
Journal of the Experimental Analysis of Behavior, 43, 165-183.

Cartwright, M., and D’Orso, M. (1993). For the Children: Lessons from a Visionary Principal.
Doubleday.

Catania, A.C., Matthews, B.A., and Shimoff, E. (1990). Properties of rule-governed behaviour
and their implications. In D.E. Blackman and H. Lejeune (Eds.), Behaviour analysis in
theory and practice. Hillsdale, New Jersey:Erlbaum.

Cohen, S.L., Chelland, S.A., Ball, K.T., and LeMura, L.M. (2002). The effects of fixed-ratio
schedules of reinforcement on exercise by college students. Perceptual and Motor
Skills, 94, 1177-1186.

Eckerman, D.A., and Lanson, R.N. (1969). Variability of response location for pigeons responding
under continuous reinforcement, intermittent reinforcement and extinction. Journal of
the Experimental Analysis of Behavior, 12, 73-80.

France, K.G., and Hudson, S.M. (1990). Behavior management of infant sleep disturbance.
Journal of Applied Behavioral Analysis, 23, 91-98.

Galbicka, G. (1994). Shaping in the 21st century: Moving percentile schedules in applied settings.
Journal of Applied Behavioral Analysis, 27, 739-760.

Student: Alan Cummins 1165236 Course: PSY286 18 of 20

Garcia, E., Guess, D. and Byrnes, J. Development of syntax in a retarded girl using procedures of
imitation, reinforcement, and modelling. Journal of Applied Behavior Analysis, 6, 299-
310.

Goldstein, A.P., and Krasner, L. (1987). Modern applied psychology. New York: Pergamon.

Hall, R.V., Lund, D., and Jackson, D. (1968). Effects of Teacher attention on study behavior.
Journal of Applied Behavioral Analysis, 1,1-12.

Hayes, C. (1951). The ape in our house. New York: Harper and Row.

Howie, P.M., and Woods, C.L. (1982). Token reinforcement during the instatement and shaping
of fluency in the treatment of stuttering. Journal of Applied Behavioral Analysis, 15, 55-
64.

Kazdin, A.E., and Polster, R. (1973). Intermittent token reinforcement and response
maintenance in extinction. Behavior Therapy, 4, 386-391.

Kazdin, A.E. (1994). Behavior modification in applied settings. Fifth Edition. Pacific Grove,
California: Brooks/Cole.

Midgely, M., Lea, S.E.G., and Kirby, R.M. (1989). Algorithmic shaping and misbehaviour in the
acquisition of token deposits by rats. Journal of the Experimental Analysis of Behavior,
52, 27-40.

Nation, J.R., and Woods, P. (1978). Persistence: The role of partial reinforcement in
psychotherapy. Journal of Experimental Psychology:General, 109, 175-207.

Pear, J.J., and Legris, J.A. (1987). Shaping by automated tracking of an automated response.
Journal of the Experimental Analysis of Behavior, 47, 241-247.

Pinkston, E.M., Reese, N.M., LeBlanc, J.M., and Baer, D.M. Independent control of a pre-school
child’s aggression and peer interaction by contingent teacher attention. Journal of
Applied Behavior Analysis, 6, 115-124.

Student: Alan Cummins 1165236 Course: PSY286 19 of 20

Skinner, B.F. (1938). The Behaviour of Organisms. New York: Appleton-Century-Crofts.

Stephens, C.E., Pear, J.J., Wray, L.D., and Jackson, G.C. (1975). Some effects of reinforcement
schedules in teaching picture names to retarded children. Journal of Applied Behavioral
Analysis, 8(4), 435-447.

Tierney, K.J., and Smith, H.V. (1988). The effect of different combinations of continuous and
partial reinforcement on response persistence in mentally handicapped children.
Behavior Psychotherapy, 16, 23-37.

Student: Alan Cummins 1165236 Course: PSY286 20 of 20