Staff Training Manual

Conducting Direct Preference

Assessments using Multiple
Stimuli Without Replacement
(MSWO)

Nana Doan and Parveen Begum

 TABLE OF CONTENTS

SECTION 1: TRAINING SKILLS FOUNDATION AND PRINCIPLES

Chapter 1 - Training Model
- What is Evidence Based Practices and Behavioural Skill Training?
- Components of Evidence Based Practices
- Performance-based and Competency-based
Chapter 2 - Preference Assessment and Multiple Stimuli Without Replacement
- What is preference assessment?
- Why assessment?
- What is Multiple stimulus without replacement (just def)?
- Who can perform the assessment with the target person?
- Who is the target person for the preference assessment?
SECTION 2: INSTRUCTIONS ON THE APPLICATION OF TRAINING
- Selecting environment
- Instructions on ‘set up’ for training session
- Direct instruction on the training procedure
- Direct instructions on prompting during training
- Direct instructions on reinforcement for correct responses of trainees
- Direct instructions on error correction
- What is performance checklist?
- Content of performance checklist
- Direct instructions on taking data using the performance chechklist
- (provide examples on how you (trainer) would use data from PC as feedback)
- Standards?
SECTION 3: EVALUATION OF THE TRAINING
- Direct instructions on administering the pre/post test
- Direct instruction on analyzing the data from pre/post test
- (graph)
- Direct instructions on how to collect direction observation data in/during the training
- Direct instructions on analyzing data from direct observations
- (graph on direct observations in/ during training)
- Tool for assessing social validity
- (graph/analysis of your social validity data
- Direct instructions on how to collect observation data at follow up
- Direct instructions on analyzing data from follow up
- (graph of direct observation at follow up)

SECTION 1: TRAINING SKILLS FOUNDATION AND PRINCIPLES

CHAPTER 1:
Model used for training
Evidence Based Practices​ ​(EBP)
or recently referred to as Behavioural Skill Training (BST) :the training is data-based; obtained
to document that trainees demonstrate the target skills at established proficiency criteria. This
method includes the following components: instructions, modeling, practice,and feedback (Reid
et al., 2003).

Components of EBP
Instructional method​: verbal information, lectures

Modeling​: in this method, the trainer demonstrates the correct application of procedure to
trainee(s), followed by an opportunity for the trainee to apply the same procedure to a particular
client (Jahr, 1998). Video modelling - In definition, it is a method of teaching that uses video
recording and display equipment to provide a visual demonstration of the targeted skill for the
learner. (Franzone, E., & Collet-Klingenberg, L., 2008).

Practice/Rehearsal​: The trainee practices/rehearses the procedures (Reid, et al., 2003).

Feedback​: A form of evaluation, either presented in oral and/or written format. The aim is to
improve the trainee performance (Jahr, 1998). Supportive feedback for accurate performances,
or corrective feedback for incorrect performances.

EBP protocol for conducting a training session consists of the following steps. See APPENDIX
A..

EBP model is performance based - “The description includes a specific means of conducting a
behavioral skills training session with a group of trainees followed by on-the-job training
requirements.”(Reid et al., 2003). In other words, being able to perform the skills correctly taught
after the training, rather than just being able to just talk about it.

EBP consists of performance and competency-based strategies (Reid et al., 2003).

The phrase ​performance-based​ refers to what the trainer and trainees do (i.e., actively
perform the specific responses being trained) during the training.

The phrase ​competency-based​ refers to the practice of continuing training until trainees
competently demonstrate the skills to safely and effectively perform a procedure(i.e.,
meet established mastery criteria). (Reid et al., 2003)

CHAPTER 2
What is preference assessment?
Stimulus preference assessment refers to a variety of procedures used to determine 1)
the stimuli that the target person prefers, 2) the preference values of those stimuli (high
 vs low), and 3) the conditions under which those preference values change
(cooper,275).

Preference assessment only helps identify the potential reinforcer temporarily.

Therefore, that stimuli is not necessarily reinforcing to them all the time.

Why assessment?
Everyone has their own preference. Doing a direct preference assessment helps direct
support workers of the individual to understand more accurately about the target person with
regards to their current wants and needs which is based on facts, rather than assumptions.
It is empowering to learn how to and have the ability make choices to help them find the
motivation to do something, specifically for individuals who are unable to support themselves
due to the severity of intellectual disability and/or physical disability and require assistance in
their life.
Preferences often change, day to day or even hour to hour.

What is MSWO?
In MSWO method, the chosen stimulus is removed from the array, the order or placement of the
remaining item is rearranged, and the next trial begins with a reduced # of items in the array
(Cooper, 279). Every instance of the target person indicates a preference among displayed
stimuli, the assessment taker will record it down on a MSWO data sheet. In the end, the
assessment taker will see the preference hierarchy of the most prefered stimulus (1st chosen) to
the least prefered (last chosen) stimulus.
MSWO identified preferred stimulus more accurately than single stimulus method, and required
half the time in comparison to the MSWR and paired stimulus methods (DeLeon and
Iwata,1996)

Who can perform the assessment?

Paraprofessional (special ed teachers, staff in group home setting, parents, RBTs) anyone with
at least high school education, who have proper training by a certified ABA trainer, and pass the
training assessment.

Who can have the assessment?

Children or young adults with autism, behavioural disorders, developmental disabilities, no vocal
skill, typically developing children, and even adults with dementia.

SECTION 2: INSTRUCTIONS ON THE APPLICATION OF TRAINING

Environments for the assessment

The preference assessment can be done at IB/ABA clinics, at school, and at home.

Instructions on ‘set up’ for training session

 - Trainer created a powerpoint on Multiple Stimuli without Replacement (MSWO)
preference assessment using the information in section 1, chapter 2 in point form.
Include the following video demonstration on MSWO:
https://www.youtube.com/watch?v=4hRet6DcJ10&t=161s
- Arranging chairs for the audience in semi circle facing a white board.
- Five different toys on a table located in front of the white board.
- Markers (red, black, blue) and a board wiper, used for illustrations if necessary.
- Printed copies of the powerpoint for trainees to be handed out at the beginning of
training session.

Direct instruction on the training procedure

Part 1: Trainer and trainees are present at the training session.
Part 2 (instructional): The trainer goes thru the PowerPoint on MSWO with trainees. The trainer
provides information about MSWO and the description of the behaviours required to perform the
preference assessment.
Part 3 (modeling): Once the lecture is complete (trainees have heard and read the description of
the actions to perform the skill), the trainer demonstrates the skill. The trainer provides detailed
instructions to the trainee to ensure the trainee knows exactly what should be done during the
demonstration. Ideally, the trainer should stop at certain points and ask the trainee what is being
done and why; from there, trainer knows what to clarify to the all the trainees.
Part 4 (rehearsal): Trainees rehearse performing the skill in a role play similar to the trainer
demonstration. The trainer will provide prompts where necessary. This step is also when
trainees ask questions and the trainer answers.
Part 5 (feedback): Have the trainees practice again and this time the trainer uses the
performance checklist to evaluate the trainees. Provide feedback at the end of each trainee’s
performance.

What is performance checklist?

A written task analysis with a short and clear description of all steps a trainee must perform to
complete a skill satisfactorily, used by a trainer to evaluate the trainee’s performance during
rehearsals (Reid & Parsons, 1995). A completed checklist is also an antecedent for the trainer
to respond in certain ways. See appendix 1 for content of the performance checklist for MSWO
preference assessment.

Content of the performance checklist for MSWO (this will be moved to appendix when
edited on MicrosW)
Step Action Yes No N/A

TRIAL 1

1 Place 5 items in a row with 5cm space in between on the table

2 Tell the target person “Pick/Choose one” and wait for him/her to
respond

3 If the person chooses one, allow him/her to interact with the chosen
item for 15 seconds

4 In the meantime, take a look at the data sheet:

Starting at trial 1, the assessor records the name of the chosen item,
circles “x” where the item was positioned

5 Place the blocker behind 4 items and moves the leftmost item over to
the rightmost position. Rearrange so that there is 5 cm of space in
between them

6 After 12-15 seconds, tell the person “All done” and retrieves the item
and puts it away

7 Remove the blocker

TRIAL 2

1 Tell the target person “Pick/Choose one” and wait for him/her to
respond

2 If the person chooses one, allow him/her to interact with the chosen
item for 15 seconds

3 In the meantime, take a look at the data sheet:

At trial 2, the assessor records the name of the chosen item, circles “x”
where the item was positioned

4 Place the blocker behind 3 items and moves the leftmost item over to
the rightmost position. Rearrange so that there is 5 cm of space in
between them

5 After 12-15 seconds, tell the student “All done” and retrieves the item
and puts it away

6 Remove the blocker

TRIAL 3

1 Tell the target person “Pick/Choose one” and wait for him/her to
respond

2 If the person chooses one, allow him/her to interact with the chosen
item for 15 seconds
 3 In the meantime, take a look at the data sheet:
At trial 3, the assessor records the name of the chosen item, circles “x”
where the item was positioned

4 Place the blocker behind 2 items and moves the leftmost item over to
the rightmost position. Rearrange so that there is 5 cm of space in
between them

5 After 12-15 seconds, tell the person “All done” and retrieves the item
and puts it away

6 Remove the blocker

TRIAL 4

1 Tell the target person “Pick/Choose one” and wait for him/her to
respond

2 If the person chooses one, allow him/her to interact with the chosen
item for 15 seconds

3 In the meantime, take a look at the data sheet:

At trial 4, the assessor records the name of the chosen item, circles “x”
where the item was positioned

4 Place the blocker behind the final item.

5 After 12-15 seconds, tell the person “All done” and retrieves the item
and puts it away

Accuracy of Performance Score

(#yes/#steps scored x 100)

Previous Performance Score

Direct instructions on taking data using the performance checklist

For every trainee’s rehearsal performance, the trainer will observe and assess them using the
performance checklist. Starting from trial 1 to trial 4, in order from the top to the bottom, if the
trainee correctly performed at any step, put a check mark under “yes” column; if incorrectly, put
a check mark under “no” column.
At the bottom of the performance checklist, calculate the percentage of the trainee’s
performance’s correctly completed steps by taking the total number of “yes” checkmarks divided
by the total number of steps across the four trials, then times 100.

Direct instructions on prompting during training

During the rehearsal, the trainee will perform all the steps listed in the performance checklist to
do the preference assessment. For instant, when the trainee performs a step incorrectly or they
forget the next step, the trainer will provide short, succinct verbal prompts to remind the
immediately the trainees or give suggestions on what to do.

Direct instructions on reinforcement for correct responses of trainees

The trainer will provide a short generic praise such as “good” so that they know they are doing
the step correctly and continue to perform the step in the same way in the future.
By the end of the practice, using the completed checklist, the trainer will then give specific,
detailed positive feedback and praises on the trainee’s accurate performance or what the
trainee has done well on their performance (i.e. “After you told the person to pick an item of
choice, you immediately took data instead of waiting to do it later despite you still have more
than 10 seconds. That is a good technique to avoid rushing to record at the end of 15
seconds”).

Direct instructions on error correction

During the practical training session, when the trainee performs a step incorrectly, the trainer will
give them a short feedback such as “no” and provide the verbal prompt to redirect the trainee.
By the end of the practice, the trainer will use the completed checklist to give corrective
feedback for incorrect or missing steps (i.e. “In all trials, when the person takes the item of
choice, you kept forgetting to take data on the item selected. So remember, as soon as the
person chooses the item, you immediately record the item on the data sheet, and do not wait
even if you have 12 seconds before the next step.)

How much training is required?

The trainer will train until the trainees meet all mastery criteria (100%) on the performance
checklist.
The first trainee’s evaluation during the rehearsal, the trainee is expected to get 100% correct
on all steps required in the skill performance checklist.
If the trainee fails, the trainer will provide feedback on errors and that he/she will be trained
again later.

Section 3: Evaluation of the training

Pre/post-test on the knowledge of your procedure

Pre/post test is a method of evaluating the quality of the training, but mainly reflecting the
trainees’ knowledge prior to and after the training that are completed by the trainees.
The pre and post tests are printed double sided.

Administering the pre/post test

At the beginning of the training session, the trainer will hand out the pre-test sheets to the
trainees. Give them about 5 minutes to complete the pre-test. At the end of the procedure,
instruct them to complete the Post-test on the flip side. Give them 5 minutes to complete.

Analyzing the data from the pre/posttest.

Calculate the percentage of the correct answers achieved by each of the trainee by dividing the
correct answers over the total questions of pretest (before the training) and posttest (after the
training). See APPENDIX C.

Graph of pre/post test data

See APPENDIX D.

How to collect direct observation data in or during the training.

With the performance checklist in hand, start observing and evaluate each of the trainee from
the beginning of their rehearsal until the end of their rehearsal.
Read the instruction on how to do the performance checklist above.

Analyzing data from direct observations.

Complete the performance checklist (see APPENDIX B) for each trainee by the end of their
rehearsals. Calculate the percentage of the correct responses by dividing the total checkmarks
on correct responses (Yes) by the total steps of the performance checklist.
I.e 15/24 x 100 = 62.5%
If it is the first training, do not mind about “previous score” section. If it is not, record the score
achieved in the last training session.
If any trainees scored below 90%, they need to be trained again on a different scheduled date. If
above, no further training needed.

Graph of direct observations in or during your training.

See APPENDIX D.

“Mechanism” to assess social validity or “satisfaction”.

Satisfaction survey sheets will be handed out to the trainees after the training session.

Graph/analysis of your social validity/satisfaction data.

See APPENDIX E.

How to collect direct observation data at follow up.

The trainer will collect another direct observation data at follow up 6 months since the previous
training session using the performance checklist.
Read the instructions on how to do the performance checklist

Analyzing data from follow up.

Read the instructions on how to analyze direct observation data above.

Graph of direct observations at follow up.

See APPENDIX D.
Appendix

APPENDIX A. Parsons, M. B., Rollyson, J. H., & Reid, D. H. (2012). Evidence-Based Staff Training: A Guide for
Practitioners. ​Behavior Analysis in Practice,​ ​5​(2), 2–11.

Trainees Baseline Post-training

 Jill 16% 83%

Becky 33% 83%

Anne 66% 100%

Josh 50% 100%

Emily 33% 100%

APPENDIX B.

PRE/POST TEST

APPENDIX C.

APPENDIX D
 .
APPENDIX E