Critical appraisal for speech and language therapists: single subject design (SSD) study

The purpose of this tool is to provide a structured framework for reading and appraising reports of intervention studies which use a single subject (participant; case) experimental design. Such studies rank relatively low in the standard evidence hierarchy; however, they provide practical and flexible methods for testing interventions for which little or no empirical evidence base yet exists. You can use this tool for appraising case studies in general, but remember that these provide only anecdotal information. Only experimental studies contribute to the evidence base i.e. designs which involve replicable methods, pre-planned interventions and accurate, reliable measurement. Use the following questions to assess the extent to which you feel the study was well-designed, apt and relevant, how reliable (replicable) and valid (true) the results are and whether you could implement the intervention with your own clients. There is space for you to add your own notes. The prompts are there to highlight why an issue is important but they shouldnt limit your thinking. There is quite a lot of overlap between questions.

Study reference:

Important clinical question?

Did the study ask a clearly focused question, in terms of: the Participant(s) studied

the Intervention given

the Comparator/control(s) used

the Outcomes considered

Is the question important for your clinical practice?

SCED Q1

Participant(s)

Was the participants clinical history adequately described? Age Sex Aetiology Severity of impairment/impact Response to previous intervention

Adapted using the 11 domains of the SCED Scale (www.psychbite.com/docs/The_SCED_Scale.pdf) by Jennifer Reid, Speech and Language Therapy Service, NHS Fife. Cartoons by Fran commissioned by Speech and Language Therapy in Practice. Accompanies Reid, J. (2011) Journal Club 6: single subject designs, Speech & Language Therapy in Practice Winter, pp.18-21. Please attribute these sources if you use, distribute or adapt any of this material.

SCED Q2

Measurement

Does the study identify measures of target behaviours or performance that can be used to evaluate intervention success? What are they, and are they: Precise?

Repeatable?

Operationally defined (i.e. for the purposes of the study) so that change can be observed?

Reliable across different raters, or the same rater over time?

SCED Q3

Design

Was it appropriate to use a single subject design? What kind of single subject design was used? It should include 3 phases (e.g. Pretreatment; treatment; post-treatment) and use either: Reversal or withdrawal design (A-B-A) baseline established before treatment, performance measured again during treatment and again after treatment has been withdrawn, or switched to another goal.

Multiple baselines method across different behaviours where all are being measured but only one is being targeted at a time.

SCED Q4

Baseline Phase

Was an adequate baseline established before intervention commenced?

Was sampling sufficient in terms of: Length of baseline phase?

Frequency of sampling?

Number of sampling time points?

Can performance trends be detected (i.e. stable, fluctuating, deteriorating, improving)?

Adapted using the 11 domains of the SCED Scale (www.psychbite.com/docs/The_SCED_Scale.pdf) by Jennifer Reid, Speech and Language Therapy Service, NHS Fife. Cartoons by Fran commissioned by Speech and Language Therapy in Practice. Accompanies Reid, J. (2011) Journal Club 6: single subject designs, Speech & Language Therapy in Practice Winter, pp.18-21. Please attribute these sources if you use, distribute or adapt any of this material.

SCED Q5

Intervention phase

Can a treatment response be distinguished from fluctuations resulting from other factors? You are looking for: A clear picture of the range of normal fluctuation for each participant (from baseline sampling) so you can separate these from changes which may result from treatment.

Enough sampling during the treatment phase to capture direction and extent of any apparent treatment effect.

SCED Q6

Data record

Is data displayed to show variability? You should be able to see: good visual displays of raw data in e.g. tables or graphs.

how measures of performance of individual participants vary over the phases of the study.

SCED Q7

Reliability of measures

Control of observer bias. Are measures used reliable (i.e. consistent across time and assessor)?

Was inter-rater reliability established for any of the measures used?

Adapted using the 11 domains of the SCED Scale (www.psychbite.com/docs/The_SCED_Scale.pdf) by Jennifer Reid, Speech and Language Therapy Service, NHS Fife. Cartoons by Fran commissioned by Speech and Language Therapy in Practice. Accompanies Reid, J. (2011) Journal Club 6: single subject designs, Speech & Language Therapy in Practice Winter, pp.18-21. Please attribute these sources if you use, distribute or adapt any of this material.

SCED Q8

Independence of assessors

More control of observer bias. Were assessors independent? Check whether assessors: Were different personnel from those who delivered the intervention

Rated or assessed blind to participant and/or the phase of the study

SCED Q9

Analysis

Was any statistical analysis of data undertaken?

How did the statistical analysis comparing results across the different phases of the study and did it demonstrate an intervention effect?

Did the authors provide a rationale for their selection of statistical technique?

SCED Q10

Evidence of replication

Is evidence provided that any intervention effect can be replicated? Has the effect been demonstrated: with other clients?

with different therapists?

in other settings?

Adapted using the 11 domains of the SCED Scale (www.psychbite.com/docs/The_SCED_Scale.pdf) by Jennifer Reid, Speech and Language Therapy Service, NHS Fife. Cartoons by Fran commissioned by Speech and Language Therapy in Practice. Accompanies Reid, J. (2011) Journal Club 6: single subject designs, Speech & Language Therapy in Practice Winter, pp.18-21. Please attribute these sources if you use, distribute or adapt any of this material.

SCED Q11

Generalisation and carryover

Is evidence provided for generalisation or carryover? For example: From improvements in knowledge/skill to improvements in functional communication?

From improvements in activity to increased social participation?

From changes in attitude to improvements in function, activity or participation?

Impact

Is the intervention described in enough detail for me to replicate it?

Is it potentially relevant for local service?

You can now use the SCED Scale to give the study a score out of 11

Adapted using the 11 domains of the SCED Scale (www.psychbite.com/docs/The_SCED_Scale.pdf) by Jennifer Reid, Speech and Language Therapy Service, NHS Fife. Cartoons by Fran commissioned by Speech and Language Therapy in Practice. Accompanies Reid, J. (2011) Journal Club 6: single subject designs, Speech & Language Therapy in Practice Winter, pp.18-21. Please attribute these sources if you use, distribute or adapt any of this material.