Running Head: ARTICLE CRITIQUE 1

Article Critique: First Step to Success

Stephanie Janzen
EDPS 612.01
Prof. Yvonne Hindes
April 5, 2014












ARTICLE CRITIQUE 2

Study #1
The study entitled A Randomized Controlled Trial of the First Step to Success Early
Intervention: Demonstration of Program Efficacy in a Diverse, Urban School District measures
the effectiveness of this intervention (FSS) on first, second and third graders based within a
specific diverse urban setting. This population was chosen as it distinct from the more
homogenous and suburban setting of the original study that was done in the 1990s. This article
does not state their hypothesis but rather gives a short summary of the results in the abstract.
Although the reader can assume that the intervention might achieve positive results, an estimate
as the degree of effectiveness would be helpful.
This study utilized a cohort design model in that waves of intervention and usual care
comparison students participated in either of the two school years. These cohorts were selected
from 34 elementary schools of the Albuquerque public school system. Radom assignment
occurred at the classroom level as only 1 student was selected per classroom to participate in the
study. In Cohort 1, students were assessed using universal screening procedures (including the
Systematic Screening for Behavior Disorder), and then identified at-risk students (those who
might benefit from this specific intervention) were randomly assigned to either the intervention
of the usual care comparison condition. The same then consisted of 44 students in the usual care
comparison group and 55 inter intervention group.
The design of this study is effective in that it utilizes randomization, and employs three
waves of students, thus the results should be an accurate representation for that school district.
The sample size itself is adequate, and the range of ethnic background of the student population
ARTICLE CRITIQUE 3
is significant (57 % Hispanic, 24.5% Caucasian, 4.5 American Indian, 0.5% Asian, 7% Black,
3% multiracial, 3% unknown) so that it qualify as appropriately diverse. Though one should
always be wary in applying results taken from one city to another, as North American cities are
extremely variable in their racial and cultural make-up. For instance, there is not a large
Hispanic population in Vancouver B.C. yet this sample consisted of 57% students of Hispanic
origin.
The literature review addresses a number of different studies pertaining to at-risk
students. Specifically they refer to their adjustment during the early elementary years (or lack
thereof), the emergence of evidence-based practices pertaining to at-risk students, while also
noting that relatively few studies in educational literature have been based on designs besides
randomized controlled trials. They also note that there still remains an insufficient level of
evidence on the efficacy of early childhood interventions that address at-risk children and their
later outcomes. Though it should be noted that this study was done in 2009 and there have been
several studies on First Steps to Success in particular, that were not mentioned in this literature
review. Apart from this oversight, I found this review to be adequate.
The description of the procedures is comprehensive as it details every step of their study,
from obtaining permission from principals, to having teachers rank five students with the highest
levels of externalizing behavior and then to actually implementing the intervention. The authors
also include a flowchart outlining each phase of their process, which is useful in helping the
reader understand the steps that were taken.
In the Discussion section, the authors note that they are currently undertaking the 1-year
follow-up assessment for Cohort 2 participants and 2-year follow-up for Cohort 1. It is
ARTICLE CRITIQUE 4
unfortunate that the information from these follow-ups were not included in this study, as it
would have given the reader more insight pertaining to the long-term effects of this intervention.
Instead they note that a subsequent report will describe the sustainability issue. They then go on
to state that moderate to strong effects were achieved and so it was a good test of applicability to
minority children, though they emphasize that it is especially relevant for Hispanic students
considering the sample population. I appreciated this caveat rather than them claiming that it
could be applicable to any diverse population.
The study also includes a lengthy description of the limitations they found in
implementing this the intervention. For instance there was some level of variability in the
severity of risk between students as some teachers chose not to participate and several parents
declined to participate. They also admit that they cannot necessarily deem this intervention to be
effective, as they could not determine whether the results could be sustained across multiple
years, though they try to downplay this by stating that relatively few studies appear to meet this
criterion. It should also be noted that they did not mention any ethical concerns at all despite the
fact that this study used children and that not every at-risk child would benefit from this
intervention as they were selected at random.
Study #2
The study entitle First Step to Success Early Intervention Program: A Study of
Effectiveness with Native-American Children utilizes the same intervention as the first, though
on a much smaller and specific population. Their intention is to measure the effects of FSS on
the targeted students s problem behavior, class-wide behaviors, and teacher behaviors, though
no hypothesis is stated or estimation of the results. Rather, the authors give a short summary of
ARTICLE CRITIQUE 5
the results in the abstract, and then later state their research question. The do include a short
description of the dependent and independent variable, though this is stated in the Procedures
section rather than the introduction.
The literature review in this study seems to be comprehensive in that it discusses previous
research pertaining to the negative long-term effects of children demonstrating anti-social
behavior, and how anti-social behaviors generate more negative-aggressive behaviors than their
non-antisocial peers. The authors also cite studies which link antisocial behavior patterns to
urban, low SES minority youth, such as those students that are being studied in this article. They
also confess that there is limited information on antisocial behavior among Native American
youth, but they do cite sources that reveal that this population is associated with high rates of
alcohol use, shoplifting, vandalism etc. The authors also site several studies, which state the
effectiveness of Fist Steps to Success with other populations besides the one in the original
study. Thus, the authors create a short but distinct review of literature pertaining both to the
sample being studied and the intervention itself.
The sample for this population consisted solely of four students residing in a
Southwestern Indian reservation in Arizona. Two of these students were in kindergarten and two
were in grade one, three of which were male and one female. Then study then also included their
teachers and parents. It should also be noted that of the four teachers involved, three of them
were in their first year of teaching. This is an extremely small sample, which may then impact
the reliability of the results. Although the intention of this study was to involve Native American
children, (which would result in a relatively small sample due to their small population as a
whole), 4 students is troublesome as their results may not necessarily be replicated in other
similar populations.
ARTICLE CRITIQUE 6
The instrument used in this study was the Modified Partens Social Play Scale (which
measured social play behaviors and nonsocial play behaviors), the Revised Behavior Problem
Checklist-RBPC (which measured problem behaviors) as well as the Teacher Ratings of
Behavior (used to measure teacher behaviors). They also used unstructured interviews with the
participants teachers and parents to evaluate their perceptions of the interventions use and
effectiveness. These instruments are standardized and reliable, except for the interview portion,
which could produce variable results depending on the comfort, honesty and reliability of those
involved.
The study utilized a multiple-baseline across groups design, which was appropriate for
this sample, though it should be noted that it did not involve a waitlist or control group, thus their
results cannot be compared to a similar screened group. This is a notable flaw, especially as
results with such a small sample could possibly be due to confounding effects. Rather they are
basing their results on baseline screening, post-intervention, and follow-up data. It should also
be noted that follow-up data was done only two months after than FSS intervention ended as
opposed to any longer period of time such as one or two years. Thus, the follow-up data may not
necessarily be deemed as long-term effects.
The strength of this paper is the simple and structured explanation of the procedures.
Every phase is divided up into a separate paragraph with a corresponding title, though at the
same time, explanations of various phases might have been over simplistic. For example, the
intervention phase paragraph included only 2 sentences and did not go into any detail as to what
was actually involved in implementing FSS within this sample. But at the same time it did
include rather lengthy explanations pertaining to interrater reliability as well as their efforts at
ARTICLE CRITIQUE 7
maintaining treatment integrity. The results were also discussed at length and also were
accompanied by a graph and two charts.
In the Discussion portion of this article, the authors state that the results of the
intervention produced significant positive affect on all participant students social play
behaviors. They then go on to describe the more specific ways in which their behavior had
changed as according to both observational techniques and teacher rating scales. Despite the
claim that there were significant positive affects, the authors do state that the teacher and
parents of student two saw no change in play behavior. If the sample had been larger, than such
differences would likely not be mentioned. Yet because the sample only consisted of four
students, the authors felt it worthwhile to note that both the teacher and parent of this one student
seemed to blame each other for the apparent ineffectiveness of FSS. In my opinion, this was
appropriate to include as it emphasized some of the limitations of this intervention, that being the
investment and participation of both teachers and parents.
The article does include several paragraphs pertaining to the limitations of this study.
For instance they recognize that only judgment-based assessment procedures were used, and that
a form was utilized to assess the treatment fidelity of the class component only, and not the home
component of the intervention. Thus it was difficult for them to determine which component
influenced the changes to play behavior. The authors also include several paragraphs dedicated
to directions for future research, but they do not include any ethical concerns.



ARTICLE CRITIQUE 8

References
Diken, I. H., & Rutherford, R. B. (2005). First step to success early intervention program: A
study of effectiveness with Native-American children. Education & Treatment of
Children (ETC), 28(4).
Walker, H. M., Seeley, J. R., Small, J., Severson, H. H., Graham, B. A., Feil, E. G., ... & Forness,
S. R. (2009). A Randomized Controlled Trial of the First Step to Success Early
Intervention Demonstration of Program Efficacy Outcomes in a Diverse, Urban School
District. Journal of Emotional and Behavioral Disorders, 17(4), 197-212.