EDPS 612.

03
Learning Task 2
Research Article Critique

Amy Donovan
June 26, 2014

Knox, M., Burkhart, K., & Cromley, A. (2013). Supporting Positive Parenting in
Community Health Centres: The Act Raising Safe Kids Program. Journal of
Community Psychology, Vol. 41, No.4, 395-407. Doi: 10.1002/jcop.21543
In this article, Knox, Burkhart & Cromly present and discuss their study which evaluates
the effectiveness of the ACT Raising Safe Kids (ACT-RSK) program on parenting outcomes for
families served by Community Health Centers. The ACT-RSK program is a primary prevention
program that addresses family violence and child physical abuse. Parents/caregivers of young
children attending the 8 session program are trained in effective parenting strategies (i.e.
nonviolent discipline, anger management, problem-solving skills) and are taught how to manage
their anger and resolve conflicts in a nonviolent manner. The authors of this study hypothesize
that parents/caregivers who receive services at CHCs and who complete the ACT-RSK program
will demonstrate significantly improved scores on measures of parental nurturing, positive
parenting, and nonviolent discipline relative to the control group. It was additionally
hypothesized that those who complete the program would have lower rates of psychologically
and physically aggressive behavior toward their children.
Knox, Burkhart & Cromly provide a thorough literature review that conveys the
knowledge and ideas established on their topic. It relates directly to their thesis, detailing
statistical information related to the prevalence of child maltreatment in the U.S., as well as that
demonstrating the link between child maltreatment and long-term health, behavior, and learning
problems. They present a review of the research pertaining to the benefits of prevention and
intervention strategies, aimed at both parents and caregivers, to support them in the
establishment of stable and nurturing environments and the development of positive parenting
skills. The qualities of effective programs were researched and authors conclude that those which
result in positive outcomes for children and families not only educate, but support and guide
parents as well and ideally should be embedded within existing service systems.
With the research pointing to the importance of embedded primary prevention strategies,
the authors suggest targeting Community Health Centers (CHCs). These neighborhood health
clinics which provide medical and mental health services are commonly utilized by low-income
families, a demographic found to have higher rates of child maltreatment. Knox, Burkhart &
Cromly suggest that CHCs are the ideal setting for prevention-based parenting programs as they
are familiar settings that are easily accessible for families. As such, it is their belief that accessing
the program through CHCs will address issues related to engagement, retention and attrition
which typically plague such programs offered in alternative settings.
Researchers point to the strengths of the ACT-RSK program as rationale for its inclusion in
their study. The manual, available in both English and Spanish, includes detailed, sequential
instructions for facilitators with facilitation training provided. This improves the fidelity with

which the program is implemented. The program itself is adaptable for a variety of circumstances
and can be implemented by an organization at low cost and offered free to parents/caregivers.
Previous studies of the ACT-RSK program show improved parent knowledge in anger
management, social problem-solving, nonviolent discipline, media literacy and positive
parenting practices following participation in the program.
Two CHC sites were chosen for this study, one in the Eastern U.S. and one in the
Midwestern U.S. Members of the research team actively recruited participants over the period of
one year. CHC staff were asked to refer families, flyers were handed out and presentations about
the program were given by members of the research team. A grocery card was provided as
incentive for families to participate. Inclusion criteria required participants to be living with or
currently caring for a child aged newborn-8 years while those with severe mental illness, those
currently abusing illegal substances or those currently receiving mental health services were
excluded. Informed consent was obtained from participants.
Participants were randomly divided into two groups, 38 in the intervention group and 46
in the control group. Participants in the control group were required to complete survey
measures but did not attend the ACT-RSK program while those assigned to the intervention
group were required to complete both the surveys and the program. Child care was made
available and transportation provided.
Researchers engaged in a summative program evaluation which looked at the outcomes
of the ACT-RSK program. Using A-B design, baseline data was collected using surveys
measures. While researchers aimed to determine whether parental nurturing, use of positive
parenting and nonviolent discipline improves as a result of participation in the ACT-RSK
program offered through CHCs, the A-B design does not allow researchers to draw confident
conclusions. Specifically, the A-B design does not allow researchers to rule out the possibility
that other factors beside participation in the ACT-RSK program, could account for change. As
Royce, Thyer & Padgett suggest, causal conclusions should be interpreted with caution when AB designs are used (Royce, Thyer & Padgett, 2006). For example, could improvement be
coincidental? Could the fact that parents/caregivers were seeking support from their CHC prior
to entering the study impact outcomes? Another weakness of the A-B design is the fact that it
does not allow for the measurement of long-term outcomes. Alternatively, a study that is
longitudinal in design would allow researchers to determine long-term outcomes of the ACTRSK program. This design would appear to be most valuable in order to determine the
effectiveness of the program. Participant survey responses that demonstrate learning and/or a
change in thinking following participation in the ACT-RSK program is less interesting than a
study that demonstrates actual changes in parenting practices that are maintained over time.

Multiple measures were used by researchers. The following were administered to

participants both before and after participation in the ACT-RSK program or only once in the case
of the control group: The Aggressive Behavior subscale of the Child Behavior Checklist, Parent
Behavior Checklist as a measure of parents' nurturing behaviors (high internal consistency &
test-retest reliability,) Parent-Child Conflict Tactics Scale to assess the way in which parents
respond to their children's behaviors, the Non-Violent Discipline subscale (4 items), the
Psychological Aggression subscale (5 items), the Physical Assault-Minor Assault subscale (4
items) and the ACT Parenting Behaviors Questionnaire (developed for purposes of this study).
The authors detailed the reliability and validity of each measure within their article and openly
discuss the limitations of designing their own questionnaire. Specifically, the lack of established
psychometric properties.
Results show a correlation between positive outcomes and the ACT-RSK program
delivered through CHCs. Particularly, participants reported increased rates of nurturing and
positive parenting and decreased rates of psychologically and physically aggressive behavior
toward their children immediately following program participation. However, it is not clear
whether these results are maintained long-term. Moreover, the degree to which the results are
generalizable is questionable as the study is relatively small, with a total of 84 participants.
Researchers report adhering to ethical guidelines. Participation was voluntary, informed
consent was required and freedom to withdraw was respected. Deception was not used as part of
the study and no harm resulted from participation. It should be noted that participants assigned to
the control group were not denied access to the ACT-RSK program but were given the
opportunity to participate following completion of the study. One might question whether the
grocery card incentive represents undue inducement however the amount was relatively
insignificant at $25.00. Researchers suggest that standard recruitment procedures were used for
this study yet, the degree to which the team engaged in active recruitment appears somewhat
aggressive. If this was the case, there may have been participants who felt coerced into
participating in the program. Attrition rates could be indicitative of this (149 parents/caregivers
consented to participate yet only 84 completed participation).
Though ethnically diverse, (16% Caucasian, 32% African American, 43% Latino, 1%
American Indian, 2% biracial) the sample population included a high percentage of Latino
parents, followed by African American. This likely was the result of the communities chosen. As
such, though participants were randomly assigned to the intervention or control group, the
population from which participants were drawn does not appear to be representative. The
inclusion of only two CHCs creates challenges to validity and reliability. An expanded study,
including a range of CHCs may have been more representative of the population as a whole.

Overall, Knox, Burkhart & Cromly appear to have thoroughly reviewed the literature
relevant to their topic of research. They provide a clear hypothesis and research design.
However, this study is not without weaknesses as have been pointed out throughout this
discussion. The adaptations suggested may be valuable considerations for future studies on this
worthwhile topic.
Article 2
Flower, A., McKenna, J., Muething, C.S., Pedrotty Bryant, D., & Bryant, B. (2013). Effects
of the Good Behavior Game on Class-wide Off-Task Behavior in a High School
Basic Algebra Resource Classroom. Behavior Modification, Vol. 38 (1), 45-68. Doi:
10.1177/0145445513507574
In their research, Flower et al. examined the impact of the Good Behavior Game (GBG)
on the off-task behavior of grade 9 students with high-incidence disabilities. Specifically, their
research focused on four questions: What is the effect of the GBG on class-wide off-task
behavior for high school student with disabilities who receive basic algebra instruction in a
resource room setting? In the follow-up phase, to what extent does the teacher continue to
implement the GBG procedures without researcher involvement? Are the effects on class-wide
off-task behavior maintained during the follow-up phase? And lastly, what is the student/teacher
rated social validity of using GBG?
The GBG is an evidence-based classroom intervention which involves clearly identifying
target behaviors, classroom rules and reinforcers. The class is divided into at least two teams who
compete against each other for points which can later be redeemed for daily &/or weekly prizes.
Points are awarded based on teams meeting the identified behavior criteria or deducted if criteria
is not met. The GBG provides teachers with the opportunity to acknowledge appropriate
behavior, and provide feedback and reinforcement to students. A proactive intervention, it has
been linked to improved behavior and increased student engagement.
Flower et al. present a comprehensive overview of the relevant research that is
appropriate and useful for the reader. They begin by presenting data related to teacher time spent
managing disruptive behavior in the classroom and the impact on teaching and learning; with one
study suggesting that up to 40% of teachers reported spending more time managing disruptive
behavior than teaching curriculum. Detailed research examining student engagement reveals
great variability in time on-task, with research showing that at any given time, 13-50% of
students are off-task thus highlighting the need for teachers to have effective
strategies/interventions to manage their classrooms and engage students in their learning.
Classroom management strategies/interventions, in particular group contingencies, have been
well researched. Placing less demand on teacher time and effort, group contingencies can address

the behavior of all students simultaneously and may be especially relevant for high school
students who are significantly influenced by the behavior of their peers. Flower at al. report that
most of the literature related to the GBG has been done with elementary students, with only two
reporting use with high school students. Furthermore, no previous study examined use of the
GBG with high school students with disabilities in a typical public school setting.
Flower et al. selected a public high school in Central Texas as the setting for their
research. The school was within a largely Hispanic (82%) suburban district where 87% of
students were considered economically disadvantaged. Participants included a female, Caucasian
teacher and students in two ninth-grade algebra resource classes. The majority of the students
were Hispanic, male and had been identified with a specific learning disability. The choice of
participants could pose potential problems for generalizability of results as culturally, the sample
is not representative of the population as a whole. Moreover, the classroom behavior of Hispanic
males could possibly be impacted by the fact that their teacher is a Caucasian female whom they
may believe has little understanding of their background and/or life experience. Researchers
might have sought a classroom with greater cultural diversity or have repeated the study in a
variety of classrooms across districts in order to improve the validity of results. This would also
allow for increased teacher participation beyond the single teacher who participated in this
research. Similarly, the resource room setting for this study may impact external validity. Given
the focus on inclusive educational environments, it is no longer common practice to place
students with a resource teacher. Rather, students with learning disabilities are educated with
their peers in the regular classroom setting. As such, the results of this study may not be
generalizable beyond this exclusive setting.
An ABAB reversal design was used by Flowers et al. (Baseline-GBG1-Reversal-GBG2Follow up/Maintenance). Baseline data was collected through observation during a typical
algebra lesson. Following this, the teacher was trained by researchers to implement the GBG. In
the intervention phase, the GBG was used in conjunction with regular classroom instruction. In
the reversal phase, the GBG was removed and the teacher returned to the classroom management
strategies used during baseline. Then, as per the research design, the GBG was reintroduced.
This allowed researchers to determine if the GBG was indeed responsible for behavior change
and not merely due to coincidence. Two weeks later, follow-up observations took place to
examine maintenance effects. The choice of an ABAB design was an appropriate choice for
researchers as it establishes the functional relationship between treatment and behavior. The
drawback however, as with all ABAB designs, is the possibility that treatment withdrawal may
not be ideal. In this case, the teacher might not have chosen to withdraw the GBG from students
as results show that it did improve their behavior overall and consistency is essential if classroom
management is to be successful.

It is noteworthy that while researchers provided teacher training in the GBG, which
would typically increase fidelity, authors note that fidelity was a problem. Specifically, the
teacher sometimes failed to deduct points for inappropriate behavior (though fidelity was
reported to always have been greater than 85%). This could impact the experimental validity and
as Royer et al. (2006) suggest complicates the inferences that can be drawn about the outcomes
(p.317).
Operationally defined, off-task behavior was determined to be time not attending to
classroom instruction or completing teacher-assigned tasks. A 1-minute momentary time
sampling procedure was used to code observed off-task behavior (coded when 2/3 or more
students were off-task). Data was collected throughout a typical class period. To ensure
reliability, two researchers were trained to observe and established alignment based on two, 20
minute observations where notes were reviewed and compared. This is a necessary practice to
ensure inter-rater reliability.
A modified version of the Childrens Social Validity Interview was used as the measure
of social validity. Only eight students from the two classes completed this interview. In terms of
teacher social validity, a modified version of the Intervention Rating Profile-15 was used. The
teacher was the sole respondent. In both cases, positive feedback was provided and researchers
suggest that this confirms the social validity of the GBG. However, it would appear that the few
respondents in both cases would not be enough for researchers to draw sound conclusions.
Additionally, while the teacher report was positive, when researchers returned during the followup phase, the teacher had chosen to discontinue its use in favor of her typical, somewhat
ineffective, classroom management procedures. This inconsistency is interesting and does not
support the social validity of the GBG. Also of interest, is the absence of information related to
the psychometric properties of these measures; typically, researchers report the
reliability/validity of their chosen measures. Moreover, researchers reported that a modified
version of the measures was used yet they do not specify what is meant by this. Modification
can impact the reliability of the measure and therefore it is unfortunate that clarity regarding this
was not provided.
Ethically speaking, researchers did not include any information related to practices and
procedures. For example, no mention was made of procedures for recruitment or informed
consent. While one could assume that ethics were maintained and particularly, that informed
consent was obtained from the parents of participating students, we cannot be certain in the
absence of information regarding this process. This is a significant omission on the part of the
authors.
In both classes, the GBG was associated with improved behavior. In class 1, baseline data
revealed class-wide off-task behavior to be 55.1% . Following implementation of the GBG, it
was calculated at 3.33%. Likewise, in class 2 the baseline off-task behavior was reported to be
59.09% and following the GBG, 6.38%. In the follow-up phase, researchers noted that the

teacher had discontinued the GBG in both classes. In class 1, off-task behavior did not return to
baseline levels but it was not maintained at the level noted when the GBG was in use. In class 2,
off-task behavior did return to baseline levels.
As the authors suggest, this study had the potential to fill a gap in the research related to
use of the GBG with high school students with disabilities. However, with an absence of
information related to ethical procedures, and problems potentially impacting reliability and
validity, it appears to fall short.

REFERENCES
Flower, A., McKenna, J., Muething, C.S., Pedrotty Bryant, D., & Bryant, B. (2013). Effects of
the Good Behavior Game on Classwide Off-Task Behavior in a High School Basic
Algebra Resource Classroom. Behavior Modification, Vol. 38 (1), 45-68. Doi:
10.1177/0145445513507574
Knox, M., Burkhart, K., & Cromley, A. (2013). Supporting Positive Parenting in Community
Health Centres: The Act Raising Safe Kids Program. Journal of Community Psychology,
Vol. 41, No.4, 395-407. Doi: 10.1002/jcop.21543
Royse, D., Thyer, B.A., & Padgett, D.K. (2006). Program Evaluation: An introduction (5th ed.).
Belmont, CA: Wadsworth Centage Learning.