Running Head: ARTICLE CRITIQUE

ARTICLE CRITIQUE Kelsey Heiple EDUR 7130 Educational Research Dr. Anne Marshall Georgia Southern University

ARTICLE CRITIQUE

Introductory Section Problem Area In the introduction, the authors clearly explain that summer reading setback, as linked to summer reading activity or lack thereof, is a contributor to the reading achievement gap. This achievement gap exists between children from economically advantaged and disadvantaged families. After laying the groundwork for the problem, they clearly articulate their theoretical perspective that summer reading setback is linked to the amount of summer reading activity; the more a student reads during summer, the less likely they are to experience summer reading setback. The authors further explain their belief that a childs access to books is directly related to their socioeconomic status. They present that if economically disadvantaged students had access to summer reading material, it could likely narrow the reading achievement gap that exists as a result of summer reading setback. The authors establish the educational relevance of the study by making note of a central aspect of federal educational policy making over the past fifty years has been attempts to fund interventions that close the achievement gaps existing on measures of reading proficiency. They also make mention of a recent federal initiative, the No Child Left Behind Act of 2001, which aimed at closing the achievement gap to further support its relevance. Related Literature The authors do an excellent job of providing clear and concise definitions and background on the constructs in the study. They provide history on the reading achievement gap between children from more and less economically advantaged families and reference a recent study that provides strong evidence of the pervasive nature of the problem. They also provide

ARTICLE CRITIQUE figures from the aforementioned study to further support the need for a solution. The authors

continue by clearly defining summer reading setback as a well-established phenomenon and cite multiple quality sources as support. They provide relevant studies of summer academic loss to find that more economically advantaged students gained on grade-level equivalent reading recognition tests over summer while children from low-income families lost on them. These studies help to further define the phenomenon of summer reading setback and help the reader to fully understand the achievement gap. Overall, the literature review does a good job of clearly defining the constructs on which the study is based. The authors were very thorough in explaining the history of each construct and further supporting their educational relevance. They present multiple research studies related to the problem area and show that the intended research is an extension of those studies. Each study presented appears to be current, as the article was published in 2010. The only exception to this would be the existing research on the problem, which was discussed some twenty to thirty years ago. The authors made sure to make note of this, however, and the existing research still appears to be from a credible source. Likewise, the other studies reviewed appear to be from credible sources, as well. Hypothesis and Purpose The purpose of this study was clearly stated as a way to test the effects of a lowinvolvement intervention design in addressing summer reading setback. Their objective, which was clearly stated, was to ensure that children from low-income families had easy access to books for voluntary summer reading over a three-year period. The researchers hypothesize that providing elementary school students from low-income families with a supply of self-selected trade books would improve summer reading setback. More specifically, they hypothesized that

ARTICLE CRITIQUE doing so could positively impact their voluntary summer reading activity and achievement and reduce summer reading setback and, thus, close the reading achievement gap. The research variables are easily identifiable from the purpose and hypothesis and the purpose can be studied in an unbiased manner. Method Section Participants The researchers clearly describe their target population as being students from lowincome families. They explain that students from seventeen high-poverty elementary schools in two of Floridas larger school districts were randomly selected using stratified sampling. The researchers describe the selection procedures by stating that all first- and second-grade students

in the schools were given the parental consent letters to take home, of which 1,713 were returned giving consent. They then randomly selected 1,082 children to receive the summer books, the treatment group, and 631 were randomly selected to serve as the control group. They accounted for possible attrition by selecting substantially more treatment group students. Over the entire period of the study, attrition resulted in having a total of 852 treatment and 478 control students. Taking into consideration the research design, the participants and the procedures for selecting them are appropriate. Because the participants were randomly selected from seventeen highpoverty elementary schools, the sampling is from the target population of low-income students making it the appropriate procedure. Instrumentation The instruments used in the study are the Florida Comprehensive Achievement Test (FACT) and a participant survey. The FCAT, as explained by the researchers, is administered annually to all students in grades 3 through 8 and has demonstrated reliability. The authors

ARTICLE CRITIQUE explain the utilization of the FCAT as an instrument to determine developmental scores, which would be used in their analyses. In addition to its demonstrated reliability, the instrument as it pertains to this study has reliability in that it is administered annually over the three-year period.

Furthermore, the instrument has validity in that it is measuring what it is intended to measure, the reading achievement of each student. It is an appropriate instrument for use with the target population and the research participants because it is already utilized for measuring achievement. The other instrument utilized by the researchers is a participant survey. The survey, a shortened version of the Literacy Habits Survey, is described by the authors as a paper-pencil instrument administered to each student outside of the classroom. Its use to collect data about the frequency of summer reading and access to books makes it an appropriate instrument for this study. The researchers sought to ensure reliability by reading aloud the questions and possible responses so that students understood what was being asked. The instrument provides criterionrelated validity in that the results, how much each student read, their access to books, etc., can predict the level of summer reading setback and, thus, the reading achievement level. These instruments are appropriate for the design of the study because they assist in adequately measuring the constructs of the study, summer reading setback and reading achievement level. Construct validity is achieved in that the instruments used represent the constructs in the study; the FCAT represents reading achievement and the survey represents summer reading activity and, thus, the likelihood of summer reading setback. Although the researchers provide evidence of reliability, they do not give reliability estimates for each instrument in terms of alpha. Design and Procedure In this study, the researchers are utilizing performance and behavioral measures to

ARTICLE CRITIQUE

determine if accessibility of summer reading material reduces summer reading setback and, thus, lessens the reading achievement gap between children from more or less economically advantaged families. These measures are appropriate for answering the questions at hand because they are trying to measure achievement, a performance measure determined by the FCAT. Furthermore, by asking things like how much each student read during the summer months, a behavioral measure, the researchers are collecting the data necessary to answer the questions at hand. The experimental treatments used for the treatment group are the trade-books that are issued to them on the every year over the three year period. Although not clearly defined, the independent variable is the student and the dependent variable is the reading achievement level of the student as measured by the FCAT. Results Section Data Analysis Researchers used comparative research to analyze the data. This is the most appropriate means for the research question because the researchers want to compare FCAT performances of treatment group with those of the control group. The statistical procedures for data analysis were not clearly described but the researchers did provide the statistics to show that there was, in fact, a difference in FCAT results for treatment and control groups after the three-year period. They show this by providing mean scores and standard deviation for both groups. For the second set of statistics, provided only for students in both groups who were eligible for free lunch, they also included median and skewness to analyze the data. As for the participant survey, the researchers explain that they examined the behavioral evidence for two items: 1) how often the children read during the summer and 2) where the children got most of the books they read. Based on the correlation coefficients provided, which

ARTICLE CRITIQUE show a significant correlation, they appropriately analyze the data as showing book distribution having a positive effect on the frequency of summer reading and, therefore, improvement in reading achievement levels. Threats to Validity

As for research validity, the researchers make note of a slight difference in the frequency of free lunch students between the treatment and control groups. Because of this, they conducted additional data analysis to compare FCAT results for students who were eligible for free lunch in both treatment and control groups. The behavioral measures used in the study also pose a threat to validity. The researchers intend to use the participant survey to ask the students questions about summer reading activity, access to books, and home reading support. Asking the participants to self-report their summer reading activity is a threat to validity. While the information is necessary, a true behavioral observation would have to be witnessed first-hand by the researchers. The researchers cannot be certain that the participants are being truthful when self-reporting their behavior. An external threat to research validity is the possibility of changing circumstances within the participants families over the course of the study. The researchers fail to take into consideration the fact that a family situation could change and, thus, improve a students accessibility to reading material over the three-year period for either group, treatment or control. Although it would likely not have a significant impact on the treatment group as they already have access to trade-books, it could have impacted the control group. Significance The researchers indicate statistical significance levels for the original treatment and control groups as well as the free lunch treatment and control groups. They further describe the

ARTICLE CRITIQUE statistical significance of the original and free lunch groups as being p = .015 and p = .001,

respectively, which provides reliability seeing as that it is less than p = .05. They go on to prove the practical significance of each by showing that the effect size for the original group of students was ES = .14 and the effect size for the free lunch group of students was ES = .21. Although they do not clearly interpret the statistical significance of each result, they do include that the effect sizes of each indicate that the book distribution had positive effects on the reading achievement of the economically disadvantaged students, thus proving that there is practical significance, or importance, in the results. Discussion Section Discussion (Conclusion, Recommendations) The authors do an excellent job of relating the conclusions to the research question. They provide collected data as a means to explain to the reader how they drew their conclusions from the results. Upon doing so, they prove that their conclusions are appropriate. The authors identify a weakness as being small effect sizes; however, they counteract this weakness by drawing attention to the minimal effort and cost associated with the study. They identify areas for future research to include more in-depth measures of summer reading activity to increase reliability of the link between reading activity and improvements in reading achievement. Finally, they list the studys implications as providing a more effective way of improving reading achievement through distribution of free books for summer reading.