Running head: QUANTITATIVE EVIDENCE REVIEW AND CRITIQUE

Quantitative Evidence Review and Critique A Review and Critique of Magrun, McCue, Ottenbacher, and Keefes 1981 Study Effects of Vestibular Stimulation on Spontaneous Use of Verbal Language in Developmentally Delayed Children William Unger Touro University, Nevada

QUANTITATIVE EVIDENCE REVIEW AND CRITIQUE

APA Reference: Magrun, W.M., McCue, S., Ottenbacher, K., Keefe, R. (1981). Effects of vestibular stimulation on spontaneous use of verbal language in developmentally delayed children. American Journal of Occupational Therapy, 35(2), 101-104. Research Question: What effect does exposure to vestibular input have on the behavior of developmentally delayed children? How does this study relate to your research question? The article I chose to review and critique examines one component of the relationship between vestibular input and behavior which I sought to examine in my research question. While there are many potential elements of a behavioral effect, behaviors of spontaneous use of verbal language are of particular interest within the scope of developmental delays. By evaluating such a relationship, this study contributes greatly to a better understanding of the phenomena of vestibular stimulation on a developmentally delayed childs behavior. What is the purpose of this study? The purpose of this study is to evaluate what effect, if any, the introduction of a systematic regimen of vestibular stimulation would have on a developmentally delayed childs engagement in spontaneous verbal language. What are the research questions/hypotheses? The question posed by this pilot study could be summarized as, Will the use of specific forms of vestibular stimulation result in an increase in child-initiated intelligible responses immediately following therapy? Based on the authors presentation of precedent within the literature demonstrating a positive relationship between nontraditional, nonverbal intervention

QUANTITATIVE EVIDENCE REVIEW AND CRITIQUE

strategies as well as between language use and the various sensory systems, the authors of the study imply (but do not directly state) a hypothesis that such an effect would be present within their study. The lack of an overtly stated hypothesis is likely due to the fact that the authors considered this to be a pilot study. What is the study design/type of study? What is the level of evidence? This study was considered by the authors to be a pilot study, and as such had certain design limitations with respect to population size. The participants were sent in to one of two groups, both of which received identical treatment and neither of which could be considered a control group. The authors described the study as a reversal design (ABAB). Phases of the study were alternated, with each phase lasting one school week, or 5 consecutive days. The first phase was a baseline period during which frequency data of spontaneous use of verbal language during daily five minute blocks of free-play time was collected. This initial baseline was followed by the first treatment phase, during which vestibular stimulation was provided for 10 minutes prior to a similar 5 minute block of free-play time. The next two phases were identical to the first two, including a second baseline and second treatment phase. Due to the lack of a control group, this study cannot be considered a randomized control trial, but would instead be considered a randomized trial without control. As such, the level of evidence would be considered level III, with some elements (such as randomization in participant selection) of level II evidence. How many participants? all together and in each group if applicable The participants of this study were sorted into one of two groups, each group containing 5. This resulted in a total participant population of 10 children. How were the participants recruited and selected?

QUANTITATIVE EVIDENCE REVIEW AND CRITIQUE

Participants of either group were randomly selected from one of two classrooms for developmentally delayed children. If applicable, how were participants assigned to groups? The five children from the first experimental group were randomly selected from a single preschool class for developmentally delayed children, with subject ages ranging between 3 and 6 years. The five children from the second experimental group were randomly selected from a single primary trainable class, with subject ages ranging between 6 and 10 years. How are the participants described demographics e.g. diagnosis, age, gender, race Participants ages ranged between 3 and 10 years between both groups. The placement of children within either classrooms was based on, referral following diagnostic testing by speech and educational personal. Language skills of participants ranged from near nonverbal to near normal. Children who demonstrated more proficient language skills were exclusively enrolled within the primary training class, and if randomly selected were placed in group 2. Children of both groups were ambulatory and exhibited no physical deformities or restrictions. What are the variables? Independent and dependent if applicable. The independent variable was that of the systematic intervention of vestibular stimulation provided during each of the treatment phases for both groups. The dependent variable is the frequency of spontaneous use of verbal language during the free-play periods. What measures were used? Frequency of the use of spontaneous verbal language by participants of either group was measured through recorded reports of the two classrooms teachers. These records were taken during daily 5 minute free play periods over the course of 20 days. During the baseline phases, these free play periods were independent of any vestibular intervention. During the treatment

QUANTITATIVE EVIDENCE REVIEW AND CRITIQUE

phases, however, these play periods immediately followed a 10 minute vestibular intervention period. If applicable, what is the intervention? The intervention of vestibular stimulation was provided for both groups during two treatment phases, each of which followed a school week without treatment (during which baseline data was collected) and which lasted a consecutive period of 5 school days. Both treatment and baseline phases followed identical format. During each day of the treatment phase, vestibular stimulation was introduced within a 10 minute period by either accelerating down a ramp in a prone position on a scooter, spinning seated and in prone in a net hammock, or swinging seated on a platform board. To avoid the participant associating the input with some form of reinforcement, the vestibular stimulation was not paired with any specific or functional play activity. All periods of vestibular input were administered by a therapist experienced in pediatric treatment, including sensory integrative therapy and evaluation. Children were permitted to self-initiate stimulation, causing variable duration and intensity for each child. As mentioned earlier, the 10 minute period of vestibular input was followed by a free-play period during which data was recorded on the frequency of spontaneous use of verbal language. What statistical analyses were used? The authors first analyzed data by computing the mean number of verbal responses for each of the two baseline and two treatment periods for each group. This information was then used to create graphic data displayed in the study. Next, the authors implemented a repeated measures analysis of variance (ANOVA) for each group in an effort to expand and clarify the graphic results and to further substantiate the effectiveness of vestibular stimulation on spontaneous verbal responses. Finally, a Newman-Keuls follow-up procedure was performed to

QUANTITATIVE EVIDENCE REVIEW AND CRITIQUE

examine the occurrences of statistically significant differences between the baseline and treatment phases of group 1. This final procedure was not performed for group 2, for reasons which will be discussed in a moment. What are the findings? The authors found that eight of all 10 participants demonstrated an increase in the frequency of verbal responses between the first baseline and the first treatment. They also found that nine of ten participants showed a similar increase between the second baseline and second treatment. The trend of an increase between each baseline and its paired treatment phase was a trend present in both groups, and in both groups it was observed that removal of treatment (i.e. going from the first treatment phase to the second baseline phase) resulted in a decrease in verbal responses. While both groups demonstrated a pattern of increased verbal responses during treatment phases, only in group one was this increase found to be statistically significant within accepted ANOVA standards (p < .05). The ANOVA for the second group approached this value, but did not reach it (p <.10). As such, the Newman-Keuls follow up procedure was only performed to clarify the results of data from group one. Through this procedure, it was discovered that while there were statistically significant differences between respective baseline and treatment conditions (i.e. the first baseline and first treatment), there was no statistically significant difference between the first and second baseline nor the first and second treatment conditions. Do these findings support the hypothesis? The findings partially support the trend highlighted by the authors through their review of the literature, but demonstrate a limit to which vestibular input supports the spontaneous use of language. As there was only statistically significant improvement in vocalization for the group of

QUANTITATIVE EVIDENCE REVIEW AND CRITIQUE

younger participants (ages 3-6) with fewer language skills, this pilot study cannot suggest an effect regardless of age or level of development. How do the findings relate to previous research? The prior research presented in the literature review gave evidence of a variety of relationships between the different sensory systems and cognitive development, and the findings of this study further elaborate on that relationship. The vestibular system in particular had been found to have both anatomical and neurological relationships with speech centers; a concept further reinforced by this study. Does the author state any clinical implications for the findings? The authors of study stated that their results tentatively support the idea that vestibular stimulation may be an effective stimulation technique for facilitating spontaneous verbal responses in a portion of the developmentally delayed population. They went on to highlight that this effect is more clearly evident in younger children with a greater language delay. This suggests that such an intervention might not be as appropriate for older children with a less profound language delay. What are the limitations that the author identifies? The authors identified several limitations of their study. First, they cited that the use of such a small sample without any independent control group limited the generalization of the results. Furthermore, inter-observer reliability coefficients were not obtained for the study, nor were the dependent variable operationally defined in specific language. Last, the authors identified limitations in the method in which their data was displayed through the use of graphics within the paper. They stated that some authorities feel it is not appropriate to employ

QUANTITATIVE EVIDENCE REVIEW AND CRITIQUE

inferential statistics to applied behavioral designs; designs which are traditionally represented through graphic representation. Does the author discuss implications for future research? The authors identify that, due to the amount of limitations present within such a small scale pilot study, future research should be employed with a more robust approach with greater control over confounding variables. While this study supported previous trends in research as to the relationship between the vestibular system and language use, further research is necessary to produce more credible results. Conducting a similar study in the future with a larger sample size, sufficient control groups, sufficient randomization of participant selection, proper evaluations of reliability and validity, and stricter interpretations of the operational definitions of variables would greatly increase such a studys credibility. What would you say about the sample size? Do you think it is adequate? The sample size of this study was very small, but such can often be expected in pilot studies. As a result of this, however, the study lacks the ability to adequately portray the demonstrated outcome in a manner which might be representative of the population. If the researcher did not find a significant difference between the groups, is it possible that this is due to a Type II error? If so, why do you think so? It is entirely possible that the lack of statistical significance between the baseline and treatment phases of the second group is the result of a Type II error. The sample size of the second group was only 5 participants, which could allow for the inherent variability of a single outlier to vastly skew the results. In order to overcome such an issue in future, non-pilot studies, it is imperative that researchers obtain data from an adequately sized sample.

QUANTITATIVE EVIDENCE REVIEW AND CRITIQUE

Is there a control or comparison group? If so, is the control or comparison group comparable to the experimental group on key features? There was no control group, but the two groups were compared to one another. A control group would have been an excellent addition to this study, as it might have lent even greater support to the demonstrated effect of the intervention if an absence of such an effect was found in a control group of similar peers. Are those administering the outcome measures blind to group assignment? No, the teachers recording the occurrences of spontaneous use of language are aware of which group their participants were assigned to, as the groups were separated between two classes made up of two age groups. Are the participants blind to group assignment? It would seem that the participants, due to age and developmental level, were unaware not only to the group assignment but possibly of the study entirely. With this being the case, it could be said that the participants were in fact blind to group assignment. Does the researcher account for drop-outs in the study? Could drop outs have influenced the outcomes? While the authors do not make any mention of drop outs during the study, had there been any it could of most certainly impacted the outcomes of the study. This fact is particularly true due to the small sample size of this pilot study (N=10), and the loss of a single participant could have had a tremendous impact on the overall pool of data. Does the researcher report reliability and validity of the outcome measures? Are there questions about the outcome measures chosen?

QUANTITATIVE EVIDENCE REVIEW AND CRITIQUE

10

No, unfortunately the study was lacking in more sophisticated methods of analysis, and neither figures for reliability nor validity were reported. The authors highlighted that the lack of information regarding inter-rater reliability was a limitation of the current study, and made recommendations for a more thorough approach to this process in the future. What confounding factors could contribute to or influence the study outcomes? The primary confounding factor which might have influenced the results of this study is inherent variability of the participants. Given the small sample size of this pilot study (N=10), the data were incredibly vulnerable to the statistical influence of any outliers. Furthermore, while the participants were randomly selected from each of two classrooms, they shared the common element within each group of being from the same classroom environment. Any confounding effect of their common classroom environment is not controlled for within the study, and could potentially have influenced the study outcomes. What are the major strengths of this study? (list 3) First, this study attempted to spearhead a line of research examining the link between vestibular stimulation and the use of language for developmentally delayed children. The authors followed the trend of research which had suggested links between sensory systems and overall development, and applied these theories to a population of children who could benefit greatly from OT intervention. Second, this study examined the effect of vestibular stimulation for children of multiple age groups. This allowed them to demonstrate how the efficacy of such an intervention can vary greatly depending on the age and level of language development of the participants. This element might have been completely overlooked had the study focused on only one age group.

QUANTITATIVE EVIDENCE REVIEW AND CRITIQUE

11

Third, this study employed conservative and well accepted measures of statistical significance for the efficacy of their intervention. Had they allowed a broader cutoff for significance, they might have been able to demonstrate an effect for both groups. While this might have helped mitigate the chance of a Type II error in the second group, it also might have lessened the clinical implication of the findings for the first group. What are the major weaknesses of the study? (list 3) First, this study had an extremely small sample size. While this can be a common practice in pilot studies, it most certainly hampered the ability to demonstrate the efficacy of the intervention with a high degree of confidence. The small sample size left the study vulnerable to the occurrence of errors, as well as the confounding effect of the inherent variability of participants. With only 5 participants in each group, a single outlier could have potentially misrepresented the presence or lack of a statistically significant effect. Second, there was no control group to which the intervention could be compared. This further limits the studys ability to control for confounding variables impacting the outcomes. While it could be argued that the format of the study (including 2 baseline periods during which there was no treatment) could effectively demonstrate that the gains found in spontaneous use of language were the direct result of the vestibular stimulation given during treatment periods, had a control group been present for both age groups examined in this study it would have lent much greater credibility to the findings. Third, participants within each group were all selected from the same classroom. While this aided in the small pilot studys feasibility, it also acted as an additional barrier to the ability of this study to generalize results to a larger population. How would you use this article as a therapist?

QUANTITATIVE EVIDENCE REVIEW AND CRITIQUE

12

As a therapist, I would use the findings of this article to form new strategies to work with children with language delays. The authors of this study indicated that the participants were children who had experienced a less than average response to traditional verbal intervention strategies for spontaneous language use, so I might consider this as a first approach for clients which matched that description. How does this article support/not support participation in occupation and the field of occupational therapy? This article provides evidence to support practices in occupational therapy which address the occupation of social participation. As social participation and communication is an essential underlying component of a broad range of important occupations, effective interventions to support the development of language in developmentally delayed children is paramount to their occupational performance.

QUANTITATIVE EVIDENCE REVIEW AND CRITIQUE

13

References Magrun, W.M., McCue, S., Ottenbacher, K., Keefe, R. (1981). Effects of vestibular stimulation on spontaneous use of verbal language in developmentally delayed children. American Journal of Occupational Therapy, 35(2), 101-104.