INTRODUCTION
Background and Purpose of Study
Scheduling difficulties at the elementary school level often make it necessary to
pull certain students out of the regular classroom to receive specialized instruction in a
variety of areas. Students that are members of performance-based groups such as Band
and Orchestra are often scheduled as pullouts. The problems associated with scheduling
pullouts can result in frustration among teachers and administrators. Wallick (1998)
states four problems commonly associated with pullout programs as:
1.
2.
3.
4.
Below Basic: The PSSA Achievement level descriptor according to the Pennsylvania
Department of Education described as inadequate academic performance. Below Basic
Work indicates little understanding and minimal display of the skills included in the
Pennsylvania Academic Content Standards. There is a major need for additional
instructional opportunities and/or increased student academic commitment to achieve the
Proficient Level (PDE 2004).
No Child Left Behind: According to the Federal Government, the No Child Left Behind
Act (Public Law 107-110 107th Congress) is An Act to close the achievement gap with
accountability, flexibility, and choice, so that no child is left behind.
Pennsylvania System of School Assessment (PSSA): a standards based criterionreferenced assessment used to measure a students attainment of the academic standards
while also determining the degree to which school programs enable students to attain
proficiency of the standards (PDE 2004).
Pullout Programs: MENC, the National Association for Music Education, (2004)
defines pullout programs as elective classes that take some students, but not all students,
out of the regular classroom to participate in group practices or individual lessons. This
is a common practice in many elementary schools across the country and is not limited to
elective music classes such as band and strings.
Standardized Test: Assessments that are administered and scored in exactly the same
way for all students. Traditional standardized tests are typically mass-produced and
machine scored; they are designed to measure skills and knowledge that are thought to be
taught to all students in a fairly standardized way. Performance assessments also can be
standardized if they are administered and scored in the same way for all students. (North
Central Regional Educational Laboratory Glossary of Education Terms).
II
REVIEW OF LITERATURE
Theoretical Foundation
Howard Gardner, a cognitive psychologist from Harvard University, developed a Theory
of Multiple Intelligences. In his Frames of Mind: The Theory of Multiple Intelligences,
published in 1983, he first challenged the commonly held practice of categorizing people
by single measures of intelligence and proposed that there are seven basic intelligences
(Harvey, 1997). Those seven intelligences are defined by Gardner (1983) as:
Linguistic Intelligence:
Bodily-Kinesthetic Intelligence:
Musical Intelligence:
Interpersonal Intelligence:
Intrapersonal Intelligence:
In the January 1997 article, The Musical Mind, Black quotes Gardner as saying
that music might be a special intelligence which should be viewed differently from other
intelligences. He stated that musical intelligence probably carries more emotional,
spiritual and cultural weight than the other intelligences. Gardner also stated that music
helps some people organize the way they think and work by helping them develop in
other areas, such as math, language, and spatial reasoning.
The belief that instrumental music instruction helps students in other academic
areas is supported by a large body of literature that shows students who study music are
more likely to be successful academically. There is also a great deal of research that
suggests that students who study instrumental music score better on standardized tests. It
is also important to note that when instrumental students are excused from their
classrooms for their lessons, they are not leaving instruction (Wallick). They are only
moving to another area of the building to receive instruction in another one of the
intelligence areas. It is this theoretical foundation that supports the hypothesis that
instrumental pullout students do not suffer negative effects academically and that there
will be no significant difference between IS and NS Math and Reading PSSA Test scores.
Research Foundation
scores of all students. Analysis revealed that the string students scored higher in both
mathematics problem solving and reading comprehension. In other words, string
students academic achievement was higher than the general population of students
despite the string students being removed from their regular classroom for two years of
string study. The study was repeated in 1989, and the same results were obtained.
Kvet (1985) studied 175 matched IS and NI students from four different
geographic and socioeconomic areas of a southern Ohio metropolis. Sixth-grade students
were matched according to race, sex, IQ, classroom teacher, and cumulative achievement
test scores in reading, mathematics, and language arts. The results showed no significant
difference between IS who were excused from class for music instruction and NI who
remained in the classroom.
A 1985 study by Edward Kent showed that student absence from class to study a
musical instrument does not result in lower academic achievement. He found no
difference in academic achievement between sixth grade students who were excused from
class for instrumental study and those who were not, matching variables of sex, race, IQ,
cumulative achievement, school attended, and classroom teacher (Cutietta, 1995).
Dryden studied fifth grade students achievement on the Comprehensive Tests of
Basic Skills in Kansas. When comparing students involved in a music pullout program to
students not involved in the music pullout program, the results showed that excusing
elementary students from regular classroom activities does not negatively affect
achievement in math or reading (Dryden 1992).
Wallick (1998) studied 148 String Students (SS) and 148 NS students of matched
ability from a southwestern Ohio city school district. The students were ability-matched
10
according to their performance on the verbal section of the Cognitive Abilities Test. The
scores of the Ohio Proficiency Test were then recorded and compared. The results
showed a significant difference in favor of the string students achievement in reading
and citizenship, with no significant difference between the two matched groups in the
writing and mathematics sections of the Ohio Proficiency Test.
No studies were found that examined pullout students achievement on the PSSA
specifically, which is surprising due to the weight that is placed on the results of the test.
The PSSA is relatively new, and it is expected that many studies will follow that examine
how certain populations perform on the test.
Problem Explored
While theoretical and research foundation seems to support the idea that students
do not suffer academically when pulled out of the regular classroom to receive
instrumental music instruction, more research needs to be done to explore the problem of
teachers and administrators concerns. The impact that state mandated, high stakes
testing, and the No Child Left Behind Act has on schools is significant. It is not
surprising that teachers and administrators are looking for possible causes for students not
performing at a proficient or advanced level. This study is designed to determine what
effect, if any, being pulled out of the regular classroom for instrumental music instruction
has on students academic achievement as measured by the Mathematics and Reading
sections of the PSSA.
III
METHODS
11
Subjects
Protection of human subjects was assured through the Clarion University of
Pennsylvania Institutional Review Board (IRB).
Population
Subjects were be selected from a rural, North-Western Pennsylvania School
District.
Selection of subjects
A random sample of fifth-grade instrumental pullout students wasselected from
the entire fifth-grade instrumental pullout population of the district. A random sample of
fifth-grade non-instrumental students who are not pulled out of class for instrumental
music instruction was selected from the entire non-instrumental population of the district.
All students were selected from the same building, but not necessarily the same
classroom.
Instrumentation
The PSSA
In 1999, Pennsylvania adopted academic standards for Reading, Writing,
Speaking and Listening and Mathematics. These standards identify what a student should
know and be able to do at varying grade levels. School districts possess the freedom to
design curriculum and instruction to ensure that students meet or exceed the standards
expectations. The annual Pennsylvania System of School Assessment (PSSA) is a
12
13
Reliability
The reliability of the PSSA was discussed in the 2003 Technical Analysis of the
test. The following table contains an overview of the data. The Test length (L), mean
percentage of points received (P), typical test reliability (R) and typical standard error of
measurement (SEM) are shown for each form by grade and content area.
Table 1.
2003 Summary of Reliability and Standard Error of Measurement for the Pennsylvania
Sysytem of School Assessment (PSSA)
11
Grade
L
05
105
08
105
Mathematics
P
R
65
.93
60
.94
105
51
.94
5.2
SEM
4.9
5.5
14
Grade
L
05
90
Reading
P
R
66
.93
SEM
4.1
08
11
120
132
68
66
.94
.95
4.7
5.1
Validity
The validity of the PSSA was investigated by the Human Resources Research
Organization (HumRRO) in a series of studies that examined several aspects of the
testing system. The results of those studies provide strong evidence for the convergent
validity of the PSSA (Thacker, 2004).
15
Also, demographic factors were analyzed from the PSSA to determine if the
PSSA exhibited any differential impact based on gender, ethnicity, English proficiency, or
socioeconomic status. Males and females tended to score very similarly with males
demonstrating a slight advantage in mathematics. Asian and White students means were
typically larger than Hispanic students means, which were in turn, larger than Black
students means. Students with limited English Proficiency (LEP) did not score as high
as students not designated LEP. Means for students designated as economically
disadvantaged were also lower than those for students not so designated. These results
followed typical national scoring patterns and were very similar for all comparison tests.
Despite differential scoring among various demographic groups of students, no evidence
was found that PSSA added bias toward any investigated group (Thacker, 2004).
Materials
Materials needed to carry out this study are:
1.
2.
3.
4.
The 2004 PSSA test results for entire 5th grade class from the
school district (obtained from the District Superintendent).
16
5.
6.
Procedures
The procedures to carry out this study were:
1. Obtained permission from the School Board and Administration to
conduct the study.
2. Submitted necessary forms to obtain protection of human subjects
through the Clarion University Institutional Review Board.
3. Achieved protection of human subjects through the IRB.
4. Developed permission slips/Informed consent to give to all fifth grade
students to obtain parental permission to participate in the study.
Making it clear to the parents that their childs name would never be
used, and that any data collected would be analyzed as a group, and
not individually. No PSSA test scores of individually named students
would be mentioned in the study.
5. Gave permission slips/informed consent to all fifth grade students and
set a due date for the return of the permission slips.
6. Collected permission slips from all students who have obtained
parental permission to participate in the study.
7. Submitted informed consent forms to Clarion University IRB
17
18
IV
RESULTS
Data Reduction and Tabulation
19
Table 2.
Pennsylvania System of School Assessment (PSSA) raw scores of instrumental pullout
students.
20
Subject
Group
PSSA Math
PSSA Reading
________________________________________________________
1
2
3
4
5
6
7
IS
IS
IS
IS
IS
IS
IS
1442
1645
1384
1548
1618
1789
1548
1360
1718
1928
1625
1775
1586
1718
Table 3.
Pennsylvania System of School Assessment (PSSA) raw scores of non-intrumental
students.
21
Subject
Group
PSSA Math
PSSA Reading
________________________________________________________
1
2
3
4
5
6
7
NS
NS
NS
NS
NS
NS
NS
1297
1412
1618
1358
1839
1219
1321
1459
1459
1488
1276
1928
1317
1383
NS = Non-Instrumental Student
Descriptive Statistics
22
After the raw scores were tabulated and placed into a matrix, a descriptive
statistical analysis was conducted. The Mean, Standard Error of the Mean, Standard
Deviation and quartile scores were calculated. See Table 4 and Table 5.
Table 4.
23
Mean
SE Mean
St Dev
Minimum
Q1
Median
Q3
IS
NS
1672.9
1472.9
66.8
81.5
176.7
215.6
1360.0
1276.0
1586.0
1317.0
1718.0
1459.0
1775.0
1488.0
Maximum
1928.0
1928.0
Table 5.
24
Mean
SE Mean
Maximum
1567.7
1437.7
50.7
81.9
134.0
216.7
St Dev
1384.0
1219.0
Minimum
1442.0
1297.0
Q1
1548.0
1358.0
Median
1645.0
1618.0
Q3
1789.0
1839.0
25
26
Data
1700
1600
1500
1400
1300
1200
IS
NS
Figure 1.
Boxplot of Pennsylvania System of School Assessment (PSSA) Math scores for
Instrumental Students (IS) and Non-Instrumental Students (NS).
27
Figure 2. shows a histogram of the PSSA Math scores for Instrumental Students
(IS) and Non-Instrumental students (NS). Visually, it is not clear if there is a significant
difference between the two groups. Further inferential statistical analysis will be
conducted to determine if there is a statistically significant difference.
28
Variable
G1 M
G2 M
IS
Mean StDev N
1568 134.0 7
1438 216.7 7
Frequency
1.5
NS
1.0
0.5
0.0
1000
1200
1400
Data
1600
1800
Figure 2.
Histogram of Pennsylvania System of School Assessment (PSSA) Math scores for
Instrumental Students (IS) and Non-Instrumental Students (NS).
29
30
Data
1700
1600
1500
1400
1300
1200
IS
NS
Figure 3.
Boxplot of Pennsylvania System of School Assessment (PSSA) Reading scores for
Instrumental Students (IS) and Non-Instrumental Students (NS).
31
32
IS
1.4
Mean StDev N
1673 176.7 7
1473 215.6 7
NS
1.2
Frequency
Variable
G1 R
G2 R
1.0
0.8
0.6
0.4
0.2
0.0
1000
1200
1400
1600
Data
1800
2000
Figure 4.
Histogram of Pennsylvania System of School Assessment (PSSA) Reading scores for
Instrumental Students (IS) and Non-Instrumental Students (NS).
33
Inferential Statistics
A two-sample independent T-test analysis was computed using MINITAB
(Minitab Inc., 2003) to determine if there was a statistically significant difference
between Instrumental Students (IS) and Non-Instrumental Students (NS) achievement on
the Math and Reading portions of the Pennsylvania System of School Assessment
(PSSA). It was determined that p=<0.05 would serve as the alpha level of significance
testing. For the Math portion of the PSSA, the t-value was 1.35 and the p-value was
0.207, showing no statistically significant difference between the two groups (See Table
5.). The analysis of the Reading portion of the PSSA resulted in a t-value of 1.90 and a pvalue of .084, also showing no statistically significant difference between the two groups
(See Table 6.).
34
Table 6.
Two-Sample Independent T-Test Analysis of Pennsylvania System of School Assessment
(PSSA) Math scores for Instrumental Students (IS) and Non-Instrumental Students (NS).
T-Test of difference DF
T-Value
130.000
=0 (vs not =)
10
(-84.601,344.601)
P-Value*
1.35
0.207
DF=Degrees of Freedom
*Significant at p<=0.05
35
Table 7.
Two-Sample Independent T-Test of Pennsylvania System of School Assessment (PSSA)
Reading scores for Instrumental Students (IS) and Non-Instrumental Students (NS).
T-Test of difference
DF
T-Value
200.000
(-31.890, 431.890)
= 0 (vs not =)
11
1.90
P-Value*
0.084
DF=Degrees of Freedom
* Significant at p<=0.05
36
Special Considerations
While the two-sample independent t-test analysis determined that there was no
statistically significant difference between Instrumental Pullout Students (IS) and NonInstrumental Students (NS) in either the Reading or Math portion of the Pennsylvania
System of School Assessment (PSSA), it is important to address the presence of an
outlier in the Non-Instrumental Group of Students (NS) on the Reading portion of the
PSSA. (See Figure 5.) This outlier has skewed the data, and resulted in a non-significant
p-value. The author has chosen to complete an additional t-test analysis of the data with
the exclusion of the one outlier score. (See Table 7.) This two-group independent t-test
analysis resulted in a t value of 3.65 and a p value of 0.006 showing that there is a
statistically significant difference between the two groups on the reading portion of the
PSSA with the Instrumental Students scoring higher.
37
Data
1700
1600
1500
1400
1300
1200
IS
NS
Figure 5.
Boxplot of Pennsylvania System of School Assessment (PSSA) Reading scores for
Instrumental Students (IS) and Non-Instrumental Students (NS).
*note presence of NS outlier
38
Table 8.
Two-Sample Independent T-Test of PSSA Reading scores for Instrumental Students (IS)
and Non-Instrumental Students (NS) with the NS outlier excluded.
T-Test of difference DF
T-Value
200.000
=0 (vs not =)
(101.754,449.960)
P-Value*
3.65
0.006
DF=Degrees of Freedom
*Significant at p<=0.05
39
V
DISCUSSION
General Comments
This study supports the following answers to the two questions posed in the
original hypothesis.
1. There was no significant difference between Instrumental Students (IS) and
Non-Instrumental Students (NS) in the Math portion of the Pennsylvania
System of School Assessment (PSSA).
2. There was no significant difference between Instrumental Students (IS) and
Non-Instrumental Students (NS) in the Reading portion of the Pennsylvania
system of School Assessment (PSSA).
The evidence suggests that students who were pulled out of the regular academic
classroom for instrumental music instruction did not suffer negative effects in their
academic performance, as measured by the Math and Reading portions of the
Pennsylvania System of School Assessment (PSSA).
40
Limitations of Study
The greatest limitation of this study was the relatively small sample size. Because
this study examined the effects of a pullout instrumental music program on student
achievement in rural schools, it was difficult to acquire a larger sample. Most
instrumental music departments at rural schools are small, and result in a smaller sample
size.
Another limitation of the study is that while the study shows that students who left
the classroom for instrumental music instruction did not suffer negative effects on their
academic achievement as measured by the PSSA, many variables could have caused this
effect, and the empirical evidence necessary to draw specific conclusions from this
sample of students lies beyond the reach of this study.
Theoretical Support
Howard Gardner has stated that music helps some people organize the way they
think and work by helping them develop in other areas, such as math, language, and
spatial reasoning. His theory of multiple intelligences challenges the commonly held
practice of categorizing people by single measures of intelligence and proposes that there
are seven basic intelligences of which music is included. With that understood, then it is
acceptable to conclude that when instrumental music students leave the regular classroom
to receive instrument lessons, they are not leaving instruction, they are only moving to a
different area of the building where they are receiving instruction in a different area of
intelligence (Wallick).
41
42
Proficient
29%
Advanced
71%
Figure 6.
Pie Chart of Instrumental Students (IS) Pennsylvania System of School Assessment
(PSSA) Math Achievement Level
Advanced = raw score of 1460 and higher
Proficient = raw score of 1300-1459
Basic = raw score of 1170-1299
Below Basic = raw score of 1169 and below
43
Basic
28.5%
Advanced
28.5%
Proficient
43%
Figure 7.
Pie Chart of Non-Instrumental Students (NS) Pennsylvania System of School Assessment
(PSSA) Math Achievement Level
Advanced = raw score of 1460 and higher
Proficient = raw score of 1300-1459
Basic = raw score of 1170-1299
Below Basic = raw score of 1169 and below
44
Proficient
14%
Advanced
86%
Figure 8.
Pie Chart of Instrumental Students (IS) Pennsylvania System of School Assessment
(PSSA) Reading Achievement Level
Advanced = 1480 and above
Proficient = 1300-1479
Basic = 1160-1299
Below Basic = 1159 and below
45
Basic
14%
Category
1 Advanced
2 Proficient
3 Basic
Advanced
29%
Proficient
57%
Figure 9.
Pie Chart of Non-Instrumental Students (NS) Pennsylvania System of School Assessment
(PSSA) Reading Achievement Level
Advanced = 1480 and above
Proficient = 1300-1479
Basic = 1160-1299
Below Basic = 1159 and below
46
VI
SUMMARY AND RECOMMENDATIONS FOR FUTURE RESEARCH
While many educators will continue to be concerned with pullouts, the fear that
instrumental music students suffer academically when they leave the regular classroom
for music instruction may be alleviated when the results of this and similar studies are
examined.
Further research needs to be conducted on this topic with the utilization of
additional rural schools and larger sample sizes. Additionally, research needs to be done
across all pullout programs, including gifted and remediation programs.
47
REFERENCES
Bessom, M.E. (1969). Supervising the Successful School Music Program. West Nyack,
NY: Parker Publishing.
Black, S. (1997). The Musical Mind. The American School Board Journal, pp. 20-22.
Circle, D. (1989) Cognitive Growth of Students in Music as Measured by ITBS.
Unpublished report. Shawnee Mission Public Schools, Music Library and Fine
Arts Center, 7200 Belinder Road, Shawnee Mission, KS 66208.
Dryden,. (1992). The Impact of Instrumental Music Instruction on the Academic
Achievement of Fifth Grade Students. ERIC Document Reproduction Services
No. ED 368 634.
English, F. (1984). Pull-outs: How much do they erode whole-class teaching?
Principal, pp. 32-36.
Friedman, B. (1960). An Evaluation of the Achievement in Reading and Arithmetic of
Pupils in Elementary School Instrumental Music Classes. Dissertation Abstracts
Internationl. [online]
Gardner, H. (1983). Frames of Mind. New York: Basic Books.
Groff, F.H. (1963). Effects on Academic Achievement of Excusing Elementary School
Pupils from Classes to Study Instrumental Music. Dissertation Abstracts, 25,
5014-5015. (University Microfilms No. 64-3536).
Harvey, A. (1997). An Intelligence View of Music Education. [Electronic version]
www.menc.org/publications.
Hennessey, G.S. (1984). Pull-outs Disrupt Class Teaching. Instructor, 94, 18.
Cutietta, H. (1995). Spin-Offs: The Extra-Musical Advantages of a Musical Education,
United Musical Instruments U.S.A., Inc.
Kvet, E.J. (1985). Excusing Elementary School Students From Regular Classroom
Activities for the Study of Instrumental Music: The Effect on Sixth Grade
Reading, Language, and Mathematics Achievement. Journal of Research in
Music Education, 32, 45-54.
Mead, R., & Smith, R.M. (2003). Technical Analysis of the Pennsylvania System of
School Assessment. PA Dept. of Education. [Electronic version].
48
MENC: National Association for Music Education. (2004). Position Statement: Pullout
Music Instruction. [Electronic version].
www.childrensmusicworkshop.com/advocacy/pulloutposition.html.
Northcentral Regional Educational Laboratory. (2002). Glossary of Education Terms
and Acronyms. [online]. www.ncrel.org
Pennsylvania Department of Education. (2004). Assessment: Pennsylvania
Accountability System. Harrisburg, PA: Author.
Pennsylvania Department of Education. (2004). Glossary of Terms: Pennsylvania
Accountability System. Harrisburg, PA: Author.
Pennsylvania School Board Associon (PSBA). (2003). The PSSA and Its Impact on
School Districts and Students: A PSBA Statement. [Electronic version].
www.pascd.org.
Robitaille, J.P., & ONeal, S. (1981). Why Instrumental Music in Elementary Schools?
Phi Delta Kappan, 63, 213.
Thacker, A. A., (2004). Relationships Among the Pennsylvania System of School
Assessment (PSSA) and Other Commonly Administered Assessments. [Electronic
version]. www.humrro.org
United States Department of Education. (2001). Public Law print of PL 107-110, the No
Child Left Behind Act of 2001. Washington D.C.: Author.
Wallick, M.D. (1996). A Comparison Study of the Ohio Proficiency Test Results
between Fourth-Grade String Pullout Students and Those of Matched Ability.
Journal of Research in Music Education, 46, 239-247.
49
APPENDIX A
Human Subjects Approval
50