Anda di halaman 1dari 18

Review of Educational

Research http://rer.aera.net

The Role of the Laboratory in Science Teaching: Neglected Aspects of


Research
Avi Hofstein and Vincent N. Lunetta
REVIEW OF EDUCATIONAL RESEARCH 1982 52: 201
DOI: 10.3102/00346543052002201

The online version of this article can be found at:


http://rer.sagepub.com/content/52/2/201

Published on behalf of

American Educational Research Association

and

http://www.sagepublications.com

Additional services and information for Review of Educational Research can be found at:

Email Alerts: http://rer.aera.net/alerts

Subscriptions: http://rer.aera.net/subscriptions

Reprints: http://www.aera.net/reprints

Permissions: http://www.aera.net/permissions

Citations: http://rer.sagepub.com/content/52/2/201.refs.html

>> Version of Record - Jan 1, 1982

What is This?
Downloaded from http://rer.aera.net at UNIV PRINCE EDWARD ISLAND on February 19, 2013
Review of Educational Research
Summer, 1982, Vol. 52, No. 2, Pp. 201-217

The Role of the Laboratory in Science Teaching:


Neglected Aspects of Research
Avi Hofstein
The Weizmann Institute of Science

Vincent N. Lunetta
The University of Iowa

The laboratory has been given a central and distinctive role in science
education, and science educators have suggested that there are rich benefits in
learningfrom using laboratory activities. At this time, however, some educators
have begun to question seriously the effectiveness and the role of laboratory
work, and the case for laboratory teaching is not as self-evident as it once
seemed. This paper provides perspectives on these issues through a review of
the history, goals, and research findings regarding the laboratory as a medium
of instruction in introductory science teaching. The analysis of research
culminates with suggestions for researchers who are working to clarify the role
of the laboratory in science education.

The laboratory has long been a distinctive feature of science education. In 1970
the Commission of Professional Standards and Practices of the National Science
Teachers Association thought that the case for school science laboratories was too
obvious to argue (Ramsey & Howe, 1969): "That the experience possible for students
in the laboratory situation should be an integral part of any science course has come
to have a wide acceptance in science teaching. What the best kinds of experiences
are, however, and how these may be blended with more conventional classwork, has
not been objectively evaluated to the extent that clear direction based on research is
available for teachers" (p. 75). Less than 10 years later, the case for the laboratory in
science instruction was not as self-evident as it once seemed. Science laboratory
requirements are currently of special concern because there is now a trend to retreat
from student-centered science activities, resulting in less time and therefore experi-
ence in the science laboratory (Gardner, 1979). Our primary goals are to review the
research studies relating to teaching and learning in the laboratory in introductory
level science courses and to suggest further research that might be needed to assess
the value of laboratory work. More specific objectives are:
(1) to review briefly the history and goals of the laboratory in introductory science
teaching;
(2) to review and critically analyze research findings regarding the effectiveness of
laboratory instruction;
(3) to suggest specific dimensions of potential relevance for research on teaching
and learning in the laboratory; and
(4) to provide a synthesis of suggestions for researchers working to clarify the role
of the laboratory in science education.
For our purposes, laboratory activities are defined as contrived learning experiences
201

Downloaded from http://rer.aera.net at UNIV PRINCE EDWARD ISLAND on February 19, 2013
HOFSTEIN AND LUNETTA

in which students interact with materials to observe phenomena. The contrived


experiences may have different levels of structure specified by the teacher or
laboratory handbook, and they may include phases of planning and design, analysis
and interpretation, and application as well as the central performance phase. Labo-
ratory activities usually are performed by students individually or in small groups,
and our definition does not include large-group demonstrations, science museum
visits, or diffused field trips.
Many research studies have been conducted comparing the effects of methods of
practical work in the laboratory with other instructional methods over the past
decades. For example, Coulter (1966) compared inductive laboratory experiments
with inductive demonstrations in high school biology; Yager, Engen, and Snider
(1969) compared three groups, namely, a "laboratory group," a "demonstration
group," and a "discussion group" in biology; Lunetta (1974) compared a control
group to a computer-simulation group in physics; and Ben-Zvi, Hofstein, Kempa,
and Samuel (1976a) compared a laboratory group to a group viewing filmed
experiments in chemistry. Most of these research studies have shown no significant
differences between the instructional methods as measured by standard paper-and-
pencil tests in student achievement, attitude, critical thinking, and in knowledge of
the processes of science. Not surprisingly, the one area in which the laboratory
approach showed measurable advantage over other modes of instruction was in the
development of laboratory manipulative skills. Because many studies comparing the
effects of laboratory learning with more conventional forms of instruction have
resulted in nonsignificant differences, some science educators (e.g., Bates, 1978;
Dickinson & Sanders, 1979) have questioned the value of laboratory work. However,
it is important to note that similarly poor evidence for success has been found for
almost all attempts to improve schooling. In a sweeping review of educational
research, Stephens (1967) noted that changing instructional techniques seems to
hinder learning as often as it seems to aid it. Among the questions that science
educators ought to ask are:
(1) Has the design of investigations and the choice of outcome measures been
adequate to probe the purported benefits of laboratory work?
(2) Are there sufficient data to accept or dismiss the claims made for the laboratory
in science teaching?
If a strong affirmative response cannot be provided, then it is important to continue
to search for greater understanding through carefully designed research studies.

Brief Review of Laboratory Goals1


The history of laboratory work as an integral part of school science learning has
roots in the 19th century. The laboratory in the science classroom has long been used
to involve students in concrete experiences with objects and concepts. In 1892 Griffin
wrote: "The laboratory has won its place in school; its introduction has proved
successful. It is designed to revolutionize education. Pupils will go out from our
laboratories able to see and do" (cited by Rosen, 1954). In the years following 1910,

1
Certain parts of this section are based on Tamir, P., The role of the laboratory in science
teaching (Tech. Rep. No. 10.). Iowa City, Iowa: The University of Iowa, Science Education
Center, 1976.

202

Downloaded from http://rer.aera.net at UNIV PRINCE EDWARD ISLAND on February 19, 2013
NEGLECTED ASPECTS OF RESEARCH

the progressive education movement had a major impact on the nature of science
teaching in general, and on the role of laboratory work in particular. John Dewey,
leader of the progressive education movement, advocated an investigative approach
and "learning by doing" (p. 202). During this period, textbooks and laboratory
manuals began to acquire a more applied, utilitarian orientation. Nevertheless, even
while the progressive education movement was gaining momentum, debate about
the proper role of laboratory work also was developing. The arguments raised against
extensive student laboratory activities included:
(1) few teachers in secondary schools are competent to use the laboratory effec-
tively;
(2) too much emphasis on laboratory activity leads to a narrow conception of
science;
(3) too many experiments performed in secondary schools are trivial; and
(4) laboratory work in schools is often remote from, and unrelated to, the capa-
bilities and interests of the children.
Following World War I, laboratory activities came to be used largely for confirming
and illustrating information learned from the teacher or the textbook. This orientation
remained relatively unchanged until the "new" science curricula of the 1960s, which
resulted in several departures from tradition in the role of laboratory work. In "the
new curricula which stress the processes of science and emphasize the development
of higher cognitive skills, the laboratory acquired a central role, not just as a place
for demonstration and confirmation, but as the core of the science learning process"
(Shulman & Tamir, 1973, p. 1119). Contemporary science educators (e.g., Hurd,
1969; Lunetta & Tamir, 1978; Schwab, 1962) have expressed the view that uniqueness
of the laboratory lies principally in providing students with opportunities to engage
in processes of investigation and inquiry. According to Ausubel (1968) the laboratory
"gives the students appreciation of the spirit and method of science, . . . promotes
problem-solving, analytic and generalization ability,. . . provides students with some
understanding of the nature of science" (p. 345).
In a review of the literature, Shulman and Tamir (1973) proposed a classification
of goals for laboratory instruction in science education:
(1) to arouse and maintain interest, attitude, satisfaction, openmindedness and
curiosity in science;
(2) to develop creative thinking and problem-solving ability;
(3) to promote aspects of scientific thinking and the scientific method (e.g.,
formulating hypotheses and making assumptions);
(4) to develop conceptual understanding and intellectual ability; and
(5) to develop practical abilities (e.g., designing and executing investigations,
observations, recording data, and analyzing and interpreting results).
Anderson (1976) summarized the goals of laboratory work in four main areas:
(1) to foster knowledge of the human enterprise of science so as to enhance student
intellectual and aesthetic understanding;
(2) to foster science inquiry skills that can transfer to other spheres of problem-
solving;
(3) to help the student appreciate and in part emulate the role of the scientist; and
(4) to help the student grow both in appreciation of the orderliness of scientific
knowledge and also in understanding the tentative nature of scientific theories
and models.

203

Downloaded from http://rer.aera.net at UNIV PRINCE EDWARD ISLAND on February 19, 2013
HOFSTEIN AND LUNETTA

These objectives for the laboratory are almost synonymous with those defined for
science courses in general. Thus there is a need to define goals for which laboratory
work could make a special and significant contribution and to capitalize on the
uniqueness of this mode of instruction.

Critical Analysis of Past Research


Are goals for laboratory teaching and learning achieved in practice? To answer
such a question sufficient data must be available from appropriate research and there
is considerable reason to assert that studies conducted this far have not been
sufficiently comprehensive. Past research studies generally examined relatively nar-
row bands of laboratory-related skills, and the conclusions that were drawn may
apply to a narrow range of teaching techniques, teacher and student characteristics,
and learning outcomes. More specifically, many research studies suffered from a
number of particular weaknesses:
1. Selection and control of variables. Researchers failed to examine or report
important variables descriptive of student abilities and attitudes. Generally, they
failed to note the kind of prior laboratory experience that most students involved in
the studies almost certainly had. Not enough attention was given to control over
extraneous factors, such as instruction outside the laboratory while the research study
was conducted.
2. Group size. Researchers used comparatively small groups. Furthermore, student
samples were of limited diversity and most of the research studies did not examine
the effect of different subsets of the population (e.g., less able or more able students)
(as reviewed by Belanger, 1971; Bradley, 1968; Cunningham, 1946).
3. Instrumentation. Researchers in science education were often more concerned
with the nature of the treatment than with the validity of the instrumentation used
to measure outcomes of their studies.
Welch (1971) noted that 30 research reports concerning instructional procedures
(including laboratory instruction), made no connection between the instructional
procedure and the test chosen to measure the effect.
In fact, we find that standardized achievement instruments, which were not
designed specifically to measure outcomes of laboratory work, were often used.
Science teaching traditionally has emphasized the learning of scientific
"information," concepts, principles, and facts, with little emphasis on the develop-
ment of problem-solving skills, and this orientation is reflected in many of the test
instruments that were used. Tests often emphasized student ability to identify or
recall facts at relatively low taxonomic levels but seldom have assessed development
of higher level skills that involve application, analysis, synthesis, and evaluation
(Bloom, 1956). Even if laboratory work does contribute to success on conventional
achievement tests, it may do so insufficiently. Equal time spent in drill might well
have greater payoff in test performance. By this reasoning, null differences on
conventional achievement measures might be interpreted as supporting the effective-
ness of laboratory experiences because laboratory work presumably will enhance
other desirable outcomes that conventional achievement tests do not assess. One of
many examples of standardized measures used (e.g., Yager et al., 1969) is the Watson-
Glazer-Critical-Thinking Appraisal (WGCTA, in Watson & Glazer, 1961), which
has little to do with science teaching in general nor with laboratory work in particular.
This instrument was constructed and validated for use in the social sciences and is

204

Downloaded from http://rer.aera.net at UNIV PRINCE EDWARD ISLAND on February 19, 2013
NEGLECTED ASPECTS OF RESEARCH

concerned with social and historical phenomena. While one can argue that "transfer
of learning" is a desirable outcome of instruction, the difference between science
laboratory experiences and historical and social events is very large. Ramsey and
Howe (1969) pointed out that the inquiry method, designed to have the student
working with the processes of science, is likely to produce different outcomes than
conventional procedures. Tamir (1972) noted that the laboratory in science education
is not only a unique mode of instruction but also a unique mode of assessment.
Therefore it is desirable to develop more sensitive evaluation instruments that will
provide information about what the student does in the laboratory and about his
growth and ability to develop inquiry and other laboratory-related skills.
4. Teaching behavior. Most research studies failed to look at teacher behavior,
classroom learning environment, and variables identifying teacher-student interac-
tion. Most research studies failed to assess and report what is really happening in the
classroom and how the teacher translates the curriculum into action (Connelly, 1979;
Silberstein, 1979). An experiment can be open ended and inductive when taught by
one teacher yet deductive when taught by another. Eggelston, Galton, and Jones
(1976) found that teaching style tends to be consistent no matter what form of
activity takes place; deductive-oriented teachers teach practical work authoritatively,
while more inquiry-oriented teachers teach investigative methods of learning. There
is a need for obtaining more objective information about the interactions between
teachers, curriculum resources, and students, and about teacher and student behaviors
during a laboratory-based learning sequence.
An important attempt to develop a systematic classroom interaction analysis to get
more information on what actually happens in the laboratory was made by Penick,
Shymansky, Filkins, and Kyle (1976), who developed the Science Laboratory Inter-
action Category (SLIC-Student), and by Shymansky, Penick, Kelsey, and Foster
(1976), who developed the SLIC-Teacher. Using these two instruments one can
obtain information about the kind of teaching and learning that takes place in the
science laboratory. Tamir (1977) used the classroom observation schedule developed
by Smith (1971) to observe students conducting experiments in the biology laboratory.
This instrument provides a record of teachers' and students' pre- and postlaboratory
activities.
Barnes (1967) developed an instrument (paper-and-pencil) called the Biology
Laboratory Activity Checklist (BLAC). This instrument measures the nature and
extent of laboratory instruction and activities in high school biology instruction as
perceived by the students. The laboratory activities and information that were
evaluated included prelaboratory activities, laboratory activities, postlaboratory ac-
tivities, and general student reaction to the laboratory. This practical instrument
enables one to find out the extent to which high school biology laboratory activities
of teachers agree with the activities advocated by the curriculum developers.
5. Laboratory manual. A key component of learning in the laboratory is the
students' laboratory manual. The laboratory manual plays a major role for most
teachers and students in defining goals and procedures for laboratory activities. It is
also supposed to help focus observations and the development of inferences, expla-
nations, and other activities in laboratory investigation (Lunetta & Tamir, 1979).
Recognizing the need to examine the quality of written laboratory manuals, Fuhr-
man, Lunetta, Novick, and Tamir (1978) designed a task analysis inventory, the
Laboratory Analysis Inventory (LAI). This inventory was found to be a useful

205

Downloaded from http://rer.aera.net at UNIV PRINCE EDWARD ISLAND on February 19, 2013
HOFSTEIN AND LUNETTA

instrument in analyzing laboratory manuals. The task categories include actual


behaviors required to perform prescribed laboratory work and inquiry. Comprehen-
sive research into the learning effectiveness of laboratory instruction should also
include a summary and analysis of the kind of laboratory manual that has been
used. Studies could be undertaken to find out how published materials are actually
used in the laboratory.
There are vast differences in learning strategy from one kind of laboratory activity
to another that are bound to affect learning outcomes, and these differences should
be analyzed and reported in studies of the effects of laboratory teaching and learning.
Some activities are organized deductively and students gather data to verify and
further understand laws or relationships that have been outlined earlier in the text or
in class. Other activities precede formal introduction of a topic and involve students
in gathering information about materials or phenomena from which they will
subsequently infer relationships and make generalizations. Also, one laboratory
activity can vary greatly from another in the amount of guidance provided to
students. Some laboratory investigations are highly structured and students follow
"cook book" instructions while others are much more "open" and involve students
in elements of the experiment's planning and design. Some laboratory activities
emphasize manipulation of materials, while others emphasize observational skills,
the interpretation of data, or the application of procedures to new problems. While
it is highly probable that such differences among laboratory activities affect learning
outcomes, it is even more probable that interaction effects will exist between the style
of the laboratory and the prior learning and logical development of individual
students.
In addition to a careful monitoring of the variables mentioned in this section,
however, the examination of student learning and growth must be expanded to
gather data in areas of high potential interest and relevance that were ignored in
many of the studies conducted thus far.

Research Areas with Potential for Contemporary Study


In a recent literature review, Bates (1978) wrote:

Teachers who believe that the laboratory accomplishes something special for
their students would do well to consider carefully what those outcomes might
be, and then to find ways to measure them. If it is anything, this paper is an
invitation to systematic inquiry for the answer has not yet been conclusively
found: What does the laboratory accomplish that could not be accomplished
as well by less expensive and less time consuming alternatives? (p. 75)

Many research studies on the effect of laboratory instruction (summarized by


Bates, 1978) attempted to measure learning outcomes in the cognitive domain. While
these studies often examined growth in the understanding of concepts of particular
science disciplines, generally they failed to examine growth in other cognitive
variables such as creative thinking, problem solving, scientific thinking, and intellec-
tual development. These important cognitive variables are probably interrelated and
have prima facie potential for research study in the laboratory learning context.
Furthermore, the effects of laboratory experiences on the development of practical
skills, attitudes, and social variables also warrant careful research study.
206

Downloaded from http://rer.aera.net at UNIV PRINCE EDWARD ISLAND on February 19, 2013
NEGLECTED ASPECTS OF RESEARCH

Creative Thinking and Problem-solving


In the 1960s there was a call for a "new" kind of science teaching. Articles and
books appeared arguing that the methods used in the past no longer met the challenge
of the present (Getzels, 1963). The change was to emphasize inquiry, discovery,
creativity, and generalization. Romey (1970) defined creativity as the ability to
combine ideas, things, techniques, or approaches in a new way. Certain kinds of
open-ended laboratory activities in which the student is involved in a problem-
solving situation might provide the best opportunities for students' creative thinking
to develop. In the laboratory a problem may be given for which the students have
not yet learned a method of solution, or a situation can be created in which a
problem exists but has yet to be identified by the student. These are situations in
which students can be encouraged to develop skills considered to be creative and
original. Ramsey and Howe (1969) took this point of view when they considered the
laboratory an important place in which to introduce students to problem-solving
through experimental methods, as well as to increase comprehension.
However, very few studies in the literature describe attempts to measure creativity
and problem-solving as outcomes of laboratory work. Those that have been under-
taken hint at positive results. Penick (1976) reports findings that suggest a growth in
creativity in fifth grade students as a result of science laboratory experiences. In an
elaborate research study, Hill (1976) found that college student creativity, as measured
by the Minnesota Test of Creative Thinking, improved with involvement in chemistry
laboratory activities. It is clear that this is an area where careful research could reveal
important relationships.

Scientific Thinking
Many educators claim that the laboratory is one of the important vehicles for
teaching and understanding the processes of scientific thinking. According to Lucas
(1971), students can understand how scientists work and think and also how to
acquire new knowledge themselves by personally practicing the use of inquiry.
Burmester (1953), for example, designed a carefully validated paper-and-pencil test
to measure some aspects of students' ability to think scientifically. Under the heading
"scientific thinking" she included the ability to
(1) recognize problems;
(2) understand experimental methods;
(3) organize and interpret data;
(4) understand the relation of facts to the solution of problems;
(5) plan experiments to test hypotheses; and
(6) make generalizations and assumptions.
It has been hypothesized that inquiry-based laboratory activities in which the student
examines an interesting problem could enhance the attainment of many of these
abilities. A research study conducted by Kaplan (1967) showed student pretest-
posttest gains on Burmester's Inventory resulting at least in part from the use of a
laboratory manual designed to teach explicit aspects of scientific thinking.
A careful study reported by Reif and St. John (1979) showed that students in a
college-level physics laboratory course based on inquiry training developed high-
level skills more successfully than did students in a conventional physics laboratory
course. The students in this laboratory course used instructional materials that
207

Downloaded from http://rer.aera.net at UNIV PRINCE EDWARD ISLAND on February 19, 2013
HOFSTEIN AND LUNETTA

presented "information in a carefully organized way [hierarchical explanation of


problem but no detailed instructions] and incorporated specific features stimulating
students to think independently" (p. 952) (e.g., filling in blanks in the scheme of the
investigation, making estimates before calculations, writing summaries of experi-
ments). A practical laboratory test examined the students' ability to:

(1) apply the underlying theory of an experiment to solve a similar problem


involving a different physical situation; or
(2) modify the experiment to find a different quantity, or to find the same
quantity by using different methods; or
(3) predict the effect of an error in an experimental procedure or measurement.
(p. 954)

Wheatley (1975) reported on a research study in college biology in which the


students were provided with special laboratory activities. Student performance at the
higher level of Bloom's cognitive taxonomy increased. High level cognitive abilities
were also measured in a study by Raghubir (1979). He found that students developed
important skills like formulating hypotheses, making assumptions, and designing and
executing investigations while conducting laboratory investigations in biology. Thus,
the enhancement of scientific thinking skills appears to be an important probable
outcome of laboratory instruction, and such skills should appear as measured
variables in more research studies.

Intellectual Development
Since 1960, instructional strategies in science have been influenced by Piaget's
developmental model. As a result, contemporary curricula frequently included the
manipulation of "concrete materials" in laboratory settings, requiring active involve-
ment on the part of the students (e.g., Lawson & Wollman, 1976).
Renner and Lawson (1973) and Karplus (1977) proposed a learning cycle to
promote science learning and students' intellectual development that consists of:
(1) Exploration: The student manipulates concrete materials and explores ques-
tions and relationships of interest;
(2) Concept introduction (invention): The teacher introduces scientific concepts
and terminology and structures relevant to the materials that have been
explored; and
(3) Concept application (discovery): The student investigates further questions
and applies the new concept in related but novel situations.
Fix and Renner (1979) used Piaget's (1970) model and Karplus' (1977) learning
cycle as a basis for teaching certain concepts in high school chemistry in Oklahoma.
Their method involved extensive exploration with laboratory materials followed by
the invention of explanations for the observations made. They reported that as a
result of this laboratory-centered chemistry curriculum, the enrollment in chemistry
doubled and student scores on the ACT chemistry subtest increased over 10 years.
The work reported by Fix and Renner may have been an important advance in
science teaching because the program made a well-developed attempt to match
curricula to students' intellectual development. On the other hand, it would be
unwise to generalize broadly on the basis of the data reported in this study, without
extending the study to other samples. Furthermore, it is difficult to discern from the

208

Downloaded from http://rer.aera.net at UNIV PRINCE EDWARD ISLAND on February 19, 2013
NEGLECTED ASPECTS OF RESEARCH

published account of this study how much of the reported improvement in achieve-
ment is actually due to work with materials in the laboratory and how much is due
to other variables that are unrelated to laboratory investigation.
It is quite clear that many interpreters of Piaget (e.g., Hineksman, 1973; Karplus,
1977; Lawson & Wollman, 1976) inferred that work with concrete objects is an
essential part of the development of logical thought, particularly prior to the time
that an individual reaches the developmental stage of formal operational thought.
However, it is not entirely clear whether this inference is data-based or simply an
assumption of the Piagetian paradigm. Certainly a thorough analysis of this question
has the potential to shed light on the need for school laboratory experience. As Bates
(1978) claimed, "Continuing research on the role of science teaching in nurturing
cognitive development may, in the relatively near future, provide important new
science teaching strategies in which properly designed laboratory activities will have
a central role" (p. 75).

Practical Skills and Abilities


Kelly and Lister (1969) claimed that "practical work involves abilities both manual
and intellectual, which are in some measures distinct from those used in non-practical
work" (p. 122).
Jeffrey (1967) suggested six abilities associated with laboratory work in chemistry:
(1) Communication: identification of laboratory equipment and operations;
(2) Observation: recording of observations and detecting errors in techniques;
(3) Investigation: accurate recording of measurable properties of an unknown
substance;
(4) Reporting: maintenance of a suitable laboratory record;
(5) Manipulation: skills in working with laboratory equipment; and
(6) Discipline: maintenance of an orderly laboratory and observation of safety
procedures.
Kempa and Ward (1975) suggested a four-phase taxonomy to describe the overall
process of practical work in science education:
(1) Planning and design of an investigation in which the student predicts results,
formulates hypotheses, and designs procedures;
(2) Carrying out of the experiment, in which the student makes decisions about
investigative techniques and manipulates materials and equipment;
(3) Observation of particular phenomena; and
(4) Analysis, application, and explanation, in which the student processes data,
discusses results, explores relationships, and formulates new questions and
problems.
Tamir (1974) designed an inquiry-oriented laboratory examination for Israeli
biology students. The student is evaluated on the bases of manipulation, self-reliance,
observation, experimental design, communication, and reasoning.
In summary, Jeffrey (1967), Kempa and Ward (1975), and Tamir (1974) created
what could serve as an organizer of laboratory work objectives that could help in the
design of meaningful instruments to assess outcomes of laboratory work.
On the whole, most past research studies on the educational effectiveness of
laboratory work neglected the important questions: What is the student really doing
in the laboratory? And, what are appropriate ways to measure his or her activity?
Grobman (1970) observed that in the "new" science teaching projects:

209

Downloaded from http://rer.aera.net at UNIV PRINCE EDWARD ISLAND on February 19, 2013
HOFSTEIN AND LUNETTA

With few exceptions evaluation has depended on written testing . . . there has
been little testing which requires actual performance in a real situation, or in
a simulated situation which approaches reality . . . to determine not whether
the student can verbalize a correct response but whether he can perform an
operation, e.g., a laboratory experiment or an analysis of a complex pro-
blem This is an area where testing is difficult and expensive yet since in
the long run primary aims of projects generally involve doing something rather
than writing about something this is an area which should not be neglected in
evaluation of curricula, (pp. 192-193)

The need to incorporate practical examinations when assessing the educational


effectiveness of the science laboratory is based on philosophical ideas as well as on
research findings. Kruglak (1958) suggested that "it is impossible to measure certain
neuro-muscular laboratory skills by means of paper-and-pencil tests. A student might
get a perfect score on written tests but not be able to handle apparatus in the
laboratory" (p. 32).
Robinson (1969), Tamir (1972), and Ben-Zvi, Hofstein, Samuel, and Kempa (1977)
found that a low correlation exists between laboratory-based practical examinations
and written paper-and-pencil-type tests. Although some recent attempts have been
made to incorporate practical examinations within evaluation projects (Ben-Zvi et
al, 1976a; Eglen & Kempa, 1974; Golmon, 1975) only a few research studies are
using practical examinations within the context of science research evaluation studies.
In summary, it is reasonable to suggest with Olson (1973) that "the laboratory
provides conditions for the acquisition of both intellectual and motor skillsnamely
an occasion for performance as well as feedback" (p. 34). Thus, the assessment of
these laboratory skills certainly should not be overlooked both in teaching and in
evaluation projects.

The Affective Domain


Attitude and interest. Developing favorable attitudes toward science has often been
listed as one of the important goals of science teaching. Generally, writers have
assumed that the availability of a wide variety of instructional materials will enable
teachers to vary classroom procedures, to avoid monotony, and to arouse interest
and attention. Smith, Walberg, Poorman, and Schagrin (1968), Selmes, Ashton,
Meredith, and Newal (1969), Ben-Zvi, Hofstein, Samuel, and Kempa (1976b),
Hofstein et al. (1976), and Raghubir (1979) found, for example, that students enjoy
laboratory work in some courses and that it generally results in positive and improved
attitudes toward, and interest in, the sciences.
In a research study conducted by Ben-Zvi et al. (1976a), chemistry students were
asked to rate the relative effectiveness of instructional methods. Students reported
that personal laboratory work was the most effective instructional method for
promoting their interest and learning when contrasted with teacher demonstrations,
group discussions, filmed experiments, and lectures. Similar results were obtained by
Bybee (1970) in comparing laboratory versus lecture demonstrations in a college-
level earth science course. Charen (1966) and Smith et al. (1968) found that laboratory
work enhanced student attitudes toward learning chemistry. Johnson, Ryan, and
Schroeder (1974) compared three groups of sixth grade science students: (1) a group
who learned science from a textbook; (2) a group that used a textbook and lab

210

Downloaded from http://rer.aera.net at UNIV PRINCE EDWARD ISLAND on February 19, 2013
NEGLECTED ASPECTS OF RESEARCH

materials; and (3) an activity-centered group that worked primarily with materials.
They found that students who interacted with concrete materials developed signif-
icantly more positive attitudes toward learning science than those who studied from
books alone. Thus, it seems that the laboratory can strongly affect attitude.
Curiosity. Curiosity has been identified as one of the important affective compo-
nents underlying inquiry skills (Bingman, 1969). Aiken and Aiken (1969) called traits
such as intellectual honesty, openmindedness, and curiosity "the more cognitive
scientific attitudes" (p. 295).
Sears and Hilgard (1964) claimed that curiosity is one of the neglected motives for
learning and that in the future it should receive more attention. A recent research
study conducted by Raghubir (1979) showed that biology students involved in a
laboratory-based investigative approach developed higher gains in curiosity when
compared to students who were involved in a lecture approach. Scientific curiosity
could be measured by an instrument similar to the one developed by Campbell
(1972) and used by Tamir (1978) and Hofstein, Ben-Zvi, and Welch (1981) in
inquiry-type science curricula. This inventory is limited to the three levels of the
taxonomy (receiving, responding, and valuing) in the affective domain (Krathwohl,
Bloom, & Masia, 1964), and it attempts to measure how far the student would be
willing to go to satisfy his or her curiosity. The extent to which science laboratory
teaching is successful in developing curiosity has not been clearly established, but it
is reasonable to hypothesize that appropriate laboratory work could help develop
curiosity in certain students.
In summary, if one agrees that "we are entering an era when we will be asked to
acknowledge the importance of affect, imagination, intuition and attitude as outcomes
of science instruction as at least as important as their cognitive counterparts"
(Shulman & Tamir, 1973, p. 1139), affective outcomes of laboratory instruction
should certainly be given more emphasis in research studies.

Social Variables
The learning environment was defined by Anderson (1973) as "the interpersonal
relationship among pupils, relationship between pupils and their teachers, relation-
ship among pupils and both the subject matter studied and the method of learning
and finally, pupil reception of the structural characteristics of the class" (p. 1).
According to Walberg (1969) perception of the learning environment is sensitive to
instructional and psychological treatments; measures of perception of the learning
environment can differentiate between groups of students using different instructional
methods. In recent years, conceptualization and assessment of the learning environ-
ment has received considerable attention (Anderson, 1973; Anderson & Walberg,
1974), which is reflected in many recent studies. Studies using the Learning Environ-
ment Inventory (LEI) have shown that student perceptions of classroom environment
are related to cognitive, affective, and behavioral measures of learning (Walberg,
Rentoul & Frazer, 1979). Rentoul and Frazer (1979) in Australia found that a
modified LEI distinguished between inquiry-based science classes using the ASEP
curriculum and noninquiry-based classes using a conventional science course. Stu-
dents involved in inquiry-type activities found the classroom environment more
satisfying. It has been suggested that certain learning environment variables are
affected by the kinds of laboratory work activities in which the student is involved.
Support for this assumption is given by Egelston (1973), who found that inductive

211

Downloaded from http://rer.aera.net at UNIV PRINCE EDWARD ISLAND on February 19, 2013
HOFSTEIN AND LUNETTA

laboratory activities created a different learning environment as measured by the


LEI. This was also supported by a preliminary study (Hofstein et al., 1980) that
reported results concerning the use of the LEI in two populations (high school and
technical school) that differed in time spent in the chemistry laboratory.
Laboratory activity has the potential to enhance constructive social relationships
as well as positive attitudes and cognitive growth. The cooperative team effort
required by many laboratory activities may promote positive social interactions
involving cohesiveness, task orientation, goal direction, democracy, satisfaction, and
other factors measured on the LEI.
Because creating a healthy learning environment is an important goal for many
contemporary educators, there is a need for more research that will assess how the
time spent in laboratory work and how specific activities in the laboratory affect the
learning environment. It would also be desirable to study further the effects of
different modes of practical work (such as gathering data to confirm relationships
versus developing generalizations through open inquiry) on the learning environ-
ment.

Summary
The laboratory provides a unique medium for teaching and learning in science
education, and science educators have suggested that rich benefits in learning accrue
from its use. Yet, researchers have not comprehensively examined the effects of
laboratory instruction on student learning and growth in contrast to other modes of
instruction, and there is insufficient data to confirm or reject convincingly many of
the statements that have been made about the importance and the effects of
laboratory teaching (Watson, 1978). The research has failed to show simplistic
relationships between experiences in the laboratory and student learning. This
revelation should not be especially surprising considering the complexity of human
learning; much more information and study are needed to clarify the relationships
that do exist. It is unreasonable to assert that the laboratory is an effective and
efficient teaching medium for achieving all goals in science education. On the other
hand, sufficient data do exist to suggest that laboratory instruction may play an
important part in the achievement of some of these goals. Appropriate laboratory
activities can be effective in promoting logical development and the development of
some inquiry and problem-solving skills. For example, they can assist in the devel-
opment of manipulative and observational skills and in understanding scientific
concepts. They also can promote positive attitudes, and they provide opportunities
for student success and foster the development of skills in cooperation and commu-
nication.
Research studies have often compared one method of laboratory work with others
or with more conventional classroom teaching over relatively short periods of time.
Many of these studies have reported nonsignificant results, meaning that the labo-
ratory medium was at least as effective in promoting student growth on the variables
measured as were more conventional modes of instruction. If differences in learning
did occur between students using a laboratory mode of learning and those involved
in more conventional instruction, those differences were probably masked by con-
founding variables, by insensitive instrumentation, or by poor experimental design.
Indeed, as reported earlier, the variables that have been controlled and measured
often have been only a subset of important dependent and independent variables.

212

Downloaded from http://rer.aera.net at UNIV PRINCE EDWARD ISLAND on February 19, 2013
NEGLECTED ASPECTS OF RESEARCH

Seldom has attention been given to the characteristics of the student sample or even
to describing the nature of the laboratory instruction. Researchers need to be
especially careful in the selection, control, and documentation of activities prior to
and during the research study. They should also look carefully at promising variables
that have been insufficiently examined in the past.
Research must now be done on specific conditions and strategies of laboratory
work, on their effects on a range of learning outcomes, and on their interactions.
Research into the effectiveness of science laboratory experiences should use valid
methods to monitor dependent and independent variables more carefully than studies
have in the past. Important variables in the laboratory environment that should be
studied include:
(1) Teacher attitudes and behavior
(2) Content and nature of laboratory activities
(3) Instructional goals
(4) Social variables/learning environment
(5) Management (temporal placement of activities in curriculum; method of
student evaluation; time allotted to activities; method of grouping students;
and availability of space and materials).
Important student characteristics that should be studied include:
(1) Student behavior
(2) Intellectual development
(3) Conceptual understanding
(4) Skill level (inquiry and problem-solving skills; mathematical skills; reading
skills; and manipulative skills)
(5) Attitudes toward a variety of relevant issues (interest and curiosity).
Researchers must examine the goals of science teaching and learning with care to
identify optimal activities and experiences from all modes of instruction that will
best facilitate these goals. While the relevant variables are interrelated and complex,
there is a real need to pursue vigorously research on learning through laboratory
activities to capitalize on the uniqueness of this mode of instruction for certain
learning outcomes. With more precise information on these important questions and
relationships, more comprehensive teaching models can be designed to incorporate
information about goals, the nature of science, and the way people learn, which will
enable science teachers to become more effective in facilitating student learning and
development.

References
Aiken, L. R., & Aiken, D. R. Recent research on attitude concerning science. Science Education,
1969, 53, 295-305.
Anderson, G. J. The assessment of learning environments. Halifax, Nova Scotia: Atlantic Institute
of Education, 1973.
Anderson, G. J., & Walberg, H. J. Learning environments. In H. J. Walberg (Ed.), Evaluating
educational performance. Berkeley, Calif.: McCutchan, 1974.
Anderson, R. O. The experience of science: A new perspective for laboratory teaching. New York:
Columbia University, Teachers College Press, 1976.
Ausubel, D. P. Educational psychology. New York: Holt, Rinehart, & Winston, 1968.
Barnes, L. W. The development of a student checklist to determine laboratory practices in his
high school biology. In A. E. Lee (Ed.), Research and curriculum development in science
education (Vol. 1). Austin, Tex.: The University of Texas, 1967.

213

Downloaded from http://rer.aera.net at UNIV PRINCE EDWARD ISLAND on February 19, 2013
HOFSTEIN AND LUNETTA

Bates, G. R. The role of the laboratory in secondary school science programs. In M. B. Rowe
(Ed.), What research says to the science teacher (Vol. 1). Washington, D.C.: National Science
Teachers Association, 1978.
Belanger, M. Learning studies in science education. Review of Educational Research, 1971, 39,
377-395.
Ben-Zvi, R., Hofstein, A., Kempa, R. F., & Samuel, D. The effectiveness of filmed experiments
in high school chemical education. Journal of Chemical Education, 1976, 53, 518-520. (a)
Ben-Zvi, R., Hofstein, A., Samuel, D., & Kempa, R. F. The attitude of high school students to
the use of filmed experiments in laboratory instruction. Journal of Chemical Education, 1976,
53, 575-577. (b)
Ben-Zvi, R., Hofstein, A., Samuel, D., & Kempa, R. F. Modes of instruction in high school
chemistry. Journal of Research in Science Teaching, \9, 14, 433-439.
Bingman, R. M. (Ed.). Inquiry objectives in the teaching of biology. Mid-Continental Regional
Educational Laboratory and the Biological Science Curriculum Study, 1969.
Bloom, B. S. (Ed.). Taxonomy of educational objectives. Handbook I: Cognitive domain. New
York: David McKay, 1956.
Bradley, R. L. Is the science laboratory necessary for general education science courses? Science
Education, 1968, 52, 58-65.
Burmester, M. A. The construction and validation of a test to measure some of the indicative
aspects of scientific thinking. Science Education, 1953, 37, 131-140.
Bybee, R. W. The effectiveness of an individualized approach to a general education earth
science laboratory. Science Education, 1970, 54, 157-161.
Campbell, J. R. Is scientific curiosity a viable outcome in today's secondary school science
program? School Science and Mathematics, 1972, 72, 139-146.
Charen, G. Laboratory methods build attitudes. Science Education, 1966, 50, 54-57.
Connelly, M. Implementation, evaluation, and professional development of science teaching. In
P. Tamir, A. Blum, A. Hofstein, & N. Sabar (Eds.), Proceedings of the Hebrew University
conference on Curriculum Implementation and its Relationship to Curriculum Development in
Science. Jerusalem, 1979.
Coulter, J. C. The effectiveness of inductive laboratory demonstration and deductive laboratory
in biology. Journal of Research in Science Teaching, 1966, 4, 185-186.
Cunningham, H. A. Lecture demonstration versus individual laboratory method in science
teaching: A summary. Science Education, 1946, 30, 70-82.
Egelston, J. Inductive vs. traditional method of teaching high school biology laboratory
experiments. Science Education, 1973, 57, 467-477.
Eggleston, J. F., Galton, M., & Jones, M. C. Process and product of science teaching. (School
Council Research Series) London: McMillan Education, 1976.
Eglen, J. R., & Kempa, R. F. Assessing manipulative skills in practical chemistry. School
Science Review, 1974, 56, 261-273.
Fix, W. T., & Renner, J. W. Chemistry and experiments in the secondary school. Journal of
Chemical Education, 1979, 56, 737-740.,
Fuhrman, M., Lunetta, V. N., Novick, S., & Tamir, P. The laboratory structure and task analysis
inventory (L.A.I.): A user's handbook. (Tech. Rep. No. 14). Iowa City, Iowa: The University
of Iowa Science Education Center, 1978.
Gardner, M. Trends in development and implementation of science curriculum in the U.S.A.
In P. Tamir, A. Blum, A. Hofstein, & N. Sabar (Eds.), Proceedings of Hebrew University
conference on Curriculum Implementation and its Relationship to Curriculum Development in
Science. Jerusalem, 1979.
Getzels, J. W. Creative thinking, problem-solving and instruction. In E. R. Hilgard (Ed.),
Theories of learning and instruction. The 63rd Yearbook of the National Society for the Study
of Education. Chicago: University of Chicago Press, 1963.
Golmon, M. Assessing laboratory instruction in biology. Iowa Science Teacher Journal, 1975,
79,4-10.

214

Downloaded from http://rer.aera.net at UNIV PRINCE EDWARD ISLAND on February 19, 2013
NEGLECTED ASPECTS OF RESEARCH

Grobman, H. Developmental curriculum projects: Decision points and processes. Itasca, 111.:
Peacock Publishers, 1970.
Hill, B. W. Using college chemistry to influence creativity. Journal of Research in Science
Teaching, 1976, 13, 71-77.
Hineksman, N. O. The function of the school laboratory. The Australian Science Teacher
Journal, 1973, 19, 81-86.
Hofstein, A., Ben-Zvi, R., & Samuel, D. The measurement of interest in, and attitude to
laboratory work amongst Israeli high school students. Science Education, 1976, 60, 401-411.
Hofstein, A., Ben-Zvi, R., Gluzman, R., & Samuel, D. A comparative study of chemistry
students' perception of the learning environment in high schools, and vocational schools.
Journal of Research in Science Teaching, 1980, 17, 547-552.
Hofstein, A., Ben-Zvi, R., & Welch, W. W. Some aspects of scientific curiosity in secondary
school students. Science Education, 1981, 65, 229-235.
Hurd, P. D. New directions in teaching secondary school science. Chicago: Rand McNally, 1969.
Jeffrey, J. Identification of objectives of the chemistry laboratory and development of means for
measuring student achievement of some of these objectives. Unpublished Ph.D. thesis, Univer-
sity of Texas, 1967.
Johnson, R. T., Ryan, F. L., & Schroeder, H. Inquiry and the development of positive attitudes.
Science Education, [974,58, 51-56.
Kaplan, E. H. The Burmester test of aspects of scientific thinking as means of teaching the
mechanics of scientific method. Science Education, 1967, 51, 353-357.
Karplus, R. Science teaching and the development of reasoning. Journal of Research in Science
Teaching, 1977, 14, 169-175.
Kelly, P. J., & Lister, R. F. Asssessing practical ability in Nuffield A-level biology. In J. F.
Egelston & J. F. Kerr (Eds.), Studies in Assessment. London: English University Press, 1969.
Kempa, R. F., & Ward, J. E. The effect of different modes of task orientation on observational
attainment in practical chemistry. Journal of Research in Science Teaching, 1975, 12, 69-76.
Krathwohl, D. R., Bloom, B. S., & Masia, B. B. Taxonomy of educational objectives: Handbook
IT. Affective domain. New York: David McKay, 1964.
Kruglak, H. Evaluating laboratory instruction by use of objective type tests. American Journal
of Physics, 1958,26,31-32.
Lawson, A. E., & Wollman, W. Encouraging the transition from concrete to formal cognitive
functioning: An experiment. Journal of Research in Science Teaching, 1976, 12, 413-430.
Lucas, A. M. Creativity, discovery and inquiry in science education. The Australian Journal of
Education, 1971, 15, 185-196.
Lunetta, V. N. Computer-based dialogs: A supplement to the physics curriculum. The Physics
Teacher, 1974, 12, 355-356.
Lunetta, V. N., & Tamir, P. An analysis of laboratory activities in two modern science curricula:
Project Physics and PSSC. Paper presented at the annual meeting of the National Association
for Research in Science Teaching, Toronto, Ontario, 1978.
Lunetta, V. N., & Tamir, P. Matching lab activities with teaching goals. The Science Teacher,
1979, 46, 22-24.
Olson, D. R. What is worth knowing and what can be taught? School Review, 1973, 82, 27-43.
Penick, J. E. Creativity of fifth grade science students: The effect of two patterns of instruction.
Journal of Research in Science Teaching, 1976, 13, 307-314.
Penick, J. E., Shymansky, J. A., Filkins, K. M., & Kyle, W. C , Jr. Science Laboratory Interaction
Category (SLIC) - Student. Iowa City: Science Education Center, University of Iowa, 1976.
Piaget, J. Structuralism (Trans, by Chaninah Maschler). New York: Basic Books, 1970.
Raghubir, K. P. The laboratory investigative approach to science instruction. Journal of
Research in Science Teaching, 1979, 16, 13-18.
Ramsey, G. A., & Howe, R. W. An analysis of research on instructional procedures in secondary
school science: Part II. The Science Teacher, 1969, 36 (4), 72-81.

215

Downloaded from http://rer.aera.net at UNIV PRINCE EDWARD ISLAND on February 19, 2013
HOFSTEIN AND LUNETTA

Reif, F., & St. John, M. Teaching physicists thinking skills in the laboratory. American Journal
of Physics, 1979, 47 (11), 950-957.
Renner, J. W., & Lawson, A. E. Promoting intellectual development through science teaching.
The Physics Teacher, 1973, 11, 273-276.
Rentoul, A. J., & Frazer, B. J. Conceptualization and assessment of enquiry-based or open-
classroom learning environments. Journal of Curriculum Studies, 1979, 11, 233-245.
Robinson, T. J. Evaluating laboratory work in high school biology. The American Biology
Teacher, 1969, 34, 226-229.
Romey, W. D. What is your creativity quotient? School Science and Mathematics, 1970, 70, 3 -
8.
Rosen, S. A. History of the physics laboratory in the American public schools (to 1910).
American Journal of Physics, 1954,22, 194-204.
Saunders, W. L., & Dickinson, H. D. A comparison of community college students' achievement
and attitude change in lecture-only and lecture-laboratory approach to general education
biological science courses. Journal of Research in Science Teaching, 1979, 16, 465-472.
Schwab, J. J. The teaching of science as inquiry. In J. J. Schwab & P. F. Brandweine (Eds.),
The teaching of science. Cambridge, Mass.: Harvard University Press, 1962.
Sears, P. S., & Hilgard, E. R. The teacher role in the motivation of the learner. In E. R. Hilgard
(Ed.), Theories of learning and instruction. The 63rd Yearbook of the National Society for the
Study of Education. Chicago: University of Chicago Press, 1964.
Selmes, A. F., Ashton, B. G., Meredith, H. M., & Newal, A. B. Attitude to science and scientists.
School Science Review, 1969, 51, 7-22.
Shulman, L. D. & Tamir, P. Research on teaching in the natural sciences. In R. M. W. Travers
(Ed.), Second handbook of research on teaching. Chicago: Rand McNally, 1973.
Shymansky, J. A., Penick, J. E., Kelsey, L. J., & Foster, G. W. Science Laboratory Interaction
Category (SLIC) - Teacher. Iowa City: Science Education Center, University of Iowa, 1976.
Silberstein, M. Are curriculum implementation constraints a part of curriculum development?
In P. Tamir, A. Blum, A. Hofstein, & N. Sabar (Eds.), Proceedings of a Hebrew University
conference on Curriculum Implementation and Its Relationship to Curriculum Development in
Science. Jerusalem, 1979.
Smith, D. M., Walberg, H. J., Poorman, E. L., & Schagrin, M. Affective response to different
media in a multi-media system. Science Education, 1968, 52, 16-22.
Smith, J. P. The development of a classroom observation instrument relevant to the earth
science curriculum project. Journal of Research in Science Teaching, 1971, 89, 231-235.
Stephens, J. M. The process of schooling: A psychological examination. New York: Holt,
Rinehart, & Winston, 1967.
Tamir, P. The practical mode: A distinct mode of performance. Journal of Biological Education,
1972,6, 175-182.
Tamir, P. An inquiry-oriented laboratory examination. Journal of Educational Measurement,
1974, 11, 23-25.
Tamir, P. How are laboratories used? Journal of Research in Science Teaching, 1977, 14, 311-
316.
Tamir, P. Inquiry and curiosity in biology. Journal of Biological Education, 1978, 12, 215-223.
Walberg, H. J. Predicting class learning: A generalized regression approach to the class as a
social system. American Educational Research Journal, 1969, 4, 529-542.
Walberg, H. J. Educational environment effects, evaluation, policy and productivity. Berkeley,
Calif.: McCutchan, 1979.
Watson, F. G., (cited by Fairbroder, R. W.) Assessment ofpractical work. In J. G. Jones & J. L.
Lewis (Eds.), The role of the laboratories in physics education. Proceedings of the International
Research Group on Physics Teaching Conference, Oxford, U.K. (1978).
Watson, G., & Glazer, E. M. W.G.C.T.A.: Watson-Glazer Critical Thinking Appraisal. New
York: Harcourt, Brace, & World, 1961.
Welch, W. W. Science Education Information Report: Research Review Series: Science Paper 7:

216

Downloaded from http://rer.aera.net at UNIV PRINCE EDWARD ISLAND on February 19, 2013
NEGLECTED ASPECTS OF RESEARCH

A Summary of Research in Science Education for the Years 968-1969. Secondary School
Level. Columbus, Ohio: ERIC Science Mathematics, and Environmental Education Clear-
inghouse, Ohio State University, 1971.
Wheatley, J. H. Evaluating cognitive learning in the college science laboratory. Journal of
Research in Science Teaching, 1975, 12, 101-109.
Yager, R. E., Engen, J. B., & Snider, C. F. Effects of the laboratory and demonstration method
upon the outcomes of instruction in secondary biology. Journal of Research in Science
Teaching, 1969, 5, 76-86.

AUTHORS
AVI HOFSTEIN, Senior Member of the Chemistry Group, Dept. of Science
Teaching, The Weizmann Institute of Science, Rehovot 76100, Israel. Specializa
tions: Science curriculum; development, implementation, and evaluation.
VINCENT LUNETTA, Professor, Science Education Center, The University of
Iowa, Iowa City, IA 52242. Specializations: Science education; curriculum devel-
opment; teacher evaluation.

217

Downloaded from http://rer.aera.net at UNIV PRINCE EDWARD ISLAND on February 19, 2013

Anda mungkin juga menyukai