Embedding Cooperative Learning into the

Design of Integrated Learning Systems:

Rationale and Guidelines
[] Thomas A. Brush

An Integrated Learning Systera (ILS) is an

advanced computer-based instructional system, generally consisting of a set of computerized courseware covering several grade levels
and content areas, and complex classroom
management and reporting features. Although
ILSs have become increasingly popular in
schools over the past five to ten years, they
introduce several potential factors that could
have negative effects on students' academic
and social growth. These factors include:
(a) de-emphasis of affective outcomes and
increased student isolation, (b) lack of teacher
involvement in curriculum planning and delivery, and (c) disparate effects on student
achievement based on students' academic level.
One possible strategy for addressing these
concerns is integrating cooperative learning
with ILS instruction. This paper will examine
the research dealing with integrating cooperative learning strategies and computer-based
instruction and provide guidelines and
strategies for designing ILS instruction that
enhances opportunities for cooperative
learning.

[] An Integrated Learning System (ILS) is an

advanced computer-based instructional system,
generally consisting of a set of computerized
courseware covering several grade levels and
content areas, complex classroom management
and reporting features, and supplementary
materials (workbooks, audiotapes, manipularives) to be used by students when they are not
engaged in the computer-based activities (Becker & Hativa, 1994; Mills, 1994; Robinson, 1991;
Wiburg, 1995). The ILS software is usually
stored on a central file server and distributed to
lab- or classroom-based workstations via the
school'6 local area network (Roblyer, Edwards,
& Havriluk, 1997; Sherry, 1990).
The design of most current TLSs is based on
the theory that learning is best facilitated not by
providing opportunities for social interaction
and dialogue among learners, but by meeting
the unique needs of each individual (Becker,
1992b; Hativa, 1994; Mevarech, 1994). Thus, ILSs
are designed primarily to be used by students
individually so that learners can receive instruction, feedback, and remediation that is tailored
to individual levels optimal for learning. This
approach perpetuates a design that includes
components for determining student entry
skills, placing students at a level of instruction
consistent with their entry skills, providing
opportunities to practice skills with immediate
and appropriate feedback, and testing for mastery and providing remediation if needed (Becker, 1992b).
For example, a typical ILS curriculum
sequence begins by administering an individual
placement test to each student in order to determine academic entry level. Once this is determined, the ILS places each student at the

ETR&D. Vol. 46. No. 3. 1998, pp. 5.-18 ISSN 1042-1629

ETR&D.

beginning of an instructional unit consistent

with the appropriate level. This unit is divided
into several sections or lessons comprised of
interactive information presentation and practice activities. If the student is having difficulty
with some of the content within a particular lesson, the ILS branches the student to additional
remediation activities. Once the student has
completed the sequence of lessons for the unit, a
more extensive test is administered in order to
provide both the student and the teacher with
information regarding the student's mastery of
the concepts included in the unit. Many systems
also administer a comprehensive achievement
test at the end of an entire curriculum sequence
(e.g., the fifth-grade math sequence) in order to
provide schools with data regarding students'
academic growth throughout the year. Figure 1
illustrates a typical ILS design sequence.

Figure 1 []

Vo146.

The ILS has become increasingly popular with

K-12 schools over the past five to ten years. Estimates are that between 11% and 25% of schools
in the United States currently own ILSs (Brush &
Bannon, 1998; Clariana, 1996; White, 1992), and
that ILSs account for nearly 50% of total educational software purchases (Bailey, 1993). There
are two explanations for the prevalence of these
systems. First and foremost, school leaders
believe ILSs are effective in raising standardized
tests scores, particularly with older students and
students who have difficulty learning from traditional classroom-based methods (i.e., low and
high achievers). Several research studies have
concluded that ILSs have a positive impact on
academic achievement (Alfrangis, 1989; Bender,
1991; Clariana, 1994, 1996; Hativa, 1994; Van

Courseware Sequence

Louon 1

Unit 2

Lesson 1

.....

LEmon 1

:' . o~. . . . . . . . . . .

Test
Lesson 2

ILS POTENTIAL A N D PROBLEMS

T y p i c a l ILS d e s i g n s e q u e n c e .

Unit I

No

'."

-."

I..elNl~l 2

, . . : . . ! ......

.. ~o .

:~" Unit ".

' Test
- . . . . ,

;....
,,-

COOPERATIVE LEARNINGILSDESIGN

Dusen & Worthen, 1994; Worthen, Van Dusen,

& Sailor, 1994). Second, many school leaders
believe these systems provide a "turnkey"
implementation process for integrating computer-based instruction into the curriculum
(Becket, 1994; Shockley, 1992; Wiburg, 1995). In
fact, many ILS vendors use ease of implementation and the availability of courseware, training,
and support from one source as major reasons
for buying their products.
These perceived benefits of ILSs are not without their critics. Some researchers believe that
the academic impact of ILSs is somewhat disappointing when compared to the investment in
monetary and personnel resources needed to
purchase and maintain these systems. Becker
(1992, 1994) conducted meta-analyses of numerous studies examining the academic impact of
ILSs. Although he found that ILSs had moderately positive effects on student achievement, he
concluded that ILSs would have a greater
impact if teachers and school leaders abandoned
their "mindless adherence to the principle of
individualized instruction" (Becker, 1994, p. 78),
and explored alternative implementation strategies for these systems. Both Wiburg (1995) and
Maddux and Willis (1992) also cautioned that
research regarding ILSs and academic achievement was inconclusive and that further study
was needed in this area.
From the standpoint of ease of implementation, studies have shown that, in many
instances, a belief in the turnkey approach leads
to lack of teacher involvement during ILS sessions. As Becker (1994) stated, "Because such
programs can run with little intervention from
the teacher.., it is tempting for schools to allow
ILS programs to run essentially unattended
except for the technical support provided by
systems managers . . ." (p.78). This lack of
teacher involvement has led to improper coordination between classroom-based and computerbased instructional activities, inadequate
student support while they are completing ILS
instruction, and lack of teacher understanding
regarding effective strategies and procedures for
using ILSs (Becket, 1992b, 1994; Brush, 1997;
Hativa, 1994; Sherry, 1990).
In addition to the lack of clear evidence
regarding the academic impact of ILSs and the

7
need for more teacher involvement when
implementing these systems, researchers have
cautioned that long-term individualized use of
ILSs, which is generally the recommended
method for using these systems in schools
(Hativa, 1994; West & Marcotte, 1993-94), can
have a variety of adverse effects on students
(Becket, 1992b; Brush, 1997; Mevarech, 1994).
Research on affective dimensions of ILSs has
shown that long-term individualized usage of
ILSs by students leads to anxiety and hostility
toward the subject matter (Brush, 1997; Lepper,
1985), increased feelings of inadequacy and
helplessness (Hativa, Swissa, & Lesgold, 1992;
Mevarech, 1994), a general dislike of the ILS
activities, particularly among low-achieving students (Brush, 1997; Hativa, 1994), and a decrease
in teacher interaction with students (Becker,
1994).
A second concern is that, while ILSs appear
to have a positive effect on achievement levels of
low- and high-achieving students, they do little
to foster academic growth of average students.
Osin, Nesher, & Ram (1994) analyzed achievement results from 15 different schools using ILSs
for math instruction and found a curvilinear
relationship, with low and high achievers performing better than medium achievers. Similar
results have been reported by Becker (1992b,
1994) and Hativa (1994). As Becker (1992b)
stated, "[ILSs] are much less likely to help stu"dents in the middle of the class distribution, who
are less likely to need a different level or pace of
instruction compared to what they receive in
traditional whole-class teaching" (p.10). Thus,
the traditional individualized delivery model
for ILS instruction does not appear to equally
address the individual needs of all students
effectively.
The research reviewed above suggests that
ILSs introduce several potential factors that, at
the very least, impede the overall effectiveness
of the systems and, at their worst, could have
negative effects on students' academic and
social growth. These factors include: (a) deemphasis of affective outcomes and increased
student isolation, (b) lack of teacher involvement
in curriculum planning and delivery, and (c)
disparate effects on student achievement based
on students' academic level.

How might the design of ILSs and/or the

delivery of ILS instruction be improved to
address these problems while maintaining (or
increasing) the benefits these systems provide to
schools (i.e., improved test scores, ease of
implementation)? One strategy that has been
effective with computer-assisted instruction
(CAI) but has not been widely researched with
regard to ILSs is the use of cooperative learning
groups (Brush, 1997; Hooper & Hannafin, 1991;
Hooper, Temiyakarn, & Williams, 1993;
Mevarech, 1994). Research has demonstrated
that cooperative learning improves students'
social interaction skills (Lloyd, Crowley, Kohler,
& Strain, 1988, Mesch, Lew, Johnson, & Johnson,
1986), promotes more teacher involvement with
individual students (Hertz-Lazarowitz &
Schachar, 1990; Sharan & Sharan, 1992), and has
positive academic affects for students of all ability levels (Simsek & Hooper, 1992; Slavin, 1991;
Yager, Johnson, & Johnson, 1985). Thus, it is possible that the instructional issues associated with
ILSs could be addressed through the integration
of cooperative learning strategies with ILS
instruction.
Although combining cooperative learning
with ILSs appears to be a potential solution to
many of the problems associated with ILSs,
there are currently no ILS systems that overtly
embed opportunities for cooperative learning
into their courseware, additionally, few systems
provide any procedures or materials to assist
teachers with integrating cooperative learning
activities into ILS instruction. Furthermore,
while researchers have discussed the potential
benefits of combining cooperative learning with
CAI and ILS instruction and have provided limited strategies for incorporating cooperative
learning into computerized instruction (e.g. Becker, 1992b; Hooper, 1992; Rysavy & Sales, 1991),
there has been little discussion regarding how to
design computer-based instruction (and ILS
instruction in particular) that integrates key
components of cooperative learning. If design
guidelines provided examples and strategies for
embedding cooperative learning strategies into
ILS instruction, many of the concerns and criticisms of long-term individualized exposure to
ILSs could be addressed, As schools invest more
resources into ILSs and push for longer and

ETR&D.Vo146.No. 3

more frequent student interactions with ILS

courseware, alternative strategies for delivery
and management of ILS instruction become
more critical. This paper examines the research
dealing with integrating cooperative learning
strategies and computer-based instruction and
provides guidelines and strategies for designing
ILS instruction that enhances opportunities for
cooperative learning.

OVERVIEW OF
COOPERATIVE LEARNING

Cooperative learning is defined by Deutsch

(1962) as a learning situation in which students
working in groups can achieve the goals of an
instructional activity only if the other students
with whom they are working achieve the goals
as well. This can be contrasted to individualistic
learning, in which students' achievement of
goals is not dependent on other students' work
(Yager, Johnson, Johnson, & Snider, 1985). However, there is still confusion as to what learning
situations constitute cooperative learning. Placing students in groups in order for them to complete an instructional activity does not in itself
promote cooperation between and among the
group members (Johnson & Johnson, 1991). In
fact, unstructured group activities could discourage rather than encourage performance
from the individual members of the group
(Johnson & Johnson, 1991; Slavin, 1995). In some
group activities, the "free rider" effect may
occur, where less able members allow other
group members to complete the majority of the
activities (Hooper, 1992; Kerr & Bruun, 1983;
Slavin, 1995). Such activities may also lead to the
"sucker effect," where the more able members of
the group expend less effort so that they avoid
having to do all the work for the group (Hooper,
1992; Johnson & Johnson, 1991; Kerr, 1983).
Various cooperative learning strategies have
been developed in an attempt to address these
factors. Some of the more popular strategies
include group investigation (Sharan & Sharan,
1992), learning together (Johnson & Johnson,
1987, 1991), and student team iearningmethods
such as Jigsaw (Aronson & Patnoe, 1997), Jigsaw
II (Slavin, 1986), and Team Accelerated Instruc-

COOPERATIVELEARNINGILSDESIGN

tion (Slavin, Leavey, & Madden, 1986). While

these models vary in implementation, several
key components are present in each: positive
interdependence, individual accountability, and
collaborative skills.

Positive Interdependence
"Positive interdependence is the perception that
you are linked with others in a way so that you
cannot succeed unless they do (and vice versa);
that is, their work benefits you, and your work
benefits them" (Johnson & Johnson, 1991, p.
127). In other words, each group member
believes he or she has a key role or responsibility
within the group and that the success of the
group depends upon each member succeeding
in that role. Several types of positive interdependence are outlined by Johnson and Johnson
(1991). These include positive goal interdependence, in which students perceive that they can
achieve their goals only if all members of the
group achieve their goals also; positive reward
interdependence, where all group members
receive the same reward for completing the task;
positive resource interdependence, in which each
group member has only a portion of the materials or information needed for the task and the
resources must be combined to complete the
task; positive role interdependence, in which each
member of the group is assigned a unique role
complementary to the roles of other group members; identity interdependence, in which the group
establishes a unique identity through a group
name or symbol; and environmental interdependence, in which the group members are forced to
be together due to environmental constraints
such as an assigned group meeting area.

Promoting positive interdependence. According

to Johnson and Johnson (1991), positive interdependence begins with the establishment of
group goals and rewards (i.e., establishing goal
and reward interdependence). This goal/
reward structure helps maintain the cohesivehess of the group and provides students with an
incentive to help and encourage each other
(Slavin, 1993). One of the most widely used
goal/reward structures involves measuring the
success of the group based on a combination of
individual achievement measures of group

members. This may include giving bonus points

to a group for each member scoring above 80%
on a quiz (Johnson & Johnson, 1991); rewarding
a group with additional free time when all members of the group pass a test (Johnson & Johnson,
1991; Slavin, 1995); computing a "team score"
for a test by averaging the individual grades of
all group members and providing achievement
certificates to the group with the highest team
score (Slavin, 1985,1990); and posting the names
of all groups whose members score 100% on
periodic unit tests (Brush, 1997).
Researchers have also developed strategies
for promoting other forms of positive interdependence. Aronson and Patnoe (1997) promote
positive role and resource interdependence by
dividing resources and tasks among group
members so that each member has unique information and responsibilities needed to complete
the assignment or project. Positive identity interdependence can be established by constructing a
competitive environment among groups in the
class (Aronson & Patnoe, 1997; Johnson & Johnson, 1991; Slavin, 1995) or by providing each
group with a unique identity or meeting area
(Johnson & Johnson, 1991).

Individual A c c o u n t a b i l i t y

Individual accountability means that each member of the cooperative group should master the
information for which the group is responsible
(Aronson & Patnoe, 1997; Brush, 1997; Slavin,
1995). As Slavin (1995) stated, "From early on,
reviewers of the cooperative learning literature
have concluded that cooperative learning has its
greatest effects on student learning when groups
are recognized or rewarded based on the individual learning of their members" (p. 41).

Establishing individual accountability. The simplest method for incorporating individual

accountability into cooperative learning is to
require each group member to complete a test
on the material individually after the cooperative learning activities have been concluded
(Slavin, 1995). This measure of individual
achievement may be used in a variety of ways.
Student grades may be based on an average of
individual test scores of all group members
(Slavin, 1995). The teacher may also select one

10
student at random and have that student's test
score represent the score for the group (Aronson
& Patnoe, 1997; Johnson & Johnson, 1991). Other
methods for promoting individual accountability include randomly requiring a group member
to explain an answer (Johnson & Johnson, 1991),
determining individual grades based on peer
evaluations of group members (Slavin, 1995),
and evaluating students based on the unique
tasks they needed to complete individually in
order for an overall group product to be completed (Aronson & Patnoe, 1997).

Teaching and Reinforcing

Collaborative Skills
Creating a cooperative learning structure that
promotes positive interdependence and establishes individual accountability does not guarantee the success of the cooperative learning
activity. Students also need to learn collaborative skills in order to work effectively with other
members of a group. As Johnson & Johnson
(1991) stated, "Students who have never been
taught how to work effectively with others cannot be expected to do so" (p. 146). Collaborative
skills include communicating ideas, building
and maintaining trust among group members,
providing leadership, and managing group conflicts (Aronson & Patnoe, 1997; Johnson & Johnson, 1990, 1991). Slavin (1995) also suggested
training students on specific learning strategies
such as prediction, summarization, and question generation, as well as providing structured
methods for using these strategies with their
partner(s).

Methods for teaching collaborative skills. Many

of the methods for teaching and building collaborative skills employ preinstructional activities
to introduce the skills and provide opportunities
for students to practice them. For example,
Aronson and Patnoe (1997) suggested several
team building activities for students to complete
prior to engaging in the target instructional
activity: "Learning to Listen," which emphasizes turn taking and listening to others; "Group
Picture," which helps students understand the
importance of each member's contribution to the
group; and "Broken Squares," which promotes
teamwork and encourages group members to

I:TI~&D. Vol 46. No. 3

actively help one another Sharan and Sharan

(1992) suggest that teachers promote and
encourage student interaction prior to beginning
cooperative group work through skill-building
exercises that emphasize appropriate discussion
behaviors. These exercises, which center around
discussions of printed materials such as short
stories, encourage students to participate in discussions and reach group consensus regarding
an issue only after receiving input from all
group members.
In addition to providing training on collaborative skills prior to beginning the instructional
activity, theorists have also offered suggestions
for encouraging and supporting collaborative
skills during cooperative learning. Johnson and
Johnson (1990, 1991) suggested awarding bonus
points to either individual students or cooperative groups to recognize when students practice
collaborative skills. These points could be used
for either academic credit or other rewards.
Another effective strategy is to have students
reflect on their use of collaborative skills after
the completion of a group activity in order for
them to assess their performance and improve
their 6se of these skills (Aronson & Patnoe, 1997;
Sharan & Sharan, 1992). This assessment can be
conducted as part of a culminating group activity or through individual use of a "collaborative
skills checklist" (see Aronson & Patnoe, 1997, p.
40, for a sample checklist). Finally, prompts and
cues could be embedded into an instructional
activity to remind students of their roles in the
group and the collaborative skills they should be
using while completing the activity (Kagan,
1985; Sherman & Klein, 1995; Slavin, 1995).

Benefits and Criticismsof Cooperatlve

Learning
A number of studies have dealt with the effects
of cooperative learning groups on the academic
achievement of the group participants. Many of
these studies compared the achievement of students participating in cooperative learning with
students learning individually. Slavin (1983,
1987, 1995) has examined over 100 studies in
which cooperative learning groups were compared with individual instruction and found
that a vast majority (nearly 75%) reported a sig-

COOPERATIVELEARNINGILSDESIGN

nificant increase in achievement levels for students participating in cooperative learning

groups. Siavin further noted that the studies that
did not show learning gains for students in
cooperative learning groups did not incorporate
one or more of the important aspects of cooperative learning discussed previously; that is, individual accountability, positive interdependence,
and group training.
In addition to improving academic achievement, research has found that cooperative learning also has an impact on numerous other
factors directly related to academic achievement. Cooperative learning activities have been
shown to produce increased time on task
(Cohen & Benton, 1988), increased motivation
for learning activities (Garibaldi, 1979; Nastasi &
Clements, 1991), increased school attendance
(Janke, 1977), and improved self-esteem (Johnson & Johnson, 1983; Slavin, 1983).
The positive effects of cooperative learning
are not unanimously accepted, however.
Tateyama-Sniezek (1990) reviewed 12 studies
comparing cooperative learning to individual
instruction. Based on the results of these studies,
she concluded that enthusiasm for cooperative
learning may not be warranted, particularly
with students with special needs, because a
number of the studies did not produce significant positive results. Of the 12 studies, she found
only 6 that reported significant results favoring
cooperative learning. This led her to conclude
that " . . . the opportunity for students to study
together does not guarantee gains in academic
achievement" (p. 436), mainly because it was
difficult for teachers to consistently integrate
methods to ensure positive interdependence
and individual accountability within the cooperative learning activities.
USING COOPERATIVE LEARNING IN
CONJUNCTION WITH ILS

While much of the research regarding cooperative learning deals with its effectiveness in a
classroom setting, with students completing
such activities as studying for a test, delivering a
group presentation, or completing a research
paper, there is a growing research base focusing
on the benefits of using cooperative learning
strategies to help students complete computer-

11

based instructional activities (I-looper, 1992;

Hooper & Hannafin, 1991; Hooper, Temiyakarn,
& Williams, 1993; Mevarech, Stern, & Levita,
1987; Neuwirth & Wojahn, 1996). However, the
research specifically investigating the effects of
cooperative learning with advanced computerbased instruction such as ILSs is limited and
does not provide a great deal of insight into the
methods with which cooperative learning strategies can be effectively integrated into ILS activities. For example, Beyer (1993) conducted an
evaluation of a three-year project to use ILSs for
math and reading instruction in two Pennsylvania middle schools, and to integrate ILS instruction with computer- and classroom-based
cooperative learning activities. The evaluation
determined that the ILS activities did not have a
significant effect on student achievement in
reading or math and that neither school effectively used cooperative learning activities to
enhance the ILS instruction. The latter factor was
because of the logistical difficulties teachers had
with implementing cooperative learning in the
computer lab and the lack of knowledge and
strategies for combining cooperative learning
and |I_S-based instruction.
Other research specifically examining the
effect of combining cooperative learning with
ILS instruction suggests that the integration of
cooperative [earning with ILSs has both academic and social benefits for students. Mevarech
(1994) found that students completing ILS activities in dyads academically outperformed students
completing
the
same
activities
individually. These differences were evident
both on activities covering basic mathematics
skills and on activities promoting higher cognitive processes. One drawback of this study is
that the methods for integrating the key components of cooperative learning (i.e., positive interdependence, individual accountability, and
collaborative skills) were not discussed; thus it is
difficult to cull guidelines or procedures for promoting cooperative learning in an ILS setting
from this methodology.
A second study that examined the academic
and social impact of ILS instruction delivered to
students in cooperative pairs was conducted by
Brush (1997). In this study, students completed
ILS activities either individually or in dyads.

12

ETR&D. Vol 46. No, 3

Positive interdependence was promoted

through shared resources (the computer) and
group goals; individual accountability was
established with unit tests given periodically to
all students; and training on collaborative skills
was provided to all students assigned to the
cooperative treatment before they began working on the ILS instruction. Results showed not
only academic gains for the cooperative treatment but also significant differences in attitudes
toward both the ILS activities and the content
area in general. Students working in dyads
reacted favorably toward the ILS instruction,
had positive attitudes toward math, and
believed that the computer-based activities were
helping them perform better on classroombased mathematics assignments. Students working individually did not enjoy the ILS activities,
had extremely negative attitudes toward math,
and did not see a relationship between the ILS
instruction and their work in the classroom.
Based on the limited research above, delivering ILS instruction to students in cooperative
groups may be a viable instructional strategy
that can have positive effects on students' academic and social development. However,
implementing this instructional strategy is not
an easy task. ILSs do not incorporate any inherent cooperative learning components into their
design. In fact, the design and implementation
of ILSs more likely promotes student isolation

and competition (Becker, 1992b, 1994; Brush,

1997; Hativa, 1994). Thus, it is unlikely that simply allowing students to complete ILS activities
(as they are currently designed) in groups
would have any academic or social benefits.
Strategies and techniques that promote cooperative learning need to be embedded into the
design and delivery of ILSs in order for schools
to be able to use these systems effectively for
other than long-term individualized instruction.

INTEGRATINGCOOPERATIVE
LEARNING INTO ILS DESIGN

Because ILSs have traditionally been designed to

promote individualized instruction, there are no
embedded design constructs within ILSs pertaining to cooperative learning. The following
section provides some guidelines for designing
ILSs that promote positive interdependence,
individual accountability, and collaborative
skills. Table 1 presents a summary of these
guidelines.

Positive Interdependence

Positive interdependence means that each member of the group believes that the success or failure of the group depends upon individual
success (Johnson & Johnson, 1991). Designing
ILS software to facilitate this interdependence

Table I [] Methods for promoting cooperative learning within ILS courseware a n d

m a n a g e m e n t system.
CL Component

ILS Courseware Addition

ILS Management Addition

Positive
Interdependence

Embed roles/tasksfor
students.

Options for group and

individual tracking and
reporting.
Prompt teacher when g r o u p s
have met criteria for rewards.

Individual
Accountability

Design unit tests to be

completed individually.
Include periodic individual
assessment items within
lessons.
On-line cooperative learning
tutorial activities.
Embed prompts to reinforce
use of collaborative skills.

Options for group and

individual tracking and
reporting.

Collaborative
Skills

Options for suspending or

delaying feedback to allow
for group discussion.

13

COOPERATIVELEARNINGILSDESIGN

can be accomplished in a variety of ways.

Within the ILS courseware itself, group members can be given specific roles and tasks, thus
establishing positive role and resource interdependence (Aronson & Patnoe, 1997). "Cooperative" computer software such as The Great Ocean
Rescue from Tom Snyder Productions (1996) has
already been designed to promote positive role
interdependence through the completion of
unique off-line preinstructional activities by
each group member. With this software, students are assigned roles (and tasks associated
with those roles) as part of an environmental
clean-up team. Each team member is responsible
for specific information, and the team must
decide which information is needed to develop a
solution to the environmental problem presented in the software.
In a second example, Sherman and Klein
(1995) designed a simple CAI activity in which
the computer assigned roles to group members
and consistently prompted and reminded them
of those roles throughout the activity. They
found that cooperative groups using CAI with
these embedded interdependence cues performed significantly better on an individual
posttest and demonstrated more helping behaviors (e.g., giving help and encouragement) than
did groups who completed a similar CAI activity without the cues. Similar strategies for promoting positive interdependence within CAI are
discussed by Cardelle-Elawar and Wetzel
(1995), Holden, Holcolmb, and Wedman (1992),
Neal (1994), Rysavy and Sales (1991), and
Siowck-Lee (1994).
Positive interdependence could be integrated
into ILS courseware in a similar fashion to the
methods outlined for CAI activities. For example, at the beginning of an instructional activity,
the ILS courseware could designate one group
member to be responsible for reading and summarizing information and a second group member to record important information and review
that information with the group. Throughout
the computer-based activity, the courseware
could remind group members of their responsibilities to the group, thus reinforcing the interdependence among the group members. After an
activity was completed, the ILS could select different roles for the group members, thus provid-

ing each student with the opportunity to perform a unique and necessary function within the
group for every activity.
Redesign of the ILS management system can
also facilitate positive interdependence. Currently, there are limited means for grouping students in the management system for reporting
purposes (Becker, 1992b; Hativa & Becker, 1994).
Providing options within the management system that would allow the teacher to designate
student groupings in the class database and
receive
computer-generated
performance
reports for each within-class group would aid
the establishment of positive goal interdependence (Johnson & Johnson, 1991; Slavin, 1995).
The teacher could easily use the information
from these reports to determine grades for each
group based on performance on the courseware
activities. In addition, the management system
could provide teachers with the capability to
assign unique names or icons to the groups, thus
assisting with the establishment of identity
interdependence (Johnson & Johnson, 1991).
The management system could also be
designed either to distribute group rewards
(such as achievement certificates, bonus points,
or class recognition) based on criteria input by
the teacher or to report electronically when a
group has met the criteria for a reward so that it
could be personally delivered by the teacher
(Litchfield, 1993; Rysavy & Sales, 1991; Yeuh &
Alessi, 1988). With either method, the ILS would
handle some of the management tasks necessary
for successful cooperative learning activities,
thus freeing the teacher to concentrate on assisting the groups with the material presented in the
courseware.

Individual A c c o u n t a b i l i t y

One of the most effective methods for ensuring

individual accountability is to require each student in a cooperative group to complete a test of
the material presented in the lesson (Slavin,
1995). Within ILS courseware, this could be
accomplished in several ways. Short quizzes
could be embedded within the courseware, with
the requirement that each student in the group
complete the quizzes individually (Hooper,
1992). The difficulty with this method is that

14
groups would need to disperse during the ILS
activity in order to move to separate computers
to complete the quizzes, then reconvene to complete the lesson. This could prove difficult from
a classroom management perspective. An alternative method would be to have students complete individual assessment activities at logical
points in the ILS courseware. For example, since
the traditional design of ILS courseware breaks
the instruction into units, students could be
required to complete on-line unit tests individually. With this method, groups would disperse
to separate computers at logical points within
the instruction, complete the tests individually,
then reconvene in their groups to begin a new
unit of instruction. The ILS management system
would have the ability to track both group progress on activities within the units and individual
scores on unit tests. Ideally, the management
system would allow the teacher to control how
the individual test scores were used in the overall assessment of the group. The teacher could be
provided with options for having the unit tests
act as individual measures of achievement
(Siowck-Lee, 1994; Yeuh & Alessi, 1988), averaging the scores together and using the average
score as a group assessment (Slavin, 1995), or
randomly selecting one score and using it as an
assessment of the entire group (Aronson &
Patnoe, 1997; Johnson & Johnson, 1991).
Another method for individual accountability that could be incorporated into ILS
courseware would involve having the
courseware periodically prompt individual students to respond to questions or short activities
embedded in the lesson (Dockterman, 1995;
Johnson & Johnson, 1991; Lookatch, 1996). Other
members of the group would be told to allow
that individual to complete the activity without
assistance. The ILS management system would
have the ability to track individual responses
within the courseware and determine the next
individual to be prompted (Lookatch, 1996).

Teaching and Reinforcing

CoUaboratlve Skills
Design components could be added to ILSs that
would introduce appropriate collaborative skills
to group members prior to engaging in the

ETF~&D. Vo146. NO 3

instructional activities, remind groups to use

those skills once groups were engaged in the
activities, and provide opportunities for group
interaction within the activities. Several
researchers have suggested that students complete on-line tutorials designed to introduce collaborative skills and stimulate students to use
those skills once they begin the instructional
activities (Hooper, 1992; Rysavy & Sales, 1991;
Siowck-Lee, 1994). The ILS could include an online preinstructional activity that discusses the
purpose and benefits of cooperative learning,
provides guidelines for working effectively in
groups, and gives group members opportunities
to practice collaborative skills prior to beginning
the instruction. A practice activity such as "The
NASA Exercise" (Aronson & Patnoe, 1997, p.
129), in which students work together to
develop a strategy for sending a mission to
Mars, could easily be converted to a computerbased activity; this design not only would reinforce group processing and collaboration skills,
but also would provide groups with the chance
to become comfortable working cooperatively
on the computer.
There are also numerous methods for reinforcing collaborative skills once groups are
engaged in ILS courseware. Hooper (1992) and
Rysavy and Sales (1991) suggested embedding
prompts within the courseware reminding
group members to practice appropriate collaborative skills. They also suggested providing time
between computer-based activities for students
to reflect on the effectiveness of their group and
discuss methods for improving collaboration.
Sherman and Klein (1995) embedded verbal
interaction cues within their CAI activities in
order to foster group interaction. These cues
included prompts to ask questions about information that wasn't understood and reminders to
review and summarize information. These
embedded cues and reminders could easily be
incorporated into the design of ILS courseware.
Finally, ILS courseware should provide
opportunities for group discussion while comple0_ng activities. This approach may involve
eliminating the immediate feedback provided
by the ILS management system in order to allow
time after a response has been made for further
discussion among the group (Hooper, 1992;

COOPERATIVELEARNINGILSDESIGN

lS
placement test i n d i v i d u a l l y . Based on the results
of the placement test the teacher (ideally, w i t h
assistance from the ILS management system)
w o u l d determine student groups and designate
an initial curricular placement for each group.
After group assignments had been made, students w o u l d complete on-line collaborative
training activities that w o u l d help prepare them
to w o r k cooperatively in the courseware.

Litchfield, 1993). For example, one student could

enter a response to a problem, then the ILS management system could prompt the group to
reconfirm that the response was agreed upon by
all members of the group. Once the group has
been given an opportunity to double-check the
response and make sure that everyone in the
group concurred, the management system could
determine the accuracy of the response and provide appropriate feedback.

In the ILS courseware itself,additions would

need to be made to designate and rotatestudent
roles/responsibilitieswithin the groups and to
provide prompts and cues to reinforce the
assigned roles,to question individual students
within the groups, and to remind groups to collaborateeffectively.These additions would help
establishpositiveinterdependence and individual accountability, thus providing consistent
integration of these components within the
instruction. For the most part, the activities
within the units would be completed entirely in
cooperative groups, with the exception of the
tests administered at the end of each unit. For

TOWARD A COOPERATIVE" ILS DESIGN

If the strategies for integrating cooperative

learning with an ILS were incorporated into a
new ILS design, what might that design look
like? A major addition to this new design would
involve integration of several on-line preinstructional activities to be completed both
individually and in groups. In a typical ILS, the
only on-line preinstructional activity completed
by students is the placement test. In a cooperative ILS, students would still need to complete a

Figure 2 [ ] Integrcrted ILS/CL design sequence.

Courxwam

Seque, nce
-,'-

Unit I

Unit 3

Unit2

..~ ": .
Lesson I

i ~

Lesson1

':

Collaborative
Training

| lk~,~,: !1

Lesson 2

Ro~es
Lesson 1

. !... ....

Lesson 2

,.

o ,

*.,,?~

""'i'~;

:
~ ?

~'

Teat

o o o o o

~,
Lesson2
~;........
..

'

AcSvW

Activities

A,a,~
[]

Teac~erl
Management
System

16

ETI~&D.Vol 46. No. 3

these activities, the courseware would need to

be designed so that it prompted students to
move to separate computers, complete the tests
individually, then reconvene in their groups and
continue with the next unit. Figure 2 displays an
ILS design integrating cooperative learning.
SUMMARY AND SUGGESTIONS
FOR FURTHERRESEARCH
This paper has both provided a rationale for

combining cooperative learning strategies with

ILS instruction and outlined some design guidelines for embedding components of cooperative
learning into ILS courseware and management
system functions. It is hoped that this paper will
assist developers in considering designs that
advance the "individualized instruction" model
of most ILSs and promote more opportunities
for social interaction and learning within these
systems.
Based on the ideas presented in this paper,
there are several lines of research that might be
pursued. While there have been a few studies
examining the academic and social impact of
delivering ILS instruction to students in cooperative learning groups (Brush, 1997; Mevarech,
1994), there needs to be a continuation of this
research in order to determine which cooperative learning models are most effective when
used with ILSs and whether various strategies
for combining cooperative learning and ILS
instruction (whether those strategies are embedded within the on-line activities or are supplementary to those activities) are more or less
effective.
Research also needs to examine various
grouping strategies and structures to determine
which are most effective for ILS instruction. Currently, there are a wide variety of opinions
regarding the optimal group size for cooperative
learning activities, particularly when those
activities are computer-based (e.g., Hooper,
1992; Rysavy & Sales, 1991; Yager, Johnson, &
Johnson, 1985). In addition, researchers continue
to debate which grouping structures (i.e., homogeneous versus heterogeneous) are most effective with students completing computer-based
activities cooperatively (Brush, 1997b; Slavin,
1995; Simsek & Hooper, 1992; Yager, Johnson, &
Johnson, 1985). Continuing this line of research

will assist designers both with (a) providing

guidelines for teachers wishing to integrate
cooperative learning with ILS instruction and (b)
developing features within the ILS management
system to assist in defining student groups. []

Thomas A. Brush is Assistant Professorof

EducationalTechnology at Arizona StateUniversity
where he can be reachedat Educational Media and
Computers, Arizona StateU., Box 870111,Tempe,AZ

85287-0111.
REFERENCES

Alfrangis, C.M. (1989). A Critical Analysis of the Components of an Integrated Learning System and a Measure of
the System's Effect on Mathematics and Reading. Doctoral dissertation, George Mason University.
Aronson, E., & Patnoe, S. (1997). The Jigsaw Classroom.
New York: Addison Wesley Longman.
Bailey, G.D. (1993). Wanted: A road map for understanding Integrated Learning Systems. In G.D. Bailey (Ed.), Computer-based Integrated Learning Systems.
Englewood Cliffs, NJ: Educational Technology Publications.
Becker, H.J. (1992). Computer-based Integrated-learning systems in the elementary and middle grades: A
critical review and synthesis of evaluation reports.
Journal of Educational Computer Research, 8, 1--41.
Becker, H.J. (1992b). A model for improving the performance of integrated learning systems: Mixed individualized / group / whole class lessons, cooperative
learning, and organizing time for teacher-led remediation of small groups. Educational Technology,
32(9), 6--15.
Becker, H.J. (1994). Mindless or mindful use of integrated learning systems. International Journal of Educational Research, 21(1), 65-79.
Becket, H.J., & Hativa, N. (1994). History, theory, and
research concerning integrated learning systems.
International Journal of Educational Research, 21(1), 5-12.
Bender, P.V. (1991). The effectiveness of integrated
computer learning systems in the elementary school.
Contemporary Education, 63,19-23.
Beyer, F.S. (1993). The CAI/cooperative learning project: Third year evaluation report. ERIC Document
Reproduction Service # ED374791.
Brush, T.A. (1997). The effects on student achievement
and attitudes when using Integrated Learning Systems with cooperative pairs. Educational Technology
Researchand Development, 45(1), 51--64.
Brush, T.A. (1997b). The effects of group composition
on achievement and t'iTne-on-taskfor students completing ILS activities in cooperative pairs. Journal of
Research on Computing in Education, 30(1), 2-17.
Brush, T., & Bannon, S. (1998, April). The integration of
technology into K-12 classrooms: A five-state eompari-

COOPERATIVE LEARNINGILSDESIGN

son. Paper presented at the annual meeting of the

American Educational Research Association, San
Diego, CA.
Cardelle-Elawar, M., & Wetzel, K. (1995). Students and
computers as partners in developing students' problem-solvingskills. Journal of Researchon Computing in
Education, 27(4), 387--401.
Clariana, R.B. (1994). The effects of an Integrated
Learning System on student performance in mathematics and reading in one third-grade classroom.
Journal of Computer-Based Instruction, 21(1), 12-15.
Clariana, R.B. (1996). Differential achievement gains
for mathematics computation, concepts, and applications with an Integrated Learning System. Journal

of Computers in Mathematics and Science Teaching, 15,

203-215.
Cohen, E.G., & Benton, J. (1988). Making groupwork
work. American Educator, 12(3), 10-17, 45--46.
Deutsch, M. (1962). Cooperation and trust: Some theoretical notes. In M. Jones (Ed.), Nebraska symposium
on motivation. Lincoln, NE: University of Nebraska
Press.
Dockterman, D. (1995). Interactive learning: It's pushing the right buttons. Educational Leadership, 53(2),
58--59.
Garibaldi, A. (1979). Affective contributionsof cooperative and group goal structures. Journal of Educa-

tional Psychology, 71,788-794.

Hativa, N. (1994). What you design is not what you get
(WYDINWYG): Cognitive, affective, and social
impacts of learning with ILS - An integration of findings from six-years of qualitative and quantitative
studies. International Journal of Educational Research,
21(1),81-111.
Hativa, N., & Becker, H.J. (1994). Integrated learning
systems: Problems and potential benefits. International Journal of Educational Research, 21(1), 113-119.
Hativa, N., Swissa, S., & Lesgold, A. (1992). Competition in individualized CAI. Instructional Science, 21,
393-428.
Hertz-Lazarowitz, R., & Schachar, H. (1990). Teachers'
verbal behavior in cooperative and whole-class
instruction. In S. Sharan (Ed.), Cooperative Learning:
Theory and Research. New York: Praeger Publishers.
Holden, M.C., Holcolmb, C.M., & Wedman, J.E. (1992).
Designing hypercard stacks for cooperative learning. The Computing Teacher, 19(5), 20-22.
Hooper, S. (1992).Cooperative learning and computerbased instruction. Educational Technology Research
and Development, 40(3), 21-38.
Hooper, S,, & Hannafin, M. (1991).The effects of group
composition on achievement, interaction, and learning efficiency during computer-based cooperative
instruction. Educational Technology Research and
Development, 39(3), 27--40.
Hooper, S., Temiyakarn, C., & Williams, M.D. (1993).
The effects of cooperative learning and learner control on high- and average-ability students. Educational Technology Research and Development, 4(2),
5-18.

17
Janke, R. (1977, April). The Teams-Games-Tournament

(TGT) method and the behavioral adjustment and academic achievement of emotionally impaired adolescents.
Paper presented at the annual meeting of the American Educational Research Association, Toronto,
Canada.
Johnson, D.W., & Johnson, R.T. (1983). The socialization and achievement crisis: Are cooperative learning experiences the solution? In L. Bickman (Ed.).
Applied Social Psychology Annual. Beverly Hills, CA:
Sage.
Johnson, D.W., & Johnson, R.T. (1987). Learning

Together and Alone: Cooperative, Competitive, and Individualistic Learning. Englewood Cliffs, NJ: PrenticeHall.
Johnson, D.W., & Johnson, R.T. (1990). Cooperative
learning and achievement. In S. Sharan (Ed.), Cooperative Learning: Theory and Research. New York: Praeger Publishers.
Johnson, D.W., & Johnson, R.T. (1991). Learning
Together and Alone. Englewood Cliffs, NJ: Prentice
Hall.
Kagan, S. (1985). Learning to cooperate. In R. Slavin, S.
Sharan, S. Kagan, R. Lazarowitz, C. Webb, & R.
Schmuck (Eds.), Learning to Cooperate, Cooperating to
Learn. New York: Plenum Press.
Kerr, N.L. (1983). Motivation losses in small groups: A
social dilemma analysis. Journal of Personality and
Social Psychology, 45, 819-828.
Kerr, N.L., & Bruun, S.E. (1983). The dispensability of
member effort and group motivation losses: Freerider effects. Journal of Personality and Social Psychology, 44, 78-94.
Lepper, M.R. (1985). Microcomputers in education:
Motivational and social issues. American Psychologist, 40, 1-18.
Litchfield, B.C. (1993, April). Design factors in multime-

dia environments: Researchfindings and implications for

instructional design. Paper presented at the annual
meeting of the American Educational Research
Association, Atlanta, GA.
Lloyd, J.W., Crowley, E.P., Kohier, F.W., & Strain, P.S.
(1988). Redefining the applied research agenda:
Cooperative learning, prereferral, teacher consultation, and peer-mediated interventions. Journal of
Learning Disabilities, 21, 43-52.
Lookatch, R.P. (1996). Collaborative learning and multimedia: Are two heads still better than one? TechTrends, 41(4), 27-31.
Maddux, C.D., & Willis, J.W. (1992). Integrated learning systems and their alternatives: Problems and
cautions. Educational Technology, 32(9), 51-57.
Mesch, D., Lew, M., Johnson, D.W., & Johnson, R.T.
(1986). Isolated teenagers, cooperative learning, and
the training of social skills. The Journal of Psychology,
120, 323-334.
Mevarech, Z.R. (1994). The effectiveness of individualized versus cooperative computer-based integrated
learning systems. International Journal of Educational
Research, 21 (1), 39-52.

18
Mevarech, Z.R., Stern, D., & Levita, I. (1987). To cooperate or not to cooperate in CAI: That is the question.
Journal of Educational Research, 80, 164-167.
Mills, 5.C. (1994). Integrated learning systems: New
technology for classrooms of the future. TechTrends,
39(1), 27-28, 31.
Nastasi, B.K., & Clements, D.H. (1991). Research on
cooperative learning: Implications for practice.
School Psychology Review, 20, 110-131.
Neal, J.5. (1994). The interpersonal computer. Science
Scope, 17(4), 24-27.
Neuwirth, C.M., & Wojahn, P.G. (1996). Learning to
write: Computer support for a cooperative process.
In T. Ko~damann (Ed.). CSCL: Theory and Practice of
an Emerging Paradigm. Mahwah, NJ: Lawrence
Erlbaum Associates.
Osin, L., Nesher, P., & Ram, I. (1994). Do the rich
become richer and the poor poorer: A longitudinal
analysis of pupil achievement and progress in elementary schools using computer-assisted instruction. International Journal of Educational Research,
2l(1), 53--64.
Robin.~on"5. (1991). Integrated learning systems: From
teacher-proof to teacher empowering. Contemporary
Education, 63, 15-18.
Roblyer, M.D., Edwards, J., & Havriluk, M.A. (1997).
Integrating educational technology into teaching. Upper
Saddle River, NJ: Merrill.
Rysavy, S., & Sales, G.C. (1991). Cooperative learning
in computer-based instruction. Educational Technology Researchand Development, 39(2), 70-79.
Sharan, Y., & 5haran, 5. (1992). Expanding cooperative
learning through group investigation. New York:
Columbia University.
Sherman, G.P., & Klein, J.D. (1995). The effects of cued
interaction and ability grouping during cooperative
computer-besed science instruction. Educational
Technology Researchand Development, 43(4), 5-24.
Sherry, M. (1990). Implementing an integrated instructional system: Critical issues. Phi Delta Kappon, 72,
118-120.
5hockley, H.A. (1992). Turnkey or turnkey.> Integrating an integrated learning system. Educational Technology, 32(9), 22-25.
Simsek, A., & Hooper, 5. (1992). The effects of cooperative versus individual videodisc learning on student Performance and attitudes. International Journal
of Instructional Media, 19, 209-218.
Siowck-Lee, G. (1994, June). Developing and using

coursewarefvr cooperative learning activities in the classroom. Paper presented at the Asia Pacific Information Technology in Teaching and Education
Conference, Brisbane, Australia.
Slavin, R.E. (1983). Cooperative Learning. New York:
Longman.
51avin, R.E. (1985). Team-assisted individualization:
Combining cooperative learning and individualized
instruction in mathematics. In R. 51avin, 5. Sharan, 5.
Kagan, R. Lazarowitz, C. Webb, & R. Schmuck

ETI~&D.Vo146, No. 3

(Eds), Learning to Cooperate, Cooperating to Learn.

New York: Plenum Press.
Slavin, R.E. (1986). Using Student Team Learning (3rd
ed.). Baltimore, MD: Johns Hopkins University.
Slavin, R.E. (1987). Cooperative learning and the cooperative school. Educational Leadership, 45(3), 7-13.
Slavin, R.E. (1990). Learning together. The American
School BoardJournal, 177(8), 22-23.
Slavin, R.E. (1991). Are cooperative learning and
"untracking" harmful to the gifted.> A response to
Allan. Educational Leadership, 48(6), 68-71.
Slavin, R.E. (1993, April). Cooperative learning and
achievement: An empirically-based theory. Paper presented at the annual meeting of the American Educational Research Association, Atlanta, GA.
Slavin, R.E. (1995). Cooperative Learning: Theory,
Research, and Practice. Boston: Allyn and Bacon.
Slavin, R.E., Leavey, M.B., & Madden, N.A. (1986).
Team Accelerated Instruction: Mathematics. Watertown, MA: Charlesbridge.
Tateyama-Sniezek, K.M. (1990). Cooperative learning:
Does it improve the academic achievement of students with handicaps? Exceptional Children, 56, 426437.
Tom 5nyder Productions. (1996). The Great Ocean Rescue. [Computer Programl. Watertown, MA: Author.
Van Dusen, L.M., & Worthen, B.R. (1994). The impact
of integrated learning system implementation on
student outcomes: Implications for research and
evaluation. International Journal of Educational
Research, 21 (1), 13-37.
West, R.C., & Marcotte, D.R. (1993-94). The effects of
an integrated learning system (ILS) using incremental time allotments on ninth grade algebra achievement. Journal of Educational Technology Systems, 22,
283-294.
White, M.A. (1992). Are ILSs good education? Educational Technology, 32(9), 49-50.
Wiburg, K. (1995). Integrated learning systems: What
does the research say? The Computing Teacher, 22(5),
7-10.
Worthen, B.R., Van Dusen, L.M., & Sailor, P.J. (1994). A
comparative study of the impact of integrated learning systems on students' time-on-task. International
Journal of Educational Research, 2 I(1), 25-37.
Yager, 5., Johnson, R.T., & Johnson, D.W. (1985). Oral
discussion, group-to-individual transfer, and
achievement in cooperative learning groups. Journal

of Educational Psychology, 77, 60-.66.

Yager, S., Johnson, R.T., Johnson, D.W., & 5hider, B.
(1985). The effect of cooperative and individualistic
learning experiences on positive and negative crosshandicap relationships. Contemporary Educational

Psychology, 10,127-138.
Yeuh, J., & Alessi, 5.M. (1988). The effects of reward
structure and group ability composition on cooperative computer-besed instruction. Journal of Computer

Based Instruction, 15, 18-22.