13

Instructional Science 11 (1982) 13-28

Elsevier Scientific Publishing Company, Amsterdam - Printed in The Netherlands

MINIMIZING THE BLACK BOX PROBLEM TO ENHANCE THE

VALIDITY OF THEORIES ABOUT INSTRUCTIONAL EFFECTS

PHILIP H. WINNE
Instructional Psychology Research Group, Simon Fraser University,
Burnaby, B. C., Canada

ABSTRACT

Prior research on learning from instruction has used a three-stage model of effects.
The model posits that (1) an instructional stimulus cues the learner to use particular
cognitive processes to operate on content, (2) these operations are executed, and (3)
the result facilitates learning. While theories focus on all three stages, experiments have
provided only indirect evidence about how well each is accomplished and rarely have
controlled directly all three key sources of variance. An illustrative analysis of learning
from instruction is presented, and a procedure is proposed that probabilistically increases
control over the theoretically significant cognitive causes of instructional effects.

Introduction

Research on teaching has come a long way in the 80 years since Joseph
Rice (1897) surprised educators with the finding that achievement in spelling
did not vary proportionally with time spent on drill. Today, there is a small
collection of empirically determined relationships between teacher behaviors
and student achievement, called "teacher should" statements by Gage (Gage,
1980). In recent experiments, teachers' applications of these principles have
yielded the gains predicted for students' learning (Anderson et al., 1979;
Good and Grouws, 1977). These advances offer proof that research on teach-
ing has valuable contributions to make to educational practice (Travis, 1979).
The contemporary empirical literature about teacher-student interac-
tions and their relationships to student achievement, nonetheless, is still in a
developmental stage with respect to both practical potency and theoretical
sophistication. For the few partially proven "teacher should" prescriptions
about instruction, there are very many more unvalidated "maybe teacher

0020-4277/82/0000-0000/$02.75 © 1982 Elsevier Scientific Publishing Company

14

might t r y " statements. Scanning almost any current text about teaching
methods (Cooper et al., 1977; Hoover, 1976) will expose a plethora of these
latter kinds of statements. Though one might quibble whether these sugges-
tions are better ignored or should be cautiously adopted within a framework
o f individual experimentation, there is much consensus that an essential
next step for the field's advance is to generate and test theories of teaching.
As Dunkin and Biddle (1974) concluded after their comprehensive review of
the research on teaching: "Until adequate, empirically based theories are
developed, this field will continue to exhibit a complex and somewhat
chaotic visage" (p. 425).
Developing empirically based theories of teaching and its effects is a
complex and demanding enterprise. In research on teaching, and in research
on instruction generally, there are a set of relatively c o m m o n assumptions
about the way that instructional events influence learning. Also, almost all
the research done within this set of assumptions has adopted a model for
explaining the effects observed in research that will be shown in this paper
to be importantly deficient. To analyze the nature of this deficiency in expla-
nations about how instruction influences learning, the paper focuses on one
particular example Of a relatively well studied area of research. Although this
selection is necessary to limit the length of the analysis, it should be viewed
as merely a representative instance of the set of variables that researchers
examine as potential causes of learning in instructional settings. The argu-
ments made are believed to generalize to the host of other variables that
serve as independent variables in research on instruction. The first step in
this analysis, however, is to characterize the nature of instruction and the
assumptions that underlie explanations about how instruction affects learning.

Explaining Instructional Effects with Black Boxes

Most contemporary theories of learning view the learner as a naturally

cognitively active and strategically purposeful processor of information. For
instance, social learning theory posits that a learner engages in an activity
after using features of the current environment as cues for searching memory.
This search of m e m o r y yields predictions about the consequences contingent
on different responses and estimates of the likelihood that each different
response can be effected in that environment. Bandura ( 1 9 7 7 ) l a b e l e d these
o u t c o m e expectations and efficacy expectations, respectively. If one allows
for differences in terms, the possibility that learners branch and recycle
through series of Cognitions, and that the contents of cognitions vary accord-
ing to the task posed for the learner, similar theoretical descriptions arise in
theories covering tasks from classical conditioning (Rescorla, 1972) to
dynamic problem solving (Greeno, 1978).
 15

If one accepts that this assumption o f inherent cognitive activity and

strategic purpose accurately characterizes learners, a question follows logically
about how learners learn from instruction: what relations exist between
these naturally occurring cognitive processes and the instructional events
that an instructor creates to approach the goal of acquiring predefined con-
tent (i.e. the curriculum)? This question implies that the learner is con-
fronted with at least two distinctly different kinds o f stimuli in an instruc-
tional setting. One set of stimuli is the content to be learned - the law of
inverse squares, a definition of peninsula, or understanding the political
pressures on the nuclear power industry. What distinguishes the instructional
setting from a non-instructional one such as play or exploration is that the
instructor purposefully uses stimuli other than content that are intended
to control the learners' natural cognitive processing of the content to be
learned. For instance, in experiments on the effects of presenting learners
with adjunct questions, the sentences comprising the question itself are not
what will be on an achievement test. Rather, the instructor (or text writer)
intends that the question will cue the learner to perform one or several
cognitive activities that operate on the content named in the question and
available elsewhere in the text.
Presumably, these cognitive processes would n o t otherwise have been
applied if the question had not been provided. The adjunct question is an
instructional stimulus. It is intended to guide the learner in the process of
learning whatever content is named in the question. Any stimulus provided
by an instructor that is intended to direct the learner's cognitive processing
o f content in a particular way is defined here to be an instructional stimulus.
Instructional stimuli are a means to an end. They cue or otherwise encourage
learners to cognitively operate on some information in a way that they
would not under natural, i.e. non-instructional, conditions. Learning the
instructional stimulus itself is not the goal of instruction. But learners' use of
the instructional stimulus as the instructor intends is one necessary precursor
to effective instruction.
Thus, what learners mentally do to content as a function of having
been exposed to an instructional stimulus is the central concern of theories
o f instruction. What will be argued here is that theoretical advancement in
instructional psychology has been impeded because of a failure to demon-
strate the correspondence between the cognitive processing of content that
learners do, in fact, engage and that which is hypothesized to follow upon
exposure to a particular instructional stimulus. In other words, causal prop-
ositions in instructional theories are black boxes. While the learner's cog-
nitive processes during instruction have been described post hoc by referring
to the products they produced, they have been controlled only weakly and
indirectly in experiments by the theorist's experimental design.
A host of significant difficulties arise when black-box processes are not
16

operationalized. To illustrate the nature of these problems, what follows is

an analysis of one relatively well-researched instructional stimulus - the
adjunct question in text - as an instance of the general case. In the course
o f this analysis, it will be demonstrated that researchers have used post hoc
data on learning effects inappropriately in theorizing about the cognitive
processes that produced that observed effect.

Some Contradictory Findings

The adjunct question in text materials has been researched widely

since its popularization by R o t h k o p f nearly 20 years ago. In a compre-
hensive review o f this research, Rickards and Denner ( 1 9 7 8 ) i n f e r r e d
from careful analyses o f the studies that learners engage one or more of
several cognitive processes to operate on content in response to an adjunct
question. For instance, R o t h k o p f and Bisbicos (1967) asked students to
read a text divided into two halves. They found that, when readers were
n o t allowed to review, postquestions about names and measures in the first
half o f the text did not increase readers' achievement of this intentional
content. However, learners' achievement on similar c o n t e n t presented in
the second half of the text, where review also was prohibited and where
there were no adjunct questions on names and measures, was statistically
greater than predicted.
Rickards and Denner (1978) hypothesized that learners invoked a
selective forward-looking processing strategy whereby these categories of
content were isolated and further processed for acquisition during reading
o f the second half of the text. This post hoc hypothesis a b o u t learners'
cognitive response to the instructional stimulus o f adjunct postquestions
where review was prohibited has the features of a black box. The post-
questions, the instructional stimulus of interest, and some surrounding con-
tent are input to the box. These were directly controlled b y the design of the
experiment. The incidental learning that was observed is o u t p u t from the
box. That the learning was incidental is an empirical fact. What actually went
on in the middle, i.e. what was interpreted to be a selective forward-looking
processing strategy, remains an inference. No data describe the processing
directly. The only data available to the experimenter describe the products
o f learners' cognitive processes, not the processes per se. Unless it is b o t h
logical and empirically supported that only the hypothesized cognitive pro-
cesses were at work to produce these products, the key events in a theory of
instruction that tries to explain how learning was affected by those instruc-
tional stimuli remain undemonstrated.
Almost all hypotheses about h o w learners deal with instructional stimuli
are black box hypotheses. The information processing in those boxes is both
 17

uncontrolled and not demonstrated as the learner responds to the instruc-

tional stimuli that the experimenter manipulates. The c o m m o n l y accepted
m e t h o d for strengthening the validity of black box hypotheses is to repeat
the study. Rickards et al. (1976) essentially did this. Unfortunately for the
theoretical explanation, they failed to replicate the earlier finding. In addi-
tion, the theory was complicated further by their hypothesis that a specific
backward processing mechanism produced the effect observed in their exper-
iment.
In many other areas of research on instruction, similar difficulties have
been encountered in pinning down black box interpretations of observed
effects. For example, though some o f the findings from b o t h correlational
and experimental studies of teachers' use of higher cognitive questions have
been favorable, many studies showed no instructional effects, and some
revealed negative effects on learning (Dunkin and Biddle, 1974; Winne,
1979). And, even if one combines findings in ways that compensate for low
statistical p o w e r in the presence of moderate real effect sizes (Gage, 1978),
theorists still use a black b o x to explain how learners use higher cognitive
questions to learn content. To test rigorously theoretical propositions about
instructional effects, the black box processes on which cause-and-effect
propositions rest must be demonstrated rather than inferred post hoc.
Ultimately, the processes that theories ascribe to these boxes must be manip-
ulated in experiments as directly and as exactingly as are content stimuli and
instructional stimuli in current research.

Penetrating Black Boxes in Instructional Research

The assumption about learners' cognitive activity and strategic purpose-

fulness permits two characterizations of what happens when they encounter
an instructional stimulus that is intended b y the instructor to cue the learner
about h o w to operate on content. A simple fabricated experiment in which
the instructional stimulus of interest is adjunct postquestions provides a
setting that illustrates these two characterizations.

THE THEORIST'S VIEW

Upon being given the text and asked to learn it, the learner begins to
read. Because the hypothesis has been framed in terms o f a difference
between group means, and because students have been randomly assigned
to one o f two groups that experience identical external stimuli except for
the experimental group's exposure to the instructional stimuli of interest
(i.e. adjunct postquestions), there are two theoretically relevant sources of
variance to account for in the analysis of scores on a post-reading multiple-
18

choice test: variance due to the group difference in instructional stimuli,

and variance arising from individual differences in cognitive processing of
the text or the text plus instructional stimuli within the groups. There are
still other sources of variance in the test scores such as that arising from
unreliability of the test, b u t the experimenter attempts to dampen these
as much as possible by manipulations of the environment, selection of
participants, and/or statistical adjustments (Cook and Campbell, 1979).
These sources influence the precision of the experimenter's statistical test
b u t they are not accountable to nor objects of the theory.
The statistical model assumed by the theorist adds a constant to each
student's multiple-choice score for those who experience the text with
adjunct postquestions. If the constant is large enough relative to a variance
term that collectively represents individual differences, unreliability in the
test scores, and other unaccounted for and presumed random factors, the
theorist is justified in speculating that students exposed to postquestions
all had their multiple-choice test scores augmented by a constant because
o f exposure to the instructional stimuli. This follows from the constant
e f f e c t assumed in the statistical model. A series of experiments with varying
manipulations of external stimuli would become the basis on which the
theorist would hypothesize whether a specific forward effect or a specific
backward effect was the readers' functional operationalization of the theo-
fist's intent behind the instructional stimuli. Note again that the cognitive
machine that directly produced these effects by its active and strategic
response to the instructional stimuli is an empirical mystery that assumes
the status of a theoretical reality.

THE READERS' VIEWS

The text is a surrogate for the instructor, and all of i~s features are
either content to be learned or signals to the learner to process content in
some particular cognitive way. All b u t one of these signals, the postquestions,
are structurally identical for readers in the two groups. For instance, all
participants in the study have the same opportunity to use paragraph inden-
tation as an instructional stimulus signalling h o w to process concepts to create
a category structure that the text writer believes is optimal for acquiring
content in the text.
Variability in learners' use of paragraph indentation and other stimuli
produces noise or residual variance in the dependent variable. To date, instruc-
tional psychologists have been Content, for the most part, to control sources
o f variance arising from "individual differences" by statistical rather than
operational procedures.
The readers in the experimental group have no difficulty isolating the
instructional stimulus - the adjunct postquestion is printed by itself on a
 19

page following the segment of text wherein the text writer knows content
can be found to answer the question. It seems reasonable to presume without
requiring empirical verification that every student registers the occurrence
o f this instructional stimulus injected into the instructional environment. As
an inherently active and strategic information processor, the reader next
might process the question for its meaning, though this is not necessarily the
case as will be argued momentarily.
Imagine that every reader in the experiment is exceptionally metacog-
nitively conscious so that each constantly monitors the cognitive processing
o f content they're reading. If these readers talked aloud a b o u t h o w they were
grappling with the text and the experimenter's instructional stimulus, their
descriptions might sound like the following.
Since this adjunct question is an instructional stimulus that the text
writer intended to guide the reader a b o u t h o w to process content in the
manner just described, one reader metacognitively may assume that the
stimulus signals: "Review the content that is still in m e m o r y to produce
an answer to the question." This first reader does what the theorist intended.
Another reader may respond: "Scan the content still in m e m o r y to perform
a recognition check about the presence of content that might be used to
answer the question, b u t don't bother to retrieve and m o n i t o r the answer as
such." Yet a third reader may react: "Process the question to set up a
monitoring routine that will flag content related to this topic in the next
section of text." Other readers may perform similar or different cognitive
operations in response to the instructional stimulus. Since neither the text
writer nor the experimenter provided a criterion for readers a b o u t which
processing strategy was intended, readers are left to their own devices regard-
less of other controls exerted by the experimental design that is in force.
Variance in these criteria, and hence the actual processing strategies adopted
b y readers in response to the instructional stimulus, is uncontrolled in the
experiment. Note, however, that these varying strategies are the direct causes
o f whatever effects are observed - t h e y are the theoretical foci in the study
that will be embodied in one inference about what the learners did in response
to the adjunct questions, i.e. a specific forward processing strategy. Note also
that the theorist assumed the immediate products of these strategies were
equivalent across learners in the experimental group to justify belief in the
hypothesis o f a constant effect in the statistical model.
Having isolated and comprehended the instructional stimulus, the reader
n o w m a y have a cognitive decision to make: "Is it worth the effort to carry
o u t the operations on content that I understand are suggested by the post-
question?" In other words, is there sufficient motivation to use the instruc-
tional stimulus? Motivation in this instructional setting is learned and depends
on the reader's reinforcement history and information gained from experience
in previous similar settings. Some readers m a y initiate execution of the cog-
20

nitive operations they perceived were intended by the instructional stimulus,

whatever they were. Others may be motivated to conserve cognitive process-
ing and cascade from complicated processing strategies to simpler ones
(Bruner et al., 1956). For instance, if the answer to the question was not
directly retrievable upon a shallow scan of memory, generating secondary
retrieval cues and initiating a sustained network search of cognitive structure
(Gagn6, 1979) m a y be judged t o o cognitively expensive. A simpler recogni-
tion check that such a probable retrieval path was available m a y be judged
"good enough." On the other hand, rather than engage any cognitive pro-
cessing b e y o n d reading the question, the reader may choose to note: "I
read the question, like all the other sentences, and n o w should move on to
the next segment of text." For a reader w h o knows he is slow, and w h o has
concerns a b o u t finishing the entire text, this might be a reasonable strategy
to insure that he at least is exposed to all the content.
At this stage in a reader's dealing with the adjunct postquestion, let us
assume that the instructional stimulus was isolated, that the reader's inter-
preted operationalization was to answer the question, and that there was a
motivational state to go ahead with the processing. There still remains the
possibility of variability in readers' execution of the cognitive response.
Some readers, failing to retrieve the answer directly, may think: "Well, I
couldn't answer that one, so I'll pay special attention to similar material if
it's encountered later." For quite different reasons than described earlier,
here is the genesis of a specific forward effect. Other readers may invest
heavily in cognitive processing to answer the question if the answer was not
directly and immediately retrievable. These readers m a y generate secondary
retrieval cues and activate surrounding cognitive structure (Gagn6, 1979).
These cognitive events rather than the postquestion itself m a y facilitate the
encoding of later-encountered similar information because of selective activa-
tion of prior knowledge. Hence, this artifact o f the instructional effect is
another possible genesis for interpreting data in terms of a specific forward
effect.
Overall, then, there is a possibility, if n o t a probability, that widely
differing cognitive processes used by readers give rise to an effect that the
theorist can view only as a constant. A constant effect is a necessary element
in validating theoretical propositions that explain what caused the observed
effect. A constant effect, however, is not a sufficient condition upon which
to conclude that what the theory must identify, i.e. the proposed cause in
cause-effect relations, also is a constant. This was illustrated in the analysis
o f the theorists' observation o f a specific forward effect when readers were
exposed to adjunct postquestions under conditions where review o f already
read text was prohibited. What is in d o u b t here is the construct validity of
the c a u s e (Cook and Campbell, 1979) o f this effect. Hence the explanation
o f w h y the effect was observed, i.e. the theory that explains why an effect
was caused, also is in doubt.
 21

Building Glass Boxes

Whether the kinds and amounts o f incommensurability between theo-

retical and actual causes of observed effects illustrated hypothetically in the
preceding section exist and are obstacles to adequate theorizing is an empiri-
cal question. That so much divergence of effects is observed in reviews of
studies conducted under relatively similar conditions lends some support to
the proposition, b u t does not prove it. Some empirical evidence on this ques-
tion will be mentioned later. But merely penetrating the black box o f instruc-
tional effects is not sufficient for testing theories. The ultimate objective o f
future m e t h o d o l o g y in the service of theory-oriented research on instruction
should be to increase the probability that the theorist controls what goes on
inside these boxes. This achievement would answer the presently m o o t empir-
ical question a b o u t the viability o f positing varying responses to instructional
stimuli. In turn, this would clarify the construct validity o f putative causes
o f effects in theories of instruction.
The assumption about learners' inherent cognitive activity and strategic
purposefulness on which was based the preceding analysis o f sources of
variance for an instructional effect also suggests what might be done about
these sources. Rather than treat as black boxes learners' processing o f con-
tent directed b y instructional stimuli, theorists could build well-designed
glass boxes to test directly the hypotheses o f interest in their theory-testing
research. Glass boxes would contain machines for cognitive processing that
did what theorists wanted them to in response to instructional stimuli. And,
they would provide some overt evidence during instruction that signaled to
the theorist that the machine was working within its design parameters. Since
the learner's naturally occuring cognitive processing of instructional stimuli
probably was learned from prior experience, it seems logical that glass boxes
could be built by training learners to process instructional stimuli in accord
with the theorist's favored process rather than an u n k n o w n naturalistic one.
And, by requiring that learners provide observable o u t p u t from such pro-
cessing that would be otherwise impossible or very unlikely to occur if the
glass-box, cognitive-processing machine were n o t used as intended by its
designing theorist, demonstration that these machines were under control
could be achieved. Experiments then could be conducted to test the validity
o f hypotheses a b o u t h o w instructional effects are produced. F o r example,
learners trained to use a specific backward focusing strategy when encoun-
tering adjunct postquestions could be compared to another group of learners
trained to engage a forward selection strategy. The qualitative and quan-
titative differences in their learning, the o u t p u t from a glass box, then would
be explainable in terms of k n o w n differences in their cognitive processes.
Though simple to state, the nature o f such training and the monitoring
of its use b y learners would be complicated. It seems likely that several differ-
22

ent kinds o f training are required. First, learners may need training to dis-
criminate the occurrence of an instructional stimulus from content and other
instructional stimuli. Questions printed on otherwise blank pages would be
difficult for almost anyone to miss. But subtle instructional stimuli like a
teacher's conceptual-level question may not be discriminated from other
kinds of questions by students in recitation lessons (Stayrook et al., 1978;
Winne, 1979).
A second kind o f training may be required to introduce to students and
to equate among them what the theorist intends they should do as they
respond cognitively and overtly to identified instructional stimuli. For
example, much disagreement can be found as to whether a teacher's feed-
back cognitively serves as neutral information, as emotional reinforcers or
punishers, as bases for future attributions, or some combination of these.
Regarding learners' possible overt responses, does a teacher's feedback tell
students to revise notes, state that this feedback makes them feel like they
never get answers right, or a t t e m p t another answer if they feel confident?
Training that standardizes what learners perceive as the thing to do when the
teacher uses an instructional stimulus like feedback may clarify this debate.
Studies in which different groups of students are trained to do different
things will clarify the relation between feedback and achievement.
If a learner can identifY an instructional stimulus, knows what it com-
municates about how to operate on content, and can behave so that an
observer can log a congruence between intent and action, there still is no
guarantee that these actions will be carried out. Thus, a third kind of training
m a y be needed in which the learner learns to be motivated to use the instruc-
tional stimulus. Some understood and achievable advantage such as better
grades probably must be seen by learners to reward the effort involved in
doing what the instructional stimulus tells to be done cognitively and overtly.
Presumably, the closer, in some sense, this advantage is to the cognitive pro-
cessing required by the instructional stimulus, the greater the 'likelihood that
the learner will engage in the appropriate cognitive processing.
Finally, since theoretical interest centers on how execution of a cognitive
or behavioral process in response to instructional stimuli influences the learn-
ing of content, training learners to master the process itself also may be
required. Otherwise, individual differences in learners' ability to apply the
process intended by an instructional stimulus will mask the cause-effect
relation between the stimulus and learning by introducing variability into
what must be a constant cause for the theory. But mastery of the process
m a y n o t be enough. It also may be important to provide learners with a
criterion for judging whether they have successfully applied the process, and
to train them explicitly about what to do if they fail to reach this criterion.
Otherwise, their cognitive processing will again revert to uncontrolled varia-
bility as the causes of individuals' learning. Gathering supplementary data
 23

a b o u t the frequency with which the intended cognitive process was engaged
b u t the criterion was not reached then can be used informatively to covary
effects in tests o f theoretical propositions.
Thus, enhancing the probability of achieving control over several sources
o f variance previously unaccounted for in learners' responses to instructional
stimuli likely involves four types of training: discrimination training to isolate
these stimuli, paired-associate training to link each instructional stimulus
with knowledge of a dominant cognitive processing response that operates
on content, reinforced practice to build the learner's motivation to engage
the appropriate cognitive process, and direct training in components of the
cognitive processing. The latter training should provide criteria for learners
to use in recognizing success, plus procedures for coping when they fail to
execute the cognitive processing called for by the instructional stimulus.
Moreover, some overt evidence that all these events occured during instruc-
tion is needed to validate the equivalence o f the theorist's nominal stimulus
and the learner's operationalization of that stimulus.
Beyond this logical argument, there also exists some empirical evidence
to support the construction o f glass boxes. Brown et al. (1981) reviewed
some of this literature, but did not cast their analysis toward the goal of
testing theoretical propositions. Nonetheless, they, like Reder (1980) and
Sternberg (1981 ) support the utility of such training in research.
Winne and Marx (1979) and Winne (1981) have conducted studies
approaching the four kinds of training necessary to control learners' instruc-
tional processing to test theoretical propositions about h o w instructional
stimuli affect learning. In two experiments wherein university students were
trained to make specific cognitive responses to discrete lecturer behaviors,
Winne and Marx found that it was easy to achieve the first two stages o f
operationalizing students' cognitive responses to instructional stimuli. Their
study failed, however, with respect to the latter two stages: first, because of
a fundamental difference between students' goals to record lecture content
versus the lecturer's intent that students learn lecture content as it was being
presented; and second, because the cognitive processes students were trained
to use overtaxed them and conflicted with their habitual procedures for
recording lecture content. Equally importantly, there was incommensurability
between the level o f specificity of the cognitive processes e m b e d d e d in instruc-
tional stimuli students were trained to use and the tests that gauged their
learning. The latter were considerably more molar than the former.
Winne (1981) attempted to remedy these flaws in a more tightly
controlled study of learning from text materials where the instructional
stimuli of interest were either instructional objectives or adjunct questions.
His study improved on the degree to which success was achieved in all four
kinds of training, and provided evidence that variance in achievement due to
individual differences in aptitudes was lessened when learners' use o f instruc-
24

tional stimuli was controlled. These two studies, among others (Winne,
1982), provide some preliminary empirical support about gains claimed for
experimental control and theoretical interpretability when the methodological
strategy outlined here is implemented. Thus, on b o t h logical and empirical
bases, making glass boxes out of heretofore black boxes seems worthy of
further effort.

Conclusion

This paper has attempted to achieve three objectives. The first was to
provide an operational definition of instruction. Based on the assumption
that learners are inherently cognitively active and strategically purposeful,
instruction was operationally defined as providing the learner with instruc-
tional stimuli that cue the learner to use particular cognitive operations on
the content presented to him. This guidance of cognitive processing sets
the stage for better content learning than would have occurred had these
instructional stimuli been absent.
Several features of this view warrant notice. The fact that instruction is
cast as communication introduces the notion that the theorist's signal about
h o w the learner should operate on content can be attenuated by noise when
it reaches the learner over the channel of communication, i.e. the operationally
defined independent variable in an experiment. U n d e r w o o d (1963) and
R o t h k o p f (1976) previously distinguished these two stimuli as nominal and
functional. Too little attention to this distinction and to achieving equivalence
among the two stimuli has been given in research on teaching and research
on instruction in general (Winne and Marx, 1979).
Another feature of this definition is that instruction is defined by its
effects rather than its appearances. Parallels to the operational definition of
reinforcement are obvious and welcomed. And, like the emerging cognitive
trend in behavioral theory (Bandura, 1977), instructional theory also should
investigate the cognitive processes learners use to prepare content for input
to learning processes per se. Focusing on the effects of instruction also
allows instructional psychology to be separated from the psychology o f learn-
ing. Learning content, by this definition, need not necessarily follow success-
ful instruction. Learners who receive a clear message about h o w the instructor
intends them to operate on content before that content is stored in an
appropriately retrievable form, and who successfully perform these prepara-
tory operations on the appropriate content are defined to have been success-
fully instructed. However, they still may be judged to fail at having learned
the content either because of a flaw or deficit in executing the storage pro-
cesses that set the stage for learning to be demonstrated, or because they can
n o t demonstrate learning by retrieving stored information in response to a
given retrieval cue, e.g. a test item (Bandura, 1977).
 25

This operational definition for instruction also points toward heretofore

unexamined sources of variance that rarely were controlled in prior research
on learning from instruction. An analysis o f these sources o f variance and
ways that they may lead a theorist to invalidly hypothesize a single cause for
an observed effect was the second objective o f this paper.
In an example o f h o w readers' might a t t e m p t to use adjunct postques-
tions as instructional stimuli in aid o f learning content, four loci for sources
o f unwanted variability in producing instructional effects were suggested:
identification o f the instructional stimulus, knowledge o f the message it
carried, motivation to engage in the cognitive operations intended b y the
instructional stimulus, and ability to execute these operations on content.
Alternative cognitive processes that might serve as generators o f a specific
forward effect in response to nominally equivalent instructional stimuli were
hypothesized in terms of these loci. In addition, an argument was made that
since these loci are fulcrums on which rest theoretical propositions a b o u t the
causes o f instructional effects, the probability o f equivalence between nominal
and actual instructional events must be maximized since the validity o f the
theory rests on this equivalence. Fundamentally, this is a problem of changing
u n k n o w n but theoretically crucial individual differences into controlled
individual equals. A n o t h e r way of phrasing this point is that tests o f theories
can tolerate variance arising from individual differences in response to an
instructional stimulus whose role as a cause is not being investigated, as was
the case for paragraph indentation earlier. Tests o f theoretical cause-effect
propositions can n o t tolerate variability in the cause o f a singular observed
effect, i.e. the cognitive processing of content cued by an instructional stim-
ulus like an adjunct question. Winne's experiment illustrated that some
progress toward making a constant out o f natural variance in learners' use o f
instructional stimuli can be achieved (Winne, 1981). Whether this procedure
will advance theories of instructional effects is an empirical q u e s t i o n that
can be answered only by further experimentation.
The third objective o f this paper was to indicate briefly that much
already is known about h o w to achieve control over these four loci o f instruc-
tional effects. Specifically, training learners to criterion in each locus can
build solidly on what already is k n o w n a b o u t discrimination training, paired-
associate learning where the associate response is knowledge o f a rule for
cognitive processing of content, motivation training via reinforced practice,
and skill training. Incorporating these kinds of training as preparation for
experiments in the service of instructional theory m a y help to penetrate
black boxes which heretofore concealed the elements of theoretical interest.
It also will force experimenters to build glass boxes that reveal explicitly what
the theory is all about.
The requirement that overt evidence be obtained on the operation o f
the cognitive machinery inside the glass boxes was vaguely explained here
26

because work on how to accomplish this without disrupting instruction is

just beginning. For example, Winne (1981) attempted to have learners
demonstrate that they processed adjunct postquestions by having them under-
line parts of preceding text that provided the information for an answer to
the question, and by indicating on their paper the degree to which their
mental answer to the question was correct. Winne and Marx ( 1 9 8 2 ) a s k e d
learners to circle parts of examples on worksheets that represented the
defining attributes of concepts being taught in simulated recitation lessons.
This was intended to signal whether learners had analyzed content in accor-
dance with an instructional stimulus they had been trained to identify. Both
procedures were only partially successful. Another possible strategy may be
to ask learners what their cognitive processing was during instruction, though
Nisbett and Wilson's (1977) seminal paper on such procedures delineates
some possibly insurmountable obstacles inherent in this approach.
The ultimate value of trying to build glass boxes is that glass shatters
easily. Theories of instructional effects and their links to learning will become
more directly testable because inferences about operations behind opaque
screens will be replaced by manipulated events hypothesized to be causes.
This will make instructional events into independent variables that are
operationally defined as rigorously as are variables that comprise the tradi-
tional elements of experimental designs in instructional research. This, in
turn, will enhance the validity of tests of hypotheses about why certain
effects were observed. These more rigorous tests may play a role in generating
further hypotheses about instruction, but their most immediate value lies in
reducing the degree to which interpretations of effects must be inferential.

Acknowledgements

The authors thank Roger Gehlbach, Bryan Hiebert, Janet Kendall,

Jack Martin and especially Ron Marx for thoughtful comments on earlier
drafts.

References

Anderson, L. M., Evertson, C. M. and Brophy, J. E. (1979). "An experimental study of

effective teaching in first grade reading groups," Elementary School Journal 79:
193-223.
Bandura, A. (1977). Social Learning Theory. Englewood Cliffs, N.J.: Prentice Hall.
Brown, A. L., Campione, J. C. and Day, J. D. (1981). "Learning to learn: on training
students to learn from texts," Educational Researcher 10 (2): 14-21.
Brunet, J. S., Goodnow, J. J. and Austin, G. A. (1956). A Study of Thinking. New York:
John Wiley.
 27

Cook, T. D. and Campbell, D. T. (1979). Quasi-experimentation: Design and Analysis

Issues for Field Settings. Chicago: R a n d - M c N a l l y .
Cooper, J. M. (Ed.) (1977). Classroom Teaching Skills: A Handbook. Lexington, MA:
D. C. Heath.
Dunkin, M. J. and Biddle, B. J. (1974). The Study of Teaching. New York: Holt, Rinehart
and Winston.
Gage, N. L. (1978). The Scientific Basis of the Art of Teaching. New York: Teachers
College Press.
Gage, N. L. (1980). "The causal connections underlying teacher education: their status
and prospects," paper presented at the annual meeting of the American Educational
Research Association, Boston.
Gagn~, E. D. (1979). "Long-term retention of information following learning from prose,"
Review of Educational Research 48: 6 2 9 - 6 6 5 .
Good, T. and Grouws, D. (1977). "Teaching effects: a process-product study in fourth
grade mathematics classrooms," Journal of Teacher Education 28: 4 9 - 5 4 .
Greeno, J. G. (1978). " A study of problem solving," in R. Glaser (Ed.), Advances in
Instructional Psychology, Vol. 1. Hlllsdale, N.J.: Lawrence Erlbaum.
Hoover, K. H. (1976). The Professional Teacher's Handbook. Boston: Allyn and Bacon.
Nisbett, R. E. and Wilson, T. D. (1977). "Telling more than we can know: verbal reports
on mental processes," Psychological Review 8 4 : 2 3 1 - 2 5 9 .
Reder, L. M. (1980). "The role of elaboration in the comprehension and retention of
prose: a critical review," Review o f Educational Research 50: 5 - 5 3 .
Rescorla, R. A. (1972). " I n f o r m a t i o n variables in Pavlovian conditioning," in G. Bower
(Ed.), The Psychology o f Learning and Motivation, Vol. 6. New York: Academic Press.
Rice, J. M. (1897). "The futility of the spelling grind," Forum 2 3 : 1 6 3 - 1 7 2 , 4 0 9 - 4 1 9 .
Rickards, J. P. and Denner, P. R. (1978). "Inserted questions as aids to reading t e x t , "
Instructional Science 7 : 3 1 3 - 3 4 6 .
Rickards, J. P., Anderson, M. C. and McCormick, C. B. (1976). "Processing effects of
common-word and number questions inserted in reading materials," Journal of Educa-
tional Research 69: 2 7 4 - 2 7 7 .
Rothkopf, E. Z. (1976). "Writing to teach and reading to learn: a perspective on the
psychology of written instruction," in Gage, N. L. (Ed.), The Psychology of Teaching
Methods. Chicago: University of Chicago Press.
Rothkopf, E. Z. and Bisbicos, E. (1967). "Selective facilitation effects of interspersed
questions on learning from written materials~" Journal ofEducationalPsychology 58:
56-61.
Stayrook, N. G., Corno, L. and Winne, P. H. (1978). "Path analyses relating student per-
ceptions of teacher behavior to student achievement," Journal of Teacher Education
24 (2): 5 1 - 5 6 .
Sternberg, R. J. (1981). "Nothing fails like success: the search for an intelligent paradigm
for studying intelligence," Journal of Educational Psychology 73: 1 4 2 - 1 5 5 .
Travis, L. D. (1979). "Interland schooling and branch-plant psychology: educational
psychology in Canada t o d a y , " Canadian Journal of Education 4 (4): 2 4 - 4 2 .
Underwood, B. J. (1963). "Stimulus selection in verbal learning," in Cofer, C. N. and
Musgrave, B. S. (Eds.), Verbal Behavior and Learning. New York: McGraw-Hill.
Winne, P. H. (1979). "Experiments relating teachers' use of higher cognitive questions to
student achievement," Review of Educational Research 4 9 : 1 3 - 4 9 .
Winne, P. H. (1981). "Matching students' cognitive processing to text with instructional
objectives or inserted questions," paper presented at the annual meeting of the American
Educational Research Association, Los Angeles.
28

~inne, P. H. and Marx, R. W. (1979). "Perceptual problem solving," in Peterson, P. L. and

Walberg, H. J. (Eds.), Research on Teaching. Berkeley, CA: McCutchan.
Winne, P. H. and Marx, R. W. (1980). "Matching students' cognitive responses to teaching
skills," )ournal of Educational Psychology 72: 257-264.
Winne, P. H. and Marx, R, W. (1982). "Training students in using cognitive processes for
classroom learning," paper to be presented at the annual meeting of the American
Educational Research Association, New York.