A Detailed Examination of
Scientific Method
by Craig Rusbult, Ph.D.
Most links in this page are italicized links that will keep you
inside the page and will be very fast, and (unless you're using
MS IE-Explorer for Mac) your browser's BACK-button will return
you to where you were. But the rare non-italicized links
open a new page in a new window, so this big page will remain
open in this window and you won't have to wait for it to relaod.
For easy navigation inside the page, there are three options:
A. click on any link in the brief Table of Contents below,
B. click on any element in the image-map that follows it, or
C. click on any link in the detailed Table of Contents.
And at the end of this page, the main ideas in these 9 sections
are condensed in the introductory Overview of Scientific Method.
2
These links are all inside-the-page, even though (to make them easier to read) they're not
italicized.
Introduction
A DESERVEDLY HUMBLE DISCLAIMER.
an entire book, which can treat these ideas with greater detail and sophistication than in
my brief summary.
In developing a model of Integrated Scientific Method (ISM), one major challenge was
the selection of words and meanings. If everyone used the same terms to describe
scientific methods, I would use these terms in ISM. Unfortunately, there is no consistent
terminology. Instead, there are important terms -- especially model, hypothesis, and
theory -- with many conflicting meanings, and meanings known by many names. Due to
this inconsistency, I have been forced to choose among competing alternatives. Despite
the linguistic confusion, over which I have no control, in the context of ISM I have tried
to use terms consistently, in ways that correspond reasonably well with their common
uses by scientists, philosophers, and educators. {details about terminology }
NINE SECTIONS.
The framework of ISM is divided into nine sections: three for evaluation factors
(empirical, conceptual, and cultural-personal), three for activities (evaluating theories,
generating theories, and experimental design), and one each for problem solving, thought
styles, and productive thinking. Sections 1-6 assume that during problem formulation
there already has been the selection of an area of nature to study; and in Sections 1-4 and
6, there is already a theory about this area.
. The "Goals of ISM" page makes a distinction between the ISM framework and an
elaboration of this framework by myself or by others. The overview describes the ISM
framework with minimal elaboration. In this "details" page there is lots of elaboration,
but much of this is a discussion of concepts that I consider a part of the ISM framework
because they are essential for accurately describing science. Therefore, the ISM
framework includes everything in the overview, and more. Perhaps in the future I will
try to define the precise content-and-structure of the ISM framework, but for now this
definition remains flexible, partly because my own concept of the framework keeps
changing as I continue to think about the methods used by scientists.
The following elaboration assumes the reader is familiar with the "Overview of
Scientific Method" as background knowledge. As a reminder, and so you can easily
review, at the beginning of each section there is a link to the corresponding description
(located at the end of this page) from the overview. And at the end of each section there
is a link to the Table of Contents at the top of this page.
5
REFERENCES.
EXPERIMENTAL SYSTEM
SUPPLEMENTARY THEORIES include, but are not limited to, theories used to
interpret observations. Shapere (1982) analyzes an "observation situation" as a 3-stage
process in which information is released by a source, is transmitted, and is received by a
receptor, with scientists interpreting this information according to their corresponding
theories of the source, the transmission process, and the receptor.
The label "supplementary" is based on assumptions about goals. For example, in the
early 1950s when "DNA chasers" were generating and evaluating theories for DNA
structure, this DNA theory was the main theory, while theories about x-rays (including
their generation, transmission, interaction with DNA, and detection) were the
supplementary theories. But these x-ray theories -- in a different context, during an
earlier period of science when the main goal was to develop x-ray theories -- were
considered to be the main theories.
PREDICTIONS.
By using a model that is based on a specified system and theory, scientists can make
predictions in more than one way: by logical deduction beginning with a composition-
and-operation model, by calculation, by "running a model" mentally or in a computer
simulation, or by inductive logic that assumes the results will be similar to those in
previous experiments with similar systems. If predictions can be made in several ways
for the same system, this will serve as a cross-check on the predictions and on the
predicting methods. {more on thought experiments}
It can be useful to think of combining two sources -- a general domain-theory (that
applies to all systems in a domain) and a specific system-theory (about the
characteristics of one system, especially about the initial system-conditions) -- in order to
predict the final system-conditions. Thinking in terms of a domain-theory and a system-
theory is also useful for the retroductive generation of ideas for a theory. { In additon to
"retroductive generation..." in Section 5, I've recently written more about how a domain-
theory and system-theory are combined to construct a model and make predictions, in the
Overview of Scientific Method. }
DEGREE OF AGREEMENT.
"Was the prediction likely to agree with the data even if the model under
consideration does not provide a good fit to the real world?" (Giere, 1991, p. 38)
Would the results be surprising if the model was not a good representation of
the system? (this question applies the "Surprise Principle" of Sober, 1991)
Should an agreement between predictions and observations elicit a response of
"So what?" or "Wow!" ?
To what extent does the experiment provide a crucial test that can discriminate
between this theory and alternative theories?
What is the degree of contrast between the predictions of this theory and the
predictions of plausible alternative theories?
For any experiment, a degree of predictive contrast can be estimated by asking one or
more of these five questions. For example, the results of the hat-lifting experiment are
likely to occur even if OW is false, so we wouldn't be surprised by this observation even
if OW was false, and a response of "so what" is justified; the experiment does not
discriminate between theories, because there is no contrast between the predictions of
OW and the predictions of plausible alternative theories.
A consideration of predictive contrast is useful because it functions as a
counterbalance to the skeptical principle that a theory is not proved by agreement
between predictions and observations. Despite the impossibility of proof, the status of a
theory increases when it is difficult to imagine any other plausible theory that could make
the same correct predictions. Of course, an apparent lack of alternative explanations
could be illusory, due to a lack of imagination, but scientists usually assume that a high
degree of predictive contrast increases the justifiable confidence in a claim that there is a
connection between a prediction-observation agreement and a model-system similarity.
An empirical evaluation should include all experiments, past and present that seem
relevant for achieving the goals of the evaluators. When they generate a theory from
multiple sources of data, scientists use art and logic.
Table of Contents
ISM follows Laudan (1977) in making a distinction between empirical factors and
conceptual factors, and between conceptual factors that are internal and external. Internal
conceptual factors (regarding components and logical structure) involve the
characteristics and logical interrelationships of a theory's own components, while external
conceptual factors are the external relationships between a theory's components and the
components of other theories (either scientific or cultural-personal). Because this is such
a long section, it is split into four parts: three to discuss internal characteristics
(simplicity, constraints, utility), and one for external relationships.
2A. Simplicity
LOGICAL SYSTEMATICITY.
To illustrate logical structure, Darden (1991) compares two theories that claim to
explain the same data; T1 contains an independent theory component for every data
point, while T2 contains only a few logically interlinked components. Even if both
theories have the same empirical adequacy, most scientists will prefer T2 due to its
logical structure.
When one component is not logically connected to other components, it is usually
considered an ad hoc appendage that makes a theory less logically systematic and less
desirable. If scientists perceive T1 as an inelegant patchwork of ad hoc components that
have no apparent function except to achieve empirical agreement with old data, they will
not be impressed with T1's predictions, and will they not expect T1 to successfully
predict new data.
Another perspective: T1 has specialized components, by contrast with the generalized
components of T2.
Internal consistency, with logical agreement among a theory's components, is highly
valued. Systematicity is weakened by an independence of components (with no
relationships) as in T1, but inconsistency among components (with bad relationships) is
the ultimate non-systematicity.
SIMPLIFIED MODELS.
first approximation, and then making adjustments. When applying Newton's Theory to a
falling object, a stripped-down model might ignore the effects of air resistance and the
change in gravitational force as the ball changes altitude. For some purposes this
simplified model is sufficient. And if scientists want a more complete model, they can
include one or more "correction factors" that previously were ignored. The inclusion of
different factors produces a family of models with varying degrees of completeness, each
useful for a different situation and objective.
For example, if a bowling ball is dropped from a height of 2 meters, air resistance can
be ignored unless one needs extremely accurate predictions. But when a tennis ball falls
50 meters, predictions are significantly inaccurate if air resistance is ignored. And a
rocket will not make it to the moon based on models (used for making calculations) that
do not include air resistance and the variation of gravity with altitude. In comparing
these situations there are two major variables: the weighting of factors (which depends
on goals), and degrees of predictive contrast. Weighting of factors: for the moon rocket
a demand for empirical accuracy is more important than the advantages of conceptual
simplicity, but for most bowling ball scenarios the opposite is true. Predictive contrast:
for the rocket there is a high degree of predictive contrast between alternative theories
(one theory with air resistance and gravity variations, the other without) and the complex
theory makes predictions that are more accurate, but for the bowling ball there is a low
degree of predictive contrast between these theories, so empirical evaluation does not
significantly favor either model.
These conflicts are common. For example, in a famous statement of simplicity known as
Occam's Razor -- "entities should not be multiplied, except from necessity" -- a
preference for ontological economy ("entities should not be multiplied") can be overcome
by necessity. But evaluation of "necessity," such as judging whether a theory revision is
improvement or ad hoc tinkering, is often difficult, and may require a deep understanding
of a theory and its domain, plus sophisticated analysis.
A common reason for non-simplicity is a desire for empirical adequacy, since
including additional components in a theory may help it predict observations more
accurately and consistently. Another reason is to construct a more complete model for
the composition and operation of systems.
11
Wimsatt (1987) discusses some ways that a false model can be scientifically useful.
Even if a model is wrong, it may inspire the design of interesting experiments. It may
stimulate new ways of thinking that lead to the critical examination and revision (or
rejection) of another theory. It may stimulate a search for empirical patterns in data. Or
it may serve as a starting point; by continually refining and revising a false model,
perhaps a better model can be developed.
Many of Wimsatt's descriptions of utility involve a model that is false due to an
incomplete description of components for entities, actions, or interactions. When the
erroneous predictions of an incomplete model are analyzed, this can provide information
about the effects of components that have been omitted or oversimplified. For example,
to study how "damping force" affects pendulum motion, scientists can design a series of
experimental systems, and for each system they compare their observations with the
predictions of several models (each with a different characterization of the damping
force); then they can analyze the results, in order to evaluate the advantages and
disadvantages of each characterization. Or consider the Castle-Hardy-Weinberg Model
for population genetics, which assumes an idealized system that never occurs in nature;
deviations from the model's predictions indicate possibilities for evolutionary change in
the gene pool of a population.
Table of Contents
Scientific communities develop preferences for the types of components that should
(and should not) be used in a theory. For example, prior to 1609 when Kepler introduced
elliptical planetary orbits, it was widely believed that in astronomical theories all motions
should be in circles with constant speed. This belief played a role in motivating
Copernicus:
Copernicus attacks the Ptolemaic astronomy not because in it the sun moves
rather than the earth, but because Ptolemy has not strictly adhered to the precept
that all celestial motions must be explained only by uniform circular motions or
combinations of such circular motions. ... It has been generally believed that
Copernicus's insistence on uniform circular motion is part of a philosophical or
metaphysical dogma going back to Plato. (Cohen, 1985; pp 112-113)
In every field there are implicit and explicit constraints on theory components --- on
the types of entities, actions and interactions to include in a theory's models for
composition and operation. These constraints can be motivated by beliefs about ontology
(after asking "Does it exist?") or utility (by asking "Will it be useful for doing science?").
For example, an insistence on uniform circular motion could be based on the ontological
belief that celestial bodies never move in noncircular motion, or on the utilitarian
rationale that using noncircular motions makes it more difficult to do calculations.
A positivist believes that scientific theories should not postulate the existence of
unobservable entities, actions, or interactions. For example, behaviorist psychology
avoids the concept of "thinking" because it cannot be directly observed. A strict
positivist will applaud Newton's theory of gravitation, despite its lack of a causal
explanatory mechanism, because it is an empirical generalization that is reliable and
approximately accurate, and it does not postulate (as do more recent theories of gravity)
unobservable entities such as fields, curved space, or gravitons. But most scientists,
although they appreciate Newton's descriptive theory for what it is, consider the absence
of explanation to be a weakness.
some comments about terminology: Positivism was proposed in the 1830s by Auguste
Comte, who was motivated partly by anti-religious ideology. In the early 20th century a
philosophy of logical positivism was developed to combine positivism with other ideas.
In current use, "positivism" can be used in a narrow sense (as Comte did and as I do
here) or it can refer to anything connected with logical positivism, including the "other
ideas" and more. Logical positivism can also be called logical empiricism. {Notice that
empiricism (i.e., positivism) is not the same as empirical. A theory that is non-empiricist
(because it contains some components, such as atoms or molecules, that are
unobservable) can make predictions about empirical data that can be used in empirical
evaluation. }
Although positivism (or empiricism, the name typically given to current versions) is
considered a legitimate perspective in philosophy, it is rare among scientists, who
welcome a wide variety of ways to describe and explain. Many modern theories include
unobservable entities and actions, such as electrons and electromagnetic force, among
13
their essential components. Although most scientists welcome a descriptive theory that
only describes empirical patterns, at this point they think "we're not there yet" because
their limited theory is seen as just a temporary stage along the path to a more complete
theory. This attitude contrasts with the positivist view that a descriptive theory should be
the ending point for science.
The ISM framework includes two types of theories (and corresponding models) --
descriptive and explanatory -- so it is compatible with any type of scientific theory,
whether it is descriptive, explanatory, or has some characteristics of each. My own anti-
positivist opinions, which are not part of the ISM framework, are summarized in the
preceding paragraph, and are discussed in more depth on a page that asks Should
Scientific Method be Eks-Rated?
Table of Contents
ALTERNATIVE REPRESENTATIONS.
Even for the same theory, representations can differ. For example, a physics theory can
symbolically represent a phenomenon by words (such as "the earth orbits the sun in an
approximately elliptical orbit"), a visual representation (a diagram or animation depicting
the sun and the orbiting earth), or an equation (using mathematical symbolism for objects
and actions). More generally, Newtonian theory can be described with simple algebra (as
in most introductory courses), by using calculus, or with a variety of advanced
mathematical techniques such as Hamiltonians or tensor analysis; and each mathematical
formulation can be supplemented by a variety of visual and verbal explanations, and
14
A SYNTHESIS?
Philosophy of science and cognitive psychology overlap in areas such as the structuring
of scientific theories (studied by philosophers) and the structuring and construction of
mental models (studied by psychologists). Research in this exciting area of synthesis is
currently producing many insights that are helping us understand the process of thinking
in science, and that will be useful for improving education.
Of course, these two aspects of scientific utility are related. In particular, cognitive
utility plays an important role in making a theory useful for doing research.
15
Laudan (1977) observes that even when a theory has weaknesses, and evaluation
indicates that it is not yet worthy of acceptance (of being treated as if it were true),
scientists may rationally view this theory as worthy of pursuit (for exploration and
development by further research) if it shows promise for stimulating new experimental or
theoretical research:
Laudan suggests that when scientists judge whether a theory is worthy of pursuit,
instead of just looking at its momentary adequacy, they study its rate of progress and
potential for improvement. Making a distinction between acceptance and pursuit is
useful when thinking about scientific utility, because a theory can have a low status for
acceptance, but a high status for pursuit. If a theory is judged to be worthy of pursuit but
not acceptance, it needs development but it shows enough promise to be considered
worth the effort.
Working out the logical relations between components may require some period
of time. And it may even be useful to consider generating hypotheses inconsistent
with some other component; maybe the other component is the problematic one.
(Darden, 1991; p. 258)
For a developing theory, some criteria are less rigorous, but other characteristics --
such as a flexibility that allows easy revision, and extendability for adapting to a
widening domain -- may be more important than in a mature theory.
A new theory can promote research by offering a new perspective on the composition
and operation of experimental systems, and by inspiring ideas for new systems and
techniques. { Of course, even after a theory has passed through the pursuit phase and is
generally accepted, there may be opportunities for experimenting (to explore the old
16
theory's application for new systems) and theorizing. But often the opportunities for
exciting research are more plentiful with a new theory. }
TESTABILITY.
Table of Contents
In the late nineteenth century, natural selection and isolation were viewed as rival
explanations for the origin of new species; the evolutionary synthesis showed that
the two processes were compatible and could be combined to explain the splitting
of one gene pool into two. (Darden, 1991, p. 269)
Of course, a declaration that "both factors contribute to speciation" is not the end of
inquiry. Scientists can still analyze an evolutionary episode to determine the roles played
17
by each factor. They can also debate the importance of each factor in long-term
evolutionary scenarios involving many species. And there can be an effort to develop
theories that more effectively combine these factors and their interactions.
A different type of coexistence occurs with Valence Bond theory and Molecular
Orbital theory, which each use different types of simplifying approximations in order to
apply the core principles of quantum mechanics for describing the characteristics of
molecules. Each approach has advantages, and the choice of a preferred theory depends
on the situation: the molecule being studied, and the objectives; the abilities, experience,
and thinking styles of scientists; or the computing power available for numerical
analyses. Or perhaps both theories can be used. In many ways they are complementary
descriptions, as in "The Blind Men and the Elephant," with each theory providing a
useful perspective. This type of coexistence (where two theories provide two
perspectives) contrasts with the coexistence in speciation (where two theories are
potential co-agents in causation) and with the non-coexistence in oxidative
phosphorylation (where one theory has vanquished its former competitors).
When two or more theories are in conflict, as described above, there is a conceptual
difficulty for all of the theories, but especially for those in which scientists have less
confidence. Conversely, agreement about the characteristics of shared components can
lend support to these components. For example, many currently accepted theories
contain, as an essential component, time intervals of long duration. Physical processes
occur during this time, and these processes are necessary for empirical adequacy in
explaining observations; if the time-component is changed to a shorter time (such as the
10,000 years suggested by young-earth creationists) the result will be erroneous
predictions about a wide range of phenomena. Theories containing an old-earth
component span a wide range, with domains that include ancient fossil reefs, sedimentary
rock formations (with vertical changes), seafloor spreading (with horizontal changes) and
continental drift, magnetic reversals, radioactive dating, genetic molecular clocks,
18
paleontology, formation and evolution of stars, distances to far galaxies, and cosmology.
In a wide variety of theories, the same type of component (for amount of time) always
has the same general value: a very long time. This provides support for the shared
component -- an old earth (and an old universe) -- and this support increases because an
old earth is an essential component of many theories that in other ways, such as the
domains they claim and the other components they use, are relatively independent. This
independence makes it less likely -- compared with a situation where two theories are
closely related and share many essential components, or where the plausibility of each
theory depends on the plausibility of the other theory -- that suspicions of circular
reasoning are justified. { Of course, the relationships that do exist between these old-
earth theories can be considered when evaluating the amount of circularity in the support
claimed for the shared component. }
But in these theories, is the age of the earth a component or a conclusion? It depends
on perspective. In most cases the age can be viewed as a conclusion reached by "solving
an equation" (such as the one describing the earth's rate of cooling) for time; all of the
theories claim to describe the same type of phenomenon (involving time), so they share a
domain rather than a component. But it also makes sense to think of time as a component
because, in each case, time is one aspect of a theory whose main goal is to explain the
phenomenon being studied -- a fossil reef, rock formation, seafloor spreading,... -- not to
explain the time. Or perhaps the long time-interval can be viewed as a supplementary
theory that in each area is needed to produce adequate models. With any of these
perspectives, the conclusion (of strong support for a long period of time) is similar.
THEORIES WITH WIDE SCOPE. Another type of relationship occurs when one
theory is a subset of another theory, as with DNA structure and atomic theory. During
the development of a theory for DNA structure, scientists assumed the constraint that
DNA must conform to the known characteristics of the atoms (C, H, O, N, P) and
19
domain of Dominance. Their initial theory was thus modified into a sub-theory with a
narrower scope, and other sub-theories were invented for parts of the original domain not
adequately described by dominance. Eventually, these sub-theories were combined to
construct an overall mega-theory of genetics that, compared with the initial theory of
dominance, had the same wide scope, with greater empirical adequacy but less simplicity.
Two types of coexistence: when each competing theory describes a causal factor, or
when each provides a useful perspective. A third type of coexistence, described in the
paragraph above, is when sub-theories that are in competition (because they describe the
same type of phenomena) "split up" the domain claimed by a mega-theory that contains
both sub-theories as components; each sub-theory has its own sub-domain (consisting of
those systems in which the sub-theory is valid) within the larger domain of the mega-
theory.
Newtonian Physics is another theory whose initially wide domain (every system in the
universe!) has been narrowed. This change occurred in two phases. In 1905 the theory
of special relativity declared that Newton's theory is not valid for objects moving at high
speed. And in 1925, quantum mechanics declared that it is not valid for objects with
small mass, such as electrons. Each of these new theories could derive Newtonian
Physics as a special case; within the domain where Newtonian Physics was
approximately valid, its predictions were duplicated by special relativity (for slow
objects) and by quantum mechanics (for high-mass objects). But the reverse was not
true; special relativity and quantum mechanics could not be derived from Newton's
theories, which made incorrect predictions for fast objects and low-mass objects.
Even though quantum mechanics is currently considered valid for all systems, it is
self-limited in an interesting way. For some questions the theory's answer is that "I
refuse to answer the question" or "the answer cannot be known." But a response of "no
comment" is better than answers that are confidently clear yet wrong, such as those
offered by the earlier Bohr Model. Some of the non-answers offered by quantum
mechanics imply that there are limits to human knowledge. This may be frustrating to
some people, but if that is the way nature is, then it is better for scientists to admit this (in
their theories) and to say "sorry, we don't know that and we probably never will."
Table of Contents
the quantum era in 1900) and Erwin Schrodinger (who formulated a successful quantum
theory in 1926).
[Planck, in a letter to Schrodinger, says] "I am reading your paper in the way a
curious child eagerly listens to the solution of a riddle with which he has struggled
for a long time, and I rejoice over the beauties that my eye discovers."
[Schrodinger replies by agreeing that] "everything resolves itself with
unbelievable simplicity and unbelievable beauty, everything turns out exactly as
one would wish, in a perfectly straightforward manner, all by itself and without
forcing."
What are the results of mutual interactions between science and society? How does
science affect culture, and how does culture affect science?
SCIENCE AFFECTS CULTURE. The most obvious effect of science has been its
medical and technological applications, with the accompanying effects on health care,
lifestyles, and social structures. But science also influences culture, in many modern
societies, by playing a major role in shaping cultural worldviews, concepts, and thinking
24
patterns. Sometimes this occurs by the gradual, unorchestrated diffusion of ideas from
science into the culture. At other times, however, there is a conscious effort, by scientists
or nonscientists, to use "the authority of science" for rhetorical purposes, to claim that
scientific theories and evidence support a particular belief system or political program.
CULTURE AFFECTS SCIENCE. ISM, which is mainly concerned with the operation
of science, asks "How does culture affect science?" Some influence occurs as a result of
manipulating the "science affects culture" influence described above. If society wants to
obtain certain types of science-based medical or technological applications, this will
influence the types of scientific research that society supports with its resources. And if
scientists (or their financial supporters) have already accepted some cultural concepts,
such as metaphysical and/or ideological theories, they will tend to prefer (and support)
scientific theories that agree with these cultural-personal theories. In the ISM diagram
this influence appears as a conceptual factor, external relationships...with cultural-
personal theories. For example, the Soviet government supported the science of
Lysenko because his theories and research supported the principles of Marxism. They
also hoped that this science would increase their own political power, so their support of
Lysenko contained an element of self-interest.
the Soviet political policies would gain popular support if there was a belief that this
policy was based on (and was consistent with) reliable scientific principles. And if
science "plays a major role in shaping cultural...thinking patterns," the government
wanted to insure that a shaping-of-ideas by science would support their ideological
principles and political policies. The government officials also wanted to maintain and
increase their own power, so self-interest was another motivating factor.
FEEDBACK. In the ISM diagram, three large arrows point toward "evaluation of
theory" from the three evaluation factors, and three small arrows point back the other
way. These small arrows show the feedback that occurs when a conclusion about theory
status already has been reached based on some factors and, to minimize cognitive
dissonance, there is a tendency to interpret other factors in a way that will support this
conclusion. Therefore, each evaluation criterion is affected by feedback from the current
status of the theory and from the other two criteria.
CONTROVERSY. Among scholars who study science there is a wide range of views
about the extent to which cultural factors influence the process and content of science.
These debates, and the role of cultural factors in ISM and in science education, are
discussed on the "Hot Debates about Science" page. Briefly summarized, my opinion is
that an extreme emphasis on cultural influence is neither accurate nor educationally
beneficial, and that even though there is a significant cultural influence on the process of
science, usually (but not always) the content of science is not strongly affected by
cultural factors.
Table of Contents
4. Theory Evaluation
This is a relatively short section because I don't want to duplicate the many discussions
of evaluation in Sections 1-3 (three types of evaluative inputs), 5 and 6 (using evaluation
to generate theories and experiments), 7 and 8 (evaluation in research and thought styles),
26
and 9 (critical thinking). And the EKS-RATED page discusses many controversial ideas
related to theory evaluation.
The overview briefly describes the main concepts of evaluation: inputs from three
types of factors (empirical, conceptual, and cultural-personal), and an output of status that
is an estimate of a theory's plausibility and/or usefulness; decisions to retain, revise, or
reject; pursuit and acceptance; rationally justified confidence instead of proof or
disproof; intrinsic status and relative status.
This section will not review these concepts, but will discuss (in more detail than
elsewhere) four topics: delayed decision, intrinsic and relative status, variable-strength
conclusions and hypotheses, and conflicts between different evaluative criteria.
DELAY. A fourth option for a decision (in addition to retain, revise, and reject) is not
shown in the ISM diagram: there can be a delay in responding, while other activities are
being pursued. Sometimes there is no conscious effort to reach a conclusion because
there is no need to decide. However, a decision (and action) may be required even
though evaluation indicates that only a conclusion of "inconclusive" is warranted. In this
uncomfortable situation, a wise approach is to make the decision (and do the action) in a
way that takes into account the uncertainties about whether or not the theory is true.
If a conclusion is delayed and a theory is temporarily ignored while other options are
pursued, and this theory is eventually revived for pursuit or acceptance, then in hindsight
we can either say that during the delay the theory was being retained (with no application
or development) or that it was being tentatively rejected with the option of possible
reversal in the future. But if this theory is never revived, then when it was ignored it was
actually being rejected.
INTRINSIC STATUS and RELATIVE STATUS. A theory has its own intrinsic
status that is an estimate of the theory's plausibility and/or usefulness. And if science is
viewed as a search for the best theory -- whether "the best" is defined as the most
plausible or the most useful -- there is implied competition, so each theory also has a
relative status.
A change in the intrinsic status of one theory will affect the relative status of
competitive theories. In the ISM-diagram this feedback is indicated by a small arrow
pointing from "alternative theories" to "status of theory relative to competitors."
A theory can have low intrinsic status even if it is judged to be better than its
competitors and therefore has high relative status, if evaluation indicates that none of the
current theories is likely to be true or useful. For example, before publication of the
famous double helix paper in April 1953, an honest scientist would admit that "we don't
know the structure of DNA." After the paper, however, among knowledgeable scientists
this skepticism quickly changed to a confident claim that "the correct structure is a
double helix." In 1953 the double helix theory attained high intrinsic status and relative
status, but before 1953 all theories about DNA structure had low intrinsic status, even
though the best of these would, by default, have high relative status as "the best of the
bad theories."
is an educated estimate rather than certainty. This concept is useful because it allows a
flexibility that doesn't force thinking into dichotomous yes-or-no channels.
Another stimulater of flexible, careful thinking is ISM's definition (based on Giere,
1991) of a hypothesis as a claim that a system and a theory-based model are similar in
specified respects and to a specified (or implied) degree of accuracy. With this
definition, different hypotheses can be framed for the same model. The strongest
hypothesis would claim an exact correspondence between all model-components and
system-components, while a weaker hypothesis might claim only an approximate
correspondence, or a correspondence (exact or approximate) for some components but
not for all. If a theory is judged to be only moderately plausible, the uncompromising
claims of a strong hypothesis will be rejected, even though scientists might accept the
diluted claims of a weak hypothesis.
Table of Contents
28
5. Theory Generation
An Overview of Scientific Method, Section 5
The rest of this section describes strategies for selecting or inventing theories.
RETRODUCTION and DEDUCTION. In contrast with deductive logic that asks, "If
this is the model, then what will the observations be?", retroductive logic -- which uses
deduction supplemented by imaginative creativity -- asks a reversed question in the past
tense, "These were the observations, so what could the model (and theory) have been?"
The essence of retroductive inference is doing thought-experiments, over and over, each
time "trying out" a different model that is being proposed (by selection or invention) with
the goal of producing deductive predictions that match the known observations.
Basically, the goal is to find a theory that, if true, would explain what has been observed.
Retroduction is useful when, after an experiment is over, scientists are not sure that
they know how to interpret what happened. In this context of uncertainty they search for
a theory (either old or new) that will help them make sense of what they have observed.
possibilities for revising these theories, an inventor may discover relationships between
them. In particular, a domain-theory (about all systems in the theory's domain) will
usually influence a system-theory about one system in this domain.
An interesting example of revising a system-theory was the postulation of Neptune. In
the mid-1800s, data from planetary motions did not precisely match the predictions of a
domain-theory, Newtonian Physics. By assuming the domain-theory was valid, scientists
retroductively calculated that if the system contained an extra planet, with a specified
mass and location, predictions would match observations. Motivated by this newly
invented system-theory with an extra planet, astronomers searched in the specified
location and discovered Neptune. Later, in an effort to resolve the anomalous motion of
Mercury, scientists tried this same strategy by postulating an extra planet, Vulcan,
between Mercury and the Sun. But this time there was no extra planet; instead, the
domain-theory (Newtonian physics) was at fault, and eventually a new domain-theory
(Einstein's theory of general relativity) made correct predictions for the motion of
Mercury. In these examples, both of the components used for constructing a model were
revised; there was a change in the system-theory (with Neptune) and in the domain-
theory (for Mercury).
In another example, described earlier, the discovery of radioactivity in 1903 caused a
revision of a system-theory for the earth's interior geology. This revised system-theory,
combined with observations (of the earth's temperature) and a domain-theory
(thermodynamics), required a revision in another theory component (the earth's age),
thereby settling an interfield conflict that began in 1868.
What are the results of theory generation? In the ISM-diagram, arrows point from
theory generation to system-theory and domain-theory, because both are needed to
construct a model. Three more arrows point to "theory" and "supplementary theory"
(because both can be used for constructing a domain-theory) and to "alternative theory"
because a newly invented theory competes with the original unrevised theory. Or the
original theory might become an alternative, since labeling depends on context; what
scientists consider a main theory in one situation could be alternative or supplementary in
other situations.
GENERATION AND EVALUATION. Although C.S. Peirce (in the 1800s) and
Aristotle (much earlier) studied theory invention, as have many psychologists, most
philosophers separated evaluation from invention, and focused their attention on
evaluation. Recently, however, many philosophers (such as Hanson, 1958; and Darden,
1991) have begun to explore the process of invention and the relationships between
invention and evaluation. Haig (1987) includes the process of invention in his model for
a "hypothetico-retroductive inferential" scientific method.
Generation (by selection or invention) and evaluation are both used in retroduction,
with empirical evaluation acting as a motivation and guide for generation, and generation
producing the idea being evaluated. It is impossible to say where one process ends and
the other begins, or which comes first, as in the classic chicken-and-egg puzzle.
The generation of theories is subject to all types of evaluative constraints. Empirical
adequacy is important, but scientists also check for adequacy with respect to cultural-
personal factors and conceptual criteria: internal consistency, logical structure, and
external relationships with other theories.
INVENTION BY REVISION. Invention often begins with the selection of an old (i.e.,
previously existing) theory that can be revised to form a new theory.
ANALYSIS AND REVISION. One strategy for revising theories begins with analysis;
split a theory into components and play with them by thinking about what might happen
if components (for composition or operation) are modified, added or eliminated, or are
reorganized to form a new structural pattern with new interactions.
31
According to Lakatos (1970), scientists often assume that a "hard core" of essential
theory components should not be changed, so an inventor can focus on the "protective
belt" of auxiliary components that are devised and revised to protect the hard core.
Usually this narrowing of focus is productive, especially in the short term. But
occasionally it is useful to revise some hard-core components. When searching for new
ideas it may be helpful to carefully examine each component, even in the hard core, and
to consider all possibilities for revision, unrestrained by assumptions about the need to
protect some components. By relaxing mental blocks about "the way things must be" it
may become easier to see theory components or data patterns in a new way, to imagine
new possibilities.
Or it may be productive to combine this analytical perspective with a more holistic
view of the theory, or to shift the mode of thinking from analytical to holistic.
And sometimes a knowledge of theories in other areas will lead to the recognition that
an existing theory from another domain can be generalized, as-is or modified, into the
domain being studied by a scientist. This is selection rather than invention, but it still
"brings something new" to theorizing in the domain. And the process of selection is
similar to the process of invention, both logically and psychologically, if (as in this case)
selection requires the flexible, open-minded perception of a connection between domains
that previously were not seen as connected.
Table of Contents
When scientists generate and evaluate experiments (i.e., when they design
experiments), they consider the current state of theory evaluation; they check for gaps in
their knowledge of systems; and they do thought-experiments for a variety of potential
experimental systems, looking for systems that might produce useful results.
GOAL-DIRECTED DESIGN. Sometimes experiments are done just to see what will
happen, to gather observations for an empirical database that can be interpreted in the
future. Often, however, experiments are designed to accomplish a goal. The next five
subsections (with *s) examine some ways in which the pursuit of scientific goals can
motivate and guide the design of experiments
empirical knowledge of a domain, they may find a gap in system knowledge that reveals
an opportunity for learning. Thus, when scientists design an experiment they can be
mainly interested in learning about either a theory or an experimental system.
For either type of goal, interpretive logic is available. For a particular experimental
system, if scientists assume they know the system-theory, they can make inferences
(either hypothetico-deductive or retroductive) about a domain-theory. But if they assume
the domain-theory is known, their inferences are about a system-theory.
This principle, that inference can involve a domain-theory or system-theory, is useful
for designing experiments with different goals. For example, scientists may assume they
know a domain-theory about one property of a chemical system, and based on this
knowledge they design a series of experiments for the purpose of developing system-
theories that characterize this property for a series of chemical systems. But the goal
changes when scientists use a familiar chemical system and assume they have an accurate
system-theory (about a number of chemical properties that are well characterized due to
the application of existing domain-theories) in order to design an experiment that will let
them develop a new domain-theory about another chemical property.
Often, however, both types of knowledge increase during experimentation. Consider a
situation where scientists assume a domain-theory about physiology, and use this theory
to design a series of experiments with different species, in order to learn more about each
species. While they are learning about these systems, they may also learn about the
domain-theory: perhaps it needs to be revised for some species or for all species; or they
may persuade themselves about the truth of a claim (that the same theory can be
generalized to fit all the species being studied) that previously had been only an
assumption.
Sometimes, in the early stages of developing a theory in an underexplored domain,
scientists can assume neither a system-theory nor a domain-theory; their knowledge gap
is both empirical and theoretical, with very little data about systems, and no satisfactory
theory. An example of dually inadequate knowledge occurred in the early 1800s when
atomic theory was being developed, and chemists were also uncertain about the nature of
their experimental systems, such as whether in the electrolysis experiment of "water -->
hydrogen + oxygen" the hydrogen was H or HH, the oxygen was O or OO, and the water
was HO or HOO or HHO.
When an area opens up due to any of these changes, for awhile the possibilities for
research are numerous. To creatively take advantage of a temporary window of
opportunity, an open-minded awareness (to perceive the possibilities) and speed (to
pursue possibilities before they vanish due to the work of others) are often essential. For
example, Humphrey Davy used the newly developed technique of electrolysis to discover
7 elements in 1807 and 1808. Of course, in science (as in the rest of life) it helps to be
lucky, to be in the right place at the right time, but to take advantage of opportunity a
person must be prepared. As Louis Pasteur was fond of saying, "Chance favors the
prepared mind." Many other scientists were working in the early 1800s, yet it was Davy
who had the most success in using the new technique for discovery.
by using deductive logic, but there are essential differences in objectives. During
experimental design the divergent objectives -- looking for outcomes that might be
interesting or useful -- are less clearly defined than in retroduction where, despite a
divergent search for theories, the convergent goal is to find a model whose predictions
match the known observations. And in hypothetico-deduction, mental experiments are
even more constrained, being done with only one theory and one system.
In addition, thought-experiments can be used for deductive exploration, by using a
theory to imagine what would happen in an exotic difficult-to-attain system. In this
context there are no physical constraints, so the only limits are those imposed by the
imagination. And the only cost is the time invested in designing and running the mental
experiments.
Table of Contents
useful to define a project as either "a problem plus a decision to pursue a solution," or as
"a problem and a plan for solving it, plus a decision to pursue this plan of action."
CONCLUSION. The central step in action evaluation -- comparing the current now-
state with the goal-state -- can be viewed as an evaluation of potential solutions. As the
project continues, usually the now-state becomes increasingly similar to the goal-state.
Eventually the now-state may be evaluated as satisfactorily similar, based on criteria
defined by the problem constraints, and the problem is solved. Or at some point there
may be a decision to abandon the project, at least temporarily, because progress toward a
solution is slow, or because despite satisfactory progress the research group decides that
working on another project is likely to be even more productive.
3Ps and 4Ps. A 3Ps model of science (Peterson & Jungck, 1988) interprets scientific
problem solving in terms of posing, probing and persuasion. A brief summary of the 3Ps
41
is that scientists pose a problem, then probe the problem in an effort to solve it, and try to
persuade themselves and others that their solution is satisfactory. This simple model,
which portrays the overall flow of research, was initially proposed for the main purpose
of influencing science education. In this role it has stimulated a great deal of productive
thinking about science and science education, thereby attracting many enthusiastic
advocates, including myself. When discussing the actions that occur during a project, it
is convenient to use a "4Ps" terminology (the original 3Ps, plus one I've added) that, in
addition to being compact, is intrinsically clear because the common meaning for each
term is the intended meaning in ISM. The 4Ps are preparing (reading,...), posing
(formulating a problem), probing (doing actions to probe the problem and pursue a
solution), and persuading.
A group can increase the effectiveness of its actions by coordinating its work on all of
the sub-problems that contribute to the solution of a larger problem. And if a group is
working on several problems simultaneously, an action may help to solve more than one
problem. Or projects can be related sequentially; work on a current project may inspire
ideas for a future project, and at the same time the current project is using results from an
earlier project while these results are being written up for publication. A family of
related projects, simultaneous or sequential, can be produced by developing variations on
a research theme.
Some of the most important interactions involve knowledge. During a current project,
scientists can search backward for what they have learned (about observations and/or
interpretations) from their past work, or they can look forward to potential future uses for
what is being learned now, or sideways for possibilities of sharing knowledge among
concurrent research projects. Learning that occurs during research will help the group
that does the research, in their current and future projects. And if a group's work is
published or is shared informally among colleagues, their experience can help other
scientists learn.
Table of Contents
8. Thought Styles
An Overview of Scientific Method, Section 8
This section describes what thought styles are, and how they affect the process and
content of science.
often unconsciously assumed as "the way things are done" rather than being explicitly
stated. But these ideas exist nevertheless, and they affect the process and content of
science, producing effects that span a wide range from the artistic taste that defines a
theory's "elegance" to the hard-nosed pragmatism of deciding whether a project to
develop a theory or explore a domain is worth the resources it would require.
A thought style will influence (and when viewed from another perspective, is
comprised by) the problem-posing and problem-solving strategies of individuals and
groups. There may be a preference for projects with comprehensive "know every step in
advance" preliminary planning, or casual "steer as you go" improvisational serendipity.
One procedural decision is to ask "Who will do what during research?" Although it is
possible for one scientist to do all the activities in ISM, this is not necessary because
within a research group the efforts of individual scientists, each working on a different
part of the problem, can be cooperatively coordinated. Similarly, in a field as a whole,
each group can work on a different part of a mega-problem. With a "division of labor,"
individuals or groups can specialize in certain types of activities. One division is
between experimentalists who generate observations, and theorists who focus on
interpretation. But most scientists do some of both, with the balance depending on the
requirements of a particular research project and on the abilities and preferences of
colleagues.
There will be mutual influences between thought styles and the procedural "rules of
the game" developed by a community of scientists to establish and maintain certain types
of institutions and reward systems, and procedures for deciding which people, topics, and
viewpoints are presented in conferences and are published in journals. A thought style
will affect attitudes toward competition and cooperation and how to combine them
effectively, and (at a community level) the ways in which activities of different scientists
and groups are coordinated. The logical and aesthetic tastes of a community will affect
the characteristics of written and oral presentations, such as the blending of modes
(verbal, visual, mathematical,...), the degree of simplification, and the balance between
abstractions and concrete illustrations or analogies.
A thought style will tend to favor the production of certain types of observation-and-
interpretation knowledge rather than other types. Effects on observation could include,
for example, a preference for either controlled experiments or field studies, and data
collection that is qualitative or quantitative. There will also be expectations for the
connections between experimenting and theorizing.
An intellectual environment will favor the invention, pursuit and acceptance of certain
types of theories. Some of this influence arises from the design of experiments, which
determines what is studied and how, and thus the types of data collected. Another
mechanism for influence is the generation and selection of criteria for theory evaluation.
For example, thought styles can exert a strong influence on conceptual factors, such as
preferences for the types of components used in theories, the optimal balance between
simplicity and completeness, the value of unified wide-scope theories, the relative
importance of plausibility and utility, and the ways in which a theory or project can be
useful in promoting cognition and research. Thought styles will influence, and will be
influenced by, the goals of science, such as whether the main goal of research projects
should be to improve the state of observations or interpretations, whether science should
44
focus on understanding nature or controlling nature, and what should be the relationships
between science, technology, and society.
The influence exerted by thought styles and cultural-personal factors is a hotly debated
topic, as discussed on the EKS-Rated page.
a PUZZLE and a FILTER. Bauer (1992) compares science to solving a puzzle. In this
metaphor (from Polanyi, 1962) scientists are assembling a jigsaw puzzle of knowledge
about nature, with the semi-finished puzzle in the open for all to see. When one scientist
fits a piece into the puzzle, or modifies a piece already in place, others respond to this
change by thinking about the next step that now becomes possible. The overall result of
these mutual adjustments is that the independent activities of many scientists are
coordinated so they blend together and form a structured cooperative whole.
Bauer supplements this portrait of science with the metaphor of a filter, to describe the
process in which semi-reliable work done by scientists on the frontiers of research, which
Bauer describes in a way reminiscent of the "anything goes" anti-method anarchy of
Feyerabend (1975), is refined into the generally reliable body of knowledge that appears
in textbooks. In science, filtering occurs in a perpetual process of self-correction, as
individual inadequacies and errors are filtered through the sieve of public accountability
by collaborators and colleagues, journal editors and referees, and by the community of
scientists who read journal articles, listen to conference presentations, and evaluate what
they read and hear. During this process it is probable, but not guaranteed, that much of
the effect of biased self-interest by one individual or group will be offset by the actions of
other groups. Due to this filtering, "textbook knowledge" in the classroom is generally
more reliable than "research knowledge" at the frontiers, and the objectivity of science as
a whole is greater than the objectivity of its individual participants. { But a byproduct of
filtering, not directly acknowledged by Bauer, is that the collective evaluations and
dominant thought styles of a scientific community introduce a "community bias" into the
process and content of science. }
THE 4Ps AND THOUGHT STYLES. The puzzle and filter metaphors provide useful
ways to visualize posing and persuading, respectively. While scientists watch what
others are doing with the puzzle of knowledge, they search for gaps to fill, for
opportunities to pose a problem where an investment of their own resources is likely to
be productive. And the process of filtering is useful for describing the overall process of
scientific persuasion, including its institutional procedures.
PREPARATION. There are mutual influences between thought styles and three
45
ways to learn. First, the formal education of students who will become future scientists is
affected by the thought styles of current scientists and educators; in this way, current
science education helps to shape thought styles in the future. Second, thought styles
influence what scientists learn from their own past and current research experience, to use
in future research. Third, thought styles influence the types of ideas that survive the
"filtering" process and are published in journals and textbooks.
POSING. The thought style of a scientific community will affect every aspect of
posing a problem: selecting an area to study, forming perceptions about the current state
of knowledge in this area, and defining a desired goal-state for knowledge in the future.
Problem posing is important within science, and it plays a key role in the mutual
interactions between science and society by influencing both of the main ways that
science affects culture. First, posing affects the investment of societal resources and the
returns (such as medical-technological applications) that may arise from these
investments. Second, the questions asked by science, and the constraints on how these
questions are answered, will help to shape cultural worldviews, concepts, and thinking
patterns.
PROBING. As described above, both types of probing activities -- observation and
interpretation -- are influenced by thought styles.
PERSUASION. For effective persuasion, arguments should be framed in the
structure of current knowledge (so ideas can be more easily understood and appreciated
by readers or listeners), with an acceptable style of presentation, in a way that will be
convincing when judged by the standards of the evaluators, by carefully considering all
factors -- empirical, conceptual, and cultural-personal -- that may influence the evaluation
process at the levels of individuals and communities. Doing all of these things skillfully
requires a good working knowledge of the thought styles in a scientific culture.
VARIATIONS. Thought styles vary from one field of science to another, and so does
their influence on the process and content of science. For example, the methodology of
chemistry emphasizes controlled experiments, while geology and astronomy (or
paleontology,...) depend mainly on observations from field studies. And experiments in
social science and medical science, which typically use a relatively small number of
subjects, must be interpreted using a sophisticated analysis of sampling and statistics, by
contrast with the statistical simplicity of chemistry experiments that involve a huge
number of molecules.
Differences between fields could be caused by a variety of contributing factors,
including: 1) intrinsic differences in the areas of nature being studied; 2) differences in
the observational techniques available for studying each area; 3) differences, due to self
selection, in the cognitive styles, personalities, values, and metaphysical-ideological
beliefs of scientists who choose to enter different fields; 4) historical contingencies.
Variation and change are a part of science, and the study of methodological diversity
and transformation can be fascinating and informative. But these characteristics of
science should be viewed in proper perspective. It is important to balance a recognition
of differences with an understanding of similarities, with an appreciation of the extent to
which differences can be explained as "variations on a theme" -- as variations on the
basic methods shared by all scientists.
Table of Contents
9. Productive Thinking
An Overview of Scientific Method, Section 9
Creative thinking is thinking patterned in a way that leads to creative results. ...
The ultimate criterion for creativity is output. We call a person creative when that
48
person consistently gets creative results, meaning, roughly speaking, original and
otherwise appropriate results by the criteria of the domain in question. (pp. 18-19)
Of course, getting "appropriate results by the criteria of the domain" requires critical
evaluation. This close connection between creativity and criticality is similar to the
connections between generation and evaluation. In fact, it can be useful to consider
generation and evaluation as the result of creative thinking and critical thinking,
respectively. This perspective is adopted in the "red plus blue makes purple" color
coding used in the ISM diagram: generation plus evaluation yields productive thinking in
design. But this interpretation, although interesting, is not logically rigorous, because a
process of generation that is truly productive (to get a high-quality idea, not just an idea)
is usually guided by critical evaluation, even in the initial stages, so equating generation
with pure creativity is not justified. Instead, it's better to consider the entire combination
of "motivation and memory, creativity and critical thinking" that results in productive
thinking with the generation of a theory (or experiment, product, strategy, action,...) that
is evaluated as being useful, and actually is useful.
Considering the close connection between creativity and criticality, perhaps a process
of productive thinking that skillfully combines creative and critical thinking could be
called "creatical" thinking? Well, maybe not. But calling it productive is certainly
appropriate.
The process of inventing useful ideas requires both modes of thinking (creative and
critical) but being overly critical, especially in the early stages of invention, can stifle
creativity. We shouldn't hinder the motion of a car by driving with the brakes on, and we
shouldn't hinder the flow of creativity by thinking with restrictive criticism. But a car
needs brakes, and a creative person needs critical thinking. One strategy for creativity is
to "play games" with the modes by shifting the balance in favor of creativity for awhile,
by experimenting with different balances between the modes during different stages in
the overall process of productive thinking.
For example, instruction designed to enhance creative thinking often uses a technique
of brainstorm and edit. During an initial brainstorming phase, critical restraints are
minimized (this can be done in various ways) to encourage a free creativity in generating
lots of ideas; in a later editing phase these ideas can be critically checked for plausibility
and/or utility. During the brainstorming phase, inventors can afford to think freely (by
consciously trying to see in a new way, to imagine new possibilities without critical
restrictions) because they have the security of knowing that their wild ideas will not be
acted on prematurely before these ideas have been critically evaluated during the editing
phase that follows. The principle of this strategy is to allow the effective operation of
both creativity and criticality.
Human "theories of the world" are essential to our learning and making sense of
the world. However, there is a curious paradox about schemata. Just as they are
the basis of human perception and understanding, so too are they "blinders" to
interpretations that fall outside their scope. ... Creativity involves the ability to go
beyond the schema normally used to approach a problem... and reframe the
problem so it might appear in a different light. Characteristically, the creative
person has the ability to look at a problem from one frame of reference or schema
and then consciously shift to another frame of reference, giving a completely new
perspective. This process continues until the person has viewed the problem from
many different perspectives. (Marzano, et al, 1988, p. 26)
One of the most important actions in science (or in life) is to recognize an opportunity
and take advantage of it, whether this involves observation or interpretation. In science
the imaginative use of available observation detectors — either mechanical or human, for
controlled experiments or planned field studies, for expected or unexpected results — can
be highly effective in converting available information into recorded data. Following
this, an insightful interpretation of observations can harvest more meaning from the raw
data. Sherlock Holmes, with his alert awareness, careful observations, and clever
interpretations, provides a fictional illustration of the benefits arising from an effective
gathering and processing of all available information. Of course, being aware, careful,
and clever are also valuable assets for a real scientist.
ENDNOTES
orbits that were elliptical, not circular. Later, Newton provided a theoretical explanation
for Kepler's elliptical orbits by showing how they can be derived by combining his own
laws of motion and principle of universal gravitation. For a wide range of reasons,
scientists considered these theories -- which postulated noncircular celestial motions -- to
be successful, both empirically and conceptually, so the previous prohibition of
noncircular motions was abandoned. In this case the standard portrait of science was
reversed. Instead of using permanently existing criteria to evaluate proposed theories,
already-accepted theories were used to evaluate and revise the evaluation criteria.
Laudan (1977, 1984) describes a similar situation, with conflict between two beliefs,
but this time the resolving of dissonance resulted in a more significant change, a change
in the fundamental epistemological foundations of science. Some early interpretations of
Newton's methods claimed that he rigidly adhered to building theories by inductive
generalization from observations, and refused to indulge in hypothetical speculation.
Although these claims are disputed by most modern analyses, they were influential in the
early 1700s, and the apparently Newtonian methods were adopted by scientists who tried
to continue Newton's development of empiricist theories (with core components derived
directly from experience), and philosophers developed empiricist theories of knowledge.
But by the 1750s it was becoming apparent that many of the most successful theories, in
a variety of fields, depended on the postulation of unobservable entities. There was a
conflict between these theories of science and the explicitly empiricist goals of science.
Rather than give up their non-empiricist theories, the scientists and philosophers "sought
to legitimate the aim of understanding the visible world by means of postulating an
invisible world whose behavior was causally responsible for what we do observe. ... To
make good on their proposed aims, they had to develop a new methodology of science,...
the hypothetico-deductive method. Such a method allowed for the legitimacy of
hypotheses referring to theoretical entities, just so long as a broad range of correct
observational claims could be derived from such hypotheses. (Laudan, 1984; p. 57)"
what it is:
Integrated Scientific Method (ISM) is a model of scientific action.
It is a synthesis of ideas -- mainly from scientists and philosophers,
but also from sociologists, psychologists, historians, and myself -- that
describes the activities of scientists: what they think about and what they do.
It shows how the mutually supportive skills of creativity and critical thinking
are intimately integrated in the problem-solving methods used by scientists.
inductive.
Most of this page was written in 1997. Later, in April 2006, I said in the "Designing of
Scientific Theories" section of An Introduction to Scientific Method,
In their daily work, scientists rarely design large-scale generalized theories, such as
the theories of gravity, invariance, or evolution developed by Newton, Einstein, or
Darwin. Instead, they typically are applying generalized theories that already are
accepted, in their study of particular systems (and experimental situations) for which they
are designing small-scale specialized mini-theories or sub-theories.
Later, I'll take the time to think about this more rigorously and will incorporate these
concepts into the box above, to describe the relationships between different types of
theories: domain, system, mini, and sub.
Estimates for degrees of agreement and predictive contrast are combined to form an
empirical evaluation of current hypothesis. This evaluation and the analogous
empirical evaluations of previous hypotheses (that are based on the same theory as the
current hypothesis) are empirical factors that influence theory evaluation.
4. Theory Evaluation
A theory is evaluated in association with supplementary theories, and relative to
alternative theories. Inputs for evaluating a theory come from empirical, conceptual,
and cultural-personal factors, with the relative weighting of factors varying from one
situation to another. The immediate output of theory evaluation is a theory status that is
an estimate of a theory's plausibility (whether it seems likely to be true) and/or usefulness
(for stimulating scientific research or solving problems). Based on their estimate of a
theory's status, scientists can decide to retain this theory with no revisions, revise it to
generate a new theory, or reject it. {or delay a decision} When a theory is retained after
evaluation, its status can be increased, decreased, or unchanged. A theory can be retained
for the purpose of pursuit (to serve as a basis for further research) and/or acceptance (as a
proposed explanation, for being treated as if it were true). According to formal logic it is
impossible to prove a theory is either true or false, but scientists have developed
analytical methods that encourage them to claim a "rationally justified confidence" for
their conclusions about status. Each theory has two types of status: its own intrinsic
status, and a relative status that is defined by asking "What is the overall appeal of this
theory compared with alternative theories?"
5. Theory Generation
Generating a theory can involve selecting an old theory or, if necessary, inventing a
new theory. The process of inventing a new theory usually occurs by revising an
existing "old theory." Some strategies for invention are: split an old theory into
components that can be modified or recombined in new ways; borrow components (or
logical structure) from other theories; generalize an old theory, as-is or modified, into a
new domain; or apply the logic of internal consistency to build on the foundation of a
few assumed axiom-components. Often, a creative analysis of data (to search for
patterns) is a key step in constructing a theory.
theory whose predictions will match known observations. If there is data from several
experiments, retroduction can aim for a theory whose predictions are consistent with all
known data. During retroduction a scientist, curious about puzzling observations and
motivated to find an explanation, can adjust either of the two sources used to construct a
model: a general domain-theory (that applies to all systems in a domain) and a specific
system-theory (about the characteristics of one system). Usually, a scientific "inference
to the best explanation" involves a creative use of logic that is both inductive and
deductive.
With retroduction or hypothetico-deduction (which are similar, except that in
retroduction a model is proposed after the observations are known), similar logical
limitations apply. Even if a theory correctly predicts the observations, plausible
alternative theories might make the same correct predictions, so with either retroduction
or hypothetico-deduction there is a cautious conclusion: IF system-and-observations,
THEN MAYBE model (and theory). This caution contrasts with the definite conclusion
of deductive logic: IF theory-and-model, THEN prediction.
experimental systems. When an area of science opens up due to any of these changes,
opportunities for research are produced. To creatively take advantage of these
opportunities requires an open-minded awareness that can imagine a wide variety of
possibilities.
7. Problem-Solving Projects
The activities of science usually occur in a context of problem solving, which can be
defined as "an effort to convert an actual current state into a desired future state" or, more
simply, "converting a NOW-state into a GOAL-state." If the main goal of science is
knowledge about nature, the main goal of scientific research is improved knowledge,
which includes observations of nature and interpretations of nature. Before and during
problem formulation, scientists prepare by learning (through active reading and listening)
the current now-state of knowledge for a selected area, including observations, theories,
and experimental techniques. Critical evaluation of this now-state may lead to
recognizing a gap in the current knowledge, and imagining a potential future state with
improved knowledge. When scientists decide to pursue a solution for a science problem
(characterized by deciding what to study and how to study it) this becomes the focal point
for a problem-solving project.
Problem formulation -- by defining a problem that is original, significant, and can be
solved using available resources -- is an essential activity in science. During research a
mega-problem (the attempt by science to understand all of nature) is narrowed to a
problem (of trying to answer specific questions about one area of nature) and then to sub-
problems and specific actions. In an effort to solve a problem, scientists generate,
evaluate, and execute actions that involve observation (generate and do experiments,
collect data) or interpretation (analyze data, generate and evaluate theories); action
generation and action evaluation, done for the purpose of deciding what to do and
when, is guided by the goal-state (which serves as an aiming point in searching for a
57
8. Thought Styles
All activities in science, mental and physical, are affected by thought styles that are
influenced by cultural-personal factors, operate at the levels of individuals and sub-
communities and communities, and involve both conscious choices and unconscious
assumptions. A collective thought style includes the shared beliefs, among a group of
scientists, about "what should be done and how it should be done." Thought styles affect
the types of theories generated and accepted, and the problems formulated, experiments
done, and techniques for interpreting data. There are mutual influences between thought
styles and the procedural "rules of the game" that are developed by a community of
scientists, operating in a larger social context, to establish and maintain certain types of
institutions and reward systems, styles of presentation, attitudes toward competition and
cooperation, and relationships between science, technology and society. Decisions about
which problem-solving projects to pursue -- decisions (made by scientists and by
societies) that are heavily influenced by thought styles -- play a key role in the two-way
interactions between society and science by determining the allocation of societal
resources (for science as a whole, and for areas within science, and for individual
projects) and the returns (to society) that may arise from investments in scientific
research. Thought styles affect the process and content of science in many ways, but this
influence is not the same for all science, because thought styles vary between fields (and
within fields), and change with time.
9. Mental Operations
The mental operations used in science can be summarized as "motivation and memory,
creativity and critical thinking." Motivation inspires effort. And memory — with
information in the mind or in "external storage" such as notes or a book or a computer
file — provides raw materials (theories, experimental techniques, known observations,...)
for creativity and critical thinking. At its best, productive thinking (in science or in
other areas of life) combines knowledge with creative/critical thinking. Ideally, an
effective productive thinker will have the ability to be fully creative and fully critical, and
will know, based on logic and intuition, what blend of cognitive styles is likely to be
58
color symbolism: The connections between generation (red) and evaluation (blue) are
symbolized by purple — because red plus blue makes purple (with pigments) — in the
"creativity and critical thinking" part of the diagram (which is described in Section 9) as a
59
reminder of the continual productive interplay between creative thinking (the main mode
of thinking in generation) and critical thinking (the main mode of thinking in evaluation).