Educational Research and Evaluation

ISSN: 1380-3611 (Print) 1744-4187 (Online) Journal homepage: http://www.tandfonline.com/loi/nere20

Acquisition of Complex Systemic Thinking: Mental

Models of Evolution
Sylvia T. d'Apollonia , Elizabeth S. Charles & Gary M. Boyd
To cite this article: Sylvia T. d'Apollonia , Elizabeth S. Charles & Gary M. Boyd (2004) Acquisition
of Complex Systemic Thinking: Mental Models of Evolution, Educational Research and
Evaluation, 10:4-6, 499-521, DOI: 10.1080/13803610512331383539
To link to this article: http://dx.doi.org/10.1080/13803610512331383539

Published online: 09 Aug 2010.

Submit your article to this journal

Article views: 67

View related articles

Citing articles: 8 View citing articles

Full Terms & Conditions of access and use can be found at

http://www.tandfonline.com/action/journalInformation?journalCode=nere20
Download by: [Bibliothques de l'Universit de Montral]

Date: 26 October 2016, At: 08:57

Educational Research and Evaluation

2004, Vol. 10, Nos. 46, pp. 499521

Acquisition of Complex Systemic Thinking:

Mental Models of Evolution?
Sylvia T. dApollonia1 , Elizabeth S. Charles2 , and Gary M. Boyd3
1

Concordia University and Dawson College, Montreal, Que., Canada, 2Georgia Institute
of Technology, College of Computing, and 3Concordia University, Montreal,
Que., Canada

ABSTRACT
We investigated the impact of introducing college students to complex adaptive systems
on their subsequent mental models of evolution compared to those of students taught in
the same manner but with no reference to complex systems. The students mental models
(derived from similarity ratings of 12 evolutionary terms using the pathfinder algorithm) were
significantly similar to their teachers mental models and were correlated to their performance on an essay on evolution. Introducing students to complex systems facilitated their
understanding of the mechanism of inheritance, the mechanism of evolution, and the role of
chance in evolution.

MISCONCEPTIONS ABOUT EVOLUTION

Researchers in science education have shown that, despite instruction,
postsecondary students retain deep-rooted misconceptions in several scientific
disciplines (Driver & Easley, 1978; Gardner, 1986; Griffiths & Grant, 1985;
Hackling & Garnet, 1986; Halhoun & Hestenes, 1985). Although Darwin and
Wallaces theory of evolution by natural selection is a central theme within the
discipline of Biology (American Association for the Advancement of Science,

? A draft of this paper was presented at the Annual Meeting of the American Educational
Research Association, New Orleans, LA, April 2002.
This research was supported in part by Social Sciences and Humanities Research Council of
Canada and Programme daide a` la recherche sur lenseignement et lapprentissage.
Address correspondence to: Sylvia T. dApollonia, Concordia University and Dawson College,
3040 Sherbrooke Street W., Montreal, Que., Canada H3Z 1A4. E-mail: sdapollonia@place.
dawsoncollege.qc.ca
10.1080/13803610512331383539$16.00 # Taylor & Francis Ltd.

500

SYLVIA T. DAPOLLONIA ET AL.

2000), many studies have shown that college and university students have
persistent misconceptions about its characteristics (Brumby, 1984; DemastesSoutherland, Good, & Peebles, 1995).
Neo-Darwinism or the modern synthesis (of genetics and evolutionary
biology) is traditionally taught in terms of four principles (Ohlsson & Blee,
1992). Firstly, there is pre-existing genetic variability in the population which
has been brought about by random mutations in individual genotypes
(producing a qualitative change in an inheritable characteristic) and by the
variation resulting from sexual reproduction. Secondly, environmental
pressures lead to differential survival/reproduction of individuals. Thirdly,
the combination of random mutations and a selection mechanism leads to
changes in the population gene pool of successive generations (changes in
allelic frequencies). Fourthly, over long periods of time, there is a change in
the characteristics of the population such that it has become more adapted to
the environment (more individuals will have the adaptive trait).
In traditional instruction, each postulant is explicitly taught and tested. In
some cases, the role of these postulants in an evolutionary case study such as
the evolution of DDT resistant mosquitoes or changes in the proportions of
dark and light peppered moths is specifically described. In addition, some
biologists also teach that non-adaptive mechanisms such as genetic drift
influence evolution (Gould, 1980).
Nevertheless, researchers have shown that students have persistent difficulties in connecting what they know of genetics to evolution (DemastesSoutherland, Good, Sundberg, & Dini, 1992; Hallden, 1988; Jensen & Finley,
1994). Bishop and Anderson (1990) have shown that students have difficulties attributing the origin of new traits to chance and their survival to natural
selection. That is, they have difficulties distinguishing qualitative changes to
genotypes from quantitative changes in the proportion of individuals that carry
the trait (Bishop & Anderson, 1990). That is, they have difficulties distinguishing what happens at the individual level from what happens at the
population level (Hallden, 1988).1
Researchers have also suggested that non-domain-specific factors, such
as religious and cultural beliefs (Matsumura, 1998), difficulties with the
scientific meaning of common terms (e.g., evolutionary adaptation) (ZaimIdrissi, Desautels, & Larochelle, 1993), and inadequate textbooks (American

This was resolved through mathematical modeling by Godfrey Hardy, an English mathematician,
and Wilhelm Weinberg, a German physician.

ACQUISITION OF COMPLEX SYSTEMIC THINKING

501

Association for the Advancement of Science, 2000; McComas, 1997), may be

a factor in students persistent misconceptions.
This research is based on several theoretical perspectives, described below.
It is based on the assumptions that students use mental models or explanatory
frameworks to organize knowledge and enhance its retrieval during problemsolving tasks. When these mental models are fragmented and/or inadequate,
understanding is inhibited. Thus, one approach to enhancing students understanding of evolution is to help them acquire the appropriate conceptual
framework as they construct their understanding of evolution. Thus, this research is based on ideas from the study of Complex Systems and from the
misconception literature, specifically Michelene Chis (1993) work.
COMPLEX SYSTEMS THINKING
Many researchers (Auyang, 1998; Bar-Yam, 1997; Kaput, Bar-Yam, & Jacobson,
1999) have argued that the study of complex systems may provide learners with
a unifying and cross-disciplinary framework that allows students to construct a
betterunderstandingofmanysciencephenomena.Complexsystemsarehierarchically organized collections of interacting components (called agents) operating
under specified rules and resulting in the emergence of higher level associations
which behave differently from what one could predict from knowledge of the
agents (see also, Charles & dApollonia, 2003). For example, there are at least
three levels (or associations) to contend with in understanding evolution: genes,
individuals, and populations. Genes mutate, individuals survive and reproduce,
and populations evolve. Students have difficulty in understanding how these
three different levels interact and how a random event (mutation) at the gene
level emerges as an evolutionary pattern at the population level.
Several researchers (Jacobson, 2000; Resnick, 1994, 1996; Resnick &
Wilensky, 1997; Wilensky, 1999) have suggested that many of the difficulties
students face in understanding such topics as ecology, evolution, predatorprey interactions may reflect confusion with such complex system properties
as emergent levels, self organization, and the role of chance on complex
systems. Jacobson and his colleagues (Jacobson & Archodidou, 2000;
Jacobson, Sugimoto, & Archididou, 1996) explored the relationship between
students complex systems thinking and their conceptual understanding of
evolution. They identified four knowledge components2 (origin of new traits,
2

These correspond to the basic principles of evolution.

502

SYLVIA T. DAPOLLONIA ET AL.

mechanism of inheritance, mechanism of evolution, and final causes). For

each knowledge component, they subsequently defined two stages in the
learners understanding (based on reductive clock-work mental models or
based on emergent mental models). They were thus able to describe eight
stages in learners mental models of evolution from Nave to Neo-Darwinian
based on their appropriate use of the knowledge components in their
explanations of evolution. Although the results from Jacobsons study
(Jacobson & Archodidou, 2000) were obtained from a small sample, they
indicate that novice learners use reductive thinking in explaining evolution
while experts use emergent thinking.
Thus, in this study, we introduced students in the experimental group to
complex systems thinking as an organizing framework for the study of
biology. We subsequently taught the aforementioned principles of evolution to
both the experimental group and the control group. We hypothesize that,
without prompting, students who have been exposed to complex systems
thinking will subsequently construct better mental models of evolution than
students taught biology along traditional lines and will therefore use emergent
thinking in solving problems in evolution.

CHIS CONCEPTUAL CHANGE THEORY

Several researchers (Chi, 1993; Chi, Slotta, & deLeeuw, 1994; Ferrari & Chi,
1998; Slotta & Chi, 1999; Vosniadou, 1994) have proposed that learners use
mental models or explanatory frameworks (general concepts used to interpret
and explain the world) to make sense of many scientific phenomena. The
basic assumption of their theory is that learners relate all new conceptions to
ontological categories on the basis of the language used to describe the conceptions. These schema-like associations act as facilitators or inhibitors of
future knowledge retrieval and problem solving. Chi and her coworkers (Chi
et al., 1994; Ferrari & Chi, 1998; Slotta & Chi, 1999) argue that learners use
explanatory frameworks to organize and express their ontological commitments. When learners categorize new conceptions to inappropriate ontological
categories, they cannot subsequently use it.
One such general ontological category is the category of processes.
Within this category there are both event processes and emergent
processes. Chi and her colleagues hypothesize that most misconceptions
occur when concepts such as evolution are assigned to the event process

ACQUISITION OF COMPLEX SYSTEMIC THINKING

503

rather than to the emergent process category. Slotta and Chi (1999) argue
that once a concept has been assigned to the wrong category, it is very difficult
to change. Reassignment may entail the learning of a new explanatory
framework. This may require learning new terminology, acquiring new mental
models, and developing different attitudes and values.
Representational Models of Learning and Memory
Traditional constructivist theorists (Ausubel, 1963, 1968; Novak, 1988;
Piaget, 1954) hold a representational model of memory in which domainspecific declarative knowledge is stored as a network in long-term memory
(e.g., Anderson, 1983). These network models assume that concepts are stored
as nodes that are interconnected to form a vast associative network. Most
researchers believe that the nodes are organized in a hierarchical manner such
that more general concepts are superordinate and more specific concepts are
subordinate. In associative models, the links are unlabelled and therefore the
same. Propositional network models (Anderson, 1983; Anderson & Bower,
1973) hold that propositions (a combination of concepts having both a subject
and a predicate that have a truth value, e.g., water is a molecule) rather than
unitary concepts (e.g., water) form the nodes. The links are labelled and
therefore are not identical.
However, understanding science involves not only knowing what
(declarative knowledge) but also knowing how (procedural knowledge).
Therefore, representational models propose that procedural rules on how to
manipulate the nodes are also stored (Anderson, 1983). Procedural rules for
general situations are stored as schemata (Rumelhart, 1980) while procedural
rules for specific subject-matter domains are stored as mental models
(Johnson-Laird, 1983; cf. Gentner & Stevens, 1983). Thus, mental models are
analogies of the events or situations along with procedural rules to mentally
manipulate the event or situation. Individuals use these models to predict
future events, answer comprehension questions, or solve problems. Initially,
individuals construct conceptual structures that include only declarative
knowledge; however, under appropriate conditions, learners downplay these
semantic features and construct mental models of the situation, by encoding procedures, goals, and relationships (McNamara, Miller, & Bransford,
1991). Whether a learner opts to encode text propositionally or construct a
mental model appears to be a function of text features, task difficulty,
expertise, and knowledge of subsequent testing procedures (McNamara et al.,
1991).

504

SYLVIA T. DAPOLLONIA ET AL.

Assessment of Mental Models and Associated Conceptual Structures

Methods of inferring the way in which people organize domain-specific
information are very diverse (Adelson, 1981; Egan & Schwartz, 1979;
Murphy & Wright, 1984; Shavelson, Ruiz-Primo, & Wiley, in press;
Shoenfeld & Herrmann, 1982). However, all are introspective, beginning
with individuals making metacognitive judgements of what they know. These
methods can be categorized as verbal reports, clustering methodologies, and
scaling techniques (Koubek & Mountjoy, 1991). Each method involves three
stages: collecting the declarative knowledge that has been acquired,
generating a representation of the conceptual structure, and quantifying the
degree of conceptual organization.
Concepts and relationships can be elicited by essays, concept maps, ordered
lists, and pair-wise similarity ratings such as described in Pathfinder
(Schaneveldt, 1990). In this study, we elicited students concepts about
evolution by both asking them to write an essay and by asking them to rate the
degree of relationship between 12 pairs of evolutionary terms.
Cognitive psychologists have developed several methods of generating
representations of the way in which declarative knowledge is structured
(Olson & Biolsi, 1991). Firstly, the subjects verbal protocols can be parsed
into propositions and the knowledge structure represented as a network of
labeled interconnected nodes. (dApollonia, De Simone, Dedic, Rosenfield,
& Glashan, 1993; Frederiksen & Breaueux, 1990; Mosenthal & Kirsch,
1992). Secondly, written or spoken text can be converted into concept maps.
For example, Novak and Musonada (1991) interviewed students on
their understanding of chemistry, converted their protocols into concept
maps, and subsequently assessed the maps. Thirdly, subjects performances
on multiple choice tests can be coded into historical or developmental stages
in the theoretical development of the domain (Jacobson & Archodidou,
2000; Vosniadou & Brewer, 1987). Finally, concept maps, similarity ratings,
and other such data can be transformed into general weighted networks such
as Pfnets by Pathfinder (Schaneveldt, 1990) or by multidimensional scaling.
The Pathfinder algorithm generates networks in which the links may be
either directed or non-directed. The Pathfinder algorithm also generates
several measures of coherence and network similarity: the coherence or
consistency of the generated network, the similarity between two networks, a
test for the probability that the number of links in common between two
links could arise by chance, and the information that two networks have
in common.

ACQUISITION OF COMPLEX SYSTEMIC THINKING

505

Thus, several methods of generating and analyzing conceptual structures

have been developed. They are all based on representational models of
memory. They all maintain that conceptual structures are symbolic internal
representations of external reality stored in long-term memory. They are based
on strong assumptions that conceptual structures are relatively stable (once
learned) and meaningful (have semantic properties), and can be inferred from
an individuals overt behaviour.
GOALS OF REPORTED STUDY
The basic objectives of our research described in this article are threefold:
1. To demonstrate that rating scales capture experts mental representations of
evolution and that these measures have face validity.
2. To demonstrate that teaching students about complex adaptive systems
influences their mental representations of evolution.
3. To demonstrate that students who were first taught about complex adaptive
systems would subsequently have a better understanding of evolution.
RESEARCH DESIGN AND METHODOLOGY
Participants and Intervention
One class of 37 college students enrolled in a 1st-year biology course was
taught complex adaptive systems during the 1st week of the course. We
emphasized the hierarchical nature of biological systems, the emergent
properties of life, and the role of chance and probability. The concepts were
reinforced periodically by asking students to relate a biology topic (e.g., cell
structure, cell division, genetics) to complex adaptive systems (see Appendix A
for summary of instruction and selected students assignments). Six weeks
later they were taught principles of evolution using a traditional pedagogy.
The same teacher taught principles of evolution using only the traditional
pedagogy to another group of 36 students. Thus, the only difference in
instruction between the two groups was that students in the experimental
group were taught about complex systems and encouraged to develop this as
an explanatory framework for biological processes. Students subsequently
reviewed evolution in light of genetics and were asked to complete two tasks.
They were asked to rate the degree of relatedness among all pairs of 12

506

SYLVIA T. DAPOLLONIA ET AL.

evolutionary terms (adaptive (natural) selection, chance, change in allelic

frequency, differential survival, environmental pressure, genetic drift, genetic
variation, genotype, individual, mutation, non-adaptive selection, population).
We selected these terms on the basis of the major headings in the students text
book (Campbell, Reece, Mitchell, & Taylor, 2003) and the results of prior
pilot tests of the rating scale. They were subsequently asked to write an inclass essay in which they explained how whales may have evolved 60 million
years ago from the ancestors of ungulates (sheep, cows, etc.), a previously
unseen evolutionary case study.3
Deriving Conceptual Structures From Similarity Ratings
We used the Pathfinder software, MacKnot (Interlinks) to (a) generate Pfnets
for each student. These Pfnets represent the students overall structural
organization of evolutionary concepts; (b) generate graphs depicting the
multidimensional distances among the terms; and (c) generate composite4
Pfnets for four domain experts, 30 students in the experimental class, and 20
students in the control class. See Schaneveldt (1990) for a discussion of the
mathematics that underlies the Pathfinder algorithms.
Deriving Conceptual Structures From Essays
The students essays were segmented into propositions. The investigators
prepared a template that was used to classify the idea units onto the 12
evolutionary terms. For example, when students indicated that some
organisms would survive, while others would not, we substituted differential
survival. Each proposition was subsequently translated into propositions
containing the 12 evolutionary terms.
We subsequently coded each proposition (if relevant) on the basis of the
students understanding of evolutionary concepts using a coding schema
(presented in Table 1), modified from Jacobson and Archodidou (2000). We
coded the frequency of statements a student made, at each level (novice (!1)
or expert (1)), for each Knowledge Component. The students understanding
of evolutionary concepts is the sum of the four frequencies.
We subsequently coded each statement (if relevant) on the basis of the
students understanding of the emergent nature of evolutionary processes.
3

See (http://www.nature.com/nsu/010920/010920-11.html).
The MacKnot software includes a program which computes the average proximity matrix
across several proximity matrices and thus generates a mental representation of the groups
understanding of evolution.
4

507

ACQUISITION OF COMPLEX SYSTEMIC THINKING

Table 1. Coding Schema Used to Score Students Understanding of Knowledge Components

of Evolution.
Knowledge component

Levels of understanding and mental model

(E is expert, N is novice)

Origin of New Traits

Prior mutations
Environmental pressure

E
N

Mechanism of
Inheritance

Transmission of genes from parent to offspring

Confusion (acquired traits, or other transmission)

E
N

Mechanism of Evolution

Differential survival ! change in allelic frequencies E

Gradual developmental of individuals
N

Role of Chance in
Origin of New Traits

Probabilistic
Teleological/deterministic

E
N

Table 2. Coding Schema for Students Understanding of the Emergent Nature of Evolutionary
Processes (E is expert, N is novice).
Processes

Genetic Transmission
Mutation
Differential Survival
Selection
Environment
Change in gene pool
Genetic variability
Evolution

Hierarchical levels
Ind

Pop

Processes

Genetic Transmission
Mutation
Genetic Drift

Differential Survival
Natural Selection

Final Cause
Prob

Dir.

That is, we coded the frequency of statements a student made, (novice (!1) or
expert (1)), on the basis of the hierarchical level to which students situated
evolutionary processes and on the role of chance (probabilistic causes) versus
direction (teleological causes) (see Table 2). The students understanding of
the emergent nature of evolutionary processes is the sum of the two
frequencies. Repeated statements were counted more than once, only if the
contexts were different. For example, if the probabilistic nature of
evolutionary processes was described for mutations and for genetic drift it

508

SYLVIA T. DAPOLLONIA ET AL.

was counted twice. However, if the probabilistic nature of evolutionary

processes was described twice for mutations, it was counted once.
The inter-rater reliability (%agreement) over 20% of the essays for each
category was the following: Origin of New Traits (100%), Mechanism of
Inheritance (100%), Mechanism of Evolution (75%), Role of Chance (75%),
Emergent Levels (75%), Final Cause (75%). In all cases, the differences were
in the strength of the coded relationship, not the direction. That is, students
sometimes repeated a statement and coders sometimes coded it twice rather
than once. See Appendix B, for the data generated by one student in the
experimental group.

RESULTS AND DISCUSSION

The results of this research are organized around three questions: Do rating
scales (as analyzed by the Pathfinder algorithm) capture mental models?
Are these measures sensitive to different instruction? Does exposure to the
explanatory framework of complex systems facilitate the acquisition of
appropriate mental models of evolution and enhance the understanding of
evolution?
Do Rating Scales Capture Experts Mental Representations
of Evolution?
The collective experts mental model of evolution derived from the composite
Pfnets is presented in Figure 1. It clearly indicates that the domain experts have
neo-Darwinian mental models on the Origin of New Traits (mutations
affecting genotype of individuals), on the Mechanism of Evolution
(environmental pressure on individual related to differential survival resulting
in natural selection changing the population gene pool), and on Role of Chance
(random forces affect both mutation and changes in gene pool). Unfortunately,
the selected terms do not capture the Mechanisms of Inheritance. In future
studies, we will add terms to capture this knowledge component.
The experts composite Pfnet was analyzed using multidimensional
scaling. The 12 terms were clearly separated into four clusters on the basis
of two dimensions: levels (population/individual) and role of chance
(probabilistic/directional). Thus, it appears that the Pfnets determined from
rating scales do capture the relevant dimensions of evolution. Thus, rating
scales can be used to capture the mental models of evolution.

ACQUISITION OF COMPLEX SYSTEMIC THINKING

509

Fig. 1. Composite Pfnets from four experts.

Does Teaching Students About Complex Systems Influence

Their Subsequent Mental Representations of Evolution?
Figures 2a and 2b shows the composite mental models of 30 students in the
experimental group and 20 students in the control group derived from the
composite Pfnets.
The two mental models derived from the composite Pfnets are surprisingly
similar. They differ in only 6 relationships. The two models differ in the role of
chance, with the students in the experimental class putting more weight on its
role in genetic drift and non-adaptive selection; while the students in the
control class put more weight on its role in mutation. The two models also
differ on the role of differential survival, natural selection, and genetic
variability. The students in the experimental class may view differential
survival affecting genetic variation through the mediation of natural selection;
while the students in the control class may view genetic variation causing
differential survival resulting in natural selection. These are tentative
speculations on how the two models may differ.
What is striking is the degree to which they are alike. Both classes appear to
have taken the same constructs from their teachers explanation. This may
reflect the powerful impact that a teachers mental model has on student
learning. Both groups of students view the impact of the environmental

510

SYLVIA T. DAPOLLONIA ET AL.

Fig. 2a. Composite Pfnet from students taught Complex Adaptive Systems. The dotted lines are
the links missing relative to the experts Pfnets. The bold dashed lines are the links
added by the students.

Fig. 2b. Composite Pfnet from control students. The dotted lines are the links missing relative
to the experts Pfnets. The bold dashed lines are the links added by the students.

511

ACQUISITION OF COMPLEX SYSTEMIC THINKING

Table 3. Characteristics of Pfnets From Experts and Students.

Group

Experts
Students in control group
Students in experimental group

Sample size

4
20
30

Coherence

Number of links

Mean

SD

Mean

SD

.38
.19
.32

.15
.30
.21

21
26
22

6.7
5.7
7.9

pressure to be at the population level rather than at the individual. Similarly,

both groups of students relate the impact of mutations and genotype on genetic
variability of the population to be direct rather than mediated by individuals.
That is, they do not appear to have the concept of the emergence of population
characteristics from activities at the individual levels.
Table 3 compares characteristics of the mental models of experts, students
introduced to complex adaptive systems before being taught evolution, and
students taught evolution directly. The experts mental models of evolution are
more coherent and have fewer links than did those of students either in the
experimental or control group. The mental models of students introduced to
complex adapted systems appeared to be more similar to those of the experts.
However, there was a high degree of variability in individual mental models.
Therefore, we were not able to show that these differences were significant at
an a level of 0.05. However, there is a trend that as expertise develops,
learners begin distinguishing clusters of related terms, rather than considering
all the terms to be related. The students that produced the worse essays on
evolution had mental models in which the link to node ratios were extremely
high.
All the experts mental models were significantly similar (t prob p < .05) to
that of the researcher who taught the experimental and control classes.
Similarly, 63.3% (19/30) of the students in the experimental group had mental
models of evolution significantly similar to that of the teachers; but only 40%
(8/20) of the students in the control class did.
Table 4 shows the degree of similarity (measured as the information content
(!log2 pdue to chance)) between the teachers mental model and those of the students in the control and experimental classes. The students who had been
introduced to complex adaptive systems constructed mental models of
evolution that were significantly (F 6.66, df 48,1, p .01) more similar to

512

SYLVIA T. DAPOLLONIA ET AL.

Table 4. Similarity (Information) Between Teachers Mental Models, and Students.

Treatment

Sample size

Control students
Experimental students

20
30

Similarity
Mean

SD

4.13
6.73

2.86
3.86

the teachers mental model than were the students not introduced to complex
adaptive systems. Thus, the introduction of complex adaptive systems affects
students mental models.
Does Teaching Students Complex Adaptive Systems Facilitate
Their Subsequent Understanding of Evolution?
We first analyzed the students essays in order to determine whether they
picked up the same concepts captured by the derived mental models (i.e.,
their Pfnets). Subsequently, we analyzed the coded essays to determine
whether students who had been introduced to complex adaptive systems have
a better understanding of evolution than do the students not introduced to
complex adaptive systems.
Table 5 shows the correlations between each students mental model and
their scores on the essay. The derived mental models are correlated to the
students level of understanding of three of the four evolutionary concepts
(origin of new traits, mechanism of inheritance, mechanism of evolution) and
one out of two of the emergent properties of evolution (hierarchical levels).
Students who were first introduced to complex systems had a significantly
better (F5 5.02; df 2,47; p .01) understanding of both evolutionary concepts and the emergent properties of evolutionary processes (Means 3.57
Table 5. Correlation Between Students Derived Mental Models (Similarity Index to the
Teachers Mental Model) and Their Scores on the Essays.

Origin of
new traits

Mechanism of
inheritance

Mechanism
of evolution

Role of
chance

Levels

Final
cause

.31$

.32$

.45$

!.04

.31$

.03

Note. $ Significant at the .05 alpha level.

5

Using multivariate tests of significance.

ACQUISITION OF COMPLEX SYSTEMIC THINKING

513

and 3.43, respectively) than did the control students (Means .59 and 1.34,
respectively). When the subcomponents were subsequently analyzed, significant differences were found for the origin of new traits, the mechanism
of inheritance, the mechanism of evolution, and on the emergent levels of
evolution. Thus, the intervention enhanced students understanding of the
mechanisms of evolution but not of the role of chance in these mechanisms.
This may reflect either problems with coding the essays or the difficulties
inherent in understanding the role of chance in the emergence of systems.

GENERAL DISCUSSION AND CONCLUSIONS

Similarity ratings appear to be an effective and valid method of capturing the
knowledge representations (mental models) held by both students and teachers.
The composite experts derived mental models are interpretable, corresponding to a neo-Darwinian model of evolution. When the experts similarity
ratings were analyzed using multidimensional scaling, two dimensions (the
role of chance, and emergent levels) separated the concepts into four clusters.
Students mental models were similar to their teachers mental models. Thus,
the Pfnets captured the underlying organizational structures held by the
teachers. Although the teachers mental models were significantly similar,
there were indications that the newer teachers mental models more closely
resembled the mental models imbedded in textbooks.
The knowledge representations were sensitive to an instructional strategy
(the introduction to the students of complex adaptive systems as an
organization framework in biology). Students who had been introduced to
complex systems had derived mental models more similar to the teachers than
did students not introduced to complex systems. Moreover, their mental
models predicted their performance on a writing task in which they were
asked to explain the evolution of whales.
Thus, we concluded that the collection of similarity ratings of 12
evolutionary terms from students had elicited information on their conceptual
structures and that the Pathfinder technique was an effective and valid method
of representing and analyzing their mental models.
The selection of appropriate terms (both the number and the terms
themselves) is crucial to obtaining reliable and valid mental models. We pilot
tested several sets of terms before deciding on the present set. However, the
knowledge representations lack certain ideas (and misconceptions). For

514

SYLVIA T. DAPOLLONIA ET AL.

example, they do not capture the misconception that many students stated that
ungulates slowly develop traits that adapt them to an aquatic environment.
Thus, in future studies we will add additional terms (to capture these
misconceptions) and ask students to add their own terms.
Introducing students to complex adaptive systems prior to the traditional
teaching of evolution, facilitated their subsequent understanding of evolution.
These students had a more profound understanding of the source of new traits,
on the mechanism of inheritance, and on the mechanism of evolution.
However, they did not acquire a better understanding of the role of chance in
evolution than did the control students. This may reflect the brevity of the
intervention. We only introduced students to complex systems for one lecture
and gave them one reinforcement assignment. In subsequent research we will
increase the amount of time that students engage in interactive activities
(simulations and discussion) that emphasize probabilistic reasoning. We will
also test their understanding both of complex systems and of other phenomena
that are based on complex systems, for example, predator-prey relationships.
Mental models of evolution are like mental maps that students may use
when they attempt to explain a phenomenon. There are several routes that they
can take between many concepts. For example, they can make a direct link
from mutation to genetic variability in a population, or they can link the two
indirectly via the influence of mutations on an individual. Having been
introduced to the ideas of complex adaptive systems may provide an
incentive for them to take the longer and correct route rather than the
shorter and incorrect route. In other words, the idea from complex system that
stable characteristics of a collective could emerge from the random
activities of agents can act as scaffolding for their evolutionary explanations.
Thus, the intervention may have helped students subsequently understand
evolution, without their necessarily understanding fully (or even marginally)
complex systems.
In conclusion, we found that the Pathfinder technique of eliciting students
and teachers conceptual structures, representing them as pfnets, and
analyzing their similarity was an effective method of deriving their mental
models of evolution. The teachers mental models strongly influenced their
students mental models. The similarity of the students mental models to that
of their teacher predicted their performance in writing an explanation of the
evolution of whales. Introducing students to adaptive systems, stimulated their
understanding of the mechanism of inheritance, the mechanism of evolution,
and the role of chance in evolution.

ACQUISITION OF COMPLEX SYSTEMIC THINKING

515

REFERENCES
Adelson, B. (1981). Problem solving and the development of abstract categories in
programming languages. Memory and Cognition, 9(4), 422433.
American Association for the Advancement of Science. (2000). http://www.project2061.org/
research/textbook/hsbio/default.htm
Anderson, J.R. (1983). Problem solving and learning. American Psychologist, 48, 3544.
Anderson, J.R., & Bower, G.H. (1973) Human associative memory. Washington, DC: Winston.
Ausubel, D.P. (1963). The psychology of meaningful verbal learning. New York: Grune &
Stratton.
Ausubel, D.P. (1968). Educational psychology: A cognitive view. New York: Rinehart and
Winston.
Auyang, S. (1998). Foundations of complex-system theories in economics, evolutionary
biology, and statistical physics. Cambridge, UK: Cambridge University Press.
Bar-Yam, Y. (1997). Dynamics of complex systems. Reading, MA: Addison-Wesley.
Bishop, B.A., & Anderson, C.W. (1990). Student conceptions of natural selection and its role in
evolution. Journal of Research in Science Teaching, 27, 415427.
Brumby, M.N. (1984). Misconceptions about the concept of natural selection by medical
biology students. Science Education, 68, 493503.
Campbell, N.A., Reece, J.B., Mitchell, L.G., & Taylor, M.R. (2003). Biology: Concepts and
connections (4th ed.). San Francisco: Benjamin Cummings.
Charles, E.S., & dApollonia, S.T. (2003). A systems approach to science education. Final
Report submitted to Programme dAide a la recherche sur lenseignement et
lapprentissage (PAREA), Gouvernement du Quebec. ISBN 1-55016-169-5.
Chi, M.T.H. (1993). Barriers to conceptual change in learning science concepts: A theoretical
conjecture. In W. Kintsch (Ed.), Proceedings of the Fifteenth Annual Cognitive Science
Society Conference (pp. 312317). Hillsdale, NJ: Erlbaum.
Chi, M.T.H., Slotta, J.D., & deLeeuw, N. (1994). From things to processes: A theory of
conceptual change for learning science concepts. Learning and Instruction, 4, 2743.
dApollonia, S., De Simone, C., Dedic, H., Rosenfield, S., & Glashan, A. (1993). Cooperative
networking: A method of promoting understanding in the sciences. Final Report
submitted to Programme dAide a la rescherche sur lenseignement et lapprentissage
(PAREA), Gouvernement du Quebec. ISBN 0-969728-1-0.
Demastes-Southerland, S., Good, R.G., & Peebles, P. (1995). Students conceptual ecologies
and the process of conceptual change in evolution. Science Education, 79, 637666.
Demastes-Southerland, S., Good, R.G., Sundberg, M., & Dini, M. (1992, March). Students
conceptions of natural selection and its role in evolution: A replication study and more.
Presented at the Annual Meeting of the National Association for Research in Science
Teaching, Boston, MA.
Driver, R., & Easley, J. (1978). Pupils and paradigms: A review of literature related to
concept development in adolescent science students. Studies in Science Education, 5,
6184.
Egan, D.E., & Schwartz, B.J. (1979). Chunking in recall of symbolic drawings. Memory and
Cognition, 7, 149158.
Ferrari, M., & Chi, M.T.H. (1998). The nature of nave explanations of natural selection.
International Journal of Science Education, 20, 12311256.

516

SYLVIA T. DAPOLLONIA ET AL.

Frederiksen, C.H., & Breaueux, A. (1990). Applying cognitive task analysis and research
methods to assessment. In N. Frederiksen, R. Glaser, A. Lesgold, & M.G. Shafto (Eds.),
Diagnostic monitoring of skill and knowledge acquisition (pp. 143165). Hillsdale, NJ:
Lawrence Erlbaum.
Gardner, P.L. (1986). Physics students comprehension of motion with constant velocity.
The Australian Science Teachers Journal, 31, 2732.
Gentner, D., & Stevens, A. (Eds.). (1983). Mental models. Hillsdale, NJ: Erlbaum.
Gould, S.J. (1980). Is a new and general theory of evolution emerging? Paleobiology, 6, 119130.
Griffiths, A.K., & Grant, B.A.C. (1985). High school students understanding of food webs:
Identification of a learning hierarchy and related misconceptions. Journal of Research in
Science Teaching, 22, 421436.
Hackling, M.W., & Garnet, P.G. (1986). Chemical equilibrium: Learning difficulties and
teaching strategies. The Australian Science Teachers Journal, 31, 813.
Halhoun, I.A., & Hestenes, D. (1985). The initial knowledge state of college physics students.
American Journal of Physics, 53, 10561065.
Hallden, O. (1988). The evolution of the species: Pupil perspectives and school perspectives.
International Journal of Science Education, 10, 541552.
Jacobson, M.J. (2000, April). Butterflies, traffic jams, & cheetahs: Problem solving and
complex systems. Paper presented at the Annual Meeting of the American Educational
Research Association, Atlanta.
Jacobson, M.J., & Archodidou, A. (2000). The design of hypermedia tools for learning:
Fostering conceptual change and transfer of complex scientific knowledge. The Journal
of the Learning Sciences, 9(2), 149199.
Jacobson, M.J., Sugimoto, A., & Archididou, A. (1996). Evolution, hypermedia learning
environments, and conceptual change: A preliminary report. In D.C. Edelson & E.A.
Domeshek (Eds.), International Conference on the Learning Sciences, 1996: Proceedings of ICLS 96 (pp. 151158). Charlottesville, VA: Association for the Advancement of
Computing in Education.
Jensen, M.S., & Finley, F.N. (1994). Changes in students understanding of evolution resulting
from different curricular and instructional strategies. Paper presented at the Annual
Meeting of the National Association of Research in Science Teaching, Anheim, CA.
Johnson-Laird, P.N. (1983). Mental models. Cambridge, MA: Harvard University Press.
Kaput, J., Bar-Yam, Y., & Jacobson, M. (1999). Planning documents for a national initiative on
complex systems in K-16 education (Report to the National Science Foundation).
Cambridge, MA: New England Complex Systems Institute. Online source: http://
necsci.org/events/cxed16.html
Koubek, R.J., & Mountjoy, D.N. (1991). Toward a model of knowledge structure and a
comparative analysis of knowledge structure measurement techniques. ERIC_NO:
ED339719
Matsumura, M. (1998). Is it fair to teach evolution? Reports of the National Center for Science
Education, 18, 1921.
McComas, W.F. (1997). The discovery and nature of evolution by natural selection:
Misconceptions and lessons from the history of science. American Biology Teacher,
59, 492500.
McNamara, T.P., Miller, D.L., & Bransford, J.D. (1991). Mental models and reading
comprehension. In R. Barr, M. Kamil, P. Mosenthal, & P.D. Pearson (Eds.), Handbook of
research in reading (pp. 490511). New York, NY: Longman.

ACQUISITION OF COMPLEX SYSTEMIC THINKING

517

Mosenthal, P.B., & Kirsch, I.S. (1992). Learning from exposition: Understanding knowledge
acquisition from a knowledge model perspective. Journal of Reading, 35, 356370.
Murphy, G.L., & Wright, J.C. (1984). Changes in conceptual structure with expertise:
Differences between real-world experts and novices. Journal of Experimental
Psychology: Learning, Memory, and Cognition, 10, 144155.
Novak, J.D. (1988). Learning science and the science of learning. Studies in Science Education,
15, 77101.
Novak, J.D., & Musonada, D. (1991). A twelve-year longitudinal study of science concept
learning. American Educational Research Journal, 28, 117153.
Ohisson, S., & Blee, N.V. (1992). The effect of expository text on childrens explanations of
biological evolution. OERI Report. Santa Cruz, CA: Learning and Development Center.
Olson, J.R., & Biolsi, K.J. (1991). Technique for representing expert knowledge. In K.A.
Ericsson & J. Smith (Eds.), Toward a general theory of expertise (pp. 240285).
Cambridge: Cambridge University Press.
Piaget, J. (1954). The construction of reality in the child. New York: Basic Books.
Resnick, M. (1994). Turtles, termites and traffic jams: Explorations in massively parallel
microworlds. Cambridge, MA: MIT Press.
Resnick, M. (1996). Beyond the centralized mindset. Journal of the Learning Sciences, 5(1),
122.
Resnick, M., & Wilensky, U. (1997). Diving into complexity: Developing probabilistic
decentralized thinking through role-playing activities. Journal of the Learning Sciences,
7(2), 153172.
Rumelhart, D.E. (1980). Schemata: The building blocks of cognition. In R.J. Spiro, B.C. Bruce,
& W.F. Brewer (Eds.), Theoretical issues in reading comprehension (pp. 3358).
Hillsdale, NJ: Erlbaum.
Schaneveldt, R.W. (1990). Pathfinder associative networks: Studies in knowledge organization.
Norwood, NJ: Ablex.
Shavelson, R.J., Ruiz-Primo, M.A., & Wiley, E. (in press). Windows into the mind.
International Journal of Higher Education.
Shoenfeld, A.H., & Herrmann, D.J. (1982). Problem perception and knowledge structure in
expert and novice mathematical problem solvers. Journal of Experimental Psychology:
Learning, Memory, and Cognition, 8, 484494.
Slotta, J.D., & Chi, M.T.H. (1999). Overcoming robust misconceptions through ontological
training. Paper presented at the Annual Meeting of the American Educational Research
Association, Montreal.
Vosniadou, S. (1994). Capturing and modeling the process of conceptual change. Learning and
Instruction, 4, 4569.
Vosniadou, S., & Brewer, W.F. (1987). Theories of knowledge restructuring in development.
Review of Educational Research, 57, 5167.
Wilensky, U. (1999). GasLab: An extensible modeling toolkit for connecting micro- and
macro-properties of gases. In W. Feurzeig & N. Roberts (Eds.), Modeling and simulation
in science and mathematics education (pp. 151178). New York: Springer-Verlag.
Zaim-Idrissi, K., Desautels, J., & Larochelle, M. (1993). The map is the territory! The
viewpoints of biology students on the theory of evolution. The Alberta Journal of
Educational Research, 39, 5972.

518

SYLVIA T. DAPOLLONIA ET AL.

APPENDIX A
The introduction to Complex Systems consisted of one, 75 min, class in which
Complex Adaptive Systems was defined. The following characteristics of
complex adaptive systems were described and then students were asked to
come up with examples of these characteristics.
% Simple agents aggregate to make more complex components or systems
which usually have a hierarchical structure.
% Agents can be reused as components of many different structures
(modularity).
% There is variety among the components.
% The components maintain their identity.
% The more complex components emerge dynamically from the random
actions of the simpler agents.
% The flow of information and feedback maintains the individuality of the
components and the functionality of the system.
% There is a selection mechanism whereby the most suitable components
and systems survive and contribute to future systems.
% The relationships among the components are non-linear and change over
time.
The students were then asked to discuss online why complex adaptive
systems are relevant to biology. Below are some of their comments:
The properties of Complex Adaptive Systems (CAS) are relevant to biology
because it involves components interacting with other components that adapt
to their environment. Like in all living systems, complexity refers to the
presence of hierarchical level. That is, the components are formed by
depending on those below them. When studying biology, we know that we
have the hierarchy which involves: ecosystem-community-populationOrganism-Organ-System-Organ-Tissue-Cell-Molecule. I believe that if we
understand this hierarchical system we should be able to understand Complex
Adaptive Systems. Basically what I am trying to say is that CAS is relevant to
Biology as it gives knowledge about components and their adaptation to the
environment.
To understand how Complex Adaptive Systems are relevant to Biology we
must first understand what they are composed of. A Complex Adaptive System
is composed of various levels of hierarchy. Each level is made up of the
components of the level below; however, the behaviour of the higher level can

ACQUISITION OF COMPLEX SYSTEMIC THINKING

519

not be determined by its components behaviours. At each level components

interact with the environment and other components according to a set of rules
and procedures. Based on information received from other components or
outer stimuli, the system adapts by changing the rules. Over a period of time, it
will develop new procedures and behave in a different manner. Since, Biology
is the study of living things and how they interact and Complex Adaptive
Systems are living things that interact with their environments, therefore, the
properties of Complex Adaptive Systems are not only relevant to biology but
essential to it.
The properties of complex adaptive systems (CAS) are relevant to biology
because most of the living organisms have many of the features of CAS. They
involve adapting to a particular environment and having a certain hierarchy.
When a component of the system changes its environment, it changes its rules
to better adapt to that new environment by producing complex temporal
patterns of adapting components. At each level of the hierarchy, the
components are created by the combination of the components of the level
below. Similarly, in biology, all the organisms are part of a hierarchical
organization and depend on the organisms from the lower levels from which
they have evolved. In both, the CAS and biology, the components work and
interact together to set new rules for their survival in a new environment.

520

SYLVIA T. DAPOLLONIA ET AL.

APPENDIX B
Figure 3 represents the derived mental model (from similarity ratings) for one
student. This is contrasted to the same students essay.
Students Essay: Numbers Refer to Extracted Propositions
Biologists have found fossils of sheep-like animals that lived on the banks of
rivers in Africa that have the same ankle bones as whales. An explanation for
this is that when the sheeps lived on the river banks some of them might have
had a change in allelic frequencies (p1). The individuals had to develop certain
genotypes to survive given the environmental pressures (p2, p3, p4). Those
who had the adaptive features, suitable to this environment were selected by
Natural Selection (p5). This selection mechanism results in members of the
population being more suited (p6) to the environment. Individuals contribute
their genes (p7) to the next generation resulting in individuals becoming more
and more suited for the environment (p8).
Extracted Propositions
p1. Some sheep might have had a change in allelic frequencies.
p2. Individuals developed certain genotype (mutations).

Fig. 3. Students mental model of evolution derived from Pfnet.

ACQUISITION OF COMPLEX SYSTEMIC THINKING

p3.
p4.
p5.
p6.
p7.
p8.

521

Individuals had to . . . in order to survive.

Individuals . . . given the environmental pressures.
Those (individuals) with adaptive features were selected.
Natural Selection leads to Differential Survival of individuals.
Individuals contribute their genes.
Individuals become more and more suited.

Coding of Essay: Knowledge Components

Origin of New Traits: !.5 (>.5 for p1 implied, !1 for p4).
Mechanism of Inheritance: 1 for p7.
Mechanism of Evolution: 0 (1 for p5 and p6 (reverses causation), !1 for p8.
Role of Chance: !1 for p3.
Coding of Essay: Emergent Nature of Evolutionary Process
Hierarchical Levels: 6 (6 for p2, p3, p4, p5, p6, p7 and !2 for p1 and p8).
Final Causes: 1 (2 for p1, p5 and !1 for p3).
Thus, this students appears to have three misconceptions; namely, that
environmental pressures cause certain mutations to arise, that the origin of
adaptive traits is determinate (i.e., not a result of random forces), and that
individuals gradually become more and more suited to the environment. This
is apparent both in Figure 3 and in the essay. For example, there is an
inappropriate link between mutation and environmental pressure in Figure 3
that is reflected in p4. Although this student appears to have a good grasp of
the appropriate levels at which evolution operates, she relates the impact of
mutations and genotype on genetic variability of the population directly rather
than indirectly mediated by individuals.