Serious Games in the Classroom: Throwing Constructivism a Lifeline?

Ben Cooperman ETEC 511 University of British Columbia Fall, 2010

Richard Clarks article, Learning from Serious Games? Arguments, Evidence and Research Suggestions and the many emotional responses it has elicited, clearly

illustrates the sharp division with Constructivist thinking on one side and the sort of direct instruction propounded by the Cognitive Load theorists on the other. While the use of simulation-based computer games in the classroom tends to be justified by a Constructivist vision of pedagogy, which has recently come under attack in outcomesbased educational reforms, I will argue that simulation-based computer games can accommodate both arguments and presents a powerful teaching tool that should not be ignored. Although there are theoretical shortcomings, many of the criticisms levied against Constructivism in education are based less upon pedagogical concerns than on economical ones. This essay will come to terms with critiques and merits of Constructivism as a learning theory and will examine the role that simulation-based computer games can play in alleviating some of the concerns. First and foremost, Constructivism is a philosophy, with its roots in epistemology. Ernst Von Glasersfeld (1989) summarizes Constructivism according to two basic principles: (1) knowledge is not passively received but actively built up by the cognizing subject; (2) the function of cognition is adaptive and serves the organization of the experiential world, not the discovery of ontological reality. (p.162) Von Glasersfeld point out that the idea that learners construct their own understanding goes back at least as far as Socrates. Manus (1996) similarly contrasts the rhetoric of the Sophists of ancient Greece, lecturing and modeling to a passive audience, with the dialectical model employed by Socrates in Platos dialogues. For the Sophists, education was intended to provide students with the necessary knowledge and skills for obtaining positions within the newly formed democratic citystate, whereas Socrates perceived it as a quest for wisdom (cited in Manus, 1996, p.312) Socrates directly engaged his students in dialogue and through a step-by-step process of argumentation whereby their assertions could be reinforced, modified or rejected. While we see the dialectic in a variety of cultures throughout history, it is nevertheless the teacher-directed, rhetorical model that has predominated our education systems. The origins of this, Manus argues, is in the ideology of the industrial revolution where knowledge was considered as a commodity and the classroom designed to mirror the assembly line in an attempt to maximize production. Outcomes-based curricula, standardized tests, end-of-year promotions and retentions were all built in as quality controls. (Manus, 1996) This model was further refined by the influence of the behaviourist model from the 1960s onwards, which assumed a direct correlation between student learning and stimuli presented in the classroom. By correctly restructuring the learning environment and teacher instruction, student learning could be maximized. (Jones, Brader-Araje, 2002) The shift to a more student-centered approach is due, in large part, to the work of educational theorists such as Piaget, Dewey, Vygotsky and Bruner. Piaget believed that learning was not only a cognitive process but also a biological one of assimilation and accommodation. Through extensive research, particularly with children, he concluded that our understanding of reality is constantly being revised and reconstructed with each new experience. We form logical structures, often referred to as schema, to help us make sense of the world and make modifications when

necessary as we accommodate new information. The notion of truth according to an objective reality is replaced with one of viability according to existing schema in the mind of the learner. (Jones, Brader-Araje, 2002) In Vygotsky and Bruner, we no longer see the child as the lone scientist described by Piaget but instead see learning reframed as a socially mediated process where the child is provided with instructional scaffolding (Bruner, cited in Larkin, 2002) as he progresses through the zone of proximal development (Vygotsky, 1978) which represents the distance between what he can do independently and what he can do with support and guidance from an adult or more capable peer. In a Paper for the President's Commission for a National Agenda for the 80s, Seymour Papert (1980), who had worked with Piaget in Geneva in the late 1950s and early 1960s, recommended a foundational shift in public education around the introduction of computers in the classroom. For Papert, the computer is not a tool for delivering instruction but a tool to think with. It can facilitate the transition described by Piaget whereby childrens thinking moves from the concrete to the formal. Through a process of experimentation and play, students apply their expanding knowledge and explore possibilities in Microworlds on the computer, particularly through programming language LOGO, and in the process, begin to think about their own thinking. (Papert, 1993) Papert makes an economic argument as well, suggesting that computers can replace many of the more traditional classroom tools. Offering direct feedback to students, they can scaffold student learning and possibly reduce the number of teachers needed as well. (Papert, 1993) The impact of these theories cannot be underestimated. Our understanding of learning has been reframed as a process rather than an outcome. Teachers and students engage in inquiry learning, Problem-Based learning, experiential learning and more recently, Challenge-Based Learning. We see classrooms where students are arranged in groups rather than in rows so as to facilitate discussion. (Jones, BraderAraje, 2002) We see examples of student work displayed prominently. Report cards add an emphasis on soft-skills such as attitude, effort and problem-solving. Most importantly, with a focus on how students learn rather than what students learn, we see less teacher-directed instruction and a greater emphasis on discovery and play and on context and learning-by-doing. Differentiation has become commonplace and since the 1980s, we have begun to discuss students according to a theory of Multiple Intelligences. (Gardner, Hatch, 1989) Teachers incorporate multimedia and computers to accommodate different learning styles. And while programming may no longer be at the center of contemporary computer use in the classroom, it is the Constructivist approach has largely been the vehicle to usher in its expanded use. Rather than receiving input from the teacher, students conduct research independently and work at their own pace. Educational software is increasing interactive and there is a new emphasis on making learning entertaining and fun. Industry has recognized an expanding market for Edu-tainment. (Derryberry, 2007) But not everyone has welcomed this change. A political shift to the right has brought with it a return to outcomes-based education policy. In the United States, lower than expected scores on various international assessments such as TIMSS and PISA has led to increased political pressure for change. In 2004, the Wall Street Journal warned, Economic Time Bomb: U.S. Teens Are Among the Worst at Math. The New

York Times warned, Math and Science Tests Find 4th and 8th Graders in U.S. Still Lag Many Peers. Momentum that began with the publication of A Nation at Risk: The Imperative for Educational Reform during the Reagan administration (1983) ultimately led to the passing of No Child Left Behind in 2001 with a reintroduction of standards-based reforms such as curriculum frameworks that outline specific knowledge students are expected to acquire, criterion-based assessments that are aligned with specific curriculum expectations, and high-stakes tests. (Hamilton, Stecher, Yuan, 2008) Around the world, we have seen similar reforms that encourage a return to direct teaching methods such as the introduction of the Literacy Hour and Numeracy Hour in the U.K. In the most recent (2010) results from PISA, the United States came out 25th in math, 17th in science, and 14th in reading out of the 34 Organization for Economic Cooperation and Development (OECD) countries. China on the other hand, participating in PISA for the first time came out near the top in each category. ABC News (December 7, 2010) cited comments by Education Secretary Arne Duncan in response. "The findings, I have to admit, show that the United States needs to urgently accelerate student learning to try to remain competitive in the knowledge economy of the 21st century," Duncan said at a press conference in Washington. "Americans need to wake up to this educational reality, instead of napping at the wheel while emerging competitors prepare their students for economic leadership." These comments certainly appear to echo the sentiments of the Sophists 2500 years earlier. (It may be interesting to note that this article appeared in the Politics section.) Many researchers, particularly proponents of Cognitive Load Theory, would support this shift. Strong critics of Constructivist education, they argue in favour of a more traditional teaching model of direct instruction. For Paul A. Kirschner, John Sweller and Richard E. Clark (2006) studies of human cognitive architecture provide the key to understanding the best method of classroom instruction. For them, the dominant structure of human cognition is our long-term memory. The goal of all instruction is an alteration of long-term memory. If nothing has changed in long-term memory, nothing has been learned. (Sweller et al., 2006, pp.77) The vehicle by which information is processed and transferred into long-term memory is referred to as working memory. While there do not appear to be temporal limits to affect our ability to bring previously learned information out of long-term memory into working memory the research suggests that we do when it comes to new information. Sweller et al. cite well-known studies that indicate that (a) information stored in working memory without being rehearsed is lost within 30 seconds and (b) we can only manage a limited number of very small elements simultaneously in working memory (in most people, between 4 and 7.) (Sweller et al., 77) They argue that problem-solving, which is central to the Constructivist methodology, puts an excessive burden or, cognitive load, on working memory, and instead, support a teaching method based on direct instruction where new concepts are first introduced in the abstract and only contextualized in realistic problem scenarios once the learners have internalized the

information. They argue that if working memory is being used to search for problemsolutions, it is not available to process information to long-term memory. It therefore cannot be used to learn. Thus methods that provide minimally guided instruction are inefficient. Students may search for extended periods of time with little or no change in long-term memory. By contrast, they recommend building a knowledge foundation based on strongly guided instruction and the use of worked examples, which take students step-by-step through a problem and teach each part of the task in isolation to ensure understanding. (Sweller et al., 2006) They reject the just-in-time approach to teaching content and argue that there is no evidence to support the idea that presenting students with partial rather than complete information will enhance their ability to construct representations of reality. Students must construct schema regardless of whether or not they have a complete set of information. Class time will be spent more efficiently and students will be less prone to misunderstandings when guided through direct instruction. In response, Hmelo-Silver, Duncan and Chinn (2007) argue that the approach advocated by most Constructivists is not minimally guided at all, as claimed by Sweller and his colleagues, and in fact, provides extensive scaffolding and support to facilitate student learning. Furthermore, the sort of modeling provided in Projectbased learning (PBL) and Inquiry learning (IL) are very similar to those of the worked examples and the various data-visualization tools that are common in Constructivist classrooms are similar to the process worksheets recommended by Kirschner, Sweller and Clark. (Hmelo-Silver, 102) Clark and his colleagues reference a study by Klahr and Nigam that suggests that through direct instruction, students not only better retain information but are also better able to transfer that information to new contexts. (Sweller et al., 2006, p.79) Kuhn and Dean Jr. (2006) repeated this study 2 years later but in contrast to Klahr and Nigams design, followed student progress over an extended time period. Their results (below) remarkably showed that while in the short term, a combination of direct instruction (DI) and practice (PR) showed the highest test scores, over time, discovery alone led to the highest information retention. (Dean Jr. and Kuhn, 2006) Multipletrace memory models would suggest that it takes some time for some details from trace memory to be absorbed into existing schema (Levitin, 2006).

Another strong criticism of Constructivism made by Clark and colleagues is that it confuses the epistemology of a discipline with the pedagogy of teaching or learning it. For them, the practice of a profession is not the same as learning to practice the profession, arguing that it is inappropriate to expect the novice to use the tools and methods of the professional and this again puts an excessive load on working memory, making it less likely that learners will be able to simultaneously acquire any new knowledge. (Sweller et al., p.83) Hmelo-Silver and colleagues counter that current Constructivist approach represents a worldwide change in instructional focus that recognizes the inadequacy of the simple recall of facts in preparing students for the future and places increased emphasis on disciplinary epistemologies. (Hmelo-Silver et al., p.105) As part of their Quest Atlantis project, researchers at Indiana University have developed 3D storyworlds where students engage in transformational play as they take on the roles of experts in dealing with real-world problems. Merely playing a game does not ensure that a student is engaged in transformational play. To play transformationally, a player must become a protagonist who uses the knowledge, skills, and concepts embedded in curricular content to make sense of a fictional situation and make choices that transform that situation. (Barab et al., 2009, p.77) In Taiga World, a unit about water quality that takes place in a fictitious virtual park, students investigate the problem of fish decline in the rivers. They collect data and ultimately make decisions, such as to outlaw logging in the park or ban farming near the rivers to try to solve the problem. They then experience the consequences of their choices when the simulation transports them 10 years in the future to witness the results. In a comparative study where 2 groups were taught the same content, one using Taiga World and the other using more traditional teaching methods, the

virtual world group showed deeper understanding of the science concepts presented, higher levels of engagement, greater intrinsic motivation and improved retention of knowledge over time. (Barab et al., 2009) In a study based out of North Carolina State University, researchers were interested in the effect of narrative on learning outcomes. Their Crystal Island project, an intelligent, game-based, 3D storyworld engages students in a science mystery. The user, as protagonist attempts to discover the origin of a mysterious illness at a research station on the island. The researchers worked with 4 groups of students who had recently completed a microbiology unit in class; a control group that did not participate in the game, a group that played the game, a group that played a modified version of the game but had all of the narrative features stripped out of the experience and a fourth group that was instead taught all of the knowledge content in the form of PowerPoint presentations. Unlike the Quest Atlantis study, post-test scores showed the most significant gains by the direct instruction (PowerPoint) group. The researchers concluded that the lower scores in the full narrative group were the result of student cognition being overloaded by the additional story content. The full narrative group however, reported higher motivation and self-efficacy and research continues to identify specific factors that support student problem solving, increase student effort, persistence, and resilience when confronted with failure, and raise the levels of success students are likely to achieve. (McQuiggan, Rowe, Lee and Lester, 2008, p.8) Constructivist practitioners are having to come to terms with an apparent mismatch between assessment regarding assessment tools. While knowledge acquisition is usually measured by more traditional testing methods that quantitative data for researchers, this is proving much more difficult when it comes to measuring the disciplinary epistemologies and soft-skills that are emphasized in the Constructivist classroom. (Salen, 2009) Nevertheless, we are seeing evidence of increased demand for soft-skill development. As evidenced by the annual Job Outlook survey (below) by the National Association of Colleges and Employers, which lists the skills most sought by employers. In 2010, so-called soft-skills topped the list. Figure 1: Employers Rank Top 5 Candidate Skills/Qualities
1. 2. 3. 4. 5.

Communication Skills Analytical Skills Teamwork Skills Technical Skills Strong Work Ethic

Source:Job Outlook 2010, National Association of Colleges and Employers. In his 2007 critique of serious games, Clark reiterates his (1983) argument that, There are no learning benefits to be gained from employing any specific medium to deliver instruction. For Clark methods and media are separate entities and it is methods

that influence learning while medias influence is only on cost and distribution. He suggests a replaceability test whereby we consider whether any learning outcome from a given task could have been similarly accomplished using any other medium. If this is the case, then we cannot attribute specific learning to any particular medium and should choose the least expensive option. (Clark, 1983, 2007). He claims that it is not possible to extract from existing research any evidence that games are more effective than any other considered instructional practices and argues solidly against the proposition that games replace direct instruction. He discounts any studies of the effectiveness of games that do not offer comparisons with other teaching methods. Parker, Becker and Sawyer (2007) take issue with this argument, pointing out that in most classrooms, teachers employ a variety of presentation methods and students are encouraged to seek information from a variety of sources. An effective teaching tool should be considered in its own right. If cost is to be the determining factor however, they identified a number of errors in Clarks calculations. Costs of digital learning tools can be spread over large populations at no increased cost. And while the cost of developing commercial video game titles can be tens of millions of dollars, serious games, with a narrower focus, simpler objective and fewer levels, rarely cost anything close to that. Serious game developers may work with found art and music and use existing game engines such as the Panda 3D game engine, developed by Disney studios, the Quake 3 game engine and the Blender game engine that are all freely available under open-source licenses. Crystal Island was developed with a 2.5 million dollar grant from the National Science Foundation. Becker and Sawyer claim that of the many serious games they have worked on, none have cost more than $20,000. Kurt Squire (2007) demonstrated the effectiveness of using commercial off-the-shelf (COTS) games in the classroom, using Civilization III to teach history. Rollercoaster Tycoon is often used to teach physics, Sim City to teach government and even the free, online Lemonade Stand for the basics of economics. Blunt (2009) cites 3 studies that showed significant gains for students who supplemented their learning in class with games including university economics students who played the free, online game Zapitalism and business students who played Industry Giant II. It is noteworthy that the standardized tests that have become the hallmark of No Child Left Behind and other outcomes-based education reforms are themselves a source of ongoing debate. Estimates range over the true costs of standardized tests in the US, with a GAO study putting the number at about 500 million dollars annually and study by CSTEEP that puts the number 45 times higher. (Phelps, 2006) Robert Kozmas (1994) considers how particular media, possess particular characteristics that make them both more and less suitable for the accomplishment of certain kinds of learning tasks. (Kozma, 1994) Second-language learners and students with special educational needs clearly benefit from additional support using audio and visuals. Parker et al. (2007) echo Kozmas sentiments, suggesting that the creation of effective learning experiences requires a synergy of content and design. They argue that many of the educational computer games of the past were failures because they were little more than reformulated drill and practice worksheets, wrapped in something sweet but still the same good-for-you vegetable. (p.41) They also point out that games are not just a teaching tool; they are a medium of expression and communication.

Recently, we have seen a variety of projects where students are involved in the development of the games themselves. Scratch, an outgrowth of Paperts work at MIT, is a simple, block-based programming language that allows students to create their own games and animations and share them online. To date, there are over a million projects, many of them games with educational content. Another project, Gamestar Mechanic, teaches students elements of design as they create and share their own games. As part of the Educate to Innovate initiative by the Obama administration, Gamestar Mechanic is currently running the STEM Video Game Challenge. The programs developers claim that, "making video games is a complex activity that involves critical 21st century skills such as systems thinking, iterative design, problem solving, collaboration and creativity, and builds a strong motivation for STEM subjects. (GamePolitics, September 22, 2010) While recent studies, including a 2008 publication by the Association for Psychological Science, have called into question some of the research that supports theories of Learning Styles (Pashler et al., 2008), we know that students have varied strengths and preferences. Research done with at-risk learners has shown that these students are more motivated to participate when they are involved in the development of a product where they can see practical value to what they are learning. (Stager, 2001) And at a time when 3 out of 10 students in the United States drop out of school without a diploma, (New Media Consortium, 2009) issues of motivation should not be considered lightly. With its emphasis on hands-on learning experiences, there is no doubt that Constructivist pedagogy is resource intensive. The Crystal Island researchers acknowledged, exploratory learning is known to suffer from efficiency issues: exploratory learning, even guided exploratory learning, often is more time-consuming than more didactic approaches. (Mott et al., p.6) However, when we expand our view of learning to consider soft-skill development, recognize the importance of motivation and try to reach out to all students rather than trying to raise average scores at the expense of students who drop out, we gain a new perspective on what may be the most efficient use of class time. Furthermore, new research on memory shows that much of our brains computational power comes from its capacity for interconnection of information. Information is easier to access when it is connected to more things. (Levitin, 2006) We develop these interconnections when we learn things in context. We are well advised to consider the important contributions that Cognitive Load theory has made for education. Just-in-time teaching is not enough. Learners need to develop depth and breadth to their knowledge foundation as they form schema to make sense of the world around them. Inquiry-based programs now describe a process of front-loading information before beginning an inquiry. The discovery model has been replaced with an increased recognition of the need for effective scaffolding. Games and simulations can however, play an important role in this process. They can accelerate time or slow it down. They can simplify reality and limit choices. They allow us to zoom in on details and move along at our own speed. They allow us to experience failure without dire consequences. (Squire, 2007) Studies by Lester et al. (1999) have shown that intelligent, animated pedagogical agents can support students by delivering just-in-time assistance when a little extra hint is needed. Artificial

intelligence may not replace our teachers but it certainly gives them another valuable tool to support students and maximize the limited time they have available. It is unlikely that the teaching style debate will end any time soon. There are countless articles, and studies to support both sides. An AERA symposium that followed publication of Sweller, Kirschner and Clarks 2006 article resulted in publication of Constructivist Instruction: Success or Failure? (Tobias, Duffy, 2009) It offers another 392 pages to confound todays educator. But as educators, we must remind ourselves that most of the time, while researchers may be arguing opposing sides, most of us are doing something somewhere in the middle. And despite what Clark and his colleagues argue, that is where serious games belong. Video games are a $20 billion per year industry in the U.S. alone (NPD Group, 2010). Serious games are used in emergency services training, corporate education, health care and military training. Even McDonalds is using serious games to train personnel in customer service, store operations and employee supervision. (Derryberry, 9) Like Kurt Squire (2007), many of our students are already learning from games, sometimes long before they encounter those same things in the classroom. He points out that we can choose to play a role in this inevitable development or not. As educational technologists, we can choose to join those embracing more expansive views of research and pedagogyor we can stay on the sidelines as the 21st century marches ahead. (Squire, 2007, p.54)

Sources: Arenson, Karen W. (2004, December 15) Math and Science Tests Find 4th and 8th Graders in U.S. Still Lag Many Peers. New York Times. p.T1. Bangeman, Eric. (2008, January 4). Growth of gaming in 2007 far outpaces movies, music. Retrieved Nov. 17, 2010 from http://arstechnica.com/gaming/news/2008/01/ growth-of-gaming-in-2007-far-outpaces-movies-music.ars. Barab, Sasha A., Gresalfi, Melissa & Arici, Anne (2009). Why Educators Should Care About Games. Educational Leadership. 67(1), 76-80. Bruce, Mary. (2010, December 7) China Debuts at Top of International Education Rankings U.S. Continues to Lag Behind Other Developed Countries. ABC News. Retrieved December 7, 2010, from http://abcnews.go.com/Politics/china-debuts-topinternational-education-rankings/story?id=12336108 Clark, R.E. (1983). Reconsidering Research on Learning from Media.Review of Educational Research. 53(4) 445-459. Clark, R. (1994). Media will never influence learning.Educational Technology Research and Development. 42(2) 21-29.

Clark, R. E. (2007).Learning from Serious Games? Arguments, Evidence, and Research Suggestions. Educational Technology. 47(3) 56-59. Dean Jr., D. and Kuhn, D. (2007), Direct instruction vs. discovery: The long view. Science Education. 91384397. doi:10.1002/sce.20194 Derryberry, A.(2007) Serious Games: Online Games for Learning. Adobe Systems Incorporated. Dialectic (n.d.) In Wikipedia. Retrieved November 17, 2010, from http:// en.wikipedia.org/wiki/Dialectic GamePolitics (September 22, 2010). BrainPop Supports National STEM Video Game Challenge. Retreived December 1, 2010 from http://gamepolitics.com/2010/09/22/ brainpop-supports-national-stem-video-game-challenge Gardner, H. and Hatch, T. (1989). Educational Implications of the Theory of Multiple Intelligences.Educational Researcher, 18(8):4-10. Gash. H. (2009) What you always wanted to know about constructivist education... Review of "Constructivist Instruction: Success or Failure." Edited by Sigmund Tobias & Thomas M. Duffy. Routledge, London 2009. Constructivist Foundations. 5(1) 64-65 Hamilton, L.S., Stecher, B.M., and Yuan, K. (2008). Standards-Based Reform in the United States: History, Research, and Future Directions. Technical report. Harel, I. and Papert, S. (1991) Situating Constructionism in Constructionism. Norwood, NJ: Ablex Publishing. Hmelo-Silver, Duncan, & Chinn. (2007). Scaffolding and Achievement in Problem-Based and Inquiry Learning: A Response to Kirschner, Sweller, and Clark (2006) Educational Psychologist. 42(2), 99107. Jones, Gail M. & Brader-Araje, Laura. (2002). The Impact of Constructivism on Education: Language,Discourse, and Meaning. American Communication Journal. 5(3) 1-10. Kirschner, P. A., Sweller, J., & Clark, R. E. (2006). Why minimal guidance during instruction does not work: An analysis of the failure of constructivist, discovery, problem-based experiential and inquiry-based teaching.Educational Psychologist. 41(2), 75-86. Kozma, R. (1994). Will media influence learning: Reframing the debate. Educational Technology Research and Development. 42(2), 7-19. Kronholz, June. (2004, December 7) Economic Time Bomb: U.S. Teens Are Among the Worst at Math. The Wall Street Journal World News. p.B1. Larkin, M. (2002). Using scaffolded instruction to optimize learning. Arlington, VA: ERIC Clearinghouse on Disabilities and Gifted Education. (Retrieved December 1, 2010

from ERIC at EBSCOhost, ERIC No. ED. 474 301). Lester, James C., Stone, Brian A., Converse, Sharolyn A., Kahler, Susan E. & Barlow, Todd S. (1999). Animated Pedagogical Agents and Problem-Solving Effectiveness: A Large Scale Empirical Evaluation. In Proceedings of Eighth World Conference on Artificial Intelligence in Education, pages 23-30, Kobe, Japan. Manus, Alice L. (1996). Procedural versus Constructivist Education: A Lesson from History. The Educational Forum. 60(4) 312-16 Maxwell, John W. (2006) Resituating Constructionism. in International Handbook on Virtual Learning Environments, Volume I. Ed. J. Weiss, J. Nolan, J. Hunsinger & P. Trifonas. Netherlands: Springer. 279298. Mayer, Richard E. (2003). Elements of a Science of E-Learning. Journal of Educational Computing Research. 29(3) 297-313. McQuiggan, S., Rowe, J., Lee, S., and Lester, J. (2008). Story-Based Learning: The Impact of Narrative on Learning Experiences and Outcomes. In Woolf, B.P., Ameur, E., Nkambou, R., and Lajoie, S., editors,Intelligent Tutoring Systems, volume 5091 ofLecture Notes in Computer Science, chapter56, pages 530-539. Springer Berlin Heidelberg, Berlin, Heidelberg. McQuiggan, Scott, Lee, Sunyoung, & Lester, James. (2007)Early Prediction of Student Frustration. InProceedings of the Second International Conference on Affective Computing and Intelligent Interactions, Lisbon, Portugal, pp. 698-709.

Mott, B., Callaway, C., Zettlemoyer, L., Lee, S., and Lester, J. (1999). Towards Narrative-Centered Learning Environments.Working Notesof the 1999 AAAI Fall Symposium on Narrative Intelligence, pp. 78-82, Cape Cod, MA, November 1999. Mott, Bradford, McQuiggan, Scott, Lee, Sunyoung, Lee, Seung, & Lester, James.(2006) Narrative-Centered Environments for Guided Exploratory Learning. In theProceedings of the AAMAS 2006 Workshop on Agent-Based Systems for Human Learning, Hakodate, Japan, pp. 22-28, 2006. Mott, B. and Lester, J. (2006). Narrative-Centered Tutorial Planning for Inquiry-Based Learning Environments. InIntelligent Tutoring Systems, pages 675-684. Nanjappa, A. & Grant, M.M. (2003). Constructing on Constructivism: The Role of Technology. Electronic Journal for the Integration of Technology in Education, 2(1). http://ejite.isu.edu/Volume2No1/nanjappa.htm (Retrieved Oct. 30, 2010). New Media Consortium. (2009) Challenge-Based Learning: An Approach for Our Time. The New Media Consortium. Apple Inc. NPD Group. (January 15, 2010) 2009 U.S. Video Game Industry and PC Game Software Retail Sales Reach $20.2 Billion. Retreived on December 1, 2010 from http:// www.cdrinfo.com/Sections/News/Details.aspx?NewsId=26708

Papert, S.A. (1993).Mindstorms: Children, Computers, And Powerful Ideas. Basic Books, 2 edition. Papert, S.A. (1980) Paper for the President's Commission for a National Agenda for the '80s',Paper for a commission formed by President Jimmy Carter. Parker, J.R., Becker, Katrin & Sawyer, Ben. (2008) Re-reconsidering Research on Learning from Media: Comments on Richard E. Clarks Point of View Column on Serious Games. Educational Technology Magazine. 48(1), 39-43. Pashler, H., McDaniel, M., Rohrer, D., and Bjork, R. (2008). Learning Styles.Psychological Science in the Public Interest, 9(3):105-119. Phelps, Richard. (2006). Estimating the Costs and Benefits of Educational Testing Programs. Education Consumers Clearing House Retrieved from http:// www.education-consumers.com/briefs/phelps2.shtm Prensky, M. (2001). Digital Game-Based Learning. New York: McGraw Hill.ng. New York: McGraw Hill. Squire, K. D. (2007). Games, learning, and society: Building a field.Educational Technology, 4(5),51-54. Salen, K. (2009). Big Thinkers: Katie Salen on Learning with Games [Video file] Retrieved from http://www.edutopia.org/digital-generation-katie-salen-video Stager, G. S. (2001). Constructionism as a high-tech intervention strategy for at-risk learners. Paper delivered at National Educational Computing Conference, Building on the Future, July 25-27, 2001, Chicago, IL. United States. National Commission on Excellence in Education. (1983) A nation at risk : the imperative for educational reform : a report to the Nation and the Secretary of Education, United States Department of Education / by the National Commission on Excellence in EducationThe Commission : [Supt. of Docs., U.S. G.P.O. distributor], Washington, D.C. Von Glasersfeld, Ernst (2010). Why People Dislike Radical Constructivism. Constructivist Foundations. 6(1)19-21. Von Glasersfeld, Ernst (1989). Constructivism in Education. In: T. Husen & T. N. Postlethwaite, (eds.) (1989) The International Encyclopedia of Education, Supplement Vol.1. Oxford/New York: Pergamon Press, 162163. Vygotsky, L. (1978). Interaction between Learning and Development (pp. 79-91). InMind in Society. (Trans. M. Cole). Cambridge, MA: Harvard University Press. Warrick, W. R. (2007).Constructivism: Pre-historical to post-modern.Retrieved Nov. 21, 2010, from http://mason.gmu.edu/%7Ewwarrick/Portfolio/Products/ constructivism.html