Augmented Reality and Education

Current Projects and the Potential for Classroom Learning Brett E. Shelton I still remember vividly my experience with the big purple dinosaur. It didnt look friendly like Barney, though. This dinosaur actually looked like a purple T-rex that was poised and ready to bite me. Of course, this was a virtual dinosaur and existed as a 3D object in 3D space sitting among a very real and familiar environment of desk, chair, and walls. The dinosaur was part of an augmented reality experiment being conducted in the Human Interface Laboratory (HITLab) at the University of Washington. For me it was and still isa very interesting experience to see the real world when blended with virtual objects. As an educator, the potential for using this kind of technology for learning is what strikes me the most. Augmented reality as a science and practice has been receiving more and more attention recently as evidenced by articles in mainstream literature (see Feiners Scientific American articlei) and the growing number of developers attending conferences dedicated to this kind of technology (see ISMAR 2002ii). The term augmented reality (AR) has been used and misused to describe a wide variety of devices and people. But for the developers dedicated to creating and using it, the term is defined as a system of tools that allows a person to view one or more virtual 3D objects in the real-world environment. The virtual objects may be stationary or manipulated, seen on a large flat screen or in a heads-up display. AR technology allows for viewing things in a natural environment that otherwise would be impossible to show, such as labels on parts of an engine or forces on the poles of a magnet.iii The project that really got the ball rolling was MagicBook, developed by Mark Billinghurst and Hirokazu Kato, among others, at the HITLab. MagicBook was a book just like any other, complete with a story written on pages that could be read without the help of AR technology. However, the pages also contained virtual animated figures, which once viewed with a heads-up display would act out the story in 3D space above the pages. MagicBook had the additional ability to completely immerse a reader in the land of the characters so that a reader could become a virtual object within the virtual environment of the story.iv MagicBook was a huge hit when it was demonstrated to thousands of visitors at SIGGRAPH 2000, the worlds largest computer graphics conference, and got a lot of people interested in developing this kind of technology for industrial applications. Much of the current
Copyright 2002 by New Horizons for Learning Vol. 9 No. 1 December 2002 For permission to redistribute please contact -- http://www.newhorizons.org

development is sponsored by corporations interested in using AR for design and diagnostics. They imagine designing a car in three dimensions, being able to make onthe-fly changes to the shapes of fenders or the placement of wheels. They might also have the non-working parts of an engine create a virtual pop-out that explains to the technician what needs to be fixed. Another area of interest in industry and the military are its possible uses for training. Certainly, taking advantage of using the inherent spatial nature of AR may be useful for practicing physical skills. Current projects are looking closely at using AR for training physicians for virtual surgeries and military specialists for hazardous materials handling. But what about more mainstream education? Can we use AR for teaching classroombased curricula? Very little of this technology has been used outside of a laboratory setting, let alone inside a classroom. But some efforts have been made in this direction and more answers may be forthcoming. For my part, I believe the AR interface is a visualization technology that can take advantage of the limitations offered by other visual means of communication for learning. In choosing a topic to study I wanted to focus on curriculum that naturally lends itself to 3D space. I chose astronomyspecifically the concepts involving seasonal variation of light and temperatureor what we refer to as earth-sun relationships. The virtual sun and earth are manipulated on a small hand-held platform that changes its orientation in coordination with the viewing perspective of the student. The student controls the angle of viewing in order to understand how unseen elements work in conjunction with those that were previously seen.v

Copyright 2002 by New Horizons for Learning Vol. 9 No. 1 December 2002 For permission to redistribute please contact -- http://www.newhorizons.org

The theory is that the traditional methods of learning spatially-related content by viewing 2D diagrams creates a sort of cognitive filter. This filter exists even when working with 3D objects on a computer screen because the manipulation of the objects in space is made through mouse clicks. I am investigating the possibility that the physical manipulation of the earth-sun virtual models in augmented reality will provide a more direct cognitive path toward understanding of content. Additional research is looking at other topics that also naturally lend themselves to 3D space, such as the 3D structure of molecules. The HITLab, in conjunction the Scripps Research Institute and the University of Utah, has initiated a research project that teaches molecular biology concepts to high school students. Teachers and students experiment with different kinds of 3D molecular models and discover new ways of interacting with them. Instructors at a Seattle high school are working with the research team to develop lessons that may be taught using AR technology. So far, the response from all parties has been positive. The high school will use AR for teaching biology and chemistry this winter, with plans to expand the program to include more complex concepts and techniques in the near future.vi Developers and researchers in Switzerland have created a kind of AR virtual chemistry laboratory. Students can view and acquire simple atoms through a virtual drag-and-drop technique. Atoms get combined by matching the spinning outermost electrons of a particular atom to ones that fill its required shell. Once combined, a new

structure is seen and additional atoms can be added using the same method as before. Labels that give the name of the structure appear when completed molecules are
Copyright 2002 by New Horizons for Learning Vol. 9 No. 1 December 2002 For permission to redistribute please contact -- http://www.newhorizons.org

formed. This way students can construct their own complex molecules while being bound by the subatomic rules that govern molecular interactions. This feature offers a clear advantage over traditional methods of building models using styrofoam and straws.vii But there are potential educational applications besides those in science and engineering. Researchers at the University of Singapore have developed a system that uses motion capturing, that when applied to computer models, mimics the actions of dancers as they perform. When viewed through an AR system, students and instructors can experience the performance from any angle as many times as they want in order to make modifications to a scene, critique actions, or simply enjoy it. Seen here, virtual dancers perform life-sized on a rooftop.viii

This article highlights only a few of the educational applications that are underway using this technology. Other applications include museum exhibits, architecture and archeology just to name a few. But despite current efforts, research into the how, what, and why is lagging far behind the technological development of the AR systems. I hope the educational research community remains enthusiastic about the building of AR tools aimed at teaching and learning. My experience with the big purple dinosaur was nearly three years ago, and little has been done since that time that explores the true value to education that this technology may offer. It remains important to remind ourselves to examine the effectiveness of advanced visualization tools for particular kinds of instruction, for which students these tools may be advantageous, and what instructional support may be necessary. These issues need to be clarified and studied so that we know when continued implementation in our schools is warranted.

Copyright 2002 by New Horizons for Learning Vol. 9 No. 1 December 2002 For permission to redistribute please contact -- http://www.newhorizons.org

Author note: Brett E. Shelton is a Learning Sciences Researcher for the Program for Educational Transformation Through Technology in the College of Education at the University of Washington. He has worked as an educational technology consultant and a lecturer of graphic communications. Contact bshelton@u.washington.edu.

Available in the April 2002 issue http://www.sciam.com/2002/0402issue/0402feiner.html. The website of this years conference is still available for exploring at http://www.ismar2002.org/. iii For an excellent review of augmented reality and its development, see Ronald Azumas August 1997 article in Presence titled A Survey of Augmented Reality pp. 355-385. iv Photo courtesy Mark Billinghurst and Hirokazu Kato from the Human Interface Technology Laboratory, University of Washington and the University of Canterbury, New Zealand, http://www.hitl.washington.edu/ and http://www.hitlabnz.org. v A paper presented at the 2002 IEEE ARToolKit Workshop on this topic is available at http://depts.washington.edu/pettt/papers/shelton-hedley-art02.pdf , photo courtesy of Nicholas R. Hedley and Brett E. Shelton. vi Photo courtesy TSRI Molecular Graphics Laboratory. Thanks to Dr. William Winn and Suzanne Weghorst for their interviews. vii Photo courtesy of Benedikt M. Voegtli benedikt.voegtli@graviton.ch, Patrick Juchli pju@hyperwerk.ch at Hyperwerk, Fachhochschule Beider Basel (FHBB), Basel, Switzerland and Morten Fjeld http://www.fjeld.ch/ Man-Machine Interaction (MMI) at the Institute for Hygiene and Applied Physiology, Zurich, Switzerland. viii Photo courtesy of Adrian Cheok and Simon Prince, National University of Singapore, contact adriancheok@nus.edu.sg.
ii

Copyright 2002 by New Horizons for Learning Vol. 9 No. 1 December 2002 For permission to redistribute please contact -- http://www.newhorizons.org