Mixed-Reality Robotics A Coherent Teaching

Framework
Reinhard Gerndt1, Jens Lssem2
Ostfalia University, Wolfenbuettel, Germany
University of Applied Sciences Kiel, Kiel, Germany
1
r.gerndt@ostfalia.de
2
jens.luessem@fh-kiel.de

Abstract Robotics evolved as a central issue in teaching for specific student set, like undergraduate or graduate students
scientific and engineering disciplines. Robotics inherently and towards a specific subject like robotic locomotion or
encompasses a spectrum of sciences and technologies and
vision. They fail to integrate all the aspects into a teaching
qualification levels. However, most current teaching approaches,
related to robotics, concentrate on individual aspects or small framework that addresses robotics as a coherent learning
student groups. In this paper we present the mixed-reality framework. Therefore, with the mixed-reality approach we
robotics educational approach. With our mixed-reality robotics address both aspects of robotics in education, a robotic
teaching system we reached a true interdisciplinary setup, platform and a teaching approach.
addressing different qualification levels. The system allows for Whilst all students typically are very enthusiastic about
aspects like peer education, learning-by-teaching, problem-based
robotics, school students typically lack the skills and their
learning and competition-based (self-) assessment. It consists of
the mixed-reality robotics platform and a teaching concept. The schools the financial background to do an immediate
overall approach has been used in teaching robotics at secondary transition from the entry-level domain to the senior domain. In
school, undergraduate and graduate level. Student and their robotics biography there often is a gap between
instructor feedback is very positive. graduating from school and advancing in the course of studies
to a level to re-enter robotics at a late stage at university. The
KeywordsRobotics education, Mixed-Reality, problem-based presented robotic platform and the educational approach
learning, learning-by-teaching, robotic platform, European intend to close the gap between both domains. The mixed-
qualification framework, robotic programming tools, artificial reality robotic platform allows easy access to a basic control
intelligence, swarm robotics of the robots and opens a path to control of complex behavior
of cooperating real robots. The basic idea of the platform is
I. INTRODUCTION
based on the EcoBe! micro-robots as presented in [1], [2] and
Robotic problems offer a system-level approach to [3]. It was designed with demonstrating, teaching and research
teaching. The overall objective, a functional robotic device, in the domain of advanced robot cooperation and swarm
can only be accomplished if all components, i.e. mechanics, robotics in mind. Figure 1 shows a setup of the system in a
electronics, and computational intelligence, interoperate five-vs.-five robotic mixed-reality soccer setup.
properly. Success or failure becomes immediately obvious
when the robot is set to work. Not only theoretical
competencies, but also practical skills are required to reach a
functional system. Robotics thus addresses different
disciplines from mechanical engineering, over electrical
engineering to computer science, just to mention the most
prominent ones. It also addresses different qualification levels,
starting with school children, spanning the undergraduate
level and finally reaching the graduate level.
However, many current robotic teaching approaches fail to
use the full potential of robotics in education. In our view, this
is often related to the robotic platform used and the classical
lecture approach taken. Robotics kits often have a too narrow
focus towards a specific qualification level. At entry level
many sophisticated platforms are too complicated to operate
and too costly to purchase and to maintain. At advanced level
many low-cost platforms lack the opportunity for serious Fig. 1. Mixed-Reality Setup.
research. Classical lectures typically are focused to a specific
faculty, like mechanical engineering or computer science, to a
 At entry level, the system is used as an easily operated
robotic platform with a few robots. At a higher level it can be
used as a robotic kit for the domain of a large number of
cooperative robots and swarm robotics. In both situations the
cost of the individual robot is a crucial aspect, due to the small
budget when entering robotics and due to the large number of
robots required in the field of cooperative and swarm robotics.
Size of the individual robot also is very important due to the
space required to operate the robots. Versatility of the
environment and thus the spectrum of possible applications
are allowed for by using a horizontally placed screen as robot
arena, to display a virtual environment and virtual objects.
Real objects, like the robots, just can be placed on the screen.
This way, mixed setups with real and virtual objects can be
realized. With easy programming of the robots at entry level
in mind, a graphical programming environment was
introduced in addition to the already available programming in
C and JAVA.
The overall system has been used for hands-on teaching at
secondary school level and at university level [4], [5]. One of
the main objectives of using robots in teaching was raising
awareness on the interfacing between computers and the real
world. Another aspect was to provide a test bench for
requirements engineering, project management, software
engineering, robotics and artificial intelligence teaching.
The educational concept involves learning-by-teaching or
peer instruction as well as competitive aspects, both within the
same university and among different universities. Over the
time, at a number of universities in the world a student
community was established to work on and with the mixed-
reality robotic platform. For learning by teaching, senior
students pass on their knowledge to less mature students and
by this also gain deeper insight into technological aspects and
improve their social competencies (fig. 2). This takes place
within robotic classes and spans over different faculties as
well as different universities. Typical methods for this are
joint student workshops to work on specific aspects of the
system or applications. During workshops students may
discuss their individual findings and research results with their
fellow students and eventually compare their development
results with others. They may also join forces to address more
complex objectives.
Another aspect of the teaching concept is competitions.
During competitions students are exposed to certain aspects of
robotics. Often, competitions are handled by senior students.
A typical entry-level competition setup is a two-vs.-two
robotic soccer game. At the advanced level teams compete in
currently seven-vs.-seven mixed-reality robotic soccer games
within the RoboCup, robotic soccer championship. However,
the set of applications is not limited to robotic soccer.
The remaining paper is organized as follows: After this
introduction, we will shortly describe three other robotics
platforms for teaching robotics at school and university level.
In the then following section we will present the hardware and
software architecture of the mixed-reality robotic platform
with its graphical programming environment. Then we present
our teaching approach. Following that, we present first results
and finish with an outlook to future work.
Fig. 2. Peer teaching at workshop with competition.
II. EDUCATIONAL ROBOTIC PLATFORMS
There exist numerous robotic kits for education and
research. Many teaching approaches are based on a specific
hardware platform. Therefore, short descriptions of three
robotic platforms that are used in teaching robotics or teaching
with robots are presented. The platforms typically are targeted
for a distinct aspect of teaching and impose specific
constraints on the teaching approach.
A. Lego Mindstorms
The best-known programmable robot kit probably is the
Lego Mindstorms construction kit. This system is available
on the market since 1998 [6]. Based on the Lego construction
bricks it requires building a robot prior to working with it. On
the other hand it opens a vast design space for different types
of robots that may be optimized for specific tasks and may be
built to individual creative ideas of the users. Many students
are familiar with Lego bricks since their early childhood days.
The central control device is the Robotics Command
System (RCX) or the NXT for more recent kits. The NXT
encompasses a microcontroller that can be connected to a
number of drives and sensors. Typical sensors are touch,
ultrasonic and light sensors. With the mechanical components,
mostly based on the versatile Lego Technic kits, almost any
robot or other automated device can be built (Fig. 3).
The RCX programming environment is based on RCX
Code, a brick-oriented graphical programming language. In
RCX, the graphical blocks represent instructions and control
elements. To form a program, blocks can be directly attached
to each other in a graphical programming environment. It is
also possible to group multiple blocks in order to form new
blocks to improve readability of the program. After putting
together the program on a PC, the code is then transferred to
the RCX brick, where it is executed autonomously. The more
recent NXT versions use national Instruments LabView-based
NXT-G for programming. In addition there is a wide variety
of development kits for different levels of programming skills.
The Lego system is also used by the Roberta approach [7].
Roberta is a project to expose school children, especially girls
to robotic technology in a playful manner. Within the Roberta
project a set of training examples and material was developed
and training for school children was carried out.
Fig. 3. Lego Mindstorms System at RoboCup Junior Rescue.
B. MA-VIN Robot Kit
The MA-Vin robot kit is based on a differential drive robot
with a size of roughly 12 by 10 cm. There is almost no
construction work that needs to be done and thus little
flexibility in the individual design of the robots.
The robot is controlled by ATMEGA64L microcontroller
[8]. Unlike Lego, the mechanical construction can be altered
only marginally. In its basic configuration the MA-VIN (fig. 4)
is equipped with 6 optical sensors for collision avoidance and
to survey the ground. Additional sensors can be added to any
of five terminals. The robotic kit comes with several I/O
modules, like a touch sensor, a light sensor, a buzzer and an
LED line, to mention just some.
Fig. 4. MA-VIN Robot Kit.
Programming at entry level is done with a graphical
programming interface. Programs are developed on a PC and
then transferred to the robot, such that the robot operates
autonomously. More advanced users may also use the
programming language C.
Typical educational applications are a line tracer,
pathfinder in a maze or robotic sumo, with the objective to
push an opposing robot out of a starting circle.
C. Jasmine Open-Source Micro-Robots
When searching for a price worthy small robot, scientific
swarm robotic projects offer a good entry point. There exist a
number of activities related to swarm robotics. The open-
source micro-robotic project of the universities of Stuttgart
and Karlsruhe [9] is one of them.
The size of the Jasmine robots (fig. 5) is roughly 2.5 cm
cube. Figure 3 shows the most recent version of Jasmine III
with the typical on-board sensors. The locomotion principle of
Jasmine, as of most other swarm robots is a differential drive.
Expandability, e.g. to add additional sensors or actuators,
typically is allowed for by means of stacking PCB boards on
top of the robots. Being targeted for applications with large
numbers of robots, similar to the MA-VIN robots, only few
different configurations are available. However, the overall
robotic platform is kept open, in order to allow individual
configurations by special of special add-on boards, which then
requires circuit design skills.
Fig. 5. Jasmine III Open-Source Micro-Robot [5].
Programming is done in C. Programs have to be loaded to
the robots. The robots are typically self-contained and operate
autonomously. The kit does not foresee any specific
environment, as long as locomotion is feasible.
 III. THE MIXED-REALITY APPROACH AT A GLANCE or move a virtual object, attached to the robot. Agents may
This section outlines the structure of the Mixed-Reality connect remotely to the framework.
Robotic Kit.
A. Hardware Setup
The Mixed-Reality hardware setup consists of the micro-
robots, the augmented reality display or screen and a tracking
camera system. The robots size is 2.5 cm cube (fig. 6).
Differential drive was chosen as locomotion principle. From a
mechanical point of view, the robots consist of a body with
two stepper drives and the rechargeable batteries. It may be
equipped with different controller PCB.
The controller board encompasses the motor drivers, the IR
communication link and the battery electronics, as well as an
extension connector. They typically differ with respect to the
microprocessor or set of microprocessors. A basic controller
board uses a single small AVR microprocessor. In a more
advanced configuration an ARM-7 processor provides Fig. 7. Camera display setup.
computational power for independent operation of the robots.
The vision-tracking module captures the camera output and
determines positions and poses of the robots equipped with
markers and possibly other real objects. The world state
generator generates an individual view for every single robot
in the system. The individual ego-views are then
communicated to the agents that control the robots.
The switch module separates the commands issued by the
agents into commands that affect virtual objects and real
robots. The robot control module takes care of interfacing and
communicating with the robots. The ODE container wraps the
physics engine and takes care of simulation of the virtual
objects. It processes data of real objects, like position and
space occupied, and commands that affect virtual objects. The
physics of virtual objects can be defined freely. In the soccer
system for example, different friction profiles can be
implemented for the virtual ball, to account for different fields.
Fig. 6. ARM-7 version of EcoBe! micro-robots. The graphics module displays all virtual objects on the screen
as defined by an XML data set.
An essential part of the mixed reality environment is the
horizontally mounted augmented reality display. Robots and
real objects can be placed on the screen. Virtual Objects are
displayed on the screen. If required, virtual objects can be
virtually attached to the robots. A tracking camera overhead
captures any object placed on the screen and provides a global
system view (fig. 7). For identification purpose, robots and
other real object can be equipped with optical markers.
Currently a variety of markers can be used. The spectrum
ranges from the markers initially used, over VR markers, e.g.
according to ART toolkit, to a miniature version of the
RoboCup small size league markers.
B. Software Setup
The Mixed-Reality software framework consists of a
number of modules for in- and output, simulation and control
(fig. 8). The most central components of the system are the
software agents. Each agent controls a single robot. The Fig. 8. Structure of the Mixed-Reality system.
agents implement the intelligence of the micro-robots. They
may also take control of virtual objects, e.g. kick a virtual ball,
 The Mixed-Reality kit significantly differs from many other
kits. PC-based software agents control the robots. Even
though the robots do not have any on-board sensors, any kind
of (virtual) sensor can be foreseen for the robots. By
processing the camera image, any sensor data stream can be
generated for the robot agents. It also differs with respect to
the versatility of the environment. As long as it remains
virtual, any kind of environment can be generated and
displayed on the horizontally mounted screen. If required,
interdependencies between robots and environment can be
implemented in software. Real objects, placed on the screen
can be identified by image processing or with the help of
optical markers, attached to them.
C. Graphical Programming Environment
Easy access to programming the robots is considered as a
key aspect for further activities in robotics. Therefore, aside
from standard ways of programming in JAVA and other
programming languages, we implemented a graphical
programming environment, we named Be!Brick. The
objective was to enable inexperienced students to implement
cooperative and swarm robotic behavior [10]. Inspired by its
success among children, we had chosen the Lego Robotics
Invention Studio as a starting point for the development of
Be!Brick.
Be!Brick therefore also uses graphical instruction and
control bricks (fig. 9). A screen may contain multiple
programs, which may be assigned to different robots.
Communication among robots is assured by means of global
variables that are accessible by all programs and event
messages, similar to sensor events. The names and
identification details for the available robots are read from a
configuration file and thus only have to be prepared once.
Fig. 9. Screenshot of the Be!Brick programming environment
An important issue in robot programming is localization
with respect to other robots and possible landmarks in the
environment. The proposed system readily provides bearings
and distances with respect to the robots position and pose (fig.
10). In the soccer exercise, goal poles and corner flags are
typical landmarks. Localization thus can be reduced to simple
vector calculations. Similarly, target positions can be
calculated, like a halfway distance between two robots or, in
the robot soccer benchmark, the center of the goal line
between the two goal posts as an advantageous position for
the goalkeeper or a goal kick. However, if required, a more
advanced scenario can be generated and robots may carry out
localization based on the data from any virtual sensor that can
be calculated by the server.
Robot movements can be controlled by low-level
commands, controlling direction and speed of the two
differential drives. A more convenient way is to use high-level
commands like drive forward or drive a right curve.
However, the proposed system also foresees an even more
abstract go-to instruction to directly drive to certain
positions on the field. The go-to instruction-brick takes the
target position as a parameter. The command remains active
over multiple cycles and tries to minimize the distance
between the target position and the robot. The execution
terminates when the distance is small enough. Then the next
instruction block will start.
Fig. 10. World model of an agent with vectors to soccer field
landmarks
Be!Brick allows for concurrent control of multiple robots.
This makes it an ideal platform for exploring cooperative
behavior with robots. In order to aid exploration of swarm
behavior, Be!Brick contains instruction blocks that represent
the basic rules used for the Boids system developed by
Craig Reynolds in 1986. Namely these are cohesion,
separation and alignment [11]. The instruction blocks can
be added to any Be!Brick program and can be configured to
allow experimenting with different configurations. As the
Boids simulation consists of multiple concurrent rules, a
special brick allows merging the results of the behavioral
bricks.
 IV. CROSS-UNIVERSITY TEACHING APPROACH
A. Goals of our teaching approach
The main goal of our teaching approach was to reach a
higher level of qualification than typically reached. For this
we set up robotics courses in a systemic way, which enables
us to link courses, within different faculties, within different
universities and on the secondary school, bachelor and master
level [12], [13], [14] and [15].
We therefore had to combine traditional learning
methods with problem-based learning and learning-by-
teaching approaches. Using this combination of learning
approaches we are able to reach the dedicated learning
outcomes for the levels 6 and 7 (first and second cycle in the
Framework for Qualifications of the European Higher
Education Area) in the European Qualifications Framework
(EQF) [16].
B. Content of our teaching approach
In terms of content we focus on:
Robotics
o Locomotion
o Kinematic and dynamic motion
o Trajectory control
o Robot Vision
o Localization
Artificial Intelligence
o Agents
o Heuristic Search Strategies
o Path planning
o Machine learning
o Swarm Intelligence
Systems Engineering
o Requirement Analysis
o Software / Hardware Development
o Software / Hardware Testing
o Refactoring
o Software / Hardware Maintenance
Project Management
o Project Planning
o Project Management
o Project Controlling
Integration of lecturers in the learning process (also
within the cross-university-approach)
Nevertheless, a large part of our teaching activities is done
in a rather traditional way we offer lectures on specific
topics with additional exercises. The robotics teaching
approach then helps to correct misconceptions. During early
programming lectures students typically develop a false
understanding of reliability of programs. They seem to believe
that no difference between intended and real behavior is
possible. Working with real robots results in questioning this
believe. Experience from working with real robots leads to
developing a concept of reliability. With respect to sensor data
students (re-) develop a basic understanding of filtering, for
example by means of averaging. When Boolean variables, e.g.
on presence of a certain state that is derived from sensor data
is concerned, implementation of counters to count presence
respective absence of a specific perception during subsequent
cycles is a typical approach. These concepts not only apply to
robotics, but also to communication protocols over unsecure
channels and real-world data bases, just to mention a few.
The robotics approach also helps to introduce some
concepts in a natural way. A multitude of independent robot
entities also physically demonstrates the concept of
parallelism and the need for synchronization, which is an issue
in larger and many real-world programs, to which students
will only be exposed later in their university career.
Distributed computing systems also require an understanding
of parallelism and synchronization. A hands-on challenge for
the students in undergraduate AI and control theory classes is
implementing a two-versus-two robot soccer game with the
mixed-reality system. Having two robots that need to co-
operate as a team, requires parallel operation and
specialization at the same time. Both robots need to operate
simultaneously but also need to specialize, e.g. as a goalie and
a field player at the same time. We established regular block
courses to enable learning-by-teaching among students of the
participating universities, based on the robotics approach.

Further on, aspects like event management, human

resources management, logistics and a few others are also
touched.
C. Learning features of our teaching approach
Core features of our approach (fig. 11) are:
Integration of students on different levels from
secondary school over Bachelor on to Master level
Development of competences in accordance with the
European Qualification Framework
Cross-qualification learning communities
Cross-university learning communities
Integration of short feedback cycles to foster learning
success
Integration of competitions Fig. 11. Components of our learning approach.
 The robotic soccer game also introduces a competitive 300 students participated in the activities over a period of over
element among individual students. In case of competitions, 5 years.
the evaluation of the learning effort and the solution to the In the center of our attention were the students
specific problem is left to the experiment of a game. It is expectations from a cross-university workshop. Interestingly,
independent of subjective assessment and has a much higher students focus most on information exchange. Realization of
acceptance by students. Competitions also play an important new ideas and having fun when meeting with fellow students
role in our cross-university-approach. As part of the regular appears to be less important (fig. 12). Students are interested
block courses, cross-qualification teams are competing with in having a possibly different approach to a subject, when
each other playing robot soccer games. Complexity starts with presented by a lecturer or mature students from a different
two-robot soccer teams and reaches up to eleven-robot soccer university and are eager to discuss their own ideas actively
with their fellow students.
teams.
Additionally, we established cross-university working
groups in which students on different levels work together in a
nearly self-organized way. The cross-university working
groups are based working groups at each participating
university. Working groups often meet during joint lectures
and organize own workshops. The workshops with student
groups from several universities are used to advance the
overall system and if special competencies are needed that
may be available only by one university or individual
participant. Roles of participants typically evolve by means of
experience or commitment. More mature students typically
take the role of advisors to the others. Overall objectives are
typically defined by the goal to improve the robotic system at
hand and to apply concepts presented in lectures. After
reaching consensus on the next partial goals, students decide Fig. 12. Expectations from a cross-university workshop
on the general approach, on the work plan and the team
We were also interested on what conditions a sustainable
structure. Faculty staff provides assistance and guidance only
students network can be established (fig. 13). Besides the
if necessary. Cross-university learning based on the block
argument that the number of workshop participants should be
lectures and on the (self-) organized working groups can be
reasonably small, students ask for regular cross-university
seen as the backbone of our approach.
projects and specific projects to tackle on. A clear framework,
as set out by specific subjects and supported by social media,
The two-versus-two game also is used for workshops with
as well as financial aid or awarding credits for participation
students from secondary school as part of the recruitment
appears to play a less important role.
activities of the participating universities or students at early
undergraduate level. Typically these students are able to
implement a basic two-versus-two robotic soccer exercise
within a few hours. The most common approach is a
specialization of the two robots in each team, with a goalie
and a field robot. Initially the typical behavior paradigm is
reactivity. Other groups, typically with a higher degree of
experience and confidence in their skills implemented
dynamically changing goalkeepers. Implementing basic
cooperation in a team of five robots typically takes them a
little longer. Students, even with little programming
experience, successfully use JAVA, C or C++ as a
programming language for the robot agents. Students with
even more advanced experience often start working on basic
principles of swarm behavior, typically with up to 10 robots, Fig. 13. Conditions for a sustainable students network
and define individual robotic projects to be implemented with
the Mixed-Reality system.
V. CONCLUSIONS
D. Evaluation of our teaching approach
In this paper we presented an educational system, suitable
Evaluations [4], [5] show that students are very motivated for teaching and experiencing cooperative and swarm
by these learning-by-teaching approaches. Over all, more than robotics. It consists of a robotic platform and a teaching
framework. It combines elements from many fields of
computer science, electrical engineering and mechanical
engineering. A graphical programming environment provides
access to novice users. Advanced users may use high-level
programming languages to implement the robotic behavior by
means of agents. The system thus is well suitable for students
with very little programming skills as well as for serious
research. The teaching approach enables cross-university and
cross-qualification learning. Students invested significant
resources to address subjects independently of their course of
studies. Participating students evaluated it very positively.
An objective for the further improvement of our teaching
approach is the better use of distance learning methods and
technologies.

REFERENCES
[1] Guerra, R. da S., Boedecker, J., Yanagimachi, S., Asada, M.:
Introducing a New Minirobotics Platform for Research and
Edutainment. Symposium on Autonomous Minirobots for Research
and Edutainment, Buenos Aires (2007)
[2] Guerra, R. da S., Boedecker, J., Mayer, N. M., Yanagimachi, S.,
Hirosawa, Y., Yoshikawa, K., Namekawa, K., Asada, M.: Introducing
Physical Visualization Sub-League. Lecture Notes in Computer
Science (2008)
[3] Gerndt, R. Bohnen, M., Guerra, R. da S., Asada, M.: The RoboCup
Mixed Reality League, in Dubois E., Gray P., Nigay L. (eds.) The
Engineering of Mixed Reality Systems 2010. pp. 399 418 Springer,
(2010)
[4] Gerndt, R., Lssem, J.: RoboCup Wettbewerb als ein
Ausbildungskonzept?, Norddeutsche Konferenz fr Informatik an
Fachhochschulen, Wolfenbttel 2010
[5] Gerndt, R., Lssem, J: Robotik in der standortbergreifenden
Informatik-Ausbildung, Norddeutsche Konferenz fr Informatik an
Fachhochschulen, Elmshorn 2011
[6] The Lego Mindstorms webpage, mindstorms.lego.com
[7] The Roberta webpage, www.roberta-home.de
[8] The MA-VIN robot kit, www.robonova.de
[9] The Open-Sorce Micro-Robot project, www.swarmrobot.org
[10] Krupop, S.: Be!Brick Graphical Programming System for Multi-agent
Systems, Ostfalia University, Computer Science Department, Bachelor
Thesis 2009
[11] Reynolds, C., Flocks, herds and schools: A distributed behavioral
model., SIGGRAPH '87: Proceedings of the 14th annual conference on
Computer graphics and interactive techniques (Association for
Computing Machinery), pp. 2534, (1987)
[12] Aebli, H.: Zwlf Grundformen des Lehrens: Eine Allgemeine Didaktik
auf psychologischer Grundlage. Medien und Inhalte didaktischer
Kommunikation, der Lernzyklus. Stuttgart: Klett-Cotta. 2006
[13] Lssem, J. et al.: Combining Learning Paradigms to Ensure Successful
Learning Outcomes in the Area of Software Development,
EDULEARN 2011, Barcelona 2011
[14] Bielaczyc, K., Collins, A., Learning communities in classrooms: A
reconceptualization of educational practice. In C. M. Reigeluth (Ed.),
Instructional design theories and models. Volume II: A new paradigm
of instructional theory. Mahwah, NJ: Erlbaum. 1999
[15] Winkler, K., Mandl, H., Learning Communities. In P. Pawlowsky &
R. Reinhardt (Ed.), Wissensmanagement in der Praxis. Neuwied:
Luchterhand. 2002
[16] N.N.: Recommendation oft he European Parliament and oft he Council
of 23 April 2008 on the establishment of the European Qualifications
Framework for lifelong learning. Official Journal of the European
Union, Bruxelles 2008