Advances in E-learning-Experiences and Methodologies

Advances in E-Learning:
Experiences and Methodologies
Francisco J. García Peñalvo

University of Salamanca, Spain
InformatIon scIence reference

Hershey • New York
Acquisitions Editor: Kristin Klinger
Development Editor: Kristin Roth
Senior Managing Editor: Jennifer Neidig
Managing Editor: Sara Reed
Copy Editor: Jeannie Porter
Typesetter: Jeff Ash
Cover Design: Lisa Tosheff
Printed at: Yurchak Printing Inc.
Published in the United States of America by

Information Science Reference (an imprint of IGI Global)
701 E. Chocolate Avenue, Suite 200
Hershey PA 17033
Tel: 717-533-8845
Fax: 717-533-8661
E-mail: cust@igi-global.com
Web site: http://www.igi-global.com
and in the United Kingdom by

Information Science Reference (an imprint of IGI Global)
3 Henrietta Street
Covent Garden
London WC2E 8LU
Tel: 44 20 7240 0856
Fax: 44 20 7379 0609
Web site: http://www.eurospanonline.com
Copyright © 2008 by IGI Global. All rights reserved. No part of this publication may be reproduced, stored or distributed in any form or by
any means, electronic or mechanical, including photocopying, without written permission from the publisher.
Product or company names used in this set are for identification purposes only. Inclusion of the names of the products or companies does
not indicate a claim of ownership by IGI Global of the trademark or registered trademark.
Library of Congress Cataloging-in-Publication Data
Advances in e-learning : experiences and methodologies / Francisco José García-Peñalvo, editor.
p. cm.
Summary: “This book explores the technical, pedagogical, methodological, tutorial, legal, and emotional aspects of e-learning, considering
and analyzing its different application contexts, and providing researchers and practitioners with an innovative view of e-learning as a
lifelong learning tool for scholars in both academic and professional spheres”--Provided by publisher.
Includes bibliographical references and index.
ISBN 978-1-59904-756-0 (hardcover) -- ISBN 978-1-59904-758-4 (ebook)
1. Internet in education. 2. Continuing education--Computer-assisted instruction. I. García-Peñalvo, Francisco José.
LB1044.87.A374 2008
371.33’44678--dc22
2007032055
British Cataloguing in Publication Data

A Cataloguing in Publication record for this book is available from the British Library.
All work contributed to this book set is original material. The views expressed in this book are those of the authors, but not necessarily of
the publisher.
If a library purchased a print copy of this publication, please go to http://www.igi-global.com/reference/assets/IGR-eAccess-agreement.

pdf for information on activating the library's complimentary electronic access to this publication.
Table of Contents
Preface . ............................................................................................................................................... xiv
Acknowledgment . ............................................................................................................................. xxii
Chapter I
RAPAD: A Reflective and Participatory Methodology for E-Learning and
Lifelong Learning .................................................................................................................................. 1
Ray Webster, Murdoch University, Australia
Chapter II
A Heideggerian View on E-Learning . .................................................................................................. 30
Sergio Vasquez Bronfman, ESCP-EAP (European School of Management), France
Chapter III
Philosophical and Epistemological Basis for Building a Quality Online Training
Methodology ......................................................................................................................................... 46
Antonio Miguel Seoane Pardo, Universidad de Salamanca, Spain
Francisco José García Peñalvo, Universidad de Salamanca, Spain
Chapter IV
E-Mentoring: An Extended Practice, An Emerging Discipline ........................................................... 61
Angélica Rísquez, University of Limerick, Ireland
Chapter V
Training Teachers for E-Learning, Beyond ICT Skills Towards Lifelong Learning
Requirements: A Case Study ................................................................................................................ 83
Olga Díez, CEAD Santa Cruz de Tenerife, Spain
Chapter VI
The Role of Institutional Factors in the Formation of E-Learning Practices . ..................................... 96
Ruth Halperin, London School of Economics, UK
Chapter VII
E-Learning Value and Student Experiences: A Case Study ................................................................ 112
Krassie Petrova, Auckland University of Technology, New Zealand
Rowena Sinclair, Auckland University of Technology, New Zealand
Chapter VIII
Integrating Technology and Research in Mathematics Education:
The Case of E-Learning ...................................................................................................................... 132
Giovannina Albano, Università di Salerno, Italy
Pier Luigi Ferrari, Università del Piemonte Orientale, Italy
Chapter IX
AI Techniques for Monitoring Student Learning Process .................................................................. 149
David Camacho, Universidad Autonoma de Madrid, Spain
Álvaro Ortigosa, Universidad Autonoma de Madrid, Spain
Estrella Pulido, Universidad Autonoma de Madrid, Spain
María D. R-Moreno, Universidad de Alcalá, Spain
Chapter X
Knowledge Discovery from E-Learning Activities............................................................................. 173
Addisson Salazar, Universidad Politécnica de Valencia, Spain
Luis Vergara, Universidad Politécnica de Valencia, Spain
Chapter XI
Swarm-Based Techniques in E-Learning: Methodologies and Experiences....................................... 199
Sergio Gutiérrez, University Carlos III of Madrid, Spain
Abelardo Pardo, University Carlos III of Madrid, Spain
Chapter XII
E-Learning 2.0: The Learning Community......................................................................................... 213
Luisa M. Regueras, University of Valladolid, Spain
Elena Verdú, University of Valladolid, Spain
María A. Pérez, University of Valladolid, Spain
Juan Pablo de Castro, University of Valladolid, Spain
María J. Verdú, University of Valladolid, Spain
Chapter XIII
Telematic Environments and Competition-Based Methodologies:
An Approach to Active Learning......................................................................................................... 232
Chapter XIV
Open Source LMS Customization: A Moodle Statistical Control Application.................................... 250
Miguel Ángel Conde, Universidad de Salamanca, Spain
Carlos Muñoz Martín, CLAY Formación Internacional, Spain
Alberto Velasco Florines, CLAY Formación Internacional, Spain
Chapter XV
Evaluation and Effective Learning: Strategic Use of E-Portfolio as an Alternative
Assessment at University . .................................................................................................................. 264
Nuria Hernández, Universidad de Oviedo, Spain
Chapter XVI
Formative Online Assessment in E-Learning...................................................................................... 279
Izaskun Ibabe, University of the Basque Country, Spain
Joana Jauregizar, Quality Evaluation and Certification Agency of the Basque University System,
Spain
Chapter XVII
Designing an Online Assessment in E-Learning................................................................................. 301
María José Rodríguez-Conde, Universidad de Salamanca, Spain
Chapter XVIII
Quality Assessment of E-Facilitators................................................................................................... 318
Evelyn Gullett, U21Global Graduate School for Global Leaders, Singapore
Chapter XIX
E-QUAL: A Proposal to Measure the Quality of E-Learning Courses . ............................................. 329
Célio Gonçalo Marques, Instituto Politécnico de Tomar, Portugal
João Noivo, Universidade do Minho, Portugal
Compilation of References ............................................................................................................... 350
About the Contributors .................................................................................................................... 386
Index.................................................................................................................................................... 394
Detailed Table of Contents
Preface . ............................................................................................................................................... xiv
Acknowledgment . ............................................................................................................................. xxii
Chapter I
RAPAD: A Reflective and Participatory Methodology for E-Learning and
Lifelong Learning .................................................................................................................................. 1
Ray Webster, Murdoch University, Australia
This chapter introduces RAPAD, a reflective and participatory methodology for e-learning and lifelong
learning. It argues that by engaging in a reflective and participatory design process for a personalized e-
learning environment, individual students can attain a conceptual change in understanding the learning and
e-learning process, especially their own. Students use a framework provided by the concept of a personal
cognitive or learning profile and the design and development of a personalized e-learning environment
(PELE) to engage with key aspects of their learning. This results in Flexible Student Alignment, a process
by which students are better able to match their learning and e-learning characteristics and requirements
to the practices, resources, and structures of universities in the emerging knowledge society. The use
of Web-based technologies and personal reflection ensure that RAPAD is well-placed to be an adaptive
methodology which continues to enhance the process of lifelong learning.
Chapter II
A Heideggerian View on E-Learning . .................................................................................................. 30
Sergio Vasquez Bronfman, ESCP-EAP (European School of Management), France
This chapter introduces some ideas of the German philosopher Martin Heidegger and how they can be
applied to e-learning design. It argues that heideggerian thinking (in particular the interpretation done
by Hubert Dreyfus) can inspire innovations in e-learning design and implementation by putting practice
at the center of knowledge creation, which in the case of professional and corporate education are real
work situations. It also points out the limits of distance learning imposed by the nature of human beings.
Furthermore, the author hope that Heidegger ideas will not only inform researchers of a better design
for e-learning projects, but also illuminate practitioners on how to design e-learning courses aimed at
bridging the gap between “knowing” and “doing.”
Chapter III
Philosophical and Epistemological Basis for Building a Quality Online Training
Methodology ......................................................................................................................................... 46
Antonio Miguel Seoane Pardo, Universidad de Salamanca, Spain
Francisco José García Peñalvo, Universidad de Salamanca, Spain
This chapter outlines the problem of laying the groundwork for building a suitable online training
methodology. In the first place, it points out that most e-learning initiatives are developed without a
defined method or an appropriate strategy. It then critically analyzes the role of the constructivist model
in relation to this problem, affirming that this explanatory framework is not a method and describing
the problems to which this confusion gives rise. Finally, it proposes a theoretical and epistemological
framework of reference for building this methodology based on Greek paideía. The authors propose that
the search for a reference model such as the one developed in ancient Greece will allow us to develop
a method based on the importance of a teaching profile “different” from traditional academic roles and
which we call “tutor.” It has many similarities to the figures in charge of monitoring learning both in
Homeric epic and Classical Greece.
Chapter IV
E-Mentoring: An Extended Practice, An Emerging Discipline ........................................................... 61
Angélica Rísquez, University of Limerick, Ireland
This chapter integrates existing literature and developments on electronic mentoring to build a constructive
view of this modality of mentoring as a qualitatively different concept from its traditional face-to-face
version. The concept of e-mentoring is introduced by looking first into the evasive notion of mentoring.
Next, some salient e-mentoring experiences are identified. The chapter goes on to note the differences
between electronic and face-to-face mentoring, and how the relationship between mentor and mentee
is modified by technology in unique and definitive ways. Readers are also presented with a collection
of best practices on design, implementation, and evaluation of e-mentoring programs. Finally, some
practice and research trends are proposed. In conclusion, the author draws an elemental distinction
between both modalities of mentoring, which defines e-mentoring as more than the defective alternative
to face-to-face contact.
Chapter V
Training Teachers for E-Learning, Beyond ICT Skills Towards Lifelong Learning
Requirements: A Case Study ................................................................................................................ 83
Olga Díez, CEAD Santa Cruz de Tenerife, Spain
This chapter describes an experience in teacher training for e-learning in the field of adult education. It
takes into account the models offered by flexible life long learning as the proper way to develop training
for teachers in service, considering the advantages of blended learning for the target audience. The chapter
discusses the balance between mere ICT skills and pedagogical competences. In this context the learning
design should always allow that the teachers in training integrate in their work ICT solutions that fit to
the didactic objectives, renew teaching and learning methodology, facilitate communication, give place
to creativity, and allow pupils to learn at their own pace. By doing so, they will be closer to the profile
of a tutor online, as a practitioner that successfully takes advantages of the virtual environments for
collaborative work and learning communication
Chapter VI
The Role of Institutional Factors in the Formation of E-Learning Practices . ..................................... 96
Ruth Halperin, London School of Economics, UK6
This chapter explores institutional and socio-organisational factors that influence the adoption and use
of learning management systems (LMS) in the context of higher education. It relies on a longitudinal
case study to demonstrate the ways in which a set of institutional and organisational factors were drawn
into the formation and shaping of e-learning practices. Factors found to figure predominantly include
institutional conventions and standards, pre-existing activities and routines, existing resources available to
the institution, and, finally, the institution’s organisational culture. The analysis further shows that socio-
organisational factors may influence e-learning implementation in various ways, as they both facilitate
and hinder the adoption of technology and its consequent use. It is argued that institutional parameters
have particular relevance in the context of hybrid modes of e-learning implementation, as they illuminate
the tensions involved in integrating technological innovation into an established system.
Chapter VII
E-Learning Value and Student Experiences: A Case Study ................................................................ 112
Krassie Petrova, Auckland University of Technology, New Zealand
Rowena Sinclair, Auckland University of Technology, New Zealand
This chapter focuses on understanding how the value of student learning and the student learning
experience could be improved given pertinent environmental and academic constraints of an e-learning
case. Believing that a better understanding of student behaviour might help course design, the chapter
revisits the outcomes of two studies of e-learning and analyses them further using a framework which
conceptualises the value of e-learning from a stakeholder perspective. The main objective of the chapter
is to identify some of the important issues and trends related to the perceived e-learning value. The
analysis of the emerging and future trends indicates that in the future blending of e-learning and face-to-
face learning is likely to occur not only along the pedagogical, but also along the technological and the
organizational dimensions of e-learning. Therefore, new blended learning and teaching models should
emphasise further the alignment of learning with work/life balance.
Chapter VIII
Integrating Technology and Research in Mathematics Education:
The Case of E-Learning ...................................................................................................................... 132
Giovannina Albano, Università di Salerno, Italy
Pier Luigi Ferrari, Università del Piemonte Orientale, Italy
This chapter is concerned with the integration of research in mathematics education and e-learning. We
provide an overview of research on learning processes related to the use of technology and a sketch
of constructive and cooperative methods and their feasibility in an e-learning platform. Moreover, we
introduce a framework for dealing with language and representations to interpret students’ behaviours
and show examples of teaching activities. Finally some opportunities for future research are outlined.
We hope to contribute to overcome the current separation between technology and educational research,
as their joint use can provide matchless opportunities for dealing with most of the learning problems
related to mathematical concepts as well as to linguistic, metacognitive, and noncognitive factors.
Chapter IX
AI Techniques for Monitoring Student Learning Process .................................................................. 149
David Camacho, Universidad Autonoma de Madrid, Spain
Álvaro Ortigosa, Universidad Autonoma de Madrid, Spain
Estrella Pulido, Universidad Autonoma de Madrid, Spain
María D. R-Moreno, Universidad de Alcalá, Spain
The evolution of new information technologies has originated new possibilities to develop pedagogical
methodologies that provide the necessary knowledge and skills in the higher education environment.
These technologies are built around the use of Internet and other new technologies, such as virtual
education, distance learning, and long-life learning. This chapter focuses on several traditional artificial
intelligence (AI) techniques, such as automated planning and scheduling, and how they can be applied
to pedagogical and educational environments. The chapter describes both the main issues related to
AI techniques and e-learning technologies, and how long-life learning processes and problems can be
represented and managed by using an AI-based approach.
Chapter X
Knowledge Discovery from E-Learning Activities............................................................................. 173
Addisson Salazar, Universidad Politécnica de Valencia, Spain
Luis Vergara, Universidad Politécnica de Valencia, Spain
This chapter presents a study applied to the analysis of the utilization of learning Web-based resources
in a virtual campus. A huge amount of historical Web log data from e-learning activities, such as e-mail
exchange, content consulting, forum participation, and chats is processed using a knowledge discovery
approach. Data mining techniques as clustering, decision rules, independent component analysis, and
neural networks, are used to search for structures or patterns in the data. The results show the detection of
learning styles of the students based on a known educational framework, and useful knowledge of global
and specific content on academic performance success and failure. From the discovered knowledge, a
set of preliminary academic management strategies to improve the e-learning system is outlined.
Chapter XI
Swarm-Based Techniques in E-Learning: Methodologies and Experiences....................................... 199
Sergio Gutiérrez, University Carlos III of Madrid, Spain
Abelardo Pardo, University Carlos III of Madrid, Spain
This chapter provides an overview of the use of swarm-intelligence techniques in the field of e-learning.
Swarm intelligence is an artificial intelligence technique inspired by the behavior of social insects. Taking
into account that the Internet connects a high number of users with a negligible delay, some of those
techniques can be combined with sociology concepts and applied to e-learning. The chapter analyzes
several of such applications and exposes their strong and weak points. The authors hope that understanding
the concepts used in the applications described in the chapter will not only inform researchers about
an emerging trend, but also provide with interesting ideas that can be applied and combined with any
e-learning system.
Chapter XII
E-Learning 2.0: The Learning Community......................................................................................... 213
Nowadays, most of electronic applications, including e-learning, are based on the Internet and the Web.
As the Web advances, applications should progress in accordance with it. People in the Internet world
have started to talk about Web 2.0. This chapter discusses how the concepts of Web 2.0 can be transferred
to e-learning. First, the new trends of the Web (Web 2.0) are introduced and the Web 2.0 technologies
are reviewed. Then, it is analysed how Web 2.0 can be transferred and applied to the learning process,
in terms of methodologies and tools, and taking into account different scenarios and roles. Next, some
good practices and recommendations for E-Learning 2.0 are described. Finally, we present our opinion,
conclusions, and proposals about the future trends driving the market.
Chapter XIII
Telematic Environments and Competition-Based Methodologies:
An Approach to Active Learning......................................................................................................... 232
This chapter provides an overview of technology-based competitive active learning. It discusses

competitive and collaborative learning and analyzes how adequate the different strategies are for different
individual learning styles. First of all, some classifications of learning styles are reviewed. Then, the
chapter discusses competitive and collaborative strategies as active learning methodologies and analyzes
their effects on students’ outcomes and feelings, according to their learning styles. Next, it shows how
networking technology can mitigate the possible negative aspects. All the discussion is supported by
significant study cases from the literature. Finally, an innovative system for active competitive and
collaborative learning is presented as an example of a telematic versatile learning system.
Chapter XIV
Open Source LMS Customization: A Moodle Statistical Control Application.................................... 250
Miguel Ángel Conde, Universidad de Salamanca, Spain
Carlos Muñoz Martín, CLAY Formación Internacional, Spain
Alberto Velasco Florines, CLAY Formación Internacional, Spain
This paper reflects the possibility of doing adaptations on a learning management system (LMS) depending
on the necessities of a company or institution. In this case, ACEM allows the definition of course-level
and platform-level reports and the automatic generation of certificates and diplomas for Moodle LMS.
These adaptations are intended to complement all the different learning platforms by contributing
added-value features like the generation of customizable diplomas and certificates and reports, which
allow the obtaining information about both grades and participation in every activity of a course. All
this necessities are not provided by default.
Chapter XV
Evaluation and Effective Learning: Strategic Use of E-Portfolio as an Alternative
Assessment at University . .................................................................................................................. 264
Nuria Hernández, Universidad de Oviedo, Spain
This chapter analyses evaluation as a strategic instrument to promote active and significant learning
and how, in that strategy, the use of alternative assessment and technology-aided learning-and-teaching
processes could be of great help. There is an important margin to allow the teachers to design the
assessment in a strategic manner and modify the nature of the students’ learning activities. So, the central
question is analysing whether the use of an electronic portfolio as an assessment tool in the subject
“International Economic Relations,” has been used strategically. In other words, is the type of desired
learning really being achieved? Is significant and deep learning being stimulated? If not, what kind of
learning is being stimulated? How should the assessment be modified to achieve the desired results?
To help answer all these questions, we have analysed whether the activities and products which make
up the “International Economic Relations” portfolio fulfil the conditions that characterise a strategic
evaluation.
Chapter XVI
Formative Online Assessment in E-Learning...................................................................................... 279
Izaskun Ibabe, University of the Basque Country, Spain
Joana Jauregizar, Quality Evaluation and Certification Agency of the Basque University System,
Spain
This chapter provides an introduction to formative assessment, especially applied within an online or
e-learning environment. The characteristics of four strategies of online formative assessment currently
most widely used—online adaptive assessment, online self-assessment, online collaborative assessment,
and portfolio—are described. References are made throughout recent research about the effectiveness
of online formative assessment for optimizing students’ learning. A case study in which a computer-
assisted assessment tool was used to design and apply self-assessment exercises is presented. The
chapter emphasizes the idea that all type of assessment needs to be conceptualized as “assessment for
learning.” Practical advices are detailed for the planning, development, implementation, and review of
quality formative online assessment.
Chapter XVII
Designing an Online Assessment in E-Learning................................................................................. 301
María José Rodríguez-Conde, Universidad de Salamanca, Spain
In this chapter we carry out analysis of the term “assessment,” applied over all the elements which
constitute the environment of formation (evaluation), and also particularizing in the assessment of the
learning process, developed in the frame of what we call e-learning. The perspective guiding text is of
a methodological and pedagogical nature. We try to plan the assessment process in online formation
environments dealing in depth with the different elements which constitute it: objectives and functions
of assessment, assessment criteria and indicators, people involved and assessment agents, software
instruments and tools for the collection of data, and analysis of the information and reports. We raise a
discussion about institutional strategies for the incorporation of this e-assessment methodology in higher
educational institutions and come to the final conclusions about the validity and appropriateness of the
e-learning assessment processes.
Chapter XVIII
Quality Assessment of E-Facilitators................................................................................................... 318
Evelyn Gullett, U21Global Graduate School for Global Leaders, Singapore
Organizations, in particular HR/Training departments, strive to set forth good practices, quality assurance,
and improvement on a continuing basis. With the continuous growth of online university programs, it is
crucial for e-learning establishments to include service quality assessments along with mechanisms to
help e-facilitators consistently maintain the highest quality standard when lecturing, teaching, guiding,
administering, and supporting the online learner. This chapter discusses the application of an e-quality
assessment matrix (e-QAM) as part of a quality assessment model that promotes continuous improvement
of the e-learning environment. This model will serve as a tool for online universities and organizations to
achieve a base standard of consistent quality that is essential for program accreditation and satisfaction
of global customers.
Chapter XIX
E-QUAL: A Proposal to Measure the Quality of E-Learning Courses . ............................................. 329
Célio Gonçalo Marques, Instituto Politécnico de Tomar, Portugal
João Noivo, Universidade do Minho, Portugal
This chapter presents a method to measure the quality of e-learning courses. An introduction is first
presented on the problematics of quality in e-learning emphasizing the importance of considering the
learners’ needs in all the development and implementation stages. Next several projects are mentioned,
which are related to quality in e-learning, and some of the most important existing models are described.
Finally, a new proposal is presented, the e-Qual model, which is structured into four areas: learning
contents, learning contexts, processes, and results. With this chapter, the authors aim, not only to draw
the attention to this complicated issue but above all to contribute to a higher credibility of e-learning
proposing a new model that stands out for its simplicity and flexibility for analyzing different pedagogical
models.
Compilation of References ............................................................................................................... 350
About the Contributors .................................................................................................................... 386
Index.................................................................................................................................................... 394
xiv
Preface
IntroductIon
Web-based training, actually known as e-learning, has experienced a remarkable evolution and growth
in the last few years. This is certainly due to enormous advances in information and communication
technologies (ICT), and also to the increasing demands to make training compatible with the profes-
sional and personal lives of any citizen, and not just something created for young students looking for a
degree. Training must be available as a lifelong experience, both for academic studies and for nonformal
or informal situations. E-learning is supposed to be an excellent solution for the old problem of mass
education, beyond that of an impractical apprenticeship method, since there are far too many knowledge
seekers and not enough knowledge providers.
The initial increase and even euphoria associated with e-learning, due to the new possibilities it
seemed to offer, gave place to a generalized feeling of disillusionment, because results did not show
e-learning to be a tool for quality training, and ROI were not really satisfactory. This was contrary to
what we one could have thought initially (García-Peñalvo & López-Eire, 2007). There exists no single
reason that can explain the failure of so many e-learning initiatives. Perhaps lack of maturity could
be the most realistic and global cause. This situation was mainly caused, among other variables, by a
pre-eminence of technological factors above other methodological or didactical elements. E-learning
started as something mainly technological, not as an activity whose aim was human learning. In fact,
most books on the subject show this unbalance clearly because human aspects are considered as if they
were unnecessary or, in many cases, because the human factor in e-learning is considered different from
any other learning modality. Consequently, the inefficiency of e-learning seemed to be due to techno-
logical elements, because the responsibility of success or failure in e-learning processes depended on
the technological tools available. This was, of course, not true. Rosenberg (2006) points out very well
this situation presenting the evolution of e-learning field in three phases. The first concerns itself with
contents, that is, with the quantity of courses, and with the investment in technology needed to deliver
them. This effort is focused on technology itself, taking as criteria for success how much you do, how
quickly you do it, and how many courses you offer. A second stage is about quality and impact factors,
and in this way success is related to innovative instructional applications, learning by doing models,
and higher cost-benefit ratios. Finally, the third phase tackles business performance to design more
comprehensive solutions that include training, improved knowledge sharing, and offer more intelligent
ways of collaboration and interaction, all in the context of work. Business measures like productivity,
customer and employee satisfaction, organizational agility, and marketplace performance are the metrics
that matter here.
The real situation is that many organizations that are bogged down in the first stage. They have intro-
duced different kinds of technology artifacts in a variety of innovative ways, and have met widely vary-
ing levels of success. Unfortunately, there are too many examples that show a very disturbing situation:
xv
these organizations do not get a reasonable relationship between investments in training and the results
they obtain. This situation presents us with “black and white” e-learning, as Martínez (2006) says.
In spite of everything, the growth of e-learning is unstoppable, and every important institution (aca-
demic, enterprise, or otherwise) knows about the necessity of creating and developing a department or
service specially devoted to this subject. E-learning deserves to be considered as real revolution, “The
Globalization of Training.” This is not only because this sort of training is given on the Internet, but also
because of the implication of entities very different from those traditionally “authorized” to do so, that
is, academic institutions. Any institution (not just academia) can plan its own training strategy, and so
learning is now possible anytime and anywhere.
Actual perspectives about e-learning initiatives are more realistic, and show a more mature concep-
tion of this field, but there is still a long way to go. The idea of “quality in e-learning” must guide us if
we want to meet successfully our educational challenges.
In order to show possibly successful ways to plan and carry out such a complex project, we are going
to study in depth the most relevant obstacles that hinder the e-learning process. After this, as a preface
to the practical knowledge and contrasted high-value experiences enclosed in the next chapters, we can
propose a complete e-learning perspective in keeping with the concept of quality in e-learning.
A FrAmework to AvoId e-LeArnIng PItFALLs
There exist quite a few works that describe a sad paradox in the deployment of e-learning systems. Many
of them are in institutions in which a learning platform is in place (more than one in many cases), but
only to be used by less than half of the teaching staff. This paradox is especially true in the context of
higher education institutions, that is to say, in universities. While it is true that some sectors demand
investments in teaching technology, trying to get equipment whose utility has been tested before it is
demanded, one can also find other institutional investments for which there is no clear need. If the teach-
ing community sees no need for these resources, it will resist using them. This is probably the cause of
the lack of interest one sees towards e-learning in the teaching staff: they do not appreciate any utility
in its use in the context of standard teaching, because institutions tend to think that “everybody knows”
what to do with these platforms. If bad comes to worse, there is a feeling that teachers will somehow
end up knowing how to use them.
Now even this is clearly something to worry about; it is by no means the only problem that precludes
a proper use of these resources. One could try to synthesize three categories in which one can group
other causes.
there is no real Intent in Institutions (“use the Platform or suffer”)
If no need has been created before deploying the e-learning platform, it is essential to do it as soon as
possible, and to do it properly. In most institutions there is a lack of a real policy as concerns ICT, and
more precisely about e-learning. Setting up a virtual campus is a much more radical change than the
incorporation of any other technology or means that has been added in a reasonable past. Using this vir-
tual campus means a real shift in the training paradigm. Hence, on must prepare for this change, and for
that it is necessary to develop specific policies about e-learning, with a clearly defined strategic model.
The proper policy concerning e-learning must be complemented with investments in human resources,
in technology, and in methodology. Without this trio of elements, the tool itself is pointless, which is
the worst possible outcome in training terms.
xvi
users are Alone
Any teacher that decides he or she is going to make use of an online training system, be it out of curiosity
or just as a personal challenge, is going to meet a whole range of problems when trying to work things
out just by himself. Which methodology should I use? Who will help me to create materials? How is this
evaluated? Who will solve technical problems for me? How could I make this platform supply this or
that need that I have in the subject I teach? Who will help me tutor if I have about 200 students? Many
of these questions find no answer. The teacher, who so far was able to handle his class and managed to
fulfill his duties, meets quite a few new tasks for which he has no training, and perhaps this lack is not
his or her fault. E-learning necessitates many support services for teaching; without them, the teacher’s
job is severely limited, and consequently any formative possibilities are lost.
there is no recognition for the teaching effort needed for any online Action
There are two rather common fallacies among those who know little about e-learning. One of them is that
e-learning is virtual, that is to say, that it is a subproduct of training and not “real” training like presential
teaching. The other is that any activity derived to an e-learning platform frees the teacher from a part
of his or her duties, thus reducing the teacher’s dedication. Nothing could be further from the truth, as
is well known to those who are dedicated to online teaching. Rather on the contrary, correctly helping a
group of students in the context of an e-learning methodology certainly enhances the trainee’s experience,
but it tends to increase remarkably the amount of time that the teacher must invest in teaching tasks, in
formative training, and in tutorial activity. Regrettably, as a consequence of these prejudices, teachers
(and this is especially true in university contexts) are “penalized” when using e-learning as a complement
to their teaching activity. If they opt for meeting the challenge, they will get exactly no recognition in
academic or economic terms. A large amount of time will have to be dedicated to this “silent” teaching
effort, and the rest of the community will take no notice. Since everything happens in a “virtual” context,
there will be no visible tracks left, no classroom or lab reservations. Any time dedicated to this job by
the teacher is considered “virtual” in all respects. But his time is all too real.
This type of situations, which have a most negative impact, should move any organizations that have
an interest in online teaching towards the adoption of a strategic policy that will fulfill the requirements
of a society that wants and needs information and knowledge in a flexible context. This society, however,
is fairly strict as concerns the quality of the product it is going to consume. The context in Europe is
expressed quite clearly in the definition of the European Higher Education Space (European Ministers
of Education, 1999) which is definitely in favor of a lifelong training, since this will contribute to the
improvement of the citizens’ opportunities according to their aspirations and abilities, and consequently
enhance their personal, social, and professional development (Cieza, 2006).
Any ad hoc solutions for this situation are bound to produce a small and not very positive return
on our investments. Any attempts to make serious use of e-learning should be strategic, in such a way
that the deployment of an e-learning platform must be one of the visible vertices in a polyedric set of
measures. These must constitute a whole strategic plan, which will affect training of course, but also
research, services, administration, and even the management and leadership of universities. If this is
not done in such a way, one will face the risk of having to redo part of the job if it was initiated in an
erratic way through lack of foresight, or one can reach a state of rigidity in the electronic “structure,”
thus producing a fragmentation that would be harmful since it would keep apart organs of the institu-
tions that should be perfectly well coordinated. The strategic foundations, which an institution must use
when trying to adopt a policy for the deployment of an e-learning structure, can and should be based on
the concept of “quality in e-learning.”
xvii
QuALIty In e-LeArnIng
Before talking about quality in e-learning, one must define what we exactly mean when we refer to e-
learning. The application of Web-based tools for learning purposes could be considered a simple definition
of e-learning. However, a clearer e-learning definition, including a conceptualization of its modalities,
is the best starting point in order to understand the quality reference framework on which we would like
to develop this book. Hence, one could define e-learning as:
a teaching-to-learning process aimed to obtain a set of skills and competences from students, trying
to ensure the highest quality to the whole process, thanks to, mainly, the use of Web-based technologies,
a set of sequenced and structured contents based upon pre-defined but flexible strategies, the interaction
with the group of students and tutors, the appropriate evaluation procedures, both of learning results and
the whole learning process, a collaborative working environment with space-and-time deferred presence,
and finally a sum of value added technological services in order to achieve maximum interaction.
It is quite common to associate adjectives like “virtual” or “distance” to “learning,” in order to build
synonyms for “e-learning.” But it is important to clarify that we are not thinking about virtual learn-
ing or distance learning when we refer to e-learning, at least not necessarily. When we try to develop a
quality e-learning initiative, the development of skills and knowledge is easier to demonstrate than in a
traditional or presential context. So if we consider “virtual” as the opposite term of “real,” e-learning is
just real and not virtual learning. But, from a philosophical point of view, virtual is “all that can induce
an effect.” If we consider that e-learning is different from many other forms of “learning” because of
its active approach, it is clearly “virtual”; that is to say, it has the virtuality to “create” and not only to
“assume” knowledge and skills. Concerning distance learning, it’s a common mistake considering e-
learning as a form of distance learning, and applying its methods and categories to e-learning the results
will be really poor. This is because e-learning is not nonpresential like distance learning is. The actors
in this process are present, on a different time and a different place, but their presence is verifiable, and
they leave certain tracks. So e-learning is more than distance learning, and this is because of the human
presence behind the technology, the net, and the computers.
One of the main issues in e-learning (and of course in every learning experience, as for any product
or service), is the notion of “quality.” This concept, in fact, does not belong exclusively to the universe
of industry and economics. The academic world is fairly used to the need to measure certain items in
order to determine quality in their learning processes.
Quality in e-learning has a twofold significance. First, e-learning is associated in many discussion
papers and plans with an increase in the quality of educational opportunities, ensuring that the shift to
the information society is more successful. This context is named “quality through e-learning.” Second,
there is a separate but associated debate about ways of improving the quality of e-learning itself; this
context is called “quality for e-learning” (Ehlers, Goertz, Hildebrandt, & Pawlowski, 2005).
Learning outcomes are at the heart of respondents’ understanding of quality in the field of e-learning.
When we talk about quality in e-learning, we assume an implicit consensus about the term “quality.” The
ISO (ISO 8402, 1986, p. 3.1) defines quality as follows: “The totality of features and characteristics of a
product or service that bear on its ability to satisfy stated or implied needs.” In fact, however, “quality”
means very different things to most e-learning providers. García-Peñalvo (2006) points out five factors:
technology, services, evaluation/accreditation, contents, and human factor (tutoring). Harvey and Green
(2000) have suggested the following set of categories: exceptionality, perfection or consistency, fitness
for purpose, and adequate return. Ehlers (2004) adds a fifth category, transformation, which describes
the increase in competence or ability as a result of the learning process as transformation.
xviii
Matching these ideas, we can define quality in e-learning as:

the effective acquisition of a suit of skills, knowledge and competences by students, by means of
developing appropriate learning contents given with a sum of efficient Web tools supported via a net
of value-added services, whose process—from content developing to the acquisition of competences
and the analysis of the whole intervention—is ensured by an exhaustive and personalized evaluation
and certification process, and it is monitored by a human team practicing a strong and integral tutorial
presence through the whole teaching-to-learning process.
orgAnIzAtIon oF the Book
In a few words, the idea behind this book is that a quality e-learning process is much more than tech-
nology. Technical issues will have an important place in this book, of course, but the whole question
must be considered within other issues such as pedagogical, methodological, tutorial, evaluation, com-
munication, strategic, and so on.
Advances in E-Learning: Experiences and Methodologies is addressed to any scholar, technical,
academic, or manager that could play a role in the field of e-learning, so the public is extremely het-
erogeneous. In fact, it is difficult to determine a field of knowing or activity, because any field and any
professional role could be potentially interested on e-learning because of its enormous capabilities ap-
plicable to institutions, schools, universities, enterprises, associations, and so forth. Above all, it will
not give a restricted vision about e-learning, but a multidisciplinary, rich, and complete analysis of the
different issues involved, intending to become a reference on e-learning literature because the different
issues will not be studied as separate matters, but any question related to e-learning studied in this book
will be pointed to get the highest quality in e-learning activities.
In fact, the book is organized into nineteen chapters. A brief description of each of the chapters fol-
lows:
In Chapter I, Ray Webster presents RAPAD, a reflective and participatory methodology for e-learning
and lifelong learning. It is a proposal of an adaptive method where students can participate with peers,
developers, teachers, and trainers to think about their learning, discuss it, and apply their thoughts to
the design and development of Web sites which can serve as Personalized E-Learning Environments
(PELE), promoting a deep understanding of learning on a metacognitive and personal level.
Chapter II introduces some ideas of the German philosopher Martin Heidegger and how they can be
applied to e-learning design. This approach argues that practice must be the center of knowledge creation,
which in the case of professional and corporate education is a real work situation. The chapter has been
written by one of the most renowned e-learning consultants in the world, Dr. Sergio Vásquez.
Following with the philosophical approaches, Chapter III by professors Seoane-Pardo and García-Pe-
ñalvo, outlines the background concepts in order to construct a human-centered methodology for online
training. This chapter analyzes in a critical way the constructivism paradigm, stating that this framework
is not a method and explaining the problems that are derived from this confusion.
Chapter IV, by Angelica Rísquez, addresses the issue of mentoring in the online teaching as a quali-
tatively different concept from its traditional face-to-face version, and how the relationship between
mentor and mentee is modified by technology in unique and definitive ways. The chapter introduces a
set of best practices on design, implementation, and evaluation of e-mentoring programs.
In Chapter V, Dr. Olga Díez deals with the issue of lifelong learning and describes an experience in
teacher training for e-learning in the field of adult education. The chapter discusses the balance between
mere ICT skills and pedagogical competences. The author argues that the learning design should always
xix
allow that the teachers in training integrate in their work ICT solutions that fit to the didactic objectives,
renew teaching and learning methodology, facilitate communication, give place to creativity and allow
pupils to learn at their own pace.
Chapter 6VI is about institutional and socio-organizational factors that influence the adoption and
use of Learning Management Systems in higher education institutions. Ruth Halperin presents a hybrid
e-learning case study to explore these factors, where institutional parameters have particular relevance
underlining the tensions involved in integrating technological innovation into an established system.
Krassie Petrova and Rowena Sinclair focus Chapter VII on understanding how the quality of student
learning and the student learning experience could be improved given the pertinent environmental and
academic constraints of an e-learning case. The main objective of the chapter is to identify some of the
important issues and trends related to the perceived e-learning value. They state that new blended learn-
ing and teaching models should emphasize further the alignment of learning with work/life balance.
Chapter VIII, by Giovannina Albano and Pier Luigi Ferrari, provides an overview of research on
learning processes related to the use of technology and a sketch of constructive and cooperative methods
and their feasibility in an e-learning platform in the Mathematics education context.
David Camacho et al. describe in Chapter IX both the main issues related with artificial intelligent
(AI) techniques and e-learning technologies, and how lifelong learning processes and problems can be
represented and managed by using an AI-based approach in order to implement a group-based adaptation
based on the actions not of an individual student but of a set of students who have accessed the system
along a period of time.
Chapter X shows a study applied to the analysis of the utilization of learning Web-based resources
in a virtual campus. The authors, Addisson Salazar and Luis Vergara, use this case study to detect of
and specific content on academic performance success and failure.
In one of most computationally-oriented chapters of this book, Sergio Gutiérrez and Abelardo Pardo
describe, in Chapter XI, the use of swarm-intelligence techniques in the field of e-learning, analyzing
several of such applications and expose their strong and weak points. Swarm intelligence is an AI tech-
nique inspired by the behavior of social insects. Taking into account that the Internet connects a high
number of users with a negligible delay, some of those techniques can be combined with sociology
concepts and applied to e-learning.
Chapter XII is devoted to Web 2.0 applied to the e-learning area. Luisa Mª Regueras et al. pres-
ent how this technology movement can be transferred and applied to the learning process, in terms of
methodologies and tools, and taking into account different scenarios and roles in order to emphasize
the collaborative way of learning.
As an example of the ideas expressed in the chapter before, in Chapter XIII Elena Verdú et al. discuss
about competitive and collaborative learning; they analyze how adequate the different strategies are for
different individual learning styles, all of them in an active learning context. The ideas are supported by
a case study and an active learning system.
Chapter XIV presents a report system plug-in for Moodle developed by Clay Formación Interna-
cional Team. It presents the possibility of doing adaptations on a LMS depending on the necessities of
an institution. This is an interesting example of how combine the Open Software ideas into a enterprise
context.
Nuria Hernández analyzes in Chapter XV evaluation as a strategic instrument to promote active and
significant learning. Inside of this strategy, the author argues that an electronic portfolio as assessment
element will be able to help the student to generate suitable learning.
xx
Chapter XVI presents a very valuable state of art of the formative assessment in e-learning-based
systems. Izaskun Ibabe and Joana Jauregizar describe the four most used strategies for online formative
assessment: online adaptive assessment, online self-assessment, online collaborative assessment, and
portfolio. Through a case study, they argue that all type of assessment needs to be conceptualized as
“assessment for learning.”
In Chapter XVII, which is related to the previous one, Dr. Mª José Rodríguez-Conde analyzes the as-
sessment term, applied over all the elements which constitute the environment of formation (evaluation),
and also particularizing in the assessment of the learning process, developed in the frame of e-learning.
The most interesting part of this chapter presents a high valuable discussion about institutional strategies
for the incorporation of this e-assessment methodology in higher educational organizations.
Dr. Evelyn Gullett discusses in Chapter XVIII the application of an e-quality assessment matrix
(e-QAM) as part of a quality assessment model that promotes continuous improvement of the e-learn-
ing environment. This model must be a reference tool for organizations to achieve a base standard of
consistent quality that is essential for program accreditation and satisfaction.
In the last chapter, Célio Gonçalo Marques and João Noivo introduce a method to measure the quality
of e-learning courses. They present a new quality reference model, e-Qual model, which is derived from
the analysis of reference frameworks developed through international projects. E-Qual is very flexible
in order to adapt itself to the evaluator’s perspective (learners, producers, and distributors) and to the
contents and contexts perspective.
reFerences
Cieza, J. A. (2006). E-learning factors. A lifelong learning challenge inside the European space for higher
education framework. In F. J. García, J. Lozano & F. Lamamie de Clairac (Eds.), Virtual campus 2006
post-proceedings. Selected and extended Papers–VC’2006, CEUR Workshop Proceedings. Retrieved
November 1, 2007, from http://CEUR-WS.org/Vol-186/
Ehlers, U. -D. (2004). Qualität im e-learning aus lernersicht: Grundlagen, empirie und modellkonzep-
tion subjektiver qualität. Wiesbaden: VS Verlag.
Ehlers, U. -D., Goertz, L., Hildebrandt, B., & Pawlowski, J. M. (2005). Quality in e-learning. Use and
dissemination of quality approaches in European e-learning. A study by the European Quality Obser-
vatory. Cedefop Panorama series, 116. Luxembourg: Office for Official Publications of the European
Communities.
European Ministers of Education. (1999, June 19). The European higher education area - Bologna
declaration, Bologna.
García-Peñalvo, F. J. (2006). Introducción al eLearning. In F. J. García-Peñalvo et al. (Eds.), Profesiones
emergentes: Especialista en eLearning. Salamanca, Spain: Clay Formación Internacional.
García-Peñalvo, F. J., & López-Eire, A. (2007). Successful e-learning case studies in Spanish University.
Journal of Cases on Information Technology (JCIT), 9(2), 1-3.
Harvey, L., & Green, D. (2000) Qualität definieren: fünf unterschiedliche ansätze. Zeitschrift für Päda-
gogik: Qualität und Qualitätssicherung im Bildungsbereich: Schule, Sozialpädagogik, Hochschule, 41,
17-39.
xxi
ISO. (1986). Quality–Vocabulary. ISO 8402. Geneva: International Organization for Standardization.
Martínez, J. (2006). E-learning en blanco y negro. Learning Review, 14.
Rosenberg, M. J. (2006). Beyond e-learning. Approaches and technologies to enhance organizational
knowledge, learning, and performance. San Francisco, CA: Pfeiffer.
xxii
Acknowledgment
It is imperative to begin these few lines with my special thanks to the authors and reviewers of every
chapter, whose labour and dedication where so remarkeable as to make it easy to complete this work.
I am equally grateful to those who helped with the blind review process, without whom it would be
impossible to achieve a book of this caliber. But my special thanks in reviewing go to my colleagues of
the Researching Group on InterAction and eLearning (GRIAL) who gave their time and effort to provide
constructive and comprehensive feedback extremely useful to finish this work. They include Valentina
Zangrando and Antonio Seoane, who helped me with the final revision of the entire book.
I would also like to thank the editor Jessica Thompson for her efficiency and generosity in working
with us, and the publishing team of IGI Global for their competence and expertise.
Finally, I express my gratitude to the Education and Science Ministry of Spain, National Program in
Technologies and Services for the Information Society, since this book has been developed inside the
KEOPS research project context (Ref.: TSI2005-00960) financed by the Govern of Spain.
Francisco José García Peñalvo

University of Salamanca

Chapter I
RAPAD:
A Reflective and Participatory Methodology
for E-learning and Lifelong Learning
Ray Webster
Murdoch University, Australia
ABstrAct
This chapter introduces RAPAD, a reflective and participatory methodology for e-learning and lifelong
learning. It argues that by engaging in a reflective and participatory design process for a personalized
e-learning environment, individual students can attain a conceptual change in understanding the learn-
ing and e-learning process, especially their own. Students use a framework provided by the concept of
a personal cognitive or learning profile and the design and development of a personalized e-learning
environment (PELE) to engage with key aspects of their learning. This results in Flexible Student Align-
ment, a process by which students are better able to match their learning and e-learning characteristics
and requirements to the practices, resources, and structures of universities in the emerging knowledge
society. The use of Web-based technologies and personal reflection ensure that RAPAD is well-placed
to be an adaptive methodology which continues to enhance the process of lifelong learning.
IntroductIon ing, discuss it, and apply their thoughts to the

design and development of Web sites which can
This chapter describes a reflective and participa- serve as personalized e-learning environments
tory methodology for the design of personalized (PELE). This process, RAPAD, is a methodology
virtual e-learning environments—reflective for enhancing e-learning and lifelong learning
and participatory approach to design (RAPAD) because it promotes a deep understanding of
(Webster, 2005). With RAPAD, students and us- learning on a metacognitive and personal level.
ers reflect and participate with peers, developers, The metacognitive and self-regulatory improve-
teachers, and trainers to think about their learn- ments brought about by using RAPAD causes a
Copyright © 2008, IGI Global, distributing in print or electronic forms without written permission of IGI Global is prohibited.
RAPAD
conceptual shift in the understanding and applica- chapter, are introduced. This is followed by a
tion of each individual’s attitudes to personalized background section which discusses the need for
learning. Enabling this conceptual shift is seen as new and personalised approaches for supporting e-
a necessary prerequisite for improving the qual- learning. The next sections consider the changing
ity of student learning (Vermetten, Vermunt, & conceptions of learning, discuss the complexity
Lodewijks, 2002). The quality of student learn- of learning, and, in order to provide a coherent
ing is of central importance in the transition to a overview of the work, offer a systems perspec-
knowledge-based economy. Because of the strong tive of the student, methodology, and PELE as
links between education, training, and the needs learning system. The concept of Flexible Student
of knowledge workers in industry and commerce, Alignment (Webster, 2005), which is partially
participatory methodologies like RAPAD can enabled by taking a systems perspective, is then
become very important mechanisms for devel- introduced before the need for human-centred
oping e-learners and lifelong learners for the e-learning systems design and participatory de-
Knowledge Society. sign (as an example of a human-centred design
As a reflective and participatory methodology, methodology) are outlined.
RAPAD provides a framework and set of proce- The development of RAPAD as a participatory
dures to enable each individual to understand his methodology is then summarized. This is followed
or her learning preferences and thus enhances by a broad description of the research phases and
e-learning and lifelong learning. Two core mecha- empirical work which comprised the development
nisms are used within RAPAD to strengthen the of RAPAD as an e-learning methodology. Future
reflective and participatory process. These are trends are then suggested before conclusions are
the cognitive profile and the personalised e-learn- drawn and the chapter is summarized.
ing environment (PELE). Using the concept of a
cognitive profile enables the personalisation of Definitions
the PELE by structured reflection on individual
learning related characteristics. The cognitive The reflective and participatory approach to
profile, as used here, consists of measures of design is an iterative process in which key ele-
each student’s cognitive style, learning style, and ments include the student as a codesigner in the
personality type. This reflects the assertion that it production of a system or PELE. The method is
is the combination of these three measures which used as a mechanism to help each student ac-
best reflects each “individual’s combination of quire the self-regulatory skills associated with
aptitude/trait strengths and weaknesses” in terms autonomous learning. The methodology provides
of learning (Jonassen & Grabowski, 1993, pxii). a conceptual framework of structure and process
Students undertake a series of profile associated for the student to function within.
tests at the start of the exercise and are given their The next section briefly introduces some key
results. They then discuss, reflect, and comment terms. The terms are defined with reference to RA-
on those results before using them in designing PAD and their use in that context is explained.
their PELE. Designing and developing the PELE
with specific reference to the personal learning Reflective
profile gives both a context and a focus to the
development of the e-learning support system The term “reflective” as used here derives from
that the PELE represents. Schön’s (1983, 1991) use of the term in both the
The structure of the chapter is as follows. phrase and the sense of “a reflective practitio-
Several definitions and key terms, as used in this ner.” Schön considered that many professionals

RAPAD
in fields such as law, engineering, architecture, centred approaches in that the user is a partner in
and medicine, developed and consolidated their the development process rather than the client of
learning by reflecting on their practice and also it, a key difference in terms of involvement. As
reflecting in the performance of their practice. the name suggests, a participatory approach is
Schön (1987) then applied the concept of learn- introduced to encourage deep user involvement
ing by doing and continuing to learn through in the design process (Preece et al., 2002). The
reflection and problem solving to education. It result is that both the system designer (or design
is considered that students need to develop as team) and the user(s) can benefit from and learn
reflective practitioners with respect to their own from each other.
learning. The purpose is to help them to function
and participate effectively in a systemically dif- Cognitive Profiles
ferent system of higher education.
The reflective and participatory approach was
design operationalized by the use of student cognitive
profiles, which were applied to the design of the
The term “design” comes before “participatory” PELE. This method of profiling gave the students
because the latter term is a subset of the concepts a framework in which to structure and apply their
encapsulated within the term “design.” Design is reflection. A cognitive profile is made up of three
used in the sense of Systems Analysis and De- core elements: measures of the student’s cogni-
sign which, at a conceptual level, derives from tive style, learning style, and personality type
systems theory (Checkland & Holwell, 1998) (Jonassen & Grabowski, 1993). The cognitive
and uses associated concepts to understand the style measure chosen was Riding’s Cognitive
activity of all types of systems. The term is most Styles Analysis—CSA, (Riding, 1991, 2001). This
often used in the context of information systems comprises a computer-based test which measures
design and development and has becomes almost personal preferences for representing and process-
synonymous with information and communica- ing information. The learning styles instrument
tions technology based systems design. It is used was the Approaches to Study Skills Inventory for
in a broader manner in this work, as the research Students—ASSIST, (Tait et al., 1998; Entwistle et
considers the design of learning environments al., 2000). This instrument measures deep, surface
from a systems perspective. The systemic per- and strategic approaches to learning, with each
spective can embrace and contain the systematic approach containing several sub-categories. The
and analytic methodology associated with much personality instrument was the Myers-Briggs
systems analysis and design, while the reverse Type Indicator—MBTI, (Myers et al., 1999) a
is rarely true. widely used instrument for measuring personality
type. As with most instruments in this area, there
Participatory is continuing debate concerning reliability and
validity (Bayne, 1995; Nowak, 1997; Peterson et
“Participatory design” is a phrase used in the al., 2003; Coffield et al., 2004). Although care was
information and technology design fields to taken to choose well tried, tested and widely used
indicate the very close and full participation of instruments, this was considered less critical for
the system users in the process of the design and the purposes of this project than for experimental
development (and testing and implementation research designs as the measures were being used
and review) of the system in question (Preece, as a framework for reflection and design rather
Rogers, & Sharp, 2002). It differs from other user- than for the purposes of category labeling. The

RAPAD
students could disagree with the results but had A personalised e- learning environment (PELE)
to say why, explain which category and learning is a system which is designed to support the
traits they considered correct and use these new information retrieval, information handling,
criteria as part of the reflective design process. and learning support needs of the student. In its
The students took the three tests at the start of entirety, the PELE system which is developed as
the process. After some discussion of the ideas a mobile (laptop, server, organiser, phone, flash
and concepts involved, they were given the test drive) based Web site to replicate as many of the
results and asked to write about them in reflective Learning Resource Centre functions as possible.
journals. The involvement of the students in re- These functions can include: Learning Support,
flecting on their own responses and then applying Study Skills, Media Services, IT Support (Ad-
them to the learning environment design formed ministrative), IT Support (Academic), Learning
a central part of the application of RAPAD. Resources, and Career Services. The PELE should
allow the student to store, retrieve, and manipulate
Learning Environment information from internal sources (hard drives,
digital documents, and images, and so forth) and
A general definition of a learning environment external sources (Internet, WWW, etc.).
was provided by Wilson, who suggested that a
learning environment is “a place where people can The use of the three measures plus an itera-
draw upon resources to make sense out of things tive process of discussion, design, and feedback
and construct meaningful solutions to problems” provided a more holistic and systemic methodol-
(Wilson, 1996, p.3). ogy for the design and development of the PELE.
A more specific definition of the term “learn- The context and overview of the Reflective and
ing environment,” which was provided within the Participatory Approach to Design is shown in
context of the management of change in higher Figure 1.
education in general and universities in particular,
is “a learning environment is a community with
its own culture and values providing a variety of BAckground
learnplaces that support student learning” (Ford
et al., 1996, p.146). A personalized e-learning environment (PELE)
This second definition was adapted to describe is a virtual learning environment which acts as
the concept of the personalised e-learning envi- an interface to learning resources as well as to
ronment used in this research in the following other learning systems and environments. The
way: a personalised e-learning environment is process of developing the PELE is seen as a way
a Web-based virtual environment reflecting the of enabling students to develop as autonomous
culture and values of the individual student and learners in that it helps them to think about their
providing links to a variety of possible learning own learning in a structured manner. This is
communities and a learnplace that supports au- considered to be a prerequisite for students in
tonomous student learning. a system of mass higher education in which the
concept of e-learning as the basis for active and
Personalized e-Learning resource-based learning is often promoted but
Environment (PELE) not explained. The associated personal activi-
ties—how to function at an individual level and
The practical definition of the e-learning environ- as an active e-learner in a resource based e-learn-
ment as given to the students for design purposes ing environment—can remain something of a
was in the following form:

RAPAD
mystery to the new student. By using a reflec- perience, a large number of factors have changed
tive and participatory approach to design, the dramatically from even 10 and certainly 20 years
interface concept can be extended to encourage ago; especially in the OECD countries (many
students to contemplate how they interface with of these changes occurred earlier in the USA,
learning materials, learning processes, and learn- although the Web related changes are similar
ing environments, including the university and for all). The number of students has increased
its associated subsystems. As we move towards significantly. The number of academic staff has
the processes and practices of the Knowledge stayed largely the same resulting in increased
Society, using RAPAD and developing a PELE staff-student ratios and the need for online and
also helps develop the reflective, metacognitive, distributed learning resources (DfES, 2003). The
and self-regulatory skills necessary for Lifelong backgrounds of the students have become more
Learning. varied, with some universities having more than
50% mature students (Laurillard, 2002).
the need for new Approaches for A continuing problem with the current scenario
supporting e-Learning in higher education is that although there may
have been a much expanded student intake, with
Recent changes in higher education have produced the move to a mass system, many of the processes
a set of circumstances that need a new approach and practices in use are those developed for an
to supporting and enabling student learning. instruction-based elite system and the introduc-
The development methodologies for e-learning tion of e-learning systems and activities. While
systems, whether they be human centred or many of the traditional procedures and systems
techno-centred, will play a central role the new will remain useful and relevant, we have to ensure
approaches which emerge. Although learning that those in use are suitable for functioning ef-
remains central to all students’ educational ex- fectively within the resources and constraints of a
Figure 1. Overview: reflective and participatory approach to design (Webster, 2005)

RAPAD
mass system. Some central processes and practices and teaching with e-learning environments plus
(forms of assessment, tutorials which functioned the drive towards personalized learning being
effectively with 8 participants but struggle with 16 experienced in OECD countries. This personaliza-
to 20, personal tutoring) are increasingly under- tion is reflected in the quotations directly below
resourced and under strain. from U.S. educationalists and UK and Australian
In addition, a perception has developed, espe- politicians. The key assertion is that education in
cially amongst higher education managers, that general and higher education in particular are mov-
the provision of information and communication ing into an era of personalized learning. Metros
technologies will, by themselves, provide useful and Bennett (2002), echoing Twigg (1994), also
and cost saving solutions. This approach often go further in identifying the central role of using
misses the point that the learning systems we cognitive profiles to enable this personalization.
are concerned with are social systems of which This is a central element RAPAD.
technology is only one aspect, often acting simply “Personalized learning can become a reality
as an information carrier or interaction enabler. when a learner’s profile, determined by prelimi-
The central and most important component nary assessment, is used to structure and sequence
remains the student. Laurillard (2002, p.145) the learning components” (USA—Metros &
quotes Carol Twigg’s suggestion that an increased Bennett, 2002).
understanding of how individuals learn has its “The key strategy is personalised learning”
corollary in that “increased individualization of (Australia—Bishop, 2006).
the learning process is the way to respond to the “A mission to realise the full potential of each
diverse learning styles brought by our students” young person through a system of education in-
(Twigg, 1994, p.1). creasingly personalised around the needs of each
Technology and e-learning systems offer inno- child, with a new concept of lifelong learning”
vative ways of reconceptualising our approaches (UK—Blair, 2004).
to learning and teaching delivery systems, but In this scenario, attempts are made to match
learning itself remains is the central and human the learning experience of the student with his
component of any e-learning system. One way or her learning needs on an individual basis.
of rethinking learning and e-learning support The Web, e-learning methodologies, and their
can be to develop the metacognitive skills of the integration as e-learning systems will play a key
individual student by using individual cognitive role in these developments.
profiles to help construct personal interfaces for
interacting with e-learning environments. The changing conceptions of Learning
need for students to become more actively involved and e-Learning
in the management of their own learning implies
an associated need for each student to be more Our conceptions and understandings of learning
aware of and to increasingly draw on his or her and the learning process have been steadily chang-
personal resources, including the components of ing over the past two or three decades. In this last
his or her cognitive profile. decade, the pace of change has perhaps increased
in response to the central facets of massification
the need for Personalized impacting more fully and more consistently on
E-Learning Environments university teaching. The second edition (2002) of
Laurillard’s influential text Rethinking University
The focus for this work comes from a combina- Teaching shows subtle changes of emphasis that
tion of observed personal experience of learning reflect the shift in focus within the sector. For

RAPAD
example, whereas Laurillard has Marton and teaching and, by inference, student learning.
Ramsden (1988) listing “teaching strategies” in the These tenets are that learning is active, social,
first edition (Laurillard, 1993, p. 82), in the second reflective, and occurs in a context. This concurs
edition they list “implications for the design of a with Goodyear (2001) who considers learning
learning session” (Laurillard, 2002, p. 69). The from a cognitive perspective through the lens of
subtitle of the second edition also shows a shift in Shuell’s (1992) work. In this framework, learn-
emphasis from the use of the phrase “educational ing can be conceptualised as passive reception,
technology” to “learning technologies.” discovery, knowledge deficit and accrual, guided
These changes, while minor in quotations instruction, with this last form fitting “best with
from a given text, represent a more substantial current scientific ideas about learning” (Goodyear,
shift in our thinking in the relationship between 2001, p.71). Within this model, the significant
teaching and learning and, as a subtext, the role elements of learning are then formulated as ac-
of learning technologies in that relationship. A tive, individual, cumulative, self-regulated, and
further requirement is presented by the need goal oriented.
to make sense of the plethora of terms used to A mode of implementation for these ap-
describe different “types” of learning—distance proaches is put forward by Simons et al. (2000,
learning, active learning, e-learning, resource p. 9), who suggest that “new instruction should be
based learning, student centred learning, self aiming for the new outcomes of learning through
regulated learning, networked learning. Unless the facilitation of the new learning processes
academics and university teachers have a clear and strategies in which a new balance between
appreciation of the form and content of the pro- guided learning, experiential learning and action
cess that constitutes student learning, it will be learning occurs.”
difficult for them to make sense of the variety A major consideration of these models and
of approaches to learning confronting them in perspectives is that each suggests that the design
their professional life. However, Laurillard (1999, of systems for learning needs to be a systemic
p. 113) did suggest that “it is difficult to find an as well as systematic process. The systemic per-
academic with a theory of learning. Or even one spective then logically holds for the individual,
who thinks it is his job to have one.” This point group, or organisational level and takes the fac-
and related issues were well explored in a paper tors into consideration. An example of elements
from the same conference (Banathy, 1999). With of this systemic and systematic approach is more
reference to systems thinking and change in fully contextualised and presented by Goodyear
higher education, this author used a hypothetical (2002, p.11). It is a point that has been made quite
conversation between “a subject-matter professor strongly by several authors in recent times (Ford
and a systems thinker” (Banathy, 1999, p.133). et al., 1996; Knight, 2001; Trowler, Saunders,
The paper, while illustrating Laurillard’s point, & Knight, 2003; Weil, 1999) and has resonance
also provided an accessible systems based com- with the writings on both organisational and
mentary and analysis on the differences between educational change of Argyris and Schön (1996)
learning and instruction focused approaches to and Checkland (1990).
higher education.
In considering the role of learning technologies Learning is complex
in the teaching and learning relationship, Driscoll
(2002) asserts that there are four basic tenets In the case of learning itself, dictionaries often
that need to be considered when we, as teachers, provide a simple definition of the phrase “to
think about the use of technology to support our learn.” For example, the Shorter Oxford English

RAPAD
Dictionary (3rd ed.) offers: “To get knowledge of is commented on at greater length in another
(a subject) or skill in (an art, etc.) by study, experi- section. Within this context, a further definition
ence, or teaching” (Onions, 1983, p. 1191). emphasises that learning is also an active process
A more problematic issue, referred to by and one to which we are well suited: “Learning
Driscoll’s principles (Driscoll, 2002) is that of is a basic, adaptive function of humans. More
understanding how we learn, or in a more complex than any other species, people are designed to
way, how we move from gathering information be flexible learners and active agents in acquir-
about something to gaining an understanding of ing knowledge and skills” (Bransford, Brown, &
that information within our own social, affec- Cocking, 1999, p.45).
tive, and cognitive domains. The roles of oth- As with learning, there are many forms and
ers—parents, friends, peer groups and, especially, phrases to describe e-learning. If we accept that
teachers—are important here. Additionally, and the “learning” part of e-learning is effectively
within the framework of this discussion, it is seen encapsulated in the above quote, then we con-
as important that the individual student gains an sider that Goodyear (2005) provided an exten-
appreciation of how he or she learns or acquires sion and clarification of the term “e-learning”
that understanding. When, with reference to pro- which emphasises the learning aspects when he
fessional learning, Trowler and Knight (2000, p. suggested that:
37) state that “much professional learning is social, The terms e-learning, Web-based learning
provisional, situated, contingent, constructed and and online learning now have wide currency in
cultural in nature,” it follows that this is also true education. I use the term networked learning to
of student learning. It might not be necessary, mean a distinctive version of these approaches. I
possible, or even desirable to try to explain all define networked learning as: “learning in which
of these aspects to new university entrants, but ICT is used to promote connections: between one
some knowledge of an individual’s own learning learner and other learners; between learners and
processes and how to use them effectively has to tutors; between a learning community and its
be a useful resource for each student. learning resources.” (Goodyear, 2005)
One reason for this is that the types of learn-
ing engaged with in higher education are more
complex than those encountered at school (Knight, A systems PersPectIve oF
2001). This is true both of the types of learning rAPAd And PeLe
in themselves and the social and organisational
setting in which many undergraduates find them- This section comprises an overview of the sys-
selves as they emerge into adulthood. Comment- tems approach to the problem and how it affected
ing within the context of considering the process the development of RAPAD as a methodology
of curriculum-making, Knight states that “it is and PELE as a system. The systems paradigm
this complexity that especially distinguishes or systems inquiry is an approach which uses
university study from school study” (Knight, the elements and organisation of systems theory
2001, p. 369). (the core transformation at the conceptual level,
It is now widely accepted that an important part hierarchy, system boundary, environment, etc.)
of the learning process is that each of us builds as a lens for investigating student learning and
or constructs new knowledge on the basis of the e-learning system design in higher education.
existing knowledge (Goodyear, 2002; Knight The approach encourages us to be systemic as
& Trowler, 2001; Simons et al., 2000; Vermunt, well as systematic.
1998). This is the “constructivist” paradigm which

RAPAD
Ontologically, systems philosophy takes a temic and systematic viewpoints allows analysis
systems view of the world and thus provides a to be used as a useful tool rather than as an end in
holistic perspective. This holistic perspective itself. Researchers such as Schon (1991), Argyris
allows us to envisage the university as system (2004), Argyris and Schon (1996), Checkland
with the student as learning system (SLS) as (1981, 2000), Checkland and Hollwell (1998), and
subsystem (both with and without the individual Banathy (1996, 1999) have all worked to apply a
e-learning environment (ILE)). The university and systems approach and systems concepts to com-
SLS can also be conceptualised in terms of their plex social systems including higher education.
relationships with other systems and subsystems. By viewing the various scenarios systemically
Systems philosophy provides a process oriented and in terms of a hierarchy of related systems
view and the organisation of the relationships and and subsystems, an analytical approach can be
processes between relevant entities is central to adopted and used without losing sight of impor-
the emergence of the properties which help define tant systems relationships. Systems theory thus
a given system. In the case of this research, the provides tools and techniques for organising and
arrangements and relationships between the stu- understanding complexity. Properties such as
dent, the PELE subsystems and e-learning support hierarchy and emergence allow us to define the
processes and materials help define the emergent student and PELE within the context of the uni-
system. Different actors will view the system in a versity and related e-learning systems. Systems
range of ways. However, the framework provided methodology provides strategies and models for
by the cognitive profile helps ensure that the applying systems theory to complex systems and
viewpoint represented by the student and PELE problems. Systems methodology can be used in
as SLS is that of the student. two related but separate modes. The first is to
Epistemologically the systems approach takes use it as a way of organising and implement-
synthesis as both the starting point and objective ing enquiry about systems. The second is as a
of systems inquiry. The combination of the sys- framework for making sense of the system from
Figure 2. Student as reflective and participatory system designer for PELE

RAPAD
within any events which might be taking place. suboptimally interfaced with many important
In this study, systems methodology was used systems. The student can be conceptualised as
in both these ways. The models, methods, and being tightly coupled to subject learning through
strategies were used to define and explore, for the provision of prescribed materials and pro-
example, the concept of the student and PELE cesses, the use of the transmission method of
as an e-learning system. In addition, the use of teaching and learning, and a lack of metacogni-
systems methodology was, in itself, an iterative tive awareness and learning autonomy. In terms
and self-reflexive process in which the methodol- of other subsystems, the student may be poorly
ogy was a tool that was refined and developed by interfaced because of some of these factors plus a
the process of being used. lack of process knowledge—for example, a poor
knowledge of administrative procedures or of how
The Student as a Reflective and to access information on those procedures.
Participatory system designer for The student and individual e-learning environ-
PeLe ment combine as SLS to produce a more effective
e-learning system. This new system produces a
From a systems perspective, the student can be tight coupling between the student and the PELE.
considered to be part of the university conceptu- This then allows a loose and flexible coupling
alised as a human activity system. This system with the subjects as e-learning systems and
then contains several related subsystems, each other university e-learning and learning support
made up of people, processes, and technology. systems. This is the concept of Flexible Student
We can also conceptualise the student as being Alignment (Webster, 2005). By enhancing each
part of an e-learning system and, consequently, student’s metacognitive skills and self-regulatory
as a system, being combined with and interacting awareness, the locus of control is shifted towards
with relevant processes (attending, studying, using the student. The more autonomous system that
the library) and technologies (books, television, emerges is better able to handle the demands
computers). As with all human activity systems, of active and independent e-learning. Figure 3
there can then be several different conceptualisa- presents an overview of this process.
tions and viewpoints of the component parts and The PELE is necessarily an open system de-
makeup of the student as learning system (aca- signed with the student in order to help to student to
demic, administrator, parent, peer, etc.). However, interact with all facets of his or her environment in
the perspective which is the most important is order to support and sustain the learning process.
that of the student him or herself. The initial interface of a prototype Individual E-
As suggested above, we can further consider Learning Environment is shown in Figure 3. It
the student as learning system (SLS) to be part is built around the personal learning activities of
of a larger learning system, the university. The the student and also allows for more personal ele-
SLS interfaces with many other subsystems which ments to be included. The student’s cognitive style
function as e-learning or e-learning support (Analytic-Imager) in this case impacts both the
systems. The interfaces between the SLS and design (e.g., structure) and content (e.g., balance
these other systems are of crucial importance in of text and graphics) of the page. The student’s
the functioning of the student as an active and learning styles affect the content (e.g., time man-
autonomous learner. agement, learning organisation procedures, and
From the perspective of a compliant student resources, resource links for identified areas of
functioning with the transmission model of e- study weakness). Personality type impacts the look
learning, the student can be considered to be and feel of the learning environment but also the
0
RAPAD
information related processes via the instrument’s their view of constructive alignment to consider
information based dimensions. the concept of “alignment to students” (McCune,
2003, p. 24). She also suggested that learning mea-
Flexible student Alignment (FsA) sures and questionnaires had their limitations in
providing descriptions of the complexity of align-
Flexible student alignment (FSA) is produced ment in any given situation and stated that:
by the student and PELE subsystems forming
an adaptive system for interfacing with the While a teaching-environment may seem well
subsystems of the university e-learning system. aligned in terms, for example, of the correspon-
Biggs (1996, 1999, 2003) proposed the use of the dence between the forms of learning encouraged
concept of “constructive alignment” and sees by the different aspects of the teaching and assess-
the process as “aligning curriculum objectives, ment, this does not mean that this environment will
teaching/learning activities and assessment tasks” be equally suitable for all of the students involved.
(Biggs, 1999, p. 65). This concept has become a (McCune, 2003, p. 24)
generally accepted approach to viewing the teach-
ing-learning process. It takes a constructivist per- We can paraphrase this to say that: while
spective on learning and aims to align objectives the e-learning systems and environments may
expressing the types of understanding required of seem well aligned in terms of, for example, the
the student with assessment tasks which help us correspondence between the forms of e-learning
to see that those objectives have been met. The required for the overall efficient functioning of
teaching context and the assessment tasks also their university, this does not mean that this sys-
help students to undertake suitable e-learning tems and environments will be equally suitable
activities and the assessments clearly articulate for all the students involved.
what the students need to do. This is a useful and What is need is a series of personalized sub-
productive approach. It does, however, consider systems which can interface with the university
alignment largely from the teacher and teaching e-learning systems and environments with the
enabled learning perspective. software processes, information, and learning
We can also adapt this to the idea of develop- objects arranged with and by the individual student
ing e-learning systems and environments—that for each student’s e-learning purposes.
is, most current systems and environments are The work reported here focused on the learner
developed from the organisation’s perspective. and consequently considered alignment from the
However, if we recognise the need for and ad- student perspective as well. There is a close fit
vantages of the personalization of learning and and tight-coupling between the student and the
e-learning for lifelong learning in the knowledge PELE as e-learning support system. This and the
society, then we need adaptive systems and en- facility for loose coupling and flexibility between
vironments. The RAPAD methodology allows the PELE and the university as an e-learning
us to develop personalized e-learning systems environment enables students to better align
and environments to promote Flexible Student themselves with the various teaching-learning
Alignment via the involvement of the student in environments they encounter. Flexible Student
the design and development process. Alignment allows the student to use the SLS-PELE
McCune (2003) recognised this when reporting system to exercise individual flexible alignment
extensive work on university teaching-learning with respect to the multiplicity of teaching-learn-
environments (Entwistle, 2003; Entwistle, Mc- ing environments and other university e-learning
Cune, & Hounsell, 2002). The team had modified support systems encountered.

RAPAD
technology or human-centred is intended to demonstrate an acute case of the

e-Learning systems design? kind of reflection which Schön (1983) advocates
in ‘The Reflective Practitioner’” (Checkland &
Many of the changes in education and society Scholes, 1990, p. 276).
in recent years have been technology driven. The development of RAPAD then draws on
In most OECD countries (excluding the USA, the theoretical and applied work of both men—
where a mass or even universal system of higher separately and together. Separately because the
education has long been in place) there has also individual contributions included Schön’s “The
been a shift from and elite to a mass system of Reflective Practitioner: and Checkland’s “Soft
higher education (Trow, 1973). This shift has Systems Methodology.” Together, in that they
meant an increase in participation rates from 10- both draw extensively on systems theory and
15% to 30-40% of the 18-21 age group alongside Vickers’ concept of “appreciative systems” to help
wider participation from the population in general gain an understanding of the operations of both
(DfES, 2003). This combination of changes (and individuals and organisations. This is the basis of
reductions in student per capita funding) has Checkland’s “Human Activity Systems” (Check-
meant that new methods of teaching and learning land, 1981, 2000). The learning system produced
have become necessary. Technology is seen as a by the integration of the students, RAPAD, PELE,
major enabler, but the learning is still done by the and supporting technologies is considered to be
student, aided by good teaching. This means we an example of such a system.
need student-centred learning systems rather than
technology-centred systems. The changes have Information systems methodologies
been placed in a broad context above and will be and user-centred and Participatory
focused on at the individual level with reference design
to learning and to organisations in general and
universities in particular. The user-centred design In the development of RAPAD, several informa-
perspective and systems approach adopted is set tion systems methodologies were drawn on at
within a systems theory framework and much different times. These include Checkland’s Soft
of the theoretical thrust comes from an integra- Systems Methodology, Vora’s Human Factors
tion of the ideas of Donald Schön (1971, 1983, Methodology for developing Web sites (1998) and
1987, 1991) and Peter Checkland and co-workers the Human Factors for Information Technology
(Checkland, 1981, 2000; Checkland & Holwell, methodology and tool kit, HUFIT (HUSAT, 1990),
1998; Checkland & Scholes, 1990). Schön and which was used for the interface design guidance.
Checkland were concerned with change in society There are an enormous number of methodologies
and organisations. Schön is perhaps most closely for the development of information systems. Most,
identified with education and learning; Checkland fundamentally, are products in the market place
with organisational change and information sys- so each has its own tools and techniques, all of
tems. Checkland acknowledges the strong links which are claimed to be superior to all the others
between the central theses of the two authors for doing essentially the same things—conduct-
(Schön and The Reflective Practitioner, Checkland ing the activities of the systems development life
and Soft Systems Methodology) in the second cycle. Some authors (Avison & Fitzgerald, 2003;
of his major texts, Soft Systems Methodology Avison & Wood-Harper, 1990; El Louadi, Gal-
in Action (Checkland & Scholes, 1990). In the letta, & Sampler, 1998) have suggested using a
final chapter, entitled Gathering and Learning the “contingency approach” to system development.
Lessons, Checkland comments that “this chapter This allows for the selection of different sets of

RAPAD
methods and techniques according to criteria such The participatory approach in this study was
as the complexity of the system under develop- operationalized by the use of cognitive profiles
ment, the role of the user in the system and the and the involvement of students in reflecting on
expertise of the system developer. RAPAD can their own responses and then applying them to
also be considered to be a contingency methodol- learning environment design. Using the three
ogy drawing, as it does, on a range of tools and measures plus an iterative process of discussion,
techniques which can be adapted for a variety of design, and feedback gave a more holistic and
circumstances. systemic approach to the design of the PELEs.
This flexibility can be useful in dealing with In the information systems arena, there is a
complex scenarios where an innovative approach central statement indicating that you cannot design
might be useful. This is often the case in higher a better or improved system without fully under-
education where there are additional reasons for standing how the current system works—and no
complexity. As well as the different cultural and one understands the day to day working of a system
social norms encountered learning support sys- like the users. As with many well-worn sayings,
tems have to have sound pedagogic aims, objec- it is uttered frequently but followed rarely. Giving
tives, and achievements. Consequently, it can be students the relatively comprehensive information
argued that the implementation of such systems concerning their approaches to learning and their
can be more difficult than “normal” business information processing preferences (with refer-
information systems. ence to the layout and structure of learning materi-
Participatory design was pioneered in Scandi- als and, by inference, interfaces) allows them to
navia in the 1960s and 1970s (Preece et al., 2002). reflect and comment on both the accuracy of the
As its name suggests, it is designed to encourage measures and their applicability to the tasks in
user involvement in the design process and, along hand—including thoughts on how and why they
with contextual design, is one of the user-centred learn. The use of the additional learning style
approaches to interaction design. Whereas contex- and personality elements of the cognitive profile
tual design aims to use an ethnographic approach also allows comparison between the measures
to help the designer to understand the user in his and an extension of the individual differences
or her social, work and cultural context, participa- being considered
tory design encourages the active involvement of
the user in the design process. We can consider Why Use a Reflective and
the similarities between contextual design and Participatory methodology?
participatory design. Contextual design has seven
activities: contextual inquiry, work modeling, The overall process for the individual student is
consolidation, work redesign, user environment one of reflecting on the elements of a personal
design, mockup, and test with customers (Preece cognitive profile and then, after discussion and
et al., 2002, p. 296). One form of participatory consideration, applying the results of those reflec-
design, as used here, is to broadly follow these tions to the development of a Web technology-
activities, but to ensure that the user (or learner in based personalised e-learning environment.
this case) is dynamically and iteratively involved This approach taken has several key features
in the full design and development process. This that contribute to its effectiveness. These include
involvement is not always an easy task to ensure, the following:
although the participation of students studying a
human computer interaction unit in the first and • Participation in the process helps students
main iteration of this study greatly facilitated to develop metacognitive awareness and
the process.

RAPAD
Figure 3. RAPAD provides the guiding methodology but the cognitive profile and PELE are key compo-
nents to help reconceptualize learning and e-learning
RAPAD:
The methodology
provides the
overall process
and framework
Cognitive Profile: PELE:

Enables learner Provides design
focused reflection and development
on learning context and focus
characteristics
self-regulatory skills and to explore their • As a product of the process, students get a
attitudes to learning and e-learning in a resource which works in several ways and
manner which promotes Lifelong Learn- on several levels—an information organiser,
ing. a e-learnplace, a virtual/physical interface,
• Students produce a personalised Web site a cognitive interface, and an organisational
or Personalized E-Learning Environment interface.
(PELE) which provides a personalized • The design process helps give participants
access to learning materials and support a better understanding of students learning
systems. and e-learning systems design.
• The student is a major contributor to and
participator in the design and development student engagement with rAPAd
process, but it is not assumed that the student
can do this alone—the instructional designer In terms of student engagement with RAPAD and
and teacher have key roles in facilitating the the process of reconceptualizing their understand-
process. ing of personalized learning, the following are
• A framework is provided that affords both key steps in the application of the methodology
a structure to work within and a process to (several of these tasks are performed iteratively or
follow. in parallel over the life cycle of the process):
• Participation in the process helps students to
learn about user-centred, learned-centred, 1. Continuous reflection and comment on all
and participatory approaches to technology aspects of the process via mechanisms such
based e-learning environment design as discussion, reflective journals, tutorial
and assessment tasks, and learning related
design task.

RAPAD
2. An introduction to learning and the possible knowledge and PELE into all learning ac-
variations in and impact of cognitive styles, tivities.
learning styles, and learning preferences on
the learning process A version of the above scenario is presented
3. Taking the cognitive profile tests, consider- in Table 1 as implemented for the Learning at
ing personal results (and being allowed to University course.
disagree with them—with the proviso of To summarize, a reflective and participatory
explaining why), discussing the results and approach to design is a developmental method-
commenting on them within the context of ology which encourages reflection within the
current individual conceptions of personal context of a participatory approach to design. In
learning. this case it is reflection by students on aspects of
4. Producing a basic learning/personal Web site their own learning and participation in the process
as part of the first assessment task (along with of the design and development of personalised
a written version of the previous activity). e-learning environments. It is not assumed that
5. Engagement with online learning resources students can easily or naturally contribute to the
from a variety of sources to consider personal design and development process, so the concept of
preferences for learning tasks and activities the cognitive profile has been introduced to help
(structure and form of educational materi- the process. A cognitive profile is considered to
als, doing assignments, individual and col- consist of measures of an individual’s cognitive
laborative learning, information retention, style, learning style, and personality type. In
revising, etc.). terms of the design of a personalised e-learning
6. Doing a series of tutorial-based profile and environment, the term “reflective” is used as
design related tasks and producing an initial in Schön’s phrase “the reflective practitioner”
design document and series of draft screens (Schön, 1983).
for the PELE (second assessment task). Participatory design is an approach to design
7. Discussing tutor feedback on the design which is not only user-centred (or learner-centred),
document in group and individual sce- but actively involves the user (student) in the design
narios. process. This is especially important where there
8. Developing a series of personalised learning is a large element of interaction between the user
strategies for the degree course, the current and the system being designed. One mechanism
year, a semester, a unit, an assignment, and for doing this is student or user involvement in
considering how they might be integrated the design process, that is, a form of participatory
into the PELE. These strategies are seen as design where students can draw on and develop
flexible and dynamic, to be adjusted accord- their knowledge and understanding of how they
ing to varying constraints. learn within a framework and discourse provided
9. Developing, documenting (i.e., explaining by academic staff, university teachers, and stu-
the design with reference to one’s personal dent peers.
learning profile as part of the final assessment
task), presenting, and receiving feedback on
the actual Personalized E-Learning Envi- the deveLoPment oF rAPAd
ronment.
10. Reflecting on the overall process, changing There were four main phases in the development
personal conceptions of individual learning, of RAPAD:
and integrating the new learning related

RAPAD
1. The initial development and formulation of episodes occurred against the backdrop of a series
ideas from observed teaching practice of university reorganizations. The reorganizations
2. A structured research study with Level 3 reflected both social and technological changes
Human Computer Interaction students in higher education and responses to government
3. The development and reformulation of ideas policy and suggested a need to rethink student
from 1 and 2 with post-graduate conversion learning support resources at a personal level.
students taking several iterations of an In- I explored some of these ideas in several of the
formation Systems Development course courses I taught over the next few years. These
4. The fourth phase saw RAPAD developed, included courses in Human Computer Interaction
restructured for less technologically ex- and Information Systems Design. One newly
perienced students, and used as the major developed course allowed me to explore more
part and focus of a unit entitled “Learning of the cognitive and interface issues emerging
at University” for over 400 pre-university with Internet and Web developments—Intelligent
students. Interfaces for the Internet.
The four main phases are discussed in more Phase 2: Formal research Program
detail. As with all dynamic user-centred meth-
odologies, further use brings new developments A formal research program was designed to
and refinements. explore several of the questions raised by the
experiences of the students and myself in the first
Phase 1: Initial Formulation of the phase. Curriculum and syllabus changes allowed
need for Personalised e-Learning the redesign of a human computer interaction
Environments course to integrate the cognitive and interface is-
sues into the course material and assessment. The
The roots of the development of RAPAD lie in stated aim of the research was “to consider how
the period following the advent of the World cognitive profiles and a reflective and participa-
Wide Web in the UK. In the mid 1990s, the Web tory approach to the design and development of
and associated work-related factors initiated a a Web-based learning environment can be used
process of thinking in a more structured manner to enable autonomous learning and help students
about emerging themes and problems. The first interface with learning processes, materials, and
of these was when I observed a personalized and environments” (Webster, 2005, p.3).
individual interface (for a partially sighted student) Three well known and reliable measures,
in practice. The second was a concurrent period Riding’s Cognitive Styles Analysis (Riding &
of major organizational change, not uncommon Rayner, 1998), Entwistle’s Approaches and Study
in modern higher education, which had a nega- Skills Inventory for Students (Tait, Entwistle,
tive impact on the student using the personalized & McCune, 1998), and the Myers-Briggs Type
interface. Ideas concerning information overload Indicator (Myers, McCaulley, Quenk, & Hammer,
and attempts to enable students to handle the 1999) were used to develop the cognitive profile.
ever-increasing availability of masses of relatively Computer-based and self-report tests for each of
unstructured information were initially devel- the above measures were administered to a group
oped. Thoughts on interface preferences were of 64 students participating in a human computer
further prompted when I supervised the above interaction unit. The results of the tests were made
student taking a written exam with the specially available to the students within one week of each
constructed interface. Both of these reflective measure being administered. The students were

RAPAD
then asked to reflect on and write about their emergent Issues

thoughts on the accuracy and relevance of the
measures. Later in the unit, each student had to The initial period of analysis involved using the
develop a Web-based personalized e-learning quantitative data to provide a broad overview
environment (PELE) to a series of e-learning of the profiles, responses, and attitudes of the
related information resources. This required the respondents. This was done using the data from
application of elements of the cognitive profile to each of the cognitive profile measures plus the
the design and development process. In addition, quantitative data from the survey. However, as
the students were asked to document the reasons would be expected and as suggested by Sum-
for their design. A range of qualitative and quanti- merville (1999), the qualitative data provided
tative measures was collected. Student reflections much greater insights into the individual aspects
on and responses to the process were considered of e-learning.
via the use of a questionnaire, reflective journal The student comments and associated qualita-
and interviews. The comments on the form and tive data indicated that engaging in the process
content of the Web sites created contained in the of reflecting on the characteristics of one’s own
documentation were also analysed. individual cognitive profile did have an effect
Two related metaphors were used to help on the design, development, and content of the
the students to conceptualise the design of the individual e-learning environment. Several
PELE. The first was that of the Learning Re- students queried their prior lack of knowledge
source Centre (LRC) which is basically a modern of this type of information and commented that
university library integrating digital information they would have preferred to have access to this
management and learning support services. One type of metacognitive information in their high
definition used was: school (or even their university) careers.
The Learning Resource Centre (LRC) is a The participants often had a vague awareness
meeting place for all those who wish to learn. and sketchy understanding of their preferences
It is the electronic hub of the university and our for information handling, but this remained in
surrounding communities, linking us to the wider an unstructured and unfocused form. The infor-
global community. It harnesses new technologies mation from their cognitive profile gave them
effectively to make learning more adaptable and an opportunity to look at this scenario and their
flexible and more widely available. The LRC is preferences in a much more informed and struc-
at the centre of the university’s concept of a new tured manner. This then helped inform the PELE
learning environment. This environment focuses design, from the perspective of an impact on both
all our available resources into a teaching and the structure and form of the environment. Feed-
learning strategy based on our understanding of back and comments indicated that the CSA and
the changing trends in the learning community. its dimensions provided the most useful data and
The second metaphor was that of the PELE criteria in terms of developing the “look and feel”
conceptualised as a small personal house which of the PELE. The MBTI and ASSIST measures
the student could enter and find the personalized also provided personal learning and information
learning resources in a set of rooms design to processing preference details and these, while hav-
support each specific learning activity. This is a ing less impact on the design and construction of
similar, but more personal and individual use of the PELE, proved useful with specific reference to
the “house” metaphor to that used in the “Book- the learning process. This then impacted on the
house” (Pejtersen, 1989). PELE in terms of materials accessed to support
e-learning preferences.

RAPAD
More important, however, was the manner in To improve the quality of student learn-
which several students commented on broader ing, instructional measures should address the
aspects of their learning experiences and ap- conceptual domain of learning conceptions and
proaches to learning and sometimes identified beliefs, of which students have to become aware,
key incidents which affected their learning de- and which they have to develop, for example by
velopment. Others commented on the difficulties means of critical reflection. (Vermetten et al.,
they had in adjusting to the different demands of 2002, p. 263)
studying at university. They also pointed out that In addition, the responses suggested that both
the way they studied in the later parts of their the range of issues students considered as affecting
time at university was very different from that their learning and the manner in which these issues
adopted in the earlier stages. The manner of this interacted was very wide yet produced an individual
transition appeared to be a random one, often mix for each student. This outcome appeared to sup-
enabled by personal recognition of the problem port the comments of Summerville (1999) and Pillay
and self-help or the requested intervention of a (1998) on the need for a more process based approach
lecturer, tutor, or counsellor. comprising the collection of qualitative data.
Consideration of these and other examples from In addition, social issues such as the intervention
the difference types of data sources, especially the of others or the need to make sense of a process which
reflective journals, process documentation, survey students felt they should understand (how to study
comments, and interviews indicated several emer- effectively at university) yet clearly didn’t, indicated a
gent issues. need for a revision and extension of the methodology
The first issue to emerge was that the real impact and e-learning system.
of the cognitive profile measures was in enabling
students to reflect on their e-learning habits and Phase three: the Introduction of
processes in a structured manner. The actual scores ssm techniques
were less important than providing each student with
a set of relevant learner categories and characteris- The third phase saw the development and refor-
tics—whether imager or analytic, extraversion or in- mulation of ideas from the first two phases with
tuition, “interest in ideas” or “fear of failure”—which post-graduate conversion students taking several
could be used to think about their own e-learning iterations of an information systems development
experiences. The measures and activities provided course. A major outcome of this phase was the
a framework and a structured set of processes with introduction of specific techniques from Checkland’s
which the participants could engage reflectively with Soft Systems Methodology (SSM) (Checkland, 1981,
important features of the own learning. By critically 2000), especially Rich Pictures, in the process and
assessing their own learning needs and applying their research. The use of Rich Pictures at the student mod-
assumptions and conclusions to an iterative design elling phase was introduced after the initial research
process aimed at supporting their personal learning and Human Computer Interaction unit iteration. The
requirements, the students could effectively engage purpose of its introduction was to see if it could be
with understanding how they learn at an individual used to draw out issues relating to the social and
level. This leads to a much needed “conceptual interactive elements of learning. It then provided the
shift” in students understanding of individual (and basis for the “organisational interface” by allowing
thus collaborative) learning, the need for which was the student to place him or herself at the centre of the
suggested by Vermetten et al. (2002). university as organisation in a pictorial format. An
example Rich Picture is shown in Figure 4.

RAPAD
Figure 4. A rich picture to help clarify learning support needs and PELE design
The Rich Picture has been described as a “tool for Phase Four: the “Learning at
reasoning about work context” (Monk & Howard, University” Unit
1998) and both the technique and the methodology
have been applied to educational scenarios by several For the fourth phase of RAPAD’s development
authors and practitioners in addition to Checkland, saw the methodology developed and restructured
its originator (Briggs, 2003; Kassabova & Trounson, for less technologically experienced students. The
2000; Patel, 1995). In the systems development unit, reflective and participatory model developed in
the students were asked to reflect and comment on the the previous three phases—including the cog-
perceived learning support needs of different types of nitive profile, Rich Pictures, and personalised
student (undergraduate, pos-graduate, part-time, full- e-learning environment—was introduced as a
time, etc.). After various exercises and discussion in pre-university unit that formed the central unit
the context of systems development they were asked of a university preparation course for more than
to produce a Rich Picture of their own situation with 400 pre-university students.
respect to learning support resources and systems. This development represented an attempt to
Again, the concept of the Learning Resource Centre change the unit or course from one format—study
was used to illustrate and aid this exercise. The ex- skills based, nonelectronically supported—to a
amples produced illustrated a variety of individual format which is supported by Blackboard Learner
perspectives of how different students see themselves Management System. The course syllabus (see Table
acting and interacting within the context of the 1) adapted and used the model presented above to
university as e-learning environment––very much develop the metacognitive and self-regulatory skills
a personalized viewpoint. The students then used of the students about to enter university life and to
their Rich Pictures to define the PELE as a system help enable e-learning and lifelong learning.
in systems development terms (see below). The differences between 64 predominantly third
year students doing a Level 3 unit in “Human Com-
puter Interaction” and more than 400 pre-university

RAPAD
students completing a “Learning at University” unit with the redevelopment of a series of university
are significant. However, the exercise proved very preparation units to integrate the material into a
successful and by the second iteration of the course keystone unit for a university preparation course.
the methodology as unit was successfully integrated This unit, Learning at University, was aimed at
with Blackboard and the unit assessment practices. helping students to understand their own learning
This phase also provided additional data and material more full and thus to help provide the individual
for consideration in the development of the RAPAD as metacognitive skills and strategies necessary for
a methodology to enable students to reconceptualize each student to more fully benefit from the other
their learning within e-learning environments. units comprising the course.
The methodology and the unit can also be
“Learning at University”: seen as parts of a learning system designed and
Participatory methodology and developed to help the student to develop as an
unit as a Learning system autonomous learner. This is within the context
of the different systemic demands of mass higher
The research has a practical focus. It was always education (educational and social). In Banathy’s
intended that the research and methodology (1999) terms of key entity, key function, and or-
would provide the basis of several short courses ganising the education for learning outcomes (i.e.,
and also longer units if possible. The main target of the learning system), we have the following:
group were first year students and it was hoped
that short courses could be provided in the first • The key entity is the student
semester, although it was recognised that the • The key function is to enable autonomous
best time could be before commencing university e-learning
study. A variety of courses, including one half-day, • How to “organize the education for attain-
one day, and one week courses were designed for ing the best possible learning outcomes?"
students (and staff in one case), but there were dif- is achieved via the current and proposed
ficulties with fitting into the current diet of study implementation of the “Learning at Uni-
skills courses. However, an opportunity did arise versity” unit
Figure 5. Conceptual model of “Learning at University” as a learning system
0
RAPAD
Again using techniques from the Soft Systems focused on producing a design document for a
Methodology (Checkland, 1981, 2000), we can PELE with specific reference to their profiles.
define the elements of the systems as shown below. Following feedback and further exercises and
These are followed by a Root Definition, which discussion, the final assessment had several com-
draws the elements together and a Conceptual ponents. These were: to produce a final version
Model which presents the minimum subsystems of the e-learning environment (PELE), describe
needed to allow the system defined in the Root and critically analyse the structure of the PELE
Definition to function. according to each individual’s cognitive profile,
and finally, to orally and visual present and
• Client: The individual student demonstrate their e-learning environments to the
• Actors: The individual student, other respective workshop groups.
students, university staff (academics and The integration of the processes and materials
administrative) into the unit as a set of lectures, tutorials, and
• Transformation: Identification and satis- workshops is shown in Table 1.
faction of the individual students needs to
develop as an autonomous e-learner and life
long learning concLusIon
• Worldview: Autonomous e-learning is a
desirable learner attribute in mass higher This chapter has covered a lot of ground and
education and the knowledge society summarized the development work of much of the
• Owner: The university past decade. More detailed information, data, and
• Environment: Social and educational results concerning the formal research program
change, university as e-learning environ- and other developments can be found in several
ment, peer group, work opportunities related publications (Webster, 2002, 2003, 2004,
2005). With reference to the information provided
Root Definition by the three measures comprising the cognitive
profile, this allowed students to reflect on their
The “Learning at University” unit and associated learning related characteristics and preferences
personnel and resources comprise a system, owned in a much more structured and informed man-
by the university and operated by the student and ner. The outcome of applying the results of this
university staff, which identifies and satisfies the reflection was enhanced metacognitive skills
individual student’s need for autonomous e-learn- and knowledge. The design of the personalized
ing capabilities. It operates in an environment e-learning environment was an iterative process
enhanced and constrained by the academic and which both enabled the reflection and was affected
social resources and relationships. by the user profile in terms of structure and con-
In practical terms, this meant the integration tent. Many found that the dimensions of the CSA
of the framework, processes, and activities of gave them the most directly useful information
RAPAD into the Learning at University unit. in terms of the format and content of the PELE
The unit was assessed by a series of linked and and interface design. In contrast, the MBTI and
integrated assessments. The first required the ASSIST measures provided personal e-learning
students to produce a simple Web site, following and information preference details which were
lab material provided, plus an initial cognitive informative and had greater relevance to the e-
or learning profile based on their results, tutorial learning process. These details could then be either
discussions, and reflection. The second assessment integrated into the ‘look and feel’ of the PELE

RAPAD
Table 1. RAPAD implemented as the “Learning at University” unit
WEEK
LECTURE TUTORIAL WORKSHOP/LAB.
1 University learning and you: indi- Introduction to the unit Introduction to the lab.
vidual differences and indepen- Logging on.
dent learning ASSIST questionnaire Accessing Blackboard
2 Student cognitive and learning University learning and you: indi- Introduction to Web design for e-
profiles vidual differences and indepen- learning environment development
dent learning
3 Learning styles and learning Student cognitive and learning Web design for e-learning environ-
strategies profiles ment development (continued)
4 Cognitive styles and individual Learning styles and learning Cognitive styles and e-learning
preferences in layout and content strategies environment development
5 Personality types—how your per- Cognitive styles and individual Learning styles and learning envi-
sonality can affect your learning preferences in layout and content ronment development ASSIGN-
MENT 1 DUE
6 Learning Resource Centres Personality types—how your per- Learning communities and e-learn-
(LRC), Web sites, and Personal- sonality can affect your learning ing environment development
ized E-Learning Environments
(PELE)
7 Online learning and Web us- Learning Resource Centres Learning strategy features for e-
ability—tips on good learning (LRC), Web sites, and Personal- learning environment development
environment design ized E-Learning Environments
(PELE)
8 Rich Pictures and you—seeing Online learning and Web us- Learning support features for e-
yourself in the context of your ability—tips on good learning learning environment development
learning environment design ASSIGNMENT 2 DUE
9 Ideas for your PELE content—the Rich Pictures and you—seeing PELE development
BookHouse and the LearnHouse yourself in the context of your
learning
10 Developing learning strategies— Ideas for your PELE content—the PELE development
units & assessment BookHouse and the LearnHouse
11 Developing learning strategies— Developing learning strategies— PELE development
semester, year and course units & assessment
12 Unit review Presentations Presentations
ASSIGNMENT 3 DUE
13 Feedback sessions Feedback sessions Feedback sessions
or used more directly to suggest the inclusion of a taught unit, can be seen as an e-learning system
specific e-learning related features. which helps the student to produce a series of
Later iterations of the process and methodol- interfaces for integrating with learning environ-
ogy introduced further elements such as the Rich ments at the same time as aiding the development
Picture to enable students to consider additional of the student as an autonomous e-learner. There
aspects of how they might interface with both was a considerable difference between developing
online learning environments and the university the methodology with a cohort of 64 second and
as e-learning environment. In this way, the meth- third year Human Computer Interaction students
odology and techniques, as applied in the form of and a much larger number of students taking a

RAPAD
university preparation course. Each iteration that there is a need to consider the process as
played an important role in the overall develop- well as the outcomes and that the qualitative data
ment of the methodology and its emergence as a provided by student comments are the most useful
tool which could be used with a broad range of sources of explanatory data. Systems theory and
general students as in the university preparation a systems approach enabled this and helped the
course, Learning at University. concept of flexible student alignment to emerge
With the more general type of course exem- with the production of adaptive personalized e-
plified by the Learning at University course, an learning environments.
initial concern was the apparently large potential Flexible student alignment focuses on the
difference in the likely skills available to each learner and considers alignment from the student
group in terms of developing the e-learning perspective. As suggested above, a close fit and
environment as Web site. There was an empha- tight-coupling between the student and the PELE
sis throughout the process that this was not a as e-learning support system plus the facility for
technical or technology-based process, but one loose coupling and flexibility between the PELE
of reflection and design. The form and content and the university as e-learning environment
of the environment is given far greater emphasis enables students to better align themselves with
that the technical “bells and whistles” that can be the different teaching-learning environments
added using technology, no matter how valuable encountered. In this way, using RAPAD to enable
its contribution may be. To this end, the current flexible student alignment allows the student to
generation of Web development tools such as exercise individual flexible alignment. This is an
FrontPage (and even, at a stretch, Word) and important characteristic when considering the
their associated tutorials provide an initial set of many and varied teaching-learning environments
pages which can be developed with relative ease. and other university e-learning support systems
The experience for the student continues to vary likely to be encountered by each student.
enormously in terms of success and frustration, The concept of process reengineering in the
but increasing familiarity with personalizing information systems field draws on the idea that
mobile phone interfaces adds to the confidence developments in new information and communi-
of many students. The sense of achievement in cations technologies allow us to do many things
having developed a personal e-learning environ- in fundamentally different ways than previously.
ment and the associated skills is often mentioned Instead of using the technology just to further
as one of the tangible benefits by the students in improve how something is done, reengineering
the feedback survey. suggests we look for ways of reconceptualising
The combination of RAPAD and the cognitive how things are done. The use of an iterative,
profile instruments afford a framework and a set participatory process for effective technology
of processes for enabling students to engage with design is part of this reconceptualisation. The
their own and other profile elements and apply student becomes a central part of the technol-
them in a reflexive manner to a practical design ogy design process, whether as specialist (e.g.,
exercise. It is a complex scenario, but the repeated HCI) student or, with more help, pre- or first year
failure of many quasi experimental attempts to university student. In doing so, each individual
uncover significant relationships between learn- actively engages with fundamental aspects of his
ing measures and learning material presentation or her learning in ways that produce a valuable
(or interface design) suggested a need for a more e-learning environment plus improved metacog-
sophisticated approach to e-learning systems nitive and self-regulatory characteristics. The
design. Several major studies have concluded use of RAPAD produces a PELE as an effective

RAPAD
e-learning support system and the student and opportunities to personalize the environments in
e-learning environment combine to form an ef- more effective ways. Software agents, part of an
ficient learning support system for e-learning and earlier iteration of the work, have developed and
lifelong learning. become more mainstream. Their potential for
This chapter has presented the background, the gathering, filtering, and selection of relevant
content, and empirical use of the RAPAD meth- learning information and materials has been en-
odology. Definitions and key terms were provided hanced by their increased use for these purposes
and followed by a section which discussed the in the business arena. The use of XML (eXtensible
need for new and personalised approaches for Markup Language) will enable software agents
supporting e-learning. The changing concep- to better match the content of documents to the
tions of learning and the complexity of learning cognitive preferences of the individual student.
were considered. In order to provide a coherent All of these examples represent the potential for
overview of the work, a systems perspective of research and development in fertile areas.
the student, methodology, and PELE as a learning
system was presented. The concept of Flexible Cognitive, Virtual, and
Student Alignment was then introduced before organisational Interfaces
the need for human-centred e-learning systems
design and participatory design was outlined. Subsequent work has suggested that students
The development of RAPAD as a participatory can use personal cognitive profile knowledge to
methodology was then summarized. This was develop a series of different but individually re-
followed by a broad description of the research lated e-learning interfaces. Each interface serves
phases and empirical work which comprised the a separate but important function in helping the
development of RAPAD as an e-learning method- student to develop a series of strategies for inter-
ology. Future trends were then suggested before facing with the university at different levels—the
concluding points were made. personal, the virtual, and the organisational.
The first interface would operate at the level
Future reseArch dIrectIons of self-awareness. Here the knowledge and under-
standing of an individual’s cognitive profile would
In terms of future research directions develop- provide a framework in which that individual can
ments, several prospects exist to develop RAPAD better formulate a series of learning strategies
and take the personalized e-learning environment (based on, for example, subject, course, year,
forward. These include developing advanced semester, unit, etc.). These learning strategies
adaptive virtual environments. The enormous would then become part of the learning resources
success and developments in alternative digital on which the student can draw.
environments such as Second Life (http://secon- The second interface operates at a more
dlife.com) suggest that this is possible and likely. functional level and consists of a Web-based
Developing the skills of learning and gaming and interface for information management purposes.
integrating them with mobile virtual environ- The development of the first interface will help
ments means that e-learning environments can inform the design and development of the second
become more personalised, powerful, and ac- interface. In addition, besides being structured
cessible. Other developments include matching around the individual student’s cognitive profile,
its form and content to the additional cognitive the awareness of preferences in terms of the format
preferences of individual students. Developments and content of educational materials helps each
in auditory and visual digital data offer exciting student to interact more effectively with learning
materials.

RAPAD
The third interface is at the level of the virtual Avison, D.E., & Wood-Harper, A.T. (1990). Mul-
organisation. The techniques associated with tiview: An exploration in information systems de-
Checkland’s Soft Systems Methodology (Check- velopment. Henley-on-Thames: Alfred Waller.
land & Scholes, 1990), especially rich pictures,
Banathy, B.H. (1996). Systems inquiry and its
root definitions, and conceptual models, are used
application in education. In D.H. Jonassen (Ed.),
to help each individual student to locate himself
Handbook of research for educational commu-
or herself at the centre of an organisational e-
nications and technology (pp. 74-92). New York:
learning system. Again, the development of the
Prentice Hall.
first two interfaces serves to enhance the students
understanding of the individual aspects of their Banathy, B.H. (1999). Systems thinking in higher
own e-learning requirements in the context of the education: Learning comes to focus. Systems Re-
university as e-learning system. search and Behavioral Science, 16, 133-145.
The development of the concept of the three
Bayne, R. (1995). MBTI: A critical review. London:
interfaces has been the product of several related
Chapman & Hall.
iterations of the initial study in a series of taught
units. A more detailed explanation of the initial Biggs, J. (1996). Enhancing teaching through
research and the theoretical background of the constructive alignment. Higher Education, 32,
overall research and methodology development 347-364.
are reported more fully elsewhere (Webster,
Biggs, J. (1999). Teaching for quality learning at
2002, 2003, 2004, 2005). The use of Web-based
university: What the student does. Buckingham:
technologies and the adoption of these tech-
Open University Press.
nologies in to personalise corporate computing
(Computing: Work-Life Balance, 2007) ensures Biggs, J.B. (2003). Teaching for quality learning
that RAPAD is well placed to be extended as an at university (2nd ed.). Buckingham: SRHE &
adaptive methodology to enhance the process of Open University Press.
lifelong learning in the workplace.
Bishop, J. (2006, February). Training talk newslet-
ter. Retrieved October 16, 2007, from http://www.
dest.gov.au/sectors/training_skills/publications_
reFerences
resources/trainingtalk/issue_20/
Argyris, C. (2004). Double-loop learning and Blair, A. (2004, May 3). Speech to NAHT confer-
implementable validity. In H. Tsoukas & N. My- ence. Retrieved October 16, 2007, from http://
lonopoulos (Eds.), Organizations as knowledge www.number10.gov.uk/output/Page5730.asp
systems: Knowledge, learning, and dynamic
Bransford, J.D., Brown, A.L., & Cocking, R.R.
capabilities (pp. 29-45). New York: Palgrave
(Eds.). (1999). How people learn: Brain, mind,
Macmillan.
experience and school. Washington: National
Argyris, C., & Schon, D.A. (1996). Organizational Academic Press.
learning II. Reading, MA: Addison-Wesley Pub-
Briggs, J. (2003). Rich pictures of UK education.
lishing Company.
Retrieved October 16, 2007, from http://www.
Avison, D., & Fitzgerald, G. (2003). Information reengage.org/go/Article_111.html
systems development: Methodologies, techniques
Checkland, P. (1981). SystemsThinking, Systems
and tools (3rd ed.). Maidenhead: McGraw-Hill.
practice. Chichester: John Wiley.

RAPAD
Checkland, P. (2000). Soft systems methodology: Entwistle, N., Tait, H., & McCune, V. (2000).
A 30-year retrospective. Systems Research and Patterns of response to an approach to studying
Behavioral Science, 17, S11-S58. inventory across contrasting groups and contexts.
Paper presented at the European Journal of the
Checkland, P., & Holwell, S. (1998). Information,
Psychology of Education.
systems and information systems. John Wiley
and Sons. Ford, P., Goodyear, P., Heseltine, R., Lewis, R.,
Darby, J., Graves, J., et al. (1996). Managing
Checkland, P., & Scholes, J. (1990). Soft systems
change in higher education: A learning environ-
methodology in action. John Wiley & Sons.
ment architecture. Society for Research in Higher
Coffield, F., Moseley, D., Hall, E., & Ecclestone, Education and Open University Press.
K. (2004). Learning styles and pedagogy in post-
Goodyear, P. (2001). Effective networked learn-
16 learning: A systematic and critical review.
ing in higher education: Notes and guidelines
London: Learning and Skills Research Centre.
(Deliverable 9) (Vol. 3). Lancaster: CSALT,
Computing: Work-Life Balance. (2007). The Lancaster University.
Economist, 23/12/06-5/1/07, 99-100.
Goodyear, P. (2002). Online learning and teaching
DfES. (2003). Widening participation in higher in the arts and humanities: Reflecting on purposes
education. London: Department for Education and design. In E.A. Chambers & K. Lack (Eds.),
and Skills. Online conferencing in the arts and humanities
(pp. 1-15). Milton Keynes: Institute of Educational
Driscoll, M.P. (2002). How people learn (and
Technology, Open University.
what technology might have to do with it). ERIC
Digest, Syracuse University. Retrieved October Goodyear, P. (2005). Educational design and
16, 2007, from http://www.ericdigests.org/2003- networked learning: Patterns, pattern languages
3/learn.htm and design practice. Australasian Journal of
Educational Technology, 21(1), 82-101.
El Louadi, M., Galletta, D.F., & Sampler, J.L.
(1998). An empirical validation of a contingency HUSAT. (1990). The HUFIT planning, analysis
model for information requirements determina- and specification toolset. Loughborough: HUSAT
tion. ACM SIGMIS Database archive, 29(3), Research Institute, Loughborough University.
31-51.
Jonassen, D.H., & Grabowski, B.L. (1993). Hand-
Entwistle, N. (2003). University teaching-learn- book of individual differences, learning, and
ing environments and their influences on student instruction. Hillsdale, NJ: Lawrence Erlbaum
learning: An introduction to the ETL project. In Associates.
Proceedings of the 10th Conference of the Euro-
Kassabova, D., & Trounson, R. (2000). Apply-
pean Association for Research on Learning and
ing soft systems methodology for user centred
Instruction (EARLI). Padova, Italy: EARLI.
design. In Proceedings of the NACCQ 2000 (pp.
Entwistle, N., McCune, V., & Hounsell, J. (2002). 159-165). Wellington.
Approaches to studying and perceptions of univer-
Knight, P.T. (2001). Complexity and curriculum: A
sity teaching-learning environments: Concepts,
process approach to curriculum-making. Teaching
measures and preliminary findings. Edinburgh:
in Higher Education, 6(3), 369-381.
University of Edinburgh.

RAPAD
Knight, P.T., & Trowler, P. (2001). Departmental Onions, C.T. (Ed.). (1983). The shorter Oxford
leadership in higher education. Buckingham: English dictionary (3rd ed.). Oxford: Oxford
Society for Research in Higher Education & Open University Press.
University Press.
Patel, N.V. (1995). Application of soft systems
Laurillard, D. (2002). Rethinking university methodology to the real world process of teaching
teaching: A conversational framework for the and learning. International Journal of Educa-
effective use of learning technologies (2nd ed.). tional Management, 9(1), 13-23.
London: Routledge.
Pejtersen, A.M. (1989). The BOOKHOUSE: An
Laurillard, D. (1993). Rethinking University icon based database system for fiction retrieval
Teaching: a framework for the effective use of in public libraries. In Proceedings of 7th Nordic
educational technology. London: Routledge. Information and Documentation Conference,
Århus, Denmark.
Laurillard, D. (1999). A conversational framework
for individual learning applied to the ‘learning Peterson, E. R., Deary, I. J., & Austin, E. J. (2003).
organisation’ and the ‘learning society’, systems The reliability of Riding’s Cognitive Style Analy-
research and behavioral science (vol. 16, pp. sis test. Personality and Individual Differences,
113-122). 34, 881-891.
Marton, F., & Ramsden, P. (1988). What does it Pillay, H. (1998). An investigation of the effect
take to improve learning? In P. Ramsden (Ed.), of individual cognitive preferences on learning
Improving learning: New perspectives. London: through computer-based instruction. Educational
Kogan Page. Psychology, 18(2), 171-182.
McCune, V. (2003). Promoting high-quality Preece, J., Rogers, Y., & Sharp, H. (2002). Inter-
learning: Perspectives from the ETL project. action design: Beyond human computer interac-
In Proceedings: 14th Conference on University tion. Wiley.
and College Pedagogy. Fredrikstad: Norwegian
Riding, R., & Rayner, S. (1998). Cognitive styles
Network in Higher Education.
and learning strategies: Understanding style
Metros, S.E., & Bennett, K. (2002). Learning differences in learning and behaviour. London:
objects in higher education. Educause Research David Fulton Publishers.
Bulletin, 19, 2. Retrieved October 16, 2007, from
Schön, D.A. (1971). Beyond the stable state: Pub-
www.educause.edu/ir/library/pdf/ERB0219.pdf
lic and private learning in a changing society.
Monk, A., & Howard, S. (1998, March-April). Temple Smith.
The rich picture: A tool for reasoning about work
Schön, D.A. (1983). The reflective practitioner.
context. Interactions, 21-30.
NewYork: Basic Books.
Myers, I.B., McCaulley, M.H., Quenk, N.I., &
Schön, D.A. (1987). Educating the reflective
Hammer, A.L. (1999). MBTI manual: A guide
practitioner: Toward a new design for teaching
to the development and use of the Myers-Briggs
and learning in the professions. San Francisco:
Type Indicator. Paolo Alto, CA: Consulting Psy-
Jossey-Bass Publishers.
chologist Press.
Schön, D.A. (1991). The reflective turn: Case
Nowack, K. (1996). Is the Myers Briggs Type
studies in and on educational practice. New York:
Indicator the right tool to use? Performance in
Teachers Press, Columbia University.
Practice, 6.

RAPAD
Shuell, T. (1992). Designing instructional comput- Vora, P. (1998). Human factors methodology for
ing systems for meaningful learning. In M. Jones designing Web sites. In C. Forsythe, E. Grose &
& P. Winne (Eds.), Adaptive learning environ- J. Ratner (Eds.), Human factors and Web develop-
ments. New York: Springer Verlag. ment. Hillsdale, NJ: Lawrence Erlbaum.
Simons, R. J., van der Linden, J., & Duffy, T.
(Eds.). (2000). New learning. Dordrecht: Kluwer
Webster, W.R. (2002, July). Metacognition and
Academic.
the autonomous learner: Student reflections on
Summerville, J. (1999). Role of awareness of cog- cognitive profiles and learning environment devel-
nitive style in hypermedia. International Journal opment. In A. Goody (Ed.), Spheres of influence:
of Educational Technology, 1. Ventures and visions in educational development.
Proceedings of ICED 2002, UWA, Perth, Austra-
Tait, H., Entwistle, N.J., & McCune, V. (1998).
lia: University of Western Australia.
ASSIST: A reconceptualisation of the approaches
to studying inventory. In C. Rust (Ed.), Improving Webster, W.R. (2003). Cognitive styles, metacog-
student learning (pp. 262-271). Oxford: Oxford nition and the design of e-learning environments.
Centre for Staff and Learning Development. In F. Albalooshi (Ed.), Virtual education: Cases
in teaching and learning (pp. 225-240). Hershey,
Trow, M. (1973). Problems in the transition from
PA: Idea Group Publishing.
elite to mass higher education. Berkley, CA:
Carnegie Commission on Higher Education. Webster, W.R. (2004, November 2-3). A learner-
centred methodology for learning environment
Trowler, P., & Knight, P.T. (2000). Coming to know
design and development. In Exploring integrated
in higher education: Theorising faculty entry to
learning environments. Proceedings, Online
new work contexts. Higher Education Research
Learning and Training 2004, Brisbane. Brisbane,
& Development, 19(1).
Australia: Queensland University of Technol-
Trowler, P., Saunders, M., & Knight, P.T. (2003). ogy.
Change thinking, change practices: A guide to
Webster, W.R. (2005). A reflective and partici-
change for heads of department, programme
patory approach to the design of personalised
leaders and other change agents in higher educa-
learning environments. Unpublished PhD Thesis,
tion. Learning and Teaching Support Network,
Lancaster, Lancaster University.
Generic Centre.
Weil, S. (1999). Re-creating universities for
Twigg, C.A. (1994). The changing definition of
beyond the stable state: From dearingesque sys-
learning. Educom Review, 29(4).
tematic control to post-dearing systemic learning
Vermetten, Y.J., Vermunt, J.D., & Lodewijks, and inquiry. Systems Research and Behavioral
H.G. (2002). Powerful learning environments? Science, 16, 170-190.
How university students differ in their response
Wilson, B.G. (1996). What is a constructivist
to instructional measures. Learning and Instruc-
learning environment? In B.G. Wilson (Ed.),
tion, 12, 263-284.
Constructivist learning environments: Case stud-
Vermunt, J.D. (1998). The regulation of con- ies in instructional design (pp. 3-8). Educational
structive learning processes. British Journal of Technology Publications.
Educational Psychology, 67, 149-171.

RAPAD
AddItIonAL reAdIngs Sternberg, R.J., & Zhang, L.F. (Eds.). (2001).

Perspectives on thinking, learning and cogni-
Goodyear, P. (2002). Online learning and teaching tive styles. Mahwah, NJ: Lawrence Erlbaum
in the arts and humanities: Reflecting on purposes Associates.
and design. In E.A. Chambers & K. Lack (Eds.),
Tsoukas, H., & Mylonopoulos, N. (Eds.). Or-
Online conferencing in the arts and humanities
ganizations as knowledge systems: Knowledge,
(pp. 1-15). Milton Keynes: Institute of Educational
learning, and dynamic capabilities. New York:
Technology, Open University.
Palgrave Macmillan.
Haag, S., Cummings, M., & McCubbery, D.J.
Wierstra, R.F.A., Kanselaar, G., Van Der Linden,
(2004). Management information systems for
J.L., Lodewijks, H.G.L.C., & Vermunt, J.D. (2003).
the information age (4th ed.). Boston: McGraw-
The impact of the university context on European
Hill.
students’ learning approaches and learning en-
Riding R., & Rayner, S.G. (Eds.). International vironment preferences. Higher Education, 45,
perspectives on individual differences: Cogni- 503-523.
tive styles (Vol. 1). Stamford: Ablex Publishing
Corporation.

0
Chapter II
A Heideggerian View on
E-Learning
Sergio Vasquez Bronfman
ESCP-EAP (European School of Management), France
ABstrAct
This chapter introduces some ideas of the German philosopher Martin Heidegger and how they can be
applied to e-learning design. It argues that heideggerian thinking (in particular the interpretation done
by Hubert Dreyfus) can inspire innovations in e-learning design and implementation by putting practice
at the center of knowledge creation, which in the case of professional and corporate education are real
work situations. It also points out the limits of distance learning imposed by the nature of human beings.
Furthermore, the author hope that Heidegger ideas will not only inform researchers of a better design
for e-learning projects, but also illuminate practitioners on how to design e-learning courses aimed at
bridging the gap between “knowing” and “doing.”
IntroductIon one can observe; actual PCCE systems produce

people who get a lot of knowledge but who are
In the field of professional, continuous, and unable to put it into practice.
corporate education (PCCE)1 there is a recurrent One of the main reasons for this knowing-do-
complaint concerning the effectiveness of the ing gap (Pfeffer & Sutton, 2000) is what I call
educational process (Mintzberg, 1988, 1996, 2004; infocentrism, which is a wrong interpretation of
Schön, 1983). Effectiveness is “the ability of a what learning is. Infocentrism says that learning
system to produce what it must produce.” There- is a kind of information system: knowledge is
fore, in an effective PCCE system people should transmitted to learners through lectures and/or
learn to do what they must do when working in accessed through readings, learners must retain
their companies. Unfortunately this is not what this knowledge, and finally professors organize
A Heideggerian View on E-Learning
tests of knowledge retention that we call exams.2 dealt with many topics, but in this chapter I will
In good educational settings, exercises and case focus on his thinking about human activity and
studies are also performed. Implicitly, the info- the relationship between theory and practice.
centric perspective makes the hypothesis that if My argumentation is based on the work done
knowledge is transmitted properly (i.e., lectures by Hubert Dreyfus, one of the best specialists
are clear) then application (practice) is obvious. In on Heidegger philosophy and who has made
fact this hypothesis is falsified. Hence the know- important contributions applying Heideggerian
ing-doing gap comes into existence. thinking to learning and artificial intelligence
As a business school professor and a practi- (Dreyfus, 1986, 1991, 1992, 2001).
tioner I am committed in research and practice The essence of Heidegger thinking is that
to bridge this gap. I think that my professional Western philosophy, from Plato onward, has
activity is an opportunity for innovation and hence misunderstood the nature of being. In particular,
I design and implement educational experiences he argues that metaphysical and scientific theories
to add value to professional, continuous, and have tended to favour all questions about being into
corporate education, by using information and categories, better suited to describe the detached
communication technologies (ICT) and applying contemplation of inert objects. As a result, accord-
innovative pedagogical methods. ing to Heidegger, philosophers and scientists have
In this journey—thanks to Fernando Flores overlooked the more basic, pretheoretical ways of
and his collaborators (Spinosa, Flores, & Dreyfus, being from which their theories derive, and, in
1997; Winograd & Flores, 1986)—I have been in applying those theories universally, have confused
contact with the ideas of Martin Heidegger whose our understanding of human existence. To avoid
philosophy I have found one of the most valuable these deep-rooted misconceptions, Heidegger
for learning innovation (including the use of believes he must restart philosophical inquiry
information and communication technologies to in a different way, using a novel vocabulary and
add value to learning). With the exception of the undertaking an extended criticism of the history
seminal work of Hubert Dreyfus, little research of philosophy (Wikipedia, 2006)
has been done on the impact of Heidegger phi- Heidegger says that our everyday action is
losophy on learning and e-learning.3 Now, this rooted in the ability to act pre-reflectively when
impact could inspire important innovations or, we are thrown in a situation.4 Most of the time
at least, an accurate interpretation of learning, our lives happen this way (to dress ourselves,
hence resulting in good e-learning design and to go to a place or another, to work, to eat, etc.).
implementation. The purpose of this chapter is We do not think, we just do, and we cope with
then to show how Heidegger ideas can illuminate the situation. There is only some little fraction
learning innovation (including e-learning) and, of time where life happens in the conscious and
in particular, help to bridge the knowing-doing deliberate way of doing.
gap. Heidegger is not against theory. He says that
theory is an important and powerful instrument,
but a limited one, only a subset of the way human
heIdegger PhILosoPhy And Its beings cope with things. In particular, Heidegger
ImPLIcAtIons For LeArnIng observes that in order to do something, even a
high level cognitive action, we do not need to have
Martin Heidegger (1889-1976) was a very influ- a theory of the domain in which we are taking
ential German philosopher, probably the most action.5 He also says that it is impossible to have
influential one of the 20th century. His work a theory about what makes theory possible. If he

is right, his analysis question one of the most im- them through “laws” (as in natural sciences) or
portant postulates in Western philosophy, which rules and procedures (as in structuralism and
finds its roots in Descartes and until Plato: that cognitivism). Therefore, teaching follows this
human activity can be explained with theories, path: we first present theory (the more abstract
that human beings are conscious subjects who it is, the better) and then applications (examples,
observe objects, and that a theoretical and de- exercises, or case studies), giving raise to the
tached perspective is better than a practical and knowing-doing gap.
involved one. Moreover, when talking about learning, people
Instead of this, Heidegger says that a theoretical usually confuse different kinds of learning. In
and detached knowledge imply a practical and in- order to clarify the discussion I will make some
volved know-how, which precedes the theoretical important distinctions on learning.6
knowledge and that cannot be explained by him.
Even theoretical knowledge depends on practi- • Learn about, for example, negotiation, com-
cal skills. The detached knower should then be munication, history, medicine, software
replaced by an involved doer. design, and so forth
As human beings, our relationship with • Learn to do, for example, how to negotiate,
“things” is always purposeful. Heidegger says how to communicate well, how to run a re-
that we do not find “simple things”; rather we use search in history, how to diagnose illnesses,
things in order to achieve something. Heidegger how to design software, and so forth
call these things “equipment,” in a very large • Learn to be, for example, a negotiator, a
sense which includes tools, material, clothes, toys, communicator, a researcher in the field of
machines, houses, and so forth. The fundamental history, a doctor, a software designer, and
characteristic of the equipment is its purposeful so forth
use; in fact, Heidegger defines a piece of equipment
in terms of its purposeful use. When everything One can love history and be interested in
is working well, equipment is characterized by medicine or in human communication. By reading
its transparency: it is “ready at hand.” Heidegger books on these topics, attending conferences, do-
called this availableness (zuhandenheit). ing courses (online or face-to-face), and so forth,
However, when we face a breakdown (i.e., one can learn a lot about history, medicine, and
a “surprise”), when something is not “ready at human communication, but that does not mean that
hand,” we move to what Heidegger calls occur- one will be able to conduct research in history, to
rentness (vorhandenheit). According to Dreyfus diagnose illnesses, or to communicate effectively.
there are some stages in this move, going from In other words, one will not be able to do.
conspicuousness (a short breakdown, easily Following the same logic, if one has been suc-
repaired), to obstinacy (which implies stop and cessfully conducting a first piece of research in
think, planned reflection—“what ifs,” “if-then- history, has diagnosed some simple illnesses, or
elses,” and so forth—all of this in a context of has solved a communicational problem, that does
involved action), and to obstrusiveness (detached not mean that one will be considered a historian,
theoretical reflection). a doctor, or a professional in the field of human
The basic postulates of Western philosophy communication. In other words, one will be
have had dramatic consequences in learning and able to do, but one will not yet be (a professional
teaching. As a matter of fact, in every discipline recognized as such by his/her peers). In order
people try to find context-free elements, basic to reach this level one must have a significant
concepts, attributes, and so forth, and relate amount of practice in the appropriate community

(negotiators, communicators, historians, doctors, An example of the above is a series of speeches

software designers, etc.). that I have designed for the IT Department of a
Bearing in mind these learning distinctions, Spanish petroleum company. The concern of the
it can be said that one of the causes of the know- chief information officer (CIO) was to improve
ing-doing gap is that the vast majority of the the technical culture of the computer professionals
educational offer satisfy only the “learn about” and to share the accumulated knowledge across
kind of learning, and that many people expect at the different specialities of the department. For
least “learn to do.” Because educational practices instance, in designing the speech on networks and
needed in order to “learn about” are not sufficient telecommunications we started by a very concrete
when one needs “learn to do,” there is an impor- situation for all of the targeted IT professionals:
tant discrepancy between supply and demand in when one sends a message from Building A to
education. Building B,8 what happens (technically speaking)?
As I have said, Heidegger philosophy can help The presentation described step by step the differ-
us to give new insights in order to design innova- ent technologies involved in this process: comput-
tive educational offers. Summarizing Heidegger ers, servers, switches, routers, optical fibre, and
contributions, one can say that good learning so forth, then showed the different IP addresses
design should: of all the elements included in this network, and
continued moving from significant examples (sig-
• To throw the learner in the situation he/she nificant for the IT professionals of this company)
must know which is the context for the to definitions and again to examples.
practice that must be mastered All of the other presentations were designed
• Always start by local and concrete examples this way. The result was a good evaluation and
and/or involved practice, and then move a significant attendance of the IT professionals
gradually to detached reflection to the series of speeches, while these attendance
• Design situations where one must deal with and evaluation were poor in the past.
breakdowns In the next section I will present a model for
• Make available a vast repertoire of situa- the two others kinds of learning: “learn to do...”
tions, cases, and so forth, on the topic to be and “learn to be...”; in other words, learning a
learned skill and learning to be a professional.
• Design or use technology that is “ready at
hand,” that is, easy to use, transparent7
PhenomenoLogy oF LeArnIng
For instance, in order to improve “learn A skILL
about...” teaching practices, one should not start
by presenting definitions of the topic to be studied, Hubert Dreyfus has done a major contribution
and then move from general abstractions to par- describing the process for which one is able to
ticular situations, from theory to practice. Instead learn and master a skill. According to Dreyfus,
of that, one should do exactly the opposite: start this process is always a kind of apprenticeship.
by presenting particular stories where the studied In the first presentation of his model (Dreyfus,
phenomenon shows itself (because our encounter 1986), he distinguished five levels: novice, ad-
with a new phenomenon happens always through vanced beginner, competence, proficiency, and
particular and concrete examples), and then move expertise. Later, he added two more levels: mastery
to a definition of the “thing,” hence going from and practical wisdom (Dreyfus, 2001). Inspired
practice to theory, from concrete examples to by his work, I will present a simplified model of
abstract definitions.

apprenticeship with only three levels: beginner, word processor for an unemployed person) and,
competent, and expert. in this process, make him learn the main func-
tionalities of Word.
Beginner Another example is the e-learning courses I
have designed for the new employees of la Caixa,
The instruction process should, of course, start the most important Spanish savings bank. Using
throwing the learner in a situation close to the participative course design methods, the design
real work situation the learner will be in. The team worked with end-users of the courses, that is,
instructor gives learners the information (facts, new employees and their managers. For instance,
rules, procedures) they need to cope with the when designing a course on insurance, we asked
situation and coach the learners. them: what is the everyday coping of la Caixa’s
In corporate learning (i.e., learning pro- new employees on insurance? The answer helped
grammes especially designed for only one com- us to focus on the skills that new employees must
pany), and in executive education, one can benefit come to master when dealing with insurance (for
from the experience of the learners and work with instance, to sell insurance that takes care of cus-
their situations instead of giving them case studies tomers’ concerns). Then we asked for recurrent
that bring practice to the classroom but which is situations faced by the new employees in this
not students’ own practice. Therefore, the situa- field, which lead us to write a sequence of mini
tions where students should be thrown must be cases. At the end of each mini case learners have
based in the everyday coping of the learners with to answer questions like: “What would you do in
the situation, that is, the way they cope every day this situation?” “What kind of products can you
with some subject or topic, the way they face it offer to this client?” “What would be your advice
every day at the workplace.9 to this customer?” and so forth. Answers must
In undergraduate education, where students generally be sent to a forum for discussion with
do not have professional experience, one should the online classroom colleagues, moderated by
move to traditional case studies, role playing, their online trainer (which is a branch manager).
and/or computer simulations. Relevant information in order to perform these
In any case, a learner should be thrown in a activities is suggested to learners (which they can
situation which is significant to him/her (e.g., access on the Web pages of the courses).
work situations) in order to provoke emotions and All of the courses were structured as a series
involvement, which is also necessary to bridge the of mini-cases. The learner must start always with
knowing-doing gap because at work we experi- a case (which thrown him/her in a situation based
ment emotions and involvement.10 on the everyday coping of la Caixa employees with
For instance, when learning to use a technol- the topic of the course), therefore being concrete
ogy (e.g., a software), the designer must create examples of involved practice. The material they
situations focused on the purpose of the use of can access in order to perform these activities gives
this technology for the learner, and throw the them definitions and general knowledge they can
learner in these situations. If unemployed people apply to different particular situations.
should learn how to use a software like Word,
traditional learning design will present all of competent
the functions of the software and then move to
applications. A Heideggerian-based design will As the learner becomes competent in coping with
rather ask the learners to write a curriculum vitae “normal” situations, in applying general rules and
(which is probably the main purpose of using a procedures to particular situations, the instructor

should move to a different kind of situations and • What am I trying to do?

make the learner cope with breakdowns: some- • What is stopping me from doing it? What
thing produces unexpected results, an error resists is the problem?
correction, or we begin to look at something in a • What action will I take in order to overcome
new way.11 The learner response to the situation, the obstacles?
based on rules and procedures, will gradually be
replaced by situational discriminations accompa- It is in this process that people learn from each
nied by associated responses. In the beginning of other and create new knowledge.
this level, the learner will have mainly reasoned In this school of thought, learning involves
responses, will need to “stop and think,” but as programmed knowledge (knowledge one gets
he/she becomes really competent, intuitive be- from outside the set through lectures, seminars,
haviour will gradually take place. books, etc.), but the majority of the learning oc-
In order to become competent, the instructor curs through fresh questions that help the person
must then design situations that provoke break- addressing the problem to look at it in different
downs. In this sense, role-playing could be better ways so that better solutions can be found.
than case studies, especially in undergraduate Another important point here is that learning
education. For instance, in courses like commu- means implementation (stopping at the analysis
nication or negotiation, one can easily imagine and recommendations phase will not be sufficient).
role plays where learners face “surprises.” Course Action learning is then a cyclical process: it starts
sessions can therefore start by role plays and then with problem discussion; people look for new
reflect on the observed behaviours; the instruc- ways of seeing the problem, finding solutions,
tor can use these reflective moments to present implementing solutions, and observing results,
theories, concepts, methods and techniques. and the process starts again with the discussion
In corporate learning and executive education, of problems with implementation.
the instructor should, in addition to the above, One can also say that in applying action
design discussions of learners’ breakdowns at learning techniques and/or the learning methods
work (Mintzberg, 2004), and use action learning suggested by Mintzberg, instructors are promot-
techniques (Pedler, 1991; Revans, 1980). Action ing “reflection-on-action.” Elsewhere (Vasquez
learning was invented by Reginald Revans when Bronfman, 2005) I have shown the parallels of
he was leading the training department at the a Heideggerian view of learning with Donald
National Coal Board in the United Kingdom. It Schön’s interpretation of reflection, which is quite
is based on two important points: (a) work on the different of the traditional interpretation of the
real problems faced by learners, and (b) work on concept as a detached way of knowing. On the
problems where there is confusion, ignorance, basis of his observations of the artistry showed
where nobody has the answer. This is done in a by competent practitioners, Schön propose two
“learning set,” that is a group of 5-8 people whose fundamental concepts in order to explain this
main goal is to learn from their own experience artistry. These are knowing-in-action on the one
through questioning and reflecting. hand, reflection-in-action and reflection-on-ac-
The group decides on the common problem/ tion on the other hand (Schön, 1983, 1987).
opportunity on which to work. People look for Knowing-in-action refers to the know-how
new interpretations, new ways of settling the revealed in our daily action when doing our jobs,
problem/opportunity. A good guide to doing this for example, the instant analysis of a balance
is to work on the following questions: sheet. According to Schön, there are in fact many
actions we perform spontaneously, without hav-

ing to think on them. Often, we are not aware of expert

having learned to perform these actions. “Even if
sometimes we think before the action, it is still true The competent performer, immersed in the world
that most of the time our spontaneous behaviour of skillful activity, sees what needs to be done,
concerning practical skills does not come from a but still has to decide consciously how to do it.
previous intellectual operation. Nonetheless, we In front of a breakdown, the expert not only sees
show a kind of knowledge” (Schön, 1996). what needs to be achieved but, thanks to a vast
Our knowledge-in-action allows us to cope repertoire of situational discriminations, but he/
with daily life. However, sometimes we experi- she sees immediately what needs to be done and
ence “surprises,” either good or bad. An error simply takes action (Dreyfus, 2001). In Donald
in a computer programme resists correction, Schön’s words, a competent performer still needs
the outputs of an advertising TV spot are much to reflect-on-action while the expert is able to
more better than expected, a carefully designed reflect-in-action.
information system is rejected by its users, and so In a delightful description of the phenomenon,
forth. Something unexpected reveals then to us. Sir Arthur Conan Doyle gets the point in his first
In Schön’s interpretation, “reflection” starts Sherlock Holmes novel Study in Scarlet (Chapter
when there is a surprise (in other words, when 2, The Science of Deduction):
there is a breakdown): something produces
unexpected results, and/or we begin to look at From long habit the train of thoughts ran so swiftly
something in a new way. We may respond to this through my mind that I arrived at the conclusion
situation by reflection and we may do so in one without being conscious of intermediate steps.
of two ways. There were such steps, however. The train of
We may reflect on action, thinking back on reasoning ran, “Here is a gentleman of a medical
what we have done in order to discover the causes type, but with the air of a military man. Clearly
of the unexpected outcome (stop-and-think). And an army doctor, then. He has just come from the
we may reflect in action, that is, in the midst of tropics, for his face is dark, and that is not the
action without interrupting it, carrying out on-the- natural tint of his skin, for his wrists are fair.
spot experiments to change the situation, “think- He has undergone hardship and sickness, as his
ing on our feet.” The point for reflection-in-action haggard face says clearly. His left arm has been
is that we can think about something while doing injured. He holds it in a stiff and unnatural man-
it, it is the capacity to respond to surprise through ner. Where in the tropics could an English army
improvisation on the spot. doctor have seen much hardship and got his arm
Table 1 summarises the parallels between wounded? Clearly in Afghanistan.” The whole
Heidegger philosophy and Schön’s interpretation train of thought did not occupy a second. I then
of reflection.
Table 1.
What happens Schön’s interpretation Heidegger’s interpretation

No breakdowns, no surprise Knowing-in-action Absorbed coping, availableness
Short breakdown Reflection-in-action Conspicuousness
Persistent breakdown Reflection-on-action Obstinacy, occurrentness
Flaw Reflection-on-action Obstrusiveness, occurrentness

remarked that you came from Afghanistan, and in the social world, from a cognitive process to
you were astonished. a social practice.
All of this means that nobody can master a
The question is then how to train to become job and become an expert outside of a commu-
an expert. First, we believe that an expert cannot nity of practitioners. If one wants to learn the
be trained only in a classroom. To become an job of a doctor (i.e., to learn to be a doctor), one
expert one must have a significant professional must practice inside a community of doctors; if
experience, where one has been coping with many one wants to become an entrepreneur, one must
different situations, in particular situations leading practice entrepreneurship inside of a community
to breakdowns. In addition to that, one must have of entrepreneurs.
an impressive record of cases in one’s profession Lave and Wenger (1991) created the concept
(other people practices). Also, it is necessary to of legitimate peripheral participation (LPP) to
study with a master in order to imitate his/her draw attention to the point that learners inevita-
actions and to “steal” part of his/her knowledge bly participate (more or less) in communities of
(Brown & Duguid, 1996). In order to do this, the practitioners and that the mastery of knowledge
best way is to be (again) thrown in daily work requires newcomers to move toward full participa-
situations where one can work side by side with tion in the socio-cultural practices of a community.
a master—a good manager, a chief engineer, an Therefore, they stress the point that, in order to
experienced technician, a senior scientist, a well facilitate learning, one must create an environment
known artist, and so forth—and look at the master that facilitates LPP, facilitates access to practice,
way of doing. access to ongoing work activities, and access to
What the classroom can do for learning ex- practical expertise.
pertise is (a) to provide an important collection of Building on situated learning, Etienne Wenger
external practices in order to enrich the expert’s (1998) developed the concept of communities of
repertoire of situations (a repertoire of cases al- practice, which are informal structures that gather
low people to make situational discriminations people linked through a common practice, which
while being in action), and (b) a place to reflect is also recurrent and stable in time. Communities
on practice with peers. In corporate learning, of practice always develop around what matters to
communities of practice are the best candidates its members; therefore, if one wants to facilitate
to provide the above (Wenger, 1998; Wenger, LPP and to “learn to be,” a community of practice
McDermott, & Snyder, 2002). is a good candidate.
When a learner moves from “competence” Following this logic, la Caixa has started to
to “expertise” the learner is also moving from cultivate some emergent communities of practice,
“learning to do...” to “learning to be...”. I think for instance communities of branch managers.
that in order to “learn to be...” one must go beyond There is a big online community where branch
teaching. The works of Jean Lave and Etienne managers have discussions on their ongoing
Wenger (Lave & Wenger, 1991) and John Seely problems at work, thus sharing their practical
Brown and Paul Duguid (Brown & Duguid, 1991, knowledge. And there are some local communities
2002), have clearly shown that learning is a social of practice where branch managers meet online
process. Moreover, this kind of learning takes and face-to-face.
place in a situated action (in space and time). Participation in communities of practice will
The above authors make a breakthrough in the certainly allow for being thrown in the specific
theory of learning by shifting the focus from the situations and context of the professional practice
individual as learner to learning as participation one wants to master. Also, it will give access to a

vast repertoire of cases on the topic to be learned. characteristics of ICT is the possibility to access
However, if the participation in communities of information “anywhere anytime.” If one wants to
practice is obviously a good way of professional learn about cosmology, one can easily imagine
development, that doesn’t mean that it will allow oneself accessing resources like e-books, articles,
per se a given professional to become an expert. simulations, interviews of well known cosmolo-
It can of course help, but in order to become an gists, and documentaries on the topic.
expert one must also work with a master and have If one wants to learn a skill (i.e., “learn to
the will to continuously improve one’s professional do...” and “learn to be...”), ICT can still support
practice which cannot be reached only by sharing and enhance learning but with some nuances. In
knowledge with peers. the classroom, ICT can always open possibilities
in order to support and enhance learning. But if
we move to distance learning, things are much
Added-vALue And LImIts oF Ict more difficult because of the role of the body in
For the dIFFerent kInds oF learning.
LeArnIng In order to clarify the discussion, I will present
examples of a motor skill (e.g., learning to play
Following Heidegger, when thinking at ICT as a soccer, karate, or learning to dance), and a cogni-
tool to enhance learning, the educational designer tive skill (communication, negotiation, economy,
must ensure that tools will be ready-at-hand. information systems implementation, etc.).
That means that ICT should essentially be “easy At level “Beginner,” if one wants to learn a
to use.” For instance, learners should not need to motor skill, ICT can support learning by giving
change tools in order to access discussions at a access to videos and documentaries especially
distance: in this sense, using e-mail in order to designed for this purpose, including exercises
participate in a community of practice could be and sequences showing particular aspects and
better than a dedicated platform, except if every techniques. A good example of this is Jane Fonda’s
contribution to the online discussion is routed to famous videos on aerobic dancing. Also, a learner
the learners’ e-mail box. could be filmed and then watch and discuss this
But technology is not only a tool. Its impres- recording with a distant coach in order to look
sive power comes usually from its disrupting for what needs to be improved. The same ap-
characteristic of being a possibilities opener. plies if one wants to start learning a martial art
Therefore, above all, the educational designer like karate, or improve his soccer techniques.
must ask himself: which new possibilities are this However, it seems obvious that in order to learn
technology opening in order to support/enhance to play soccer, one must play with other players
learning? Or, more precisely, which new possibili- and that it is impossible to do this at a distance.
ties are ICT opening in order to support/enhance Playing soccer is not the same as playing soccer
heideggerian based learning? with a Play Station!
As long as one address only the “learn about...” If one wants start to learn a cognitive skill, ICT
kind of learning, ICT can always support and can allow the learner to access at a distance a series
enhance learning: to calculate quickly, to draw of well designed cases that will throw the learner
and redraw, to accelerate and to slow time hence in the proper situations, give the learner access
seeing what is otherwise impossible to see, and to rules and procedures, and allow the learner
so forth, all of this allowing to create microworlds to experiment with emotions and involvement.
where one can experiment without risks (Papert, Thanks to computer-based cooperative work tools,
1993). In particular, one of the most powerful the distant professor can also organise discussions

on the cases. Obviously, at this level ICT can also of the most difficult words. An important
enhance classroom teaching. feature is that the article disappears from the
Examples are: screen after 10 minutes. In other words, the
programme throws the student in the same
• The Technology Enhanced Active Learning situation the student will be in during the
(TEAL) project at the Massachusetts Insti- exam and helps to prepare the discussion;
tute of Technology (MIT), where students also it gives the students a vast repertoire of
learn physics moving seamlessly between cases on the topic to be learned. Technology
non traditional lecture, hands-on experi- is “easy to use,” “ready at hand.”
ments, and discussion. Classrooms consist of • The Practicum in Law at the Open University
13 tables with 9 students per table. Most of of Catalonia is an online simulation of the
the student work involves building, running, practical training that students must run in
and experimenting with simulation models law firms. The students access a simulated
and then solving problems. No traditional office (with tables, chairs, computers, tele-
lecture takes place; rather, professors and phones, law books, and a virtual boss) where
their teaching assistants walk around from there is some work to do. The virtual boss
table to table, see what interesting issues are asks something of the student by letting
unfolding, and occasionally interrupt the messages on the virtual table which, after
entire class to discuss something that a par- clicking on, the student can read. All of
ticular table is encountering (Brown, 2005). the documentation necessary to do what is
In particular, TEAL provides impressive requested is available. The student must do
media-rich visualizations and simulations the requested work, fill the documents, and
delivered via laptops and the Internet that send them to the boss (in fact, a professor)
allows students to “see” what is otherwise who will comment and suggest actions to
impossible to see: electromagnetism, elec- take. The process continues until the work is
trostatics, and so forth. By doing this, the completely done. Again, the system throws
whole TEAL system throws students in the the learner in the situation where the learner
context of research in physics; also, every must know and, in the context of the practice,
session start by concrete examples and in- must master, always starting by local and
volved practice. concrete examples.
• A CD-ROM designed in a school of the
Chamber of Commerce and Industry of At level “Competent,” if one wants to learn a
Paris, which help students in preparing their motor skill, it is necessary to enter a face-to-face
business English exam. During the exam the apprenticeship. At this level, mastering karate
students must read an article from the busi- needs a significant experience in fighting because
ness press (Business Week, The Economist, it is in fights (and not in exercises) where one will
Financial Times, Fortune, etc.) and then be confronted to breakdowns (caused by the oppo-
summarise the text in a discussion with nent). Furthermore, the same applies when learn-
the professor. Only 10 minutes are allowed ing a dance and a collective sport: ICT can only
to the student to read and understand the help to record the learner movements and separate
article and prepare the discussion that lasts it into its elements in order to analyse errors, as it
for other 10 minutes. The CD-ROM contains is done with high performance athletes.
a random selection of ad hoc articles and has In learning a cognitive skill at this level,
a dictionary that allow for rapid consultation ICT can still support significant enhancements,

because ICT-based scenarios can throw students never properly handled in PCCE. Case studies
in situations where they will be confronted to usually give all the information in one time but
breakdowns. Examples of this are: it never happens like that in real life. In doing
this, traditional case studies train students to
growltiger analyse facts of the past rather than to cope with
present situations. By giving the case in several
This is a software for simulating structures in parts, one can create breakdowns hence training
civil engineering at MIT. This programme was students to respond to the changes and to reframe
conceived in the beginning as a design tool but the problem in the light of new information on
quickly became a very powerful learning tool. It the situation.
incorporated a finite element algorithm for study- In his book Testaments Betrayed: An Essay
ing equilibrium forces. Students could draw on in Nine Parts, Milan Kundera summarises bril-
the screen a structure such as a beam of a truss liantly this question of time and our ontological
for a bridge, specify the materials and the dimen- impossibility to know the future. In the chapter
sions, then lead the bridge, and the programme “Paths in the Fog” he says that man proceeds in
showed them deflections, moment diagrams, and the present always as the one who walk in the
so forth. fog: unsure of what the next moment may bring.
Students could simulate the structure’s be- Walking in the fog one can see the edge of the
haviour under different load conditions, explore path, what happens near, and react, and one
the space of possible bridge designs, and find can see 50 meters ahead, but not beyond. This
“surprises” in this process. We can see here re- fundamental truth should be scenarised in our
flection-in-action: “interacting with the model, case studies: instead of training students in the
getting surprising results, trying to make sense illusion of rigorous planning based on data, one
of the results, and then inventing new strategies should train them to work with uncertainty and
of action on the basis of this new interpretation. breakdowns. In other words, one must put fog in
Students could iterate very quickly with this de- case studies.
sign tool” (Schön, 1996). In Heidegger’s words, Online learning open new possibilities to do
Growltiger helps to design situations where one this. Case studies can then take place in some
must deal with breakdowns; also, it gives the short “chapters” where the professor gives new
student a vast repertoire of situations, cases, and information on the situation, hence changing it
so forth, on the topic to be learned. and asking students: “What will you do now?”
Organising an asynchronous discussion will
walking in the Fog allow for reflection-on-action, while running a
discussion with a small group using synchronous
This is a case study on IT project implementa- discussion tools (e.g., a chat room) will force
tion that I have implemented in different univer- reflection-in-action. In both cases the instructor
sity settings, both face-to-face and online. The can design situations where one must deal with
case—called NetActive City—tells the story of breakdowns.
the implementation of a virtual school of entre-
preneurs and a virtual incubator. The main point A Blended Learning course on
is that the case is given in several “parts” and the communication
situation change as time goes by (as it happens
in real life!). As a matter of fact, the question of Another example of a course aimed at learning
time is a fundamental one but, unfortunately, from breakdowns is the one designed on commu-
0
nication for la Caixa employees.12 The course is subsequent online discussions allowing them to
structured in six learning units, every unit having reflect on action.
the same structure as below: Applying Heidegger ideas, one can see that
in this course the learner is thrown in the situ-
• First, trainees read a story (a mini-case that ations where the learner must know or which is
tells a story with a breakdown, a surprise, the context for the practice that must be mastered.
that can be interpreted in terms of human Every learning unit starts by local and concrete
communication) and participate in an online examples (those of the bank) and then move gradu-
discussion of this story in a forum. The mini ally to detached reflection; also, the situations are
cases create not only breakdowns but also designed in order to deal with breakdowns.
emotional involvement because the stories At level “expert,” if one is learning a motor skill
are real stories of what happens in the daily almost nothing can be done with ICT in order to
work at this savings bank. support or enhance learning. To reach this level,
• Second, trainees are encouraged to access one must train the body to respond skilfully to
some readings on communication theory different situations, and this can only be done by
that allow for a new interpretation of the practising the skill; more precisely, by putting the
story. Then follows an online discussion body to practise the skill.
of participants’ own examples of the same In the case of a cognitive skill, the best way
kind of story. to become an expert is still to practise the skill
• Third, following a given procedure, trainees again and again, under the guidance of a master,
must run a face-to-face exercise on human and to acquire a vast repertoire of cases. ICT
communication (with a colleague, a friend, can help to do the later by giving access to a lot
etc.), then report the results via e-mail, and of material and allowing participation in virtual
finally participate in an online discussion communities of practice and/or in virtual learn-
on what happened in this exercise. ing sets. However, the density of interaction in
• Finally, trainees must write an evaluation learning sets—learner/learner interaction and
report of the above exercises, in the light learner/instructor interaction—is usually very
of what they have learned. high; therefore it is difficult to have good discus-
sions at a distance. It is better to run it face-to-face,
As we can see, this course is not a completely in a classroom or in a workroom.
online course. Participants must do some face-to- Moreover, in order to imitate a master’s actions,
face activities. This is because human communi- one needs to work side by side with the master,
cation is an embodied phenomenon. As human because in order to experience how to respond
beings, we are not like minds in a vat; we have directly to the risky and perceptually rich situa-
bodies and our bodies play an important role in the tions that the world presents, in order to capture
communication process. Therefore, if one wants the expert’s style, in order to learn abilities for
to learn to communicate (which is not the same which there are no rules, and so forth, we must
than to learn about communication) one must experience with our whole bodies, with the five
also train the body to communicate and reflect senses, and not only those that can be easily me-
on what happens to the body in the face-to-face diatised by ICT (e.g., sight and hearing).
exercises. Moreover, we think that in human com- In other words, I completely agree with Hu-
munication courses face-to-face exercises are the bert Dreyfus when he says that, at the level of
only way to allow people to reflect in action, the expertise, distance apprenticeship is an oxymoron
(Dreyfus, 2001).

Future reseArch dIrectIons social skills. Massive Multiplayer Online Games

(MMOG), like World of Warcraft, involves
As I have said earlier, if one wants to learn a hundreds of thousand kids lined up (Thomas &
motor skill at level “Competent,” one must enter Brown, 2006). I strongly believe that game-based
a face-to-face apprenticeship. However, from a learning, and in particular especially designed
theoretical point of view, it could be possible to MMOGs, could be an important trend in e-learn-
reach competence in learning a motor skill without ing innovation.
having to interact face-to-face with other human
beings. Chilean biologists Humberto Maturana
and Francisco Varela discovered that our nervous reFerences
system is a closed system. As a consequence, our
nervous system is unable to distinguish from two Brown, J.S., & Duguid, P. (1991). Organizational
identical stimulii coming from outside (Maturana learning and communities of practice: Toward a
& Varela, 1984). More precisely, the nervous unified view of working, learning and innovation.
system will react identically if our senses are Organization Science, 2(1).
stimulated by another human being (e.g., another
Brown, J.S., & Duguid, P. (1996). Stolen knowl-
fighter or dancer) or by a virtual reality system.13
edge. In H. McLellen (Ed.), Situated learning
Therefore, one can imagine learning to dance with
perspectives (pp. 47-56). Englewood Cliffs, NJ:
a virtual reality system. Even if nowadays this is
Educational Technology Publications.
not the most cost-effective system to learn how
to dance (to say the least!), virtual reality seems Brown, J.S., & Duguid, P. (2002). The social life
a promising field of research in order to design of information. Boston, MA: Harvard Business
ICT-based learning systems to reach competence School Press.
in a motor skill.
Brown, J.S. (2005). New learning environments
As virtual reality could be a future trend in
for the 21st century. Paper presented at the Forum
reaching competence when learning a motor skill,
for the Future of Higher Education’s 2005 Aspen
videogames can open new possibilities when
Symposium.
learning a cognitive skill at this level. Every father
who has carefully observed his son playing with Dreyfus, H.L. (1986). Mind over machine. New
a videogame, whether it is on a computer or on York, NY: Free Press.
other devices like a mobile phone, a Gameboy,
Dreyfus, H.L. (1991). Being-in-the-world. Cam-
or a Play Station, could notice how quickly he
bridge, MA: MIT Press.
decides and take action. If one is not extremely
good at pattern recognition, sense-making in Dreyfus, H.L. (1992). What computers still can’t
confusing environments, and multitasking, one do: A critique of artificial reason. Cambridge,
will not succeed in the game world. In this world, MA: MIT Press.
one is immersed in a complex, information-rich,
Dreyfus, H.L. (2001). On the Internet. London:
dynamic realm where one must sense, infer, de-
Routledge.
cide, and act quickly, always responding at new
situations (Brown, 2005). In other words, one must Lave, J., & Wenger, E. (1991). Situated learning.
be good at reflection-in-action, hence becoming Legitimate peripheral participation. Cambridge,
a master in dealing with breakdowns. UK: Cambridge University Press.
Moreover, thanks to the Internet, a new gen-
eration of videogames allow for the learning of

Maturana, H., & Varela, F. (1984). El árbol del Thomas, D., & Brown, J.S. (2006). The play of
conocimiento. Santiago de Chile: Editorial Uni- imagination: Extending the literary mind (Work-
versitaria. ing Paper). Retrieved October 17, 2007, from
http://www.johnseelybrown.com
Mintzberg, H. (1988). Formons des managers, non
des MBA! Harvard-L’Expansion, nº 51, 84-92. Vasquez Bronfman, S. (2005, September). A
Heideggerian perspective on reflective practice
Mintzberg, H. (1996). Musings on management.
and its consequences for learning design. Paper
Harvard Business Review, 74(4), 61-67.
presented at the 11th Cambridge International
Mintzberg, H. (2004). Managers, not MBAs. San Conference on Open and Distance Learning,
Francisco, CA: Berrett Koehler. Cambridge (UK).
Papert, S. (1993). Mindstorms: Children, com- Wenger, E. (1998). Communities of practice:
puters, and powerful ideas (2nd ed.). New York: Learning, meaning and identity. Cambridge, UK:
Basic Books. Cambridge University Press.
Pedler, M. (1991). Action learning in practice. Wenger, E., McDermott, R., & Snyder, W.M.
London: Gower. (2002). Cultivating communities of practice.
Boston, MA: Harvard Business School Press.
Pfeffer, J., & Sutton, R. (2000). The knowing-do-
ing gap. Boston, MA: Harvard Business School Wikipedia. (2006). Martin Heidegger. Retrieved
Press. October 17, 2007, from http://en.wikipedia.org/
wiki/Heidegger
Revans, R. (1980). Action learning: New tech-
niques for management. London: Blond & Winograd, T., & Flores, F. (1986). Understanding
Briggs. computers and cognition: A new foundation for
design. Norwood, NJ: Ablex.
Schön, D.A. (1983). The reflective practitioner:
How professionals think in action. New York: Wrathall, M., & Malpas, J. (2000). Heidegger,
Basic Books. coping, and cognitive science: Essays in honor
of Hubert L. Dreyfus (Vol. 2). Cambridge, MA:
Schön, D.A. (1987). Educating the reflective
MIT Press.
practitioner. San Francisco: Jossey Bass.
Schön, D.A. (1996). Reflective conversation with
materials. In T. Winograd et al. (Eds.), Bringing
AddItIonAL reAdIng
design to software (pp.171-184). Reading, MA:
Addison-Wesley.
Brown, D., Richards, M., & Barker, J. (2006).
Schön, D.A. (1997). A la recherche d’une nouvelle Massively multi-player online gaming: Lessons
épistémologie de la pratique et de ce qu’elle impli- learned from an MMOG short course for high
que pour l’éducation des adultes. In J.M. Barbier school students. In T. Reeves & S. Yamashita
(Ed.), Savoirs théoriques et savoirs d’action (Eds.), Proceedings of World Conference on
(pp. 201-222). Paris: Presses Universitaires de E-Learning in Corporate, Government, Health-
France. care, and Higher Education 2006 (pp. 404-406).
Chesapeake, VA: AACE.
Spinosa, C., Flores, F., & Dreyfus, H.L. (1997).
Disclosing new worlds. Cambridge, MA: MIT Burbules, N.C. (2006). Rethinking the virtual.
Press. E-Learning, 1(2), 162-183.

Conolly T.M., & Stansfield, M. (2006, June 25-28). Foreman, J. (2004, September/October). Game-
Using interactive technologies in teaching an on- based learning: How to delight and instruct in the
line information systems course. In Proceedings 21st century. Educause Review, 51-66.
of the 2006 Informing Sciences and IT Education
Galarneau, L. (2005, June 16-20). Spontaneous
Joint Conference, Salford, UK.
communities of learning: A social analysis of
De Freitas, S. (2006). Learning in immersive learning ecosystems in massively multiplayer
worlds: A review of game-based learning. Paper online gaming (MMOG) environments. Paper
prepared for the JISC e-Learning Programme. presented at the International DiGRA Conference,
Retrieved October 17, 2007, from http://www. Vancouver, British Columbia, Canada. Retrieved
jisc.ac.uk/media/documents/programmes/elearn- October 17, 2007, from http://www.gamesconfer-
ing_innovation/gaming%20report_v3.3.pdf ence.org/digra2005/overview.php
DeWolfe Waddill, D. (2007). Action e-learning: Gibbs, P., & Angelides, P. (2004, September). Ac-
An exploratory case study examining the impact creditation of knowledge as being-in-the-world.
of action learning on the design of management- Journal of Education and Work, 17(3).
level Web-based instruction. In M.K. McCuddy
Graves, M. (1998). Learning in context (Working
et al. (Eds.), The challenges of educating people
Paper). Retrieved October 17, 2007, from http://
in a challenging world (pp. 475-497). Springer.
www.apple.com/education/LTReview/winter98/
Dreyfus, H.L. (2002). Intelligence without rep- context.html
resentation: Merleau Ponty’s critique of mental
Kreisler, H. (2005). Meaning, relevance, and the
representations. Phenomenology and the Cogni-
limits of technology: Conversation with Hubert
tive Sciences, 1(4).
L. Dreyfus. Retrieved October 17, 2007, from
Dreyfus, H.L., & Dreyfus, S.E. (1985). From http://globetrotter. Berkeley.edu/people5/Drey-
Socrates to expert systems: The limits and dan- fus/dreyfus-con1.html
gers of calculative rationality. In C. Mitcham &
Nardi, B.A., Ly, S., & Harris, J. (2007). Learning
A. Huning (Eds.), Philosophy and technology II:
conversations in world of warcraft. In Proceedings
Information technology and computers in theory
of the 40th Hawaii International Conference on
and practice. Reidl.
System Sciences, Hawaii.
Dreyfus, H.L., & Dreyfus, S.E. (1999). Appren-
Squire, K. (2005). Game-based learning: State of
ticeship and expert learning. In K. Nielsen & S.
the field. Masie Center. E-Learning Consortium.
Kvale (Eds.), Apprenticeship, learning from social
practices. Denmark: Hans Reitzels Forlag.
masie.com/xlearn/Game-Based_Learning.pdf
Ducheneau, N., & Moore, R.J. (2005). More than
Steinkuehler, C.A. (2004). Learning in massively
just “XP”: Learning social skills in massively
multiplayer online games. In Proceedings of the
multiplayer online games. Interactive Technology
6th International Conference on Learning Sci-
& Smart Education, 2, 89-100.
ences, Santa Monica, California (pp. 521-528).
Duesund, L. (2000). Teaching and learning: An
Van Manen, M. (1995). On the epistemology of
interview with Hubert Dreyfus. Pedagogiske
reflective practice. In Teachers and teaching:
utfordringer, 2. The Norwegian University of
Theory and practice. Oxford Ltd. 1(1), 33-50.
Sport and Physical Education. Retrieved October
17, 2007, from http://www.nih.no/kunnskap_om_
idrett/index.html

Wierinck, E. et al. (2005). Effect of augmented when technology is “transparent,” one is

visual feedback from a virtual reality simulation completely concentrated on what one is
system on manual dexterity training. European doing with the technology (our relationship
Journal of Dental Education, 9(1). with equipment is always purposeful).
8
The IT Department is spread across three
Yoo, Y.-H., & Bruns, W. (2005). Motor skill
different buildings.
learning with force feedback in mixed reality. In 9
Hubert Dreyfus calls skillful coping not only
Proceedings of the 9th IFAC Symposium on Analy-
the way people deals with daily work situa-
sis, Design and Evaluation of Human-Machine
tions, but mainly the smooth and unobtrusive
Systems, Atlanta, FL.
responsiveness to those situations (Wrathall
& Malpas, 2000).
10
“For the case study method to work, the
endnotes students must become emotionally involved.
So, in a business school case study, the stu-
1
Professional education refers to university dent should not be confronted with objective
education (either undergraduate or post- descriptions, but rather be led to identify
graduate) of architects, engineers, doctors with the situation of the senior manager and
in medicine, business professions, and so experience his agonized choices and subse-
forth (see Schön, 1983). “Continuous and quent joys and disappointments” (Dreyfus,
corporate education” refer to all educational 2001).
activities (either performed in-company or 11
Breakdowns have also another benefit: they
not) that do not lead to a degree. put people in the right mood for learning
2
To be rigorous, “information” rather than because it reveals what they are not able to
“knowledge” should be written here (see do.
12
Brown & Duguid, 2002). In fact, this course on communication has
3
By e-learning, we mean here not only ICT- been designed by a company whose members
based distance education, but more gener- were trained in the applications of Hubert
ally every use of ICT to support or enhance Dreyfus’ ideas, among others.
13
learning. See for instance the Wikipedia article on
4
As human beings, we are always thrown Virtual Reality (http://en.wikipedia.org/
in a given situation. A child is born in a wiki/Virtual_reality).
given hospital, city, country, and will live
with a given family. When we are at work,
we are thrown in meetings, conversations
with customers, computer programming,
architectural design, and so forth.
5
For instance, in order to innovate one does
not need to have a theory of innovation
(Spinosa et al., 1997).
6
John Seely Brown and Paul Duguid (2002)
make the distinction between learn about
and learn to be, to which I add learn to do.
7
For instance, when using e-learning tech-
nology, one does not think at it. Instead,

Chapter III
Philosophical and
Epistemological Basis for
Building a Quality Online
Training Methodology
Antonio Miguel Seoane Pardo

Universidad de Salamanca, Spain
Francisco José García Peñalvo

ABstrAct
This chapter outlines the problem of laying the groundwork for building a suitable online training
methodology. In the first place, it points out that most e-learning initiatives are developed without a
defined method or an appropriate strategy. It then critically analyzes the role of the constructivist model
in relation to this problem, affirming that this explanatory framework is not a method and describing
the problems to which this confusion gives rise. Finally, it proposes a theoretical and epistemological
framework of reference for building this methodology based on Greek paideía. The authors propose that
the search for a reference model such as the one developed in ancient Greece will allow us to develop a
method based on the importance of a teaching profile “different” from traditional academic roles and
which we call “tutor.” It has many similarities to the figures in charge of monitoring learning both in
Homeric epic and Classical Greece.
Philisophical and Epistemological Basis for Building a Quality Online Training Methodology
IntroductIon sources are devoted to perfecting these elements

in the hope of improving the learning experience,
the Failure of e-Learning without without our realizing that the solution to the
a method problem lies in another direction.
Logically, the evolution of these technological
Online training or e-learning is an authentic mediation factors will contribute to improve the
revolution in its way of conceiving learning ex- context, just as we would improve the learning
periences compared to how we thought of them experience if we renewed the blackboards, the
until very recently. It would take too long to list lighting, or the equipment of a classroom in a
all the changes that have taken place in this new traditional context. However, we all know that
educational modality, which have affected techno- this is not the main thing for achieving quality
logical elements, communication dynamics, social training. And looking back on our own experience,
factors, and new teaching and learning roles, as we all remember that we learned nothing, or very
well as the teaching-learning relationship itself, little, from the technical or logistic elements in
the value of the contents, and the methodology our schools but we did learn a lot with our good
of the process. teachers and classmates. Thus, technology must
However, despite the euphoria unleashed by be improved but we can not fall into the trap of
online training in recent years, and the fact that only blaming the tool for not being able to reach
the development of tools, training systems, and the desired objectives. Technological mediation
digital contents has been and still is extraordi- in e-learning is precisely that, a medium, and in
nary, we can not hide from the fact that there any case it is a mistake of training strategy not
is a certain skepticism or even disappointment to have had suitable resources, or not to have
when the level of user satisfaction and the out- been capable of adapting ourselves to the means
comes attained in online training are analyzed, available. The tool is, or we should make it be,
if we limit ourselves exclusively to the learning as neutral as possible.
objectives actually attained. What is important in All in all, if we study the brief history of e-
any educational intervention, whatever its type, learning we can already speak of “generations”
electronic, at a distance, or face-to-face, is none that have marked its development up until now,
other than achieving certain learning objectives: and whose evolution allows us to predict (assum-
the proof of having taught them does not suffice; ing that this is possible) where we are going in
we need to be sure that they have actually been the future (Seoane, García, Bosom, Fernández,
acquired. & Hernández, 2007). After a first generation
Since e-learning is a type of learning charac- marked almost exclusively by the development of
terized by technological mediation (this is not its technological environments and digital contents,
only peculiarity, but for the time being we will we have moved towards a concern, in recent years,
focus on this aspect) and since what is apparently for the e-learning “model” and, consequently, to
different with respect to other kinds of training a concern for the development of implementa-
seems to lie in the elements of this mediation, tion strategies and the interoperability of online
when we analyze the causes of this skepticism training environments with an institutional model
(or failure) we usually focus exclusively on the for the university, the public administration, and
technological factors: the learning environments business firm. Thus the question of a model of
are not appropriate, the digital contents are not efficiency and quality appears. However, we are
well-structured, and so forth. Consequently, an witnessing a moment in which a strange paradox
enormous amount of material and human re- is occurring: the greater the technological media-

tion and the more we implement our systems and of the internalization and assumption of specific
improve the environments and training contents information with a significant purpose. Thus, as
with a view to reducing the intervention of the can be seen, education presupposes information,
teaching roles, the worse the learning outcomes. but it is more than that. That is why educational
It is becoming necessary to go on then towards material alone can not “educate.” This can only
what we call the “second-advanced generation” be done by the subject who becomes educated by
in which the importance of the human factor internalizing, by becoming aware of the value of
in online training plays a crucial role, not only the contents, by building a meaningful universe
from the point of view of planning and strategic within him or herself, or, what is more common,
design, but also, and especially, as an element by the mediation of other human beings, who,
present in all the stages of the training itinerary. either individually (with a teacher) or collectively
The redefining and centrality of teaching roles (with a group of students or in the social context
in e-learning is the main characteristic of this itself), contribute to turning information into
generational stage, at which many institutions an educational experience in the mind of the
and training initiatives really concerned about individual. This dichotomy can be compared to
quality currently find themselves. what in philosophical terms Aristotle (and later
Thus, the cornerstone that will allow us to the Aristotelians, specially Thomas Aquinas)
explain the reason for the disappointment in e- called “active intellect” and “passive intellect”
learning up until now is the human factor. The (Aristotle, De anima, 430a 10-25; Thomas Aqui-
great fallacy of technological mediation has nas, Summa Theologica, first part, question 79)
consisted of the belief that the mediating role or to the cognitive distinction between memory
of the classroom teacher would be replaced by and consciousness.
technology, when the latter should really be at Thus, education is more than information.
the service of the teacher, who will still carry on And if we wish to attain it, we have to go beyond
playing the main mediating role in learning. technological mediation and learning objects to
This mistake, as widespread as it is serious, speak of human interaction both among students
is the consequence of the transmutation of a and with teachers, because this is where the suc-
training paradigm into one of an “informative” cess or failure of most educational or training
nature. In other words, we can say that underly- initiatives begins.
ing this matter there is an enormous confusion Hence, it seems that two major questions still
between information and education (or training). remain to be solved (perhaps because they have
This situation is not at all new and has come up not been sufficiently well-defined) before we face
when analyzing the problems of other “classical” the main problem: on the one hand, it is necessary
training paradigms, but with online training it to define a suitable interaction model for online
has been taken to its ultimate consequences, most learning, taking advantage of the fact that the
likely because of the emergence of the so-called tools available make possible new modalities of
“information society” and information and com- communication up until now impossible (Seoane
munication technologies. Their names are accu- et al., in press); on the other hand, not only do we
rate enough, but they seem to have subliminally have an unsuitable definition of the teaching at-
taken on educational aspirations. Indeed, a book, tributions and profiles in online training, but they
a newspaper, the Internet, or audiovisual mate- are also being drastically reduced or eliminated.
rial can provide us with information, but never They often end up becoming mere dynamizers
education or training. Education is a specifically and stimulators of learning, as if they were the
human activity that consists, among other things, “cheerleaders” of training. Absurd, right? But
absolutely true in many cases.

But the main problem which our e-learning fining the educational model for many e-learning
initiatives often face is the total lack of a suitable interventions and about which pages and pages
method for their development. When we speak have been written, we will discover that these
of the “second generation” of e-learning we are communities favor a high degree of interaction
referring to a strategic approach of the training and communication, but we will not be able to
model required by the entity implementing it. avoid a certain feeling of anarchy and loss of time
This strategy determines the “what” and the in most of these collective groups. To use Kantian
“for what,” but only the development of an ap- terms, there are many theses and antitheses, but
propriate methodology will make it possible to few syntheses and above all there is still great
develop “how” the pre-established objectives will difficulty in determining who has attained certain
be achieved. training objectives and to what degree. Further-
But is it really true that no methodology for more, we lack a certain criterion of authority (in
online training has been designed after all these the Latin sense of the term auctoritas) which
years? Well, an analysis of a good part of the train- makes it difficult to select the best syntheses of the
ing initiatives and even the specialized literature common task because there is a belief (generally
certainly seems to show this. On the one hand, naïve) that in these communities a final synthesis
if we focus on a purely technocentric model, in of knowledge per se is produced, when what usu-
which the addressee gains access to knowledge ally happens is that, when this does occur, each
and “interacts” with it without any other media- member contributes his/her view of the problem,
tion, there is no method with educational ends but neither a conclusion nor a consensus is reached
and the most we can affirm is whether there has on it. This is so because although e-learning envi-
been (or not) a good sequencing and organization ronments “transform the social interaction space,
of the information and whether or not the student … a deeper understanding of the ‘inside’ of the
has been able to respond suitably to some test collaborative learning processes is still missing”
items that prove that this information has been (Cecez-Kecmanovic & Webb, 2000). Of course,
acquired, but not whether real training has taken learning communities, especially when made up
place. Thus, no matter how much we theorize over of qualified adult individuals, are instruments
these aspects, we will not be going in the right with high educational capacity thanks to the
direction in our quest for a training method. possibilities of interaction and communication
Moreover, if we look at other initiatives based and their potentiality for favoring contexts of
on predominantly vertical human interaction critical and active construction of knowledge.
(student-teacher-student), we find that there is However, the problem of learning communities,
no substantial change with respect to certain at least in the shape they have taken in a good
face-to-face contexts, which leads us to the same part of prior experience, lies in their excessively
problems as in face-to-face teaching without, on “democratic” approach. Favoring a cognitive
the other hand, being able to make good use of and social presence in these communities is not
the advantages of a completely different inter- enough. In order to be able to design, direct, and
action and communication model. It would in nurture interaction in a learning community, a
any case be a similar model to that of tutoring strong teaching presence is necessary. This does
in traditional distance education, which differs not have to affect the open and critical nature of
considerably from the paradigm we are seeking these communities; what is more, the key factor
for e-learning. for success in these communities will lie in the
Finally, if we analyze initiatives and studies teacher’s ability (as in face-to-face teaching) to
on learning communities, a key concept for de- create a suitable climate that will favor the setting

up of a genuine learning community, one that is the ideas of Vygotsky (Vygotsky & Cole, 1978),
perfectly monitored and well-constructed (Gar- those of Bruner (Bruner, 1997, 1998) and even
rison & Anderson, 2003, 2005). those of Dewey (1933, 1938) form part of an ideo-
Thus, we have contexts, we have interaction logical and philosophical context developed dur-
models and, of course, technology, but we lack ing the 20th Century in opposition to the methodic
methods for the development of quality training individualism and transcendental philosophies of
initiatives. A method is nothing more than a consciousness that were developed up to the 19th
guide or instructions as to the road to follow to Century and which had their last great exponent
reach certain objectives. In this case, the method in Hegelian idealism. Philosophical approaches
has to be understood in a three-fold sense: first, in accordance with this presuppose a new type
as the set of instructions and strategies offered of rationality that replaces an idealist paradigm
to the teacher in order to achieve the learning with another of dialogical, communicative, and
objectives; second, analogous rules must exist for social rationality which we can find in key thinkers
the correct acquisition of the contents on the part of the last century such as Gadamer, Apel, and
of the student (who should also have a method); Ortega y Gasset.
finally, since e-leaning favors social knowledge Thus, constructivism explains, according to
building and social learning is by far the most the ideological presuppositions of its time, how
significant of all those that exist, a method is knowledge is constructed in the human mind. This
needed to regulate social interaction with an does not presuppose the existence of an implicit
educational purpose, especially when we are in method, or that this explanatory framework can
a “non-natural” context such as that of virtual provide us with this method by itself. In simple
learning environments. terms, thanks to cognitivist and constructivist
thinkers, we know that the cognitive process
takes place in a certain way, which does not mean
ConSTRUCTiViSm aS a GoaL, that they have told us how to get our students to
But not As A method acquire the competencies and skills we program
in a learning initiative. This is the difference
One of the terms most used in relation to e-learning between an explanation and a method: knowing
(to the point that its original meaning has become what has happened (and even knowing why) and
completely lost and it is now used gratuitously) knowing how to make it happen again, adapting
is “constructivism,” as a synonym of prestige, it to predetermined learning circumstances.
careful methodology, and good practice. This Therefore we may ask: What does constructiv-
expression can be found in essays on methodologi- ism offer us? What is it good for?
cal approaches or theories for online training, in The thesis here defended postulates that
the explanation of the instructional design of an constructivism can be considered as a goal for
initiative, in the conception of a learning object learning, even as a “table of validation” thanks to
or even (surprisingly) to advertise the virtues of which we will be able to verify the solidity of the
a software tool addressed to online training. knowledge acquired by our addressees. At most,
The problem is that constructivism is not a it could be a guide or perspective for preparing
method, nor even a theory, but rather an explana- a training methodology, but in no case must we
tory framework (Coll et al., 2005) which tells us confuse the end with the means that we intend
that de facto learning occurs in a social, collective to use for reaching our objectives. Constructiv-
context and is the fruit of construction beyond the ism is thus not valid as a method, and the need
solitary consciousness of the individual. Actually, to develop a methodology for online training
remains pending.
0
However, in the name of constructivism, many the information into training thanks to his or her
practices in e-learning have become widespread, mediation. In many cases, this mediation occurs
practices which, based on the supposed virtues “among peers” (how many of us have learned,
of the training paradigm, entail more than a few thanks to our classmates what our teachers had not
difficulties and are to a certain extent responsible been able to make us understand?) but we must not
for the high failure index of online training initia- renounce a teaching figure who, suitably adapted
tives. We now take a look at some of them. to the context, can perform this mediation.
The students, therefore, do not have to be the
the excuse of a student-Focused center of learning but the goal of this task, since
model they are the addressees of the training interven-
tion. In any case, the oft-mentioned “center”
Together with expressions such as “constructivist should be occupied by that element of human
methodology” we often find a reference to the “stu- mediation that here we call “tutor” and who adapts
dent-focused model.” In many cases this statement the training initiative (with all its technological,
is correct, but students end up discovering that it academic, didactic, and human components) to
means something completely different from what the peculiarities of each addressee, takes charge
they expected. In general, placing students at the of guaranteeing the actual acquisition of the
center of learning is usually an excuse to unload competencies and skills foreseen for the train-
the whole weight of learning on them and propose ing initiative and is ultimately responsible (often
a self-learning itinerary with as little assistance as even more so than the student) for attaining the
possible. Indeed, if students are the protagonists, training objectives.
they are solely responsible for carrying out the
learning task. This is the meaning of “occupying the existence of a community is not
the center” in many e-learning initiatives. enough for social Learning to occur
A model in which the student occupies the
center of the training scenario, far from being a Another of the presumed virtues of many online
privilege and a stimulus, in many cases ends up training initiatives with a constructivist approach
being a drawback and gives rise to results con- is the guarantee of training success based on com-
trary to those desired. To show this graphically, munity working dynamics. Gathering together in
the central position of students means that all the one room a hundred splendid musicians will not
elements revolve around them and none of these make this assembly an orchestra, the same as a
elements are a point of reference, but rather they all set of sailors enlisted on the same ship cannot be
have the student as a reference. This image, which considered a crew. For there to be a real com-
may seem somewhat strange, is disconcerting for munity (musical, nautical, sport, or learning) we
many students who are not used to an autonomous need much more than a set of related individuals
style of learning, to setting their own rhythm of in the same space-time or “virtual” context. In-
learning, and to adapting to the peculiarities of deed, as Gestalt psychologists affirmed, inspired
the environment, because the environment never by the old discussion that Aristotle initiated in
adapts to them. his Metaphysics (1028a-1041b), the whole is more
It is true that this training model adapts per- than the sum of its parts.
fectly to the peculiarities of self-taught persons No one will be surprised if we say that the social
with a great ability to turn information into train- context is one of the most efficient and common
ing by themselves. However, most individuals forms of learning, as is shown in the way we ac-
need a figure to act as guide and help them change quire knowledge of our native language—without

the need to enroll in any educational institution- the students involved, but also includes the tutor
and attain a notable mastery of it. However, when or tutors at the head of said community. It is a
we make a set of persons in a training initiative matter of achieving a dynamic or model that some
interact, we have no guarantee that they will form scholars call socioconstructivist, in which the re-
a genuine learning community. Communities sult of social construction is not the responsibility
of students are artificial societies, and making of the students or the teachers (the model is not
the “sum of the parts” into a “whole” is frankly focused on the student or the teacher), but rather
something very complex. Thus, obliging students is the outcome of interaction between learning
to work in a group does not presuppose that they contents, teaching staff, and students (Barberà,
are going to form a learning community. This 2006) by means of a design for activities that fo-
is a problem well-known to tutors and experts ment the acquisition of competencies and skills
in virtual group dynamics who, using the same and that have an eminently practical approach
strategies in seemingly analogous groups, often that favors this interaction.
attain completely different learning outcomes,
both individually and collectively. Turning a tools do not construct
group of students into a learning community is a
real art, as is turning a hundred musicians into an The third of the usual practices that can be
orchestra. The former may even be more difficult observed in many initiatives inspired by con-
than the latter, but this is coming from someone structivism is the use of technological tools and
who has never directed an orchestra. methods that are posited as constructivist per se. It
The dynamics that are set up in a learning is well known that constructivism and especially
community are complex and require detailed social constructionism is the theoretical reference
study. There are magnificent works on learning model for many developers of software for online
communities (Wenger, 1998a, 1998b) but there learning, especially open source. Possibly the
is no method capable of guaranteeing that we best-known system of this type for course man-
will be able to reproduce or build an efficient agement, Moodle (http://moodle.org), confesses
community. Nevertheless, we can affirm that on its main page that its philosophy is “social
opening up debate and promoting team work is constructionist pedagogy” based on four under-
not enough to constitute a learning community lying concepts: constructivism, constructionism,
and to “construct” a social learning context. A social constructivism, and connected and separate
group must have good leadership and be solidly (Moodle, 2007). The creator of this instrument,
structured so that guidelines for behavior can be Martin Dougiamas, has said that his reference
developed that in the end will turn this sum of the model when designing Moodle was the analysis
parts into a whole that functions as an authentic of learning communities based on constructiv-
community. In other words, the possibilities for ism and social constructionism (Dougiamas &
success in the building of learning communities Taylor, 2003).
online (or face-to-face; there are no significant However, the use of Moodle or any other e-
differences in this respect) increase when we learning tool does not guarantee social construc-
start from a situation that includes teaching roles tion, nor does it foment the achievement of certain
that regulate communication flows, establish objectives. The intentionality of the person who
guidelines and rhythms for learning, and foster constructs a tool has nothing to do with the use
the active participation of the members. The that users may make of it and the corresponding
construction of learning in a community is a task outcomes. Was Alfred Nobel responsible for the
that is shared not only by each and every one of belligerent use of dynamite, a compound origi-

nally intended to prevent the constant accidents in The methodology of our online training initia-
mines owing to the instability of nitroglycerine? tives must therefore revolve around the central
Likewise, the tools that we use may be more or and catalyzing figure of the tutor.
less suited to the aims and training strategies of
our activities, but in themselves they do not give
any guarantee whatsoever of constructivist learn- In seArch oF A groundwork
ing. What is more, it could be said that the type of For the method
tool we use is practically irrelevant (as long as it
fulfills certain minimum conditions) compared to contributions from greek Paideía
the importance of a good instructional design, a
correct training strategy, and a good human team Taking into account the starting supposition of
to head the teaching-learning process. these pages, to wit, the importance of monitor-
Let us then assume that a quality online ing learning through a specialized professional
training initiative has to have as its goal that the profile, to which the major share of the training
students should achieve significant, active, learn- methodology will fall, it is evident that we are
ing constructed within a social context whenever not looking at the traditional teaching figure, at
possible in the midst of a learning community. least as understood in our current school systems.
However, in order to achieve this objective, we It is thus a matter of a professional whose main
have to avoid three major obstacles which, like mission is not to emit knowledge but rather to
a tree in front of us, can prevent us from seeing guarantee that it reaches the addressees, in an ac-
the forest. On the one hand, the affirmation of a tive, participative, and significant context. In our
student-focused model does not at all guarantee a opinion, a large part of the success or failure of
construction (much less a social construction) of online training initiatives will depend on whether
knowledge; on the contrary, it can even hinder it. or not we have this type of professional, suitably
On the other hand, we often observe a confusion inserted in a solid and well-constructed context
between group work and learning community, or of training planning.
between group and community. Finally, we have The big question now is as follows: Has there
been able to show that the use of certain tools ever been in the history of education a professional
does not at all condition the social knowledge profile of such characteristics? Do we have any
building, because this depends on the modalities model that can serve as a reference, and from
of interaction that occur in the dynamics of train- which we can develop the role that corresponds
ing activities; thus, they have to do with humans, to our quality teacher in e-learning initiatives?
not with machines. Our answer is clearly affirmative. Indeed, in
In short, learning (in e-learning or in con- Ancient Greece we can find “teaching” models
ventional environments) is the product of social whose characteristics, despite forming part of a
interaction which as such has rules, roles, and context so different from today’s, which is not
defined structures. To extract all its potentialities, even homogeneous, are extraordinarily interest-
it must be correctly moderated and led by some- ing for the task at hand, which is none other than
one with a professionally well-defined teaching designing a suitable teaching profile for online
profile, who plays a particularly important role in training methodology. Briefly, and by way of
online training and on whom the success of our example, below we give the “professional” profile
initiative largely depends. of these personages that will serve as inspiration
for the construction of our online teacher and his
or her methodology.

the mentor as teacher of the There is no room for doubt that the prominence
homeric hero goes not to the teacher but to the disciple, but
neither can it be denied that the constant presence
One of the first testimonies of the teacher-disciple of the teacher, the security that it gives the hero
relationship and thus of interaction between teach- to know that someone is watching over him and
ing profiles and pupils in a learning context that appears when most needed, even placing him
we know of in western culture has its source in before complex situations from which he must
Greek mythology and the Homeric epic. The epic extract new teachings, is a model of training and
heroes acquire their greatness both from their permanent tutoring that is characteristic of the
ancestry and, what is even more interesting, from Greek paideía.
the presence and importance of their teachers, who However, the most characteristic example
not only educate them and prepare them to face from which we can extract greater conclusions
the hazards of the heroic life, but even carry out is that of the relationship between Mentor and
a constant follow-up of their disciple’s actions, Telemachus in the Odyssey.
intervening when they are most needed. According to Homer’s Odyssey, when Odys-
Achilles was taught by no less than the centaur seus left Ithaca and was away fighting in the
Cheiron and by his mother the goddess Thetis; Trojan War, his son Telemachus was just an infant.
the latter intervened even at the moment when So Odysseus entrusted Mentor with the care of
the hero doubted whether he should or should Telemachus and the entire royal household until
not go to the Trojan War, helping him to weigh he came back 20 years later. Although Mentor
his decision. Indeed, Achilles had to choose is not a main character in Homer’s epic poem,
between two ways of living and dying. On the he represents wisdom, trust, counsel, teaching,
one hand, if he decided to stay he would marry, protection, challenge, encouragement, and so
have children and grow old as a king and after on. (Anderson & Shannon, 1995; Carruthers,
his death his descendants would remember him. 1993). Mentor’s authority was so important to
On the other hand, if he decided to go to war, Telemachus than even the goddess Athena took
he would die young without descendants, but the figure of Mentor to persuade the hero’s son
the whole of humanity would admire his deeds to search for his father.
forever. Everyone knows the result of his choice. The role of Mentor instructing Telemachus
Odysseus, for his part, received the permanent is not quite clear in Homer’s poem, and this is
guidance of the goddess Athena, who appeared one of the most interesting questions about the
to him on several occasions to advise him, such matter. Never mind if Mentor (or Athena) is the
as on his return to Ithaca, when she changed him real “teacher” of Odysseus’ son. It is strange
into a beggar and proposed a plan to put an end that Mentor is mentioned just a few times in the
to the suitors that were harassing Penelope and Odyssey and we do not know how he “really”
ruining his property. instructed Telemachus. The only important thing
In these cases we encounter figures that apis that Telemachus achieved enough maturity to
pear in the life of the Homeric heroes, who are know how to face Penelope’s suitors and help his
undoubtedly of greater rank and importance father to complete the final revenge: he became a
than their disciples, but who do not outshine the man with the help of an old person whose mission
actions of their pupils. Rather, the opposite: they was to remain in the dark, “tutoring” Telemachus’
extol them by placing them in circumstances in steps, not helping him but following his tracks at
which they will be able to come out with flying a certain distance, because no one can drive the
colors, magnified by their bravery and preparation. fate of a man except himself. In fact, the unde-

fined, secondary but crucial role of Mentor has the education of man as a citizen:
not changed so much with regard to the excellent the sophists and socrates
“Mentors” of e-learning students nowadays.
Etymologically, “mentor” produced “monitor” Towards the second half of the 5th Century B.C.
in Latin. The verb “maneo” (to show, to indicate) and especially in the last quarter, a real revolution
comes from the Indo-European *man (to think, occurred in the way education was conceived of in
to know). So Homer’s character Mentor is an an- Greece, to be precise, in Athens. The economic,
thropomorphization of this idea: wisdom (Little, social, and political changes that occurred in the
1990), thought, knowledge (and consequently city favored the appearance of new social needs
know-how), personified by an old man whose and a fairly widespread demand for education far
purpose is to transmit these skills. above what had until then been received in the
In the figure of old Mentor we find, then, an family sphere, which only reached a certain level
excellent personification of the role that the online in the higher social strata. This growing demand
training teaching profile should play. From a sup- favored the arrival in Athens of the Sophists, who
porting role, yielding prominence to the disciple, unleashed a whole revolution in the way of con-
he nevertheless invites the latter to act, to solve ceiving education and, of course, aroused great
problems and to learn through action. Learning, controversy which, even now, has still not been
according to the principles of Greek paideía, was analyzed with sufficient neutrality.
not based on acquiring theoretical knowledge or Here it is not our intention to study what
specific practical skills, but had to be oriented the arrival of the Sophists in Athens meant for
towards achieving areté, that which the Romans education. There are several essays (in general
subsequently translated as virtus and which, er- fairly critical of the work of these thinkers) which
roneously, through Christianity, reached the West can be referred to for a more detailed analysis,
as “virtue.” In Homeric times, areté was related ranging from the more generic ones by William
to the values peculiar to heroes, to noble warriors K.C. Guthrie (Guthrie, 1971), Mario Untersteiner
and was a mixture of moral and martial ideals. (Untersteiner, 1954), and Jacqueline de Romilly
Later, in the classical age, paideía transformed (Romilly, 1992) to those that deal with specific
the meaning of areté, which now acquired a more aspects such as their role in Greek Rhetoric (Ken-
humanist and political approach. Then, “excel- nedy, 1963). On the other hand, it is our intention
lence” (a more correct translation of the Greek to call attention to a conception of education in
term than “virtue”) consisted of the acquisition which both the Sophists and Socrates coincide,
of all the values that make a man a citizen, a be- and which has to do with the active social and
ing capable of moving with ease in the polis and political nature of education. We will also deal
actively participating in the life of the city. Thus, with some of the differences that may be interest-
for the Greeks, education (understood as an activ- ing for our purpose.
ity oriented towards practice and citizenship, and The same as occurred in Homeric times, the
not as a simple learning of contents) is the key to main purpose of education for the Sophists and
the evolution of a civilization, and linked from Socrates was none other than attaining excel-
its origins to the heroic epic until its splendor lence, areté. However, although still maintaining
in Athenian democracy, it appears as the motor a certain competitive view of excellence (i.e., an
behind Greek culture. Such was the importance approach according to which areté is shown in
of education (of this type of education) in Ancient superiority over other men owing to its origin in
Greece (Jaeger, 1945). the noble and warrior class, as we have seen in
the previous section), the meaning of the term

underwent a significant variation. In this age interaction of citizens in a political context in

areté was linked to social and political success which the command of language and rhetoric plays
and, therefore, the main objective of teaching was a major role. The teaching-learning relationship
none other than to form good citizens, aware, as is an eminently linguistic activity.
the Greeks of that age were, of the importance of As regards Socrates and his particular method
social and political interaction. Thus, learning was of teaching, there are some differential elements
not something erudite and private, but had to have that we would like to call attention to (leaving aside
a social and public usefulness; in a certain sense, the polemics with the Sophists for the moment).
moreover, it was an emancipating task because it Socratic mayeutics is a method based on dialogue,
guaranteed success and social advancement and on the art of questioning the disciple so that the
what is perhaps more important, the usefulness latter will be able to find his own answers. Hence,
of learning was immediately perceived in its ap- according to Plato’s old teacher, the teacher does
plication to the social context. not really teach the disciple anything but merely
What Socrates and the Sophists did disagree helps him to find for himself the answers which,
on was the possibility of being able to teach areté. really, were already inside him.
According to Socrates it was more a quality of What is really interesting in this methodology
the soul that one did or did not have and which, at is that the student is the one who answers the
most, the “teacher” could help to find inside the questions and solves the problems. The teacher’s
disciple through the Socratic dialogical method method consists of knowing how to ask and how
known as mayeutics. The Sophists, however, to encourage the disciple to look for the answers.
considered that it was possible to teach, in an Really, he or she is more a stimulus and a guide
orderly and structured way, everything required than an open book in which to find the solution to
to be an excellent citizen; such teaching, of course, problems. Even if this is true (and probably it is),
included, among other things, rhetoric, because the virtue of the teacher consists of making the
one of the keys to social success in a civilization student believe that she has found for himself the
such as the Greek one entailed admiration and answer to the questions posed. It is a methodology
respect for those with a beautiful and persuasive that gives prominence to the student without the
diction, those who today we would call “charis- teacher disappearing; the latter is always there,
matic.” This, of course, could lead us into a debate ready to orient and advise.
as to whether charisma can be taught or not, and Thus, the Socratic method can be defined
so we would return to the polemics between the as dialogical, process-oriented (we understand
Sophists and Socrates, but let us leave this ques- learning as a process), and proactive. These
tion for the moment. characteristics are undoubtedly major elements
Protagoras, according to Plato’s dialogue of the for an online training methodology on which to
same name, used the myth of Prometheus to show construct the professional profile of our e-learn-
us that all humans have political virtue by order ing teacher.
of Zeus himself, who even ordered that all men Furthermore, sophistry has revealed that edu-
should cultivate it and practice it under penalty cation has an eminently social nature, and that it
of being exiled from the city (Plato, Protagoras, is precisely in this context where learning gains
320d-322d). Without going into whether political meaning, beyond mere erudition without specific
virtue can be taught or not, the important thing is usefulness. These elements are equally important
that education is defined as an activity oriented when constructing an appropriate method for our
towards the social sphere and above all to the new training.

concLusIon make sense if put into practice and therefore are

learned along the way. This action, which is the
What sense is there in posing a reflection of the result of knowledge, is revealed in a social context,
concept of paideía in Ancient Greece in order a context in which new knowledge is produced
to develop a methodology suited to online train- as the result of the action and interaction of the
ing? As has been seen at the beginning of this subjects. Knowledge is, then, the fruit of a social
chapter, most e-learning initiatives are set in environment. Finally, dialogue and language
motion without having a clearly defined method are the basic elements in the quest for learning,
or a strategy suited to the peculiarities of this since this is no more than a continual process of
type of training. questions and answers, answers that lead to new
Moreover, there seems to be a more or less questions…
widespread trend to accept constructivism as an The purpose of these pages was not, then, to
explanatory framework or theoretical presupposi- develop a method for online training based on the
tion. However, constructivism is a cognitive theory activity of the tutor as a catalyst in the teaching-
rather than a method and perhaps this confusion learning relationship, as has been done in previous
lies at the bottom of many serious errors related studies (Seoane Pardo & García Peñalvo, 2006;
to training paradigms for e-learning. Thus, if con- Seoane Pardo, García Peñalvo, Bosom Nieto,
structivism indicates to us a desideratum, a goal, Fernández Recio, & Hernández Tovar, 2006).
but is not a method in itself, the need remains to On this occasion, on the contrary, we opted to
set up a path on which to trace the route of learn- illustrate the groundwork on which to build this
ing in an interaction framework as peculiar as the method, starting from a model with a long tradi-
one corresponding to online training. tion and which, by the way, is to be found in the
In short, all theoretical reflection on this type of very foundations of western civilization.
training revolves around what should be done, but
there is very little effective orientation to indicate
how to achieve what we are supposed to do. Future reseArch dIrectIons
After analyzing the different conceptions of
education throughout history, we feel that the The philosophical and epistemological reflections
Greek paideía model is perfectly suited both to contained in this chapter are part of a more am-
the presuppositions of the commonly accepted bitious research concerning a new methodology
theoretical framework and to a more realistic (and for online training, especially a methodology
in a certain sense, “classical”) position, according for training “online teachers” or “tutors” (also
to which a teaching profile is necessary in order to known as “facilitators” or “e-facilitators” in other
guarantee the success of a training initiative. contexts).
The model from Ancient Greece shares with These considerations, among with the main
us the idea that training is a task that falls to the hypotheses of that methodology, are being tested
subject being trained, but which is not achieved in several initiatives developed by the University
alone and without the presence of someone who, of Salamanca that are being addressed to different
although remaining in the shadows, will always kind of users with completely different learning
appear when needed and will be capable of show- contexts and with remarkable success in all the
ing us the road to knowledge. This knowledge, various scenarios where this methodology has
however, is not understood as a simple acquisi- been proved.
tion of contents but rather will be developed in Most of the theories and even case studies
capabilities, competencies, and skills which only related to methodology and didactics in e-learn-

ing contexts analyze the learning contexts and Bruner, J. (1997). La educación, puerta de la
how students learn in these initiatives, simply cultura. Madrid, Spain: Visor.
explaining the scenario or offering a sum of sug-
Bruner, J. (1998). Desarrollo cognitivo y edu-
gestions to improve the learning experience. But
cación. Madrid, Spain: Morata.
still persists the need of a real teaching model
for e-learning activities, because teaching roles Carruthers, J. (1993). The principles and practices
are still fundamental to let trainees achieve the of mentoring. In B.J. Caldwell & E.M.A. Carter
desired goals, skills, and competences required (Eds.), The return of the mentor: Strategies for
for any learning activity. workplace learning. London: Falmer Press.
Thus the main challenge for the near future
Cecez-Kecmanovic, D., & Webb, C. (2000). To-
is the definition of a complete methodology for
wards a communicative model of collaborative
training teachers specifically adapted to online
Web-mediatic learning. Australian Journal of
contexts, among with the clear definition of their
skills and competences. These studies, actually
being proved in real learning contexts, will be Coll, C., Martín, E., Mauri, T., Miras, M., Onrubia,
completed with several “user manuals” for online J., Solé, I., et al. (2005). El constructivismo en el
teachers, students, learning content designers, and aula, Vol. 111 (15th ed.). Barcelona: Graó.
instructional designers, all of them adjusted to a
Dewey, J. (1933). How we think. Boston, MA:
rigorous quality framework that must preside the
Heath.
whole process of every learning activity that aspire
to deserve the qualifying of excellence. Dewey, J. (1938). Experience and education. New
York: Macmillan.
Dougiamas, M., & Taylor, P.C. (2003). Moodle:
AcknowLedgment
Using learning communities to create an open
source course management system. Paper pre-
This work has been partly financed by the Min-
sented at the Proceedings of the EDMEDIA 2003
istry of Education and Science (Spain), KEOPS
Conference, Honolulu, Hawaii.
Project (TSI2005-00960).
Garrison, D.R., & Anderson, T. (2003). E-learning
in the 21st century: A framework for research and
reFerences practice. London, New York: RoutledgeFalmer.
Guthrie, W.K.C. (1971). The sophists. London:
Anderson, E.M., & Shannon, A.L. (1995). Towards
Cambridge University Press.
a conceptualization of mentoring. In T. Kerry &
A.S. Mayes (Eds.), Issues in mentoring. London: Jaeger, W. (1945). Paideia: The ideals of Greek
A.S. Routledge. culture (G. Highet, Trans.). New York: Oxford
University Press.
Barberà, E. (2006). Los fundamentos teóricos de
la tutoría presencial y en línea: Una perspectiva Kennedy, G.A. (1963). The art of persuasion
socio-constructivista. In J.A. Jerónimo Montes & in Greece. Princeton, NJ: Princeton University
E. Aguilar Rodríguez (Eds.), Educación en red Press.
y tutoría en línea (pp. 161-180). Mexico: UNAM
Little, J.W. (1990). The mentor phenomenon and
FES-Z.
the social organisation of teaching. In C. B. Court-

ney (Ed.), Review of Research in Education, 16, Wenger, E. (1998b). Communities of practice.
297-351. Washington, DC: American Educational Learning, meaning and identity. Cambridge:
Research Association. Cambridge University Press.
Moodle. (2007). Philosophy. Retrieved October
17, 2007, from http://docs.moodle.org/en/Phi-
losophy AddItIonAL reAdIngs
Romilly, J.d. (1992). The great sophists in Peri-
Anagnostopoulos, D., Basmadjian, K.G., & Mc-
clean Athens. Oxford, UK, New York: Clarendon
Crory, R.S. (2005). The decentered teacher and
Press, Oxford University Press.
the construction of social space in the virtual
Seoane, A.M., García, F.J., Bosom, Á., Fernández, classroom. Teachers College Record, 107(8),
E., & Hernández, M. J. (2007). Online tutoring 1699-1729.
methodology approach. International Journal of
Ardizzone, P., & Rivoltella, P.C. (2003). Didat-
Continuing Engineering Education and Life-Long
tiche per l’elearning. Metodi e strumenti per
Learning (IJCEELL), 17(6), 479-492.
l’innovazione dell’insegnamento universitario.
Seoane Pardo, A.M., & García Peñalvo, F.J. Roma: Carocci editore.
(2006). Determining quality for online Activities.
Bereiter, C., Scardamalia, M., Cassells, C., &
Methodology and training of online tutors as a
Hewitt, J. (1997). Postmodernism, knowledge
challenge for achieving the excellence. WSEAS
building, and elementary science [Special Issue:
Transactions on Advances in Engineering Educa-
Science]. Elementary School Journal, 97(4),
tion, 3(9), 823-830.
329-340.
Seoane Pardo, A.M., García Peñalvo, F.J., Bosom
Jonassen, D.H., Carr, C., & Yueh, H.-P. (1998).
Nieto, Á., Fernández Recio, E., & Hernández
Computers as mindtools for engaging learners in
Tovar, M.J. (2006). Tutoring online as quality
critical thinking. TechTrends, 43(2), 24-32.
guarantee on e-learning-based lifelong learning.
Definition, modalities, methodology, compe- Maldonado, T. (1994). Lo real y lo virtual. Bar-
tences and skills (CEUR Workshop Proceedings). celona: Gedisa.
Virtual Campus 2006. Selected and Extended
Marcelo, C., Puente, D., Ballesteros, M.A., &
Papers, 186, 41-55.
Palazón, A. (2002). E-learning-teleformación.
Untersteiner, M. (1954). The sophists. New York: Diseño, Desarrollo y Evaluación de la Formación
Philosophical Library. a través de Internet. Barcelona: Gestión 2000.
Vygotsky, L.S., & Cole, M. (1978). Mind in so- Ruipérez, G. (2003). Educación Virtual y eLearn-
ciety: The development of higher psychological ing (1ª ed.). Madrid: Fundación Auna.
processes. Cambridge, MA: Harvard University
Scardamalia, M. (2002). Collective cognitive re-
Press.
sponsibility for the advancement of knowledge. In
Wenger, E. (1998a). Communities of practice. B. Smith (Ed.), Liberal education in a knowledge
Learning as a social system. Retrieved October society (pp. 67-98). Chicago: Open Court.
17, 2007, from http://www.co-i-l.com/coil/knowl-
Scardamalia, M., & Bereiter, C. (2003). Knowl-
edge-garden/cop/lss.shtml
edge building environments: Extending the limits
of the possible in education and knowledge work.

In A. DiStefano, K.E. Rudestam & R. Silverman of Education (2ª ed.) (pp. 1370-1373). New York:
(Eds.), Encyclopedia of distributed learning. Macmillan Reference.
Thousand Oaks, CA: Sage Publications.
Vygotsky, L.S., & Kozulin, A. (1986). Thought
Scardamalia, M., & Bereiter, C. (2003). Knowl- and language (Rev. ed.). Cambridge, MA: MIT
edge building. In J. W. Guthrie (Ed.), Encyclopedia Press.
0

Chapter IV
E-Mentoring:
An Extended Practice,
An Emerging Discipline
Angélica Rísquez
University of Limerick, Ireland
ABstrAct
This chapter integrates existing literature and developments on electronic mentoring to build a construc-
tive view of this modality of mentoring as a qualitatively different concept from its traditional face-to-face
version. The concept of e-mentoring is introduced by looking first into the evasive notion of mentoring.
Next, some salient e-mentoring experiences are identified. The chapter goes on to note the differences
between electronic and face-to-face mentoring, and how the relationship between mentor and mentee
is modified by technology in unique and definitive ways. Readers are also presented with a collection
of best practices on design, implementation, and evaluation of e-mentoring programs. Finally, some
practice and research trends are proposed. In conclusion, the author draws an elemental distinction
between both modalities of mentoring, which defines e-mentoring as more than the defective alternative
to face-to-face contact.
IntroductIon of learning possibilities (Gisbert, 2004). During

the last two decades, ICTs have offered new and
The technology revolution has changed the way we exciting opportunities for transcending the physi-
live in our world, including what we understand cal and psychological distance between people.
about mentoring and how it happens. Information Accounts of the potential of ICT for mentoring
and communication technologies (ICTs) have been relationships started appearing in the literature
made central given their potential for democratiza- in the late 1980s and early 1990s (Moore, 1991),
tion of the access to knowledge, their incorporation and have extended to become a phenomenon
to professional competences, and the improvement emerging on a world wide scale. The first online
E-Mentoring
version of the original contribution to UNESCO’s Ingersoll, 2004). Miller (2004) refers to “transi-
World Communication and Information Report tion mentoring” to describe those programs that
(Blurton, 1999) notes the potential of ICT to en- target individuals during times of transition at
able mentoring programs to provide guidance any moment in life, for example, educational and
to individuals by well-established members of a career transitions. In transition mentoring, a paired
particular community. Blurton (1999) notes that relationship is established between a more senior
“such virtual collaborations between individuals individual or mentor and a lesser experienced
are an effective way for senior member of a com- individual or mentee, in order to develop compe-
munity to teach, inspire and support newcomers” tences and orientations towards survival that the
(p.12). newcomer might otherwise only have acquired
A simple Web search using the terms “elec- slowly and with at least some difficulty.
tronic mentoring,” “e-mentoring,” “online mentor- Literature also suggests that effective mentor-
ing,” or “telementoring” identifies a large number ing relationships should be trust based and power
of programs initiated by educational institutions, free (P. B. Single & R. M. Single, 2005a). This is
corporations, and communities around the globe, often refereed to as “the value of impartiality,” the
in which support to individuals is facilitated by benefit associated with being mentored by some-
the use of ICT. This chapter presents to the reader one who has no a vested interest in your choices
the developments of the last decade in computer or ulterior motives for mentoring. Basically it
mediated mentoring, starting first with a consid- is useful to find a mentor who doesn’t have an
eration of the general concept of mentoring. interest in your performance, and with whom the
newcomer can share common experiences. Peer
to peer relationships offer useful orientations to
BAckground a mentoring system, involving a degree of social
responsibility to the community in ways that at-
a multifaceted and Elusive Concept tempt to confront and reverse an ever-increasing
individualistic, competitive approach to career,
The term “mentoring” was coined based on education, and life development (Allen et al., 1999;
Homer’s Odyssey, where the young Telemachus McLean, 2004; O’Regan, 2006). In addition to
was assigned Mentor as his companion and advi- these benefits, peer mentors may be in a better po-
sor during the long absence of his father. Since sition to share information, offer credible advice,
the late 1970s, the term was adopted to promote listen to the mentees’ concerns, and serve as a role
the value of institution or organization-based re- model than traditional mentors. Allen et al. (1999)
lationships to an individual’s personal and profes- demonstrated the effectiveness of psychosocial
sional development. Much emphasis is placed on and career-related peer mentoring showing that
empathy and trust (Eby, 1997); and most authors there are different dimensions of socialization of
agree on the idea that the benefits of mentoring newcomers that peers can facilitate (politics, per-
tend to emerge only over a relatively long period formance, and establishment of relationships with
of time (Rhodes, 2002). Mentoring is a growing organizational members). The authors underscore
practice that has been extensively documented in the valuable role that more experienced peers can
Anglo-Saxon literature as a means to facilitate serve in enhancing socialization (in abstract).
transitional adjustment and personal or profes- Arguably, peer mentors may have training and
sional development (Allen, McManus, & Russell, support needs that program organizers must take
1999; Eby, 1997; Gray & Gray, 1990; Kram, 1985; careful consideration of.
McMahon, Limerick, & Gillies, 2002; Smith &

E-Mentoring
However, what is understood for “mentoring” There can be components of mentoring in

and it manifestations is very diverse. The idea of tutoring, and of tutoring in mentoring, but the
a strongly interpersonal relationship which pro- primary goals of the two programs and different.
vides a “safe place” for the newcomer to address Most definitions distinguish mentoring from a
his or her development needs associates mentor- situation where the mentor provides solutions to
ing to the area of counseling, although there are the mentee, and emphasize instead the reciprocal,
important distinctions between the two (Stokes, complex, and multilayered nature of the relation-
Garrett-Harris, & Hunt, 2003). Mentoring is also ship. To condense this elusive concept, it is useful
often confused and mixed with other concepts, to remind the reader on what mentoring is and is
like tutoring, coaching, and advising. It is difficult not, as summarized in Table 1.
to draw a distinction between these and the term
used very much depends on local and national a European Perspective
contexts and traditions. O’Neill and Harris (2005)
draw a pretty clear distinction between “tutoring” The popularity of mentoring, long accepted in
and “mentoring” as follows: Anglo-Saxon academic and organizational en-
vironments, is strongly rising in the European
Tutoring is often confused with mentoring because context as a means for guidance, support, and
in involves an ongoing relationship between a socialization. A recent resolution by the Euro-
student [and by extension a new employee] and a pean Council, aimed to establish the policies
more knowledgeable person, but there are impor- and practices in the field of guidance through
tant differences. (…) In tutoring, the objective is life, includes mentoring in the main definition
that the student [employee] masters a well-defined of “guidance” (EC, 2004b, p. 2). The document
domain. The expert assigns the student [employee] stresses that the role of guidance and mentoring
a problem (…), then evaluates the student’s is to provide significant support to individuals
[employee’s] performance (...) Throughout, the during their transition between levels and sectors
tutors is typically in control of which problems of education, training systems, and working life
the student [employee’s] addresses. Mentoring is (2004b, p. 3). The document also strongly recom-
quite different in that interactions usually evolve mends that the beneficiaries of guidance should
around problems that the junior party brings to be at the centre of the process both in terms of
the table. (p.113) design and delivery. O’Regan (2006) highlights
mentoring is receiving a higher profile than ever
Table 1. What is mentoring?
Mentoring is… Mentoring is not…

An enhancement of other forms of social, An isolated solution to problems
emotional, psychological, and intellectual
support
A dynamic process that engages both mentee Something that is done TO an individual
and mentor in the process of self-learning,
action, and reflection
Transformational, organic, complex, multi- Passive or mechanistic
dimensional, and somewhat unpredictable.
Requires mutual engagement

E-Mentoring
before. The author quotes Gränzer’s presentation increase the likelihood of engagement in the
at ENCYMO (the Mentoring in Europe Confer- e-mentoring process, and relies on program
ence which took place in Liverpool in 2005) on evaluation to identify improvements for future
the discussions currently taking in the European programs and to determine the impact on the
Commission relating to the growth and expansion participants. (p.10)
of mentoring as a key element to the support of
individuals across multiple contexts. Ensher, Heun, and Blanchard (2003) categorize
The UK has a significant lead on other Euro- e-mentoring according to the amount of electronic
pean countries, with several millions of pounds communication that takes place within the rela-
invested from governmental funding to the Na- tionship. At one end of the continuum there are
tional Mentoring Network through the Aimhigher full e-mentoring relationships (computer based
program. communication only). At the other extreme are
face to face mentoring with ICT support, and
somewhere in the middle blended mentoring takes
iSSUES, ConTRoVERSiES, anD place as a combination of face-to-face and online
ProBLems mentoring. Hamilton and Scandura (2003) specify
further and state that, to be called e-mentoring,
e-mentoring 75% or more of the mentorship must take place
through electronic means.
Time and space constraints often create an A review of the literature focusing on support
obstacle that prohibits mentors and mentees approaches in electronic collaborative learning
meeting as frequently as they should (if at all), environments results in a variety of concepts
an outcome that has undermined traditional (e.g., e-tutoring, online mentoring, e-coaching,
face-to-face mentoring relationships more than e-moderating) being used to address the roles,
any other factor, according to Noe (1998). As a tasks, and responsibilities of online facilitators
result, organizations and institutions across the (De Smet, Van Keer, & Valcke, forthcoming).
globe have embraced the access opportunity that Much of the above discussion on the differences
computer mediated communication promises for between mentoring and tutoring would apply to
mentoring. E-mentoring is defined by Single and their electronic versions, and a case can be made
Muller (1999) as a naturally occurring or paired that what is termed “e-mentoring” is often difficult
relationship primarily using electronic communi- to distinguish from e-moderating, e-couching, or
cation that is established between a more senior e-counseling. Moreover, the technical and inter-
individual (“mentor”) and a lesser experienced personal competences required from e-mentors
individual (“protégé” or “mentee”), intended to overlap with those of e-moderators and e-tutors,
develop and grow the skills, knowledge, confi- and much of the literature dealing with best train-
dence, and cultural understanding of mentee to ing practices in e-mentoring emanate from best
help the mentee to succeed. P. B. Single and R. practices and research findings in other areas
M. Single (2005b, p. 10) elaborate further on the related to computer mediated communication
definition to structured e-mentoring programs, (CMC) (Kasprisin, Single, Single, & Muller, 2003;
informed by the work of the face-to-face struc- O’Neill & Harris, 2000).
tured mentoring field: E-mentoring systems have been introduced in
many contexts with a wide variety of purposes:
...occurs within a formalized program environ- facilitating expatriate or newcomers’ adjustment
ment, which provides training and coaching to (Beitler & Frady, 2001; Dewart, Drees, Hixen-

E-Mentoring
baugh, & Williams, 2004), career development tive Empathy-Edge in the UK (Headlam-Wells
(Tesone & Gibson, 2001; Wadia-Fascetti & et al., 2005).
Leventman, 2000), support to entrepreneurs and
SMEs (Perren, 2003; Stokes, 2001), curriculum- a Qualitatively Different Experience
based learning (O’Neill & Harris, 2000), and
higher participation in academia by minority E-mentoring programs do have some fundamental
groups (Headlam-Wells, Gosland, & Craig, 2005; similarities with their face-to-face counterparts.
McMahon et al., 2002; Single & Muller, 2001). In The starting point is essentially the same: a one
Tables 2-11, MentorNet, an outstanding example to one liaison between two individuals based on a
of e-mentoring, is presented. mutual commitment towards developing the skills
of the less experienced of them towards some
mentornet: A great success of broad organizational or institutional objective
e-mentoring (Conway, 1998). In order to function effectively,
both electronic and face-to-face systems must be
By 2003, MentorNet had served more than 2,800 planned, implemented, and monitored properly
mentees. Nowadays, the organization has around with a coordination system that supports, but is
20,000 members and has coordinated more than somewhat independent of the participants. Both
9,000 e-mentoring relationships. Importantly, the are also affected by wider organizational and
evaluation of results of nearly 10 years of experi- personal factors including culture and norms,
ence are greatly helping to canvass the potential management support, and degree of top-level
and challenges of e-mentoring (Single & Muller, commitment to the success and longevity of the
2000; Single, Muller, Cunningham, & Single, program.
2000; Single & Single, 2005a, 2005b). Many other However, e-mentoring and face-to-face men-
e-mentoring projects have been inspired on the toring are also different in many ways. A literature
work by MentorNet with the common objective review of the opportunities and challenges of
of enhancing female presence on target areas and computer mediated mentoring as opposed to it
professions, for example, the EU funded initia- traditional face-to-face version has highlighted
Figure 1. Homepage of MentorNet (www.mentornet.net)

E-Mentoring
Figure 2. MentorNet e-Forum discussion groups
Figure 3. Mentor profile (to be filled before participating in one-to-one mentoring)

E-Mentoring
Table 2. Contact. Differences between face-to-face and e-mentoring (a literature review)
FACE-TO-FACE ELECTRONIC
• Rigid, dependent on space and time • Flexible, independent of space and time
Table 3. Timing. Differences between face-to-face and e-mentoring (a literature review)
• Immediate. Pressure on immediately responding • Asynchronous tools (discussion forums, e-mail):

• Delayed, without the pressure of immediately
responding. It may be not a timely process if
responses are not quick
• Asynchronous tools (chat): Pressure on immediate
response
Table 4. Implications of the communication channel. Differences between face-to-face and e-mentoring
• Rich on nonverbal cues, wealth of emotional • Nonverbal cues are missing; alternative expres-
information sion of emotions is required
• Participants can learn from the other person’s im- • Not needing to take account of another person’s
mediate reactions immediate reactions (“self-absorption”) may
• For some individuals, face-to-face interaction is facilitate self-awareness and provision of honest
seen as warmer and richer. Others find it difficult feedback
and exposed • For some individuals, the communication style
• Misunderstandings can be clarified as they appear can be safer and less intimidating. Others perceive
if participants have the required social skills it as a cold medium
• First impressions may play a greater role • Miscommunication may happen. In extreme cases,
CMC can turn hostile as inhibitions are lowered
• Less information is exchanged so relationships
develop slowly, but it allows for greater privacy
• Hyper-relationships may happen (participants
form a better opinion of the other than they would
if they were physically interacting)
Table 5. Skills required. Differences between face-to-face and e-mentoring (a literature review)
• Conventional social skills are required • Social skills, computer literacy, good written com-
munication, and netiquette are required
• More frequent and explicit purpose-setting, prog-
ress-reporting, and problem-solving communica-
tions may be necessary

E-Mentoring
Table 6. Role of social differences. Differences between face-to-face and e-mentoring (a literature re-
view)
• Status differences play a greater role • Status differences are attenuated
Table 7. Pairing and scalability. Differences between face-to-face and e-mentoring (a literature re-
view)
• Space/time restrictions may impose limitations • Space/time flexibility is likely to provide greater
on the pairing of mentors, taking precedence over choice in the pairing of mentors and protégés and
expertise extend opportunities to participate
• Physical proximity and personal schedules may • The ease with which virtual relationships can be
pose high barriers to entry started and ended can weaken commitment. Also,
the nature of the communication can promote
minimal contact between participants, shorter
programs, inadequate planning, mentor training,
and follow-up
• Mentors often find it difficult to find out about
their mentees’ needs and frustrations, and are reli-
ant on their mentees to express them
Table 8. Records of the interaction. Differences between face-to-face and e-mentoring (a literature
review)
• There is usually no record. Information is col- • The interaction can be recorded automatically and
lected sometimes using questionnaires or rubrics just in time. Mentors and protégés tend to find
and is retrospective these records helpful
Table 9. Monitoring and evaluation. Differences between face-to-face and e-mentoring (a literature
review)
• Use of secondary sources (participants’ reports • Primary source of information (electronic records)
and coordinator’s notes) allow for content analysis, analysis of participa-
tion patterns, lurking, and so forth

E-Mentoring
Table 10. Ethical implications. Differences between face-to-face and e-mentoring (a literature review)
• The relationship is usually confidential although • Electronic records may involve additional
ethical issues must be dealt with (like participants’ confidentiality and ethical issues, which may also
selection) impact the communication
Table 11. Cost and access. Differences between face-to-face and e-mentoring (a literature review)
• Depending on geographic and time circumstances, • Depending on participants’ having easy access
it can be a cost effective solution or a cost inten- to computers and Internet, e-mentoring is a cost
sive one effective option or it may appear a digital divide
• There may be other associated costs, like activities (lesser e-mentoring opportunities for those who
during meetings can not access the technology)
differences (Bierema & Merriam, 2002; Ensher value of connections between organizations.
et al., 2003; Harrington, 1999; McCormick & For them, e-mentoring facilitates the “strength
Leonard, 1996; O’Neill & Harris, 2000, 2005; of weak ties,” since electronic communications
O’Neill, Harris, Cravens, & Neils, 2002; Single span render geographical distances irrelevant and
& Muller, 1999) summarized in Tables 12-17, provide mentoring opportunities to wider and
according to each issue considered. more diverse groups of people.
Of course, both varieties of mentoring are Given the potential drawbacks that e-mentor-
not necessarily mutually exclusive, and they can ing may involve as noted in the tables above, some
complement each other if the circumstances make authors (O’Neill et al., 2002) argue that deeply
a blended approach possible. However, the issues personal, long term relationships are likely not
raised above indicate that both types of mentor- to work so well online. However, there are also
ing represent quite different ways of striving for equally important forms of mentoring that can
a common goal. Bierema and Merriam (2002) provide people guidance and advice as they enter
share the view that e-mentoring is “qualitatively into and move through unfamiliar organizations,
different than traditional face-to-face mentoring” communities and stages in life. E-mentoring is
and that “the virtual medium provides a context likely to find its niche among these modalities
and exchange that may not be possible to replicate of mentoring, focused on more shorter-term and
in face-to-face mentoring relationships” (2002, professional or academic objectives.
p. 219). Therefore, in Harris’ words (O’Neill et
al., 2002), the important question is not whether
e-mentoring is better or worse than face-to-face Best PrActIce
mentoring, but rather what e-mentoring can
bring “for long in-depth, productive, mutually Coming from this view of e-mentoring as a
beneficial interactions when the same can’t hap- discipline and practice “in its own right,” a 306
pen face-to-face.” P. B. Single and R. M. Single degree review of effective practice along the
(2005b, p. 14) elaborate in this direction and note life span of a mentoring program is presented
that the primary benefit of e-mentoring is in the next. This review combines suggestions for best

E-Mentoring
Table 12. Statement of purpose and long-range plan. Best practice on e-mentoring DESIGN (a literature
review)
RECOMMENDATIONS AUTHOR(S)
• State what ultimate purpose the program is design for: career development, aca-
demic support, socialization, and so forth.
MENTOR, 2001
• Who, what, where, when, why,and how activities will be performed
• Realistic, attainable, and measurable goals, objectives, and timelines
• Decide on a typology of mentoring (senior to junior or peer to peer, individual or
Miller, 2002
group based, etc.)
• Carry out a pilot in small scale and introduce change gradually Ross, 2004
Table 13. Relevant populations and stakeholders. Best practice on e-mentoring DESIGN (a literature
review)
• Assessment of potential mentee’s needs and pool of mentors
MENTOR, 2001
• Adherence to general principles of volunteerism
Table 14. Contextualization. Best practice on e-mentoring DESIGN (a literature review)
• Research local and national e-mentoring schemes Ross, 2004
• Assessment of organizations’ readiness, capacity, and will to create and sustain a
high-quality e-mentoring programs, collecting input from originators, staff, potential MENTOR, 2001
volunteers, and potential mentees
• Build upon the knowledge obtained in face-to-face mentoring experiences in the
institution/organization
O’Neill, et al. (2002)
• Sustain involvement of staff with funded time, meaning it is a designated time
within their day (instead of something extra in addition to their regular duties)
• Build relationships carefully with all stakeholders
• Adjust to the institution/organization’s periods of intensive work, holidays, and so Ross, 2004
forth
0
E-Mentoring
Table 15. Technology strategy. Best practice on e-mentoring DESIGN (a literature review)
• Carry out a through IT audit of all involved Ross, 2004
• Choose a communication system:
a. appropriate to goals of the program
b. relevant to participants’ context
c. safe and reliable
d. affordable
MENTOR, 2001
• Policies regarding privacy and security of program participants’ data and communi-
cation
•
• Method for archiving e-mails to meet safety and evaluation needs
Table 16. Promotion and marketing policy. Best practice on e-mentoring DESIGN (a literature re-
view)
• Year-round marketing and public relations MENTOR, 2001
• Prepare and distribute an information pack for teachers Ross, 2004
Table 17. Safety measures. Best practice on e-mentoring DESIGN (a literature review)
• Establishment of a code of online conduct guided by common sense, basic neti-
quette, and mutual respect
• Adherence to rules and laws that apply in face-to-face mentoring, as well as those
unique to online mentoring, for example, confidentiality of program participants’
personal information MENTOR, 2001
• Comprehensive background checks and screening of mentors
• Process for raising and addressing concerns
• Exit clause

E-Mentoring
Table 18. Recruitment plan. Best practice in e-mentoring program IMPLEMENTATION (a literature
review)
• Strategies that reflect accurate expectations and benefits
• Targeted outreach based on mentees’ needs and interests
MENTOR, 2001
• Volunteer opportunities beyond mentoring
• Basis in your program’s statement of purpose and long-range plan
• Recruit early, before participants are caught up in their daily activities
• Design different “call for participants” and application forms for mentors and men-
tees
• As well as electronic communication, use alternative recruitment mediums like Single & Muller, 2005
newsletters, heads of department, student/staff representatives, and so forth.
• Manage expectations carefully before training: program goals, eligibility criteria,
frequency of expected contact, and so forth.
Table 19. Eligibility screening. Best practice in e-mentoring program IMPLEMENTATION (a literature
review)
• Reference checks for mentors, especially if working with underage mentees
• Suitability criteria that satisfy the program statement of purpose and needs of the
target population (could include personality profile, skills identification, gender, age, MENTOR, 2001
geography, language, race, career interests, level of education, previous volunteer
experience, and so forth)
Table 20. Induction and orientation. Best practice in e-mentoring program IMPLEMENTATION (a
literature review)
• Successful completion of training and orientation
• Separate orientation for mentors and mentees
Include:
a. Reinforce expectations: jobs/roles descriptions, restrictions (accountability)
b. Description of technology used and access needed
c. Level of commitment expected (time, energy, flexibility, frequency)
d. Benefits and rewards of participation
MENTOR, 2001
e. Summary of program policies, including those governing privacy, reporting,
communications, and evaluation
f. Safety and security, especially around use of the Internet
g. Cultural/heritage sensitivity and appreciation training
h. Do’s and Don’ts of managing the relationships
i. Crisis management/problem-solving resources
j. Support materials and ongoing sessions as necessary
k. Suggestions on how to get started
• Decide on a method for delivery: face-to-face, online, or blended. If choosing
online, options are:
- Moderated discussion groups Single & Muller
- Web-based threaded discussion lists (2005)
- Web-based training tutorial based on case studies, sample responses, simulation,
and so forth.

E-Mentoring
Table 21. Coaching. Best practice in e-mentoring program IMPLEMENTATION (a literature review)
• Guide the e-mentoring pairs along the relationship, starting with initiation and mov-
ing through cultivation, separation, and redefinition (Kram, 1985)
• Coach in a networked environment, using messages containing discussion sugges- Single & Muller
tions, mentoring tips, and so forth. (2005)
• Keep coaching messages short and frequent (weekly or every other week)
• Conclude coaching messages by soliciting feedback from the participants
• Consider techniques that address the development of the participant’s expectations
and role acquisition:
- Iterative cycles: give participants the chance to experience different mentors and
mentees O’Neill & Harris
- Direct facilitation: interaction by a third party, who follows and participates in the (2005)
mentoring dialogue, assisting, and suggesting
- Open access to models: shared electronic workspace that allows mentors and
mentees to observe and learn others’ e-mentoring relationships
• Deal with lurkers: check all participants know how to post/reply to messages,
provide test areas and arrivals areas, have a free-flowing social conferencing area,
Salmon (2000)
give participants time to get used to the online environment, provide areas for safe
reflections and comments
Table 22. Matching and re-matching. Best practice in e-mentoring program IMPLEMENTATION (a
literature review)
• Grounding in the program’s eligibility criteria MENTOR, 2001
• Choose a matching method:
- Participant choice: works best when those available for listing are plentiful and
when one group will be recruited before the other group; however, it may give place
to inappropriate matching and to participants not having a match
- Unidirectional matching: mentees identify preferences for a mentor, the coordina- Single & Muller
tor matches mentees’ preferences with mentors’ characteristics (2005)
- Bidirectional matching: both mentees and mentors identify preferences for e-men-
toring partners, the coordinator takes into account all preferences
• Let mentors and mentees know the process by which they will be matched
• Allow the participants to review, accept, or reject their e-mentoring partnerships
Table 23. Monitoring. Best practice in e-mentoring program IMPLEMENTATION (a literature re-
view)
• Consistent and regular communications with staff, mentors, and mentees
• Tracking system for ongoing assessment
• Written records MENTOR, 2001
• Guidelines for support and conflict resolution
• Rationale for the selection of this particular monitoring strategy
• Monitor e-mails systematically Ross, 2004

E-Mentoring
Table 24. Support, recognition and retention. Best practice in e-mentoring program IMPLEMENTA-
TION (a literature review)
• Formal kick-off
• Process for managing grievances, rematching, interpersonal problem solving, han-
dling crises, and bringing closure to the relationships that end prematurely
• Ongoing peer support for volunteers MENTOR, 2001
• Social gatherings of different groups as appropriate
• Ongoing recognition and appreciation
• Newsletters of other communications to mentees, mentors, and support staff
• Program Web site with a participant guideline posted on it
• Keep a closed mentor list, so mentors can get feedback and advice from each other. Single & Muller
A moderator prompts early introductions and periodically seeds the list with discus- (2005)
sion topics
Table 25. Closure steps. Best practice in e-mentoring program IMPLEMENTATION (a literature re-
view)
• Private and confidential exit interviews to debrief the mentoring relationship be-
tween mentees and staff, mentors, and staff and mentors and mentees
• Clearly stated policy for future contacts between mentors and mentees
MENTOR, 2001
• Assistance for mentees in defining next steps for achieving personal goals
• Organize a formal end to the programs, which might include a celebration and
certificates
Table 26. Types of data collected. Best practice in e-mentoring program EVALUATION (a literature
review)
• Obtain benchmarking data after pilot program
• During and after the program, collect three types of information:
- Involvement data: frequency of interactions, continuation of mentoring relation-
ships for the duration of the program
- Formative data: participants’ satisfaction with the program, examinations of the Single & Muller
matching protocol and content of the mentoring interactions, which will guide the (forthcoming)
future enhancement of the program
- Summative data: assessment of program goals achieved, which serve as a standard
for comparison with a control group (students that do not undergo mentoring), ad-
dress sustainability and expansion with stakeholder and founders as main audience

E-Mentoring
Table 27. Moment. Best practice in e-mentoring program EVALUATION (a literature review)
• Ongoing evaluation rather than at the end of the program
MENTOR, 2001
Table 28. Dissemination. Best practice in e-mentoring program EVALUATION (a literature review)
RECOMMENDATIONS
AUTHOR(S)
• Consideration of the information needs of the program’s board, founders, communi-

cation partners, and other supporters
MENTOR, 2001
• Sharing of program information and lessons learned with program stakeholders and
the broader mentoring community
Table 29.“Don’ts” on e-mentoring (a literature review)
On planning and running an e-mentoring program, don’t…

… rush or under-estimate the time required to set up and plan the scheme—include the coordinator, mentor, and
mentee training
… commit to a long-term scheme initially
… assume mentees and mentors have good e-mail skills or easy access to equipment
… assume the software will deal with all risks or that everything is running smoothly
… let information technology to “take over the show.” At best, IT must enable participants to meet their tradi-
tional goals in a better way that was practically possible before
… engage in e-mentoring if you do not have experience in face-to-face mentoring
… do it for marketing or public relations purposes, but only when a genuine need is perceived and a realistic plan
can be implemented long-term
… use it as a replacement for a face-to-face mentoring program particularly with populations at high risk (failure,
violence, and so on)

E-Mentoring
practice as published by diverse authors and has communities of practice that promote sharing of
been divided in three main program phases: (a) knowledge and resources. Mutual collaboration
design and planning, (b) implementation, and (c) is likely to reinforce sharing of expertise and
evaluation. Recommendations and sources are resources that combine mentoring with other
summarized in Tables 12-28 respectively. student-centered methodologies, as well as pro-
Programme managers should also keep in grammatic efforts based on best practice and
mind some important “don’ts” on planning and demonstrated outcomes. As Haaris notes (O’Neill
running a e-mentoring program, as recommended et al., 2002):
by Ross (2004) and O’Neill, et al. (2002) and
summarized in Table 29. The kind of skills, sensibilities, and problem solv-
ing abilities that will be necessary to succeed in
an increasingly complex and technologically satu-
Future PrActIce dIrectIons rated society will not be developed in learners who
look to the technology to teach them. E-mentoring
The main threat to survival of many e-mentoring is an excellent and natural vehicle for starting to
initiatives is that of long-term sustainability. In create authentic, learner centered instruction of
the case of large projects in the U.S. O’Neill et this rich and complex variety. (p. 11)
al. (2002, p. 7) see in the next years a swift move
from national scale, generalist programs to more
localized and customized versions when they state, Future reseArch dIrectIons
“The most important issue for e-mentoring as it
moves into the future is tailoring e-mentoring It has been noted that the proliferation of online
initiatives to fit local needs. (...) even so if this mentoring programs has been underpinned by
means working in a less organized way and with very practical reasons of access and convenience.
fewer resources.” However, the benefits of these initiatives has been
The authors go on to point to the importance often assumed rather than demonstrated, and their
of creating software and guidance materials that positive outcomes have largely been based on
will assist in the development of small e-mentor- speculation and anecdotal evidence. Compared
ing initiatives in those circumstances where local to the plethora of Web sites connecting mentors
knowledge is very critical to success. This may and mentees, very little research has been done
be the way forward for initiatives like Aimhigher on program effectiveness. With some exceptions
in the UK, which has just been granted an extra (Asgari & O’Neill, 2004; Calder, 2004; Carlsen
year of “grace” after the initiative had officially & Single, 2000; Dewart et al., 2004; Headlam-
run out of governmental support. Wells, 2004; Headlam-Wells et al., 2005; O’Neill
As e-mentoring becomes a more widely known & Harris, 2000), it tends to be the case that fol-
and accepted modality of support, its permeation low up research on the benefits of mentoring are
in Europe is greatly likely to increase, partially much less frequent than the introduction of such
as a result from the emphasis placed on longlife program. Comprehensive literature reviews and
learning. It is important however that the expertise theoretical papers on the subject also scarce,
developed at grassroots level is harnessed and again with exclusion of the work of a few notable
made the most of to contribute to the success of authors (Bierema & Merriam, 2002; Ensher et
new, larger initiatives in the European context. al., 2003; Harrington, 1999; Harris, O’Bryan, &
Best case scenario, the next years will witness Rotenberg, 1996; O’Neill, 2004; Perren, 2003;
the consolidation of national and cross-national Single & Muller, 1999; P. B. Single & R. M.
Single, 2005b).

E-Mentoring
Moreover, existing research agendas have been practice across institutions and countries is in-
often outlined from a comparative perspective dispensable.
between e-mentoring and traditional face-to-face
programs (for example in Ensher et al., 2003),
rather than by treating e-mentoring in its own concLusIon
right. Future research questions should, rather,
gravitate around the opportunities and limita- In summary, the experiences and research pre-
tions that e-mentoring brings, how to monitor sented paint a picture of e-mentoring which is
mentoring relationships most effectively, what diverse and packed with venues for creativity.
are the ethical and policy issues involved in It was said at the beginning of this chapter that
keeping electronic records of the interactions, the practice of e-mentoring developed upon the
how to evaluate most effectively e-mentoring foundations of the large amount of research in its
programs, and so on. Much more can also be done face-to-face modality. However, the standpoint of
to benchmark e-mentoring practices across dif- this chapter is that by measuring the effective-
ferent contexts. By comparing the potential and ness and efficacy of an e-mentoring program
dangers of e-mentoring in the academic world using traditional face-to-face arrangements as a
and working life. Single and Muller (2005, pp. benchmark, the initial rationale for setting up e-
13-19) suggest some possible research questions mentoring programs is defeated. In other words, if
in this direction: organizers come from the belief that e-mentoring
is a quick and economical choice that substitutes
• What motivates mentors to volunteer for appropriate support structure with a snazzy Web
such programs? site, taking away the pain of the administration
• Which matching variables are more strongly and monitoring; then a case for keeping traditional
related to successful outcomes? face-to-face at all costs should be made. However,
• How do matching methods and closeness if emphasis is placed on the relation between men-
of match influence mentoring outcomes, tor and mentee, on the importance of screening,
such as involvement in the program and the training, and supporting mentors, and on sound
benefits gained by both the mentors and the program evaluation, then the question is what can
students? e-mentoring do for newcomers that we would not
• Which are the most effective and efficient have reached in a traditional program.
methods for training delivery? And do these
depend on the type and the size of the mentor
and mentee populations? reFerences
• How frequently should coaching messages
be sent? What content is most useful for Allen, T. D., McManus, S. E., & Russell, J. E. A.
those engaged in online mentoring? (1999). Newcomer socialization and stress: Formal
• What is an acceptable benchmark level for peer relationships as a source of support. Journal
involvement with an e-mentoring program of Vocational Behaviour, 54(3), 453-470.
Asgari, M., & O’Neill, K. (2004). What do they
In line with Harrington’s (1999) suggestions,
mean by success? Contributors to perceived suc-
future exploration of e-mentoring programs
cess in a telementoring program for adolescents.
should also move away from positivist approaches
Paper presented at the Annual Meeting of the
towards inquiries into social activity. What is
American Educational Research Association,
clear is that at this stage, sharing research and
San Diego, CA.

E-Mentoring
Beitler, M. A., & Frady, D. A. (2002). E-learning conceptual extension of the mentoring literature.
and e-support for expatriate managers. In H. B. Journal of Vocational Behaviour, 51, 125-144.
Long & Associates (Eds.), Twenty-first century
Ensher, E. A., Heun, C., & Blanchard, A. (2003).
advances in self-directed learning (CD). Boynton
Online mentoring and computer-meadiated com-
Beach, FL: Motorola University.
munication: New directions in research. Journal
Bierema, L. L., & Merriam, S. B. (2002). E-men- of vocational behaviour, 63, 264 - 288.
toring: Using computer mediated communication
Gisbert, M. (2004). Las TIC como motor de
to enhance the mentoring process. Innovative
innovación de la Universidad. En SANGRÀ,
Higher Education, 26(3), 211-227.
A. Y GONZÁLEZ, M. (coord.): Barcelona. Ed.
Blurton, C. (2000). New directions of ICT-use in UOC. In A. Sangrà & M. Gonz’alez (Eds.), La
education. UNESCO World Communication and transformación de las universidades a través
Information Report. Paris: UNESCO. de las TIC: Discursos y prácticas (pp. 193-197).
Barcelona: Ed. UOC.
Calder, A. (2004). Online learning support: An
action research project. James Cook University. Gray, M. M., & Gray, W. A. (1990). Planned men-
Paper presented at 4th Pacific Rim First Year toring: Aiding key transitions in career develop-
Experience Conference at Queensland University ment. Career Planning and Adult Development
of Technology. Brisbane, Australia. Journal, 6(3), 27-32.
Carlsen, W., & Single, P. B. (2000). Factors related Hamilton, B. A., & T.A., S. (2003). E-mentor-
to success in electronic mentoring of female col- ing: Implications for organizational learning and
lege engineering students by mentors working in development in a wired world. Organizational
industry. Paper presented at the Annual Meeting Dynamics, 31(4), 388-402.
of the National Association for Research in Sci-
Harrington, A. (1999). E-mentoring: The ad-
ence Teaching, New Orleans, LA.
vantages and disadvantages of using e-mail to
Conway, C. (1998). Strategies for mentoring: A support distant mentoring. The Coaching and
blueprint for successful organizational develop- Mentoring Network Articles. Retrieved October
ment. New York: John Wiley and Sons. 17, 2007, from http://www.coachingnetwork.
org.uk/ResourceCentre/Articles/ViewArticlePF.
De Smet, M., Van Keer, H., & Valcke, M. (2008).
asp?artId=63
Blending asynchronous discussion groups and
peer tutoring in higher education: An exploratory Harris, J., O’Bryan, E., & Rotenberg, L. (1996).
study of online peer tutoring behaviour. Comput- It’s a simple idea, but it’s not easy to do! Practical
ers and Education, 50(1), 207-223. lessons in telementoring. Learning and Leading
with Technology. Retrieved October 17, 2007,
Dewart, H., Drees, D., Hixenbaugh, P., & Wil-
from http://emissary.wm.edu/templates/content/
liams, D. (2004, April 5-7). Electronic peer
publications/October96LLT.pdf
mentoring: A scheme to enhance support and
guidance and the student learning experience. Headlam-Wells. (2004). E-mentoring for aspir-
Paper presented at the Psychology Learning and ing women managers. Women in Management
Teaching Conference, University of Strathclyde, Review, 19(4), 212-218.
Glasgow, UK.
Headlam-Wells, J., Gosland, J., & Craig, J. (2005).
Eby, L. T. (1997). Alternative forms of mentor- There’s magic in the Web: E-mentoring for
ing in changing organizational environments: A women’s career development. Career Develop-
ment International, 10(6-7), 444-459.

E-Mentoring
Kasprisin, C. A., Single, P. B., Single, R. M., & and perspectives and developmental needs of
Muller, C. B. (2003). Building a better bridge: participants in telementoring programs. Paper
Testing e-training to improve e-mentoring pro- presented at the Annual Meeting of the American
grammes in higher education. Mentoring and Educational Research Association, New Orleans.
Tutoring, 11(1). LA.
Kram, K. E. (1985). Mentoring at work: Develop- O’Neill, K., & Harris, J. (2005). Bridging the
mental relationships in organizational life. New perspectives and developmental needs of all
York: University Press of America. participants in curriculum-based telementoring
programs. Journal of Research on Technology in
McCormick, N., & Leonard, J. (1996). Gender
Education, 37(2), 111-128.
and sexuality in the cyberspace frontier. Women
& Therapy, 19, 109-119. O’Neill, K., Harris, J., Cravens, J., & Neils, D.
(2002). Perspectives on e-mentoring: A virtual
McLean, M. (2004). Does the curriculum matter
panel holds an online dialogue. National Mentor-
in peer mentoring? From mentee to mentor in
ing Center Newsletter, 9, 5-12.
problem-based learning: A unique case study.
Mentoring and Tutoring, 12(2), 173-186. O’Regan. (2006). MOLIE: Mentoring online in
Europe. Salford: University of Salford.
McMahon, M., Limerick, B., & Gillies, J. (2002).
Structured mentoring: A career transition support Perren, L. (2003). The role of e-mentoring in
service for girls. Australian Journal of Career entrepreneurial education and support: A meta-
Development, 11(2), 7-12. review of academic literature. Education and
Training, 45(8-9), 517-525.
MENTOR. (2001). US quality standards for
e-mentoring: Elements of effective practice for Rhodes, J. E. (2002). A critical view of youth
e-mentoring. E-Mentoring Clearinghouse. Re- mentoring. Boston.
trieved October, 2005, from www.mentoring.
Ross, B. (2004). First aimhigher e-mentoring
org/emc
networking meeting. Birmingham: Middlesex
Miller, A. (2004). E-mentoring: An overview. Pa- University.
per presented at the First Aimhigher Networking
Salmon, G. (2000). E-moderating. The key to
Meeting, Aston University.
teaching and learning online. London: Kogan
Moore, G. R. (1991). Computer to computer: Page.
Mentoring possibilities. Educational Learder-
Single, P. B., & Muller, C. B. (1999). Electronic
ship, 49(3), 40.
mentoring: Issues to advance research and
Noe, R. A. (1998). An investigation of the deter- practice. Paper presented at the Annual Meet-
minants of successful assigned mentoring relation- ing of the International Mentoring Association,
ships. Personnel Psychology, 41, 457-479. Atlanta, GA.
O’Neill, K. (2004). Building social capital in a Single, P. B., & Muller, C. B. (2000, April 24-
knowledge-building community: Telementoring 28). Electronic mentoring: Quantifying the pro-
as a catalyst. Interactive Learning Environments, grammatic effort. Paper presented at the Annual
12(3), 179-208. meeting of the American Educational Research
Association, New Orleans, LA.
ONeill, K., & Harris, J. (2000, April 24-28). Is
everybody happy? Bridging the perspectives

E-Mentoring
Single, P. B., & Muller, C. B. (2001). When e- toring). Paper presented at the 10th European
mail and mentoring unite: The implementation Mentoring & Coaching Conference.
of a nationwide electronic mentoring program-
Tesone, D. V., & Gibson, J. W. (2001, October).
-MentorNet, the national electronic industrial
E-mentoring for professional growth. Paper
mentoring network for women in engineering
presented at the IEEE International Professional
and science. American Society for Training and
Communication Conference, Santa Fe, NM.
Development.
Wadia-Fascetti, S., & Leventman, P. G. (2000).
Single, P. B., & Muller, C. B. (2005). Electronic
E-mentoring: A longitudinal approach to mentor-
mentoring programs: A model to guide practice
ing relationships for women pursuing technical
and research. Mentoring and Tutoring, 13(2),
careers. Journal of Engineering Education, 89(3),
305-320.
295-300.
Single, P. B., & Muller, C. B. (forthcoming).
Electronic mentoring programs: A model to guide
practice and research. Retrieved January 2008
AddItIonAL reAdIng
from www.apesma.asu.au/mentorsonline/refe-
rence/pdfs/muller_and_boyle_single.pdf
Bierema, L. L., & Merriam, S. B. (2002). E-men-
Single, R. M., Muller, C. B., Cunningham, C. M., toring: Using computer mediated communication
& Single, R. M. (2000). Electronic communities: to enhance the mentoring process. Innovative
A forum for supporting women professionals and Higher Education, 26(3), 211-227.
students in scientific fields. Journal of Women and
Boyer, N. R. (2003). Leaders mentoring lead-
Minority? Engineering, 6, 115-129.
ers: Unveiling role identity in an international
Single, P. B., & Single, R. M. (2005a). E-mentoring online environment. Mentoring and Tutoring,
for social equity: Review of research to inform 11(1), 25-37.
program development. Mentoring and Tutoring,
Clutterbuck, D., & Cox, T. (2005, November).
13(2), 301-320.
Mentoring by wire. Training Journal, 35-39.
Single, P. B., & Single, R. M. (2005b). Mentoring
Crocitto, M., Sullivan, S. E., & Carraher, S. M.
and the technology revolution: How face-to-face
(2005). Global mentoring as a means of career
mentoring sets the stage for e-mentoring. In F.
development and knowledge creation: A learning
K. Kochan & J. T. Pascarelli (Eds.), Creating
based framework and agenda for future research.
successful telementoring programs (pp. 7-27).
Career Development International, 10(6/7).
Greenwich: Information Age Press.
Duff, C. (2000). Online mentoring. Educational
Smith, T., & Ingersoll, R. (2004). What are the
Leardership, 58(2), 49-52.
effects of induction and mentoring on begin-
ning teacher turnover? American Educational Eby, L. T. (1997). Alternative forms of mentor-
Research Journal, 41(3), 681-714. ing in changing organizational environments: A
conceptual extension of the mentoring literature.
Stokes, A. (2001). Using telementoring to deliver
Journal of Vocational Behaviour, 51, 125-144.
training to SMEs: A pilot study. Education +
Training, 43(6), 317-324. Echavarria, T. e. a. (1995). Encouraging research
through electronic mentoring: A case study. Col-
Stokes, P., Garrett-Harris, R., & Hunt, K. (2003).
lege & Research Libraries, 56(4), 352-361.
An evaluation of electronic mentoring (e-men-
0
E-Mentoring
Ensher, E. A., Heun, C., & Blanchard, A. (2003). Kasprisin, C. A., Single, P. B., Single, R. M., &
Online mentoring and computer-meadiated com- Muller, C. B. (2003). Building a better bridge:
munication: New directions in research. Journal Testing e-training to improve e-mentoring pro-
of vocational behaviour, 63, 264-288. grammes in higher education. Mentoring and
Tutoring, 11(1).
Ensher, E. A., Thomas, C., & Murphy, S. E. (2001).
Comparison of traditional, step-ahead, and peer Lavin Colky, D., & Young, W. (2006). Mentor-
mentoring on proteges’ support, satisfaction, and ing in the virtual organization: Keys to building
perceptions of career success: A social exchange successful schools and businesses. Mentoring
perspective. Journal of Business and Psychology, and Tutoring, 14(4), 433-447.
15(3), 419-438.
Lee, H., & Noh, S. (2003). Educational use of e-
Haas, A., Tulley, C., & Blair, K. (2002). Mentors mentoring to encourage women into science and
versus masters: Women’s and girls’ narratives engineering. Lecture Notes in Computer Science,
of (re)negotiation in Web-based writing spaces. 2713, 75-84.
Computers and Composition, 19(3), 231-249.
Mahayosnand, P. (2000). Public health e-men-
Hamilton, B. A., & Scandura, T. A. (2003). E-mentoring: An investment for the next millennium.
toring: Implications for organizational learning American Journal of Public Health, 90(8), 1317.
and development in a wired world. Organizational
O’Neill, K. (2004). Building social capital in a
Dynamics, 31(4), 388-402.
knowledge-building community: Telementoring
Hawkridge, D. (2003). The human in the machine: as a catalyst. Interactive Learning Environments,
Reflections on mentoring at the British Open Uni- 12(3), 179-208.
versity. Mentoring and Tutoring, 11(1), 15-24.
O’Neill, K., & Harris, J. (2005). Bridging the
Headlam-Wells, J., Craig, J., & Gosland, J. (2006). perspectives and developmental needs of all
Encounters in social cyberspace: E-mentoring participants in curriculum-based telementoring
for professional women. Women in Management programs. Journal of Research on Technology in
Review, 21(6), 483-499. Education, 37(2), 111-128.
Headlam-Wells, J., Gosland, J., & Craig, J. (2005). O’Neill, K., Harris, J., Cravens, J., & Neils, D.
There’s magic in the Web: E-mentoring for (2002). Perspectives on e-mentoring: A virtual
women’s career development. Career Develop- panel holds an online dialogue. National Mentor-
ment International, 10(6-7), 444-459. ing Center Newsletter, 9, 5-12.
Henderson, K. L. (1996). Electronic “keyboard O’Neill, K., Weiler, M., & Sha, L. (2005). Soft-
pals”: Mentoring the electronic way. Serials ware support for online mentoring programs: A
Librarian, 29(3-4), 141-164. research-inspired design. Mentoring and Tutoring,
13(1), 109-131.
Hixenbaugh, P., Dewart, H., Thorn, L., & Drees,
D. (2005). Peer e-mentoring: Enhancement of the Paul, R. (2003). Electronic mentoring. School
first year experience. Psychology Learning and Administrator, 60(10), 26-29.
Teaching, 5(1), 8-14.
Perren, L. (2003). The role of e-mentoring in
Hunt, K. (2005). E-mentoring: Solving the issue entrepreneurial education and support: A meta-
of mentoring across distances. Development and review of academic literature. Education and
Learning in Organizations, 19(5), 7-10. Training, 45(8-9), 517-525.

E-Mentoring
Rhodes, J. E. (2002). New directions for youth Stephenson, S. (1998). Distance mentoring. Jour-
development: Theory, practice and research: A nal of Educational Technology, 26(2), 181-186.
critical view of youth mentoring. San Francisco:
Villar, L., & Alegre, O. (2006). Online faculty
Jossey-Bass.
development in the Canary Islands: A study of
Richard, K. (2004). E-mentoring and pedagogy: e-mentoring. Higher Education in Europe, 31(1),
A useful nexus for evaluating online mentoring 65-81.
programs for small business? Mentoring and
Vincent, A. (1999). Using telementoring to
Tutoring, 12(3), 383-401.
overcome mentor shortages: A process model.
Rogan, J. M. (1997). Online mentoring: Reflec- International Journal of Management, 16(3),
tions and suggestions. Journal of Computing in 413-421.
Teaching, 13(3), 5-13.
Wadia-Fascetti, S., & Leventman, P. G. (2000).
Russell, A., & Perris, K. (2003). Telementoring E-mentoring: A longitudinal approach to mentor-
in community nursing: A shift from dyadic to ing relationships for women pursuing technical
communal models of learning and professional careers. Journal of Engineering Education, 89(3),
development. Mentoring and Tutoring, 11(2), 295-300.
227-239.
Weber, R. (2001). Click on to e-mentoring. People
Sinclair, C. (2003). Mentoring online about men- Dynamics, 19(8), 28-39.
toring: Possibilities and practice. Mentoring and
Woodd, M. (1999). The challenge of telementor-
Tutoring, 11(1), 79-95.
ing. Journal of European Industrial Training,
Single, P. B., & Single, R. M. (2005). E-mentoring 23(3), 140-144.
for social equity: Review of research to inform
program development. Mentoring and Tutoring,
13(2), 301-320.

Chapter V
Training Teachers for
E-Learning, Beyond ICT Skills
Towards Lifelong Learning
Requirements:
A Case Study
Olga Díez
CEAD Santa Cruz de Tenerife, Spain
ABstrAct
This chapter describes an experience in teacher training for e-learning in the field of adult education.
It takes into account the models offered by flexible lifelong learning as the proper way to develop train-
ing for teachers in service, considering the advantages of blended learning for the target audience. The
chapter discusses the balance between mere ICT skills and pedagogical competences. In this context the
learning design should always allow that the teachers in training integrate in their work ICT solutions
that fit to the didactic objectives, renew teaching and learning methodology, facilitate communication,
give place to creativity, and allow pupils to learn at their own pace. By doing so, they will be closer to
the profile of a tutor online, as a practitioner who successfully takes advantages of the virtual environ-
ments for collaborative work and learning communication.
Training Teachers for E-Learning
IntroductIon The point of view of the teacher staff may lead

us to conclude that most teachers are aware of the
The aim of this chapter is to focus on the features advantages of using ICT in education. It could
a teacher training course has to fulfil, in order to not be otherwise. Computers are a part of our
facilitate in teachers the use of ICT as a tool to daily life and ICT skills are thus among the new
reach learning goals. Skills and competences are basic skills, according to the Recommendation of
to be developed to guarantee that teachers not the European Parliament and the Council of 18
only are able to make proper use of computers, December 2006 on Key Competences for Lifelong
but also, and most important, that they are aware Learning (Recommendation, 2006).
of the mayor challenges ICT brings as a power- But if we observe at a certain scope, we can
ful means of communication and as an emerging easily notice that the current use of ICT in class-
new pedagogical model. A case study is offered rooms is mainly related to information and data
to point out possible approaches to develop train- transfer and interactive exercises. This is closer
ing programmes. to the Computer Aided Instruction (CAI), than
to a truly e-learning system. In other words, the
possibilities of the Internet as a tool to communica-
BAckground tion, collaborative learning, and development of
social spaces for sharing and building knowledge
Teaching training is a steady challenge in the remain still almost unexploited. For instance,
always evolving learning the knowledge society the flexibility that e-learning offers to support
requires. It is nowadays a common place to point and guide the learning activities of pupils who
out the advantages of ICT integration into school need to increase their learning result is an almost
as a proper way to transform information into unexplored field. In the process of developing
knowledge (Barberà-Badia, 2004). As shown in e-learning solutions for schools, teacher staff,
a survey developed for the European Union by policy makers, and other stakeholders are due
the European Schoolnet (2005), in the last years to shift to a broader understanding of the possi-
a great effort has been made to ensure the pres- bilities of e-learning within the formal learning.
ence of ICT in every school through the neces- From a content centred approach, that may help
sary infrastructure. As a result, more than 80% teachers in delivering instructional predefined
of the European teachers describe themselves as contents. It is possible to reach a more flexible
competent in using computers and the Internet e-learning model, which also correlates to the
in classroom situations; two-thirds dispose of lifelong learning objectives.
the necessary motivation for doing so (accord- This previous path is to be taken to ensure
ing to their own opinion), and 60% describe positive experiences for teachers and interest-
the ICT infrastructure in their schools and the ing learning results, and, accordingly, a natural
Internet connection as sufficiently rapid. This shift to a more open minded use of the Web as
means that most teachers use computers in their a powerful way to build and share knowledge,
everyday work, but, on the other hand, some are which will probably bring us to the almost mythi-
still reluctant to do so, mainly those who claim cal realm of e-learning 2.0, often foretold as the
that a subject does not lend itself to being taught future scenario that will allow learning in every
with computers, or that there is a lack of proper possible human situation.
didactic contents.

FLexIBLe LIFeLong The report entitled “Assessment Schemes for

LeArnIng As A modeL Teachers’ ICT Competence-A Policy Analysis,”
For teAcher trAInIng developed for the European Union by the Euro-
pean Schoolnet (2005), includes some remarkable
Once stated the necessity of a broader training key findings:
model for teachers, even in formal educational
contexts, it is to be taken into consideration which • In the future more detailed job descriptions
kind of training programme is to be developed. and specialized training profiles are needed
As a matter of fact, many training courses are for different actors in schools to cater for a
regularly offered to teacher staff by the Educa- personalised training.
tional Departments in every European country. • Training policies face the challenges to be
The local peculiarities of this offer make it dif- flexible enough for short term adjustments
ficult to establish a regular standard within the of changing training needs and incorporat-
European Union and to design a proper common ing long term goals and objectives that are
policy in teacher training. important for teachers to identify with.
A common background is given by the Com- • Countries will need to think of offering new
mon European Principles for Teacher Competenc- and flexible forms of training for teachers,
es and Qualifications (European Commission, at different times, at different places, with
2005), where the European Union views the role different means, but much more related to the
of teachers and their lifelong learning and career concept of lifelong learning. This includes a
development as key priorities. Teachers should be shift in the culture of the teaching profession
equipped to respond to the evolving challenges of from a passive consumer of training courses
the knowledge society, participate actively in it, to an active producer and organiser of its
and prepare learners to be autonomous lifelong own learning process.
learners. The key competences of teachers are • Training policies can only be successfully
set as follows. implemented and sustainable in the long
Teachers should be able to: term if they are part of an interlinked or
integrated ICT strategy that caters for tech-
• Work with others nology, pedagogy, support, organisational
• Work with knowledge, technology, and development, and (financial) solutions.
information
• Work with and in society From this point of view, e-learning solutions
are an interesting approach that allows flexible
To facilitate such an approach, ICTs are not only forms of training, but that have to be delivered
a means distribute training course for teachers at under some conditions to ensure the quality of
service, but also the logical environment where their results.
these three dimensions of the key competences E-learning is unfortunately a very broad term,
are to be developed and fulfilled. A proper use of which may lead readers to think of many different
ICT empowers the abilities needed for collabora- learning scenarios, and therefore, it seems to be
tive work, as well as requires an autonomous use necessary to define or, at least, to set the limit of
of information sources, its selection and delivery, the concept for the aims of this chapter.
and allows teachers to keep in touch with a steady Computer and learning are the two basic ideas
changing society, into which their pupils are to that come to our minds when trying to define e-
become active citizens. learning. Therefore, a first definition could point

out this relationship. For instance: “The delivery evolved development of these, we can find the
of a learning, training or education program by virtual corelate to the community of practice, that
electronic means. E-learning involves the use of is, “a shared domain of interest” where “members
a computer or electronic device (e.g., a mobile interact and learn together” and “develop a shared
phone) in some way to provide training, educa- repertoire of resources.” In others words, it is the
tional or learning material” (Stockley, 2003). Such place where learning happens (Wenger, 1998).
a definition involves the delivery of instruction via In this pedagogical approach, the new role of
CD, the Internet, or shared files on a network. It the teacher is a turning point for the development
is also called computer mediated learning. of e-learning (Kearsley, 2000).
It is not surprising that a new definition of e- In the last years many e-learning courses
learning is being developed, as far as a broader have been developed at high schools, universi-
use of the Web has been reached. The so-called ties, and enterprises, and many lessons are to
Web 2.0 (O’Reilly, 2005) enables a new definition be learned from them. In the most successful
of the concept, under the label of e-learning 2.0 experiences, the key factor is the presence of a
(Downes, 2005; Jennings, 2005; Karrer, 2006). specialized trainer who ensures the effectiveness
The interesting point of this concept is that e- of the Web-based learning process. This trainer
learning can no longer be defined only by the is skilled and competent in interaction, com-
use of ICT itself, but by a certain use of the ICT. munication, and knowledge building through
It includes communication, collaborative learn- virtual spaces. In other words, this professional
ing, social networks, and new roles for learners is the tutor online, defined as follows by Seoane
and teachers. and García (in press):
But this supposed novelty is to be tracked back
to the theories that stressed the change from a Tutor online is the teaching staff that follows a
transmission model of knowledge transfer, to a group of students on a part of their learning path,
learner-centred or activity-centred model (Gifford ensures the efficiency of teaching-to-learning pro-
& Enyedy, 1999; Reeves, 1999; Vinicini, 2001; cess, promotes the achievement of aims and skills
Wilson, 1995). predicted for the academic initiative that he leads,
The conventional classroom was the natural by creating a context of collaborative and active
metaphor in which many learning management learning, and evaluates how pre-established aims
systems (LMS) and, even more importantly, most were achieved for students and for the academic
learning designers and content creators, developed intervention (quality management).
the learning environment, from computer aided
instruction (CAI) to many online courses. They Of course, the teacher staff in schools is yet
order the sequence of information and focus on far from reaching such a level of acquaintance
the structure of the disciplinary domain. and competence as is to be found in a proper tu-
But as far as it is possible to encourage com- tor online. Nevertheless, according to the variant
munication, interaction, and collaboration in reality of schools and the different target learners
e-learning environments, this model is to be they serve, in certain kinds of educational institu-
supplied with news methods that allow achiev- tions teachers functions are getting nearer to this
ing orchestrated interdependence and autonomy profile, as far as they have in their classroom an
in e-learning. The new idea is well summarized increasing variety of pupils. This is the situation
by the image of a community, a virtual learning of centres devoted to adult education, vocational
community. (Cabero, 2006; García Aretio, 2003; training centres, and secondary schools provid-
Hudson, 2005; Paloff & Pratt, 1999). In the most ing courses to adult and young adult people who

need to improve their educational outcomes and thinking skills to perform the kind of information
validate the skills developed in their job. management and research tasks necessary for
In this field much is to be made in order to academic success.
prepare ordinary teachers to become adult teach- On the other hand, most teachers are “digital
ers with skills and competences allowing them immigrants.” This situation in the average class-
to bring to their pupils an attractive, flexible, and room reflects the digital divide that currently exists
accurate learning. In many cases this involves in Europe. Furthermore, quite often teachers feel
that they too have to learn the new skills needed less competent than their pupils in this field, and
in the knowledge society. this is the reason why they do not risk integrat-
ing ICT to a greater extent (Barnes, Marateo, &
Ferris, 2007).
deveLoPIng teAcher
trAInIng ProgrAmmes: how is it Possible to train teachers
Beyond Ict skILLs in this Evolving Educational
context?
Teacher training as an efficient way to develop
the skills needed for e-learning is not simple. Therefore, when designing a teacher training
As a matter of fact, it is a long-term aim which course, a balance between technical and didactic
should be reached step by step through minor contents is to be reached. In many cases the new
formative actions. ICT tools are introduced to teachers without point-
The role of formal learning as a fist step ing out clearly which are the didactic benefits they
towards lifelong learning is reinforced by the provide, or how far they could ease their daily
Recommendations of the European Parliament work. Moreover, a great amount of funds are spent
and the Council on key competences. Its first aim on courses whose results are rarely incorporated
is to ensure that “initial education and training ino the daily work in the classroom.
offers all young people the means to develop the A few questions are to be asked when design-
key competences to a level that equips them for ing teacher training courses:
adult life, and which forms a basis for further
learning and working life.” • What kind of skills does the course intend
It is important to notice that e-learning involves to facilitate?
the capability to acquire knowledge and develop • Are these new skills profitable for teachers
skills within Web-based means. E-learning, when at the end of the course, or could they even
properly led, facilitates the metacognitive aware- take advantage of them as they are attending
ness needed in the field of “learning to learn.” the course?
Therefore, ICT in this context is just an enabler • If the didactic advantages are clear, is the
in a new means to learn that should also encom- related ICT presented as a means or does
pass several key elements such as learning design, the course focus mainly on it?
collaborative learning, and social contexts. • How far does the course allow teachers to
In spite of the fact that younger generations develop their creativity to incorporate the
have grown up with ICT and are thus “digital new skills in their own learning context?
natives” (Prensky, 2001), they are far from being
digitally literate. Preliminary research released by With these questions in mind, we will present
Educational Testing Service (ETS) on November the experience of a teacher training course that
14, 2006, shows that many students lack the critical took place in 2005-2006. The study of this case will
provide some basis for profitable conclusions.

A case study: training teachers for management system designed to help educators
Formal Adult education within open create online courses with opportunities for rich
Learning methodology interaction, integrating resources, and activities
as well as assessment tools. The workshop was
The Educational Department of the Regional totally face-to-face, in groups of 20 participants,
Government of the Canary Islands, Spain, offered to allow hands-on experience with a computer
a training course to the teacher staff working for under the guide of an instructor, during a total
adult education. It was held during 5 months (from of 25 training hours.
February to June 2006), and certificated 100 train- The contents of the course comprised five
ing hours. It was carried out as a blended learning different thematic units:
course, that is, there was one compulsory face-
to-face meeting per month. It took place at three • Adult education features
different islands (Lanzarote, Tenerife, and Gran • Distance learning
Canaria), and 246 teachers from the seven Canary • Tutoring in adult education
Islands registered. The participants worked at • Designing learning contents for adult educa-
primary schools, vocational training centres, section
ondary schools, and Escuelas Oficiales de Idiomas • ICT supported distance learning
(schools devoted to foreign languages).
The “Curso de educación de personas adultas Every Unit was introduced by a face-to-face
en modalidad no presencial” had as its main goal session in which some practical examples of
to introduce adult education features, in order to the previous activities and units were given, the
develop the required skills for open education, main topics of the new unit were underlined and
using ICT as a helpful means. directions for the further activities were offered.
The general learning objectives of the training The face-to-face sessions were scheduled as
course were stated as follows: large classes meetings (about 80 people) where
the tutors acted mainly as traditional teachers,
a. To approach teachers to the theories and developing topics and giving general advise to
practices related to adult education. follow the Unit.
b. To deliver basic knowledge about the speci- During the month, between face-to-face ses-
fications of this field of education. sions, the teachers who had given a lecture in the
c. To recognize and analyze the features of ordinary one-to-many way, changed their func-
distance learning, the related methodology tion and supported open many-to-many discus-
and specially the tutorial and advisor role sion, as tutors online in the virtual main course.
of the teachers. Therefore, during the five months the course was
developed, every participant counted on the sup-
Accordingly, the course structure aimed to port and guidance of the tutor team, which, not
combine individual and group learning activities, only designed and delivered the learning contents
supported on the Internet, through a learning and activities of each Unit, but also provided chat
management system (LMS), and completed with meeting, forum discussion, personal e-mail advice
face-to-face sessions, once a month. and technical support.
Previously, all participants had to attend a At the end of the course, participants could
workshop in order to get basic skills on the use choose between designing a learning Unit or
of a LMS, both as a student and as a tutor. In this creating learning content for a specific subject
case, it was Moodle 1.5.4., a well-known course in the context of adult education.

A BLended APProAch and the fulfilment of individual tasks to be

delivered at the monthly meeting; the use of a
The course was developed under a blended form, computer was previewed as a way to deliver
as a proper way to initiate teachers into e-learn- written contents to the participants and to
ing. Blended learning is indeed another evasive allow them to ask questions in between. The
concept (Oliver & Trigwell, 2005) that some blended form allowed the reutilization of
authors relate to the frustration of e-learning in written contents and made a step forward, as
general terms (Bernabé, 2004). But for the goals the virtual classroom was the central point
and features of the course contents and the target of the course and the face-to-face sessions
audience, it was the chosen model (Valiathan, were intended to reinforce the online learn-
2002). The benefits of such a decision were the ing.
following: • Learner centred methodology: Due to
the very broad variety of interest, work-
• Organization of the course: As the number ing contexts, and previous experience on
of participants was about 250 teachers with ICT and adult learning of participants, the
only four instructors, a completely online blended approach made it easier to present
development of the course would have been the common points and bring together the
very difficult to fulfil without a rather high different learning situations in the face-to
rate of attrition. The long duration of the face classroom, and to work in more detail
course was another factor of risk to be taken the difficulties and interest of participants
into account. (Diaz & Cartnal, 2006). almost on demand, in the virtual classroom.
• Pace to develop ICT skills: The blended ap- It was possible to minimize the tendency
proach of the course shifted gradually from a to dispersion of participants that grows as
full face-to-face beginning in the workshops a long term online course develops, and
to an almost complete online development helped trainees to keep in mind the main
for the final assessment (Driscoll, 2002). In goals of the course.
the meantime, the monthly sessions allowed
the instructors to reinforce the motivation of The course was led and coordinated by a team
participants, present the best results of the of four tutors who were actually teachers at the
proposed tasks, and increase the informal same educational levels as the participants and
meeting of trainers and trainees at the cof- had a broad experience in adult open education,
fee-breaks. Moreover, it allowed learners to creation of learning content, online tuition, and
gradually move form the traditional role in as teacher trainers.
a classroom, to the active participation in Apart from the “common main course,” in
the virtual classroom through forums and the virtual environment every participant had a
chats as a public way to share experience and “practice course” to test and develop the contents
build knowledge. Thus, the implementation and activities of the course. Therefore, they de-
of the course fostered the development of veloped a double role in the virtual environment,
higher ICT skills as essential to the learning as students in the “common main course,” and as
process. teachers in their own “practice course.” On de-
• Course contents: The same course had been mand, 288 practice courses were implemented, as
delivered in previous years through a classic participants could choose whether to develop their
distance learning schedule which involved tasks alone or in small collaborative teams.
a lot of individual work with a handbook,

It was in the virtual common course where began to develop self-made contents, they became
the social dimension of the proposed learning more aware of the fact that surfing the Internet
path took place. Beside unit contents and tests, is not so easy, making digital contents properly
special attention was driven to foster and promote accessible through the Web requires some special
the use of communicative tools such as forums, attention, and multimedia learning contents were
chats, and internal messages. E-mail was another something they could experiment with.
possibility to ask tutors for help or advice, but By the end of the course, only 26 participants
its use was limited to the moments when strong had never entered the course, and for the rest of
technical problems took place within the virtual them only 5 delayed in the delivery of the activi-
environment. ties required for assessment. The initial dropout
The forums were the main way to develop rate was 10%, but the number of participants
communication throughout the whole course. throughout the whole course stayed the same.
A glance at the many logs they received made it At the end of the course, a evaluation ques-
clear: there were 42,816 logs in all the available tionnaire was answered by the participants. It
forums (i.e., 147 logs per participant), being the considered course development and organization,
general forum the most visited. It was the place tutors, work, communicative skills, and adequacy,
not only for general matters about the course, usefulness, and interest of course contents. Unfor-
but mainly to share experiences, to make open tunately, the results are not yet available from the
questions, and to recommend further information department that carried out the course. Neverthe-
or Web sites always in the scope of the aims of less, according to the posts sent by participants
the course. Most of this discussion was started after the end of the course, it was most successful.
out by the participants and sometimes produced They reported to have learned a lot and were in-
long threads of conversation, often moderated terested in attending further courses of this kind
by the tutors. because of it flexibility and quality.
Chat was used only by recommendation of When asked after the final on-ground meeting,
the tutors as a part of the contents of the course, the tutors also expressed their satisfaction with
not having an important role in other situations. the development of the course, the attendance of
Internal messages were used mainly to keep in participants, and the learning results.
touch with other participants, while the main way
to ask for advice to the tutors was the forums. The Learning design and Learning
tutors always answered in less than 12 hours, be- outcomes
ing the average time of answer 2 hours after the
question was made. Some important issues from this reported course
Another particular feature of the course was could be summarized as follows.
the three online workshops, devoted to technical In spite of the fact that the aim of the course
issues that participants might need, when creating was to introduce teachers into adult education
their own contents and courses. The goal of these and lifelong learning and to enable them to cre-
workshops, as stated in the course syllabus, was ate specific learning contents for adults, the final
to improve the digital literacy and ICT skills, in results also included other outcomes.
accurate information search in the Web, authoring
tools and standards contents formats, as pdf, and • About one third of the teachers had never
audio files creation. Though attendance at these before used online communication tools
workshops was not compulsory, almost every par- such as forums, chat, or messages. Many
ticipant took part in at least one of them. As they of them considered these to be part of the
0
younger generation’s habits. Through the by doing so, they were involved step by
steady use of them, they were aware of their step in a new ICT environment and tested
learning usefulness as means not only to new technological tools because they could
foster motivation and social skills, but also foresee the benefit of them.
to generate a more accurate learning. • Of course, there were participants who did
• The use of these tools also had as a result not learn as much ICT during the course.
that most teachers could express more clearly But for most of them this was the fist time
their own expectations during the course, they had to harmonize their daily routines,
being thus a way to improve metacognitive their work at school, and their virtual and
skills. The forum was a great help to reach- almost daily presence in the course during
ing better learning outcomes, but it is also several months. They wanted to take the best
remarkable that some of the participants also advantage from their effort experiencing
stated that though they were rather “lurk- thus by themselves some of the conditions
ers” at the forums, as they felt uneasy when their adult learners have to face in order to
sending posts. This was not an obstacle to obtain valuable learning results.
reach the course objectives. In other words, • Furthermore, the use of peer to peer com-
their learning styles did not suit for active munication made it possible in the most
public written participation, but they could remarkable cases to investigate the use of
benefit from the group interaction, merely a LMS as a virtual environment relevant
as lurkers. for learning activities as well as a for col-
• In spite of the fact that collaborative learning laborative work and for the dissemination of
was not a goal in this particular course, nor teaching experiences and strategies. Under
its chosen methodology, the communication these circumstances the first steps to develop
flow was so rich that it introduced some kind a virtual community of teachers could have
of collaborative synergy that was present in been taken, if the required leading conditions
the final activities. to sustain it had been given.
• The course benefited from a flexible design
that allowed the tutors to adapt it to the de-
mands of participants. It seems that in this concLusIon: From A LIst
case, the proper use of a LMS, like Moodle, oF skILLs to A set oF
as the main space for communication, made comPetencIes
it possible that the on-ground meetings were
considered more as an introduction to the As stated by the European Parliament and the
tasks that were proposed to be fulfilled Council, the aims of education are “personal
during the following month than as the fulfilment, active citizenship, social cohesion and
core of the course. From the point of view employability in a knowledge society.” In such a
of the tutors, the core of this course was social context a broad educational policy is needed.
the interaction and the work carried out in Teaching, even in formal contexts, deals no more
the virtual environment, while the face-to- with the transmission of a set of predefined learn-
face sessions were rather a companion to ing contents, but it shifts towards the development
this than the contrary. As usually happens, of capacities that enable citizens to adapt dynami-
participants wanted to learn real things, cally to a rapidly changing world.
ideas, tips, and resources that could easily From this starting point, it is obvious that
improve their work with adult pupils. And teacher staff needs to be enabled to accomplish the

required functions in an always evolving society. the learning activities beyond classroom
This implies that a large scope policy for teacher walls and to allow people to work together
training is to be developed in order not merely to within a collaborative framework.
obtain a certain list of new skills, mostly those • The development of new roles for teachers
related to the use of ICT in learning situations. and learners and the way the different in-
It actually involves that teachers require training structional designs sustain it when adapted
to apply their skills to new problems, under new to the working context. This involves set-
conditions. They should thus develop skills into ting the rules of flexible learning in order
competences, and, by doing so, integrate in their to avoid an excess that could drive courses
work ICT solutions that fit to the didactic objec- to chaotic random learning.
tives, renew teaching and learning methodology,
facilitate communication, give place to creativity, It is of particular interest the study of learning
and allow pupils to learn at their own pace. By outcomes in teachers training courses developed
doing so, they will be closer to the profile of a tutor online. The analysis of their implementation
online, as a practitioner that successfully takes and the research of the issues that guarantee the
advantages of the virtual environments for col- quality of their development would offer valuable
laborative work and learning communication. guidance.
Future reseArch dIrectIons reFerences
The study of a single case only allows an out-

line of the relationship between learning design Barberà, E., & Badia, A. (2004). Educar con aulas
and learning outcomes. The evaluation of other virtuales. Orientaciones para la innovación en el
teacher in service training courses should take in proceso de enseñanza y aprendizaje. Barcelona:
consideration the issues presented in this chapter, A. Machado Libros.
in order to establish some conclusions about the
Barnes, K., Marateo, R., & Ferris, S. (2007).
following trend topics:
Teaching and learning with the net generation.
Innovate, 3(4). Retrieved October 18, 2007,
• The supposed instructional benefits of a
from http://www.innovateonline.info/index.
blended learning course compared with
php?view=article&id=382
authentically online training courses. This
involves a study of the online learning design Bernabé, A. (2004). Blended learning. Conceptos
that should underlie the implementation of básicos. Pixel-Bit. Revista de Medios y Educación,
this kind of course in order to determine 23, 7-20. Retrieved October 18, 2007, from www.
in which conditions a blended or an online lmi.ub.es/personal/bartolome/articuloshtml/04_
design should be desirable. blended_learning/documentacion/1_bartolome.
• The integration of ICT in the daily work of pdf
teachers. Where do the barriers that embar-
Cabero, J. (2004). Bases pedagógicas del e-
rass the use of ICT in teaching lie? How it is
Learning. Revista de Universidad y Sociedad
possible to promote ICT integration through
del conocimiento, 3. Retreived October 18, 2007,
teaching training?
from http://www.uoc.edu/rusc
• Collaborative work and learning. The so-
called Web 2.0 makes it possible to broaden

Diaz, D., & Cartnal, R. (2006). Term length Retrieved October 18, 2007, from http://www.e-
as an indicator of attrition in online learning. Learningeuropa.info/index.php?page=doc&doc_
Innovate, 2 (5). Retrieved October 18, 2007, id=6494&doclng=7&menuzone=1
from http://www.innovateonline.info/index.
Jennings, D. (2005). E-learning 2.0, whatever that
php?view=article&id=196
is. Retrieved October 18, 2007, from http://alchemi.
Downes, S. (2005, October 17). E-learning 2.0. co.uk/archives/ele/e-Learning_20_wh.html
Elearn Magazine. Retrieved October 18, 2007,
Karrer, T. (2006, February 10). What is e-learning
from http://elearnmag.org/subpage.cfm?section
2.0. E-Learning Technology. Retrieved October
=articles&article=29-1
18, 2007, from http://e-Learningtech.blogspot.
Driscoll, M. (2002). Blended learning: Let’s get com/2006/02/what-is-e-Learning-20.html
beyond the hype. Learning and Training Innova-
Kearsley, G. (2000). Online education: Learn-
tions Newsline. Retrieved October 18, 2007, from
ing and teaching in cyberspace. Belmont, CA:
http://www.ltimagazine.com/ltimagazine/article/
Wadsworth.
articleDetail.jsp?id=11755
Oliver, M., & Trigwell, K. (2005). Can blended
European Commission. (2005). Common Eu-
learning be redeemed? E-Learning, 2(1). Re-
ropean principles for teacher competences
trieved October 18, 2007, from http://www.
and qualifications. Retrieved October 18, 2007,
wwwords.co.uk/pdf/viewpdf.asp?j=elea&vol=2
from http://europa.eu.int/comm/education/poli-
&issue=1&year=2005&article=3_Oliver_ELEA_
cies/2010/doc/principles_en.pdf
2_1_web&id=83.104.158.140
European Schoolnet. (2005, July 15). Insight
O’Reilly, T. (2005). Web 2.0: Compact definition?
special report on assessment schemes for
Retrieved October 18, 2007, from http://radar.
teachers’ ICT competence—A policy analysis.
oreilly.com/archives/2005/10/web_20_com-
Retrieved October 18, 2007, from http://www.e-
pact_definition.html
Learningeuropa.info/index.php?page=doc&doc_
id=6578&doclng=6 Paloff, R.M., & Pratt, K. (1999). Building com-
munities in cyberspace: Effective strategies for the
García Aretio, L. (2003). Comunidades de apre-
online classroom. San Francisco: Jossey-Bass.
ndizaje en entornos virtuales. La comunidad
iberoamericana de la CUED. In M. Barajas (Ed.), Prensky, M. (2001). Digital natives digital im-
La tecnología educativa en la enseñanza superior. migrants. On the Horizon NCB University Press,
Madrid, Spain: McGrawHill. 9(5).
Gifford, B.R., & Enyedy, N. (1999). Activity cen- Recommendation of the European Parliament and
tered design: Towards a theoretical framework for the Council of 18 December 2006 on Key Com-
CSCL. In Proceedings of the Third International petences for Lifelong Learning. (2006, December
Conference on Computer Support for Collabora- 12). Official Journal of the European Union
tive Learning. Retrieved October 18, 2007, from Retrieved October 18, 2007, from http://www.
http://www.gseis.ucla.edu/faculty/enyedy/pubs/ cimo.fi/dman/Document.phx/~public/Sokrates/
Gifford&Enyedy_CSCL2000.pdf Comenius/keycompetences06.pdf
Hudson, B. (2005). Conditions for achieving Reeves, W. (1999). Learner-centered design: A
communication, interaction and collaboration in cognitive view of managing complexity in product,
e-learning environments. E-Learningeuropa.info. information, and environmental design. Thousand
Oaks, CA: Sage.

Seoane Pardo, A.M , & García Peñalvo, F.J. (in Bonk, C.J., & Graham, C.R. (2005). Handbook
press). Tutoring & mentoring online. Definition, of blended learning: Global perspectives, lo-
roles, skills and case studies. In G.D. Putnik & cal designs. San Francisco: Pfeiffer Publishing.
M.M. Cunha (Eds.), Encyclopedia of networked Retrieved October 18, 2007, from http://www.
and virtual organizations. Hershey, PA: Idea uab.edu/it/instructional/technology/docs/blend-
Group Inc. ed_learning_systems.pdf
Stockley, D. (2003). E-learning definition. Re- Cabero, J. (2004). Bases pedagógicas del e-
trieved October 18, 2007, from http://derekstock- Learning. Revista de Universidad y Sociedad
ley.com.au/elearning-definition.html del conocimiento, 3. Retreived October 18, 2007,
from http://www.uoc.edu/rusc
Valiathan, P. (2002). Blended learning models.
Learning Circuits. Retrieved October 18, 2007, Dillenbourg, P. (1999). Collaborative learning.
from http://www.learningcircuits.org/2002/ Cognitive and computational approaches. New
aug2002/valiathan.html York: Pergamon Earli.
Vinicini, P. (2001). The use of participatory design Educational Testing Service. (2006). ICT lit-
methods in a learner-centered design process. eracy assessment preliminary findings. Retrieved
ITFORUM 54. Retrieved October 18, 2007, from October 18, 2007, from http://www.ets.org/Me-
http://it.coe.uga.edu/itforum/paper54/paper54. dia/Products/ICT_Literacy/pdf/2006_Prelimi-
html nary_Findings.pdf
Wenger, E. (1998). Communities of practice European Commission. (2006, September 29).
learning, meaning, and identity. Cambridge Benchmarking access and use of ICT in European
University Press. schools 2006. Retrieved October 18, 2007, from
http://ec.europa.eu/information_society/eeurope/
Wilson, B.G. (1995). Situated instructional design:
i2010/docs/studies/final_report_3.pdf
Blurring the distinctions between theory and prac-
tice, design and implementation, curriculum and Inaba, A., Ikeda, M., & Mizoguchi, R. (2003).
instruction. In M. Simonson (Ed.), Proceedings of What learning patterns are effective for a learner’s
selected research and development presentations. growth? In U. Hope, F. Verdejo, & J. Kay (Eds.),
Washington, DC: Association for Educational Artificial intelligence in education: Shaping the
Communications and Technology. Retrieved future of learning through intelligent technologies
October 18, 2007, from http://carbon.cudenver. (AIED2003) (pp. 219-226). Sydney, Australia.
edu/~bwilson/sitid.html
Jonassen, D.Y., & Rorher-Murphy, L. (1999).
Activity theory as a framework for designing con-
structivist learning environments. Educational
AddItIonAL reAdIng Technology: Research and Development, 46(1).
Keegan, D. (1988). Theories of distance education:
Berge, Z., Collins, M., & Dougherty, K. (2000).
Introduction. In D. Sewart, D. Keegan & B. Hol-
Design guidelines for Web-based courses. In B.
mberg (Eds.), Distance education: International
Abbey (Ed.), Instructional and cognitive impacts
perspectives (pp. 63-67). New York: Routledge.
of Web-based education (pp. 32-40). Hershey, PA:
Idea Group Publishing. Oblinger, D.G., & Oblinger, J.L. (Eds.). (2005).
Educating the net generation. Washington, DC:
EDUCAUSE. Retrieved October 18, 2007, from

http://www.educause.edu/books/educatingth- Siemens, G. (2005). Connectivism: A learning

enetgen/5989 theory for the digital age. International Journal
of Instructional Technology and Distance Learn-
Parchoma, G. (2003). Learner-centered instruc-
ing, 2(1).
tional design and development: Two examples of
success. Journal of Distance Education, 18(2). Tan, S.C., Hu, C., Wong, S.K., & Wettasinghe,
C.M., (2003). Teacher training on technology-en-
Parchoma, G. (2005). Roles and relationships in
hanced instruction—A holistic approach. Educa-
virtual environments: A model for adult distance
tional Technology & Society, 6(1), 96-104.
educators extrapolated from leadership in expe-
riences in virtual organizations. International Vygotsky, L. (l978). Mind and society. Cambridge,
Journal on E-Learning, 4(4). MA: Harvard University.

Chapter VI
The Role of Institutional
Factors in the Formation of
E-Learning Practices
Ruth Halperin
London School of Economics, UK
ABstrAct
This chapter explores institutional and socio-organisational factors that influence the adoption and use
of learning management systems (LMS) in the context of higher education. It relies on a longitudinal
case study to demonstrate the ways in which a set of institutional and organisational factors were drawn
into the formation and shaping of e-learning practices. Factors found to figure predominantly include
institutional conventions and standards, pre-existing activities and routines, existing resources avail-
able to the institution, and, finally, the institution’s organisational culture. The analysis further shows
that socio-organisational factors may influence e-learning implementation in various ways, as they both
facilitate and hinder the adoption of technology and its consequent use. It is argued that institutional
parameters have particular relevance in the context of hybrid modes of e-learning implementation, as they
illuminate the tensions involved in integrating technological innovation into an established system.
PArt I: BAckground longitudinal case study in an academic setting,

the chapter reveals the vital role of institutional
IntroductIon concerns for understanding learning technology
use and its consequences.
This chapter focuses on the institutional and By exploring institutional and organisational
socio-organisational factors that influence the factors in e-learning, this study addresses a defi-
use of learning management systems (LMS) in nite gap in the literature to date. As shown in a
the context of higher education. Drawing on a literature review, various factors that may facili-
The Role of Institutional Factors in the Formation of E-Learning Practices
tate or hinder the effective use and integration of At the core of this chapter is a set of institu-
learning technology have been explored. These tional and socio-organisational factors impinging
include technical factors such as availability, on e-learning which will be seen to arise from the
stability, and reliability, factors associated with case analysis. Factors found to figure predomi-
instructional design, and, to a large extent, user nantly include: (a) institutional conventions and
related factors, namely attitudes and perceptions. standards, (b) institutional activities and routines,
Yet, these factors are typically studied in isolation (c) organisational resources (physical, techno-
and socio-organisational factors are effectively logical—other than LMS—and human), and, (d)
ignored. The significance of studying institutional organisational culture and social relations.
factors stems not only from the potential role they After introducing the factors and demonstrat-
are likely to play, and have repeatedly been shown ing their role in the formation of e-learning prac-
to play in the context of information systems other tices, a discussion of their implications follows.
than e-learning, but more crucially in the light It is argued that these parameters have particular
of the prevailing mode of hybrid (or blended) relevance in the context of hybrid modes of e-
e-learning. Within hybrid models of integration, learning implementation, as they illuminate the
the role played by the pre-existing institutional tensions involved in integrating technological
context becomes all the more important, as the innovation into an established system. It will be
technological environment is meant to comple- shown that in cases where technology was intro-
ment, rather than replace, the existing and long duced to supplement existing arrangements, that
established learning system. is, to compensate for deficiencies affecting the
Findings presented in this chapter demon- existing “off-line” setting, the integration process
strate the ways in which a set of institutional was typically vigorous and accelerated. Clearly,
and organisational factors were drawn into the difficulties and challenges also arose as the LMS
formation and shaping of e-learning practices, was seen to compete or clash with its veteran off-
defined as the shared and recurrent activities that line counterpart. In some cases, interoperation
emerge from learners’ continuous interaction with and fusion were achieved through negotiation;
learning technology. The analysis further shows in others, technological properties were ruled out
that socio-organisational factors may influence and capabilities remained unexploited.
e-learning implementation in various ways, as The next part of the chapter provides a review
they both facilitate and hinder the technology of the literature on factors influencing the use
adoption and its consequent use. of learning technology. Although considerable
The case study reported in this chapter involves research on the topic has been undertaken, find-
the use of a standard LMS in a traditional, well- ings on the institutional and socio-organisational
established university in the UK. Focus is placed factors are strikingly absent. The aim of the pres-
on the integration of the LMS into the provision of ent study is to address this gap in the e-learning
a masters degree in a faculty of social science. Data literature.
collection encompassed three consecutive years,
starting from the point at which the technology
was first introduced in the institution. A research FActors InFLuencIng the use
design was devised so as to guide a systematic ex- oF LeArnIng technoLogy
amination of the organisational context. Relevant
institutional levels were mapped out and analysed Various factors that may facilitate or hinder the
as interconnected layers (Pettigrew, 1990). effective use and integration of learning technolo-
gies have been studied and are briefly reviewed

in the following sections. A general overview few of the factors explored. It should be noted,
is provided first, before attention is drawn to however, that instructional design of learning
research on user perspectives, representing the technology emerges as a topic in its own right
most frequently studied parameter. and so a comprehensive review of the subject is
well beyond the scope of this chapter.
overview of Factors Studies The instructional style and, in particular, the
role played by teachers/instructors in e-learn-
Some attention has been paid to strategic con- ing environments is considered a key factor
siderations associated with the implementation affecting learning interactions online (Guldberg
of learning technology (LT). For example, Boyd- & Pilkington, 2007). The primary role of the
Barret (2000) has examined six different models e-moderator in facilitating an environment for
of universities implementing LT and identified effective learning to occur is frequently advocated
three primary institutional and political charac- (Salmon, 2000). However, this approach has been
teristics that have critical influence on distance recently criticised by Oliver and Shaw (2003) as a
learning outcomes. These include private or kind of pedagogical determinism. In their study,
public emphasis, degree of dedication to online the tutor’s enthusiasm and expertise are viewed
learning, and holistic or incremental strategy. as the major factors stimulating student engage-
In addition, three secondary dimensions are ment in asynchronous discussions. Mazzolini
considered: technology mix, financial produc- and Maddison (2003) have found that different
tion models, and target markets (Boyd-Barret, roles taken by online instructors can influence
2000). Williams (2003) has identified and rated students’ participation and perceptions but not
organisational roles and competencies needed for always in expected ways. They conclude that
successful deployment of e-learning programmes the rates at which instructors participate are not
in higher education institutions. Implications for simple indicators of the quality of online discus-
staff development and training are discussed at sion and more subtle measures of the effectiveness
the managerial level. of asynchronous discussion forums for learning
Technical factors commonly addressed in the and teaching are warranted.
literature include availability and access (Chiero,
1997; Tu, 2000) as well as reliability and stability User Perspectives
of the technology in use (Webster & Hackley,
1997). Hara and Kling (2000) provided a system- A review of the e-learning literature clearly sug-
atic analysis of students’ distressing experience gests that the factors most frequently studied are
in online learning. They have demonstrated those related to users’ perspectives on technol-
how technical difficulties and communication ogy-mediated learning (Kerr and Rynearson,
breakdowns emerged as significant factors that 2006). Research in this area focuses on attitudes
actually impede learning. In a recent study, the towards the application of ICT in learning and on
role of technical support in e-learning has been perceptions, opinions, and preferences regarding
demonstrated (Ngai & Poon, 2007). learning technology. Within studies of students’
The instructional design of the technological perspectives on e-learning, perceptual and atti-
environment is frequently cited as a major factor tudinal variables are used to measure the effec-
affecting the ways in which learning technologies tiveness of the technology (Phipps & Merisotis,
are adopted and used (Penuel & Roschelle, 1999). 1999) and as indicators of learning outcomes.
Information structures (Potelle & Rouet, 2003) For example, Webster and Hackly (1997) write
and the nature of online learning tasks are but a “we suggest that attitudes towards a technology,

the perceived usefulness of the technology, and in learning, and whether learning is easier in
attitudes towards distance learning should be online environments. The results indicate that a
included as important learning outcomes” (p. computer-enriched environment is positively cor-
1284). Waxman, Lin, and Michko (2003) suggest related with student attitudes toward computers in
measuring perceptions and attitudes as indica- general, their role in teaching and learning, and
tors for affective outcomes and distinguish them their ability to facilitate communication.
from both cognitive and behavioural outcomes. Overall, research has reported that students
In other studies, however, the perspectives of hold positive attitudes towards the application
the users are taken to represent factors that may of ICT in learning (Phipps & Motistis, 1999).
bear upon the consequent adoption and use of the Favourable attitudes have been found across many
technology. For instance, student attitudes are student populations, at all levels of education and
considered a prominent motivational factor in training, and across different cultures (Mitra,
learning; therefore, positive attitudes may often 1997, 2000; Sanders & Morrison-Shetlar, 2001;
accompany effective learning (Ayersman, 1996). Selwyn, 1999). Previous research has highlighted a
The first approach seems to conflate outcomes and range of factors influencing user attitudes towards
perceptions and in so doing blurs the distinction computers and e-learning. Personal factors affect-
between learning outcomes and their potential ing attitudes such as self-efficacy (Dusick, 1998;
cause. The second approach appears more coher- Liaw, 2002) and demographic characteristics were
ent in so far as this distinction is concerned. This explored in relation to user attitudes towards LT
body of the literature will now be reviewed. (Selwyn, 1999). Although overall attitudes were
User attitudes are seen as influencing not only found to be consistently favourable, research into
the initial acceptance of IT but also the future the factors influencing them has reported mixed
behaviour regarding the use of computers. Thus, findings (Sanders & Morrison-Shetlar, 2001).
student attitudes towards technology form a User perceptions regarding learning technol-
fundamental basis for both participation and sub- ogy have been explored in numerous studies.
sequent achievements in e-learning (Liaw, 2002; Drawing on Rogers’ Diffusion of Innovation
Selwyn, 1999). In measuring and assessing atti- Theory (Rogers, 1995), Omalley and McCraw
tudes, different studies have applied the computer (1999) explored user perceived effectiveness of
attitude scale (CAS). The CAS (Selwyn, 1999), online learning. In their analysis, facets of per-
based on Davis’ (1989) technology acceptance ceived characteristics of e-learning included rela-
model (TAM), consists of four subscales: anxiety tive advantage, course and student compatibility,
related to using computers, perceived control when grades, and schedule. Research findings indicate
using a computer, perceived usefulness of using that students perceive that online learning has a
the computer, and behavioural attitudes towards significant relative advantage compared to tradi-
using a computer. This model has been applied tional methods. These advantages include saving
in various studies of learning technology users them time, fitting in better with their schedules,
(e.g., Dusik, 1998; Selim, 2003). Mitra and Stef- and enabling them to take more courses. However,
fensmeier (2000) examined pedagogic usefulness students do not believe that they learn more in
of the computer by focusing on student attitudes. online learning courses. Interestingly, students
Categories of attitude included: user comfort with seem to be ambivalent when comparing online to
computers, apprehension regarding the use of traditional methodologies. They prefer traditional
computers, the effect of online learning on com- courses although they want more online courses
munication with instructors, general preferences (Omalley & McCraw, 1999). Another study in-
for e-learning, the effect of e-learning on workload vestigating students’ perspectives on TML has

suggested that although the majority of students summary and critical remarks
taking traditional courses favour online courses,
they are less likely to enrol in them. However, A growing body of research has concentrated on
the majority of students taking online courses factors that enhance or inhibit the adoption and
find that such courses meet their academic needs use of learning technology. Some studies have
and improve their technological skills (Leonard focused on factors such as the strategic approach
& Guha, 2001). In a study on the effect of stu- of the university towards online learning (Boyd-
dents’ perceptions of their receptivity towards Barret, 2000) or the development of appropriate
TML, a “distance learning receptivity model” competencies and roles within the institution
was examined (Christensen, Anakwe, & Kessler, (Williams, 2003). The factors of technology
2001). In addition to overall attitudes towards availability and access were considered as were
LT and various demographic characteristics and technical stability and reliability (Webster &
technology perceptions (perceived usefulness, Hackly, 1997). The inhibiting impact of techni-
technological familiarity and technological acces- cal difficulties and communication breakdown in
sibility), other perceived categories were explored using learning technology was highlighted (Hara
including reputation (of the lecturers involved, of & Kling, 2000). Instructional factors appear to
the programme and of the school), constraints be fundamental. Design issues and the role of the
(e.g., commuting time, work demands, family instructor are considered critical factors influenc-
responsibilities), and learning preferences percep- ing students’ participation and engagement in
tion (towards traditional learning). The results technology-mediated learning (Tu, 2000).
reveal significant relationships between many of The most frequently studied factors relate to
these variables and LT receptivity. Findings also students’ perceptions on learning technology. A
indicate that some traditionally held assumptions, more detailed review was therefore provided of the
for example those regarding accessibility, reputa- attitudinal and perceptual factors studied to date.
tion, and constraints, may not be valid in the new A critique of the literature on “user perspectives,”
high-tech learning environment (Christensen et however, concerns the tendency to study percep-
al., 2001). tions and beliefs as isolated constructs, detached
Research exploring opinions shared by stu- from action. Studies of users’ perceptions seem
dents on issues concerning the application of to imply a straightforward, causal relationship
technology to course instruction resulted in an between perceptions (e.g., assumptions about
opinion typology. Three opinion types were identi- the technology) and action (i.e., actual use of the
fied: (1) time and structure in learning (i.e., flexible technology). Perceptions are therefore examined
time management that requires self-discipline), and measured within and among themselves. Yet
(2) social interaction in learning (i.e., individual the ways in which perceptions serve to guide
work leading to less enrichment from others) and people’s actions may otherwise be viewed as more
(3) convenience (i.e., commuting factors—time complex and thorny. For example, Picciano (2002)
and cost—less interference with work) (Valenta, points out that much of the literature is based on
Theriault, Dieter, & Mrtek 2001). Similar results students’ perceptions of the quality and quantity
were recently reported by Song, Singleton, Hill, of their interaction and performance. He suggests
and Koh (2004), indicating that the main factors going beyond student perceptions to explore actual
perceived by students as influencing successful interaction and performance. Findings indicate
online learning include time management and that while positive relationships between percep-
perceived lack of sense of community. tions of interaction and perceived performance
persist, the relationship between actual interac-
00
tion (defined by actual postings on discussion the sIgnIFIcAnce oF

boards) and actual performance measurements InstItutIonAL context In
(designed to measure specific course objectives) e-LeArnIng reseArch
are mixed and inconsistent (Picciano, 2002). It
remains the case, however, that despite obvious The significance of contextual factors in the adop-
difficulties, the most frequent research regarding tion and use of information and communication
students is the assessment of their attitudes and technologies (ICTs), although neglected in the
perceptions towards e-learning (Nachmias, 2002). e-learning discourse, has been demonstrated
Nachmias proposes that “it may well be that the widely in the case of other information systems
ease of data collection regarding these variables (Avgerou, 2001; Avgerou & Madon, 2004). Vari-
is what gives them the broad attention of the ous studies have repeatedly shown how similar
research community” (p. 219). Furthermore, the technologies yielded different results in difference
tendency to explore user perceptions and beliefs organisations, thus illuminating the crucial role
as standalone, independent constructs circum- played by contextual particularities in shaping the
vents the potential role played by other sources use of technology and its consequences (Robey
of influence and represents a decontextualised & Bourdreau, 1999).
notion of the user (Lamb & Kling, 2003). While The significance of studying institutional
opinions and beliefs held by users may well guide factors in e-learning stems not only from the
their choices to act, these are influenced by and role they are likely to play, as shown in imple-
dependent upon conditions and circumstances mentation cases of other ICTs, but more crucially
other than individual perceptions. Technologi- in the light of the prevailing mode of hybrid e-
cal and socio-organisational properties are key learning. “Hybrid” (Cookson, 2002) or “blended”
elements that are similarly and interdependently (Ginns & Ellis, 2007) modes of implementation,
drawn upon in continuous uses of technology refers to learning technology integration into
(Orlikowski, 2000). traditional on-campus education. Since hybrid
In summation, the literature overview provided implementation works within the physical en-
above demonstrates that a variety of factors influ- vironment of the university, and since learning
encing the adoption and use of learning technology technology is meant to complement, rather than
have been explored, with attention mostly given to replace the existing system, the role played by
to those of user attitudes and perceptions. Yet, the pre-existing institutional context becomes
these factors are typically studied in isolation and all the more important. Furthermore, many of
contextual factors associated with the institutional the controversial issues surrounding the highly
setting of e-learning are effectively ignored. It is researched topic of distance learning become
the aim of this study to address this gap in the less critical. Different opportunities, challenges,
e-learning literature. In the section that follows, and concerns are brought to the fore, calling for a
a case is made for the significance of contextual new research agenda (Nachmias, 2002). Although
factors, so that a better understanding of learning some initial attempts can be cited (McDonald
technology adoption and use can be achieved. In & Mcateer, 2003; Wu & Hiltz, 2004), hybrid
particular, attention is paid to the institutional e-learning largely remains an under-researched
context of higher education as the backdrop of phenomenon requiring further exploration. The
the empirical findings presented in the subsequent importance of research in this area is highlighted
part of this chapter. by the growing pervasiveness and anticipated
growth of this integrated mode across the higher
0
education sector (Allen & Seaman, 2004). As and the myriad hybrid models deployed, further
Garrison and Kanuka (2004, p. 104) conclude “it point to the potential influence of particular
is essential that researchers begin to explore the socio-organisational elements on the actual use
impact of blended learning.” of the technology. It is therefore suggested that a
For exploring hybrid e-learning, the institution systematic analysis of the context within which
into which e-learning is introduced cannot be seen learning technology is implemented and used is
to represent a plain variable. Indeed, criticism has essential for understanding both the processes
been aired against the over generalised and over and consequences of e-learning. The research
standardised assumptions about the character of reported in the next part of the chapter makes a
“universities” and “traditional learning” prevail- step in this direction, seeking to increase under-
ing in the literature (Ehrmann, 1995; Saba, 1999). standing of contextual considerations involved in
Ehrmann (1995) points to the mechanical concep- e-learning practices. It does so by eliciting a set
tion underlying comparative studies of technol- of institutional and socio-organisational factors
ogy-based methods vs. traditional methods. These emerging from an empirical analysis of LMS
studies assume that higher education operates implementation in the hybrid mode.
like a machine, and that each college is a slightly
different version of the same ideal machine. The
phrase “traditional methods” is used to represent PArt II: FIndIngs And
some widely practiced method that presumably ImPLIcAtIons
has predictable, acceptable results. Yet “traditional
methods” do not define the higher education that contextuAL FActors And
the research reveals. In fact, university learning e-LeArnIng PrActIces
is not so well-structured, uniform, or stable that
one can simply compare an innovation against Research findings presented in the following
traditional processes. Ehrmann refers back to the sections demonstrate the ways in which a set of
term “organised anarchy” coined by Cohen and institutional factors were drawn in the forma-
March (1974) to describe how higher education tion and shaping of e-learning practices. In this
institutions function. A variety of inconsistent context, e-learning practices are defined as the
goals, unclear methods and processes, and uncer- shared and recurrent activities that emerge from
tain organisational boundaries seem to capture learners’ continuous interaction with learning
both colleges and their courses (Ehrmann, 1995). technology.
The difficulty of talking about “universities” in Findings and illustrations draw on a longitu-
general was likewise acknowledged by Brown dinal study into the structuring of technology-
and Duguid (1998), stating that “the menagerie mediated learning practices in higher education
has many beasts and several species” (p. 5). Both (Halperin, 2005). The empirical setting of the
Saba (1999) and Ehrmann (1995) conclude that study involved the use of a standard LMS, namely,
the search for global answers about the compara- WebCT™ in a “traditional,” well established
tive effectiveness of technology is fundamentally university located in the heart of London, UK.
useless. Discussion of broad concepts related to Focus was placed on the integration efforts of the
traditional education or face-to-face education LMS into the provision of a masters degree in a
are vague and do not help the current discourse faculty of social science. The student body used
to shed any light on the subject of learning. as the research sample included 127 students in
The apparent diversification of learning total. The demographic features of the students
technology implementation in higher education reflected a fairly diverse collective. While some
0
similarities were apparent in terms of formal edu- institutional norms suggests that practices en-
cation acquired and, presumably, socio-economic acted online are anchored in, and conditioned by,
background, the cultural and national diversity long-established standards and regulations. An
had a strong presence. As for gender distribution illustrative example concerns the way in which
among students, two thirds of them were female institutional conventions of assessment served to
and one third male. shape e-learning practices. First, the pre-existing
Several data collection tools were employed assessment framework was thought of as a way
in the research, including in-depth interviews, to reinforce new practices and to endorse their
informal conversations, documents, off-line integration into the learning process. Organisa-
observations, and online observations (through tional “rules” were drawn upon in an attempt to
logons, tracking utilities, log files and compiled strengthen and institutionalise the online prac-
transcripts of computer-meditated communica- tice. Consequent efforts to establish the practice
tion [CMC] discussion messages). The use of through formal assessment required compliance
various data collection tools concurs with Yin with a set of institutional conventions and related
(1984) who advocates the use of multiple sources procedures, which in turn served to structure the
of information in conducting case studies. Data e-learning practice. In the case study institution,
collection encompassed three consecutive years, strong emphasis is placed on standardised and
starting from the point at which the technology “objective” assessment. Thus, introducing assess-
was first introduced in the institution. A research ment to online activities required the approval of
design was devised so as to guide a systematic a school-wide committee and the assurance of
examination of the organisational context, rely- conformity with pre-set criteria thereby aligning
ing on qualitative analysis methods. Relevant online outputs with institutional conventions.
institutional levels were mapped out and anal- Once the formal assessment of online activi-
ysed as interconnected layers (Pettigrew, 1990), ties was put into place, a new e-learning practice
and included the off-line course (the traditional, emerged. This distinct practice may be termed
face-to-face elements of the taught course), the discourse and is differentiated from other knowl-
programme, the department, and the university. edge sharing practices administered online such
A set of institutional and socio-organisational as information exchange, as discussed below.
factors impinging on e-learning have emerged The practice of discourse relied on the discussion
from the case analysis. Factors found to figure module of the LMS which was used to make public
predominantly include institutional conven- well developed statements regarding various top-
tions and standards, institutional activities and ics studied in the course. Students had referred to
routines, organisational resources (physical, this activity as a “mini-essay” and conceived it
technological—other than LMS—and human), to be an individual output for assessment rather
and organisational culture and social relations. than an integral part of an online discussion which
These are discussed and illustrated in turn. was the original idea behind it. As students com-
mented in interviews:
institutional Conventions and
standards …at some point it became a series of statements
just to prove how intelligent you were and it
Findings indicate that pre-existing institutional became very difficult to answer or reply to these
conventions and associated procedures were statements. Sort of mini essays that people were
drawn into the formation of e-learning practices. posting …
Systematic analysis of both formal and informal
0
…I noticed that people write those long spell- routines. For example, the weekly routine of
checked mini essays, which doesn’t really allow recurrent face-to-face lectures is reflected in the
discussion. way in which information is organised within the
system’s content module. While online content
Thus, the institutionalisation of the practice could otherwise be organised according to any
by means of formal assessment meant that po- chosen logic (e.g., vertical or thematic), a linear,
tentially innovative e-learning practice became week-by-week logic served as the organising
conventional. principle underlying content release and presen-
tation. Similarly, the temporal pattern of online
Pre-Existing activities and Routines interaction concurred with off-line time cycles
in so far as “term time” and “vacation time” are
Findings further suggest a strong linkage between concerned. The analysis clearly suggests that
pre-existing activities and routines enacted off- configuration of time in online practices was
line and the new online learning practices which underpinned by the pre-existing temporal profile
emerged through the continuous use of the tech- of the off-line activities.
nology. There is evidence to suggest that distinct
e-learning practices are intertwined within key organisational resources
components of the off-line course such as lectures
and face to face seminars. One example involves Characteristics of the organisational environment,
an e-learning practice which may be termed specifically the nature of campus resources, were
knowledge presenting. Relying on the students found to influence the adoption and use of the
presentation model of the LMS, the technology technology by both promoting and inhibiting
in this practice was used recurrently to present integration efforts. Organisational resources in
knowledge on given learning topics. Structured this context may be discussed in terms of physical,
activities in this context included preparation and technological, and human resources.
publishing of Web-based presentations pertaining Physical resources and features of the mate-
to various themes studied in the course. From rial environment of the organisation emerged as
the beginning of the course, a weekly-based prominent characteristics motivating the adoption
routine for publishing online presentations was of the technology. As mentioned earlier, the field
set up through the LMS. Presentations were to organisation is located at the heart of a metropolis
be uploaded regularly by a given time: on the where real estate prices are remarkably high. Space
day before the lecture and seminars. A topic and is therefore a scarce resource and the campus is
a set of articles were given each week. Students exceptionally crowded. Resources for students
were to provide a brief summary of the read- such as study rooms are limited and, in general,
ing followed by questions and criticisms, using poorly maintained. Under these conditions, and
multimedia options in their presentations. The given that no residence is available on campus,
e-learning practice of knowledge presenting was students typically preferred to work outside the
thus designed to support off-line seminar discus- university and tended to rely on remote access
sions. It was the routines and conventions of the through the LMS. As more resources became
traditional seminar that gave rise to the practice available online, the use of the system increased,
and served to shape its pattern. allowing students to move away from using cam-
Other facets of the online environment and its pus facilities such as the physical library. This was
consequent exploitation were clearly rooted within especially evident in the enactment of individual-
pre-existing and long established organisational productivity practice that characterises e-learning
0
practice in which students used the LMS to obtain as a major benefit. As one student commented in
various resources and content related to the taught an interview: “people like Naomi Klein wouldn’t
course. Through this practice, frustration with come to Hamburg.” Hence, the unique opportu-
inconvenient work conditions could be relieved. nity to participate in non-mediated interactions
Hence, the online learning environment is seen on campus hindered students’ motivation and
to compensate and complement poor and inef- interest in online interaction. For similar reasons,
ficient resources of the physical environment of students did not exploit specific online resources
the institution, and in so doing, its adoption and such as recorded talks of guest lecturers. In this
use is motivated and accelerated. respect, the resources and opportunities offered
Technological resources other than WebCT on campus are seen to compete with their online
have been brought to bear on the learning tech- counterparts and raise questions about the added
nology adoption and its subsequent integration. value provided by the technology within the par-
More specifically, a number of systems were ticular organisational context.
placed at the disposal of the students, such as
mail servers, public folders, an online administra- organisational cultural and social
tion system, and a digital library. While each of relations
these systems meets distinct requirements, when
designed functionalities are compared with the Other contextual characteristics influencing the
LMS some similarities are evident. For example, adoption and use of the technology stem from
an e-mail application is provided as a module socio-cultural features of the organisation. A
within WebCT. Yet this component was never prominent feature of the institution studied con-
exploited, as all users opted for the dedicated cerns the national and cultural diversity of its
exchange server of the organisation. Although student body. According to the graduate school
WebCT suggests itself as a definitive, all-purpose prospectus, the institution has attracted students
learning environment, comparable technolo- from 130 countries worldwide. As the director
gies implemented in the organisation appear to states (graduate school prospectus, p. 8) “the
delimit its role by providing viable alternatives. [school] is global in outlook and cosmopolitan
The fact that other, parallel systems were used in in character.” Figure 1 indicates the distribution
the organisation explains why certain modules of of graduate students by domicile.
the LMS were disregarded. It also sheds light on The cosmopolitan character of the student
some antagonistic attitudes towards the technol- body was drawn on in the learning practice and
ogy since students were required to learn and to stimulated specific activities, as is evident in the
manage several systems at once. As this burden case of an e-learning practice termed information
appeared only partly justified; consequent issues exchange. In this practice, the discussion module
of motivation become unavoidable. of the LMS was used recurrently to exchange
Being a leading academic institution located information about learning content. In particular,
in a capital city, the case study institution hosts students exchanged relevant information concern-
highly valued human resources. The university ing data of their own countries. In this way their
attracts renowned scholars and high profile in- knowledge of different languages and familiarity
dividuals from the social and political arena to with different national contexts served substantive
give talks and to take part in public debates. The aspects of the course and gave rise to a structured
intellectually-charged atmosphere of the campus learning activity online.
and the opportunity it provides to participate in a Yet, the cultural diversity has also hindered
range of events was acknowledged by the students motivation for online interaction among students
0
Figure 1. Graduate students by domicile
1%
4%
15% Australia
33% south America
2% north America
Africa
17% Asia
uk
other europe
28%
who expressed greater interest in non-mediated was further challenged by the social relationships
interaction. As indicated above, the opportunity which have developed among the students on the
for collocated interaction on campus appeared programme. Students of the masters programme
compelling, valuable and unique. As one student under investigation have managed to create a vi-
explained: brant social life. In contrast to other programmes
of the same department, they remained a relatively
The thing that makes this programme good is small group in terms of student numbers and
actually the people. What I like most is the fact distinct in that students were to leave together for
that everyone in my class has such different per- another year of study in the USA as the programme
spectives. They come from different backgrounds involved collaboration of two universities (UK
and bring different ideas into the discussion…. and the U.S.). These features of the programme
I sit there and I’m this capitalist American and may explain the formation of more tightly coupled
XXX[name of student] from Ukraine disagrees social relationships among students. Given the
with me…but now the perspective she represents frequent occasions for off-line interaction, online
becomes real…and it makes me much less dismis- communication seemed redundant and its rel-
sive than I used to be, which is good!...engaging evance was at times contested. Several students
like this with people that have experienced…things have commented that the technology “feels like an
that they lived through…so it’s not anymore some- artificial construct” and that “it creates distance
thing theoretical that is out there, it is something where there is none.”
that is right in front of you…so you take it more While students had frequently challenged
seriously … the benefits of asynchronous communication,
its inherent ability to overcome time constraints
There is evidence to suggest that students were still made it seem a valuable means of interaction.
inclined towards face-to-face interaction over Synchronous CMC, on the other hand, although
remote, online interaction, especially given the technically provided within specific WebCT
opportunity to engage with a mixed and diverse modules, was entirely ignored. Bearing in mind
community of peers in a collocated setting. The the social circumstances, the opportunity and
motivation to participate in online interaction the preference for collocated interaction, medi-
0
ated interaction in real time appeared all the Findings arising from the study indicate that
more redundant and artificial. For these reasons, the adoption and use of learning technology is
synchronous CMC options remained unexploited strongly influenced by the socio-organisational
and removed from the learning practice. environment surrounding it. More specifically,
Finally, the influence of the learning culture of institutional factors are shown to play a vital
the organisation prevailed in specific e-learning role in the formation and shaping of e-learning
practices. In particular, the individualistic culture practices within the context of LMS use in higher
and the competitive atmosphere of the institution education.
were reflected in individual-productivity and in Organisational factors found to figure pre-
discourse practices as described above. In these dominantly include institutional conventions and
cases, the use of the technology served to sup- standards, pre-existing activities and routines,
port individual efforts and achievements and existing resources available to the institution, and
coincided with the dominant learning culture of finally, the institution’s organisational culture.
the organisation. This culture manifests itself in These factors have particular relevance in the
the perception of students and lecturers, and is context of hybrid modes of e-learning implemen-
embodied in formal institutional documents as tation, as they illuminate the tensions involved
shown below: in integrating technological innovation into an
established system.
At XXX [name of institution], we believe you Further analysis of the emerging socio-or-
should be largely responsible for organising your ganisational parameters, demonstrates the ways in
own work and meeting the requirements of the which these factors can both promote and inhibit
programme. Although support with your studies the integration of LMS. The important role played
is always at hand if required, a strong emphasis by an array of organisational properties denotes
is placed on self-reliance. You will spend the the institutional embeddedness of the e-learning
majority of your time on your own work rather practice. Since institutions of higher education
than with formal instruction. There is sufficient can and do exhibit diversity in terms of their
time in your schedule for reading and reflection. socio-organisational characteristics (Ermann,
(Graduate Prospectus, p. 8) 1995), efforts to integrate learning technology
across academic organisations should expect to
Although collaborative e-learning practices encounter inconsistent and contradictory conse-
have emerged, these were typically associated quences (Robey & Bourdreau, 1999).
with sharing and exchange rather than with team Within hybrid models of e-learning implemen-
efforts or collaborative tasks which remained, by tation, the role played by pre-existing rules and
and large, individual-based. resources becomes all the more significant as the
technological environment is meant to comple-
ment, rather than replace, the existing and long
concLusIon established learning system. The analysis suggests
that, in cases where technology was introduced
The aim of the study reported in this chapter was to supplement existing arrangements, that is, to
to explore institutional and socio-organisational compensate for deficiencies affecting the existing
factors involved in the adoption and use of learn- off-line setting, the integration process was typi-
ing technology in higher education. In so doing, cally vigorous and accelerated. Clearly, difficulties
the study attempted to address a perceptible gap and challenges also arose as the online system was
in the current e-learning literature. seen to compete or clash with its veteran off-line
0
counterpart. In some cases, interoperation and dIrectIons For Further

fusion were achieved through negotiation; in reseArch
others, technological properties were ruled out
and capabilities remained unexploited. This chapter has presented preliminary findings of
Understanding the ways in which socio-or- an exploratory research study into the institutional
ganisational factors impinge on LMS use bears factors involved in e-learning. The lack of previous
practical implications. Rather than be driven by research on this topic, and hence, the exploratory
the technological capabilities and the features nature of the study, suggest that further research
available, LMS implementation efforts and course is called for in order to achieve a more rounded
design should take account of the contextual understanding of the role played by institutional
particularities associated with the educational factors in the formation of e-learning practices. In
institution in question. Particular attention should particular, further research may enable validation
be paid to the strengths and weaknesses of the of the results across cases. While the institutional
organisation as they may be viewed from the factors identified stemmed from a longitudinal
students’ point of view. This is especially rel- in-depth analysis, a single case study design was
evant in hybrid e-learning projects, where the applied thus suggesting limited generalisability.
advantages of both learning systems—online Further research may also discover institutional
and off-line—ought to be realised. As the case factors in addition to the ones reported here, and
study illustrates, the implementation process of in so doing extend the knowledge on this appar-
the LMS was accelerated when it compensated ently important topic.
for deficiencies apparent in the physical system While any mode of e-learning application is
on campus. This was evident in, for example, the embedded within some institution or broader
case of space and physical resources available for systems surrounding it, the relationship between
the students. At the same time, conflict between the existing institutions, or the off-line environ-
the systems is expected if the LMS competes ment, with the online learning environment is
with the perceived strength of the “off-line” particularly relevant in the prevailing mode of
learning environment. Of main concern here is hybrid or blended e-learning. This is so because,
the tension arising between virtual and collocated in b-learning, traditional off-line learning and in-
interaction, which directs course design efforts novative online learning are deliberately mixed
to offer complementary online and face-to-face with one another. To better understand this crucial
communication in the learning practice. relationship, the study described in this chapter has
Evident in this hybrid e-learning case study sought to address the ways in which pre-existing
is the dominant influence of the “traditional,” institutional factors influence emerging online
off-line learning system and its methods on the practices. Yet, further research is warranted on
new online practices. An illustrative example the reverse affects, that is, on the ways in which
discussed in this chapter is the role played by online learning practices influence off-line prac-
traditional assessment in structuring e-learning tices and traditional routines in learning.
practices. While established, traditional methods
may prove powerful in reinforcing the use of the
technology, they might also stand in the way of reFerences
innovative learning practice and, in so doing,
undermine the original aim of implementing the Allen, I. E., & Seaman, J. (2004). Sizing the
technology. opportunity: The quality and extent of online
education in the US, 2002 and 2003. Needham,
MA: Sloan-C.
0
Avgerou, C. (2001). The significance of context in Dusick, D. (1998). What social cognitive factors
information systems and organizational change. influence faculty members’ use of computers for
Information Systems Journal, 11, 43-63. teaching. A literature review. Journal of Research
on Computing in Education, 31(2), 123-137.
Avgerou, C., & Madon, S. (2004). Framing IS
studies: Understanding the social context of IS Ehrmann, S. C. (1995). Asking the right question:
innovation. In C. Avgerou, C. U. Cibbora & F. F. What does research tell us about technology and
Land (Eds.), The social study of ICT (pp. 162-182). higher learning? Change, 17(2), 20-27.
Oxford: Oxford University Press.
Garrison, D. R., & Kanuka, H. (2004). Blended
Ayersman, D. J. (1996). Reviewing the research on learning: Uncovering its transformative potential
hypermedia-based learning. Journal of Research in higher education. Internet and Higher Educa-
on Computing in Education, 28(4), 501-525. tion, 7(2), 95-105.
Boyd-Barrett, O. (2000). Distance education pro- Ginns, P., & Ellis, R. (2007). Quality in blended
vision by universities: How institutional context learning: Exploring the relationships between
affect choices. Information Communication & online and face-to-face teaching and learning.
Society, 3(4), 474-493. Internet and Higher Education, 10(1), 53-64.
Brown, J. S., & Duguid, P. (1998). Universities Guldberg, K., & Pilkington, R. (2007). Tutor
in the digital age. In B. L. Hawkins & P. Battin roles in facilitating reflection on practice through
(Eds.), The mirage of continuity: Reconfiguring online discussion. Educational Technology and
academic information resources for the 21st Society, 10(1), 61-72.
century (pp. 39-60). Washington, DC: Council
Halperin, R. (2005). Learning technology in
on Library and Information Resources.
higher education: A structurational perspec-
Chiero, T. C. (1997). Teachers’ perspectives on fac- tive on technology-mediated learning practices
tors that affect computer use. Journal of Research (Doctoral dissertation). London: London School
on Computing in Education, 30(2), 133-145. of Economics.
Christensen, E. W., Anakwe, U. P., & Kessler, Hara, N., & Kling, R. (2000). Student distress
E. H. (2001). Receptivity to distance learning: in a Web-based distance education course. In-
The effect of technology, reputation, constraints, formation, Communication and Society, 3(4),
and learning preferences. Journal of Research on 556-579.
Computing in Education, 33(3), 263-370.
Kerr, M. S., & Rynearson, R. (2006). Student
Cohen, M. D., & March, J. D. (1974). Leadership characteristics for online learning success. Inter-
and ambiguity: The American college president. net and Higher Education, 9, 91-105.
New York: McGraw-Hill.
Leonard, J., & Guha, S. (2001). Students’ perspec-
Cookson, P. (2002). The hybridization of higher tives on distance learning. Journal of Research
education. International Review of Research in on Technology in Education, 34(1).
Open and Distance Learning, 2(2), 1-4.
Liaw, S. S. (2002). Understanding user perceptions
Davis, F. D. (1989). Perceived usefulness, per- of WWW environments. Journal of Computer
ceived ease of use and user acceptance of infor- Assisted Learning, 18, 1-12.
mation technology. Management Information
Systems Quarterly, 13, 319-340.
0
Mazzolini, M., & Maddison, S. (2003). Sage, guide Phipps, R., & Merisotis, J. (1999). What's the dif-
or ghost? The effects of instructor intervention on ference? A review of contemporary research on
student participation in online discussion forums. the effectiveness of distance learning in higher
Computers and Education, 40, 237-253. education. Washington, DC: The Institute for
Higher Education Policy.
McDonald, J., & Mcateer, E. (2003). New ap-
proaches to supporting students: Strategies for Picciano, A. (2002). Beyond student perceptions:
blended learning in distance and campus based Issues of interaction, presence, and performance
environments. Journal of Educational Media, in online course. Journal of Asynchronous Learn-
28(2-3), 129-146. ing Networks, 6(1), 21-40.
Mitra, A., & Steffensmeier, T. (2000). Change Potelle, H., & Rouet, J. F. (2003). Effects of content
in student attitudes and student computer use representation and readers’ prior knowledge on the
in a computer-enriched environment. Journal comprehension of hypertext. International Jour-
of Research on Computing in Education, 32(3), nal of Human-Computer Studies, 58, 327-345.
417-431.
Robey, D., & Bourdreau, M. (1999). Accounting
Nachmias, R. (2002). A research framework for for the contradictory organizational consequences
the study of a campus-wide Web-based academic of information technology: Theoretical directions
instruction project. Internet and Higher Educa- and methodological implications. Information
tion, 5(3), 213-229. Systems Research, 10(2), 167-185.
Ngai, E., & Poon, J. (2007). Empirical examination Rogers, E. M. (1995). Diffusion of innovations.
of the adoption of WebCT using TAM. Computers New York: Free Press.
and Education, 42(2), 250-267.
Saba, F. (1999). Is distance education comparable
Oliver, M., & Shaw, G. P. (2003). Asynchronous to traditional Education? Retrieved October 19,
discussion in support of medical education. 2007, from http://www.distance-educator.com/
Journal of Asynchronous Learning Networks, der/comparable.html
7(1), 56-67.
Salmon, G. (2000). E-moderating: The key to
Omalley, J., & McCraw, H. (1999). Student percep- teaching and learning online. London: Kogan
tions of distance learning, online learning and the Page.
traditional classroom. Online Journal of Distance
Sanders, D., & Morrison-Shetlar, A. I. (2001).
Learning Administration, 2(4), 1-16.
Student attitudes towards Web-enhanced instruc-
Orlikowski, W. J. (2000). Using technology and tion in an introductory biology course. Journal
constituting structure: A practice lens for study- of Research on Computing in Education, 33(3),
ing technology in organizations. Organizational 251-262.
Science, 11(4), 404-428.
Selim, H. M. (2003). An empirical investigation
Penuel, B., & Roschelle, J. (1999). Designing of student acceptance of course Web sites. Com-
learning: Cognitive science principles for the in- puters and Education, 40, 343-360.
novative organization. Stanford Research Institute
Selwyn, N. (1999). Students’ attitudes towards
International, 1-26.
computers in sixteen to nineteen education.
Pettigrew, A. (1990). Longitudinal field research Education and Information Technologies, 4(2),
on change: Theory and practice. Organization 129-141.
Science, 1(3), 267-291.
0
Song, L., Singleton, E. S., Hill, J. R., & Koh, M. H. AddItIonAL reAdIng
(2004). Improving online learning: Students per-
ceptions of useful and challenging characteristics.
Internet and Higher Education, 7(1), 59-70. Bullock, C., & Ory, J. (2000). Evaluating in-
structional technology implementation in higher
Tu, C. (2000). Critical examination of factors af-
education environments. American Journal of
fecting interaction on CMC. Journal of Network
Evaluation, 21(3), 315-328.
and Computer Applications, 23, 39-58.
Huynh, M. Q., Umesh, U. N., & Valacich, J. S.
Valenta, A., Theriault, D., Dieter, M., & Mrtek, R.
(2003). E-learning as an emerging entrepreneurial
(2001). Identifying student attitudes and learning
enterprise in universities and firms. Communica-
styles in distance education. Journal of Asynchro-
tions of the Association for Information Systems,
nous Learning Networks, 5(2), 111-127.
12, 48-68.
Waxman, H. C., Lin, M., & Michko, G. M.
Kim, K., & Bonk, C. J. (2002). Cross-cultural
(2003). A meta-analysis of the effectiveness of
comparisons of online collaboration. Journal of
teaching and learning with technology on stu-
Computer Mediated Communication, 8(1), 1-31.
dents outcomes. Naperville, IL: Learning Point
Associates. Kling, R., & Iacono, S. C. (1987). The institutional
character of computerised
Webster, J., & Hackley, P. (1997). Teaching ef-
fectiveness in technology mediated distance information systems. Office: Technology and
learning. Academy of Management Journal, People, 5(1), 7-28.
40(6), 1282-1309.
Lamb, R., & Kling, R. (2003). Reconseptualizing
Williams, P. E. (2003). Roles and competencies for users as social actors. Management Information
distance education programs in higher education Systems Quarterly, 27(2), 197-235.
institutions. The American Journal of Distance
Laurillard, D. (2001). Rethinking university
Education, 17(1), 45-57.
teaching: A framework for the effective use of
Wu, D., & Hiltz, S. R. (2004). Predicting learning learning technologies (2nd ed.). London: Rout-
from asynchronous online discussions. Journal ledge Falmer.
of Asynchronous Learning Networks, 8(2), 139-
Olsen, G. M., & Olsen, J. S. (2000). Distance
152.
matters. Human-Computer Interaction, 15, 139-
Yin, R. K. (1984). Case study research: Design 178.
and methods. Thousand Oaks, CA: Sage.

Chapter VII
E-Learning Value and Student
Experiences:
A Case Study
Krassie Petrova
Auckland University of Technology, New Zealand
Rowena Sinclair
Auckland University of Technology, New Zealand
ABstrAct
This chapter focuses on understanding how the value of student learning and the student learning ex-
perience could be improved given pertinent environmental and academic constraints of an e-learning
case. Believing that a better understanding of student behaviour might help course design, the chapter
revisits the outcomes of two studies of e-learning and analyses them further using a framework which
conceptualises the value of e-learning from a stakeholder perspective. The main objective of the chap-
ter is to identify some of the important issues and trends related to the perceived e-learning value. The
analysis of the emerging and future trends indicates that in the future blending of e-learning and face-to-
face learning is likely to occur not only along the pedagogical, but also along the technological and the
organizational dimensions of e-learning. Therefore, new blended learning and teaching models should
emphasise further the alignment of learning with work/life balance.
E-Learning Value and Student Experiences
IntroductIon learning environment that promotes and supports

student success.
E-learning is used as a comprehensive term to The programme used in this case study is a
identify the use of a variety of information and typical three-year undergraduate programme.
communication technologies to enhance and sup- A cornerstone of its philosophy is to encourage
port learning, often blending their use. Online independent student led learning. Entrants to the
learning can be defined as an implementation of e- programme come from a range of backgrounds.
learning using Web-based technologies (Petrova, Due to ethnic diversity, some students might have
2007). Online learning and e-learning are used English as an alternative language and even full
as synonyms throughout the text. time students work long hours. E-learning was
Across universities worldwide, participants’ introduced in an attempt to alleviate some of these
engagement and achievement and the support problems. However there is evidence to suggest
provided by educational technology have become that the continuing effort involved in developing
the subject of intensive research, development, and and delivering e-learning courses may lead to a
discussion (Blinco, Mason, McLean, & Wilson, significant demand on academics’ time and in-
2004; Buzzetto-Moore & Pinhey, 2006; Kickul stitutional resources, as the amount of individual
& Kickul, 2006; Lee & Nguyen, 2005; Sharpe attention needed may “rival a one-to-one course”
& Benfield, 2005). The work presented here has (Tastle, White, & Shackleton, 2005, p. 249).
a specific focus on understanding how the qual- Since 1999, e-learning within the case study
ity of student learning and the student learning programme case has gradually developed into
experience could be improved whilst working two distinct teaching and learning models of
within environmental and academic constraints, Web-based online learning, known as “flexible
believing that a better understanding of student mode” and “enhanced mode.” Both models belong
behaviour might help course design. to the category of “hybrid” or “blended” learning
The main objective of the chapter is to identify (Mortera-Gutierrez, 2006; Petrova, 2001) as their
some of the important issues and trends related to delivery format combines face-to-face and online
the perceived value of e-learning. To this end, the teaching and learning.
outcomes of two studies of e-learning are revisited In enhanced mode, e-learning is used to
and analysed further using a framework, which complement (in-class) and enhance (off-cam-
conceptualises the value of e-learning. Current pus) the 3 hours per week classroom teaching
and emerging trends about the drivers of student by using the institutional e-learning platform
satisfaction are discussed and recommendations (BlackBoard™). E-learning activities include
are presented. exercises and demonstrations; off-campus they
are mostly used as a vehicle for questions and
answers about the course and assessment. As a
BAckground rule, in enhanced mode online activities are not
formally assessed.
E-learning was first introduced into the under- In flexible mode, a portion of the face-to-face
graduate business programme at the New Zea- teaching is replaced by the equivalent time in
land university used in this case study as early online activities, performed off-campus, in the
as 1999. However, since these early adoption students’ own time. Students are given detailed
days e-learning has become widespread across instructions about the e-learning activities they
the whole university and its importance is now are expected to engage in, and about the expected
recognised as a strategic approach to providing a outcomes. The “flexible” online activities may

be either individual or group, and often require More specifically Lizzio, et al. (2002) found
significant preparatory research. Typically, they that student perceptions of the teaching and
will have a fixed completion deadline, and may be learning environment and the assessment prac-
incorporated into the assessment programme. tice contribute to the development of deeper
The overall spread of e-learning in the case approaches to studying. They established that
study programme is relatively high: following their positive perceptions of the environment directly
specific study pathway, a typical undergraduate influence both measured academic outcomes, for
student might be engaged in e-learning in up to example, academic achievement and also qualita-
58% of their studies (Petrova & Sinclair, 2005). tive learning outcomes, workplace related skills.
This rather “massive” advent of e-learning has Other studies have highlighted usage patterns in
introduced a significant change to many aspects of terms of time, place, and functional components
the teaching and learning environment, including (Blinco et al., 2004; Burr & Spennemann, 2004;
stakeholder perceptions about its value. McKnight & Demers, 2002). An important point
made in the reviewed research studies and reports
inform the studies presented here: analysis of
e-LeArnIng vALue: students’ perceptions in conjunction with factual
stAkehoLder PersPectIves data can provide a valuable input to the processes
of curriculum development and management
Studies in the area of change processes and (Burr & Spennemann, 2004; Buzetto-More &
management related to the introduction of new Pinhey, 2006; Kickul, & Kickul, 2006; McKnight
educational technologies have found that students & Demers 2002; Sharpe & Benfield, 2005).
might be resistant to change. In an early article Two studies were carried out during the
on the use of information technology to enhance period 2003-2005, both investigating the case
education in business schools, Leidner and Jarv- study programme. Based on the assumption that
enpraa (1995) pointed out that there was a need to improving the scholarship of e-learning depends
understand better the role of students in learning on understanding stakeholders’ perspectives, the
models involving information technology, and overall research framework used in the studies
suggested that students would be “likely to resist (Figure 1) includes students as they interact with
the new learning models” (Leidner & Jarvenpaa, the e-learning platform in the context of courses
1995, p. 287). delivered online, while academics participate in
Students are identified as one of the recog- e-learning as course developers and implement-
nised stakeholder groups involved in e-learning, ers. E-learning is facilitated by the organizational
therefore any emerging organizational formats formats and structures of the university. The
developed to accommodate this educational work aimed to identify and explore criteria for
paradigm need to be managed carefully in order stakeholder evaluation of e-learning, to identify
to avoid early student disillusionment and the patterns of online platform usage, and to provide
subsequent failure of students to realise the full a basis for the understanding of student satisfac-
education potential of e-learning (Hunt, Thomas, tion with e-learning. Two research questions were
& Eagle, 2002; Sharpe & Benfield, 2005). Student investigated:
participation in e-learning and student perceptions
in particular have been the emphasis of research 1. What is the perceived value of e-learning
(Hisham, Campton, & FitzGerald, 2004; Lizzio, from a stakeholder perspective?
Wilson, & Simons, 2002; Phillimore, 2002; Swan, 2. Are students satisfied with e-learning and
1995; Wells, Fieger, & de Lange, 2005). what are the manifestations of satisfac-
tion?

In the first study, data were collected in 2003 A vALue FrAmework For
using an anonymous questionnaire distributed e-LeArnIng
to 44 academics, 6 managers, and 75 students.
Students and academics were selected from across The issue of “value” is central to the operation
the courses in the case programme. Managers of most organizations operating in a competitive
represented the organization at a senior level. environment. In the business sector “value” can
In the second study, data were collected from be many things, for example, offering valuable
two sources: the statistical reports provided by customer services. In the education sector the
BlackBoard™ (collated for the months of August, issue of value is broad, as there are so many
September, and October, 2004) and the responses stakeholders with differing viewpoints of what
to an anonymous questionnaire distributed to constitutes value.
students. The questionnaire was sent at the end In this market-driven education environment,
of 2004 to 730 participants in both “flexible” tertiary institutions need to establish their creden-
and “enhanced” courses (452 and 278 students tials within their niche market. Potential students
respectively). need a reliable indicator of value to enable them
Some of the findings of the two studies were to navigate the huge number of courses available
reported in more detail by Sinclair (2003b), and without falling victim to unlicensed “Web-cow-
by Petrova and Sinclair (2005). A framework for boy” operators and “digital diploma mills” (Hope,
further analysis of the issues emerging from the 2001) where the emphasis is on taking students’
findings of the studies is presented in the next money rather than on any real concern for the
section. value of the students’ learning. Poor value can be
reflected in students withdrawing from a course
Figure 1. E-learning and stakeholder framework (Adapted from Petrova & Sinclair, 2005)

or not attending. Word of mouth can result in (2002) has developed distance learning guidelines
one disaffected student telling 10 others of their at the request of the distance learning commu-
experience. This can mean a huge decrease in nity in the UK who recognised the importance
student enrolments as students take advantage of of having a code of practice to assure value in
courses from other tertiary institutions. the offered courses (Cavanaugh, 2002). In the
In the face of increasing costs tertiary institu- United States, the Institute for Higher Education
tions are looking at ways to decrease spending. Policy (2000) developed a list of 45 benchmarks
Online courses are less constrained by infra- and in Canada, the Canadian Association for
structure than face-to-face courses and have the Communication Education sponsored a project
potential to have a lower cost/student ratio. This to develop quality distance education guidelines
can lead to a tertiary institution’s decision to (Barker, 2002; FuturEd, 2002).
choose quantity over the perceived value of the Whilst these “solutions” to the issue of value
course (Heerema & Rogers, 2001). Institutions may be appropriate they have a weakness in
must realise that value should never be compro- that they may be focused on the needs of the
mised as in market-driven environments students accrediting organizations and the tertiary insti-
have the freedom of choice and will move if the tutions rather than on the needs of students and
value, in their eyes, deteriorates. It is interesting academics. The importance of a framework that
in all this discussion on value that McLoughlin looked at the perspectives of different stakehold-
and Luca (2001) consider that technology has yet ers was highlighted in 2002 when the Council for
to make significant improvements in the value of Higher Education Accreditation (2002) held an
education being offered. This possibly reflects international seminar where two of the three key
the current emphasis of online courses, which is speakers discussed the importance of a framework
to make education more accessible to students that ensured there was value in e-learning. These
and replicate, rather than improve, what is in the suggestions related to a proposal that accreditation
face-to-face courses. of higher education should be part of the General
If the perceived value of e-learning is lowered, Agreement on Trade in Services (GATS) of the
the credibility of the course will ultimately di- World Trade Organization (Council for Higher
minish in the eyes of employers when graduates Education Accreditation, 2002). This is a con-
cannot meet expected outcomes. This will lead cern, as there is the potential that international
students elsewhere, as they would want a qualifi- bureaucrats rather than the education sector of
cation that employers recognise. To ensure there each country would manage standards.
is value in the worldwide online courses, several Reflecting this concern, research was done
universities have joined global alliances such as to develop a framework that incorporated value
the Global University Alliance (2000) and the from three stakeholders groups’ opinions—aca-
World Alliance in Distance Education (2002). demics, students, and the management within the
These alliances have focused on ensuring value New Zealand University used in this case. Online
in online learning and in providing students with learning was looked at as a whole and there was
a wide variety of quality online courses that they no distinction made between the two models of
can access from different locations. e-learning, that is, flexible and enhanced. This
Another way in which tertiary institutions and lack of distinction could have affected the results,
accrediting organizations have tried to increase especially if students perceived that the flexible
the perceived value is by developing benchmarks component did not add value to the course. What
for online courses (Sinclair, 2003a). In the United could also affect results would be students who
Kingdom (UK) the Quality Assurance Agency did not consider that the flexible component of

their course was important. However, the next 2000; Western Interstate Commission for Higher
section which focuses on students’ perspectives Education, 2002).
will look at the two models of e-learning and Once these criteria were established, ques-
highlight any differences. tionnaires were developed for each of the three
Initially, a comprehensive list of criteria about stakeholder groups. Stakeholders were surveyed
what would be considered valuable in online using an anonymous questionnaire. For students
learning was developed from each stakeholder the response rate was 62% (47 questionnaires
group’s perspective. The criteria for academics returned), for academics the response rate was
was determined by a nominal group (Brahm & 32% (14 questionnaires returned), and for manag-
Kleiner, 1996) made up of experienced academ- ers the response rate was 83% (5 questionnaires
ics of online learning from different business returned). Stakeholders were asked to “rank” the
disciplines. A nominal group was considered importance of each criterion using the analytic
appropriate as it could generate ideas about value hierarchy process (AHP) scale developed by Saaty
criteria and then prioritise these ideas (Uribe, (1994). This 1-9 scale has 17 steps which sought to
Schweikhart, Pathak, Marsh, & Fraley, 2002). capture the sensitivity of criteria that were prefer-
To determine students’ criteria for value, various entially close to one another (Davies, 2001). Tul-
studies (Berman & Pape, 2001; Cashion & Palm- lous and Utecht (1994) considered that evaluating
ieri, 2000; Inglis, 1999; Lambert, 1996; Ponzurick, multiple criteria simultaneously amongst different
France, & Logar, 2000; Scott, 2001) were identified stakeholder groups was not an easy task. AHP
which looked at the different factors that made up was used to overcome this problem as it provided
online learning. To determine value criteria from a structure and procedure for incorporating dif-
the organisation’s perspective, the accreditation ferent stakeholders’ criteria. The results from the
requirements of various accrediting agencies and questionnaires were consolidated by stakeholder
tertiary institutions standards were looked at to into the top 10 criteria that would be essential for
determine and consider a list of value criteria valuable e-learning delivered courses.
(Barker, 2002; Distance Education and Training Students’ criteria focused on the materials,
Council, 2002; Institute for Higher Education teaching, and information available about the
Policy, 2000; Southern Regional Education Board, course. Cashion and Palmieri (2000) refer to
Table 1. E-learning value criteria (Adapted from Sinclair, 2003b)
1 Convenient and secure access to learning platform

2 Sufficient information supplied to students about the paper
3 Technology utilised is appropriate
4 Qualified academics
5 Qualified technical staff
6 Academics are prepared at start of the semester
7 Materials available at start of the semester
8 Materials easy to use
9 Materials up to date and accurate
10 Paper recognised by employers
11 Learning outcomes are appropriate for flexible delivery
12 Readings and activities provide academic challenge
13 Training on the platform available to students
14 Approaches to learning encourages active learning
15 Development of critical thinking skills by students
16 Academics provide clarity on flexible tasks.
17 Feedback by academic constructive

Figure 2. E-learning value framework (Derived from Sinclair, 2003b)
“learner readiness” but this research highlighted than their employability, the future. Students tend
the importance of “academic readiness,” that is, to undervalue the importance of critical thinking
the academic is prepared and materials are avail- and academic challenge over extending their
able at the start of the semester. Academics’ criteria knowledge, which would hopefully make them
covered the materials but also looked at the level more employable.
of learning that takes place, that is, considering Five distinct categories emerged from the list;
critical thinking more important than a straight accessibility, components, satisfaction, learning
recall of facts. The managers in the organization experience, and interaction. The subsequent clas-
were concerned with the materials and teaching sification of the criteria under the appropriate
but also that active learning takes place. category allowed a value framework for e-learn-
The top 10 criteria from each stakeholder ing to be constructed (Figure 2). This framework
group were merged into a common list of seven- will be used to measure the value of an online
teen criteria (Table 1). Fifteen of the criteria were course by analyzing the data which is relevant to
ranked high by all the stakeholders. Two criteria the components of the framework.
were not ranked highly by students, who did not The next section analyzes data collected from
consider that materials and approaches to learn- and about one of the stakeholder groups (students).
ing should provide an academic challenge. This The issues identified are aligned with the five
reflects the tendency of students to concentrate categories of the e-learning value framework
on passing the course, that is, the present, rather (Figure 2).

PerceIved vALue And overALL aligned with framework categories emerged from
student sAtIsFActIon wIth the findings (Figure 3).
e-LeArnIng
Issue 1: “Accessibility”
To investigate the student perspective, two sources
of data were used: the actual usage patterns of The first issue emerged from the investigation
the online platform were investigated in conjunc- of the time dimensions of the actual usage of
tion with a study of the perceived satisfaction the online platform. Online activities took place
of e-learning. Students were surveyed using an predominantly during the hours of the day, with
anonymous questionnaire. In enhanced mode, activities slowing down in the evening. The curve
the response rate was 71% (197 questionnaires for participants in flexible mode peaks later in the
returned), with 84% of the respondents regarding day compared with the curve for enhanced mode
themselves as full-time students and 50% gener- (Figure 4). The days from Monday to Thursday
ally inclined to prefer e-learning to face-to-face were characterised by heavier usage compared to
learning. In flexible mode the response rate was Friday to Sunday, with students in flexible mode
65% (294 questionnaires returned) with 85% of more active during the weekend (Figure 5).
the respondents regarding themselves as full-time As shown in more detail in Petrova and Sinclair
students and 52% preferring e-learning to face- (2005), there was very little variation in these
to-face learning. two patterns across the semester, or by course
The e-learning value framework proposed level. In summary, it appears that e-learning as
in the previous section (Figure 2) was applied undertaken in this case study is not too different
to analyse the data collected. Five main issues from face-to-face learning in terms of “when”
Figure 3. Issues with perceived value of e-learning (students)

Figure 4a. Average daily and hourly use of the online platform
Figure 4b. Average daily and hourly use of the online platform
it occurs. The 24/7 access and the possibility to line learning model: in enhanced mode the Content
study at any time may not be highly important as Area was the most commonly used feature while
most students may still prefer to “e-learn” at the in flexible mode the most used component was
same time as they would study normally. the Groups Area. The Discussion Board and to a
lesser extent the Announcements Area were used
Issue 2: “components” similarly in the two models (Figure 6).
When students were asked to indicate which
The top four most used components of Black- components they had used, both groups gave a
board™ were the Content Area, the Groups Area, similar percentage of “yes” answers for the An-
the Discussion Board, and the Announcements nouncements Area and the Content Area (Figure
Area. There were variations according to the on- 7). A greater number of “flexible” participants
0
Figure 6. Use of the online platform by component
Figure 7. Use of the online platform by component
had used the Discussion Board compared to “en- Board, and the Groups Area. There was less
hanced” participants (90% vs. 58%), and similarly emphasis on the use of online components aiming
the Groups Area (85% vs. 46%). In other words, at developing student capabilities. Students used
the value of the online platform according to BlackBoard™ predominantly for information
students is signified by its dual role as a course gathering. This pattern is fairly typical for courses
content organiser and a communication channel where delivery is enhanced with e-learning (Lee
between them and the academic. & Nguyen, 2005; Phillimore, 2002). In flexible
Data indicates that there are different compo- mode, there was a much greater use of e-learning
nent usage patterns within each mode. In enhanced for communicative activities, such as group work
mode the Content Area component was used and online discussions. A much wider range of
most followed by Announcements, Discussion capabilities are being addressed and the emphasis

on using the online platform for this purpose may questions shown in Table 2, which are used as
reflect the wide use of collaborative learning in general satisfaction indicators.
flexible mode. The overall level of student satisfaction with
It appears that the most valuable components e-learning is shown in Figure 8, with the values
of the learning platform (irrespective of the e- of the first two indicators above 50%. The two
learning model) are the ones providing storage, learning modes (enhanced and flexible) display
organization, and direct communication facilities. the same trend. However the graph highlights an
The pattern of use of the top four components issue: while most students will be happy to take
differs with respect to the e-learning model. another e-learning course (meaning that they
Components related to capability development see value in it), a lesser number are prepared to
may be more valuable to participants involved in recommend the course to a peer—especially in
flexible rather than in enhanced e-learning. enhanced mode (58%). The reluctance to declare
publicly that the course is of high value suggests
Issue 3: “student satisfaction” some uncertainty on the part of students as to
the benefits of e-learning. In other words, it can-
The survey instrument addressed student sat- not be concluded with confidence that students
isfaction with e-learning through the set of two are convinced e-learning is more beneficial than
face-to-face learning.
Table 2. General satisfaction indicators
Would you choose another [course] with a flexible (or enhanced) option? (Yes/No)
Would you recommend this [course] to another person based on its flexible (or enhanced)
mode of delivery? (Yes/No)
Figure 8. Overall student satisfaction and student experience indicators

Table 3. Student experience indicators
Statements ranked on a Likert scale from 1 (strongly disagree) to 5 (strongly agree)

So far my experiences with this course have been positive.
The online mode of this [course] met my expectations.
Table 4. Specific student satisfaction indicators
Statements ranked on a Likert scale from 1 (strongly disagree) to 5 (strongly agree)

Assessment tasks were supported by [BlackBoard™]
[BlackBoard™] supported communication between academics and students well
[BlackBoard™] helped me to keep up to date with changes, deadlines and notices.
[BlackBoard™] provided adequate storage for course materials.
[BlackBoard™] provided adequate additional course materials.
Issue 4: “student Learning in Table 4. Figure 9 shows the level of satisfaction

experience” with five different pedagogical aspects of e-learn-
ing (assessment, communication with academic,
The survey addressed students’ perceptions about communication about the course, access to course
the role of their e-learning experience through two material, access to additional material).
indicators (Table 3). Over 70% of students rated All indicators are above 60%. There was a
their experience as positive, as shown in Figure higher level of satisfaction in flexible mode with
8. However, students’ overall expectations of the the ways in which e-learning related to com-
course they had taken were met by only 62% of munication with the academic, communication
students in flexible mode and 66% of students in about the course, and assessment. In flexible
enhanced mode. mode, students were most satisfied with course
The graph in Figure 8 highlights the issue: communication (82%) while in enhanced mode
students are positive about their own e-learning students were most satisfied with the support for
experience but are not so sure if their expectations course content storage (83%).
were met. Similar to the issue discussed previ- Thus a fifth issue emerged from the data that
ously, the data analysis so far has not offered a related to the adequacy of the level of online in-
plausible explanation, except for the speculative teraction. It seems that students (who are mostly
suggestion that, in fact, students were not suffi- studying full time) are engaged in e-learning in a
ciently informed about all aspects of the course similar way in both flexible and enhanced mode;
prior to starting it, to be able to conceive reason- it may be concluded that the design differences
able expectations. between the two blended e-learning models have
not led to significant difference in the use of the
Issue 5: “online Interaction” learning environment (also supported by the data
discussed in Issue 1). In both modes the same
Specific aspects of student satisfaction with e- BlackBoard ™ components are used and emphasis
learning were addressed through the indicators is on tools which improve communication and

Figure 9. Student satisfaction with e-learning pedagogical aspects (average)
also course organization but not on tools which journey. It might be expected that students will
support the development of student capabilities. gradually develop their own effective online
This is somewhat in contrast with the expecta- study habits (Sharpe & Benfield, 2005). Still the
tion that blended e-learning models would offer future of e-learning heavily depends on academics
more and diverse opportunities for deep learning providing clear explanations about the purpose
and that the mix of face-to-face instruction and of online work and expected involvement, and
computer-based communication, including the succinct instructions addressing student respon-
Internet in a blended learning situation “will create sibilities.
a myriad of educational possibilities that reflects The case data provides evidence indicating
… pedagogical richness” (Mortera-Gutierrez, that students are reasonably well satisfied with the
2006, p. 317). level and quality of online interaction. Accord-
ing to Kickul and Kickul (2006), the perceived
value of e-learning and hence satisfaction with
Future trends its value correlates positively with the level of
online interaction among proactive learners.
Future trend patterns emerging from the data Therefore the e-learning of the future will need
about student perceptions of e-learning value, and to “embrace learning solutions that are built upon
the data about actual and perceived e-learning the principles of connectedness, communication,
platform usage cluster around “course design” creative expression, collaboration and competi-
(e.g., the e-learning model and the underpinning tiveness,” to quote Adobe Systems’ Ellen Wagner
pedagogy), and “course delivery” (e.g., the e- in Neal (2006).
learning platform). The second pattern relates to course delivery.
The first pattern relates to course design. It was It was interesting to observe, for example, that
evident from the findings that students’ experi- only half of the students who had already been
ences were positive but not overwhelmingly so exposed to e-learning (52%) showed a prefer-
as they needed a better roadmap of the e-learning ence for using it. Possible reasons may include

students not understanding the role of the medium concLusIon

in an on-campus university (Sharpe & Benfield,
2005), students finding it difficult to adapt to the The work presented in this chapter investigates
change of the educational model (Mortera-Guti- stakeholders’ perceptions about the value of
errez, 2006), students having time management online learning in a New Zealand undergradu-
problems (Hunt et al., 2002). In future learning ate business degree, based on the premise that
environments it might be expected that blending advancing e-learning needs to be grounded in a
will commonly occur in the area of e-learning good understanding of the value attributed to e-
platform support as, for example, the intelligent learning and of the indicators of overall student
tutoring system where text messages are stored satisfaction with e-learning. An e-learning value
in Web accessible format and later disseminated framework was proposed and used to study data
(Silander & Rytkohen, 2005). collected though a survey and from BlackBoard™
With regard to the supporting IT infrastruc- records.
ture, blended e-learning models may need to The analysis of the emerging and future trends
support a more diverse range of communication showed that in the future blending is likely to occur
channels and more sophisticated tools for detailed not only along the pedagogical, but also along the
feedback on assessment activities and thus to technological and even the organizational dimen-
provide more stimuli for developing students as sion of e-learning and should have an emphasis
highly motivated e-learning participants (Hisham on aligning with work/life balance. Stakeholders’
et al., 2004; Wentling, Waight, Gallaher, La Fleur, increased expectations of e-learning value will
Wang, & Kanfer, 2000). continue to present a challenge and will provide
Another trend observed was the use of the an area of fruitful further research.
e-learning platform predominantly during the
daytime on weekdays; with similar results re-
ported by Burr and Spennemann (2004) and Future reseArch dIrectIons
earlier by McKnight and Demers (2002), it seems
that whilst access 24/7 is required, the emphasis The importance of student understanding and
should be on providing sufficient capacity and satisfaction with both online delivery models and
technical support during normal business hours. features of the e-learning environment, and the
This relates to the raising importance of work/life need to provide effective interaction and partici-
balance (Goode, 2003). pation mechanisms to online learners encourages
Based on the patterns discussed above, four future research in several directions. Further
likely drivers of future learner’s satisfaction with research into student adoption of e-learning, ap-
e-learning can be identified: the appropriateness plying well established information technology
of pedagogy, the level of interaction, the level adoption models, may help to better understand
of blending of models and platforms, and the student motivation in specific contexts (Ndubisi,
balance between “life” and study. These results 2006) while studies with a focus on a particular
confirm some prior research findings (Gerbic, discipline, for example, accounting, may help
2002; Petrova, 2002; Sinclair, 2003b). The align- enhance course design (Flynn, Concannon, &
ment of the drivers is also consistent with the Bheachain, 2005; Wells et al., 2005). Along with
initial research framework (Figure 1) in which more in-depth studies of student satisfaction, mo-
the e-learning environment is created through tivation, and online learning styles (Hisham et al.,
stakeholder participation in the two basic teach- 2004; Sharpe & Benfield, 2005), a more detailed
ing and learning processes: course design and investigation of the factors driving academic
course delivery.

motivation (Tastle et al., 2005) and the required (Australia) and JISC-CETIS (UK) (Version 2).
special training is also needed. Retrieved October 19, 2007, from http://www.
The cases presented support the notion that jisc.ac.uk/uploaded_documents/Altilab04-infra-
although students are satisfied with e-learning in a structureV2.pdf
course currently taken, they might not have formed
Brahm, C., & Kleiner, B. H. (1996). Advantages
a sufficiently positive attitude towards e-learning
and disadvantages of group decision-making
in general and therefore cannot recommend it to
approaches. Team Performance Management,
others with confidence. Therefore, studying stu-
2(1), 30-35.
dent perceptions and satisfaction with e-learning
will need to continue, as also evidenced by works Britain, S. (2004, May). A review of learning
such as Flynn et al. (2005), Hisham, et al. (2004), design: Concept, specifications and tools. JISC.
Hunt et al.(2002), Ndubisi (2006), Selim (2005), Retrieved October 19, 2007, from http://www.jisc.
and Wells et al.(2005). ac.uk/uploaded_documents/ACF1ABB.doc
With the observed increase in the range of
Burr, L., & Spennemann, D. H. R. (2004). Patterns
user interfaces, physical devices and supporting
of user behaviour in university online forums.
infrastructure driven by new and emerging infor-
International Journal of Instructional Technology
mation and communication technologies (Blinco
and Distance Learning, 1(10), 11-28.
et al., 2004), further research is needed in the area
of blended models such as blending content from Buzzetto-More, N. A., & Pinhey, K. (2006). Gui-
different sources such as multimedia (Verhaart delines and standards for the development of fully
& Kinshuk, 2004), blending content with learn- online learning objects. Interdisciplinary Journal
ing processes (Britain, 2004; Buzzetto-More & of Knowledge and Learning Objects, 2, 95-104.
Pinhey, 2006), and blending delivery platforms as,
Cashion, J., & Palmieri, P. (2000). Quality in
for example, the use of mobile networks (Petrova,
online learning: Learners views. Retrieved Oc-
2007) which will help create a more satisfactory
tober 19, 2007, from http://flexiblelearning.net.
and fulfilling e-learning environment. Finally,
au/nw2000/talkback/p14-3.htm
further research will help identify and conceptu-
alise advanced blended learning models. Cavanaugh, C. (2002). Distance education qua-
lity: The resources-practices-results cycle and
the standards. Retrieved October 19, 2007, from
reFerences http://www.unf.edu/~caavanau/2569.htm
Council for Higher Education Accreditation.
Barker, K. (2002). Canadian recommended
(2002). International quality review. Retrieved
e-learning guidelines. CACE. Retrieved Oc-
October 19, 2007, from http://www.chea.org/in-
tober 19, 2007, http://www.futured.com/pdf/
ternational/inter_summary02.html
CanREGs%20Eng.pdf
Davies, M. (2001). Adaptive AHP: A review of
Berman, S. H., & Pape, E. (2001). A consumer’s
marketing applications with extensions. European
guide to online courses. School Administrator,
Journal of Marketing, 35(7), 872-893.
58(9), 14.
Distance Education and Training Council. (2002).
Blinco, K., Mason, J., McLean, N., & Wilson,
DETC accreditation overview. Retrieved October
S. (2004, July 19). Trends and issues in e-learn-
19, 2007, from http://www.detc.org/content/free-
ing infrastructure development. A White paper
Publications.html
for alt-i-lab 2004, prepared on behalf of DEST

Flynn, A., Concannon, F., & Bheachain, C. N. In R. Kinshuk, K. Lewis, R. Akahori, T. Kemp,
(2005). Undergraduate students’ perceptions of L. Okamoto, C. Henderson & H. Lee (Eds.),
technology-supported learning: The case of an Proceedings of the 9th International Conference
accounting class. International Journal on E- on Computers in Education (Vol. 2, pp. 964-968),
Learning, 4(4), 427-444. Auckland, New Zealand: Asia-Pacific Society for
Computers in Education.
FuturEd. (2002). Consumers guide to e-Learning.
Retrieved October 19, 2007, from http://www. Institute for Higher Education Policy. (2000).
futured.com/pdf/ConGuide%20Eng%20CD.pdf Quality on the line. Retrieved October 19, 2007,
from http://www.ihep.com/Pubs/PDF/Quality.
Gerbic, P. (2002). Learning in asynchronous envi-
pdf/
ronments for on campus students. In R. Kinshuk,
K. Lewis, R. Akahori, T. Kemp, L. Okamoto, Inglis, A. (1999). Is online delivery less costly
C. Henderson & H. Lee (Eds.), Proceedings of than print and is it meaningful to ask? Distance
the 9th International Conference on Computers Education, 20(2), 220-232.
in Education (Vol. 2, pp. 1492-1493), Auckland,
Kickul, G., & Kickul, J. (2006). Closing the gap:
New Zealand: Asia-Pacific Society for Computers
Impact of student proactivity and learning goal
in Education.
orientation on e-learning outcomes. International
Global University Alliance. (2000). About GUA. Journal on E-Learning, 5(3), 361-372.
Lambert, M. P. (1996). The distance education
gua.com/shell/gua/index.asp
and training council: At the cutting edge. Quality
Goode, V. L. (2003). Lifestyle in the balance. Assurance in Education, 4(4), 26-28.
Chartered Accountants Journal, 82(3), 22-24.
Lee, Y. L., & Nguyen, H. (2005). So are you online
Heerema, D. L., & Rogers, R. L. (2001). Avoiding yet?! Distance and online education today. In M.
the quality/quantity trade-off. T.H.E. Journal, Khosrow-Pour (Ed.), Managing modern organiza-
29(5), 14-21. tions with information technology. Proceedings
of the 2005 Information Resources Management
Hisham, N., Campton, P., & FitzGerald, F. (2004).
Association International Conference (pp. 1035-
A tale of two cities: A study on the satisfaction of
1036). San Diego, CA: Information Resource
asynchronous e-learning systems in two Austral-
Management Association.
ian universities. In R. Atkinson, C. McBeath, D.
Jonas-Dwyer, & R. Phillips (Eds.), Beyond the Leidner, D., & Jarvenpaa, S. (1995). The use of
comfort zone: Proceedings of the 21st ASCILITE information technology to enhance management
Conference (pp. 395-402). Perth, Australia: school education: A theoretical view. MIS Quar-
ASCILITE. Retrieved October 19, 2007, from terly, 19(3), 265-291.
http://www.ascilite.org.au/conferences/perth04/
Lizzio, A., Wilson, K., & Simons, R. (2002).
procs/hisham.html
University students’ perceptions of the learning
Hope, A. (2001) Quality assurance. In G. Farrell environment and academic outcomes: Implica-
(Ed.), The changing faces of virtual education tions for theory and practice. Studies in Higher
(pp. 125-140). London: The Commonwealth of Education, 27(1), 27-52.
Learning.
McKnight, R., & Demers, N. (2002). Evaluating
Hunt, L. M., Thomas, M. J. W., & Eagle, L. course Web site utilization by students using Web
(2002). Student resistance to ICT in education. tracking software: A constructivist approach. In

Proceedings of the Technology, Colleges and Petrova, K. (2007). Mobile learning as a mobile
Community Worldwide Online Conference 2002, business application. International Journal of
Kapio’lani, Hi: University of Hawaii. Retrieved Innovation in Learning, 4(1), 1-13.
October 19, 2007, http://kolea.kcc.hawaii.edu/tcc/
Petrova, K., & Sinclair, R. (2005). Business
tcon02/presentations/mcknight.html
undergraduates learning online: A one semester
McLoughlin, C., & Luca, J. (2001). Quality in snapshot. International Journal of Education
online delivery: What does it mean for assess- and Development using Information and Com-
ment in e-learning environments? In G. Kennedy, munication Technology, 1(4), 69-88. Retrieved
M. Keppell, C. McNaught & T. Petrovic (Eds.), October 19, 2007, from http://ijedict.dec.uwi.
Meeting at the crossroads. Proceedings of the edu/viewissue.php?id=6
18th Annual Conference of the Australasian
Phillimore, R. (2002). Face to face lectures or
Society for Computers in Learning in Tertiary
econtent: Student and staff perspectives. In R.
Education (pp. 417-426). Melbourne, Australia:
Kinshuk, K. Lewis, R. Akahori, T. Kemp, L.
Australasian Society for Computers in Learning
Okamoto, C. Henderson & H. Lee (Eds.), Pro-
in Tertiary Education. Retrieved October 19, 2007,
ceedings of the 9th International Conference on
from http://www.ascilite.org.au/conferences/mel-
Computers in Education (Vol. 1, pp. 211-212).
bourne01/pdf/papers/mcloughlinc2.pdf
Auckland, New Zealand: Asia-pacific Society
Mortera-Gutierrez, F. (2006). Faculty best prac- for Computers in Education.
tices using blended learning in e-learning and in
Ponzurick, T. G., France, K.R., & Logar, C.M.
face-to-face instruction. International Journal
(2000). Delivering graduate marketing education:
on E-Learning, 5(3), 313-337.
An analysis of face-to-face versus distance edu-
Neal, L. (2006, January 19). Predictions for 2006: cation. Journal of Marketing Education, 22(3),
E-learning experts map the road ahead. eLearn 180-187.
Magazine. Retrieved October 19, 2007, from
Quality Assurance Agency for Higher Education.
http://www.elearnmag.org/subpage.cfm?section
(2002). Distance learning guidelines. Retrieved
=articles&article=31-1
October 19, 2007, from http://www.qaa.ac.uk/pu-
Ndubisi, N. O. (2006). Factors of online learning blic/dlg/dlg_textonly.htm
adoption: A comparative juxtaposition of the
Saaty, T. L. (1994). How to make a decision: The
theory of planned behaviour and the technology
analytic hierarchy process. Interfaces, 24(6),
acceptance model. International Journal on E-
19-43.
Learning, 5(4), 571-591.
Scott, G. (2001). Assuring quality for on-
Petrova, K. (2001).Teaching differently: A hybrid
line learning. Retrieved October 19, 2007, from
delivery model. In N. Delener & C. N. Chao (Eds.),
http://www.qdu.uts.edu.au/pdf%20document/
Proceedings of the 2001 Global Business and
QA%20for%200
Technology Association International Conference
(pp. 717-727). Istanbul, Turkey: Global Business Selim, H. M. (2005). Elearning critical success
and Technology Association. factors: an exploratory investigation of student
perceptions. In M. Khosrow-Pour (Ed.), Ma-
Petrova, K. (2002). Course design for flex-
naging modern organizations with information
ible learning. New Zealand Journal of Applied
technology. Proceedings of the 2005 Information
Computing and Information Technology, 6(1),
Resources Management Association Internatio-
45-50.

nal Conference (pp. 340-346). San Diego, CA: In- tasks. Journal of Applied Business Research,
formation Resources management Association. 10(1), 132-144.
Sharpe, R., & Benfield, G. (2005). The student Uribe, C. L., Schweikhart, S. B., Pathak, D. S.,
experience of e-learning in higher education. Marsh, G. B., & Fraley, R .R. (2002). Perceived
Brookes eJournal of Learning and Teaching, 3(1), barriers to medical-error reporting: An explora-
1-9. Retrieved October 19, 2007, from http://www. tory investigation. Journal of Healthcare Ma-
brookes.ac.uk/publications/bejlt/volume1issue3/ nagement, 47(4), 263-280.
academic/sharpe_benfield.pdf
Verhaart, M., & Kinshuk, C-K., (2004). Adding
Silander, P., & Rytkohen, A. (2005). An intelligent semantics and context to media resources to enable
mobile tutoring tool enabling individualization of efficient construction to learning objects. In C.
students’ learning processes. In Proceedings of Kinshuk, K. Looi, E. Sutinen, D. G. Sampson,
the 4th World Conference on mLearning (paper I. Aedo, L. Uden, & E. Kähkönen (Eds.), Pro-
59), Cape Town, Republic of South Africa. ceedings of the 4th International Conference on
Advanced Learning Technologies (pp. 651-653).
Sinclair, R. M. S. (2003a). Components of quality
Joensuu, Finland: IEEE Computer Society.
in distance education. In G. Davies & E. Stacey
(Eds.), Quality education @ a distance (pp. 257- Wells, P., Fieger, P., & de Lange, P. (2005, July).
264). Boston: Kluwer Academic Publishers. Integrating a virtual learning environment into a
second year accounting course: Determinants of
Sinclair, R. M. S. (2003b). Stakeholders’ views of
overall student perception. Paper presented at the
quality in flexibly delivered courses. Unpublished
2005 Accounting and Finance Association of Au-
Masters Research Paper. Deakin University,
stralia and New Zealand Conference, Melbourne,
Australia: Geelong.
Australia: Accounting and Finance Association
Southern Regional Education Board. (2000). of Australia and New Zealand.
Principles of good practice. Retrieved October
Wentling, T. L., Waight, C., Gallaher, J., La Fleur,
19, 2007, from http://ww.electroniccampus.org/
J., Wang, C., & Kanfer, A. (2000). E-learning - a
student/srecinfor/publicaitons/Principles_2000.
review of literature. Knowledge and Learning
pdf.
Systems Group, University of Illinois at Urbana-
Swan, M. K. (1995). Effectiveness of distance Champaign. Retrieved October 19, 2007, from
learning courses—Students’ perceptions. In Pro- http://learning.ncsa.uiuc.edu/papers/elearnlit.
ceedings of the 22nd Annual National Agricultural pdf .
Education Research Meeting (pp.34-38), Denver,
Western Interstate Commission for Higher Edu-
CO. Retrieved October 19, 2007, from http://
cation. (2002). Best practice for electronically
www.ssu.missouri.edu/SSU/AgEd/NAERM/s-
offered degree and certificate programs. Retrie-
a-4.htm
ved October 19, 2007, from http://www.wiche.
Tastle, W. J., White, B. A., & Shackleton, P. (2005). edu/telecom/Article1.htm
E-learning in higher education: The challenge,
World Alliance in Distance Education, (2002).
effort, and return on investment. International
World alliance in distance education. Retrieved
Journal on E-Learning, 4(2), 241-250.
October 19, 2007, from http://www.wade-univer-
Tullous, R., & Utecht, R. L. (1994). A decision sities.org/index.htm
support system for integration of vendor selection

AddItIonAL reAdIngs Eaton, J. S. (2002). Maintaining the delicate

balance: Distance learning, higher education
American Federation of Teachers. (2000). Guide- accreditation, and the politics of self-regulation.
lines for good practice. Retrieved October 19, Washington: American Council on Education
2007, from http://www.aft.org/pubs-reports/ Center for Policy Analysis.
higher_ed/distance.pdf
Ehlers, U. (2004, May). Quality in e-learning from
Australian National Training Authority. (2002). a learner’s perspective. European Journal of Open
Flexibility through online learning. National Cen- Distance and E-Learning. Retrieved October 19,
tre for Vocational Education Research. Retrieved 2007, from http://www.eurodl.org/materials/con-
October 19, 2007, from http://www.ncver.edu. trib/2004/Online_Master_COPs.html
au/research/proj/nr1F12/nr1F12.pdf
E-learning in tertiary education: Where do we
Bonk, C. J., & Graham, C. R. (2006). The hand- stand? (2005). Education & Skills, 4, 11-93.
book of blended learning: Global perspectives, OECD, Centre for Educational Research and
local designs. New York: Pfeiffer Publishing. Innovation.
Borotis, S., Zaharias, P., & Poulymenakou, A. Frydenberg, J. (2002). Quality standards in e-
(2007). Critical success factors for e-learning learning: A matrix of analysis. International
adoption and sustainability: A holistic approach. Review of Research in Open and Distance Learn-
In T. Kidd (Ed.), Handbook of research on in- ing, 3(2).
structional systems and technology. New York:
Gilroy, P., Long, P. D., Rangecroft, J., & Tricker,
Idea Group Inc.
T. (2001). Evaluations and the invisible student:
Calvert, J. (2003). Quality assurance and quality Theories, practice and problems in evaluating
development: What will make a difference? In G. distance education provision. Quality Assurance
Davies & E. Stacey (Eds.), Quality education @ in Education, 9(1), 14-22.
a distance (pp. 17-29). Boston: Kluwer Academic
Hodges, C. B. (2004). Designing to motivate:
Publishers.
Motivational techniques to incorporate in e-learn-
Carnevale, D. (2000a). Assessing the quality of ing experiences. Journal of Online Interactive
online courses remains a challenge. The Chronicle Learning, 2(3). Retrieved October 19, 2007,
of Higher Education, 46(24), A59. from http://www.ncolr.org/jiol/issues/viewarticle.
cfm?volID=2&IssueID=8&ArticleID=31
Carnevale, D. (2000b). Study assesses what par-
ticipants look for in high-quality online courses. Hoppe, G., & Breitner, M. H. (2003). Business
The Chronicle of Higher Education, 47(9), A46. models for e-learning. Retrieved October 19, 2007,
from http://www.wiwi.unihannover.de/fbwiwi/
forschung/diskussionspapiere/dp287.pdf
(2001). The role of accreditation and assuring
quality in electronically delivered distance learn- International Federation of Accountants. (2000).
ing. Retrieved October 19, 2007, from http://www. Quality issues for Internet and distributed learning
chea.org/pdf/fact_sheet_2.pdf in accounting education. New York: International
Federation of Accountants. Retrieved October 19,
2007, from http://www.ifac.org/Members/Down-
(2003). Important questions about diploma mills
Loads/EDC-QualityIssues.pdf
and accreditation mills. Retrieved October 19,
2007, from http://www.chea.org/degreemills/de-
fault.htm
0
Kettunen, J., & Kantola, M. (2006). Strategies for Rosenberg, M. (2006). What lies beyond e-learn-
virtual learning and e-entrepreneurship in higher ing? Learning Circuits. Retrieved October 19,
education. In F. Zhao (Ed.), Entrepreneurship 2007, from http://www.learningcircuits.org/2006/
and innovations in e-business. Hershey, PA: March/rosenberg.htm
IRM Press.
Sahay, S. (2004). Beyond utopian and nostalgic
Lindh, J., & Soames, C. (2004). Are students’ views of information technology and education:
and teachers’ views on online courses in accor- Implications for research and practice. Journal
dance? A dual perspective on an online university of the Association for Information Systems, 5(7),
course. Electronic Journal on eLearning, 21(1), 282-313.
129-134.
Schmees, M. (2004). Integrating e-commerce
Ling, P., Arger, G., Smallwood, H., Toomey, R., into e-learning. In Proceedings of the 6th Inter-
Kirkpatrick, D., & Barnard, I. (2001). The effec- national Conference on Electronic Commerce
tiveness of models of flexible provision of higher (pp.177-186). Delft, The Netherlands: Association
education. Canberra, Australia: Department of for Computing Machinery.
Education, Training and Youth Affairs, Com-
Trentin, G. (2000). The quality-interactivity
monwealth of Australia.
relationship in distance education. Educational
McPherson, M. (2002). Organizational critical Technology, 40(1), 17-27.
success factors for managing e-learning. In R.
Twigg, C. (2001). Quality assurance for whom?
Kinshuk, K. Lewis, R. Akahori, T. Kemp, L.
Okamoto, L. Henderson & C. H. Lee (Eds.), Pro-
center.rpi.edu/pew5ym/mono3.pdf
ceedings of the 9thIinternational Conference on
Computers in Education, Vol. 2 (pp. 1540-1541). Valiathan, P. (2002). Blended learning models.
Auckland: Asia-Pacific Society for Computers Learning circuits: American society for training
in Education. and development (ASTD)’s source for e-learning.
Retrieved October 19, 2007, from http://learning-
Open and Distance Learning Quality Council.
circuits.org/2002/aug2002/valiathan.html
(2005). Standards in open & distance learning.
Retrieved October 19, 2007, from http://www. Zentel, P., Bett, K., Meister, D. M., Rinn, U., &
odlqc.org.uk/standard.doc Wedekind, J. (2003). A change process at universi-
ty—Innovation through ICT? In R. Williams (Ed.),
Parry, D. (2004). What do online learners really
Proceedings of the 2nd European Conference on
do, and where and when do they do it? Bulletin of
eLearning (pp. 507-513). Glasgow, United King-
Applied Computing and Information Technology,
dom: Academic Conferences International.
2(2). Retrieved October 19, 2007, from http://www.
naccq.ac.nz/bacit/0202/2004Parry_eLearners.
html

Chapter VIII
Integrating Technology and
Research in Mathematics
Education:
The Case of E-Learning
Giovannina Albano
Università di Salerno, Italy
Pier Luigi Ferrari

Università del Piemonte Orientale, Italy
ABstrAct
This chapter is concerned with the integration of research in mathematics education and e-learning.
We provide an overview of research on learning processes related to the use of technology and a sketch
of constructive and cooperative methods and their feasibility in an e-learning platform. Moreover, we
introduce a framework for dealing with language and representations to interpret students’ behaviours
and show examples of teaching activities. Finally some opportunities for future research are outlined.
We hope to contribute to overcome the current separation between technology and educational research,
as their joint use can provide matchless opportunities for dealing with most of the learning problems
related to mathematical concepts as well as to linguistic, metacognitive, and noncognitive factors.
Integrating Technology and Research in Mathematics Education
IntroductIon sketched and apply some of the ideas and methods

discussed there.
The main concern of this chapter is the integration The section “Future Trends and Conclusions”
of technology and research in the field of math- includes some discussion of the opportunities for
ematics education. Currently technology is too future research.
often used with little or no concern for the results In all the examples described in this chapter
of educational research, despite the fact that they we refer either to Moodle (Moodle, 2006) or to
could provide valuable help to both magnify the IWT (Intelligent Web Teacher, 2006). The latter
outcomes and keep away from some unwelcome is a distance-learning platform designed to lay
washback. Conversely, too often research in the foundation for the next generation e-learning
mathematics education disregards the impressive (for details, see Albano, Gaeta, & Salerno, 2006,
opportunities technology could provide. or Intelligent Web Teacher, 2006).
Through the chapter we focus on e-learning as
a domain appropriate for integrating technology
and educational research. We argue that nowa- BAckground
days technology is flexible enough to be used
within different theoretical frameworks (such as technology and research on
the constructivist and the socio-cultural ones) mathematics education
and at different levels (cognitive, metacognitive,
noncognitive). We also show that technology can Currently information and communication
provide matchless opportunities for dealing with technology (ICT) is not strictly linked to any
most of the learning problems related to language theoretical framework in mathematics education.
and representations. This was not the case in the past, as sometimes it
In the section “Background” we give: was naively associated to some specific cognitive
framework (e.g., information-processing theory)
• A concise overview of some outcomes of or even to some interpretation of mathematics (e.g.,
research that underline the complexity of computational ones). This may account for the
educational processes, and in particular the relatively poor role played by ICT in most studies
need for taking into account not just cogni- in the psychology of mathematics education.
tive, but also metacognitive and noncogni- We also assume that the use of ICT is not a
tive aspects; simple matter but requires the development of
• An overview of research on individual and detailed teaching paths and much research to
personal learning processes related to the fully exploit the opportunities provided and to
use of technology; keep away from any potential drawbacks.
• A sketch of the main features of constructive Research on mathematics education, converse-
and cooperative methods and their feasibility ly, has widely shown the complexity of teaching
in an e-learning platform; and learning processes, and thus the inadequacy
• A framework for dealing with language of one-dimensional models, including the belief
and representations in order to effectively that the simple addition of some technology to
interpret students’ behaviors. standard teaching practices could provide con-
siderable improvements of the outcomes.
In the section “Teaching and Learning Op- In particular any model for mathematics educa-
portuinities,” we show examples of teaching tion has to consider that students’ performances
activities which fulfil some of the requirements are affected by factors belonging to at least three
different levels:

• The cognitive level, which involves the learn- achievements occur when the instruction is exactly
ing of the specific concepts and methods of matched to the aptitudes of the learner. At first, we
the discipline, also related to the obstacles can say that individualisation regards how much the
recognized by research and practice; instruction fits students’ characteristics, creating
• The metacognitive level, which involves learning situations suitable to different students.
learners’ control of their own learning pro- In particular we refer to the individualisation at
cesses; the teaching level which, according to Baldacci
• The noncognitive level, which involves (1999), means the adjustment of the teaching to the
beliefs, emotions, and attitudes, and all individual students’ characteristics, by means of
affective aspects, which are most often specific and concrete teaching practices. Another
critical in shaping learners’ decisions and major goal is the personalisation of the teaching,
performances. which refers to the set of activities directed to
stimulate each specific person in order to achieve
As we will see below, ICT can play a part in the maximum intellectual capability. It is clear that
each of these levels, including the noncognitive neither individualisation nor personalisation are
one, as it from the one hand can deeply influence possible at undergraduate level, especially with
learners’ beliefs, emotions, and attitudes related large classes of freshman students, if teaching is
to mathematics, and from the other hand is itself still based on standard lectures. The didactical
the object of deep-rooted beliefs and can produce transposition carried out by the teacher is based
effects at the noncognitive level. on general parameters, which arise from the
So any study integrating ICT and research on average of sets of data regarding, for example,
mathematics education has to take into account previous curriculum and knowledge, attitude to
noncognitive factors related to technology as well mathematics, metacognitive awareness, and so
as to mathematics. forth, and which can hardly suit the actual needs
In the next sections we will focus on some or problems of the individuals.
issues which are regarded as critical by research On the contrary, the modality of blended
in mathematics education and could be dealt learning, that is the support by online activities
with in a more appropriate way with the help of to standard lectures, seems to give a considerable
an e-learning platform: constructive learning, contribution in the right direction. The belief
cooperative learning, language and representa- that there exist teaching methods which produce
tions, and noncognitive implications. Of course, the best outcomes has been long discarded, and
although we examine each of them separately, learning is now regarded as the result of a process
in teaching practice these issues cannot be dealt whose core is the pair person-situation, which is
with in isolation. influenced by both teaching methods and indi-
vidual differences (Jonassen & Grabowski, 1993).
individual and Personal teaching In particular the support of diversity in student’s
and Learning methods is also viewed as the guide of mathemati-
cal learning (Balacheff & Sutherland, 1999).
The individualisation of teaching is one of the From the viewpoint of individualisation, the
most critical issues in instructional practice. It is teaching procedures included in the platform
well known that some instructional strategies are should get the students to attain the basic skills
more or less effective for particular individuals by means of a choice of different learning paths,
depending upon their specific abilities. According whereas from that of personalisation teaching
to Cronbach and Snow (1997), the best learning activities should be planned in order to allow the

students to get to excellence the student’s own way, as of the specific characteristics of the knowledge
through specific opportunities to develop cogni- at stake.” (p. 2)
tive potential. In order to develop each student’s Thus the learning paths can be individualised
specific skills of, it is necessary to let the student according the student’s profile, with particular
be free to move, to choose, to plan, and to manage reference to the skills which are being acquired
some suitable cognitive situations. and the learning style. This kind of individualisa-
According to this perspective, e-learning plat- tion/personalisation can be automatically realised
forms allow teachers to create learning situations by the platform or can be constructed by each
appropriate for each student. In this context, the student through the learning process. In fact for
teacher, who might more properly be referred to each teaching unit the student can ask the system
as the author, is not just a content developer, but for the list of the other teaching units regarding
becomes an organizer of contexts in which the the same concept at stake. Moreover students can
content is aimed at the attainment of well-de- add personal annotations to the teaching units,
fined goals. All this requires the author to use a which can be simple textual notes or video and
range of skills, from those related to teaching to audio files or figures. They have also a space
the technological ones. According to Brousseau to share resources among them. In such a way
(1997), we can say that in e-learning environment students interact with the learning material in a
the role of the author is to prepare a-didactical tri-dimensional relationship: they do not restrict
situations, that are situations in which attention themselves to receive and elaborate some objects
is paid to the students and knowledge, not to the (such as in the case of the book), but produce new
teacher. There are no specific teaching constraints, learning objects starting from the ones placed
so what the learner does is not affected by any at their disposal by the platform (Maragliano,
pressure by the teacher, and knowledge system 2000).
is modified as a result of adaptation processes Resources like Moodle’s lesson or IWT’s di-
linked to the strategies performed. dactic unit may be the starting point to develop
Individualisation is possible as far as a choice individualised or personalised learning paths. In
of teaching materials, such as written texts, mul- that frame students are required to perform a test
timedia file, interactive exercises, and so on, is at the end of each unit or group of units in order
made available to the learner. The learner should to proceed to the next one. In case of satisfactory
be given a wide range of stimuli through different results each student will be automatically given
sensorial channels (auditory, visual, manipulative, access to the next unit, otherwise the student will
and so forth) for each teaching unit, in order to be kept in the current unit or will be directed to
make easier the adjustment of the teaching style a remedial unit. The questions included in the
related to the learning styles of the learners. This test may regard just the understanding of the text
way the student can learn any content more eas- from the viewpoint of language, or the specific
ily, as the teaching modalities are more suitable contents. Pros and cons of tests are more widely
to the student’s cognitive styles, allowing the discussed in the “Self-Evaluation” section.
student to overcome some learning difficulties. In the perspective of personalisation, open-
According to Balacheff (2000), “learning does not ended questions and reflection on “wrong” an-
occur because of one specific type of interaction, swers constitute the starting point of new problem
but because of the availability of all of them. One situations the learner can deal with. Such oppor-
type of interaction, or one type of agent, being tunities allow “mistakes” to play a constructive
selected depending on the needs of the learner at educational role, as they can be productively used
the time when the interaction is looked for, as well in the platform, in place of the (usually ineffective)

practice of just proposing the replication of what the concerns of the supporters of the restricted
has already been presented. According to Perrin view, through the chapter we are adopting an
Glorian (1994), sometimes the mistake is indeed inclusive definition of constructivism and focus
provoked by previous knowledge, which had its on each learner’s opportunities to interact with
owns interests and successes, which is false or the environment. Within an e-learning platform
inadequate in the new context. To be aware and the learner can freely use a range of modules to
to analyse such kind of mistakes is a fundamental construct his or her knowledge. Modules allow-
step in order to construct new knowledge. The aim ing some feedback, such as Moodle’s “lesson” or
is not to try to avoid any possible mistake, since “quiz” as well as suitable IWT interactive learning
they are intrinsic in the process of construction of objects (e.g., interactive online exercise sessions
knowledge, but rather to minimise their effects, or Virtual Scientific Experiments), are specially
interpreting them problematically and developing relevant from this perspective.
the necessary awareness.
Such kind of activities involve constructive Cooperative Learning
processes of problem solving, of interpretation,
and conversion of representations in different E-learning platforms generally provide a number
semiotic systems and also metacognitive aspects, of activities involving peer interactions or inter-
such as the method used to read and understand actions between learners and tutors. Modules
a text. In fact an increasing number of students such as Moodle’s “workshop,” “wiki,” or “task”
seems to believe that learning means being able or IWT classroom virtual space are generally
to repeat pieces of text, obviously with the help of suitable for designing activities of this kind. In
some keywords, without being worried to draw at this section we describe some experiences with
least the simplest inferences from the text. a “workshop” module at undergraduate level.
From the viewpoint of the theory of mathematics
Constructive methods education, all of these activities can be framed
within the so-called socio-cultural (or “discour-
In mathematics education, constructive methods sive”) approach. For more information see Kieran,
play a major and increasing role. An e-learning Forman, and Sfard (2001).
platform allows the learners to actively construct Our idea is to support the students by online,
new knowledge as they interact with their environ- time restricted activities based on role-play, which
ment. We are aware that some researchers adopt actively engage them and induce them to face
a more restricted definition of constructivism and learning topics in a more critical way.
they would regard some computer environments It is well known that the cognitive processes
and, more generally, some ways of using ICT induced by talking, discussing, and explain-
as inconsistent with the constructivist stance. ing to others the concepts to be learnt promote
For example, graphing a function (defined by a deeper level or higher-order thinking (Johnson
symbolic expression) by means of the facilities of & Johnson, 1987). In this framework we want to
some computer algebra system might be regarded put emphasis on peer learning (Boud, Cohen, &
as nonconstructive as some steps of the process are Sampson, 1999), which is intended as the use of
fully concealed to the learner, whereas programs teaching and learning strategies in which students
explicitly computing the coordinates of a finite learn with and from each other without the im-
set of points of the graph of the function might mediate intervention of a teacher. It includes peer
be regarded as more suitable for a truly construc- tutoring and peer mentoring. When the students
tive approach. Although we understand some of in a group act as both teachers and learners we

talk about reciprocal peer learning. This may • The construction of a representation within
incorporate self and peer assessment whereby a semiotic system, such as writing a text or
students actively develop criteria for assessment. a formula or drawing a figure.
Falchikov (2001) analysed the various peer tutor- • The treatment of representations within a
ing techniques and the benefits linked to each of semiotic system, such as summarizing a
them. She found evidence of some improvement verbal text, simplifying a formula, or trans-
in comprehension, memory for lecture content, forming a geometrical figure.
performance, and facilitation in encoding and • The conversion of representations from a
retrieval of material given by Guided Reciprocal semiotic system to another, such as verbally
Peer Questioning. describing a figure, or writing a formula to
represent the data of a word problem.
Language and representations
Duval often refers to semiotic representation
The potential of information and communication systems as “registers.” We prefer to employ “reg-
technology as regards semiotic or linguistic issues ister” to denote a use-oriented linguistic variety,
is largely underestimated. Language is growing according to the definition widely accepted in the
one of the most relevant issues for research on field of linguistics. According to Duval, the main
mathematics education. On one hand, classes goal of education as far as semiotics is concerned
including students from different linguistic groups is what he names the coordination of semiotic
pose new teaching problems. On the other hand, systems, which is the ability at using multiple
even at undergraduate level, a large share of stu- representations of the same “object” and moving
dents’ failures can be ascribed to linguistic issues. quickly from one to another. A problem involving
An increasing number of students, for example, real functions, for example, can be appropriately
seemingly cannot properly understand a written dealt with by the coordination of the verbal de-
verbal text even if it is simple and short. A description of the function, its symbolic representa-
tailed investigation of language-related students’ tion as an equation, and its Cartesian graph. The
failures is beyond the scope of this chapter. In coordination of semiotic systems improves both
this section we are going to focus on two aspects: understanding and problem solving skills. On
Duval’s (2005) investigation of semiotic represen- one hand, students who can coordinate semiotic
tation systems and the pragmatic interpretation systems are allowed to distinguish a concept from
of mathematical language. its representation (which is harder, if one can deal
with one representation only; on the other hand,
Semiotic Representation Systems and they can adopt the best strategies provided by
their Coordination each representation (for example, symbolic com-
putation of the derivative of a function or visual
Duval’s (2005) Theory of Semiotic Representa- search for a tangent on the graph).
tion Systems provides a new insight on the role of ICT provides plenty of opportunities to use
semiosis in learning. Algebraic symbol notation, multiple representations. An e-learning platform
verbal language, cartesian graphs, and geometrical can suggest a number of activities appropriate to
figures are examples of semiotic representation the goal of achieving the coordination of semiotic
systems. The main activities described by Duval systems.
(2005) are:

A Pragmatic View on Mathematical teAchIng And LeArnIng

Language oPPortunItIes
Recently, various frameworks have been pro- Self-Evaluation

posed that underline the role of languages in
the learning of mathematics. For example, Sfard Most of e-learning platforms provide the opportu-
(2000) interprets thinking as communication and nity of designing sets of questions with automatic
regards languages not just as carriers of pre-ex- evaluation of the answers. The admissible formats
isting meanings, but as builders of the meanings for the items include multiple choice, true/false,
themselves. So, under this perspective, language matching, fill-in, cloze-procedure, short answer,
heavily influences thinking. On the other hand, and numerical answer. Apart from short answer
there is evidence that a good share of students’ and numerical answer items, the other formats
troubles in mathematics, at any school level, in- only require the learners to select their answer
cluding undergraduates, can be ascribed to the out of a prearranged set and not to construct the
improper use of verbal language. More precisely, answer themselves. This might be a critical issue.
students often produce or interpret mathematical Items can be designed according to different cri-
texts according to linguistic patterns appropriate teria: they could be focused on one subject only,
to everyday-life contexts rather than to mathemati- or on a whole course. In general, correct answers
cal ones. The difference is not just a matter of equipped with some comment are made available
vocabulary, grammar, or symbols, but it heavily to students as soon as they have submitted their
involves the organization of verbal texts, their ones. Resources of this kind provide plenty of
functions and relationships with the context they teaching opportunities, and some risk too. The
are produced within. item developers have to make the most of the
Under these assumptions, a pragmatic per- benefits, exploiting the opportunities as much as
spective has proven suitable to provide tools possible, and to reduce the risks. This might make
to interpret students’ behaviors and to design the development of the items a very troublesome
appropriate teaching units. This means focus- business.
ing on language use rather than on grammar, Students might use the sets of questions indi-
and regarding the interpretation of a text as a vidually or in groups, to get immediate feedback
cooperative enterprise which involves not only about some aspects of their learning. This may
vocabulary and grammar, but also the so called greatly affect not just their knowledge, but their
encyclopedia. An e-learning platform provides confidence as well (the so called sense of autoef-
plenty of opportunities for planning activities ficacy). The opportunity of trying and making
compatible with a pragmatic perspective. It is mistakes without the judgment of another human
especially suitable for planning activities aimed being may help some students to grow more
at improving linguistic competence, including confident and to develop a more positive attitude
competence in verbal language, as it allows the towards their products. Students could even use
authors to design a wide range of communication sets of questions as a means to learn: the interaction
situations and to devise tasks forcing students with the resource could be used to add some piece
to use more refined linguistic resources. An ap- of knowledge. Using resources of this kind might
plication of these ideas to advanced mathematics prove somewhat risky, as some kinds of items
has been discussed by Ferrari (2004). All of the might prove harder to develop and implement than
activities described in the above paragraph on others. For example, currently in most platforms
cooperative learning involve plenty of exchanges is much easier and faster to insert word questions
relevant from this perspective.

with little symbolic expressions and no images. Interactions and role-Play

Moreover, items like multiple-choice or true-false
ones cannot provide complete information about The experience we are going to describe may be
students’ achievements. For example, devising inserted in the framework of cooperative learning
a solution strategy for a problem, representing, previously described. The experiment has been
and describing it with words involve fundamental carried out in 2005-2006 in the universities of
skills that should not be overlooked. Uncritical use Salerno and Piemonte Orientale, both in Italy.
of test items might also induce some high school It has been organised by selecting two working
teachers or students to neglect the skills related groups: an experimental group and a control
to argumentation. Thus users should be warned group.
that prearranged-answer items cannot provide a In our setting, the subject matter has been split
complete evaluation of their achievements, and into various sections. For each section rounds of
opportunities to deal with open-answer items different activities have been planned for the two
should be provided anyway. selected groups.
This could be achieved by means of resources The activities of the experimental group have
allowing people to post files like Moodle’s task been based on role-play. In each round each student
or “workshop” or IWT classroom shared area. has dealt with three topics:
Of course items of this sort cannot be evaluated
automatically, but require more sophisticated • The student acts as a teacher, so he or she
patterns of evaluation. devises some questions as if he or she were
On the fall of 2006 at the University of to evaluate someone other’s learning out-
Piemonte Orientale, some 150 biology, chemistry, comes;
and environmental sciences students have been • The student answers to the questions pro-
offered more than 300 quiz items covering all the posed by a peer;
topics of the “Introductory Mathematics” course, • The student again acts as a teacher and checks
from linear algebra to differential and integral the output (both questions and answers) of
calculus. On average each item has been dealt two peers.
with by 34 individuals. More precisely, students
were split into two groups. About half of them At the end of each round, the tutors revise all
visited the platform on a regular basis and tried to the files produced and made them available to
answer to a fair amount of items. The other half all the students.
visited the platform occasionally, made just few The activities of the control group have been
attempts to answer to some items, and completed, based on standard problem solving. Each member
at most, one set of them. The number of students of the group was asked to autonomously solve
regularly visiting the platform and attempting problems provided by the staff (teacher and tutors)
to answer to a reasonable amount of items is in a given time. Then the staff makes available
far beyond our expectations. Their outcomes, solution patterns for self-evaluation.
although not significant from a statistical view- An implicit selection of a third group has
point, encourage us to go on with the experience arisen: the passive users of the platform, who
and to expand and improve the offer for activities have at their disposal lecture notes, self-evalua-
on the platform. tion tests, other materials (worked-out problems,
problems with hints for solution, FAQ), and op-
portunity to contact the teacher, the tutors, and
other students.

The outcomes of the experiment have been and this is why they fail. Some students have ap-
collected at the end of the course by means of preciated to receive from their colleagues some
interviews aimed at understanding how the questions considered “tortuous” so that they
activities carried out have affected the way of have been forced to think about. Actually, if we
studying, which progress have been noticed by see the papers produced by the students, there
the students themselves, which role (among those are no really tortuous questions, as well as there
played) has been considered particularly useful are not at the exams. Anyway the feeling of the
and why. The interviews have given evidence of students simply shows their familiarity with a flat
many benefits due to peer-to-peer activities (see, and rote-learning style that is related to the lack
for example, Albano, 2006, or Albano, Bardelle, of self-posed questions. In the same direction, we
& Ferrari, 2007) strengthening communication note that most of them have found questions that
skills, critical enquiry, and reflection, clarifying they did not think of before.
subject content through discussion, viewing situ- The role of the teacher who checks the correct-
ations from different perspectives, learning how ness is not really much appreciated, essentially for
to work as a team member, becoming actively two reasons: students do not feel themselves to be
involved in the learning process, learning to learn. equal to this task or consider the task not useful
In particular, looking at the benefits identified because they think they surely will do well.
by the students for each role, we can summarize The role-play activities also affected students’
as follows. working methods. The students have acquired the
The most appreciated role has been the first habit of going into depth as a standard practice,
one, because it has allowed them to be in the and the habit of looking at something from more
teacher’s perspective, so getting able to understand viewpoints (also through the comparison with
the educational goals. Moreover, to ask questions other colleagues). This has changed attitudes to-
have helped to study in a more critical and deeper ward studying, fostering the practice of reasoning
way, with greater care, because it is not simple rather than of learning by heart. The involvement
to pose a question due to the fact that there is no in the activities proposed has given the students
method to do that. At the same time, the request of a sort of guidance for the organization of their
a certain number of questions on a topic requires study, providing time constrictions, topics to
to range over the entire programme, not only con- revise, and indications of the relevant activities.
centrating on the specific and restricted topic but Finally we want to note that some students have
also paying attention to all the other linked topics. appreciated such kind of group activity also as
It is also interesting to note that some students training for their future work.
have used this role to make critical points clear From a practical viewpoint, some management
(posing as questions exactly their own doubts). difficulties are to be mentioned. The experimental
Finally we noticed some noncognitive aspects activities described require some work for revis-
such as the trend to pose nontrivial questions, ing students’ products and this has to be done in
also for pride reasons, and this has required the itinere as much as possible, in order to influence
mastery of the topics. their further elaborations. So, on the basis of our
The second role, answering questions, has experience, the availability of a staff, composed
been considered useful because it has allowed by a suitable number of tutors, is essential: maybe
students to appreciate topics usually neglected. one tutor per 10-20 users could be appropriate. Of
It is commonly experienced by teachers the stu- course, the coordination among the teacher and
dents’ quite general assumption about questions the tutors has to be taken into account.
they consider tortuous when posed at the exams
0
communication and semiosis mathematics” and then their questions are standard
(e.g., “How a group is defined in Algebra?”) so
The activities described in the previous section are that the answers exactly conform to some piece
a good example of communication that involves of a book or lecture. Going on, students try to
the adoption of different registers (i.e., use-re- pose questions requiring some consideration for
lated linguistic varieties). The students have to different topics or registers or semiotic systems,
understand each other, but also to convey some with the obvious consequence that answers cannot
mathematical ideas. These two tasks may require exactly conform to the style of a textbook (e.g.,
different linguistic resources, and students have “Why the main coefficients of a conic after the
to switch between informal registers, in order to rotation are the eigenvalues of the original qua-
communicate each other as persons, and more dratic matrix?”, “Which is the relation between
formal ones, in order to describe mathematical the rank and the determinant of a matrix?”, “What
ideas. are the admissible representations of a vector
Looking at the files produced by the students space?”, “Is the intersection of two distinct planes
during the activities, we can find a range of ex- in R3 a vector space? Justify your answer.”). The
amples of conversion between different registers presence of non standard questions has been
and semiotic systems. The following two questions increasing as much as the activities have gone
posed by a student require conversion between an on, with an average of 45% on the total amount
informal register and a mathematical one: “Write of the questions. So, for one thing, this is a good
the Cauchy problem (in mathematical language)” advance in mathematical thinking, for another
and conversely “Explain by words the Cauchy thing, although they use a number of informal or
problem.” We also note that, even if the students even inaccurate expressions, students gradually
did not explicitly use graphical tools in their ac- improve their understanding of the meanings
tivities, some questions they posed involved some involved in mathematical expressions.
sort of figural representation, as shown by a ques- A platform, anyway, provides plenty of oppor-
tion like “In the Cauchy problem, which means tunities for designing communication situations
graphically the expression y’(x0)=y0?” which involving the use of a wide range of linguistic
requires to switch from an analytic expression resources.
to a graphical one and then to explain by words. More generally, ICT provides matchless oppor-
Furthermore, many of the students’ questions tunities for designing tasks involving conversion
regard the interpretation of symbols in a given of semiotic systems, as defined. The following
setting, such as “Which indicates cB(v)?”, “Which problem can be quite easily inserted as an item
represents the column j of the homomorphism in different e-learning modules.
representative matrix?”
If we try to trace the evolution of the use of 1. Consider the function f defined for any x∈
x5
language by the students through the activities by f(x) = − x.
described, we can say that at beginning the use 7
of language is seemingly more formal, and in a. Find f’(x) ......
some sense more precise from the mathematical b. Compute f’(0) ......
viewpoint. Actually, it is only a more rigid usage,
due to the fact that students are not used to “talk of Among the following graphs mark at least two
which do not match f. Explain.

A)
B)
C)

D)
A problem like this (administered to science has been made explicit, and it remains unwavering
freshman students) involves conversions between through the entire questionnaire. A considerable
formulas, graphs, and tables of values. It involves share of students actually expects an improvement
neither any advanced mathematical content nor in the relationship with the teacher, due to the
any sophisticated use of semiotic systems, but increased opportunity to communicate provided
it requires coordination of some piece of math- by the technological tools. We suppose that this
ematical knowledge and three different semiotic feeling of approaching (even if not physical) should
systems. Problems like this are hardly proposed be read as “it is beautiful to know that there is
in standard teaching activities if they are based someone.” In other words they greatly appreciate
on paper-and-pencil or blackboard work only. that the teacher is always at hand (by e-mail for
Nevertheless, they provide unique learning instance) if they wish or need. Through the plat-
opportunities from almost all the perspectives form the teacher is perceived closer, helpful, and
discussed in this chapter. so forth, and these factors have positive influence
on the motivation to study, on the involvement in
affective aspects: Students, the course and on understanding. In almost 50%
Teachers, and mathematics of the questionnaires, the students refer to their
expectation for wider, more frequent, and easier
The use of an e-learning platform as a support to opportunities to interact with the teacher. Such
a standard lecture-based course also affects emo- expectation is as strong as to be expressed anyway,
tional aspects. Some investigations (Albano, 2005) independently from the question posed: we might
have strongly pointed out students’ expectations be talking of either the course or their learning
and beliefs on their relationship with mathematics outcomes, or their relationship with mathematics,
and the teacher. The interviews have highlighted but in any case their expectation emerged in an
the importance of the role of the teacher as a tu- almost “intrusive” way! At the undergraduate
tor and as a guide for a proper use of technology. level, maybe this issue is felt as an important one
Otherwise, the computer may prove an obstacle if because of the larger number of students per class
the work is not properly supported by the teacher, compared to high school, which might weaken
because of the risk of getting lost due to the “dis- the relationship between teacher and student. So
persiveness” proper of the technological tools. We we can read their answers as a request for some
underline that even from the first question the contact with the teacher, who is, the students
expectation of a wider contact with the teacher feel, remote and missing. Tools as those offered

by the ICT not only make the students nearer to to underline that a teacher who uses a blended
the teacher, but induce them to communicate in course has been considered as a teacher who
a less formal, less rigid, “warmer” way. In other takes care of the learning of students, who wants
words, the relationship between teacher and stu- them to be successful in their learning outcomes,
dents becomes less asymmetrical. who wants to communicate with them. Thus, it
Note that the improvement of the quality of the positively affects students’ motivation and then
relationship between teacher and students greatly their outcomes: seeing the background activity
influence the relationship between students and of the teacher on the platform (such as materials
mathematics. Actually, 44% of the students claim updating, asynchronous interactions by e-mails
that the ICT-support, by itself, cannot change their and forum, etc.) let students feel encouraged
feelings about mathematics, but most of them and eager to learn. Moreover, being acquainted
think that the teacher can strongly influence their so as to communicate with the teacher can help
relationship with mathematics anyway. to reduce the exam-related anxiety, which often
This is true of the quality of the course too: a cannot be overcome by the mastery of the subject
teacher who does not love what is being taught only. Finally, the support offered by a blended
and who does not transmit passion to students course has proved an optimal help for students
is the main, or maybe the only, factor that can who failed previous exams. The benefits they got
“unqualify” a course. On the other hand almost not only affected their cognitive and metacogni-
20% of the learners states that a platform can tive state, but also improved their relation with
improve the quality of a course since it allows to mathematics.
improve the relationship teacher-student because
of a “direct contact” created (18.8%).
Anyway most of the learners (70%) expect Future reseArch dIrectIons
to progress in mathematics learning and perfor-
mance, thanks to the e-learning platform, because We plan to go on with research on personalisation
of the following main reasons: of teaching for students with learning difficulties.
Personalisation should take into account both
• Greater avalaibility of contents/investiga- specific content-related troubles and the student’s
tions/doubts/tests (37.2%) emotional profile. Currently platforms are often
• To be always in contact with the teacher used as Learning Content Management Systems,
(9.3%) that is, as managers of teaching resources which
• Course more interesting/practical/stimu- are labeled according to standard parameters
lant/new/involving (39.5%) such as kind of resource, school level, degree of
• Easy, fast, deep learning (23.3%) deepening, size of the resource, and so on. We are
going to adopt forms of labeling appropriate to
Further investigations on such expectations keep into account the instructional context more
have been carried out after attendance to the closely (Albano et al., 2006). This affects not just
blended course in order to compare students’ the amount of subject matter to be taught, but
expectations and the actual outcomes (Albano, also the teaching method. In our opinion almost
2006; Albano et al., 2007). It has been found that all the current research streams do not explicitly
the students’ expectations have been met quite deal with the emotional aspects of learning nor
satisfactorily. The use of an e-learning platform with the need for designing a wide range of learn-
really helps to create a relation with the teacher ing paths according to the “emotional profile”
that is quite lacking otherwise. We would like of each student. As pointed out by Di Martino

and Zan (2002), different attitudes profiles can Dynamical Geometry Systems. Actually, we
be associated to a certain belief and they require already use multiple representations, but they
different teaching actions. We mean to design are almost always pre-arranged by the teacher
teaching experiments (available to either indi- (e.g., test items involving graphs) and do not fully
viduals or cooperative learning groups) suitable exploit the opportunity of asking the student to
for specific emotional profiles and to investigate build the representations him/herself.
on the outcomes.
We also mean to investigate the viability of
open-ended remedial activities. This should be reFerences
done not by the teacher or the computer system
but through cooperative activities promoting Albano, G. (2005). Mathematics and e-learning:
reflection on errors or critical points. In a role- Students’ beliefs and waits. In International
play context (as described earlier) each student Commission for the Study and Improvement of
revises the wrong answers, but often does not go Mathematics Education 57 Congress, Changes
beyond detecting the error and providing a proper in Society: A Challenge for Mathematics Educa-
answer. From the interviews we have gathered, it tion (pp. 153-157). Piazza Armerina: Università
comes out that this role is the least interesting for di Palermo Press.
many students, as they, more often than not, pick
Albano, G. (2006). A case study about math-
the proper answer from some book. An activity
ematics and e-learning: First investigations. In
asking students to explain why a given answer
International Commission for the Study and
is to be considered wrong would be much more
Improvement of Mathematics Education 58
fruitful, as it involves:
Congress, Changes in Society: A Challenge for
Mathematics Education (pp. 146-151). Plezeň:
• Linguistic aspects (the linguistic form of
University of West Bohemia Press.
the answer)
• Cognitive aspects (processes of analysis Albano, G., Bardelle, C., & Ferrari, P. L. (2007).
and construction of knowledge, such as The impact of e-learning on mathematics educa-
construction of a counterexample) tion: Some experiences at university level. La
• Metacognitive aspects (awareness) matematica e la sua didattica, 21(1), 61-66.
Albano, G., Gaeta, M., & Salerno, S. (2006).
Activities of this sort cannot be carried out in
E-learning: A model and process proposal. In-
the frame of standard undergraduate lectures. On
ternational Journal of Knowledge and Learning,
the contrary, they can be planned and developed in
2(1/2), 73-88.
a virtual place such as a platform with cooperative
activities to be ended by an institutional meeting Balacheff, N. (2000). Teaching, an emergent prop-
(virtual o real) with the teacher. erty of e-learning environments. The Information
As far as semiotic systems are concerned, and Society for All. (IST 2000). Retrieved October
in the frame of online learning paths, we want to 21, 2007, from http://www-didactique.imag.fr/
investigate how to create interactive, open-ended Balacheff/TextesDivers/IST2000.html
tasks engaging students in “creative” activities
Balacheff, N., & Sutherland, R. (1999). Didactical
of construction, conversion, and treatment of
complexity of computational environments for the
semiotic representations within different semiotic
learning of mathematics. International Journal of
systems, in the setting of multiple-representation
Computers for Mathematical Learning, 4, 1-26.
systems such as Computer Algebra Systems or

Baldacci, M. (1999). L’individualizzazione. Kieran, C., Forman, E., & Sfard, A. (2001).
Basi psicopedagogiche e didattiche. Bologna: Learning discourse: Sociocultural approaches to
Pitagora. research in mathematics education. Educational
Studies in Mathematics, 46, 1-12.
Boud, D., Cohen, R., & Sampson, J. (1999). Peer
learning and assessment. Assessment and Evalu- Maragliano, R. (2000). Nuovo manuale di didat-
ation in Higher Education, 24(4), 413-426. tica multimediale. Editori Laterza.
Brousseau, G. (1997). Theory of didactical Moodle. (2006). Retrieved October 21, 2007, from
situations in mathematics. Kluwer Academics http://moodle.org/doc/
Publisher.
Perrin Glorian, M. J. (1994). Théorie des situa-
Cronbach, L., & Snow, R. (1977). Aptitudes and tions didactiques: Naissance, développement,
instructional methods: A handbook for research perspectives. In M. Artigue, R. Gras, C. Laborde
on interactions. New York: Irvington. & P. Tavignot (Eds.), Vingt ans de didactique des
mathématiques en France (pp. 97-147). Paris: La
Di Martino, P., & Zan, R. (2002). An attempt to
Pensée Sauvage.
describe a negative attitude toward mathemat-
ics. In P. Di Martino (Ed.), Proceedings of the Sfard, A. (2000). Symbolizing mathematical real-
Mathematics Views—XI European Workshop: ity into being—or how mathematical discourse
Research on Mathematical Beliefs (pp. 22-29). and mathematical objects create each other. In P.
Pisa: Università di Pisa Press. Cobb, E.Yackel & K. McClain (Eds.), Symbolizing
and Communicating in Mathematics Classrooms.
Duval, R. (1995). Sémiosis et pensée humaine.
Mahwah, NJ: Lawrence Erlbaum Associates.
Peter Lang.
Falchikov, N. (2001). Learning together: Peer
tutoring in higher education. Falmer Press.
AddItIonAL reAdIng
Ferrari, P. L. (2004). Mathematical language and
advanced mathematics learning. In M. Johnsen Psychology and mathematics
Høines & A. Berit Fuglestad (Eds.), Proceedings education
of the 28th Conference of the International Group
for the Psychology of Mathematics Education Bruner, J. (1986). Actual minds, possible worlds.
(Vol. 2, pp. 383-390). Bergen, Norway: Bergen Cambridge, MA: Harvard University Press.
University College Press.
Bruner, J. (1990). Acts of meaning. Cambridge,
Intelligent Web Teacher. (2006). Retrieved MA: Harvard University Press.
October 21, 2007, from http://www.momanet.
Dreyfus, T. (1991). On the status of visual rea-
it/english/iwt_eng.html
soning in mathematics and mathematics educa-
Johnson, D. W., & Johnson, R. T. (1987). Learn- tion. In F. Furinghetti (Ed.), Proceedings of the
ing together and alone: Cooperative, competi- 15thConference of the International Group for the
tive, and individualistic. Englewood Cliffs, NJ: Psychology of Mathematics Education, Assisi (I),
Prentice Hall. (Vol.1, pp. 33-48).
Jonassen, D. H., & Grabowski, B. L. (1993). Dubinsky, E. (1991). Reflective abstraction in
Handbook of individual differences, learning advanced mathematical thinking. In D. Tall (Ed.),
and instruction. Erlbaum, Hillsdale. Advanced mathemathical thinking (pp. 95-123).
Dordrecht: Kluwer.

Sfard, A. (2001). There is more to discourse than

meets the ears: Looking at thinking as communi- Information and communication
cating to learn more about mathematical learning. technology and e-Learning
Educational Studies in Mathematics, 46, 13-57.
Anderson, T., & Elloumi, F. (Eds). (2004).
Vergnaud, G. (1996). The theory of conceptual
Theory and practice of online learning. Atha-
fields. In L. P. Steffe, P. Nesher, P. Cobb, G. A.
basca University, ISBN: 0-919737-59-5. Retrieved
Goldin & B. Greer (Eds.), Theories of math-
October 21, 2007, from http://cde.athabascau.
ematical learning. Mahwah: Lawrence Erlbaum
ca/online_book/
Associates.
Conole, G., Dyke, M., Oliver, M., & Seale, J.
Vinner, S. (1997). The pseudo-conceptual and the
(2004). Mapping pedagogy and tools for effec-
pseudo-analytical thought processes in mathemat-
tive learning design. Computers and Education,
ics learning. Educational Studies in Mathematics,
43(1-2), 17-33.
34, 97-125.
Conole, G., & Fill, K. (2005). A learning design
Vygotskij, L. S. (1978). Mind in society: Develop-
toolkit to create pedagogically effective learn-
ment of higher psychological processes. Cam-
ing activities. Journal of Interactive Media in
bridge, MA: Harvard University Press.
Education, 08.
Vygotskij, L. S. (1986). Thought and language.
Jonassen, D. H., Howland, J., Moore, J., & Marra,
Cambridge, MA: The MIT Press.
R. M. (2003). Learning to solve problems with
Zan, R. (2000). A metacognitive intervention in technology: A constructivist persepective. Upper
mathematics at university level. International Saddle River, NJ: Merril/Prentice Hall.
Journal of Mathematical Education in Science
Keller, F., & Schauer, H. (2005). Personalization
and Technology, 31, 1.
of online assessments on the basis of a taxonomy
matrix. In Proceedings of the 8th IFIP World
representations and Language Conference on Computers in Education, WCCE,
Cape Town. Retrieved October 21, 2007, from
http://www.ifi.unizh.ch/ee/products/publications/
Gombert, J. é. (1992). Metalinguistic develop-
paper/WCCE05_Franziska_Keller.pdf
ment. Chicago: The University of Chicago Press.
(Original work published 1990). Khalifa, M., & Lam, R. (2002). Web-based
learning: Effects on learning process and out-
Halliday, M. A. K. (1985). An introduction to
come. IEEE Transaction on Education, 45(4),
functional grammar. London: Arnold.
350-3356.
Koper, R., & Tattersall, C. (Eds.). (2005). Learning
metacognition and noncognitive
design: A handbook on modelling and deliver-
Factors ing networked education and training, Springer
Verlag.
Morgan, C. (1998). Writing mathematically.
The discourse of investigation. London: Falmer Kramarski, B., & Gutman, M. (2006). How can
Press. self-regulated learning be supported in math-
ematical e-learning environments? Journal of
Pimm, D. (1987). Speaking mathematically: Com-
Computer Assisted Learning, 22(1), 24-33.
munication in mathematics classrooms. London:
Routledge, Kegan and Paul.

Nichols, M. (2003). A theory for e-learning.

Educational Technology & Society, 6(2), 1-10.
Retrieved October 21, 2007, from http://ifets.ieee.
org/periodical/6-2/1.html (ISSN 1436-4522)
Noss, R., & Hoyles, C. (1996). Windows on
mathematical meanings. Learning cultures and
computers. Kluwer Academic Publishers.
Tall, D., & Thomas, M. (1991). Encouraging versa-
tile thinking in algebra using the computer. Edu-
cational Studies in Mathematics, 22, 125-147.
Wiley, D. A. (2000). Connecting learning objects
to instructional design theory: A definition, a
metaphor, and a taxonomy. In D. A. Wiley (Ed.),
The instructional use of learning objects: Online
version. Retrieved October 21, 2007, from http://
reusability.org/read/chapters/wiley.doc

Chapter IX
AI Techniques for Monitoring
Student Learning Process
David Camacho
Universidad Autonoma de Madrid, Spain
Álvaro Ortigosa
Estrella Pulido
María D. R-Moreno
Universidad de Alcalá, Spain
ABstrAct
The evolution of new information technologies has originated new possibilities to develop pedagogical
methodologies that provide the necessary knowledge and skills in the higher education environment.
These technologies are built around the use of Internet and other new technologies, such as virtual edu-
cation, distance learning, and long-life learning. This chapter focuses on several traditional artificial
intelligence (AI) techniques, such as automated planning and scheduling, and how they can be applied
to pedagogical and educational environments. The chapter describes both the main issues related to
AI techniques and e-learning technologies, and how long-life learning processes and problems can be
represented and managed by using an AI-based approach.
AI Techniques for Monitoring Student Learning Process
IntroductIon and collaborative. The different research works

in the e-learning area led to the development of
The e-learning (Clark, 2001; Kozma, 1991; Meyen the IMS Learning design specification which is
et al., 2002) research field has become a hot topic currently used as a standard format for learning
in recent years. Many educators have seen it as designs (IMS LD, 2006). This specification is
a way to re-use previous courses stored in a da- based on a metalanguage which allows modelling
tabase, or in other electronic formats (Schmitz, learning processes. In IMS LD model concepts
Staab, Studer, Stumme, & Tane, 2002), and to like roles, activities, or environments are defined
give flexibility to existing ones. Moreover, the for describing learning designs.
increasing computing power and the available In higher education, the increasing tendency
network infrastructure allows sharing and dis- is to create virtual learning environments (VLE)
tributing these courses among public institutions which are designed to facilitate teachers the man-
and private corporations. These new educational agement tasks of educational courses for their
approaches are evolving to use the new informa- students. This increasing number of platforms,
tion technologies, and the Internet, as a virtual systems and tools related to virtual education has
platform where all the involved people can imple- led to the creation of different e-learning stan-
ment new ways of communication. dards. These standards, such as SCORM (2006),
Current e-learning techniques are modify- have been developed to facilitate the utilization
ing the traditional learning environment with a (and reutilization) of teaching materials (through
classroom, desktops with students, and a black- the definition and creation of learning objects).
board. These new techniques offer individualised Currently, these technologies and standards are
contents and learning methodologies, which mature enough to incorporate innovative tech-
traditional courses cannot provide, and allow niques that could provide new mechanisms to
advanced learners to speed through or bypass deal with learning processes.
contents that are redundant, whereas beginners The new virtual learning environments pro-
slow down through them (Small & Lohrasbi, vide an interesting field for different kinds of
2003). The progress made by each student can researchers. We will focus on artificial intelligence
be monitored in order to determine the main (AI) researchers that can experiment with their
problems that the students face when studying the automatic problem solving algorithms, or develop
units of a course. By knowing those problems, it is and design new algorithms in this complex do-
possible to propose e-learning activities that can main; and educational researchers that can use a
improve the quality of the learning process and, new kind of tools and techniques that could aid
as a consequence, improve the learning designs. to detect, reason, and solve (automatically) defi-
A learning design (LD) can be defined as an ap- ciencies detected in their initial learning designs.
plication of a pedagogical model for a specific One of the areas of AI most suitable to be applied
learning objective, a target group, and a specific within this context is the automated planning and
context or knowledge domain (Koper & Olivier, scheduling. Planning techniques generate a plan
2004). Different systems have been implemented (sequence or parallelization of activities) that
to help course designers to specify and imple- achieves a set of goals given an initial state and
ment LDs. Two examples are the open-source satisfies a set of domain constraints represented
system learning activity management system, or by operators schemas. In scheduling systems,
LAMS (LAMS, 2006), or the course management activities are organised along the time line by
system Moodle (Moodle, 2006), which supports having in mind the resources available. These
sequences of activities that can be both adaptive systems can perfectly handle temporal reasoning
0
and resource consumption, together with some lationships between them, and (b) the current
quality criteria (usually centred around time or situation of a given learner with respect to the
resource consumption) but they cannot produce whole learning process. These models will be
the required activities and their precedence rela- later considered, by using a particular e-learning
tions given that they lack an expressive language tool, to understand how traditional AI techniques
to represent activities. These techniques have can be incorporated into a particular e-learning
been applied with success in different real (and system.
complex) environments such as industry, robot- Several standards and guides have been pro-
ics, or information retrieval. Of special interest posed related to learning object metadata, student
in the last few years has been the development profiles, course sequencing, and so forth. The
of autonomous architectures (Muscettola, Dorais, IEEE Learning Technology Standards Committee
Fry, Levinson, & Plaunt, 2002) that can carry out (LTSC, 2006) has developed the learning object
a large number of functions, such as tracking a metadata (LOM, 2006) standard which specifies
spacecraft’s internal hardware or rover’s position, the attributes required to describe a learning
ensuring the correct working, and repairing when object, where a learning object is defined as any
possible, without (or little) human intervention. entity, digital or nondigital, which can be used,
In these new operation models, scientists and re-used or referenced during technology supported
engineers communicate the high-level goals to learning. Relevant attributes of learning objects to
the spacecraft or to the rovers, which are trans- be described include type of object, author, owner,
lated into planning and/or scheduling sequences. terms of distribution, format, and pedagogical at-
Then, a continuous status checking is performed tributes, such as teaching or interaction style. The
in order to detect any damage, and, finally, the standard also defines how LOM records should
plan is executed. These systems must also have be represented in XML and RDF.
the capability to understand that the errors oc- Promoting Multimedia Access to Education
curred during the process of accomplishing the and Training in European Society (PROMETEUS)
goals. This chapter shows how this kind of AI- tries to apply the IEEE LTSC standards into Eu-
based techniques can be appropriately used into rope context and cultures.
e-learning, and more specifically into virtual Another specification which allows the model-
education or VLE domains (Sicilia, Sánchez- ling of learning processes is the learning design
Alonso, & García-Barriocanal, 2006). We will (LD) information model (IMS LD, 2006) from
apply these techniques to a specific e-learning the IMS Global Learning consortium. A learn-
tool called Task-Vased Adaptive Learner Guid- ing design is a description of a method enabling
ance on the Web (TANGOW) developed by some learners to attain certain learning objectives
of the authors of this chapter (Carro, Pulido, & by performing certain learning activities in a
Rodriguez, 1999b). certain order in the context of a certain learning
environment.
LD is designed to integrate with other existing
rePresentAtIon FormALIsms In specifications. Among these, it is worth mention-
LeArnIng domAIns ing the IMS content packaging (IMS CP, 2006),
which can be used to describe a learning unit. A
In this section we will describe different formal- learning unit can have prerequisites which specify
isms that have been used in e-learning systems the overall entry requirements for learners to fol-
to represent (a) the learning area (domain model) low that unit. In addition, a learning unit can have
which includes the course concepts and the re- different components such as roles and activities.

Roles allow the type of participant in a unit of can be chosen among, for example, “completed,”
learning to be specified. There are two basic role “score less than,” or “time limit exceeded.” On
types: learner and staff. Activities describe the the other hand, an action could be, for example,
actions a role has to undertake within a specified “skip,” “disable,” or “hide from choice.”
environment composed of learning objects. The Aviation Industry CBT (Computer-Based
LD also integrates the IMS simple sequencing Training) Committee (AICC) is in charge of de-
(IMS SS, 2006), which can be used to sequence veloping guidelines for the aviation industry in
the resources within a learning object as well as the development, delivery, and evaluation of CBT
the different learning objects and services within and related training technologies.
an environment. Content is organized into a hier- In addition to these standards, there are other
archical structure where each activity may include specific proposals such as the GRAFCET repre-
one or more child activities. Each activity has an sentation formalism described in M’tir, Jeribi,
associated set of sequencing rules which describe Rumpler, and Ghazala (2004), which uses a graph
how the activity or how the children of the activity to represent the sequences of course concepts and
are used to create the desired learning experience. the possible learning itineraries.
The learning process can be described as the Other existing work (Ahmad, Basir, & Has-
process of traversing the activity tree, applying sanein, 2004) uses fuzzy logic to relate attributes
the sequencing rules, to determine the activities in the learner module and concepts in the domain
to deliver to the learner. model. The motivation for the use of fuzzy logic is
The general format of a sequencing rule can that it is appropriate for representing and reasoning
be expressed informally as: if condition set then with vague concepts and that the formalisation of
action. There may be multiple conditions. Condi- the level of understanding of a given concept by
tions may be combined with a single and combi- a learner is an inherently vague process.
nation (all conditions must be true) or a single or
combination (only one condition must be true).
Individual condition values may be negated before AutomAted Processes In
being combined in the rule evaluation. e-LeArnIng tooLs
The U.S. Federal Government Advanced
Distributed Learning (ADL) initiative has also Most of the current VLE contain prefixed courses
proposed a model called shareable courseware where the user navigates and learns the concepts
object reference model (SCORM) which describes that they have been planned for. Some e-learning
how the Department of Defense will use learn- tools include situation learning (SL) courses where
ing technologies to build, and use the learning the user is presented with different predefined
environment of the future. The standard defines situations where the user has to choose among
what is called “Learning Object Metadata,” which different options. The drawback of this type of
is a dictionary of tags that are used to described course is that nothing is dynamically generated
learning content in a variety of ways. For a given and a lot of effort is required to create challenging
learning object, these metadata describe, for situations that keep the user’s attention.
example, what the content is, who owns it, what Although the instructors can get statistics
costs (if any), technical requirements, educational as well as other information about the student
purpose, and so forth. The order in which learning progress, there is still a lack of feedback among
objects are presented to the learner is specified by the previous users, the tool, the instructors, what
using sequencing rules. A sequencing rule has the the user is interesting in, and the future users.
format if condition then action, where condition Among the tools that have worked on this direc-

tion we can mention the CourseVis system (Mazza • Domain theory: The STRIPS (Fikes &
& Dimitrova, 2003) and the dynamic assembly Nilsson, 1971) representation is one of the
engine (Farrell, Liburd, & Thomas, 2004). most widely used alternatives. A world state
An approach for automatic course genera- is represented by a set of logical formulae, the
tion (in some ways similar to the one presented conjunction of which is intended to describe
in this chapter) is the work of Ulrich (Ullrich, the given state. Actions are represented by
2005) who uses an AI hierarchical task network the so-called operators. An operator con-
(HTN) planner called JSHOP (Ilghami & Nau, sists of preconditions (conditions that must
2003) which assembles learning objects retrieved be true to allow the action execution), and
from one or several repositories to create a whole post-conditions or effects (usually consist-
course. Our approach not only can link learning ing of an add list and a delete list). The add
objects, but also schedule them along a period list specifies the set of formulae that are
of time and consider previous student results to true in the resulting state, while the delete
generate different learning designs. list specifies the set of formulae that are no
Since our goal is to monitor the learning longer true and must be deleted from the
process in TANGOW, the next subsections pres- description of the state. A course can be
ent a review of the main existing techniques: AI defined in terms of a set of learning activi-
Planning and Scheduling. ties that are performed by students.
• Problem: Described in terms of an initial
state and a goal. The initial state is repre-
IntroductIon to PLAnnIng sented by logical formulae that specify the
technIQues situation for which a solution is being looked
for. Examples of initial states in a learning
Planning can be defined as the sequence or paral- environment would be the students previous
lelization of activities that, given an initial state, knowledge, the resources that a course uses
achieves a set of goals and satisfies a set of domain and the time period when they are avail-
constraints represented as operators schemas. able, and so forth. Goals are often viewed
Using a high level description, the inputs of these as specifications for a plan. In a learning
systems are shown in Figure 1: environment, a possible goal would be that
the student is able to apply critical thinking
to a specific subject.
Figure 1. Inputs and outputs of an AI planner
Domain
description
Problem
Planner
description
Control
rules

• Some AI planners include a third input • An SAT-based planner takes a planning

referred to as control knowledge. It could problem as an input, guesses a plan length,
guide the solver to the right alternatives of and generates a set of propositional clauses
the search tree potentially avoiding back- that are checked for satisfiability. After the
tracking and arriving straight forward to translation is performed, fast simplification
the solution. algorithms are used to solve the problem.
• An HTN planner uses tasks networks
As an output, planners generate a plan with and tasks decomposition (methods). A task
the set of operators that achieves a state (from the network is a collection of tasks that need to
initial state) that satisfies the goals. be carried out, together with constraints on
The main AI planning techniques are de- the order in which tasks can be performed.
scribed next: The basic algorithm is to expand tasks and
resolve conflicts iteratively, until a conflict-
• A total order (TO) planner generates so- free plan can be found that consists only of
lutions that are sequences of total ordered primitive tasks.
actions. The basic structure is a tree where • A Markov decision process (MDP) is
nodes can be plans or states, and edges are defined by an initial distribution over the
actions or state transactions; then any search states, the dynamics of the system with states
algorithm can be applied. annotated by different possible actions, the
• In a Partial Order planner, nodes represent probabilistic state transitions, and a reward
partially specified plans, and edges denote function to make the transition from one state
plan-refinements operations such as the ad- to another. This kind of techniques requires
dition of an action to a plan. The planning full enumeration of all possible states what
algorithm commits to only the essential can make it intractable in most of the plan-
ordering decisions. There is no need to ning systems. Most of the work in this area
prematurely commit to a complete, total has focused on using only a subset (the most
sequence of actions. probable state space) or abstractions of the
• A Graphplan planner alternates between state space.
graph expansion and solution extraction. The • A contingent plan refers to a plan that con-
graph expansion extends the plan graphs tains actions that may or may not actually be
forward until it has achieved a necessary executed, depending on the circumstances
condition for plan existence. The solution that hold at the time. Another way to handle
extraction phase performs a backward- uncertainty is by applying probabilistic plan-
chaining search on the graph, looking for a ners which use probabilities of the possible
plan that solves the problem. If no solution uncertain outcomes to construct plans that
can be found, the cycle repeats the expansion are likely to succeed.
of the planning graph.
• A heuristic search planner (HSP) trans-
forms planning problems into problems of IntroductIon to scheduLIng
heuristic search by automatically extracting technIQues
heuristics functions from STRIPS encoding
instead of introducing them manually. The Scheduling can be defined as the process of
bottleneck is the computation in every new organising activities along the time line by tak-
state of the heuristic from scratch. ing into account the resources available. Many

techniques used in the area of scheduling systems have to be executed in parallel, so there are no
come from the operational research (OR) area (i.e., conflicts. If we impose a deadline of 5 time units
branch and bound, simulated annealing, lagrang- to the original problem (we consider that each
ian relaxation). Lately, constraint programming activity has a duration equal to 1), the solution
(CP) has been applied to the different scheduling given by the scheduler will be also time consis-
problems with very good results, that is, job-shop tent. However, a value lower than 5 will make the
scheduling and the RCPSPmax problem (Kolisch solution inconsistent.
& Hartmann, 1999). A RCPSPmax consists of a Then, scheduling techniques can be easily
set of activities where two kinds of constraints generalized and applied to a learning environment.
can be interrelated: In this case, instead of having machines and jobs
(Job-Shop Scheduling problem), we have students,
• Precedence constraints that impose the educators, and learning units (LU) in courses.
restriction that an activity cannot start before Each learning unit (operation) needs to be
its predecessor activities, and processed during a period of time for a given
• Resource constraints among activities student (machine), and the unit will be super-
that consume the same resource due to the vised by an educator. The course will also have
limited capacity of the resource itself. a limited duration (deadline). Each instructor
will have a maximum number of students (we
The objective is to find precedence and re- consider an instructor as a resource with a total
source assignments for all the activities in the resource capacity given by the number of students
horizon imposed. Figure 2 shows a simple ex- the instructor is able to advise). We need to know
ample of a Job-Shop Scheduling problem with the initial and end time of each LU considering
two resources: resource R1 with a capacity of 2, precedence constraints among them. The variable
and resource R2 with a capacity of 3. The left part values are imposed by the problem conditions:
of the figure shows the precedence constraints learning activity durations, course duration,
among activities and the resources that each one number of learners, and so forth. AI integrated
requires. The right part of the figure shows the planner and scheduler systems generate a plan
solution to the problem. Since R1 has a maximum or a set of plans if a solution exists for the given
capacity of 2, the 3 activities that consume this deadline. A plan can be seen as a sequence of
resource cannot be performed in parallel. Then, operator applications (learning activities) with
the scheduler will add a precedence constraint to a specific duration that can lead from the initial
one of them. Resource R2 has a capacity of 3 but state to a state in which the goals are reached with
none of the activities that require this resource the resources available.
Figure 2. A job-shop scheduling example with two resources
Resource capacity R R
R R
R R
R R R R R R R
R R
R

In a specific learning design, we need to impose systems, as taught to second year students of a
a deadline, that is, the total duration of the course, computer science degree. The current version
and the resources that are available, that is, the of the course is composed by a number of tasks
number of educators. Then, it is up to the educator representing theory units and also examples.
and the pedagogical responsible to study the best The example shows how tasks are decomposed
way to distribute the number of hours and their into subtasks according to specific rules. The
contents among the different units in order to as- complete course consists of four tasks: opera-
sure the quality of the education process. This task tive system overview, operative system concepts,
can be done automatically by applying Planning distributed systems and security. These tasks are
and Scheduling techniques to a new domain: e- combined through an AND rule, which states that
learning environments. Although the process will all the subtasks must be completed exactly in the
be explained in detail in the following sections, order in which they are listed in the rule. The
the basic idea is to change some parameters and operative system overview task is divided into
to use the feedback from the students that have services, security, and architecture subtasks, com-
already followed the course. bined through an OR rule. The OR rule dictates
that, in order to complete the composed task, at
least one of the subtasks must be completed, in any
tAngow: A tooL For vIrtuAL chronological sequence. Differently, the operative
educAtIon system concepts task is decomposed, through an
ANY rule, into the process, memory, scheduling,
TANGOW facilitates the development and de- input-output, and file subtasks. An ANY rule
ployment of adaptive courses. In these courses means that all the subtasks must be completed
the contents, the navigational options, and the but the order is not relevant. Following with the
flexibility of the guidance process are adapted course, it can be seen that, in order to execute the
to both the user features and their actions while Scheduling task, the student has to sequentially
interacting with the course. Adaptivity is an execute the scheduling principles and scheduling
important feature because the lifelong learning algorithms tasks. Scheduling algorithms consists
philosophy is growing in importance in many of the study of five specific algorithms in any
environments and the same virtual education order. Finally, the task round-robin algorithm
course can be accessed by students with different can be learned by reading either the description
backgrounds, age, and interests. or any of the three examples provided.
Teachers can describe adaptive courses by Besides the correct sequencing, the rule may
means of tasks and rules. Tasks are the basic units state conditions that must be fulfilled in order
in the learning process. They include topics to be for the rule to be applied. These conditions are
learned, exercises to be done, examples to be ob- expressed by means of attribute values regarding
served, and so forth, that is, tasks to be performed user features, such as personal characteristics
in order to learn or put into practice the concepts (age, language, experience, etc.), learning style
or procedures involved within the course. Rules (visual/verbal, intuitive/deductive, etc.) (Paredes
specify the way of organizing tasks in the course & Rodriguez, 2002), preferences (type of informa-
along with information about the task execution tion desired, learning strategy, etc.), and actions
(order among tasks—if any—free task selection, while interacting with the course (tasks visited,
prerequisites among tasks, etc.). exercises performed, results obtained in the tests,
Figure 3 presents an example of the (partial) time spent in every task, and so on). The latter
structure of a TANGOW course on operative type of attributes are called “dynamic attributes”

Figure 3. Partial example of a TANGOW course structure
Operative System course
Operative System Overview

Services
or Security overview
Architecture
Operative System Concepts

Processes
Memory
And Scheduling
And Scheduling principles

Any Scheduling algorithms
FCFS
Any Round-Robin
Description
or Example
Example
Example
SPN
SRT
HRRN
Input-Output
Files
Distributed Systems
Security
as they must be calculated during the student Because rules can depend on dynamic at-
interaction. In this way, prerequisite relationships tributes, the decomposition of the next task can
between tasks, for example, can be specified in only be computed just on time when the student
rule activation conditions. Moreover, an educa- selects the given task. In this way, the TANGOW
tor can describe different course structures and system consults the course description and the
dependencies for different students, by specifying data about the student and generates, step by step,
several alternative rules for the same composed a personalized course for each student, adapting
task. When a student selects the new task to be the different course aspects to each student.
carried out, the systems looks for an appropriated
rule (that is, a rule whose conditions are true for the Authoring a tAngow course
current student) describing how the task must be
decomposed. Figure 4 shows an example of three When designing an adaptive course for TANGOW,
different rules describing possible decompositions the first step is to establish the user features to
for the operative system overview task, each one be considered for the adaptation. The attributes
suitable for students with different characteristics. selected to be used in rule conditions compose the
In this case, the considered feature is whether user model. These data are stored in the student
the student is preparing to become an end user, database along with the log files containing the
an operating system designer or an application sequence of actions performed by the students.
programmer. For example, regarding the operative system

Figure 4. Task decomposition based on student characteristics
end user Operative System Overview

Services
And
Security overview
os designer Operative System Overview
Architecture overview
Services
or Security overview
Application
Operative System Overview Architecture
Programmer
Services
Any
Security overview
course analyzed in Figure 4, the relevant student order to adapt the course to student features and
feature is the student role (end user, application actions.
programmer, OS designer). The list of visited Each task will be defined as atomic or com-
tasks will also be needed, as there are prerequisite posed. Atomic tasks will be the leaves of the task
relationships between some of them (for example, tree, while composed tasks are the inner nodes
processes is required for scheduling algorithms, and will have one or more rules describing how it
Figure 3). is decomposed into subtasks. Table 1 shows some
Afterwards, the designer describes the adap- rules describing how composed tasks should be
tive course itself (Carro, Ortigosa, & Schlichter, divided into subtasks.
2003; Carro, Pulido, & Rodriguez, 1999a) by
specifying the tasks and rules that will be part tAngow Logs
of the course, as well as the contents (generally
HTML files) associated with each task and used While the student is interacting with the course,
for page generation. The designer can specify all of the student’s actions are logged. This log
different variations of any of these aspects in stores information about the tasks the student has
Table 1. Examples of rules for the operative system adaptive course
task name conditions subtasks sequencing
… … … …
‘Operative System role = ‘End user’ ‘Services’, ‘Security AND

Overview’ Overview’, Architecture
Overview’
‘Operative System role = ‘Application ‘Services’, ‘Security ANY
Overview’ programmer’ Overview’
‘Operative System role = ‘OS Designer’ ‘Services’, ‘Security OR

Overview’ Overview’, ‘Architecture’
… … … …

Figure 5. A portion of a TANGOW log file
<log-root>
<user-model>
...
</user-model>
<log-activity>
<log activity="Operative System Overview" action="START-ACTIVITY" timestamp="00--T0::00"/>
<log activity="Services" action="START-ACTIVITY" timestamp="00--T0::"/>
<log activity=" Services" score="0.“ action="END-ACTIVITY" timestamp="00--T:0:"/>
...
</log-activity>
</log-root>
visited, the corresponding time stamps, the level for new solutions (Camacho & R-Moreno, 2007;
of completeness, and the score obtained, when it R-Moreno & Camacho, 2007). This methodology
applies. Figure 5 displays a partial example of a has been implemented in a planning and schedul-
student interacting with the adaptive course. ing (P/S) system called IPSS (R-Moreno, Oddi,
Log files enable the course designer to retrace Borrajo, & Cesta, 2006). In this section, details
the interaction of the student with the system/ will be presented about how the IPSS and the
course, and can be used with different goals. For TANGOW systems have been integrated and the
example, Ortigosa and Carro (2003) use the in- advantages obtained with this integration.
formation contained in the log files to provide the
course designer with information about possible tAngow & IPss Integration
problems or improvement opportunities within
the adaptive course. In IPSS (R-Moreno et al., 2006), reasoning is
divided into two levels. The planner module
(IPSS- P) focuses on the action selection, and
monItorIng tAngow courses: the scheduler module (IPSS-S) on the time and
A cAse study resource assignment. Figure 6 shows in more
detail how the different modules (layers) of IPSS
The e-learning methodology proposed uses auto- interact. Since our planner is a total order planner
mated reasoning techniques, such as planning and (the solution is a sequence of activities), it is not
scheduling, to automatically learn from possible enough to look for a solution that minimises the
mistakes in the learning design process and to look time and the resources. We use a de-order algo-
Figure 6. The IPSS architecture

rithm (Bacstrom, 92) to eliminate the unnecessary means that the task is considered as
causal links. Then, the solution generated by the essential in the learning process).
de-order algorithm is given to the temporal and c. The teacher selects the number and
resource reasoners. During the search process, kind of dimensions that will be used to
every time the planner chooses an operator, it generate the metadata. In our example,
consults the scheduler for the time and resource the teacher has selected the knowledge-
consistency. If the resource-time reasoner finds student level (end user, application
the plan inconsistent, then the planner backtracks. programmer, OS Designer).
If not, the operator is applied, and the search 2. The students interact with TANGOW. These
process continues. To know more details about interactions generate different logs that will
the algorithm, readers can refer to (R-Moreno, be stored in the system.
2003; R-Moreno et al., 2006). 3. By using the above-mentioned information
Figure 7 shows the architecture of the system (students logs, teaching tasks, rules, task
that results from the integration of the IPSS and priorities, time estimation) the metadata is
the TANGOW systems. The monitored learning generated based on both logs and educator
process can be described as follows: estimations.
4. The metadata is appropriately mapped into
1. The educators define: an IPSS representation. This mapping pro-
a. The teaching tasks and rules to build cess generates the domain and the initial
the adaptive course (by using the problem that will be used by IPSS to solve
TANGOW tool). the defined problem.
b. The educator assigns to each task 5. IPSS looks for solutions that solve possible
both a priority and a time estimation problems existing in the initial learning
associated to the task (a high priority design by taking into account the teaching
Figure 7. IPSS integration for a specific TANGOW course
TANGOW
Student Data
Course Student
Contents Logs
Teaching Tasks Rules

Log-based Metadata
Students Priority-based Metadata
P/S metadata-based mapping
P/S Domain
(End user, Appl. programmer, OS Designer )

<stud-type, task, dur-min,dur-max>
<task, time, priority>
Planner/Scheduler
Educators
IPss
Dimensions
Sol
P/S Problem
novice
Sol
normal
Priority/Time module
Sol
(teaching Tasks) advanced
Solutions found (proposals)
0
task decomposition, their priorities and The first step for defining a domain consists of
estimated duration time. identifying the operators and the object types that
6. For each kind of student (end user, application are needed in the domain (for declaring the type
programmer, OS designer) a plan is gener- of each operator variable). Types can be defined
ated with a possible scheduled course. as structured in a hierarchy. A special type, the
infinite type, allows representing continuous val-
The following subsections describe the pre- ued variables, while finite standard types represent
viously listed processes in detail by using the nominal types. In our domain, we have, among
TANGOW course example of Figure 3. others, the following types: STUDENT, ROLE,
SUBTASK, COURSE, DURATION, PRIOR-
From tAngow metadata to ITY, and TIME. Variables of type STUDENT
P/s-Based Problems instantiate to the possible student stereotypes.
The variable ROLE represents the relevant user
As mentioned, the planner domain theory con- features (end user, application programmer, OS
tains all the actions represented by operators. The designer) and COURSE represents the courses that
language for describing an IPSS domain theory we want to track. Finally, DURATION, PRIOR-
is based on an augmentation of the representation ITY, and TIME are infinite types that allow us to
originally proposed by Fikes and Nilsson (Fikes handle numerical values needed to calculate the
& Nilsson, 1971). Since this representation is duration and priorities of each task. By following
quite restrictive, it has been extended to allow the TANGOW example described in Section 4
disjunctive preconditions, conditional effects and (Figures 3, 4, and 5 and Table 1), we can obtain
universally-quantified preconditions and effects, the needed metadata for IPSS (see Table 2).
quality metrics, durations, time and resource The IPSS operator in Figure 8 is composed of
constraints, and continuous values. the following fields:
Table 2. TANGOW metadata
subtasks
(min,max)
Duration
Priority
task name conditions sequencing

Task
… … … …
‘Operative System role = ‘End user’ ‘Services’ , AND

Overview’
‘Security Overview’ ,
‘Architecture Overview’ ,
‘Operative System role = ‘Application programmer’ ‘Services’ , ANY
Overview’
‘Operative System role = ‘OS Designer’ ‘Services’ , OR
Overview’
‘Architecture’ ,
… … … …

• Params field: Contains a list of the variables “Services,” “Security Overview,” and “Architec-
whose values will be printed out through the ture Overview” substasks. These subtasks are
user interface when a solution is found. represented within the operator as the variables
• Preconds field: Ccontains the preconditions <tk>, <tk1> and <tk2>. The <role> variable will
of the operator. be instantiated by the End_user value.
• Effects field: contains the add and delete The variables <pt>, <pt1>, and <pt2> represent
effects of the operator. priorities and can take numbers as values. We
• Constraints field: Contains the temporal need to use the gen-from-pred IPSS function to
constraints. constraint the values that these three variables
can take. This function generates a list of values
The symbols within < > are variables that are as the bindings for a variable by using the infor-
instantiated during the problem solving process. mation on the current state. In this example, in
The operator in Figure 8 has two preconditions: the case of <pt>, gen-from-pred returns the list
(compose-subtask End_User <tk> <tk1> <tk2>) of values {x} greater or equal to 1 such that the
and (student-role <st> <role>) and three add effects literal (prioritySe End_user Services x) is true in
(done <st> <tk>), (done <st> <tk1>) and (done <st> the current state.
<tk2>). This operator, called “T_EU_Operating- The variables <dur>, <dur1>, and <dur2>
System_Overview,” corresponds to the “Operat- represent task duration. As the values they can
ing System Overview” task under the “End User take are lists of two elements corresponding to the
role” condition in Table 2 which is composed of minimum and the maximum duration, we need
Figure 8. An IPSS operator corresponding to the “operating system overview” task

to use the constraint-from-pred IPSS function will force to modify the subsequent tasks, see
to constraint the values that these three variables Figure 10).
can take. This function generates a list of values Figure 10 shows the execution of different
to be possible bindings for the corresponding teaching tasks proposed by the planner. Any task
variable by using the information of the cur- could be under or over estimated depending on
rent state referred to the duration of each task external factors (students skills, labs/classrooms
that compose a specific task. IPSS will choose availability, and so forth). Although the total
a possible integer value in that range during the duration (i.e., makespan) of the course is fixed,
problem resolution. IPSS can replan the remaining tasks in order to
In order to implement static properties ef- fit them in the remaining time. IPSS will either
ficiently, IPSS allows user-defined functions to increment/decrement the duration of the tasks
represent them. In our example, we have defined based on the priorities, or add/delete subtasks
the function diff, that allows us to calculate if the based on the AND, OR, and ANY rules. Figure
objects passed as arguments are different; and the 9 shows a possible course execution scenario, in
function calculate-total-duration, that permits which Task1 was under estimated by the educators
encoding the function: time= dur+dur1+dur2. (when accessing the course, it took longer to stu-
The second input to the planner is the problem dents to execute the task). The new time duration
to be solved, described in terms of an initial state (including a time increment) will be managed by
and a set of goals to be achieved. The description the planner/scheduler in the next cycle to adjust
of an initial state is composed of a list of objects the time duration of the rest of tasks that have not
and their corresponding types together with a set of been executed yet. The planner/scheduler main
instantiated predicates (i.e., literals) that describe objective is to execute all the teaching tasks in the
the configuration of those objects. The objects in available time. For this reason, over estimations
the state must be instances of the types that are (i.e., Task 2) and under estimations (i.e., Task 1)
declared in the domain. Figure 9 shows some initial will be sequentially used in every planning/sched-
conditions and two goals corresponding to two uling cycle to (dynamically) adapt the available
student stereotypes. The goals are that learner1 time. This adaptation could result in the addition
whose student role is end user and that learner2 or removal of low priority teaching tasks.
whose student role is application programmer On the other hand, if a particular course has
must learn the Operative System course. been executed several times, we can access the
student logs to look for a <min, max> estimation
for the time duration of a particular task. If the
monitoring tAngow courses majority of students a task requires less time
than estimated by the educator, this means that
Once the metadata is generated, there are two the next time the students start the course we can
different planning/scheduling possibilities. On consider that duration as the baseline duration.
one hand, it could be the first time that this course This information will be given to IPSS that will
is executed. In this situation, the planner does adjust the whole course based on that change.
not have information about how much time is In this way, IPSS will provide a better initial
necessary for a particular task (we use the educa- solution in the next course execution. The same
tor time estimation). The system will propose a will occur if the time assigned to a task by the
schedule that can be dynamically modified once educator is underestimated. From the log files
the interactions with the students provide the we can extract that information and translate it
initial time durations for the tasks (a teaching into IPSS to reschedule the whole course. If a
task that requires more than the scheduled time rescheduling is needed, two things can happen:

Figure 9. An example of IPSS initial conditions
(objects
(learner1 learner2 STUDENT)
(End_user Application_programmer OS_Designer ROLE)
(Services Security_Overview Architecture_Overview SUBTASK)
(Operating_System COURSE))
(state
(and
; SubTasks that compose the tasks depending on the student role
(compose-subtask End_user Services Security_Overview Architecture_Overview)
(compose-subtask Application_programmer Services Security_Overview)
;...
; Features for each student sterotype

(student_role learner1 End_user)
(student_role learner2 Application_programmer)
; Priotities for each substask

(prioritySe End_user Services 1)
(prioritySO End_user Security_Overview 2)
(priorityAO End_user Architecture_Overview 3)
(prioritySe Application_programmer Services 1)

(prioritySO Application_programmer Security_Overview 2)
; ...
; Durations for each substask

(durationSe End_user Services (2 4))
(durationSO End_user Security_Overview (3 6))
(durationAO End_user Architecture_Overview (5 7))))
(durationSe Application_programmer Services (2 3))

(durationSO Application_programmer Security_Overview (2 4))
(durationSe OS_Designer Services (2 3))

; ...
(goal
(and (Learn learner1 Operative_System)
(Learn learner2 Operative_System)))
Figure 10. Sequential teaching tasks execution
Time Available
Time duration Time duration
task1: task., task.,… task2: task., task.,… task3 taskn-1 taskn
Time estimation Time estimation

Figure 11. Cyclic TANGOW course execution
)
on
mati
esti
h er
ac
: te
(ta
e
sk
tim
sk TANGOW course (nd ex)
tim
(ta
e:
s
tud
en
TANGOW course (st ex)
t lo
gs
)
T … Tn Nth ex
TANGOW course (rd ex)
th ex )
s…
log
&
on
ati
th ex stim
e
h er
ac
e : te
tim
sk
(ta
all the tasks can still be fit into the total time as- instantiated) have prerequisite relationships with
signed to the course or they cannot. In the latter others already performed, imposes the restriction
case, IPSS will have to eliminate one or more that the starting time of a task must be later in time
tasks from the course. to the ending time of its prerequisite tasks. This is
In both situations, with and without previous the case of the “T_EU_OperatingSystemConcept”
students information, IPSS can use the avail- and “T_EU_OperatingSystemOverview” tasks in
able information (in the first case a worse time Figure 11. The starting time of “T_EU_Operat-
assumption will be assumed) to schedule tasks. ingSystemConcept” is equal to the ending time of
Figure 10 shows how the cyclic execution of a “T_EU_OperatingSystemOverview”. These tasks
particular course can be used by IPSS to itera- belong to the tasks that have to be performed by
tive improve the quality of the plans. A proposed the student with the “End User” role (see Table
schedule will have a higher quality if it is better 2). Since the Sequencing is an “AND,” all the
adapted to both the students characteristics and tasks must be performed sequentially. But these
the available time. prerequisite relationships may not exist between
other tasks such as the “T_AP_OperatingSyste-
mOverview” and “T_EU_OperatingSystemOver-
the IPss Final solution view” tasks that can be executed in parallel. The
“T_AP_OperatingSystemOverview” task will
Finally, IPSS generates a plan with the sequence be performed by a student with the “application
of operators that achieves a state (from the initial programmer” role that has no conflicts with the
state) that satisfies the goals and their start and end tasks performed by the “end user” student.
times. Figure 12 shows the solution generated from
the initial conditions of Figure 9. In the solution,
IPSS instantiates each operator (that is, each task concLusIon
in TANGOW) by giving value to the task starting
and ending times, and to the subtask list that com- In the first e-learning systems, any student,
pose the task. The fact that some tasks (operators no matter what the student’s personal features

Figure 12. The IPSS final solution
were, was presented with the same materials, the in the TANGOW log files in order to refine the
same exercises were proposed to him and in the initial learning design by adjusting the duration
same order. The next generation of e-learning and the ordering of the activities proposed to the
environments are those that are able to adapt the students accessing the e-learning system. For
deployed course to the features and actions of the the integration, the TANGOW rules and condi-
student. In this way, every student will follow an tions are translated into IPSS operators, and the
individualised course specifically deployed for the TANGOW attributes into IPSS types to produce
student. We can refer to this kind of adaptation a plan that corresponds to a course instance (tasks
as an individual adaptation which is performed dependency).
based on the individual characteristics and actions Consequently, the deployed course adapts itself
of each student accessing the system. not only to the personal features and actions of
This chapter shows how it is possible to go every student who accesses the course, but also
one step further in the development of e-learning to the global actions of a group of students. With
systems and implement a group-based adaptation this approach the courses generated are automati-
based on the actions not of an individual student cally validated avoiding inconsistencies in linking
but of a set of students who have accessed the activities, durations given to each activity and
system along a period of time. The basic idea is the total duration of the course, saving time to
to register student actions when interacting with the educators. Since the planning and schedul-
an e-learning course based on an initial learning ing techniques can be applied to the collected
design. A Web-based learning system called log files as many times as required, the process
TANGOW is used for this purpose. Then, plan- of improving and refining becomes a long-life
ning and scheduling techniques as implemented in process that will only stop when the designer or
the IPSS system are applied to the data collected tutor considers it suitable.

Finally, we also want to mention the issues is a simple way to control and monitor a particular
that AI planners can gain with this approach. course. In this way, educators can easily detect
Generally, to specify the domain theory, a deep hidden problems and improve, and reach, their
understanding of the way AI planners work and final goals.
its terminology is needed. However, if we use a However, in the near future a new kind of
tool like TANGOW, the description language is adaptive intelligent education-based tools will
closer to the user and allows an automatic verifi- be completely designed and developed using
cation of the syntax through a friendly interface. this kind of technique (other related AI-based
Also, a new domain to apply and develop new AI techniques such as machine learning are cur-
algorithms has being created. rently used to learn students profiles and take
We are currently on the process of testing learning decisions). In these new tools it will be
the proposed approach. In a first stage we are possible to control, monitor, and automatically
working with synthetic logs. The next stage will solve, several detected problems in the learning
involve empirical tests with real users to verify designs deployed. These problems could be au-
improvements on the course. tomatically detected (by logging the interactions
with the students, and analysing the quantitative
results from tests and exams), or provided directly
Future reseArch dIrectIons by educators and learning designers, as we have
shown previously. These new tools will be able
The current state in computer-based education to automatically evaluate, through the long-life
technologies, tools, and standards provides some learning process of a particular course, the
new interesting perspectives to other research ar- problems and automatically modify the learning
eas like artificial intelligence. The well established designs in order to smooth them.
standards, such as IMS, SCORM, or LOM, are To achieve the previous goal it will be neces-
currently being used to define and develop new sary to adapt and integrate, well known Artificial
adaptive virtual-based education tools. These Intelligence techniques, such as Automated plan-
tools support the creation of personalized learn- ning or scheduling, which allow us to deal with
ing designs (LD). With these new designs it is problems like resource assignment or the organi-
possible to reuse and exchange useful information sation of different activities (cost, duration, time)
among different platforms. These new tools can in a particular time period. Therefore, it will be
be used by educators (and/or learning designers) necessary to define adequate mechanisms to trans-
not only to define the contents of the course (i.e., late correctly from the e-learning standards into
using the IMS LD specification), but also to cre- the planning and scheduling standard language
ate adaptable and personalized learning flows, representation. In this chapter we have presented
so that the educational system can monitor and an initial approach, based on the adaptive virtual
control the whole learning process. This chapter education and authoring tool used.
has described a particular authoring tool (TAN-
GOW) that can be used to achieve the previous
goal based on teaching rules, and has shown how AcknowLedgment
a particular AI system that integrates planning
and scheduling techniques can be used to improve This work has been funded by the follow-
the learning quality of a particular course. In our ing research projects: TSI2006-12085, UAH
approach the term quality is used to describe the PI2005/084, PAI-0054-4397, TSI2005-08225-
fact that suggesting one or several modifications, C07-06 and TIN2004-03140.

reFerences Ilghami, O., & Nau, D.S. (2003). A general ap-

proach to synthesize problem-specific planners
Ahmad, A., Basir, O., & Hassanein, K. (2004). (Tech. Rep. CS-TR-4597). University of Maryland:
Adaptive user interfaces for intelligent e-lear- Department of Computer Science.
ning: Issues and trends. In Proceedings of the 4th
IMS CP. (2006). Retrieved October 22, 2007, from
International Conference on Electronic Business
http://www.imsglobal.org/content/packaging/
(ICEB2004) (pp. 925-934).
IMS LD, IMS Learning Design. (2006). IMS
Camacho, D., & R-Moreno, M.D. (2007). Towards
Global Learning Consortium. Retrieved October
and automatic monitoring for higher education le-
22, 2007, from http://www.imsglobal.org/learn-
arning design. International Journal of Metadata,
ingdesign/index.html
Semantics, and Ontologies, 2(1), 1-10.
IMSSS, IMS Simple Sequencing. (2006). Re-
Carro, R.M, Ortigosa, A., & Schlichter, J. (2003).
trieved October 22, 2007, from http://www.ims-
A rule-based formalism for describing collabora-
global.org/simplesequencing/index.html
tive adaptive courses, KES2003. Lecture Notes
in Artificial Intelligence, 2774, 147-178. Kolisch, R., & Hartmann, S. (1999). Heuristic
algorithms for solving the resource-constrained
Carro, R.M, Pulido, E., & Rodríguez, P. (1999a).
project scheduling problem: Classification and
Designing adaptive Web-based courses with
computational analysis. Project scheduling:
TANGOW. In G. Cumming, T. Okamoto & L.
Recent Models, Algorithms and Applications,
Gómez (Eds), Advanced research in computers
147-178.
and communications in education (pp. 147-178).
Amsterdam: IOS Press. Koper, R., & Olivier, B. (2004). Representing the
learning design of units of learning. Educational
Carro, R.M., Pulido, E., & Rodríguez, P. (1999b).
Technology & Society, 7(3), 97-111.
Dynamic generation of adaptive Internet-based
courses. Journal of Network and Computer Ap- Kozma, R. (1991). Learning with media. Review
plications, 22, 249-257. of Educational Research, 61(2), 179-212.
Clark, R.E. (2001). New directions: Evaluating LAMS. (2006). Learning Activity Management
distance education technologies. In R.E. Clark System. Retrieved October 22, 2007, from http://
(Ed.), Learning from media: Arguments, analy- lamsfoundation.org/
sis, and evidence (pp. 125-136). Greenwich, CT:
LOM. (2006). Retrieved October 22, 2007, from
Information Age Publishing.
http://ltsc.ieee.org/wg12/
Farrell, R., Liburd, S.D., & Thomas, J.C. (2004).
LTSC. (2006). Retrieved October 22, 2007, from
Dynamic assembly of learning objects. In Pro-
http://ieeeltsc.org/
ceedings of 13th International World Wide Web
Conference, NY. Mazza, R., & Dimitrova, V. (2003, July 20-24).
CourseVis: Externalising student information to
Fikes, R., & Nilsson, N. (1971). Strips: A new
facilitate instructors in distance learning. In U.
approach to the application of theorem proving
Hoppe, F. Verdejo & J. Kay (Eds.), Proceedings
to problem solving. Artificial Intelligence, 2,
of the International conference in Artificial Intel-
189-208.
ligence in Education, Sydney, Australia.

Meyen, E.L., Aust, R., Gauch, J.M., Hinton, H.S., and scheduling integration. IEEE Transactions
Isaacson, R.E., Smith, S.J., et al. (2002 ). E-learn- on Knowledge and Data Engineering, 18(12),
ing: A programmatic research construct for the 1681-1695.
future. Journal of Special Education Technology,
Schmitz, C., Staab, S., Studer, R., Stumme, G.,
17(3), 37-46.
& Tane J. (2002). Accessing distributed learning
Moodle. (2006). Retrieved October 22, 2007, from repositories through a courseware watchdog. In
http://demo.moodle.com/ Proceedings of the E-Learn 2002-World Confer-
ence on E-learning in Corporate, Government,
M’tir, R.H., Jeribi, I., Rumpler, B., & Ghazala,
Healthcare for Higher Education.
H.H.B. (2004). Reuse and cooperation in e-learn-
ing systems. In Proceedings of the Fifth Interna- SCORM. (2006). Sharable Courseware Object
tional Conference on Information Technology Reference Model. Retrieved October 22, 2007,
Based Higher Education and Training, ITHET from http://www.academiccolab.org/projects/
(pp. 131-137). scorm.html
Muscettola, N., Dorais, G.A., Fry, C., Levinson, Sicilia, M.A., Sánchez-Alonso, S., & García-Bar-
R., & Plaunt, C. (2002). IDEA: Planning at the core riocanal, E. (2006, March 23-25). In Proceedings
of autonomous reactive agents. In Proceedings of on Supporting the Process of Learning Design
the Workshop Online Planning and Scheduling, Through Planners. Virtual Campus 2006 Post-
AIPS 2002 (pp. 49-55). Toulouse, France. Proceedings, CEUR Workshop Proceedings (vol.
186). Barcelona, Spain.
Ortigosa, A., & Carro, R. (2003). The continuous
empirical evaluation approach: Evaluating adap- Small, M., & Lohrasbi, A. (2003). Student perspec-
tive Web-based courses. User modeling. Lecture tives on online degrees and courses: An empirical
Notes in Computer Science, 2702, 163-167. analysis. International Journal on E-learning,
2(2), 15-28.
Paredes, P., & Rodríguez, P. (2002). Considering
sensing-intuitive dimension to exposition-exem- Ullrich, C. (2005). Course generation based on
plification in adaptive sequencing. In P. De Bra, P. HTN planning. In Proceedings of 13th Annual
Brusilovsky & R. Conejo (Eds.), Adaptive hyper- Workshop of the SIG Adaptivity and User Model-
media and adaptive Web-based systems. Lecture ing in Interactive Systems (pp. 74-79).
Notes in Computer Science, 2347, 556-559.
R-Moreno, M.D. (2003). Representing and plan-
ning tasks with time and resources. Ph.D. Thesis, AddItIonAL reAdIng
Universidad de Alcalá.
This section provides some additional references
R-Moreno, M.D., & Camacho, D. (2007). AI
related to the main research topics described in this
techniques for automatic learning design. In
chapter: AI planning and scheduling techniques,
Proceedings of the International e-Conference
virtual education, authoring tools and e-learning
of Computer Science (IeCCS 2006), Lecture
standards. We have included both classical texts
Series on Computer and Computational Sci-
and some recent publications that could be used
ences (LSCCS) (vol. 8, pp. 193-197). VSP/Brill
by readers to learn more about above-mentioned
Academic Publishers.
research themes.
R-Moreno, M.D., Oddi, A., Borrajo, D., & Cesta,
A. (2006). IPSS: A hybrid approach to planning

ADL, Sharable Object Reference Model, Brusilovsky, P., & Miller, P. (2001). Course
SCOR M. (2006). Retrieved October 21, delivery systems for the virtual university. In
2007, from http://www.adlnet.org/index. F.T. Tschang & T. Della Senta (Eds.), Access to
cfm?fuseaction=Scormabt knowledge: New information technologies and the
emergence of the virtual university (pp. 167-206).
Albers, P., & Ghallab, M. (1997). Context depend-
Amsterdam: Elsevier Science.
ent effects in temporal planning. In Proceedings
of the 4th European Conference On Planning, Burgos, D., Tattersall, C., & Koper, R. (2006).
Toulouse, France (pp. 1-12). How to represent adaptation in e-larning with IMS
learning design. Retrieved October 22, 2007, from
Allen, J.F., Hendler, B., & Tate, A. (1990). Read-
http://dspace.ou.nl/bitstream/1820/786/1/BUR-
ings in planning. Morgan Kaufman.
GOSetal_SofiaExtensionToILE_v3_210806.pdf
Anane, R., Chao, K.-M., Hendley, R.J., & Younas,
Carbonell, J.R. (1970). AI in CAI: An artificial-
M. (2003). In Proceedings of the International
intelligence approach to computer-assisted in-
Conference on Internet and Multimedia Systems
struction. IEEE Transactions on Man-Machine
and Applications (pp. 104-108). Honolulu.
Systems, 11(4), 190-202.
Andriessen, J., & Sandberg, J. (1999). Where is
Castillo, L., Fdez.-Olivares, J., & Gonzalez, A.
education heading and how about AI? Interna-
(2001). On the adequacy of hierarchical planning
tional Journal of Artificial Intelligence in Educa-
characteristics for real-world problem solving. In
tion, 10, 130-150.
Proceedings of the Sixth European Conference
Bacchus, F., & Kabanza, F. (2000). Using temporal on Planning (ECP’01).
logics to express search control knowledge for
Cesta, A., & Oddi, A. (2002). Algorithms for
planning. Artificial Intelligence, 16, 123-191.
dynamic management of temporal constraint
Berlanga, A.J., & García, F.J. (2005). Authoring networks (Tech. Rep.). Italian National Research
tools for adaptive learning designs in computer- Council.
based education. In Proceedings of the 2005
Chang, W.C., Hsu, H.H., Smith, T.K., & Wang,
Latin American conference on Human-computer
C.C. (2004). Enhancing SCORM metadata for
interaction (pp. 190-201).
assessment authoring in e-learning. Journal of
Blum, A., & Furst, M. (1997). Fast planning Computer Assisted Learning, 20(4), 305-316.
through planning graph analysis. Artificial Intel-
Clarke, M., & Wing, J.M. (1996). Formal meth-
ligence 90, 281-300.
ods: State of the art and future directions. ACM
Blythe, J. (1999). Decision theoretic planning. AI Computing Surveys, 28(4), 626-643.
Magazine, 20(2), 37-54.
Como, L., & Snow, E.R. (1986). Adapting teach-
Bonet, B., & Geffner, H. (2001). Planning as ing to individual differences among learners. In
heuristic search. Artificial Intelligence, 129(1-2), M.C. Wittrock, (Ed.), Handbook of research on
5-33. teaching. New York: McMillan.
Brusilovsky, P. (1999). Adaptive and intelligent Cristea, A. (2005). Authoring of adaptive hy-
technologies for Web-based education (Special permedia. Educational Technology & Society,
Issue on Intelligent Systems and Teleteaching). 8(3), 6-8.
Künstliche Intelligenz, 4, 19-25.
0
Currie, K., & Tate, A. (1991). O-plan: The open Hoffmann, J. (2003). The metric-FF planning sys-
planning architecture. Artificial Intelligence, 52, tem: Translating ignoring delete lists to numerical
49-86. state variables. Journal of Artificial Intelligence
Research, 20, 291-341.
De Bra, P., Aroyo, L., & Cristea, A. (2004). Adap-
tive Web-based educational hypermedia. In M. IMSCP, IMS Content Packaging. (2006). Re-
Levene & A. Poulovassilis (Eds.), Web dynamics, trieved October 22, 2007, from http://www.
adaptive to change in content, size, topology and imsglobal.org
use (pp. 387-410). Springer.
IMSQTI, IMS Question and Test Interoperability.
Dechter, R., Meiri, I., & Pearl, J. (1991). Temporal (2006). Retrieved October 22, 2007, from http://
constraint networks. Artificial Intelligence, 49, www.imsglobal.org
61-95.
Karagiannidis, C., Sampson, D., & Cardinali, F.
Drapper, D., Hanks, S., & Weld, D. (1994). A (2001). Integrating adaptive educational content
probabilistic model of action for least-commit- into different courses and curricula. Educational
ment planning with information gathering. In Technology & Society, 4(3), 37-44.
Proceedings of the 10th Conference on Uncer-
Kautz, H., & Selman, B. (1992). Planning as
tainty in Artificial Intelligence (178-186). Morgan
satisfiability. In Proceedings of the ECAI-92 (pp.
Kaufman.
359-363).
Drapper, D., Hanks, S., & Weld, D. (1994). Tempo-
Koper, R. (2005). An introduction to learning de-
ral planning with continuous change. In K. Ham-
sign. In R. Koper & C. Tattersall (Eds.), Learning
mond (Ed.), Proceedings of the 2nd International
design. A handbook on modeling and delivering
Conference on AI Planning Systems. University
networked education and training (pp. 3-20). The
of Chicago, Illinois. AAAI Press.
Netherlands: Springer-Verlag.
Edelkamp, S., & Helmert, M. (2000). On the
Mason, R. (2004). E-portfolios in lifelong learn-
implementation of MIPS. In AIPS Workshop on
ing. British Journal of Educational Technology,
Model-Theoretic Approaches to Planning (pp.
5(6), 717-727.
18-25).
Mödritscher, F. , García-Barrios, V.M., & Gütl,
Emerson, E.A. (1990). Temporal and modal logic.
C. (2004). The past, the present and the future of
Handbook of Theoretical Computer Science (pp.
adaptive e-learning. An approach within the scope
997-1072). MIT Press.
of the research project AdeLE. In Proceedings of
Ernst, M.D., Millstein, T.D., & Weld, D. (1997). teh 7th Conference on Interactive Computer-aided
Automatic SAT-compilation of planning prob- Learning (ICL 2004). Carinthia Tech Institute.
lems. In Proceedings IJCAI-97 (pp. 1169-1177).
Muscettola, N. (1994). HSTS: Integrating planning
Erol, K. (1995). HTN planning: Formalisation, and scheduling. In M. Zweben & M. Fox (Eds.),
analysis and implementation. Ph.D. Thesis, Intelligent scheduling (pp. 169-212). Morgan
Computer Science Department, University of Kaufman.
Maryland.
Paramythis, A., & Loidl-Reisinger, S. (2004).
Friesen, N., & Anderson, T. (2004). Interaction for Adaptive learning environments and e-learning
lifelong learning. British Journal of Educational standards. Electronic Journal on E-Learning,
Technology, 35(6), 679-687. 2(1), 181-194.

Penberthy, J.S., & Weld, D.S. (1994). Probabilistic Specht, M., & Burgos, D. (2006, June). Imple-
planning with information gathering and contin- menting adaptive educational methods with IMS
gent execution. In Proceedings of the AAAI-94 learning design. In Proceedings of Adaptive Hy-
(pp. 31-36). Seattle, WA. permedia. Dublin, Ireland. Retrieved October 22,
2007, from http://dspace.learningnetworks.org
Pryor, L., & Collins, G. (1996). Planning for con-
tingencies: A decision-based approach. In Journal Van Rosmalen, P., Vogten, H., Van Es, R. , Van,
of Artificial Intelligence Research, 4, 287-339. P., Poelmans, H.P., & R. Koper (2006). Author-
ing a full life cycle model in standards-based,
Puterman, M. (1994). Markov decision process:
adaptive e-learning. Educational Technology &
Discrete stochastic dynamic programming. John
Society, 9, 72-83.
Wiley and Sons.
Vogten, H., & Martens, H. (2006). CopperCore
R-Moreno, M.D., Borrajo, D., Cesta, A., & Oddi,
3.0. Retrieved October 22, 2007, from http://www.
A. (2007). Integrating planning and scheduling
coppercore.org
in workflow domains. Expert Systems with Ap-
plications, 33(2), 389-406. Wasson, B. (1997). Advanced educational tech-
nologies: The learning environment. Computers
R-Moreno, M.D., Prieto, M., & Meziat, D. (2007).
in Human Behavior, 13(4), 571-594.
An AI electrical ground support equipment for
controlling and testing a space instrument. Ap- Weld, D. (1999). Recent advances in AI planning.
plied Artificial Intelligence, 21(2), 81-98. AI Magazine, 20(2), 93-123.
Sicilia, M.A. (2006). Semantic learning designs: Wiley, D.A. (2000). Connecting learning objects
Recording assumptions and guidelines. British to instructional design theory: A definition, a
Journal of Educational Technology, 37(3), 331- metaphor, and a taxonomy. In D.A. Wiley (Ed.),
350. The instructional use of learning objects. Re-
trieved October 22, 2007, from http://reusability.
Sicilia, M.A., & Lytras, M. (2005). On the
org/read/chapters/wiley.doc
representation of change according to different
ontologies of learning. International Journal of Wiley, D. (2003). Learning objects: Difficulties
Learning and Change, 1(1), 66-79. and opportunities. Retrieved October 22, 2007,
from http://wiley.ed.usu.edu/docs/ lo_do.pdf

Chapter X
Knowledge Discovery from
E-Learning Activities
Addisson Salazar
Universidad Politécnica de Valencia, Spain
Luis Vergara
Universidad Politécnica de Valencia, Spain
ABstrAct
This chapter presents a study applied to the analysis of the utilization of learning Web-based resources
in a virtual campus. A huge amount of historical Web log data from e-learning activities, such as e-mail
exchange, content consulting, forum participation, and chats is processed using a knowledge discovery
approach. Data mining techniques as clustering, decision rules, independent component analysis, and
neural networks, are used to search for structures or patterns in the data. The results show the detection of
and specific content on academic performance success and failure. From the discovered knowledge, a
set of preliminary academic management strategies to improve the e-learning system is outlined.
IntroductIon 1 shows a general schema of the virtual campus

learning environment at UPA. The study pursued
This chapter contains a case study on knowledge to obtain knowledge about academic performance
discovery research carried out on data of gradu- success and failure of the students and analyz-
ate and undergraduate courses at the Universidad ing the e-learning event activity at the campus
Politécnica Abierta (UPA) site. This university Web to recognize patterns on learning styles of
is a virtual campus at Universidad Politécnica the students. Events covered the personal and
de Valencia and currently it has more than 6,000 collaborative use of the Web resources in course
students registered in about 230 courses. Figure activities, including content consulting, e-mail
Knowledge Discovery from E-Learning Activities
exchange, forum participation, and so on. The exercise practice, course achievement, and forum
underlying hypothesis was that there is useful participation. Date and time for each event also
hidden knowledge in data from e-learning Web were available. Besides of the information on the
activities for academic management and evalua- Web activity, the exercises achieved and grades
tion of the e-learning system. obtained by the UPA’s students were tried in the
The chapter describes an integrated methodol- knowledge discovery process. The data were col-
ogy to extract knowledge from quantitative and lected from the virtual campus Web in the period
qualitative data; the results obtained its evalua- from January 2002 to March 2005, totalizing
tion and a strategic action outline derived from 2,391,003 records.
the discovered knowledge. Different data mining The process of knowledge discovery covered
techniques were used to exploit the e-learning the following stages:
data through a knowledge discovery approach
(Cabena, Hadjnian, Stadler, Verhees, & Zanasi, • Building a reliable data warehouse, by
1997; Fayyad, Piatetsky-Shapiro, Smyth, & Uthur- filtering data inconsistencies, solving data
usamy, 1996; Maimon & Rokach, 2005). Those heterogeneity problems, and processing
techniques included, independent component data.
analysis (ICA), neural networks (NN), clustering, • Obtaining and interpreting patterns of the
linear regression, and decision trees. ICA allowed student behaviour in e-learning activities
distinguishing the independence of the events and by using independent component analysis,
detecting learning styles; NN was used to obtain neural networks, and linear regression
patterns of the student behaviour; linear regres- analysis.
sion was employed for numeric analyses of the • Obtaining homogeneous data groups by ap-
relationship between the student performance and plying clustering processing and selecting
the event activity levels. Quantitative clustering data groups, sorted out by research topics,
and qualitative conceptual clustering algorithms for the definition of decision rules.
were applied for grouping data in homogeneous • Applying a knowledge representation on
datasets. To enable qualitative analysis of the selected groups using decision trees to
data, continuous numeric data were converted obtain the decision rules of the factors that
to discrete value data and descriptions for their influence on academic achievement success
interpretation were obtained. Finally, on the and failure.
descriptive datasets, a mining association rule • Evaluating knowledge findings by experts,
process was made by applying the C4.5 deci- from the point of view of their validity,
sion tree algorithm. The obtained decision rules novelty, and simplicity.
involved global and specific content knowledge • Outlining strategies for the improvement of
that was evaluated by academic experts taking academic processes.
into account their validity, novelty, and simplicity.
The results were considered as useful for e-learn- The following sections describe the back-
ing academic management. ground and context of this work, the results
Data from the use of the UPA Web facili- obtained in each stage of the knowledge discov-
ties included the following Web log statistics ery process showing partial findings from the
about e-learning event activities: course access, data mining techniques. Final sections include
agenda using, news reading, content consulting, the global and particular conclusions about
e-mail exchange, chats, workgroup document, academic performance and learning styles and
future work.

Figure 1. Virtual campus learning environment at UPA
jkovic, 2002). Other predicting works on student’s

BAckground success, errors, or help requests are: predicting
the time spent in solving an exercise task by us-
Knowledge discovery in databases (KDD), or just ing neural networks (Beck & Woolf, 1998), and
knowledge discovery, is a subdiscipline of com- predicting in which word the student asks help in
puter science, which aims at finding interesting reading English, where information of the student
regularities, patterns, and concepts in data. Usu- (gender, approximated reading test results of the
ally knowledge discovery has been related with day, help request behaviour) and word (length,
the global process spans from data to knowledge frequency, etc.) were processed (Beck, Jia, Sison,
using different statistical and heuristic techniques & Mostow, 2003).
called data mining techniques. However in the Regarding to descriptive data modelling
current literature “knowledge discovery,” “data techniques applied to educational data, there are
mining,” and “machine learning” are often used several references in subjects such as analyzing
interchangeably. Recently, the data mining ap- factors with affect academic success, desertion
proach has been applied in academic research. and retention of students, mining navigation
Those applications include predictive or descrip- patterns in log data, analyzing student’s compe-
tive modelling on educational data. Traditional tence in course topics, analyzing student’s errors
sources of data have been databases or question- in program codes, and mining student answers
naires, and more recently data from the Web. from a Web-based tutoring tool database to get
Some of the works in educational predictive pedagogically relevant information and to provide
models from databases or questionnaires are the feedback to the teacher (Kristofic & Bielikova,
following: predicting whether the students gradu- 2005; Merceron & Yacef, 2003; Romero, Ventura,
ates in six years (Barker, Trafalis, & Rhoads, De Bra, & Castro, 2003; Salazar, Gosalbez, Bosch,
2004), selecting students who would need remedial Miralles, & Vergara, 2004; Shin & Kim, 1999).
classes (Ma, Liu, Wong, Yu, & Lee, 2000), and Data mining from Web data (Web mining) is a
predicting individual student’s final academic new research area that pursues to understand the
achievement by modelling with decision trees information flow at the Web by means of auto-
and hierarchical models (Gasar, Bohanec, & Ra- mated techniques for searching knowledge. This

Table 1. Dimensions of learning and teaching styles underlying in the Web data
Sensory Concrete
1 Perception 1 Content
Intuitive Abstract
Visual Visual
2 Input 2 Presentation
Auditory Verbal
Inductive Inductive
3 Organization 3 Organization
WEB Deductive Deductive
DATA Active Active Student
4 Processing 4
Reflective Passive Participation
Sequential Sequential
5 Understanding 5 Perspective
Global Global
1a. Student’s Learning Style 1b. Professor’s Teaching Style
Dimensions Dimensions
area has a wide range of emergent applications 2003). Recently, new holistic Web mining ap-
including e-learning, e-commerce, automated proaches considering extracting learning styles
information assistants, and many applications from the Web navigational behaviour of the
that operate through the Web (Srivastava, Cooley, students have been outlined (Garcia, Amandi,
Deshpande, & Tan, 2000). One example of Web Schiaffino, & Campo, 2007; Mor & Minguillón,
mining is the classification of Web pages based on 2004; Xenos, 2004).
understanding the textual content of e-mails based A learning-style model classifies students
on hierarchical probabilistic clustering (Larsen, according to where they fit in a number of scales
Hansen, Szymkowiak, Christiansen, & Kolenda, corresponding to the ways in which they receive
2002). Nowadays there exist many topics open in and process information. One of the most accepted
e-learning concerning to the quality evaluation learning style taxonomies for engineering students
of the system, the knowledge and control of the is Felder and Silverman (1988) (see Table 1). One
Web activities of the students, and the use of the learning style is conformed by the combination
huge quantity of outcome information from the of one feature in each dimension, for instance,
e-learning process (Kimber, Pillay, & Richards, intuitive-visual-deductive-active-global-. This
2007; Liaw, Chen, & Huang, 2006; Liu & Yang, model was used in the present research.
2005; Piramuthu, 2005; Pituchs & Lee, 2006; Our work pursued two objectives: (1) to find
Reilly, 2005; Selim, 2007; Shee & Wang, 2007; patterns on academic performance of the students,
Sun, Tsai, Finger, Chen, & Yeh, 2007). and (2) to detect student learning styles underlying
Particularly there is an increasing interest in in the Web data. Thus the framework is Web min-
Web mining of the e-learning data. Some examples ing for descriptive modelling of the educational
are: predicting drop-out on demographic data (sex, data. We contribute with an empirical study with
age, marital status, etc.) and course data in the a huge amount of data to contrast the results. The
first half scores of the course (Kotsiantis, Pier- complexity of the social studied phenomenon
rakeas, & Pintelas, 2003), predicting the course requires of such kind of analysis as we can found
score processing success rate, success at first try, in recent literature (Levy, 2007; Puntambekar,
number of attempts, time spent on the problem, 2006; Schellens & Valcke, 2006; Stephenson,
and so forth (Minaei, Kashy, Kortemeyer, & Brown, & Griffin, 2006). In contrast to using
Punch, 2003), combining several weak classifiers one technique as Bayesian networks, we used an
by boosting to predict final score (Zang & Lin, integrated approach with several techniques in

order to exploit and complement the advantages 2004; Pils, Roussaki, & Strimpakou, 2006).
of each one of techniques. This work is developed Some of the definitions of informatics ontology
in a novel research line that pursues to discover are the following: the informatics ontologies define
student learning styles in order to feedback and the kind of things that exists in the application
improve the e-learning system using no model domain, allowing no confusion of the terms and
based methods. The understanding of learning symbols (Sowa, 2000), informatics ontologies
styles is more difficult in e-learning education define a kind of explicit specification of a concept
than traditional education environment, and set (Gruber, 1995), informatics ontologies are de-
then research in this topic is always a challenge. fined as formal specification of a shared concept
Mining patterns from data is a classical task em- set (Borst, 1997). The principal motivation of the
ployed for a long time and it is especially useful informatics ontologies is allowing sharing and
and suitable in the e-learning context due to the reusing of knowledge bases computationally, by
new generating data processes come from Web using a common vocabulary.
technological innovations. Informatics ontology can be defined in several
The algorithms applied search for patterns ways, but necessarily it includes a vocabulary of
in the data and its output could be stored in a terms and some specification of their meanings.
knowledge based system. However the scope of This includes definitions and specifications on
the chapter does not comprise the stage of knowl- relationship between terms, so, in general, a
edge storage, so metadata are not included for structure is imposed on the domain and term
the output of the algorithms; although from the interpretation is constrained (Fridman & Mc-
decision rules obtained and using, for instance, Guinness D., 2001; Uschold, King, Morales, &
logic programming languages, the creation of Zorgios, 1998; Weigand, 1997). Thus, informatics
a knowledge base can be easily undertaken. In ontology can be a logic theory, a semantic formal
addition, the set of decision rules, cluster descrip- description, the vocabulary of a logic theory, and
tions, and learning styles detected in this work a specification on conceptions.
implicitly define a kind of informatics ontology
that could be used to share or reuse the discovered
knowledge, using an implementation format and dAtA PreProcessIng
language. There are several references comparing
informatics ontologies and statistical approaches A reliable data warehouse was created from the
as the included in this chapter (Caragea, Pathak, & historical (2001-2005) log Web data from the UPA.
Honavar, 2004; Gomez-Perez & Manzano-Macho, For this end, different operations were applied to
Figure 2. Structure of the Web data

the original data: filtering of missing and errone- The student activity data were added as fields to
ous data, solving data heterogeneity problems due the data warehouse. From the grades table, the
to different data sources, adaptation of variables average grade for a student was calculated and
for data processing, and data retrieval for stratified added to the data warehouse. Because all the
analysis. Figure 2 shows a simplified scheme of virtual courses do not have evaluation, only 1,873
the data entities at the UPA. of the rows of the data warehouse had a value for
The objective data were collected from the the variable average grade.
Web activity of the virtual campus in the tables: To get the qualitative descriptions of the student
grades and event. The event table contained one event activity, Tables 2 to 11 were defined.
record for each event at the virtual campus Web
site. The fields of this table were: group, course,
student code, event time and event class; see event Table 2. Course access description
classes (e-learning activities) in Tables 1 to 10.
The total number of events in the analyzed Type Course access
period was 2,391,003 of which 2,124,734 corre- 1 Almost never
sponded to student events and the rest of events 2 Occasionally
were of teachers and system administrator. From 3 Usually
the student events, only the records corresponding 4 Very frequently
to courses with more than 3 students (2120547
records) were selected for the knowledge dis-
covery study. Table 3. Agenda using description
A new event table was calculated summarizing
records for student and kind of event. The resulting
Type Agenda using
table contained the fields: group, course, student
1 Does not use it or use it a little
course, event class counter (event instance total
2 Average use
for that event class), and average event class time
3 Use it a lot
(average time of the student activity in that event
class). It was a 63,207 records table.
In order to build a data warehouse for data
mining, a new table with the projection of the event Table 4. News reading description
table on the student code was calculated. It was
an 8,909 records table. That number of records is Type News reading
due to that active and no active student data were 1 Do not read it or almost never read it
contained in the initial event table. 2 Average use it
For each student, the corresponding total

instance counter of each kind of event was cal-
culated, and a normalized value (1-100 scale) of Table 5. Content consulting description
student event activity was calculated with the
following equation,
Type Content consulting
(1)
1 Almost never
event total instancestudent 2 Occasionally

even_activitystudent = ⋅ 100
instance maximumevent 3 Usually
4 Very frequently

Table 6. E-mail exchange description The global mean and limits of the event activity
were calculated using the following equations:
Type E-mail exchange (2)
1 Sporadically exchange it
total instance numberevent
2 Usually exchange it meanevent =
3 Copiously exchange it
student number
(3)
Table 7. Chat description
limitevent = meanevent ± (maximum - minimum)event
Type Chats � 0.3
1 Not very active
2 Fairly active Equation 3 was applied after checking the
3 Very active normality of the event instance distributions.
The superior limit (suplim) for event instance was
calculated using plus in equation 3 and inferior
Table 8. Workgroup documents limit (inflim) was calculated using minus in equa-
tion 3. Thus the 60% of the probability density
Type Workgroup documents
distribution of the event instance was contained
1 Low collaboration
between the superior and inferior limits.
2 Average collaboration
Given the event total instance numbers (event
3 High collaboration
activity) for a student, the corresponding descrip-
tion values were calculated using the description
tables and the event limits. To allocate a value of
Table 9. Exercise practice description
the description table from an event activity value
Type Exercise practice for a student, the following algorithm for 4-entries
1 Few exercising description tables was used:
2 Enough exercising
3 Much exercising
If event_activity < inflimevent
allocate to Type 1
If event_activity >= inflimevent & < meanevent
Table 10. Course achievement description allocate to Type 2
If event_activity >= meanevent & < suplimevent
Type Course achievement
allocate to Type 3
1 Sporadically
If event_activity >= suplimevent
2 Usually
allocate to Type 4
3 Very frequently
In the case of 2-entries tables, Type 1 was

allocated when event_activity < meanevent and
Table 11. Forum participation description
Type 2 was allocated when event_activity >=
Type Forum participation
meanevent. For tables with 3 entries, Type 1 was
allocated when event_activity < inflimevent, Type
1 Little
2 Average
2 was allocated when event_activity >= inflimevent
3 High
& < suplimevent, and Type 3 was allocated when
event_activity >= suplimevent.

Besides of assigning description values to using the conjunctive conceptual algorithm (Mi-
event activity values, the instance time of the chalsky & Stepp, 1983).
events was discretized using the following hour Total population of the data warehouse was
intervals: [0-6) =dawn, [6-12) =morning, [12-18) divided accordingly to the course types in: gradu-
=evening, and [18-24) =night. The qualitative value ate (informal courses), doctorate, and regular
for average grade was calculated using the follow- academic career courses. In addition, each of
ing discretization intervals: [0-5) =unsatisfactory, those population divisions was divided in two
[5-7] =fair, [7-9) =good and [9-10] =excellent. subsets: cases with grades and cases with no-
At the end of the data preprocessing stage, grades. Therefore six disjunctive data subsets to
the data warehouse obtained consisted of a 8,909 analyze were generated.
(student records) x 27 (variables) as follows: 3
identification record variables (group, course,
student code), 4 variables for quantitative and IndePendent comPonent
qualitative values of average grade and average AnALysIs (IcA)
time, and 20 variables for quantitative and quali-
tative values of the activity for the different kind ICA is a powerful statistical technique that has
of e-learning events. had a successful application in different areas
of signal processing (Cichocki & Amari, 2001;
Hyvärinen, Karhunen, & Oja, 2001). ICA assumes
dAtA mInIng scheme that there is a M-dimensional zero-mean vector
s(t) = [s1(t),...,sM (t)]T, such that the components si(t)
Figure 3 shows a general schema of the relationship are mutually independent. The vector s(t) cor-
between the data mining techniques applied on responds to M independent scalar-valued source
the data warehouse. Quantitative clustering was signals si(t). The multivariate probability density
made by applying the fuzzy c-means algorithm function (p.d.f.) of the vector can be rewritten as
(Bezdek & Pal, 1992) and qualitative clustering the product of marginal independent distributions
Figure 3. Interconnection of the applied data mining techniques
0
M ICA was applied on the UPA data in order

p ( s ) = ∏ p i ( si ). A data vector x(t) = [x1(t)...xN (t)]T to identify independent “sources” (independent
i =1
is observed at each time point t, such that x(t) = event activity), that is, searching those event ac-
As(t)where A is called mixture matrix and it is tivity that can separate by an ICA algorithm as a
full rank N x M, (Hyvärinen, Karhunen, & Oja, source. Figure 4 shows the estimated activity for
2001). the 10 events (see Tables 2-11) on the UPA Web
There are several standard ICA algorithms
plus the average connection time and average
as FastICA (Hyvärinen & Oja, 1998), Extended
grade for 1,072 students of graduate courses with
Infomax (Lee, Girolami, & Sejnowski, 1999), or
grades. Note that data are displayed as signals
TDSEP (Ziehe & Müller, 1998). Those algorithms
(vectors of samples) for Figures 4 and 5. This latter
rely on assumptions about the source signals,
show the sources estimated by an ICA algorithm,
such that imply a given model for the source
note that signal of event 8 (exercise practice)
distributions or make assumptions that are only
in Figure 4 is very similar (high correlated) to
fitted to specific applications. We applied stan-
source 5 in Figure 5, it means that the activity
dard ICA algorithms and a new nonparametric
corresponds to the workgroup document event
ICA algorithm proposed in Annex 1. This latter
could be recognized as an independent source
algorithm yields the best results, because it was
for this subset of data.
more adaptable to the data. It does not assume
After analyzing the results from ICA applied
any restriction on the data, since the probability
to the different data subsets and considering
distributions are calculated directly from the
additional information about the courses and
training set through a nonparametric approach,
students in the campus, we can infer the follow-
and also focusing the independency between the
ing conclusions:
source components directly from its definition
based on the marginal distributions.
Figure 4. Data of the graduate courses with grades for the 10 events, and average grades and connec-
tion time: e1 (course access) … e10 (forum participation)

Figure 5. Sources calculated by an ICA algorithm for data of Figure 4
• E-mail exchange was independent in some PrIncIPAL comPonent AnALysIs

cases. It could be due to weakness in teaching (PcA) And IcA
strategies for promoting the student interac-
tivity. Then e-mail exchange is transformed PCA is a very well known technique that reduces
in e-mail review done as a routine. the variable dimensionality in statistical multivari-
• In courses with no grades, the workgroup ate analysis (Hardle & Simar, 2006). We applied
document event was independent. The lack PCA for grouping the events of the Web activity
of evaluation and grades discourage the in learning dimensions taking into account the
participation of students in collaborative Felder’s framework (Felder & Silverman, 1988).
tasks. PCA reduced 10 Web event activities to 5 compo-
• In some datasets the content consulting nents. To solve the problem of detecting learning
event was independent as reflect of a kind of styles in e-learning we assume that the underlying
distributed passive learning (DPL) nature of independent sources that generate the Web log
the Web platform. Thus content consulting data are dimensions of the learning styles of the
becomes a routine consisting in download students and we observe x linear combinations
materials with no interactive learning proof those styles through the use of the facilities
cess. by the students at the virtual campus. Then,
• Exercise practice and course achievement si , (i = 1, ,5 learning style dimension) cor re-
also were found as independent events for spond to the “perception,” “input,” “organization,”
some datasets. It could be due to the profile “processing,” and “understanding” dimensions
of some students that includes information (see Table 1); and the mixture matrix A provides
and telecommunications background and the relation between e-learning style dimensions
knowledge about course contents. For those and e-learning event activities,
students participating in those event activi-
ties could be irrelevant. aij, (i = 1,...,5 learning style dimension), ( j = 1,...10
e-learning activity).

Table 12 contains the six first sorted contribu- emphasis of educational strategies did not favour
tions of Web activities of the ICA mixture matrix to highlight that dimension. From Table 12, the
for the 5 sources estimated. Each source was relationship between learning style dimensions
associated with one learning dimension of Table and Web activities can be made; see Table 13
1 analyzing the weight of the Web activities and where we have added a possible Web activity
considering the principal evaluation methodolo- combination for learning dimension 1.
gies employed by teachers for graduate courses Note that some Web activities are associ-
with grades. Dimension 1 was not detected and ated with more than one dimension; it has sense
dimension 5 was detected twice. The methodolo- because a Web activity could demand several
gies assigned grades focusing on: achievement, capabilities of the students used in their learning
individual student participation, or group work. process. Allowing that kind of relationship we can
The implicit teaching styles of the evaluation obtain more real and versatile descriptions of the
methodologies encourage specific learning styles student learning styles, besides of including all the
of the students, as we explain below. dimensions of the learning framework. In Garcia
The learning dimension 1 (sensory-intuitive) et al., (2007) just three dimensions of the Felder
corresponding to “perception” was not detected and Silverman (1988) model were considered
in the ICA mixing matrix; it could be because the and the Bayesian network proposed constrained
Table 12. ICA mixing matrix (*learning style dimension, **workgroup documents)
LSD* Sorted Web activity contribution

chat forum news e-mail access exercises
2
1 .82283 .30755 .16476 .14756 .14231
**
e-mail content wg-doc exercises forum chat
4
1 .34189 .32297 .28768 .22548 .20078
wg-doc news achieve content chat e-mail
3
1 .80531 .4122 .39987 .39421 .31666
achieve content agenda access forum news
5’
1 .45124 .2117 .21116 .20087 .18239
access agenda content achieve e-mail chat
5
1 .95776 .85549 .7143 .5832 .49774
Table 13. Association between learning styles and Web activities
Learning Style Web event activity

1 Sensory-Intuitive Perception chats, forum participation, course access.
chats, forum participation, news reading,
2 Visual-Auditory Input
e-mail exchange.
workgroup document, news reading,
3 Inductive-Deductive Organization
course achievement, content consulting .
e-mail exchange, content consulting, work-
4 Active-Reflective Processing
group document, exercise practice.
course access, agenda using, content con-
5 Sequential-Global Understanding
sulting, course achievement.

relationship of the Web activities with just one We made a cluster validation procedure to
dimension of the learning model. determine best quality of cluster configuration
Figure 6 shows the sources 3, 4, and 5 (orga- for data of Figure 6. It consisted in estimating
nization, processing, understanding) obtained for the partition coefficient and the partition entropy
the grade graduate course dataset. Four labelled coefficient for different number of clusters (Hal-
characterised zones in the learning style space diki, Batistakis, & Vazirgiannis, 2001). The best
are displayed: (1) Represents the learning style cluster configuration for data of Figure 6 was 4
more important in the population. The learning clusters—a detailed explanation of cluster valida-
for the students in this zone emphasizes global tion procedure is in cluster analysis section.
understanding, active processing, and deductive Figure 7 shows three sources for graduate
logic (natural human teaching style), and high courses with no grades. The distribution of the data
grades. (2) This learning style is focused on in Figure 7 does not allow forming learning style
inductive logic (natural human learning style), groups and show all the subjects within a unique
with sequential understanding, and relative ac- learning style. As understanding and organiza-
tive processing. Students within this style could tion dimensions do not discriminate projection
have natural skills for virtual education. (3) It is of the learning styles, only the dimension of the
characterised by global understanding, deductive processing provides some discrimination. The
logic, and reflective processing. Students within unique learning style emphasises reflection over
this style would have higher abstraction skills that actuations. It would be the content consulting and
need of teaching. (4) Basically this cluster repre- exercise practice components of that dimension.
sents outliers with individual learning styles. The conclusion is the lack of assessment does not
We can conclude that dimension of understand- allow developing student learning styles.
ing enables to project clearly the learning styles, Results for regular academic career courses
and its principal components are achievement, were similar to the graduate courses results find-
content, and agenda. This finding confirms the ing meaningful learning styles for courses with
assumption that the more quickly way to change grades.
the learning style of the student is to change the Results of this section could be analyzed as a
assessment style, that is, expected evaluation kind of ontology. The conceptions are the learn-
bias how the student learns (Elton & Laurillard, ing styles detected, related with the dimensions
1979).
Figure 6. Three sources in a learning style space for graduate courses with grades

Figure 7. Three sources in a learning style space for graduate courses with no grades
Figure 8. Ontology of learning styles detected
of learning (input, organization, processing, the linear model were not statistical significant,
understanding), and ultimately with the Web so there is some nonlinear relations between the
learning activities. Figure 8 shows such a kind of variables.
ontology (numbers in the boxes corresponding to We tried with more adaptive models apply-
the numbers of learning styles in Table 1). ing the linear vector quantization (LVQ) neural
networks to classify the different datasets in for
classes depending on the average grade: [0-5)
regressIon AnALysIs And =unsatisfactory, [5-7] =fair, [7-9) =good and [9-
neurAL networks 10] =excellent. LVQ algorithm includes the self-
organizing and competitive stages; four output
A linear regression model was designed with the neurons were defined corresponding each to the
average grade as dependent variable and the 10 target classes. Seventy-five percent of the data were
Web event activities plus the average connection used in the training phase and the rest of the data
time as independent variables. The adjustment of in testing phase. Kohonen learning rate of 0.01 and

conscience learning rate of 0.001 were employed. Figure 9 shows a scheme of a data cluster
Different tests were made varying the number of structure, consisting of five clusters and the dis-
neurons of the hidden layer from 4 to 10. tances between cluster centroids. Doughnut sizes
The groups found automatically by LVQ could represent the number of records in each cluster
be interpreted with similar contents to the obtained and doughnut slides represent the 12 variable
by the fuzzy clustering algorithm described in values of cluster centroids.
next section. However some dissimilar groups The fuzzy c-means was applied using a fuzzi-
were found by LVQ, for example, students that ness degree of 1.3 (exponent m). Validity measure
only have a lot of chats and obtain an excellent was the partition coefficient and the maximal
average grade, students that exchange e-mails and number of classes used in training was 16. The
have chats with excellent average grades. calculation was carried out for all classes and the
best number of classes was determined and vali-
dated by checking the evolution through the class
cLusterIng AnALysIs number range of the partition coefficient (pc) vs.
the classification entropy (pe), see Figure 10.
In the clustering procedure, a (specified) number Figure 10 was calculated with graduate with
of clusters are calculated from a set of objects. A no-grades data subset and the best portioning is
cluster is represented by a cluster center which at c=5. The partition coefficient and the partition
defines the center point of the cluster in the feature entropy both tend towards monotone behaviour
space. A cluster center is thus an (imaginary) object depending on the number of clusters. So as to find
which defines the typical or ideal representative the “best” number of clusters c* one chooses the
of its cluster. number where the entropy value c* lies below the
rising trend and the value for the partition coef-
Figure 9. Cluster structure

Figure 10. Cluster validation
ficient lies above the falling trend. On viewing For the regular career courses with grades:
the curve of all the connected values, this point
can be identified as a kink (thus the name “elbow • The student group with the best grades
criterion”). shows a similar activity level in the different
The best partitioning of the clusters applies event types, except in course achievement
at that point with a value of c to get the highest where it has a higher activity than the other
cluster differentiation (maxima of inter-clusters groups.
mean distances) with good homogeneity within • The student group with the worst grades
cluster members (minima of distances between shows a higher exercise practice proportion
cases and centroids) (Haldiki et al., 2001). than the other groups; however the course
The application of the fuzzy c-means algo- achievement is relatively low.
rithm on the defined 6 data subsets generated 23 • The intermediate academic performance
clusters or groups. Ten groups for the data subsets groups show an imbalance in the proportion
graduate and career regular courses with grades, of activities, focusing in e-mail exchange
and thirteen groups for the same courses with and agenda using.
no-grades. For the doctorate courses no groups • This data subset does not use events that
were generated due to there were not enough require interactivity (chats, forum partici-
cases to analyze. pation, and workgroup documents) among
Besides of the fuzzy c-means, the conjunctive several students.
conceptual algorithm was used on the qualitative • The students with worst grades are devoted
variables to get logical conjunctions of relations mainly to news reading, content consult-
between the variables. From the analysis of the ing, and e-mail exchange but they do not
obtained cluster centroids, the following conclu- undertake to course achievements.
sions on academic performance were derived. • The average grade of the clusters follows a
normal distribution, being the clusters with
the best and worst grades less numerous than
the other clusters.

For the graduate courses with grades: • Every group showed similar proportion in
exercise practice and course achievement
• There are no significant differences in the events.
academic performance of the clusters.
• The worst grade group is the highest course For the graduate courses with grades:
achievement one.
• The best grade group shows similar behav- • Every group showed a high utilization of
iour in each of the e-learning activities. This interactivity events; even the workgroup
group has the highest value of course access documents activity was high.
activity. • There was a group with high values for the
• In this data subset, the events that require three interactivity activities, but the value
interactivity among students were used, but for exercise practice event is very low like
its activity was lower than the events that in the other groups.
do not require student interactivity.
• The best grade group was the second most Clusters calculated from the graduate course
numerous ones and the worst grade group with no-grades data subset are shown in Figure
was the less numerous ones. 11. Event activity values in the cluster representa-
• The best grade groups used the e-mail more tion are normalized.
frequently than the others.
For the regular career courses with no- mInIng decIsIon ruLes
grades:
Average grade was defined as outcome variable for
• Every group exhibited a good utilization of mining decision rules on academic performance;
the interactivity events forum and chats but 250 decision rules were obtained applying the C4.5
workgroup document event. algorithm (Quinlan, 1992) to the clusters of the
Figure 11. Event activity proportion at calculated clusters from graduate course data subset

graduate and regular career course data subsets For the regular career courses:
and to the group consisting of the union of both
data subsets. Decision rules involve global con- Rule No 114: If Course = Economic and Financial
tent or specific content knowledge. Some of the System and
mined decision rules are listed below including Exercise practice = Much exercising and
the success percentage of the rules. Course achievement = Usually Then
For graduate courses: Average Grade FAIR [83.54%]
Rule No 9: If Agenda using = Average use and Rule No 115: If Course = Economic and Financial
News reading = Does not use it or use it a little System and
and Average time = Evening and
Content consulting = Usually and E-mail exchange = Usually exchange it and
Forum participation = Average Then Course achievement = Very frequently Then
Average Grade GOOD [80.65%] Average Grade FAIR [73%]
Rule No 16: If Content consulting = Very fre- Rule No 116: If Course = Economic and Financial
quently and System and
Course achievement = Usually and Average time = Evening and
Forum participation = Average Then News reading = Average use it and
Average Grade GOOD [77.24%] Content consulting = Usually and
Chats = Fairly active Then
Rule No 36: If Course = Environment System Average Grade FAIR [81.5%]
Management and
Average time = Evening and Rule No 121: If Course = Economic and Financial
E-mail exchange = Copiously exchange it and System and
Chats = Very Active Then Average time = Morning and
Average Grade EXCELLENT [71.23%] Agenda using = Average use and
Content consulting = Usually Then
Rule No 55: If Course = Renewable Energy and Average Grade FAIR [79.57%]
Chats = Fairly active Then
Average Grade EXCELLENT [71.45%] Rule No 133: If Course = Management I and
News reading = Average use it and
Rule No 67: If Course = Teledetection Systems Exercise practice = Enough exercising Then
for Environment Risk Prevention and Average Grade GOOD [94.85%]
Course access = Very frequently and
Content consulting = Occasionally Then Rule No 140: If Course = Economic and Financial
Average Grade FAIR [80%] System and
Average time = Evening and
Rule No 78: If Course = Basic Environment E-mail exchange = Usually exchange it and
Technical English and Chats = Very active Then
Course achievement = Null Then Exercise practice = Enough exercising Then
Average Grade UNSATISFACTORY [80%] Average Grade GOOD [95.27%]

Rule No 141: If Average time = Evening and • To collaborate with other educational
Chats = Null and networks or virtual platforms in order to
Exercise practice = Enough exercising and reinforce teaching quality and promoting
Forum participation = High Then the creation of virtual learning networks.
Average Grade GOOD [84.24%] • To design and to implement pedagogic course
syllabus for students to understand e-learn-
Rule No 154: If Course = Management I and ing education. The appropriate utilization of
Exercise practice = Much exercising and information and communications technolo-
Course achievement = Usually and gies helps to educate more and better.
Average Grade EXCELLENT [90%] • To adapt the roles of counsellors, teaching,
supporting, and administrative staffs to the
The knowledge findings obtained in the re- classes in the cyberspace.
search were evaluated by academic administration • To promote the knowledge of the virtual
experts of the university in aspects such as validity, campus in all the university to generate
novelty, and simplicity, obtaining general score synergic relationship between university
of 8.2 points on a scale from 1 to 10. people. This can produce positive feedback
From the point of view of informatics on- to the virtual campus as new students and
tologies, the trees conformed by decision rules teachers, e-learning project creation, and
could be analyzed as “concepts” about good communications for improvements.
and bad academic achievement of the students. • To propose special events for diffusion of
Those rules provide structure for relationship of the virtual university as conferences and
the terms that define the student behaviour us- workshops online, in order to obtain par-
ing the Web e-learning activities. In (Silvescu, ticipation of the students.
Reinoso-Castillo, & Honavar, 2001) is explained • To implement a virtual library with refer-
how implicit ontologies drive the information ences to bibliographic contents of virtual
extraction and data integration procedures used courses. Thus in addition to the basic mod-
in knowledge acquisition from data, specifically ules and annexes of the courses, access to
using decision trees. bibliographical electronic resources, allow-
ing research activities, would be provided.
strAtegIc ActIon outLIne

concLusIon
The following are some preliminary strategies
for academic management that were proposed The proposed methodology applied to a real
based on the results of the knowledge discovery case with huge historical data obtained promise
process. results in detecting student learning styles in
an e-learning environment. The dimensions of
• To empower collaborative informatics to the Felder’s learning framework were modelled
include practical virtual labs. Some of the using and adaptive approach. The versatility of
technical subjects to be included are: work- the approach consists in integrated descriptive
group, workflow, data mining, searchers, modelling using several data mining techniques
multimedia, and customer research manage- for processing quantitative and qualitative data.
ment. Nonparametric independent component analysis,
a technique normally used in signal processing,
0
has been useful for detecting patterns in e-learning with more variables. Thus, the complexity of the
data. Despite the possible problems of converting analysis of the research topics can be more real-
continous numeric data to discrete value data, im- istically modelled. Among these variables could
provement of interpretation capabilities has been be gender, age, location, enrolment date, likes,
demonstrated. Modelling learning dimensions as and dislikes, and so on. Besides of teacher’s data,
a combination of Web event activities enhanced such as the course’s survey results and research
the detection of the student learning styles. topics. Those variables would be collected through
The knowledge discovery from e-learning questionnaires, or transferring automatically from
Web data found useful knowledge (of global or databases.
particular content) on academic performance The part of teaching styles of the Felder’s learn-
of the students at the Universidad Politécnica ing framework or another educational model has to
Abierta (UPA). Among the findings are the fol- be incorporated in the proposed methodology. The
lowing: (i) Events of synchronous interactivity, tuning of the learning and teaching styles to obtain
such as chats, forum participation, and events of a good performance in the outcome of the process
asynchronous interactivity empower the student would be modelled. Depending on the mixture of
academic performance; (ii) In the courses with learning and teaching styles, several adaptations
grades, academic student performance could be of the pedagogical e-learning resources could be
improved by motivating students to have course made. The results of the enhanced model could
achievement. Some students show good values for be used to adapt teaching methodologies, includ-
the different event activities, including exercise ing the critical aspect of the assessment style,
practice, but do not have evaluations. or in general to improve the e-learning system,
General results of the research were well balancing distributed passive learning (DPL) and
evaluated by academic experts taking into ac- distributed interactive learning (DIL).
count the validity, novelty, and simplicity of the The semantic information and the implicit
knowledge. All these knowledge of global and informatics ontologies defined by clusters de-
particular contents could be used to improve the scriptions, decision trees, and learning styles
e-learning system in different aspects. Strategies conceptions found in the research, could be used
to encourage interactivity between students, strat- to implement a knowledge-based system and/or
egies to design an assessment methodology that a standard ontology of the studied domain. The
reinforce the student learning styles detected, and ontology could be used to exchange and reuse the
global improvements of different components of knowledge and it would make easy to increase,
the e-learning system towards a more distributed foster, and update the knowledge obtained from
interactive learning could be proposed. Consider- the Web e-learning activities. The use of a Web
ing the findings of knowledge, a preliminary set ontology language and standard data interchange
of strategies was outlined. formats would make possible the approach to the
semantic Web.
Future work
Future reseArch dIrectIons
As a prototype, the study has yielded encouraging
results on the application of knowledge discovery The chapter has discussed the knowledge discov-
to e-learning analysis. Nevertheless, in order to ery in e-learning considering several subjects as:
obtain a complete application of this analysis is e-learning Web activities, data preprocessing, data
necessary to complement the data warehouse mining techniques, knowledge evaluation, learn-

ing and teaching styles, pedagogical innovation, automatic conversion of knowledge or semantic
and informatics ontologies. information obtained by Web mining techniques,
The balance between interactive and personal represented by structures as the decision trees, to
activities is a critical factor for e-learning systems ontologies would make possible the exchange and
(distributed passive learning (DPL) and distributed reuse of the domain knowledge. Methodologies
interactive learning (DIL)). An interesting area of to create hierarchical structures of patterns are
research is the proposal of new e-learning activi- suitable to create ontologies; to make that pos-
ties or determining the suitable mixture of those sible, conversion procedures to translate statistical
activities considering, for instance, contents, mul- information to standard data interchange formats
timedia resources, and ubiquitous networks. are needed. All of those approaches may contribute
The quality of the discovered knowledge to develop the semantic Web.
is directly proportional to the cleanness and
relevance of data. E-learning processes could
generate a lot of useless information, so efficient AcknowLedgment
algorithms to filtering and summarizing data; to
resolve inconsistencies, to estimate missing data, Special thanks go to the Universidad Politécnica
and solving data heterogeneity are valuable for Abierta personnel for giving the Web data and
the knowledge discovery approach. information about the virtual campus. This work
Pattern recognition is a wide area that includes has been supported by Spanish Administration
many kinds of machine learning algorithms. In- under grant TEC 2005-01820.
dependent component analysis (ICA) algorithms,
as applied in the present chapter, have yielded
important results in areas as image filtering and reFerences
segmentation, brain to computer interface, and
electrocardiographic diagnosis. Recently the mix- Barker, K., Trafalis, T., & Rhoads, T.R. (2004).
ture of ICAs has emerged as a flexible generating Learning from student model. In System and
model to arbitrary data densities using mixtures Information Engineering Design Symposium
of Gaussians or Laplacians distributions or non- (pp. 79-86).
parametric distributions for the components.
Beck, J.E., Jia, P., Sison, J., & Mostow, J. (2003).
Those ICA mixtures could be used to model data
Predicting student help-request behavior in an
or knowledge in the Web.
intelligent tutor for reading. In 9th International
Usually the evaluation of the knowledge is
Conference on User Modelling (pp. 303-312).
made by experts. Nowadays, aspects as novelty
or interestingness are estimated by novelty detec- Beck, J.E., & Woolf, B.P. (1998). Using a learn-
tion algorithms. Those algorithms could be used ing agent with a student model. Lecture Notes in
by intelligent agents in the Web in order to make Computer Science, 1452, 6-15.
decisions considering user behaviours. In the field
Bezdek, J.C., & Pal, S.K (1992). Fuzzy models
of e-learning it is a novel approach.
for pattern recognition: Methods that search for
Recently, second level patterns in data mining
structures in data. New York: IEEE Press.
have been studied. Those approaches have been
used in protein and ADN research, where the Borst, W.N. (1997). Construction of engineering
results of a first level of data mining conforms a ontologies for knowledge sharing and reuse.
huge knowledge domain. In Web applications that University of Twenty, NL-Centre for Telemática
kind of methods would be useful. In addition the and Information Technology.

Cabena, P., Hadjnian, P., Stadler, R., Verhees, J., learning from texts. Knowledge Engineering
& Zanasi, A. (1997). Discovering data mining: Review, 19(3), 187-212.
From concept to implementation (IBM Books).
Gruber, T.R. (1995). Towards principles for the
Pearson Education.
design of ontologies used for knowledge sharing.
Caragea, D., Pathak, J., & Honavar, V. (2004). International Journal of Human-Computer Stud-
Learning classifiers from semantically heteroge- ies, 43, 907-928.
neous data. Lecture Notes in Computer Science,
Haldiki, M., Batistakis, Y., & Vazirgiannis, M.
3291, 963-980.
(2001). On clustering validation techniques.
Cichocki, A., & Amari, S. (2001). Adaptive Journal of Intelligent Information Systems, 17(2-
blind signal and image processing: Learning 3), 107-145.
algorithms and applications. New York: John
Hardle, W., & Simar, L. (2006). Applied multivari-
Wiley & Sons.
ante statical analysis. New York: Springer.
Duda, R., Hart, P.E., & Stork, D.G. (2000). Pattern
Hyvärinen, A., Karhunen, J., & Oja, E. (2001).
classification (2nd ed.). Wiley-Interscience.
Independent Component Analysis. New York:
Elton, L.R.B., & Laurillard, D.M. (1979). Trends John Wiley & Sons.
in research on student learning. Studies in Higher
Hyvärinen, A., & Oja, E. (1998). A fast fixed-point
Education, 4(1), 87-102.
algorithm for independent component analysis.
Fayyad, U., Piatetsky-Shapiro, G., Smyth, P., & Neural Computation, 9(7), 1483-1492.
Uthurusamy, R. (1996). Advances in knowledge
Kimber, K., Pillay, H., & Richards, C. (2007).
discovery and data mining. New York: The MIT
Technoliteracy and learning: An analysis of the
Press.
quality of knowledge in electronic representa-
Felder, R., & Silverman, L. (1988). Learning and tions of understanding. Computers & Education,
teaching styles. Journal of Engineering Educa- 48(1), 59-79.
tion, 78(7), 674-681.
Kotsiantis, S.B., Pierrakeas, C.J., & Pintelas, P.E.
Fridman, N., & McGuinness, D. (2001). Ontology (2003). Preventing student dropout in distance
development: A guide to creating your first ontol- learning using machine learning techniques. In
ogy (Rep. No. KSL-01-05, SMI-2001). Proceedings of 7th International Conference
on Knowledge-Base Intelligent Information an
Garcia, P., Amandi, A., Schiaffino, S., & Campo,
Engineering Systems.
M. (2007). Evaluting Bayesian networks’ precision
for detecting students’ learning styles. Computers Kristofic, A., & Bielikova, M. (2005). Improving
& Education, 49(3), 794-808. adaptation in Web-based educational hyperme-
dia by means of knowledge discovery. In ACM
Gasar, S., Bohanec, M., & Rajkovic, V. (2002).
Conference on Hypertext and Hypermedia (pp.
Combined data mining and decision support
184-192).
approach to the prediction of academic achieve-
ment. In Workshop on Integrating Aspects of Data Larsen, J., Hansen, L.K., Szymkowiak, A., Chris-
Mining (pp. 41-52). tiansen, T., & Kolenda, T. (2002). Web mining:
Learning from the world wide Web (Special Issue
Gomez-Perez, A., & Manzano-Macho, D. (2004).
of Computational Statistics and Data Analysis).
An overview of methods and tools for ontology
Computational Statistics and Data Analysis, 38,
517-532.

Lee, T., Girolami, M., & Sejnowski, T. (1999). Mor, E., & Minguillón, J. (2004). E-learning per-
Independent component analysis using an extend sonalization based on itineraries and long-term
infomax algorithm for mixed sub-Gaussian and navigational behavior. In Thirteenth World Web
super-Gaussian sources. Neural Computation, Conference (pp. 264-265).
11, 417-441.
Pils, C., Roussaki, L., & Strimpakou, M. (2006).
Levy, Y. (2007). Comparing dropouts and per- Location-based context retrieval and filtering.
sistence in e-learning courses. Computers & Lecture Notes in Computer Science, 3987, 256-
Education, 48(2), 185-204. 273.
Liaw, S., Chen, G., & Huang, H. (in press). Users’ Piramuthu, S. (2005). Knowledge-based Web-en-
attitudes toward Web-based collaborative learning abled agents and intelligent tutoring systems. IEEE
systems for knowledge management. Computers Transactions on Education, 48(4), 750-756.
& Education.
Pituchs, K.A., & Lee, Y.-K. (2006). The influ-
Liu, H., & Yang, M. (2005). QoL guaranteed ence of system characteristics on e-learning use.
adaptation and personalization in e-learning Computers & Education, 47(2), 222-244.
systems. IEEE Transactions on Education, 48(4),
Puntambekar, S. (2006). Analyzing collaborative
676-687.
interactions: Divergence, shared understanding
Ma, Y., Liu, B., Wong, C.K., Yu, P.S., & Lee, and construction of knowledge. Computers &
S.M. (2000). Targeting the right students using Education, 47(3), 332-351.
data mining. In KDD’00: Sixth ACM SIGKDD
Quinlan, R.J (1992). C4.5: Programs form
International Conference on Knowledge Discov-
machine learning. San Mateo, CA: Morgan
ery and Data Mining (pp. 457-464).
Kaufmann.
Maimon, O., & Rokach, L. (2005). Data mining
Reilly, R. (2005). Guest editorial Web-based
and knowledge discovery handbook (1st ed.).
instruction: Doing things better and doing better
Springer.
things. IEEE Transactions on Education, 48(4),
Merceron, A., & Yacef, K. (2003). A Web-based 565-566.
tutoring tool with mining facilities to improve
Romero, C., Ventura, S., De Bra, P., & Castro,
learning and teaching. In 11th International
C. (2003). Discovering prediction rules in AHA!
Conference on Artificial Intelligence in Educa-
courses. In 9th International Conference on User
tion (pp. 41-52).
Modeling (pp. 25-34).
Michalsky, R.S., & Stepp, R.E. (1983). Auto-
Salazar, A., Gosalbez, J., Bosch, I., Miralles, R.,
mated construction of classifications: Conceptual
& Vergara, L. (2004). A case study of knowledge
clustering versus numerical taxonomy. IEEE
discovery on academic achievement, student
Transactions on Pattern Analysis and Machine
desertion and student retention. In IEEE 2th Inter-
Intelligence, 5(4), 396-410.
national Conference on Information Technology:
Minaei, B., Kashy, D.A., Kortemeyer, G., & Research and Education (pp. 150-154).
Punch, W. (2003). Predicting student performance:
Schellens, T., & Valcke, M. (2006). Fostering
An application of data mining methods with an
knowledge construction in university students
educational Web-based system. In Proceedings
through asynchronous discussion groups. Com-
of 33rd Frontiers in Education Conference.
puters & Education, 46(4), 349-370.

Scott, D.W., & Sain, S.R. (2004). Multi-dimen- Uschold, M., King, M., Morales, S., & Zorgios,
sional density estimation. In C. R. Rao, E. J. Weg- Y. (1998). The enterprise ontology. Knowledge
man & J. L. Solka (Eds.), Handbook of Statistics, Engineering Review, 13, 32-89.
Data Mining and Computational Statistics, Vol.
Weigand, H. (1997). Multilingual ontology-based
24, (pp. 229-261). Elsevier.
lexicon for news filtering. In IJCAI Workshop on
Selim, H. (2007). Critical success factors for e- Multilingual Ontologies (pp. 138-159).
learning acceptance: Confirmatory factor models.
Xenos, M. (2004). Prediction and assessment of
Computers & Education, 49(2), 396-413.
student behaviour in open and distance education
Shee, D., & Wang, Y. (in press). Multi-criteria in computers using Bayesian networks. Computers
evaluation of the Web-based e-learning system: & Education, 43(4), 345-359.
A methodology based on learner satisfaction and
Zang, W., & Lin, F. (2003). Investigation of
its applications. Computers & Education.
Web-based teaching and learning by boosting
Shin, N., & Kim, J. (1999). An exploration of algorithms. In IEEE International Conference on
learner progress and dropout in Korea National Information Technology: Research and Education
Open University. Distance Education an Inter- (pp. 445-449).
national Journal, 20, 81-97.
Ziehe, A., & Müller, K.R. (1998). TDSEP-an ef-
Silvescu, A., Reinoso-Castillo, J., & Honavar, V. ficient algorithm for blind separation using time
(2001). Ontology-driven information extraction structure. In 8th International Conference on
and knowledge acquisition from heterogeneous, Artificial Neural Networks (pp. 675-680).
distributed, autonomous biological data sources.
In International Joint Conferences on Artificial
Intelligence (IJCAI) (pp. 1-10).
AddItIonAL reAdIng
Sowa, J.F. (2000). Knowledge representation:
Logical, philosophical and computational foun- data mining and knowledge
dations. Pacific Grove, CA: Brooks Cole. Discovery
Srivastava, J., Cooley, R., Deshpande, M., & Tan,
Chi, X., & Spedding, T.A. (2006). A Web-based
P. (2000). Web usage mining: Discovery and ap-
intelligent virtual learning environment for
plications of usage patterns from web data. In
industrial continous improvement. In IEEE 4th
SIGKDD Explorations (pp. 12-23).
International Conference on Industrial Informat-
Stephenson, J.E., Brown, C., & Griffin, D.K. (in ics (pp. 1102-1107).
press). Electronic delivery of lectures in the uni-
Hammouda, K., & Kamel, M. (2006). Data min-
versity environment: An empirical comparasion of
ing in e-learning. In S. Pierre (Ed.), E-learning
three delivery styles. Computers & Education.
networked environments and architectures: A
Sun, P., Tsai, R., Finger, G., Chen, Y., & Yeh, D. knowledge processing perspective. Springer Book
(in press). What drives a successful e-learning? Series: Advanced Information and Knowledge
An empirical investigation of the critical factors Processing.
influencing learner satisfaction. Computers &
Han, J., & Kamber, M. (2001). Data mining con-
Education.
cepts and techniques. Academic Press.

Markou, M., & Singh, S. (2003). Novelty detection: Pattern recognition

A review. Part 1: Statistical approaches. Signal
Processing, 83(12), 2481-2497. Langseth, H., & Nielsen T.D. (2005). Latent
classification models. Machine Learning, 59(3),
Yang, Y., Wu, X., & Zhu, X. (2006). Mining in
237-265.
anticipation for concept change: Proactive-reac-
tive prediction in data streams. Data Mining and Lee T.W., Lewicki, M.S., & Sejnowski, T.J. (2000).
Knowledge Discovery, 13(3), 261-289. ICA mixture models for unsupervised classifica-
tion of non-gaussian classes and automatic con-
text switching in blind signal separation. IEEE
ontologies Transactions on Pattern Analysis & Machine
Intelligence, 22(10), 1078-1089.
Berners-Lee, T., Hendler, J., & Lassila, O. (2001).
Salazar, A., Igual, J., Vergara, L., & Serrano, A.
The semantic Web: A new form of Web content
(in press). Learning hierarchies from ICA mix-
that is meaningful to computers will unleash a
tures. Paper presented at the International Joint
revolution of new possibilities. Scientific American
Conference on Neural Networks.
Magazine, 284(5), 34-43.
Vergara, L., Salazar, A., Igual, J., & Serrano, A.
Cardoso, J., & Sheth, A. (2002). Semantic e-
(2006). Data clustering methods based on mixture
workflow composition. Journal of Intelligent
of independent component analyzers. Paper pre-
Information Systems, 21(3), 191-225.
sented at the ICA Research Network International
Ceccaroni, L., & Ribiere, M. (2002). Experi- Workshop, ICArn (pp. 127-130).
ences in modeling agentcities utility-ontologies
Webb, A.R. (2002). Statistical pattern recognition.
with a collaborative approach. Paper presented
John Wiley and Sons.
at the Ontologies in Agent Systems Workshop,
Autonomous Agents and Multi-Agent Systems
Conference. education
Davies, J., Studer, R., & Warren, P. (2006). Se-
Butler, K.A. (1990). Learning and teaching style:
mantic Web technologies: Trends and research
In theory and practice (2nd ed.). Gregorc Associ-
in ontology-based systems. Wiley.
ates, Incorporated.
Sowa, J.F. (2006). Categorization in cognitive
Entwistle, N. (1990). Styles of learning and
computer science. In H. Cohen & C. Lefebvre
teaching - an integrated outline of educational
(Eds.), Handbook of categorization in cognitive
psychology for students, teachers, and lecturers.
science (pp. 141-163). Elsevier.
Fulton, David Publishers.
Sowa, J.F. (2006). A dynamic theory of ontol-
Entwistle, N., & Peterson, E. (2004). Conceptions
ogy. In B. Bennett & C. Fellbaum (Eds.), Formal
of learning and knowledge in higher education:
ontology in information systems (pp. 204-213).
Relationships with study behaviour and influences
IOS Press.
of learning environments. International Journal
of Educational Research, 41(6), 407-428.

e-Learning If we assume a nonparametric model for p(y), we

can estimate the source pdf’s from a set of train-
Chen, C.M., Lee, H.M., & Chen, Y.H. (2005). ing samples obtained from the original dataset
Personalized e-learning system using item reusing (2). We propose a kernel density estimation
sponse theory. Computers & Education, 44(3), technique (Scott & Sain, 2004; Duda, Hart, &
237-255. Stork, 2000), where the marginal distribution of
a reconstructed component is approximated as:
Conole, G., Dyke, M., Oliver, M., & Seale, J.
(2004). Mapping pedagogy and tools for effec- (n ')
1 y − y 
−  m m  (5)
p (ym )= a ⋅ ∑ e
tive learning design. Computers & Education, 2 h 2
 
, m = 1...M
43(1-2), 17-33. n'
Driscoll, M. (2002). Web-based training: Design- where a is a normalization constant and h is a

ing e-learning experiences, Pfeiffer; 2 Har/Cdr constant that adjust the degree of smoothing
edition. of the estimated pdf. The learning algorithm
can be derived using the maximum-likelihood
Figini, S., Baldini, P., & Giudici, P. (2006). Non- estimation. In a probability context it is usual to
parametric approaches for e-learning data. Lecture maximize the log-likelihood of (3) with respect
Notes in Computer Science, 4065, 548-560. to the unknown matrix W:
Shih, T.K., Wang, T.H., Chang, C.Y., Kao, T.C., &
Hamilton, D. (2007). Ubiquitous e-learning with L (W ) log det W . p (y ) (6)
=
multimodal multimedia devices. IEEE transac- W W
tions on multimedia, 9(3), 487-499. log det W log p (y )
= + .
Watkins, R. (2005). 75 e-learning activities: W W
Making online learning interactive, Pfeiffer; where
Har/Cdr edition.
= (WT )
log det W −1 (7)
W
Annex 1
Imposing independence on y and using (4):
This annex contains the formulation of the para-
(8)
log p (ym )
metric ICA algorithm applied in the chapter.
log p (y ) M
The probability density function of the data x =∑ =
can be expressed as: W m =1 W
M
1 p (ym ) M 1 p (ym ) ym
p(x) = | det W|p(y) (3) ∑
m =1 p (ym ) W
=∑
m =1 p (ym ) ym W
.
where p(y) can be expressed as the product of the

marginal distributions since it is the estimate of where using (5):
the independent components: (9)
p (ym )
(n ')
1 y −y 
−  m m 
2 2  ym − ym(n ')  1
= −a ∑ e
h
 
  , m = 1...M
M
p (y )= ∏ pi ( yi ) (4)
ym n'  h h
i =1

Let us call w Tm the m-th row of W. Then ym = w Tm x, Considering the vector f(y) = [ f(y1) f(y2)...f(yM)]T
then we can write
log p (y )
= f (y )xT
ym (10) (13)
= Mm
W W
where Mm (l,l' = d(l – m)xl' Using the results of (7) and (13) we may finally
Substituting (5), (9) and (10) in (8) we have: write (6) as:
L (W )
= (WT ) + f (y )xT
−1 (14)
(11)
log p (y )= W
∑ f (y )M
M
m m
W m =1
Then we can apply (14) in the gradient updating al-
where gorithm to iteratively find the optimum matrix
 1  y − y( ) 
−  m m 
n'
 L (W )
 y ⋅ e 2  h  2

 (12) W (i + 1) = W (i )+ (i ) (15)
1 ∑ m
 W
f (ym )= 2  n '  (n ') 
− ym 
h  1 y −y
−  m m 
2 2 
 ∑e 
h


 n' 

Chapter XI
Swarm-Based Techniques in
E-Learning:
Methodologies and Experiences
Sergio Gutiérrez
University Carlos III of Madrid, Spain
Abelardo Pardo
University Carlos III of Madrid, Spain
ABstrAct
This chapter provides an overview of the use of swarm-intelligence techniques in the field of e-learn-
ing. Swarm intelligence is an artificial intelligence technique inspired by the behavior of social insects.
Taking into account that the Internet connects a high number of users with a negligible delay, some
of those techniques can be combined with sociology concepts and applied to e-learning. The chapter
analyzes several of such applications and exposes their strong and weak points. The authors hope
that understanding the concepts used in the applications described in the chapter will not only inform
researchers about an emerging trend, but also provide with interesting ideas that can be applied and
combined with any e-learning system.
IntroductIon Many research efforts are trying to take ad-

vantage of two fundamental characteristics of the
The World Wide Web does not offer only access Internet: small delays in communications (inde-
to information. It connects many people all around pendent of physical location) and a big number of
the world in a very short time. New applications, users. Social systems try to emulate the behav-
inspired in natural processes (like those that allow ior of social groups in real life. These systems
social insects to work together) are appearing. extract some information from the behavior of
Swarm-Based Techniques in E-Learning
the group and use it to get some benefit for the hallways, and ventilation systems following very
students. In other words, they take advantage of simple and local rules.
the interactions between the different members Swarm intelligence is a growing field of active
of the learning community to help each of its research, and its applications outside the Internet
members. This chapter aims at providing the are manifold. Swarm intelligence techniques have
reader with a broad overview of the state of the been applied to many different kinds of problems.
art on this emerging field, specifically regarding Examples include packet routing (Dorigo &
the problems of sequencing and filtering. Stützle, 2004), graph coloring (Costa, Hertz, &
Dubious, 1995), allocating tasks for robots in a
factory (Morley, 1996), routing a fleet of trucks
BAckground (Gambardella, Taillard, & Agazzi, 1999), as well
as many robotic applications (Bonabeau et al.,
A swarm may be defined as a population of in- 1999). It is common to use the term ant colony
teracting elements that is able to optimize some optimization (ACO) for the set of heuristics used
global objective through collaborative search of a on these problems.
space (Kennedy & Eberhart, 2001). The elements The Internet allows for low-delay communica-
may be very simple machines or very complex tions for big amounts of people. Communication
living beings, but there are two restrictions to can be direct (like it is the case for IP-telephony
be observed: they are limited to local interac- or chats) or indirect (like Internet polls or file in-
tions and usually interaction is not performed terchange), and there are many intermediate cases
directly but indirectly through the environment. (like Internet fora). Big number of participants,
The property that makes swarms interesting is indirect communication, local awareness, and
their self-organizing behavior; in other words, it local actions define a setting for the appearing of
is the fact that a lot of simple processes can lead emergent behaviors of a swarm by combination
to complex results. of the activities of all its members.
The behavior of ants is the best-known example
of swarm intelligence. In many ant species, ants
deposit a chemical substance called pheromone coLLABorAtIve seQuencIng
as they move from a food source to the nest. Ants
do not communicate directly with each other, but The problem of adaptive sequencing is that of,
they follow pheromone trails (leaving their own given a set of learning activities, finding the best
pheromones behind, so the trail is reinforced). sequence for a particular student. Given a proper
Shorter trails are more strongly reinforced, as the user model, a system can adapt the sequence of
ants cross them more times for the same period learning activities to each student. Unfortunately,
of time, so they are followed by more ants. In most approaches share a common weakness. The
the end, this positive feedback loop ends up in role of a human designer is very important and a
the path connecting the food source and the nest mistake on her part affects the whole system. The
being optimized without any global knowledge negative effect on students may vary.
of the problem by any of the agents. This process Human mistakes may be avoidable (e.g., er-
of indirect communication in a swarm is called rors when typing names of activities), but the
stigmergy (Bonabeau, Dorigo, & Theraulaz, sequencing designing process is bound to some
1999). Another example of a stigmergic process unavoidable problems. First, teachers have a
is nest building by termites, in which the insects limited capability and a limited knowledge about
are able to construct complex buildings with arcs, activities that may be of interest for their students.
00
If the sequencing strategy depends entirely on nodes and their probabilities, the sequencing of
the teacher, those activities that the teacher does the students is specified stochastically (Semet,
not know will never be accessed by the students. Lutton, & Collet, 2003).
Moreover, students evolve over time (e.g., changes As such, the graph defines possible sequenc-
in formal curricula, different access to media, and ings of the activities stochastically, but its defi-
knowledge sources, etc.). A sequencing design that nition is static (the pedagogical weights do not
works very well today may not be as good tomor- change). Thus, the system can suffer from the
row. This cannot be addressed by the teacher alone problems stated before. Thus, the Paraschool
(Gutiérrez, Pardo, & Delgado Kloos, 2006). Three system takes an approach inspired by the ACO
different approaches are presented in this section, paradigm. Every student traversing the graph acts
but all of them try to extract some information like an ant. As she interacts with the exercises,
of the behavior of the group of students and use she leaves pheromones on the arcs that influence
it to improve their sequencing strategy. their probabilities of being chosen. These phero-
mones can be of two types: positive or negative.
the Paraschool system Positive pheromones are left when the student
solves correctly an exercise, negative pheromones
Paraschool is the leading e-learning company in are left when the student fails. Combining both
France. It offers its services to several thousands pheromones and the pedagogical weights, a fitness
students in the country. It offers support on many function is calculated for every arc. After an exer-
subjects, online exercises, correction, and feed- cise has been completed by a student, all outgoing
back. Their system uses techniques based on ACO arcs are sorted by fitness and one is selected by
in order to detect bad pedagogical planifications stochastic tournament (Semet, Lutton, & Collet,
and to correct them automatically (Valigiani, 2003). That arc is “followed” by the student and
Lutton, Jamont, Biojout, & Collet, 2006). the next activity is shown.
The Paraschool system divides the activities In order to enforce the notion of learning
into courses and chapters. Courses can range path, pheromones are laid on the last four arcs
from a short training (e.g., course on security traversed by the student. Older arcs receive a
when using heavy machinery) to a full academic smaller amount of pheromone. As pheromones are
year at a school (e.g., fourth grade class). Courses deposited on the arcs, those arcs that are part of
are divided into chapters. Inside each chapter, a successful sequences of exercises are reinforced,
graph of activities is defined. Every node in this while those that lead to failure are depleted. Over
graph is an activity, typically composed of a time, the system is able to refine the original
theory Web page, an exercise that illustrates the design and even correct bad pedagogical designs
concepts presented, and a final page that corrects (Gutierrez, Valigiani, Jamont, Collet, & Delgado
the answer of the student and offers her some Kloos, 2007).
feedback. Edges of the graph represent possible Pheromone values are decreased over time
transitions after a node has been covered. Nodes (much in the same way natural pheromones
are not necessarily connected to themselves. evaporate on air) in order to give the system
Probabilities are associated to every arc. responsively to changes. First versions of the
These probabilities determine which will be the Paraschool systems used natural time, but this
next learning unit to be delivered to the student. approach presented problems of seasonality and
These probabilities are initially set by a peda- was difficult to calibrate (Valigiani, Jamont,
gogical team of teachers, as pedagogical weights. Biojout, Lutton, & Collet, 2005). Thus, a sort of
Taking into account the edges that connect the user-based time was used. Pheromone values on
0
arcs are only updated when a user goes through A final characteristic of the Paraschool system
that arc or through a neighbour (those arcs that is its ability to create new arcs that connect its
lead to the same exercise). When a student finishes exercises, apart from those that were set by the
exercise A, positive and negative pheromones are pedagogical team. Paraschool users can navigate
laid on the last four arcs that she followed. Thus, in a guided fashion (as explained above) or freely.
the pheromones on those arcs are increased. At Free navigation means that the students can ac-
the same time, pheromone values on all the other cess any of the exercises that are available on the
arcs that lead to the same exercise are decreased. site. Every time this happens, the system records
This can be seen as competition between arcs, the transition and creates a new arc from the last
as those arcs that are used more often will have exercise performed by the student to the exercise
high pheromones values, while those that are that she has gone to. From that point on, that
seldom used will see that their pheromones level arc is considered like a normal arc of the graph
“evaporate” rapidly. (only with no starting pedagogical weight) that is
The work by Valigliani (2005) has shown selectable in normal guided navigation, receive
several differences between the application of pheromones, and so forth. This is an interesting
ant-inspired heuristics to technical problems (like feature that allows finding learning sequences
those referenced in the background section), in that the pedagogical team had not thought about
which “ants” are little software agents, and their in the first place. Moreover, it opens the door to
application to social problems, in which real people new exercises to be added to the system even if
are used as “ants.” no pedagogical team creates a graph for them. As
First, real students are not as altruistic as ants connections between the arcs will appear from free
(either natural or artificial) are: students will not navigation, the algorithm will assign a fitness to
willfully sacrifice themselves for the good of those arcs over time, as pheromones are deposited
the community. In other words, a student does on them with regard of success or failure of the
not expect to be lost in the exercise graph for the students traversing them. In the end, the new set
sake of exploration. All students want to learn. of exercises will auto-organize themselves.
They are not interested in optimizing the results
of the group; they are interested in optimizing Learning networks
their own results. This is only common sense, but
it means that there are aspects of the ant-colony Learning networks (Koper, 2005) are flexible
optimization heuristics that cannot be used or do learning facilities oriented at supporting the
not perform as expected. needs of learners at various levels of competence
Secondly, the process of learning is one that throughout their lives. They support ubiquitous
must be optimized for each learner. The fact that a access to learning facilities at work or at home.
pedagogical path is optimized for a group does not Learning networks consist of learning events
mean that it is optimized for each of its members. (called activity nodes) in a given domain. Activity
In order to give an additional level of adaptation nodes can be associated to anything that supports
at a personal level, personal pheromones are used. learning (e.g., Web resource, course, workshop,
These pheromones are left by each user as the etc.). Both providers and learners can create new
user traverses the graph, and prevents the user activity nodes or adapt existing ones (even delet-
from repeating the same exercises over and over ing nodes). Thus, a learning network represents
again. Personal pheromones are laid by use and a large and ever-changing set of activity nodes
evaporate with natural time. They are a multipli- that provide learning activities for learners,
cative factor in the fitness function. from different providers and at different levels
of expertise.
0
In a learning network, users have a goal. This et al., 2004). This matrix indicates, for each activ-
is a description of the level of competence that ity node, how many learners have successfully
they want to achieve in any particular domain. For progressed from that node to the following one.
achieving goals, a learner can follow one route, This information can be fed back to other learners
sometimes more. Additionally, users in a learn- providing some guidance for their navigation in
ing network are described by their learning track an indirect fashion. The algorithm for selecting
and their position. The learning track is defined the next activity node for a user that has just
as the set of activity nodes that have already been completed an activity is as follows. First, the set
completed. The learning position includes the of activity nodes that have to be completed is
learning track and all those activity nodes than can calculated. After that, the success information
be considered as completed, either because they for each of them (starting from the current node)
are related to some node in the learning track or is retrieved from the transition matrix. Then,
because of previous study or work experience on using that information as weights, the next activ-
the part of the student. This is a complex problem ity is chosen randomly. An example will clarify
that is studied in (van Brugen et al., 2004) along this point. Let us imagine a student that has just
with some approaches to it. finished activity node A. For achieving her goal,
As it can be expected from any life-long learn- the student has to complete activity nodes B, C,
ing scenario, learning networks can give access D, and E. From the transition matrix, we can get
to a big number of activities and resources. This how many students moved from A to each of these
brings a big complexity that might hinder the learn- activity nodes and were successful completing
ing process of students. They may find it hard to them. (The definition of success may vary between
gain an overview of the number of modules and nodes: it may be a Boolean quality, for example,
the best sequence to study them. In order to help answering correctly one question, or a measure
learners during the cognitive, decision-making of different parameters, for example, answering
process required for choosing and sequencing correctly 60% of the questions or do it in a time
their learning events, Tattersall et al. (2005) take lower than some threshold, and so forth). Data at
an inspiration of how ants interact indirectly using the matrix says that 5, 2, 1, and 2 students were
pheromones to optimize the path from their nest successful completing B, C, D, and E, respectively.
to a food source. Thus, the next activity node recommended for
The system allows learners to select from a the student selected will be B with a 50% prob-
list of the learning goals in a learning network. ability, C with a 20%, and so forth. The result is
It also allows, thereby, identification of the route that the most frequently and succesfully followed
to that goal. Learners’ interactions are stored in path has a higher probability of being selected.
a log, including information of the learner, the To prevent suboptimal convergence to this path
activity node, a timestamp and an indication of (stagnation), there is a chance that the other paths
performance. Learners’ interactions with learning are selected.
resources and activities are recorded automatically This navigation support is designated to fa-
as they progress through a body of knowledge, cilitate planning decisions and reduce the risk
identifying sequences. These sequences can be of information overload by offering accessible
processed and aggregated to derive a sort of information. This information is learner centered,
“pheromone strength,” so those paths where more as it is related to the learner’s present position
learners have been successful are favoured. (i.e., his knowledge level in the current learning
Using information on the tracks of all learners, network). As the feedback makes use of success
a transition matrix can be calculated (Deshpande rates, it helps learners to make better choices based
on sequences tried and tested by their peers.
0
sIt to SIT and the platform takes the corresponding

resource and forwards it to the student.
SIT (Gutiérrez, Pardo, & Delgado Kloos, 2006b) is SIT wraps the resources in a brief user interface
a generic platform for the development of intelli- that allows the student to log out or continue the
gent tutoring systems. The platform offers several tutoring process after she has interacted with the
services to the tutoring system designer, from learning resource (e.g., read some text, solved an
authentication facilities to automatic retrieval and exercise, etc.). If the student logs out, the session
forwarding of Web resources. Intelligent tutoring is restarted at the same point the next time she
systems built on top of SIT do only have to take logs into the system. If the student continues,
care of the user model, the domain model and the all data is collected by SIT and forwarded to the
learning strategies they want to develop (Hartley, tutoring system and the cycle restarts.
1973). Most administrative tasks are performed If only one activity is selected by the tutor-
by SIT. It has been calculated that a big amount ing system as suitable for the student, that is the
of the cost of developing an intelligent tutoring activity delivered to the student. But it may be
system has to do with dealing with administrative the case that the tutoring system decides that
tasks (Murray, 1999). SIT permits to focus on the there are several activities that are suitable for
teaching and learning aspects of development. the student, given current knowledge about her.
SIT puts the focus on adapting the sequence When this happens, SIT shows a menu to the
of learning resources to the user. Thus, tutoring student with all the options, so the student can
systems built on top of SIT must implement some decide which is the next preferred activity based
model of the domain they are tutoring and some on the information given by the tutoring system
sequencing strategy that tailors the sequence of and forwarded by SIT (typically, a title or short
activities to the student’s capabilities and needs. description of every option).
It is the responsibility of the tutoring system It is this last characteristic of SIT that is im-
to know where to find the appropriate learning portant from the point of view of this chapter, as
resources. In the context of SIT, resources are SIT uses some techniques inspired on the mecha-
identified by an URL. Resources can be created nisms that ants use for optimizing the paths from
by the same pedagogical team than creates the their nests to food sources (as explained above)
tutoring system or they can use some resources to help students in their decision. Using a method
already available on the Web. SIT encourages similar to that of the learning networks explained
reuse of already existing learning resources as above, SIT collects information about the success
it has no resources on its own, but can retrieve of students when interacting with activities and
and deliver any resource to the students. They resources following a particular sequence. It then
can be static resources, like images, or dynamic feeds back some of this information to students
resources, like PHP pages that perform a full (Gutierrez, Pardo, & Delgado Kloos, 2006a).
remediation cycle of showing an exercise, getting If the administrator of the system allows it
the student’s answers and showing some feedback (this can be selected for some specific tutor, or
adapted to her answers. just for some learning modules, etc.), SIT collects
It is the responsibility of the tutoring system information about the success of the students
to analyse the input from the user, if any, as well when interacting with those learning activities
as all former knowledge about her. Using that delivered to them. Then, when different learning
information, the tutoring system decides what activities are chosen for the student and a menu
is the next learning activity or resource to be se- is presented to her, SIT shows information about
quenced to the student. This information is given how her peers have performed on those activities
0
when they where in the same state as she is now. There is no need to mark the student with any tag
The state is defined as the last activity or sequence at the beginning of the process, nor is there any
of activities performed by the student. need for a complex student modelling guiding
Information is presented as a ratio for each this classification (although this auto-organizative
available activity in the menu: n students tried procedure is complementary to any student model-
this activity when they were in the same state as ling performed by the intelligent tutoring system
you are now, m were successful. If there is no built on top of SIT). Students situate themselves
information about the success, or if that informa- in their appropriate level of skill following the
tion makes no sense (e.g., if an activity consists traces left by others. It is important to note that
on looking at a photograph for five minutes or this process has two virtues: it is automatic and
reading a piece of text, there may be no concept auto-organizative, and it is flexible. Students that
of “success”), only the first half of the information perform very well on the first activities of a mod-
is shown: n students tried this activity when they ule may not do so later, and the system can adapt
were in the same state as you are now. It may be to them at every moment. From a certain point
the case that some activities are presented with of view, there can be as many levels as students,
a success ratio while others are not. although this is dependant of the sequencing
Students then have the possibility of selecting strategy of the tutoring system. In other words,
an activity using the information they have about the “skill levels” of the students are fractal: after
their peers. Students that are conscious of their each and every step, some of the activities pres-
relative competence will select those activities ent greater challenges than others and students
that seem to be harder, looking for bigger chal- have to choose at every moment which path they
lenges or trying to avoid the longer and more want to follow.
boring sequences of learning material. Students It is obvious that the technique used by SIT
that find the learning material more difficult will bears many similarities to that of the learning
choose the longer but easier path to achieve the networks, but there are several differences.
same learning goals. In effect, the information The first difference is that SIT does not have
about the results of their peers places the students any resources of its own, but relies on forwarding
in a metacognitive state in which they do have to external resources as directed by the tutoring sys-
think about their relative performance compared tem. This flexible approach is a two-ended sword,
with other students as much as their level of skill as it means that information about success on the
in a particular domain. This makes the system activities is not always available. If the auto-orga-
specially suited as a complement to classical nizative feature of SIT is to be used, resources and
teacher-classroom-student scenarios, in which activities have to be adapted so that they can send
the students interact frequently with each other that information to the platform (this is performed
and have a more accurate picture of their relative using a tag on the HTTP request). Without success
skill level compared to the rest of the group. The or failure information, SIT is only able to show
system is valid for distance learning as well, as the number of students that have performed one
the comparative position with respect to the group or another activity and most of the advantages of
can be inferred as the learning process goes on. the stigmergic process are lost.
A student who finds that she is successful where Another difference is that the learner plays a
many others have failed will probably look for central part in the adapting and stigmergic pro-
harder challenges and vice versa. cesses. The “pheromone” information is shown
This approach discriminates between students directly to the student, and it is the student who
by skill level in a sort of stigmergic development. makes the decision, not the system. This is much
0
more flexible than the former approach and allows information that shows that transition is -prob-
for the formation of several skill levels. When ably- a very bad one.
only one activity is recommended, most students
(both skillful and modest) will tend to follow the
same path, leading to a Stalinist regime (Kauff- coLLABorAtIve FILterIng
man, 1996); even those that rebel to a path that
may not be suited to them and chose any other, Over the last years, the growth of e-commerce
will be lost in the absence of any other guidance has stimulated the use of collaborating filtering
and their traces will not be significant. Present- systems as recommender systems. The goal of
ing different options with different information a modern collaborative filtering system may be
in each case allows the users to follow different stated as predicting the utility of a certain item
paths according to their capabilities. In all cases, for a particular user based on the user’s previous
the traces left behind are significant to other stu- likings and the opinions of other like-minded us-
dents that come afterwards. ers. Collaborative filtering is based on the premise
A third difference is the definition of state of the that people looking for information should be able
learner. The state of the student is the last activity to make use of what others have already found
or sequence of activities that has been completed and evaluated. In a way, collaborative filtering
by the student. The length of the sequence is a systems are organisers of knowledge: the prefer-
parameter to be set by the administrator of the ences of the users and their processing create the
system. Longer sequences mean more precise classification scheme.
adaptation between the cognitive state of the peers Modern collaborative filtering system can be
(inferred from the information shown) and the classified into memory-based and model-based.
cognitive state of the student. Shorter sequences The first ones employ a user-item database to
mean that it is easier (i.e., takes less time) to get generate a prediction. These systems use statistical
results that are meaningful to the student. This techniques to find a set of users (neighbours) that
a common problem in most social system and is have a similar profile of agreeing with the target
referred in the literature as the cold start problem. user (Pennock, Horvitz, Lawrence, & Lee, 2000).
In the case of the learning networks, success is Model-based collaborative filtering algorithms
calculated only from the current activity node to provide item recommendation by first developing
the next one. SIT can calculate longer sequences a model of user ratings. Algorithms in this cat-
of activities from the past history of the student egory take a probabilistic approach and envision
in order to give more precise information. the collaborative filtering process as computing
Related to the former two issues, there is the the expected value of a user prediction, given the
fact the SIT calculates the success information in user’s ratings on other items. The model build-
relative terms. Observing the case where the state ing process is performed by different techniques
of the student is of length 1 (as in the learning such as Bayesian networks (Miyahara & Pazzan,
networks), SIT has not only one transition matrix, 2000), latent semantic analysis (Hofmann, 2003),
but two: one stores the successful transitions from and rule-based approaches (Boley, 2003).
the current activity to all the others, while the There have been some collaborative filtering
second one stores total transitions. In the case of systems explicitly designed for assistance in the
the learning networks, it is not important if 5 or learning process, like PHOAKS (www.phoaks.
500 students have tried to complete activity node com) or LON-CAPA, but the most interesting
B after completing A. With the SIT approach, for the scope of this chapter is CoFIND. Not
both numbers are important and it is their joint only is it a collaborative filtering system and it
0
is focused on learning, specifically, in providing of qualities, but “with purple letters” is as well.
some of the “features” of a teacher (Dron, 2002). In order to prevent the system to be flooded with
It is also designed to produce auto-organization qualities of little or no interest to the users, qualities
and stigmergic processes as the swarm of students disappear from the system if they are not used by
interact with it. users for a certain period of time. This produces a
competition between the qualities, in which only
coFInd those that are useful (and used) continue on the
system. Successful qualities are those that are
CoFIND (Dron, Mitchell, Siviter, & Boyne, 1999) used or rated (see below) more often. They are
is a collaborative filtering system that tries to help presented at first places of the list of qualities. Not
students to generate and use a list of learning successful qualities are pushed down the list by
resources. The resources are to be found by the the others, until they finally disappear because
students and organized automatically. This orga- nobody in interested in them anymore.
nization is not the result of some predefined rules In the early versions of CoFIND, qualities
by the tool designer or a pedagogical team, but it disappeared from the system if they had not been
is the result of the interactions of the students with used for a week. As it was the case when we talked
the resources they and their peers have provided, about the Paraschool system, the approach of us-
and between each other. ing natural time for the evolution of the system
CoFIND aims to facilitate the communication presented several problems, nonetheless the fact
between the learners (being this communication that if the system was not used for a week, all
direct or indirect) as well as providing information qualities (including useful ones) disappeared from
as an expert in one subject. This information is it. Thus, more recent versions of CoFIND use a
structured in a way that it is more easily assimi- weighting algorithm that uses a kind of user-based
lated by the learner. In order to reach that goal, time. New qualities start with a weighting that is
the system makes use of the votes of the users for equivalent to a count of the largest number of rat-
the resources (and their qualities) and the usage ing given to any quality currently in the system.
that the users make of them. Whenever a quality is chosen by any user, its
There are four important concepts that are weighting increases by one. When a user logs into
central in CoFIND: resources, qualities, votes, and the system, the weightings from all the qualities
topics. Resources are identified with a URL. Both are reduced by one. Thus, those qualities that are
qualities and topics describe resources. Qualities not used will see their weightings go down to zero
are metadata tags that indicate the opinion of a in a system running normally. On the other hand,
learner about a resource (e.g., interesting, bad if the system is not heavily used for any reason
explained, amusing, etc.). Topics provide thematic (e.g., summer holidays) it preserves its state.
clustering of resources. Later versions modified as well the user in-
Looking for auto-organization, CoFIND takes terface related to qualities. Those qualities that
an evolutionary approach in which resources are have been used more often to describe resources
to compete with each other. The best fitted (in the appear on the list using a larger font. The user of
sense of most useful for the students) will survive the system has therefore two kinds of information
and the other will disappear. when selecting or searching for a quality. Position
As it is the case for resources, qualities are on the list expresses the weighting: those qualities
created by the users of the system. There are no that are up the list have been used recently or they
constraints of what can be a quality. “Helpful,” are of new creation. Size of the font used express
“written properly,” or “low level” are all examples how many resources have been already described
0
with this quality. In other words, the size of the qualities. Both are topic-dependent (e.g., a quality
quality is an indicator of how many resources can like “about bridges” needs some clarification about
be found using CoFIND when searching for that the nature of the bridges to know if the resource is
particular quality, while position is an indicator more relevant to civil or electronic engineers).
of the usefulness of the quality for other users. In the same fashion as qualities and resources,
It is important to note that the number of rated topics are introduced by their own users. The
resources is a piece of explicit knowledge, but users are presented by the system with a blank
the usefulness of a quality is implicit knowledge canvas divided in four quadrants. There, they can
generated from user statistics. Another interest- fill these sectors with topics of their choosing,
ing point here is that the quality alone does not selecting which of the sectors contains their top-
describe the resource: a resource may be described ics. Therefore, the sectors will probably present
by a quality with a lot of negative votes (e.g., some clustering of related topics.
a boring document described with the quality Topic names in the same sector compete with
“amusing”). each other for the space. Each time a topic is
When users look for resources that have been selected, the font used for displaying its name
described with a particular quality, they have is increased. The font used for the other top-
the opportunity to express their agreement or ics is decreased. In order to avoid a stagnation
disagreement with the selection provided. This phenomenon observed in the earlier versions of
is performed through a voting process, in which the system, the increase in size when selecting a
the users can express how much they agree with topic is proportional to the number of neighbours
the description. The users vote using a discreet (more neighbours mean more increase) while
scale of six steps, to encourage positioning on the the decreasing of size for the other topics is pro-
part of students, avoiding the “middle point.” For portional to the inverse of the number of topics
example, if a student looks for “good for begin- (more neighbours, smaller decrease). For the sake
ners” resources and obtains a very complex one, of legibility, there is a maximum and a minimum
she may express that she totally disagrees with the bound to the size of the font. Thus, there is no
“good for beginners” quality (for that particular real death of topics, they never disappear. This
resource). This rating affects the positioning of method presents some instability problems with
resources, as explained now. old Web browsers that have very few font sizes
Resources are displayed in order of the aver- to display (e.g., only three sizes for the FONT tag
age of the ratings. On its own, this would mean of HTML), but presents a good behavior for big
that a resource that has been rated once could number of topics. By relating growth and shrink-
be considered as more successful than one that age of topic names to the number of competing
has been rated many times, although one single topics, successful ones will continue to stand out
rating might be strongly atypical. Therefore, to when there are large populations as effectively
give more importance to those resources that are as when there are small populations. The result
rated more frequently, they are displayed before of this process produces some sort of stigmergy,
those that have been rated fewer times when the in which students select popular paths, thus in-
average rating is the same. creasing the success of those paths, amplifying
Another concept of importance in CoFIND the behavioral pattern of the group to provide an
is that of topics. Topics are, in the end, binary structure for the learning material.
classifications. Topics cover the necessity of creat- There is a compromise in the size of the
ing some separation between both resources and community that can benefit from a system like
0
CoFIND. Big populations mean that the system Future reseArch dIrectIons
will be helpful for more people, but a smaller
group is more likely to make the system evolve The Internet is still far from its peak of develop-
faster. This is coherent with results from natural ment, and the effects of “social e-networks” is far
sciences which show that evolution tends to occur from being fully comprehended. In the near future,
more rapidly in isolated populations (Gould, 1978). many more applications built on the foundations
This leads to a useful mechanism for reducing the of those presented here will develop.
bloated excess of results that could be obtained Social navigation, very related to social se-
from a typical search engine or directory. If a quencing, is another emerging trend. Social navi-
small group with a common learning interest gation (Marten, Farzan, & Brusilovky, 2006) aims
compiles a set of resources (possibly found using at using navigation information from the users
search engines) there is a high probability that to provide them with clues about where to move
they are winning a higher relevance from the next. This clues might give information about the
rest of the resources (found by their colleagues). current position of users (e.g., which documents
The evolutionary model of CoFIND creates a is the user accessing now) or about the path they
context-dependent taxonomy which captures have followed. These clues can relate to groups
the usage of the group’s tacitly negotiated and of users as well. For example, documents that
agreed evaluations. Ambiguities and disagree- are more frequently accessed can be highlighted
ments are not discussed but solved in a sort of in some form.
indirect democratic process as users vote and As the users interact increasingly more
rate the resources. between themselves (either consciously or not)
during their learning process, their awareness
about their environment becomes more important.
concLusIon Learning is known to be a social process that
benefits greatly from the interaction with others,
The pervasive presence of the Internet allows yet information technologies have traditionally
for greater communication between different limited learning to an impersonal paradigm.
people than ever before, independently of their As technology is providing means for real com-
physical location. Although this situation brings munication, it becomes important to be aware of
its own risks, it opens the doors for many ways the environment with which the student interacts
of collaboration between different members of a (who is doing what; when, where, etc.). Possible
community. sources of information that can be relevant to
Taking some inspiration from the biology of the student are virtual location and near peers,
social insects, several swarm-intelligence applica- peers activities and opinions, timing, and age of
tions have appeared over the last years. Combin- activities and actors, and so forth.
ing some of their know-how with some sociology Enhancing the awareness of the students
concepts produces the appearance of social-swarm about all these issues can play an important role
applications. This chapter has presented those in their learning. Besides, being aware of your
that are focused on e-learning, showing both its peers allows you to interact with them directly.
strong points as their weaknesses. Finding the adequate equilibrium between direct
and indirect communication (like in swarms) for
promoting learning will be one the challenges in
the future.
0
reFerences of the optimal learning path. In Adaptive Hyper-

media 2006 (pp. 430-434).
Boley, H. (2003). RACOFI: A rule-applying col-
Gutiérrez, S., Valigiani, G., Jamont, Y., Col-
laborative filtering system. In 2003 IEEE/WIC
let, P., & Delgado Kloos, C. (2007). A swarm
International Conference on Web Intelligence/
appoach for automatic auditing of pedagogical
Intelligent Agent Technology.
planning. In Proceedings of IEEE ICALT 2007
Bonabeau, E., Dorigo, M., & Theraulaz, G. (1999). (pp. 136-138).
Swarm intelligence: From natural to artificial
Hartley, J., & Sleeman, D. (1973). Towards more
Systems. NY: Oxford University Press
intelligent teaching systems. International Jour-
Costa, D., Hertz, A., & Dubious, O. (1995). nal of Man-Machine Studies, 2, 215-336.
Embedding of a sequential algorithm within an
Hofmann, T. (2003). Collaborative filtering via
evolutionary algorithm for coloring problems in
Gaussian probabilistic latent semantic analysis.
graphs. Journal of Heuristics, 1, 105-128.
In 26th ACM SIGIR Conference on Research in
Deshpande, M., & Karypis, G. (2004). Selective Information Retrieval (pp. 259-266).
Markov models for predicting Web page accesses.
Kauffman, S. (1996). At home in the universe:
ACM Transactions on Internet Technology (TOIT),
The search for the laws of self-organization and
4(2), 163-184.
complexity. Oxford University Press.
Dorigo, M., & Stützle, T. (2004). Ant colony
Kennedy, R., & Eberhart, R. (2001). Swarm
optimization. MIT Press.
intelligence. CA: Morgan Kaufmann/Academic
Dron, J. (2002). Achieving self-organisation in Press.
network-based learning environments. PhD
Koper, R. (2005). Designing learning networks
doctoral dissertation.
for lifelong learners. In R. Koper & C. Tattersall
Dron, J., Mitchell, R., Siviter, P., & Boyne, C (Eds.), Learning design: A handbook on modelling
(1999). CoFIND: Experiment in n-dimensional and delivering networked education and training
collaborative filtering. In World Conference on (pp. 239-252).
the WWW and Internet (pp. 301-306).
Mertens, R., Farzan, R., & Brusilovsky, P. (2006).
Gambardella, L.M, Taillard E., & Agazzi G. Social navigation in Web lectures. In U. K. Wiil,
(1999). MACS-VRPTW: A multiple ant colony P. J. Nürnberg & J. Rubart (Eds.), Proceedings of
system for vehicle routing problems with time Hypertext Conference 2006.
windows. New Ideas in Optimization, 63-76.
Miyahara, K., & Pazzani, M. (2000). Collaborative
Gould, S. J. (1978). Ever since Darwin - reflections filtering with the simple Bayesian classifier. In
in natural history. Burnett. Pacific Rim International Conference on Artificial
Intelligence (pp. 679-689).
Gutiérrez, S., Pardo, A., & Delgado Kloos, C.
(2006a). A modular architecture for intelligent Morley, R. (1996). Painting trucks at general
Web resource based tutoring systems. Intelligent motors: The effectiveness of a complexity-based
Tutoring Systems, 753-755. approach. In Ernst and Young Center for Business
Innovation, (Ed.), Embracing Complexity: Explor-
ing the Application of Complex Adaptive Systems
(2006b). Some ideas for the collaborative search
to Business (pp. 53-58). Cambridge, MA.
0
Murray, T. (1999). Authoring intelligent tutor- AddItIonAL reAdIng

ing systems: An analysis for the state of the art.
International Journal of Artificial Intelligence in Dron, J. (2005). E-learning and the building habits
Education, 10, 98-129. of termites. Journal of Educational Multimedia
and Hypermedia, 14(4), 321-342.
Pennock, D., Horvitz, E., Lawrence, S., & Lee
Giles, C. (2000). Collaborative filtering by per- Dron, J. (2006). Social software and the emergence
sonality diagnosis: A hybrid memory- and model- of control. In 6th IEEE Conference on Advanced
based approach. In 16th Conference on Uncertainty Learning Technologies (pp. 904-908).
in Artificial Intelligence (pp. 481-488).
Dron, J. (2007). The safety of crowds. Journal of
Semet, Y., Lutton, E., & Collet, P. (2003). Ant Interactive Learning Research, 18.
colony optimization for e-learning: Observing
Dron, J., Boyne, C., & Mitchell, R. (2001). Foot-
the emergence of pedagogic suggestions. In Pro-
paths in the stuff swamp. In World Conference
ceedings of the 2003 IEEE Swarm Intelligence
on the WWW and Internet (pp. 323-328).
Symposium.
Dron, J., Boyne, C.W., & Mitchell, R. (2004).
Tattersall, C., Manderveld, J., Van den Berg, B.,
The evaluation of forms of assessment using n-
Van Es, R., Janssen, J., & Koper, R. (2005). Swarm-
dimensional filtering. International Journal on
based wayfinding support in open and distance
E-Learning, 3(4).
learning. In E. M. Alkhalifa (Ed), Cognitively
informed systems: Utilizing practical approaches Dron, J., Mitchell, R., & Boyne, C.W. (2003).
to enrich information presentation and transfer Evolving learning in the stuff swamp. In N. Patel
(pp. 166-183). (Ed.), Adaptive evolutionary information systems
(pp. 21-218). Hershey, PA: Idea Group.
Valigiani, G., Jamont, Y., Biojout, R., Lutton E.,
& Collet, P. (2005). Experimenting with a real- Farzan, R., & Brusilovsky, P. (2006a). AnnotatEd:
size man-hill to optimize pedagogical paths. A social navigation and annotation service for
In H. Haddad, L. Liebrock, A. Omicini & R. Web-based educational resources. In World Con-
Wainwright (Eds.), ACM Symposium on Applied ference on E-Learning.
Computing (pp. p4-8).
Farzan, R., & Brusilovsky, P. (2006b). Social
Valigiani, G., Lutton, E., Jamont, Y., Biojout, R., & navigation support in a course recommendation
Collet, P. (2006). Automatic rating process to audit system. In V. Wade, H. Ashman & B. Smyth
a man-hill. WSEAS Transactions on Advances in (Eds.), 4th International Conference on Adaptive
Engineering Education, 3(1), 1-7. Hypermedia (pp. 91-100).
van Bruggen, J., Sloep, P., Van Rosmalen, P., Gutiérrez, S. (2007). Sequencing of learning
Brouns, F., Vogten, H., & Koper, R. (2004). Latent activities oriented towards reuse and auto-or-
semantic analysis as a tool for learner positioning ganization for intelligent tutoring systems. Un-
in learning networks for lifelong learning. Brit- published doctoral disertation, University Carlos
ish Journal of Educational Technology, 35(6), III of Madrid.
729-738.
Gutiérrez, S., Pardo, A., & Delgado Kloos,
C.(2004). An adaptive tutoring system based
on hierarchical graphs. In P. de Bra & W. Nejdl

(Eds.), 3rd Conference on Adaptive Hypermedia Tattersall, C., Van den Berg, B., Van Es, R.,
(pp. 401-404). Janssen, J., Manderveld, J., & Koper, R. (2004).
Swarm-based adaptation: Wayfinding support
for lifelong learners. In P. de Bra & W. Nejdl
(2006). Finding a learning path: A swarm intel-
(Eds.), 3rd Conference on Adaptive Hypermedia
ligence approach. In Proceedings of IASTED
(pp. 336-339).
Web-Based Education.
Van den Berg, B., van Es, R., Tattersall. C., Jans-
Janssen, J., Tattersall, C., Waterink, W., Van den
sen, J., Manderveld, J., Brouns, F., et al. (2005).
Berg, B., Van Es, R., Bolman, C. et al. (2007).
Swarm-based sequencing recommendations in
Self-organizing navigational support in lifelong
e-learning. Intelligent Systems Design and Ap-
learning: how predecessors can lead the way.
plications, 3(3), 1-11.
Valigiani, G., Jamont, Y., Biojout, R., Lutton,
Klamma, R., Chatti, M. E., Duval, E., Hummel,
E., Fonlupt, C., & Collet P. (2006). Man-hill op-
H. G. K., Hvannberg, E. T., Kravcik, M., et al.
timization of pedagogical paths in an e-learning
(2007). Social software for life-long learning.
system. Special issue on artificial evolution. In
Technology & Society, 10(3), 72-83.
P. Collet (Ed.), Techniques et Sciences Informa-
Kurhila, J., Miettinen, M., Nokelainen, P., & Tirri, tiques. Hermes.
H. (2002). EDUCO—A collaborative learning
Valigiani, G., Lutton, E., Jamont, Y., Biojout, R., &
environment based on social navigation. In 2nd
Collet, P. (2006). Automatic rating process to audit
Conference on Adaptive Hypermedia.
a man-hill. WSEAS Transactions on Advances in
Lakoff, G. (1987). Women, fire, and dangerous Engineering Education, 1(3), 1-7.
things. IL: University of Chicago Press.
Valigiani, G., Lutton, E., & Collet P. (2006).
Mobasher, B. (2004). Web usage mining and Adapting the ELO rating system to competing
personalization. In M. P. Singh (Ed.), Practical sub-populations in a man-hill. In 13th ISPE Inter-
handbook of Internet computing. Chapman & national Conference on Concurrent Engineering
Hall/CRC Press. (pp. 766-774). IOS Press.
Prieto Linillos, P., Gutiérrez, S., Pardo, A., & Valigiani, G., Lutton, E., Jamont, Y., Biojout,
Delgado Kloos, C. (2006). Sequencing parametric R., & P. Collet (2005). Evaluating a real-size
exercises for an operating system course. IFIP man-hill. In WSEAS International Conference
Artificial Intelligence Applications and Innova- on E-Activities.
tions, 450-458.
Zlochin, M., & Dorigo, M. (2002). Model-based
Socha, K., Knowles, J., & Sampels, M. (2002). A search for combinatorial optimization: A com-
MAX-MIN ant system for the university timeta- parative study. In 7th International Conference
bling problem. In Third International Workshop on Parallel Problem Solving from Nature (pp.
on Ant Algorithms (ANTS 2002) (pp. 1-13). 651-661).
Tattersall, C., Manderveld, J., van den Berg,
B., van Es, R., Janssen, J., & Koper, R. (2005).
Self organising wayfinding support for lifelong
learners. Education and Information Technolo-
gies, 10(1-2).

Chapter XII
E-Learning 2.0:
The Learning Community
Luisa M. Regueras
University of Valladolid, Spain
Elena Verdú
María A. Pérez
Juan Pablo de Castro

María J. Verdú
ABstrAct
Nowadays, most of electronic applications, including e-learning, are based on the Internet and the Web.
As the Web advances, applications should progress in accordance with it. People in the Internet world
have started to talk about Web 2.0. This chapter discusses how the concepts of Web 2.0 can be transferred
to e-learning. First, the new trends of the Web (Web 2.0) are introduced and the Web 2.0 technologies
are reviewed. Then, it is analysed how Web 2.0 can be transferred and applied to the learning process,
in terms of methodologies and tools, and taking into account different scenarios and roles. Next, some
good practices and recommendations for E-Learning 2.0 are described. Finally, we present our opinion,
conclusions, and proposals about the future trends driving the market.
E-Learning 2.0
IntroductIon • To analyse how Web 2.0 can be transferred

and applied to the learning process, in terms
Immersed in a tremendous crisis, the evolution of of methodologies and tools, and taking into
the Web seemed to have collapsed. In a context account different scenarios and roles
where the business crashed and most future ex- • To present good practices and recommenda-
pectations were about to be frustrated, the Web tions for E-Learning 2.0
was redefined as Web 2.0 bringing back many • To present our opinion, conclusions, and
believers who were previously deceived. proposals about the future trends driving
For some it was a “web quake,” for others the market
simply a new word with an old meaning. But,
what is unquestionable is that it has contributed
to a relaunch of the Web evolution, renewing the revoLutIon In the weB:
influence of the Web in our society and affecting weB 2.0
most areas of the citizens’ life. In this chapter,
we will present some examples of how Web 2.0 Historically there have been some “killer apps”
affects different aspects of our lives. in the Internet world. First, there was the e-mail,
Social software or collaboration are key ele- which is the most used application in the Inter-
ments of the Web 2.0, but is their technology net. Second, there was the World Wide Web and
strictly new or is it simply a new approach based browsers, which have been established as the
on collaboration and participation? Technology dominant software platform for clients in the
will also have a place in this chapter. We will try network. Recently, a new kind of services has
to discover if the technology is new or simply come on stage earning the right to create a new
dressed up or reused. denomination: Web 2.0.
After examining society and technology we According to Wikipedia, the term Web 2.0
will focus on education: Does E-Learning 2.0 refers to a second generation of services available
exist? What is it exactly? Has it reached educa- on the Web that lets people collaborate and share
tion centres? information online.
We will discuss new challenges and opportuni- As O’Reilly (2005) explains:
ties and show how Web 2.0 can be transferred to
classrooms. A discussion about methodologies, The concept of Web 2.0 began with a conference
tools, scenarios, and roles for E-Learning 2.0 will brainstorming session between O’Reilly Media
also be part of this chapter. and MediaLive International. Dale Dougherty,
Finally, some recommendations and guidelines Web pioneer and vice-president of O’Reilly Me-
for good practices will be provided and, to con- dia, noted that far from having crashed, the Web
clude, we will present our opinion and proposals was more important than ever, with exciting new
about the future trends forced by the market and applications and sites popping up with surprising
the environment. regularity. In their discussion, they found that the
In summary, the objectives of this chapter are companies that had survived the collapse seemed
the following: to have some things in common, and they thought
that the dot-com collapse could have marked
• To make readers aware of the new trends of some kind of turning point for the Web. Then, they
the Web (Web 2.0) agreed that a call to action might make sense and,
• To review Web 2.0 technologies therefore, the Web 2.0 was born.

E-Learning 2.0
Table 1. Different services implementation. Web 1.0 vs. Web 2.0 (Adapted from O’Reilly, 2005)
Web 1.0 Web 2.0

Platform Netscape, Explorer… Google services, AJAX
Web pages Personal Web sites Blogs
Word processor Microsoft Word Google Docs (Writely)
Portals Content Management Systems Wikis
Encyclopedia Britannica Online Wikipedia
Knowledge Directories (taxonomy) Tagging (folksonomy)
References URLs Syndication, RSS
Lookup Domain name speculation Search engine optimization
Role Publishing Participation, collaboration
Media provision Netmeeting Skype
Content Akamai (Content delivery) BitTorrent (P2P)
Metrics Page views Cost per click
Table 1 shows many of the differences between the success of Web 2.0 or “social Web,” a lot of
the two approaches. The most popular service of people are getting ready for the boom of the next
the new Web generation is known as blog (from generation of the Internet: Web 3.0 or the “semantic
Binnacle Log). The blog is a Web site periodically Web.” As a mainstream tool, the Web is still in
updated that chronologically compiles contents its infancy and there is plenty of experimentation
of one or several authors where the most recent still happening as people grapple with how to use
appears first. Contents may be texts, pictures, it (Evans, 2006).
audio,or video and the style is usually personal
and nonformal. key techonologies for web 2.0
Other good representative of these new ser-
vices is the wiki (from the Hawaiian word wiki With Web 2.0 being more a philosophy of service
meaning fast), which is a type of Web site that than a new system, it may be hard to talk about the
allows visitors themselves to easily add, remove technology that runs it. In fact, every known tech-
or otherwise edit and change some available nology effectively supports the services of Web
content, sometimes without the need to register. 2.0. Actually, the current services that everyone
This easiness of interaction and operation makes would classify as Web 2.0 run on old platforms
a wiki an effective tool for collaborative author- and well proven technologies for Web 1.0.
ing. Perhaps the best-known use of a wiki is the There exists a handful of technologies that
Web-based encyclopedia, Wikipedia (http://www. support the majority of services in use today or
wikipedia.org), which is the maximum exponent envisioned for the next years, like HTTP as appli-
of collaborative effort in the Internet. cation communication protocol, XML and HTML
At this stage, there are proofs that 2006 has as content representation, and Java, ASP, and PHP
been the year of Web 2.0, such as the success as scripting languages for the server side.
of social communication sites (like LinkedIn, These are also well known general-purpose
MySpace, YouTube or Facebook), where the social technologies that have been fuelling the overall
technology (like blogs or wikis) have spread like Internet industry for several decades so that they
wildfire. However, when most users are tasting do not contribute specially to Web 2.0.

E-Learning 2.0
Really, the new way of using these technolo- computing services. These are named collectively
gies is what makes Web 2.0 a new real and cool as Web Services.
concept that deserves an own name. The key to success is that these services mainly
From an architectural point of view, some rely on standardized XML documents and HTTP
techniques developed as standards have to be transport, which are available in most computing
remarked. Some of them are Web services (with platforms. Hence, this approach can work in many
related technologies like REST, SOAP, UDDI, and environments with less infrastructure changes
so forth), syndication “protocols” such as RSS and than with older frameworks.
ATOM, AJAX as a new technique to freshening Web services are described by means of an
up the world of Web user interfaces, and peer to XML document known as Web service descrip-
peer networks and ubiquitous networking with tion language (WSDL) and made public in a
presence indication (SIP). All these concepts will registry named universal description, discovery
be reviewed in the following sections. and integration (UDDI). The messages involved
in the communication are XML documents with
Web Services in a Brief a schema known as simple object access protocol
(SOAP).
In the past, many services have been developed All of these components make up an infrastruc-
from scratch by voluntaries or professionals once ture to bind independent and remote systems and
and again. Most of the research on computing has configure a complex and more powerful service.
been focused on reusing and integrating previous For example, a small Web service could borrow the
efforts. This leads to a vast variety of standards, powerful search engine and mapping tools from
frameworks, libraries, Application Programming Google (http://code.google.com), mix them with
Interfaces (APIs), and component architectures the latest headlines of world news from BBC and
that conform a world of islands of standardized some local content with almost no cost.
computing technologies. Many bridges, gateways,
tunnels, and connectors have, of course, been built new Look for the Browser: AJAx
between them, but the result consists of a very
heterogeneous technology background. Simultaneously to the development of the previ-
In order to make modern Web 2.0 systems ous technological framework, the world of user
work, many sources of information and comput- interfaces has evolved too. Web applications have
ing resources should be orchestrated to compose early adopted a different model of programming
a powerful and popular service. This should be from the old Windows-based applications, called
accomplished by means of a common channel of representational state transfer (REST). In REST,
communication and method invocation protocols. the flow of the application is encoded in the docu-
In the past, many vendors have tried to stan- ments exchanged from the server to the client
dardize their own version of distributed method instead of being stored as persistent objects at any
invocation technologies such as DCOM, RMI, of the sides. This can be observed in every Web
or CORBA. However, although some of them page, where the logic of the application is repre-
are very popular, none of them has achieved the sented in the content itself, mainly in the form of
degree of universal standard. hyperlinks. During a decade, almost every mass
Recently, a set of standards proposed by the service on the Web relied on this architecture in
World Wide Web Consortium have been proposed order to relieve the servers’ load and allow more
as a respected and powerful common infrastruc- requests to be served.
ture of communication between independent

E-Learning 2.0
Figure 1. Differences between traditional Web applications (upper figure) and AJAX (lower figure). Note
the difference in servers load and the increase of interactivity.
Recently, some new patterns of Web applica- due to the cost of acquiring such information
tions have risen: Asynchronous Javascript and in a timely fashion. On the other hand, many
XML (AJAX). There seems to be a back step information providers can not obtain enough
to the age of heavy-weight client applications social repercussion in a monopolized mass media
but with a Web browser as innovative running market. Moreover, the advent of new community
environment. Root technologies for AJAX are driven services, such as blogs, has increased the
XML as format for communication, XHTML offer of information and has made social impact
and DHTML as graphical user interface toolkit, more difficult. For providers it is worth sharing the
and a full evolved Javascript support at browser content with popular services, and for the later it is
level. With a careful development of Javascript worth feeding from them. This process of mutual
logic in the documents, a full-interactive and fast- benefit is generally called Syndication.
response application could be deployed without In Web 2.0, users build spontaneous networks
installing any additional software at the client (see and communities (blogosphere) and content is
Figure 1). The result is a Web page that shows a selected and promoted by means of an organic-
cool aspect and reacts smoothly. The irritating like mechanism of selection and propagation.
reloads of pages and waiting times are avoided Interesting sources of information are cited and
and the updates on the screen happen alike they linked in more sites and their global ranking is
are expected in traditional client applications. calculated by measuring the interest of the com-
munity on them.
syndicating and mashing In order to allow such mechanism, a technical
solution should be used. This involves mainly
As the success of a service relies mainly on the a standard language of publication: RSS and
quality of content, several projects have failed ATOM are the most popular. They are ways of

E-Learning 2.0
representing information that is accessed by means wInds oF chAnge BLowIng

of classical protocols, hence, they are more lan- the weB: weB 2.0 And the
guages than protocols, but, as they include state InFormAtIon socIety
information and instructions for the access and
retrieval of information, they are usually known Web 2.0 affects all aspects of life, from the private
as protocols. Nowadays, Web service providers and fun sphere to work environments, from an
are using RSS/ATOM feeds as lightweight alterna- individual level to a group level. In short, Web
tives to SOAP. Developers are finding new ways 2.0 becomes an enabler to (Prinz, 2006):
to combine Web services from different sites into
new applications, known as “mash-ups” in the • Turn an Internet presence into a customer
lingo of Web 2.0. experience site
RSS states for Really Simple Syndication, and, • Turn a passive consumer into a creative
indeed, it is. RSS is a very simple document that content provider
provides users with a title, an abstract and a link • Enforce the link between Internet users and
to the information. To overcome the limitations Internet places
of such a simple solution, a more flexible and • Support the self-organisation of communi-
powerful format have appeared with the name ties
of ATOM. • Create a social Web on top of the technical
Web
other networks in the network
Currently, we can access lots of information
Although the Web is the most successful and and, at the same time, write our own content or
representative service in the Internet, we should leave comments. We are now the content genera-
not forget other interesting and very promising tors. Education, marketing, journalism, and even
networked services that are also earning great politics is moving towards the reality 2.0.
acceptance and expectation. Marketing 2.0 is about getting integrated into
Peer to peer networks are frequently proposed the message “constellation” (Thorson, 2006). Web
as the alternative to a centralized and easy-to-con- 2.0 gives companies the opportunity to reach
trol network. These services rely on the computing thousands of customers at a much lower cost.
resources of every participant, instead of relying Companies need to understand that their custom-
on the giant servers of commercial providers. ers are their best marketers; a positive comment
This approach allows supporting services with about a product or service is still more powerful
the resources provided by the users themselves. than any commercials or print ads. Until now, those
There are no scalability problems as newcomers comments took place at cocktail parties, where
contribute with new resources to the network. very few people knew about them. However, now
Some examples of this approach are file shar- the place is the blogosphere, where everybody can
ing, in which bandwidth and storage are shared read and comment about them. In this sense, Polo
(eDonkey, eMule or BitTorrent), computing grid (2006) gives her opinion about the power of blogs:
projects, in which spare computing time is shared “imagine a database in which you can search lots
(SETI at home or BOINC), and Internet telephony, of conversations about your products, with good
in which bandwidth and geographical proximity and bad opinions... terrifying, isn’t it?”
are shared (Skype). Another interesting example is journalism,

E-Learning 2.0
where the blog is also being extensively used as a Broadly conceived, Web-based social software
tool to allow a more direct and quick communica- could encompass older media, but some would
tion with readers, especially in the cases where restrict its meaning to more recent phenomena
journalists cover the news in the exact place where such as blogs and wikis (as defined in Wikipedia,
they happen as, for example, war reporters. http://en.wikipedia.org/wiki/Social_software).
Moreover, some citizens are already using the Social revolution is being even more important
blogosphere as political space. The blog must not and extraordinary than the technological revolu-
be a simple electoral propaganda element. Cur- tion. Maybe the most visible aspect of this revolu-
rently, politicians use blogs in order to talk about tion is the use of tools such as blogs or wikis that
general subjects, transmit their own ideas, explain are contributing to change from a Read Web to
their activity, inform about their responsibility the Read/Write Web that Tim Berners-Lee, the
areas, listen to and receive citizens’ opinions, creator of the HTML, envisioned. In the context
and share knowledge. of education, this trend will transform the way we
On the other hand, Web 2.0 facilitates a change have taught so far. Learners will not only learn,
in the way people interact online, so that a lot but they will also interact and share knowledge
of daily activities are being affected, even some since they count with tools that will allow them
that have not surrendered so far, changing habits to do it easily. The process of producing Web
strongly established. An interesting example is the contents and, therefore, e-learning materials, can
use of the blog as a public personal diary. be started and developed almost without any type
of technical knowledge and without an excessive
time investment. This fact makes the launching
e-LeArnIng 2.0 reAches of e-learning experiences an easier task and lets
cLAssrooms the teacher to remain a teacher and does not
oblige him to become an expert in information
How can the concepts of Web 2.0 be transferred to and communication technologies (ICT).
e-learning? This is something that the e-learning According to Wikipedia, the momentum in
community is wondering at the moment. the area of e-learning is based on the confluence
As well as Web 2.0 is not a completely defined of several important trends:
context but a constantly developing environment
instead, so is E-Learning 2.0. • Dramatically lower effort to compose e-
Regarding collaboration and information learning solutions based on Web 2.0 tech-
sharing, Web 2.0 gives users an experience that is nologies and tools
richer and more dynamic than the traditional static • Demand in corporate settings for training
Web pages, which were generated by individuals that requires time and engages learners in
or closed groups. the process over a course of time
In fact, one of the major components of the • Recognition in e-learning of the importance
Web 2.0 movement is the social software. The idea of blended learning
dates as far back as the 1960s and JCR Licklider’s • The trend toward student centred design
thoughts on using networked computing to con- • The theory of connectivism (Siemens,
nect people in order to boost their knowledge 2005)
and learning ability (Alexander, 2006). Social • Free-Libre Open Source Software (FLOSS)
software enables users to rendezvous, connect and open access
or collaborate through computer-mediated com- • Educational blogging
munication and to form online communities.

E-Learning 2.0
Putting all this together, we could talk of E- learning environments (PLE) using social
Learning 2.0 as the learning community that uses software. Students generate and share con-
active ICT-based collaboration and communica- tent. They interact not only with teachers
tion as the main base of the learning process. and their peers, but with anyone in the world
According to O’Reilly (2005), “there is still they can learn from.
a huge amount of disagreement about just what
Web 2.0 means, with some people decrying it as Elsewhere FitzGerald (2006) states:
a meaningless marketing buzzword, and others
accepting it as the new conventional wisdom.” In The irony of all this is that we are returning to the
our opinion, Web 2.0 is not really new, but it is a way things were before modern schools were in-
new term successfully used to refer to the recent vented. It used to be that learning occurred within,
changes experimented by the Web that, with more and supported by, the community. After schools
than 237 million citations in Google in October were developed students were removed from the
of 2006, has clearly taken hold. community into the mainly artificial environment
A similar discussion could take place about of the classroom where they were expected to learn
E-Learning 2.0: is it really a different scenario content divorced from context.
for e-learning?
There are very critical opinions. For example As Downes says, “there is nothing more virtual
Jennings (2005) states that E-Learning 2.0 is a than the classroom! Just think how students learn
rhetorical manoeuvre by e-learning suppliers foreign languages and consider how teachers try
and consultants to distance themselves from the to recreate authentic France in their classrooms”
failures of the first wave of e-learning, or the (Blamire, 2006).
bastard neologism offspring of e-learning and E-learning as we know it has been around for
Web 2.0 technologies. 10 years or so. During that time, it has emerged
Regarding the definition of E-Learning 2.0, from being a radical idea—the effectiveness of
an interesting discussion has taken place in the which was yet to be proven—to something that
EdNA (Australia) groups on social network- is widely regarded as mainstream. It is the core
ing: philosophy and pedagogy. In this context, to numerous business plans and a service offered
FitzGerald (2006) discusses whether Web 2.0 is by most colleges and universities. And now, e-
matched with E-Learning 2.0: learning is evolving with the World Wide Web as
a whole and it is changing to a degree significant
• E-Learning 1.0 was static packaged content enough to warrant a new name: E-Learning 2.0
developed by content developers such as (Downes, 2005b).
CDROMs and courseware. It had little true
interactivity and learner input and very little
(if any) contact with a tutor. mEThoDoLoGiES, TooLS, anD
• E-Learning 1.5 is best represented by chALLenges In e-LeArnIng 2.0
Learner Management Systems (LMS). Some
packaged content and some provided by the Once the technology and tools are available, the
teacher. There is more interaction with a main emphasis has to be made on the methodol-
teacher and some with peers (through forums ogy to be used in order to design and implement
and chat). the new learning strategies and procedures, tak-
• E-Learning 2.0 will follow a student-centred ing into account the scenarios and roles of the
model and will be centred on the personal different agents that take part in the learning
0
E-Learning 2.0
process. Along this section, some classifications an innovative way of constructing knowl-
for the e-learning key elements are established edge.
and commented. • (Edu)Blogs are increasingly finding a home
According to the students’ social interdepen- in education. Blogs remove the technical
dence and interaction in the classroom, learning barriers to writing and publishing online,
methods can be classified in the following three encourage students to keep a record of
types: their thinking over time and facilitate
critical feedback, by letting readers add
• Individualistic learning, which means comments—which could be from teachers,
“working by oneself to ensure that one’s peers, or a wider audience.
own learning meets a present criterion • Flickr is a free photo sharing site which
independently from the efforts of the other provides to teachers and students an easy
students” (Johnson & Johnson, 1999). way to upload, share, and add notes to the
• Cooperative learning, which is the instruc- photos on the Web.
tional use of small groups so that students • Google Docs has quickly jumped into the
work together to maximize their own and educational field. It is an easy-to-use online
each other’s learning. (Johnson, Johnson, & word processor, so that students can access
Holubec 1998). Students are rewarded on the from anywhere with an Internet connection
basis of the success of the group (Woolfolk, and work collaboratively.
2001). • Synchronous communication tools (such
• Competitive learning, which is mainly based as videoconferencing or chat) allow online
on activities carried out by students that communication in real time and can be
compete individually or in teams. Students helpful in assisting group work and peer
work against each other to achieve a good learning among students.
grade and only some of them succeed. • Asynchronous communication tools (such
as e-mail or newsgroups) allow time-inde-
Cooperative learning may be contrasted with pendent interaction between participants
competitive learning and individualistic learn- and are well suited to cooperative learning
ing. In addition, within competitive situations, strategies.
individuals seek outcomes that are beneficial
to themselves and detrimental to others. The Forums, blogs, wikis, flickr, and so forth, can
student effort is on performing faster and better all be created to surround your course with an ex-
than classmates. (Johnson & Johnson, 1999). panded set of learner resources. But, it is a need to
However, team competitions could combine the be very careful to ensure that you understand who
better of competitive learning with the better of you teach, that is, the learner’s scenario: primary
the collaborative one. school, secondary school, university education,
On the other hand, there are some “social” long-life learning, or informal learning.
tools that are very useful for e-learning: Finally, the role of teachers and students
should change.
• Wikis provide unique collaborative oppor-
tunities for education. Combining freely • Teacher: from leader to facilitator. The
accessible information, rapid feedback, teacher’s role should go from absolutely
simplified HTML, and access by multiple controlling and leading everything that hap-
editors, wikis are being rapidly adopted as pens in the classroom to staying aside and
accompanying students along their walk.

E-Learning 2.0
• Student: from passive to active. Students all, it is necessary to know and learn the use and
can rely on their teachers and class sessions features of Web 2.0 tools. This is a prerequisite
when learning something or evaluating, in every scenario and for every participant, in-
making decisions, and being responsible dependently of the teachers’ and students’ role
for their own learning. and of the methodology. Digital literacy must
cover every tool to be used, including of course,
In this context, E-Learning 2.0 is mainly synchronous and asynchronous communication
oriented to collaborative (or mixed competitive- tools, but specially focusing on the new tools and
collaborative) methodologies applied in each and their possibilities.
every scenario where the teacher tends to be a Moreover, it is necessary that students and
facilitator and the student is encouraged to be teachers are familiar and comfortable with col-
as active as possible. In addition, relatively new laborative writing and the use of these new tools.
tools, such as blogs, have already found their The social and cultural practices of collaborative
place in the e-learning framework: the edublogs. working are not in the students’ repertoire of
Examples of edublogs can be found in Web shared practices yet. So, for example, Grant (2006)
sites such as Weblogs of teachers (http://www. implemented a 3-week wiki writing segment in
superblog.org/planet/educacion), Aulablog (http:// her class of 13- to 15-year-olds and found that her
www.aulablog.com) or Edublogs (http://www. students had great difficulties writing in a public
edublogs.org). space and altering other student’s wiki work.
The collaborative aspect of blogs is what has The second important challenge has its origin
brought many teachers into the fold. Comment- in the anarchy and uncontrolled character of the
ing capabilities in many of the blogging software Internet. E-Learning 2.0 encourages socializa-
packages provide students and teachers with an tion, networking, participation, and collaboration,
easy peer review tool and make easier bringing in but how can we be sure that learners are out of
experts from outside the classroom. In this respect, all dangers that can be found in the net? These
an interesting experience is the one developed dangers can be simply distractions when students
by Will Richardson (pioneer educational blog- waste time in activities that do not contribute to
ger). In 2002, his school had adopted Sue Monk the learning objectives, but they can also be really
Kidd’s book, The Secret Life of Bees, as part of harmful as digital predators know how to virtually
the literature curriculum, and he decided to set capture their preys. This challenge is especially
up a blog for students to converse about the book important in primary and secondary education
outside class. With great delight, he watched the and informal learning where the student’s family
students post ideas as well as artistic interpreta- and environment must take a special care of the
tions. He also posted a blog for parents to join paths the teenager walks.
the discussion. The result was a truly democratic Finally, another aspect is that these new tools
learning space (O’Hear, 2006). are being used to do the same. In this sense, blogs
It is also important to underline that E-Learn- may seem to be replacing the classic Webs that
ing 2.0 raise new challenges or barriers. So, for teachers used as a notice board or as a space to
example, digital literacy should be extended to deliver materials for their students. At the same
cover the new tools (such as blogs, wikis, and so time, the efforts to motivate students to create
forth), what will be essential for the definitive their own blogs obey mainly to the classic patterns
launching of E-Learning 2.0. There are a lot of of learning practices: the teacher usually suggest
people who still have to learn there is life on the the content, periodicity of updating, number of
Web beyond e-mail and e-commerce. First of posts, style, type, and number of links, and so

E-Learning 2.0
forth, and, of course, establishes the criteria to Wikis are already making their mark in higher
evaluate the work done and its contribution to the education and are being applied to just about
final assessment of the student. any imaginable task. They are popping up like
The result is that new tools and technologies mushrooms, being faithful to the word wiki ( fast),
surrender to the predominant learning models and at colleges and universities around the world,
practices. In this way, a false taste of innovation sometimes in impromptu ways and more often
is acquired. The real innovation would only exist with thoughtful intent. Bellow, different ways to
if new technologies and tools are provided with use Wiki in education are shown:
new learning approaches.
The simple use of new tools does not guarantee • Easily create simple Web sites. Typically
by itself a change in the e-learning scenario or in when students are asked to create Web sites
results. An important challenge is to define and as part of a class project, they have to rely on
design new approaches and strategies. It is here the chance that someone in a group knows
where the E-Learning 2.0 concept is valuable and how to make a Web site, or that some sort
makes sense: the discussion about E-Learning of training is available. The wiki allows
2.0 may be the reason to slow down and redefine students to spend more time developing
e-learning with a more constructivist and col- the content of the site, instead of trying to
laborative base. learn how to make one. As more organiza-
tions adopt the wiki for collaboration and
information sharing, students will be well
Best PrActIces prepared to use it in their careers (Pearce,
2006).
Every learning experience based on constant • Course Web site. Eukaryotic Genetics and
communication and collaboration through ICT Molecular Biology course at the University
and where participants are active members of an of Maryland, Baltimore County (UMBC)
online learning community could be considered uses a wiki as course Web site. Moreover,
as a contribution to E-Learning 2.0. But which the Career Services unit at the University
are the best practices? of British Columbia is using wiki pages to
For students, a blog can be used as a living store and organize content for a major site
record of their learning: a place to pose questions, redevelopment presenting job postings and
publish work in progress, or provide links to (and career education (Brian, 2004).
comments on) relevant Web resources. Teachers • Project development with peer review.
may want to start their own subject-based blog Students can use a wiki to develop a term
where they can provide up-to-date information paper, and might start by tracking their
and comments on their subject area, as well as background research. Taking advantage of
posting questions and assignments, and linking the automatic revision history, the teacher
to relevant news stories and Web sites (O’Hear, and peers can see the evolution of the paper
2005). over time, and continually comment on it,
As with blogs, wiki software makes it possible rather than offering comments only on the
to publish a Web site with very little technical final draft (Pearce, 2006).
knowledge, putting a greater emphasis on col- • Group authoring. Using a wiki “pulls”
laborative rather than on personal publishing. group members together to build and edit
Every wiki entry has an “edit this page” button the document, which strengthens the com-
so that users can not only add new content but munity within the group. It also provides
make changes to existing pages.

E-Learning 2.0
immediate, equal access to the most recent tell about their experiences during the class
version of the document for all group mem- and leave advice for the next group.
bers (Pearce, 2006). • Presentations. Some people are using a
- Track a group project. Considering stu- wiki instead of conventional presentation
dents’ busy schedules, a wiki is very useful software, like Keynote and PowerPoint.
for tracking and completing group projects. It • Social interaction. Wikis have been used
allows group members to track their research successfully to enable hundreds of students
and ideas from anywhere with Internet ac- to participate in a collaborative icebreaker
cess, helps them save time by seeing what exercise at Deakin University. This project
sources others have already checked, and illustrates how e-learning practitioners
gives them a central place to collectively can use wiki technology to enhance social
prepare the final product. interaction amongst students online (Augar,
According to Pearce (2006), one way to do Raitman, & Zhou, 2004).
this is to give each group a wiki page in which • Planning a conference. An academic
to write a paper, and give each member of research unit at the University of British
the group a separate page to track his/her Columbia used a wiki for planning a tech-
research and ideas for the paper. The “paper” noculture conference, in order to collect sup-
page lets you see how the group is working porting resources and gather contributions
collaboratively, and individual pages let you from invited participants. They used the
track how each group member is developing wiki during the conference to record group
contributions to the paper, and gives you a work. Participants subsequently edited their
place to leave feedback and suggestions for collaborative authorings, resulting in a
each student. “conference proceedings” of an altogether
• Tracking progress in your research group. different sort.
Teachers can use the wiki in order to main-
tain a journal of work performed on group Moreover, when attempting to develop student-
projects. created wikis at the school, McPherson (2006)
• Information repositories or teacher-li- recommends starting with easy-to-manage wiki
brarians. For example, the SUNY Geneseo projects. For example, a possible wiki writing
Collaborative Writing Project at the State project for primary-aged children is to create an
University of New York that consists in animal alphabet wiki. Individuals, pairs, or groups
a Dictionary of Literary Terms (Schacht, of primary students can (a) choose an animal, (b)
2006). select a picture of that animal from the Internet
• Review and debate classes and teachers. and, (c) insert the picture and the first letter of the
Students at Brown University have started animal into their wiki. Intermediate students can
the Course Advisor Wiki, a place for students use a wiki to create a story with multiple begin-
to collaboratively write reviews of courses nings and endings or, as another example, develop
they have taken. This wiki gives reviewers their online and off-line map-reading and writing
flexibility to articulate their impressions, skills by collaboratively adding descriptive text to
while readers get richer reviews that combine an online map, such as WikiMapia (www.wiki-
multiple impressions and perspectives. mapia.org). Secondary students can use a wiki to
Rhetoric and Composition course at Penn create hyperlinks from an existing poem to pages
State University uses a wiki for students to containing their own understandings or they can
even create their own collaborative poem.

E-Learning 2.0
Finally, it is important to show that E-Learn- 2.0, include customization, flexibility, and net-
ing 2.0 is a lot more than blogs, wikis, and so working.
forth. Teachers can create their own tools based Customization can be defined as the capac-
on constant communication and collaboration, so ity to adapt to the situation of each individual
that they can also contribute to define E-Learn- and not the other way around. Customization, in
ing 2.0. As an example, we have developed the terms of tools, methodologies, and contents, is an
QUEST system (Verdú, Regueras, Verdú, Pérez, important element. The learning experience must
& de Castro, 2006). This tool presents both indi- be personalized and tailored to the characteristics
vidual and collaborative environments in which of each scenario.
intellectual challenges are proposed by teachers There is also a need for an increased focus on
or students, as in a competition, which must be the fulfilment of customer needs. It makes sense
solved in a time-constrained way. The scores to take personality differences into account when
obtained make up a league-like classification designing an education system (Schank, 2002). In
from which the final grade is taken. this sense, Freund and Piotrowski (2003) states that
the use of e-learning and the mass customization
concept will help to make a given person fit for a
recommendAtIons And Future given job or a given job fit for a given person and
trends overcome the efficiency paradox in developing
and delivering education.
In our opinion, e-learning is constantly evolving, An interesting idea is the appearance of ser-
and, currently, E-Learning 2.0 should be regarded vices similar to “eBay” but in the context of online
as the target scenario. However, it is difficult to learning where users can have access exactly to
identify the future trends in E-Learning 2.0, as E- what they need.
Learning 2.0 itself must be considered a trend. Do not make the mistake of thinking that by
As O’Hear (2005) explains, the early promise combining different online pages from different
of e-learning—that of empowerment—has not courses you can produce a “mash-up” course.
been fully realized. For many people the expe- More likely, you will only produce a “ransom
rience of e-learning has been no more than a note” course. A good trainer will look at the
hand-out published online, coupled with a simple different technologies available and mash up a
multiple-choice quiz. However by using Web 2.0 solution that will be effective for their learner
tools, e-learning has the potential to become more base (Rosen, 2006).
personal, social and flexible. Traditional e-learn- Flexibility closely linked to the prior trend
ing approaches have been based on cumbersome refers to the possibility of offering multiple e-
(and often expensive) virtual learning environ- learning experiences (methodologies, workloads,
ments, which tend to be structured around courses, objectives, and so forth) in order to guarantee the
timetables, and testing. In contrast, E-Learning personalization or customization for each learner
2.0 (as coined by Stephen Downes) takes a “small leading to the universalizing of e-learning.
pieces, loosely joined” approach that combines E-Learning 2.0 must enable learning in every
the use of discrete but complementary tools and scenario to allow the integration of the educational
Web services—such as blogs, wikis, and other life of students.
social software—to support the creation of ad-hoc In the new paradigm, e-learning should be
learning communities (O’Hear, 2006). individualized, localized, and globalized with
According to these ideas, the future trends aims to create unlimited opportunities for life
and targets of e-learning, that is, E-Learning long learning. Students are the centre of educa-

E-Learning 2.0
tion. Learning should be facilitated to meet their concLusIon

needs and personal characteristics, and develop
their potentials. Students can be self-motivated After some hard beginnings, e-learning seems
and learn by themselves with appropriate guidance to be definitively catching a massive attention
and facilitation, and the learning process becomes as the application of ICT is finally regarded as
self-actualizing, discovering, experiencing, and positive by the learning community. We could go
reflecting (Cheng, 2002). even further and say that the institutions or teach-
Networking is linked to the idea of socializa- ers that do not count with this new element are
tion and the possibility of getting advantage of doomed to become obsolete. On the other hand,
any synergy that may arise in the net, including the pressure of the learner demand for learning
reutilization of materials, ideas, strategies, meth- experiences that make use of the ICT in classroom
odologies, and so forth, that, this way, are easily activities, blended learning, or distance learning
made available to everyone. Participants establish is each time stronger and stronger.
virtual identities and do networking with other In our opinion, one cycle is completed and
participants. we are now starting a second cycle that deals
While still in early stages of development, not with e-learning but with E-Learning 2.0. At
technology is permitting new ways of accessing the moment, there exists a lukewarm welcome to
information and interacting. Rapid knowledge E-Learning 2.0, but, as time goes by, we predict
growth requires off-loading the internal act of that the learner demand will grow as quick as
cognition, sense, and meaning making, and the interest among institutions and teachers, who
filtering to a network consisting of human and should be constantly evolving.
technology nodes. Thus, E-Learning 2.0 can be With such a prediction, our recommendation
regarded as a distributed process within a network is to anticipate to needs and trends, be a pioneer
recognizing and interpreting patterns, according now that you are still on time, and contribute to
to the connectivism theory. decide which must be the next step in e-learn-
As Downes (2005a) explained in his presen- ing. Do not surrender to the trends if you do not
tation at the Transitions in Advanced Learning think that they are the best possible practices, but
Conference about “What E-Learning 2.0 Means dictate them instead.
To You,” the learning is a network phenomenon E-Learning 2.0 is about personalizing the e-
where a Web of user-generated content is an learning environment to be more disciplined and
important pillar. The network is open, diverse pedagogically specific to the educational activity
(multiple views, technologies, etc.) and connected at hand (Blackboard, 2006). This more tailored
and interactive (not integrated), made of small platform experience must be achieved through
pieces, loosely joined. specialized software extensions developed by
E-Learning 2.0 is about enabling a social and for educators, as well as with rich, interactive
experience that recognizes the course is but one digital resources.
social-organizational group in a broader education One important conclusion is that the emphasis
environment. E-Learning 2.0 is about moving is and should be more focused on methodologies
beyond the course towards a more holistic concep- than on technologies, tools, platforms, or applica-
tion of a networked learning environment. One tions. Do not make the mistake of thinking that
consequence of this shift is a hunger by educators wikis, blogs, or forums are the E-Learning 2.0.
to conduct research and benchmark various e- They are resources. E-Learning 2.0 needs struc-
learning strategies and programs using data from ture and instructional design to be effective and
peer institutions (Blackboard, 2006). provide return on investment. Courses 2.0 should

E-Learning 2.0
never be a hotchpotch assembly of old method- We consider that people, along the different
ologies delivered through new technologies. A stages of life (childhood, adolescence, youth, and
new tool is only effective if it provides you with so forth), learn in what we could consider their
a better service (Rosen, 2006). global or macro-environment, made up by dif-
The evolution of the Web does not have to do ferent micro-environments: home, family, school,
so much with the technology as one might think. university, recreational activities, centres, and
As Downes (2005b) says, “The emergence of the so forth. Instead of living isolated experiences,
Web 2.0 is not a technological revolution, it is a people should live a global experience. Hence, it is
social revolution.” necessary that the different micro-environments
Finally, in accordance to Anderson (2006), are interconnected from a technological point of
“stop thinking about e-learning 2.0 as a new view through synchronous or asynchronous tele-
toolset… It is so not about that! The E-Learning matic tools, and which is more important, that a
2.0 is about people. The tools simply allow us global educational strategy is established.
to do what we do best… and that is connecting However, the reality is that it is very difficult,
with other people to support, share and learn if not impossible, to find global learning strategies
with each other.” that can be successfully applied to such different
contexts. Moreover, this global learning experi-
ence requires an instant multiterminal access,
Future reseArch dIrectIons as different environments are characterized by
different terminals. Otherwise, the learning
Having a look around schools and universities process could be slowed down if the occasion to
all over the world, we can observe that, although live certain micro-learning-experiences is lost due
there are some pilot experiences, E-Learning 2.0 to the lack of instant access through the terminal
is not a widely extended reality. In this sense, the available at each time and place.
design and development of educational strategies Other element to be further studied and
that include the use of E-Learning 2.0 tools, such promoted is the collaborative generation of con-
as wikis, blogs, or quests, should be considered a tents. Sites like wikipedia, helpfulvideo.com,
future research direction. However, in this closing and so forth, allow the collaborative publishing
section, we would like to go further; so, beyond of contents in different formats. However, some
what we consider future fields of research, we problems appear: how to supervise the quality of
would suggest working on the following ideas. this generated by-everyone content without mak-
Masie (2006) defined himself as a nanolearner. ing authors to feel controlled or restricted. In this
What does that mean? As he explained himself, sense, future research should focus on developing
each day, he learned several things in small adequate educational strategies to solve this.
chunks, really small chunks. A 90-second conver- To sum up, in our opinion, future research
sation with an expert could trigger a huge “a-ha.” should focus on developing and validating ef-
A few moments concentrating on learning how fective learning strategies in a context where it
something works could lead to a new micro-skill. is considered that learners live a macrolearning-
Moreover, his opinion is that most people acquire experience by adding microlearning-experiences
most of their knowledge in smaller pieces. that take place in different micro-environments,
Masie pointed out this idea focusing on con- where instant access from a wide range of ter-
tinuous training, but, in our opinion, it could be minals is a requirement and multiformat contents
transferred to formal education. are also generated by learners.

E-Learning 2.0
reFerences http://elearnmag.org/subpage.cfm?section=artic
les&article=29-1
Alexander, B. (2006). Web 2.0: A new wave of
Evans, M. (2006). Goodbye Web 2.0, long
innovation for teaching and learning? EDUCAUSE
live Web 3.0. Retrieved October 25, 2007,
Review, 41(2), 32-44.
from http://evans.blogware.com/blog/_ar-
Anderson, G. (2006). E-learning 2.0 is about chives/2006/11/12/2493546.html
people. Konferenz Professionelles Wissensman-
FitzGerald, S. (2006, June 8). Social network-
agement - Erfahrungen und Visionen Live von
ing: Philosophy and pedagogy. 2006 Networks
der ICL 2006». Retrieved October 25, 2007, from
Community Forum. Edna, Australia. Retrieved
http://elearningblog.tugraz.at/archives/130
October 25, 2007, from http://www.groups.edna.
Augar, N., Raitman, R., & Zhou, W. (2004). edu.au/mod/forum/discuss.php?d=6615
Teaching and learning online with wikis. Paper
Freund, R. J., & Piotrowski, M. (2003). Mass
presented at the ASCILITE Australasian Society
customization and personalization in adult
for Computers in Learning in Tertiary Education
education and training. Paper presented at the
2004 Conference. Perth, WA.
2nd Interdisciplinary World Congress on Mass
Blackboard. (2006). Blackboard unveils black- Customization and Personalization MCPC2003,
board beyond initiative. Four bold inaugural proj- Munich, Germany.
ects will advance e-learning 2.0 vision. Retrieved
Grant, L. (2006). Using wikis in schools: A
October 25, 2007, from http://www.blackboard.
case study. Retrieved October 25, 2007, from
com/company/press/release.aspx?id=823603
http://www.futurelab.org.uk/research/discuss/
Blamire, R. (2006). Insight blog. The online diary 05discuss01.htm
of European schoolnet’s insight team. Retrieved
Jennings, D. (2005). E-learning 2.0, whatever
October 25, 2007, from http://blog.eun.org/in-
that is. DJ Alchemi Web, an individual brew
sightblog/2006/06/elearning_20.html
of learning, culture and technology. Retrieved
Brian, L. (2004) Taking a walk on the wiki side. October 25, 2007, from http://alchemi.co.uk/ar-
Campus Technology. Retrieved October 25, 2007, chives/ele/elearning_20_wh.html
from http://www.campustechnology.com/article.
Johnson, D., & Johnson, R. (1999). Learning
asp?id=9200
together and alone: Cooperative, competitive,
Cheng, Y. C. (2002). Linkage between innovative and individualistic learning. Boston: Allyn and
management and student-centred approach: Plat- Bacon.
form theory for effective learning. Paper presented
Johnson, D., Johnson, R., & Holubec, E. (1998).
at the Second International Forum on Education
Cooperation in the classroom. Boston: Allyn
Reform: Key Factors in Effective Implementation,
and Bacon.
Bangkok, Thailand.
Masie, E. (2006). Nano-learning: Miniaturiza-
Downes, S. (2005a). What e-learning 2.0 means
tion of design. Chief Learning Officer (CLO)
to you. Paper presented at the meeting of the
Transitions in Advanced Learning Conference,
http://www.clomedia.com/content/templates/
Ottawa.
clo_article.asp?articleid=1221&zoneid=173
Downes, S. (2005b). E-learning 2.0. eLearn

E-Learning 2.0
McPherson, K. (2006). Wikis and student writ- Schacht, P. (2006). The collaborative writing
ing. ProQuest Information and Learning. Teacher project. Retrieved October 25, 2007, from http://
Librarian. Retrieved October 25, 2007, from node51.cit.geneseo.edu/WIKKI_TEST/medi-
http://redorbit.com/news/education/761377/wi- awiki/index.php/Main_Page
kis_and_student_writing/index.html
Shank, R. C. (2002). Designing world-class e-
O’Hear, S. (2005). Seconds out, round two. The learning. McGraw Hill.
Guardian. Retrieved October 25, 2007, from
Siemens, G. (2005). Connectivism: A learning
http://education.guardian.co.uk/elearning/sto-
theory for the digital age. Retrieved October 25,
ry/0,10577,1642281,00.html
2007, from http://www.elearnspace.org/Articles/
O’Hear, S. (2006). E-learning 2.0 - how Web connectivism.htm
technologies are shaping education. In R. Mac-
Thorson, D. (2006). Marketing 2.0: The constel-
Manus (Ed.). Retrieved October 25, 2007, from
lation. Retrieved October 25, 2007, from http://
http://www.readwriteweb.com/archives/e-learn-
donthorson.typepad.com/don_thorson/2006/04/
ing_20.php
the_constellati.html
O’Reilly, T. (2005). What Is Web 2.0. design
Verdú, E., Regueras, L. M., Verdú, M. J., Pérez,
patterns and business models for the next gen-
M. A., & de Castro, J. P. (2006). Improving the
eration of software. O’Reilly Web. Retrieved
higher education through technology-based active
October 25, 2007, from http://www.oreillynet.
methodologies: A case study. WSEAS Transac-
com/pub/a/oreilly/tim/news/2005/09/30/what-
tions on Advances in Engineering Education,
is-web-20.html
3(7), 649-656.
Pearce, J. (2006) Using wiki in education. The
Woolfolk, A. (2001). Educational psychology.
Science of Spectroscopy. Retrieved October 25,
Boston: Allyn and Bacon.
2007, from http://www.scienceofspectroscopy.
info/edit/index.php?title=Using_wiki_in_edu-
cation
AddItIonAL reAdIngs
Polo, F. (2006). Marketing 2.0 New way to old
things. Jornada Internet de Nueva Generación:
Alexander, B. (2006). Web 2.0: A new wave of
Web 2.0, Internet 2.0. Spain. Retrieved October
innovation for teaching and learning? EDUCAUSE
25, 2007, from http://internetng.dit.upm.es/po-
Review, 41(2), 32-44.
nencias-jing/2006/polo.pdf
Augar, N., Raitman, R., & Zhou, W. (2004).
Prinz, W. (2006). Social Web applications. Jor-
Teaching and learning online with wikis. Paper
nada Internet de Nueva Generación: Web 2.0,
presented at the ASCILITE Australasian Society
Internet 2.0. Spain. Retrieved October 25, 2007,
for Computers in Learning in Tertiary Education
from http://internetng.dit.upm.es/ponencias-
2004 Conference. Perth, WA.
jing/2006/prinz.pdf
Cheng, Y. C. (2002). Linkage between innovative
Rosen, A. (2006). Technology trends: E-learning
management and student-centred approach: Plat-
2.0. Learning Solutions E-Magazine. Retrieved
form theory for effective learning. Paper presented
October 25, 2007, from http://www.readygo.
at the Second International Forum on Education
com/e-learning-2.0.pdf
Reform: Key Factors in Effective Implementation,
Bangkok, Thailand.

E-Learning 2.0
Downes, S. (2005a). What e-learning 2.0 means O’Hear, S. (2006). E-learning 2.0 - how Web
to you. Paper presented at the meeting of the technologies are shaping education. In R. Mac-
Transitions in Advanced Learning Conference, Manus (Ed.). Retrieved October 25, 2007, from
Ottawa. http://www.readwriteweb.com/archives/e-learn-
ing_20.php
Downes, S. (2005b). E-learning 2.0. eLearn
Magazine. Retrieved October 25, 2007, from O’Reilly, T. (2005). What Is Web 2.0. Design
http://elearnmag.org/subpage.cfm?section=artic patterns and business models for the next gen-
les&article=29-1 eration of software. O’Reilly Web. Retrieved
October 25, 2007, from http://www.oreillynet.
Fraser, J. (2005, April 21). It’s a whole new Internet.
com/pub/a/oreilly/tim/news/2005/09/30/what-
Adaptive Path. Retrieved October 25, 2007, from
is-web-20.html
http://www.adaptivepath.com/publications/es-
says/archives/000430.php Masie, E. (2006). Nano-learning: Miniaturiza-
tion of design. Chief Learning Officer (CLO)
Freund, R. J., & Piotrowski, M. (2003). Mass
customization and personalization in adult
http://www.clomedia.com/content/templates/
education and training. Paper presented at the
2nd Interdisciplinary World Congress on Mass
Customization and Personalization MCPC2003, McPherson, K. (2006) Wikis and student writing.
Munich, Germany. ProQuest Information and Learning. Teacher
Librarian. Retrieved October 25, 2007, from
Grant, L. (2006). Using wikis in schools: A
http://redorbit.com/news/education/761377/wi-
case study. Retrieved October 25, 2007, from
kis_and_student_writing/index.html
http://www.futurelab.org.uk/research/discuss/
05discuss01.htm Pearce, J. (2006) Using wiki in education. The
Science of Spectroscopy. Retrieved October 25,
Jennings, D. (2005). E-learning 2.0, whatever
2007, from http://www.scienceofspectroscopy.
that is. DJ Alchemi Web, an individual brew
info/edit/index.php?title=Using_wiki_in_edu-
of learning, culture and technology. Retrieved
cation
October 25, 2007, from http://alchemi.co.uk/ar-
chives/ele/elearning_20_wh.html Polo, F. (2006). Marketing 2.0 new way to old
things. Jornada Internet de Nueva Generación:
Johnson, D., Johnson, R., & Holubec, E. (1998).
Web 2.0, Internet 2.0. Spain. Retrieved October
Cooperation in the classroom. Boston: Allyn
25, 2007, from http://internetng.dit.upm.es/po-
and Bacon.
nencias-jing/2006/polo.pdf
Johnson, D., & Johnson, R. (1999). Learning
Prinz, W. (2006). Social Web applications. Jor-
together and alone: Cooperative, competitive,
nada Internet de Nueva Generación: Web 2.0,
and individualistic learning. Boston: Allyn and
Internet 2.0. Spain. Retrieved October 25, 2007,
Bacon.
from http://internetng.dit.upm.es/ponencias-
O’Hear, S. (2005). Seconds out, round two. The jing/2006/prinz.pdf
Guardian. Retrieved October 25, 2007, from
Richardson, W. (2006). Blogs, wikis, podcasts,
http://education.guardian.co.uk/elearning/sto-
and other powerful Web tools for classrooms.
ry/0,10577,1642281,00.html
Thousand Oaks, CA: Corwin Press.
0
E-Learning 2.0
Rosen, A. (2006). Technology trends: E-Larning Verdú, E., Regueras, L. M., Verdú, M. J., Pérez,
2.0. Learning Solutions e-Magazine. Retrieved M. A., & de Castro, J. P. (2006). Improving the
October 25, 2007, from http://www.readygo. higher education through technology-based active
com/e-learning-2.0.pdf methodologies: A case study. WSEAS Transac-
tions on Advances in Engineering Education,
Shank, R. C. (2002). Designing world-class e-
3(7), 649-656.
learning. New York: McGraw Hill.
Woolfolk, A. (2001). Educational psychology.
Siemens, G. (2005). Connectivism: A learning
Boston: Allyn and Bacon.
theory for the digital age. Retrieved October 25,
2007, from http://www.elearnspace.org/Articles/
connectivism.htm

Chapter XIII
Telematic Environments and
Competition-Based
Methodologies:
An Approach to Active Learning
Elena Verdú
Luisa M. Regueras
María J. Verdú
Juan Pablo de Castro

María A. Pérez
ABstrAct
This chapter provides an overview of technology-based competitive active learning. It discusses com-
petitive and collaborative learning and analyzes how adequate the different strategies are for different
individual learning styles. First of all, some classifications of learning styles are reviewed. Then, the
chapter discusses competitive and collaborative strategies as active learning methodologies and ana-
lyzes their effects on students’ outcomes and feelings, according to their learning styles. Next, it shows
how networking technology can mitigate the possible negative aspects. All the discussion is supported
by significant study cases from the literature. Finally, an innovative system for active competitive and
collaborative learning is presented as an example of a telematic versatile learning system.
Telematic Environments and Competition-Based Methodologies
IntroductIon It also has to be taken into account that the

implementation of an education model based on
New active learning methodologies are arising active methodologies is not an easy task, as a
as more effective learning methods than the number of difficulties usually arise. Some of these
traditional ones. The effectiveness of a learning difficulties are: the rejection of new methods by
process is achieved when its results are lasting both students and teachers; the number of students
and transferable to other situations. Various in each class; and even current classroom set-up,
studies have shown an important improvement which appears more suited to taking notes rather
in long term retention of what is learnt when ac- than doing group work (Verdú, Regueras, Verdú,
tive learning techniques are introduced into the Pérez, & de Castro, 2006a).
learning process (Canós & Mauri, 2005; Tim- On the other hand, the results obtained when
merman & Lingard, 2003). For example, Hyland applying active learning techniques have not
(2002) shows that people are able to remember always been positive. These techniques are more
about 20% of what they listen to (passive), 70% effective in order to achieve some objectives,
of what they say (active), and 90% of what they whereas traditional classes are better for some
say and do (active). others (McCarthy & Anderson, 2000). Besides,
The use of active methodologies contributes students have different learning styles and, conse-
to develop in students the capacity to actively quently, not every student benefits the same from
research and undertake prominent roles and active learning. For example, Zywno and Waalen
leadership in their own learning process, fac- (2002) compare the effect of individual learning
ing the resolution of problems with their own styles on student outcomes in conventional envi-
resources. ronments with the same in hypermedia assisted
Active learning is not a new concept, although environments. When applying hypermedia sys-
the interest in applying it has increased a lot over tems, they find better results with students who
the last few years. This increase has become more prefer active, sensing, and global learning. In
noticeable with the spread of information and any case, they state that, due to the multimodal
communication technology (ICT). attributes involved, hypermedia is more effective
ICT opens up numerous possibilities with in reaching all types of students and reducing
regard to the implementation of active method- differences in the academic performance among
ologies. It permits the development of remote several learning styles.
cooperative activities, where the teacher’s role Therefore, we can assert that the selection of
can be easily adapted to the model in which stu- a learning-teaching strategy entails previously
dents must actively lead their learning process not only determining the cognitive activity of
(Bryndum & Montes, 2005). the learning, that is, the type of skills, competen-
When applying ICT-based active learning cies, and techniques to be developed (Fandos &
methods, the teacher should understand the González, 2005), but also takes into account the
educative value of the telematic tools and know individual learning styles of students. The strategy
how to use the technology in order to improve the implemented in a certain learning process may
learning process. In this sense, it is commonly not be the right one for every student.
believed that technology is effective when it comes One of the key aspects for the success of a
accompanied by a constructivist pedagogy, which learning methodology is motivation. The recom-
supports the learning based on investigation. mendations about learning strategies normally
Wirsig (2002) agrees about it and, moreover, she include techniques to encourage students. Nowa-
states that technology can add a considerable days we have more resources than ever to design
cognitive value to the learning process.

strategies with high possibilities with regard to • To present, as an example of a telematic

motivation. Technology-based materials, such as versatile learning system, Quest Environ-
hypermedia and interactive contents, promote a ment for Self-managed Training (QUEST),
divergent thought, stimulate the research instead an innovative tool for active, competitive,
of imposing certain thoughts, and contribute to an and collaborative learning (Verdú, Regueras,
autonomous learning. Students are encouraged to Verdú, Pérez, & de Castro, 2006b; Verdú, de
create problems and select sources and analyze Castro, Pérez, Verdú, & Regueras, 2006).
them with a critical view (Bryndum & Montes,
2005). Moreover, the multiple possibilities of ICT
allow us to design strategies adaptable to a wider LeArnIng styLes:
range of learning styles. cLAssIFIcAtIons
Technology-enhanced learning has generated
a lot of experiences on computer supported col- Numerous categorizations of learning styles are to
laborative learning (CSCL). Motivation is one of be found in the literature. One of the most frequent
the most positive aspects of collaborative work. classifications (Burd & Buchaman, 2004; Canós
However, some students feel more motivated & Mauri, 2005; Marqués, 2001) is established
through competition. Team competition has a dual by Kolb, who identifies four learning styles ac-
nature: it is both competitive and collaborative cording to the way information is received (from
and, therefore, offers a lot of possibilities when concrete experiences to abstract concepts) and
facing a heterogeneous group of students. how it is processed (from active experimentation
Although some authors do not recommend to reflective observation): diverger, converger,
competitive learning (Brightman, 2006; Johnson assimilator, and accommodator. Divergers rely
& Johnson, 1988), we can also find studies in on a combination of concrete experience and re-
which good results are obtained when applying flective observation. They like working in group
this type of learning with the support of ICT in order to compile information and often prefer
(Chang, Yang, Yu, & Chan, 2003; Philpot, Hall, to observe rather than to participate. Convergers
Hubing, & Flori, 2005; Titcomb, Foote, & Car- prefer high levels of abstract conceptualization
penter, 1994). Some of these case studies will be together with active experimentation. They
reviewed along this chapter. seek to learn via the direct application of ideas,
In summary, the objectives of this chapter problem solving or deductive decision-making,
are: without taking into account social and personal
relationships. Assimilators, as the name suggests,
• To review different classifications of learning prefer to combine abstract conceptualization with
styles that can be found in the literature. reflective observation and are good at assimilating
• To study how ICT can promote active learn- a lot of information and arranging it in a more
ing and the improvement of the learning logical form. Their greatest strength lies in their
process, through the review of some study ability to develop theoretical models. Finally, ac-
cases. commodators combine concrete experience with
• To discuss about competitive and collabora- active experimentation and like working in group.
tive strategies as active learning methodolo- Their greatest strength lies in doing things, car-
gies and analyze their effects on students’ rying out plans and experiments, and involving
outcomes and feelings, according to their themselves in new experiences.
learning styles, with the support of some Other categorizations take into account the
examples. channels through which information arrives. For

example, Mehlenbacher, Miller, Covington, and examines a learner’s intellectual approach to as-
Larsen (2000) classify students according to the similation of new information. The layer beyond
categorization of learning styles developed by examines social interaction and how students
Felder. They distinguish active from reflective prefer to interact with each other. The outermost
students, visual from verbal students, sensing layer, of instructional preference, tends to relate
from intuitive students, and sequential from global to external factors such as physiological and
students. Active students prefer to process infor- environmental stimuli associated with learning
mation through engagement in physical activity, activities. The outermost layers are more influ-
discussion, and in groups, whereas reflective stu- enced by external factors (and more observable)
dents tend to work alone. Visual students base their whereas the innermost layers are considered to
work on pictures and graphics, whereas verbal be more stable psychological constructs and less
students prefer written and spoken words. Sens- susceptible to change; however these are much
ing students tend to work with visual and sound less easily measured.
aids, whereas intuitive students prefer memories Traditional learning methodologies have been
and ideas. Finally, sequential students like doing oriented for reflective students and individualized
logical incremental steps, whereas global students learning. Nowadays, there is a greater interest in
prefer total picture reasoning. applying learning methods suitable for different
In another line, Kim and Sonnenwald (2002) learning styles. In this sense, the multiple pos-
use the scale of learning preferences of Owens sibilities that ICT offers allow us to adapt the
and Barnes to identify three learning styles: co- used methodologies to a bigger range of learning
operative, competitive, and individualized. The styles.
cooperative learning style indicates a preference
for achieving individual goals while working in
group. The competitive learning style indicates Ict-BAsed ActIve LeArnIng
a preference for learning in competition with
others, often achieving individual goals. Lastly, There are different types of telematic tools and
the individualized learning style indicates a pref- educational material. Most of them are currently
erence for achieving individual goals having no based on Web and data base technologies:
involvement with other students.
Finally, some authors briefly modify traditional • Tools for the management of courses and
learning styles models in order to adapt them to students: enrolment, student’s data record,
new technology based systems. For example, student’s achievement record, and so
Brown, Cristea, Stewart, and Brailsford (2005) forth.
extend the Curry’s “onion” model for adaptive • Tools for online lectures: slides, videos,
hypermedia systems. They integrate prior knowl- videoconferences, electronic blackboards,
edge layer as an additional layer to those included and so forth.
in the “onion” model: instructional preference, • Material for the support to the classes: exer-
social interaction, information processing style, cises, self-assessments, interactive tutorials,
prior knowledge, and personality style. The in- virtual encyclopaedias, multimedia books,
nermost layer, cognitive processing style, seeks hypertext references, and so forth.
to measure an individual’s personality, specifi- • Virtual laboratories: animations, simula-
cally related to how they prefer to acquire and tions, study cases, and so forth.
integrate information. Moving outwards, the next • Communication tools: electronic mail, chat,
layer measures information processing style and discussion forums, instant messages ser-

vice, queries boxes, distribution lists, news activities. This fact has generated a lot of experi-
boards, news groups, multiconferences, and ences in what has been called computer supported
so forth. collaborative learning (CSCL), many times as
• Tools for collaborative learning: coordina- the opposite of traditional competitive learning.
tion of group work, virtual spaces for sharing However, in our opinion, there is not opposition
information and resources, management of between collaboration and competition since
document versions, and so forth. both techniques can be used in a complementary
way. Team competition could be a good example
Nowadays, these tools are not used alone but of it.
integrated into learning management systems
(LMS), which permits the scheduling, implemen-
tation, and management of the whole learning coLLABorAtIve And
process. comPetItIve LeArnIng
Most of the current available LMS platforms
(WebCT, Blackboard, Angel, Centra, Moodle, Collaborative work or group work is the work
Claroline, and so forth) include many of those that a group of people do with the aim of getting
common tools. They are used in different active a common objective. Researches in group tech-
learning contexts, since they facilitate the interac- niques suggest that group work improves the way
tions between the teacher and the student, among of perceiving obstacles and determines the group
students and between the student and the course as an element of support and motivation to face
material. These interactions are very important up to learning (Fandos & González, 2005).
components of active learning (Mehlenbacher Although sometimes it is used as synonym of
et al., 2000). collaborative learning, cooperative learning puts
As well as management, communication, and the emphasis more on the product that is obtained
interaction tools, interactive contents, such as during the group learning process. Besides, the
online interactive exercises, can be very efficient scheduling and guidance of the teacher has a more
when used as instruments for active learning. In important role. However, in spite of that slight
fact, whereas most of the available e-learning nuance of meaning, both types of learning are
material consists of static hypertext pages, at different to the traditional one in the same things
best with Flash animations, the current learning (Marqués, 2001):
theory suggests that the student’s achievement
improves more when the educational resources • They centre on the student.
are more interactive and multimedia (Morozov, • There is an intrinsic motivation.
Tanakov, Gerasimov, Bystrov, & Cvirco, 2004). • They are focused on knowledge-building.
However, it has to be taken into account that • The responsibility of the learning falls es-
interactivity is a critical design objective of the pecially on the student.
educational Web sites, since it requires a hard- • There is a greater motivation.
working process. Whereas to develop static or • The development of higher-order reasoning
quasi-static hypertext pages is cheap and easy, is promoted.
the design and implementation of interactive • More abilities are developed, research, group
material takes a lot of time and is a complex and work, problem solving, public presentations,
expensive task. social abilities, prevention and mediation in
Active methodologies based on the use of ICT conflicts, and so forth.
promote active learning and permit cooperative

In spite of these positive aspects, student satis- strong contact culture within students as reasons
faction in courses requiring a collaborative effort why online debates were not used for interactive
among peers is heavily influenced by the project communication among collaborative groups, since
team experience (Reichlmayr, 2005). they preferred to debate, if possible, in a face-to-
The development of ICT has enabled the face manner. However, only 11% of the class felt
creation of tools which make collaborative work that the BSCW platform did not help them in their
easier (Fandos & González, 2005): learning process whereas only 39% perceived that
the use of the platform created cohesion among
• Communication tools: The electronic mail the members of the class, owing to the reasons
for the exchange of information among the described above. These results make sense in a
members of a group, the discussion space hybrid learning context like the one described.
or forum for sharing ideas, and the chat or Although the BSCW system was not a key element
the videoconference. in group discussions, which were mainly achieved
• Organizational tools: Schedule, notice face-to-face, other of its characteristics certainly
board, and so forth. facilitated the learning process.
• Tools for the presentation of ideas: The The Intelligent and Cooperative Systems
electronic blackboard and the remote desk- Research Group of the University of Valladolid
top applications are good examples of this uses also BSCW in experiences of project-based
type of resources. learning. In some of their studies they state that
• Tools for sharing and managing docu- the computational support to share documents
ments: These tools allow the members of through BSCW has been revealed as important,
the workgroup to access documents located even in a face-to-face classroom setting (Mar-
in shared remote work spaces, while the tínez, Gómez, Dimitriadis, Jorrín, Rubia, &
system automatically manages the different Vega, 2005).
versions of the documents. Frees and Kessler (2004) use a group of tools
for collaborative learning called Cimel (Cimel Col-
Some examples of use of these tools in CSCL laborative Tools). The system consists of a contact
are described below. list, a chat client, a searchable database, and a set
Van der Linde (2005) uses the collaborative of screen sharing applications collectively named
work platform basic support for cooperative work ShowMe. ShowMe applications promote a “learn
(BSCW) as a complement and support to the tra- by doing” approach that enables an instructor to
ditional class based sessions. The use of BSCW view the student’s desktop and use annotations and
was intended to increase the level of discussion text messages in order to guide students in their
and knowledge by sharing information outside learning process. Students have complete control
the classroom. Van der Linde (2005) observed over which parts of the screen they want to share
that, although students uploaded online contents with the teacher and the system captures the local
and participated in debates created by the teacher, user’s mouse cursor position and stream it to the
there was only limited interaction within and be- peer, so the teacher can see where the student is
tween groups. The author thinks that this limited pointing. Thus, the authors try to promote active
use of the BSCW system may be explained by learning, forcing students to perform the appropri-
the lack of experience in the use of virtual tools. ate actions in order to solve the problem, while
Students cited time pressures as the reasons for allowing the instructor to oversee what is going
their low participating rates in online debates. on, make suggestions and guide students to the
They also mentioned traditional learning and the solution. The creators of this tool observed that

student feedback was positive, with most students computing challenges and allows participation
indicating that it would be very useful to them if in online contests. Since its launch in 1997, it has
made available in all their classes. received over 5 million submissions. This fact
Mickle, Shuman, and Spring (2004) present reflects the success of this type of competitive
other system, used for active and collaborative learning projects.
learning, called computer augmented support The idea of competition is usually linked with
for collaborative authoring and document editing gaming, because of the motivational nature of both
(CASCADE). This system permits to insert com- methods. There are some systems that implement
ments, diagrams and graphical representations games for learning, which is an effective method
into shared documents and handles automati- in order to increase motivation, fun, and learning
cally when and who made the comment. In that (Philpot et al., 2005).
way, the CASCADE system makes it possible Yu, Chang, Liu, and Chan (2002) present a
for the teacher to review the comments made by study conducted to examine students’ preferences
students over multiple big documents in a matter towards different kinds of competition using the
of seconds, via automatically provided dynamic online system JOYCE. The Joyce system is a
hypertext representations. competitive board game that allows students to
Although it is easier to find successful experi- compete against each other or against a simple
ences in CSCL or pure collaborative learning, in computer simulated agent. Players have to respond
part because of the fact that they are more numer- multiple-choice questions correctly in order to win
ous, in the next section, we briefly describe some the game (Chang et al., 2003). The authors justify
studies in which ICT-based competitive learning the fact of choosing “gaming” as the instructional
has been successfully applied. method because instructional games have been
Chu, Chang, and Hsia (2004) study the com- suggested as a powerful technique to capture
petitive behaviour of some students that compete and hold student interest. Besides, they include
with each other in designing a Web site. They find an element of competition to further promote the
that, during the competition, when two teams motivation. In their study, students were exposed
are level on points and have a chance to win the to three competition modes (Yu et al., 2002):
contest, both of them obtain higher scores than
in any other situation. On the other hand, when a • Anonymity mode: The identities of the
team distances from another one, the lower scoring participants are concealed.
team ends up giving up the struggle to win. • Face-to-face mode: Participants are sit-
Chang, Wang, Liang, Liu, and Chan (2004) ting next to each other during competition
describe, on the one hand, the use of a Web site (therefore, it is not anonymous).
to make online contests and, on the other hand, • Distance mode: The identities of the
the wireless system EduClick to make contests participants are revealed, but they are geo-
within the classroom or between different class- graphically distant from each other while
rooms located in the same or different schools. competing in the system.
In both cases, contests lie in answering questions
automatically processed by the systems. Their results show that students prefer the
Revilla, a lecturer in the department of Applied anonymous competition because it is more stimu-
Mathematics at the University of Valladolid, has lating and more likely to reduce the stress and
developed a successful project named “Online- other negative emotions, whereas the face-to-face
Judge” (http://acm.uva.es). The developed system competition is what they dislike most.
corrects the solutions to a number of online

As Yu et al. (2002) state, some previous re- cognitive activity, that is, the goals of learning,
search showed that competition has a negative the type of skills or abilities to be developed, and
effect on interpersonal relationship, emotional the desired level of cognition: from the recall
states, and group process. Those studies were of information to more abstract levels such as
conducted in face-to-face situations, in which synthesis and evaluation, following the Bloom’s
the identities of the participants cannot be hid- taxonomy of learning.
den without networking technological support. Once identified the goals and skills to be de-
However, the authors state that it remains to be veloped, it is necessary to develop strategies and
seen whether the negative effects of face-to-face activities to lead students to the desired level of
competition can be mitigated with the anonymity cognition. Moreover, when designing the global
inherent in network competition and synchronous strategy or concrete classroom activities, it should
e-learning environments. be taken into account, not only the desired cogni-
Chang et al. (2003) evaluate the effectiveness tive activity, but also the students’ motivation.
of the Joyce system and successfully observe that Therefore, instructors should try to adapt their
most students think that playing a game is a good learning strategies to the most common individual
way to learn, since it allows them to memorize learning styles of their students, and then choose
much knowledge. Besides, they state that the sys- the ICT tools that can better support the selected
tem increases motivation, as their studies revealed strategies.
that students want to read more articles and books
to find answers and to win the game. motivation
Philpot et al. (2005) develop some computer-
based interactive games for learning some specific In general, it can be asserted that motivation
engineering subjects. Those games make use of has a great influence on the learning process. It
repetition and carefully constructed levels of stands to reason that if students want to learn they
difficulty in order to help students to get a bet- will get more involved in the learning process,
ter performance in their learning. Students can getting, then, a better performance. There are
participate in games at their own pace but in a several factors that influence motivation, such
competitive way. As in the examples described as the connection of educational activities to the
above, students’ quantitative ratings and com- real world or the achievement of activities that
ments to these games were very positive. Even facilitate the constructive learning. The key is that
more, students who used games scored signifi- a well designed and executed learning strategy
cantly higher on quizzes than those who learned involves motivation; and this is the reason why
via traditional lectures. the work of the teacher during the educational
In all these examples, it has been shown the design is so important.
success of different learning strategies. But, how Motivation is even more critical in a distance
do we choose one of them? learning context, in which the teacher cannot
interact with students in a face-to-face way. In
these cases, interactivity can be used as an ele-
choosIng A LeArnIng ment of motivation in order to capture and hold
strAtegy the students’ interest. The teacher should provide
an interactive and dynamic environment to com-
As it has been said before, the choice of a learning- pensate for the physical distance. Even more, if it
teaching strategy implies several considerations. is well designed, results could be improved.
The first thing to take into account is the desired

Hislop (1999) presents some interesting data learning styles depending on factors such as age,
obtained when evaluating a learning experience personality, culture, and environment. Agreeing
developed completely online: 95% of students felt with this principle, we believe that not every
that they had better access to the instructor, and student must be treated in the same way, and that
43% felt that they actually communicated with the a set of activities that contribute to facilitate the
instructor more than they would in a traditional learning process of the different students must
class. These data confirm the hypothesis that be proposed by the team of teachers. The idea is
Web tools facilitate the interaction between the to respect diverse talents and ways of learning
teacher and the student. However, 51% of students (Chickering & Ehrmann, 1996): students need
missed face-to-face lectures, 40% felt that they opportunities to show their talents and learn in
had to work harder in the online course, and 15% ways that adapt better to their learning styles.
felt that the online class was more boring than a Therefore, teachers must apply different learn-
traditional class. These data reflect, on the one ing techniques: collaborative learning, practical
hand, the benefit of considering telematic systems sessions, self-assessments, and so forth, when
as additional resources to be used taking into ac- designing their classes.
count the students’ profile and the educational, However, we also believe that students must
social, and professional context. On the other hand, be adequately prepared to successfully undertake
they reflect the idea that online education does their professional careers. Then, teachers should
not work well for everyone, since the success also focus on the use of active strategies where coopera-
depends on the different learning styles. tive and competitive learning activities, and not
only individual learning activities, take place.
Learning styles As it has been explained before, although
motivation is one of the most positive aspects
The interactivity provided by the Web seems to of collaborative work, some students feel more
be positive, since it is a good motivation element. motivated through competition. Team competi-
However, as it has been mentioned along this chap- tion has a dual nature; it is both competitive and
ter, it must be always taken into account the dif- collaborative and, therefore, offers a lot of pos-
ferent learning styles. Results from Mehlenbacher, sibilities when facing a heterogeneous group of
Miller, Covington, & Larsen (2000) show that, in students. This could be taken into account when
a Web environment, reflective and global learners selecting and designing a learning strategy.
are performing better than active and sequential When designing a competitive learning
learners. They conclude that reflective learners, strategy, besides the selection of individual or
who prefer solitary, quiet problem-solving as op- team competition, other factors should be ana-
posed to group discussion of problems, may have lyzed, since the competitive methodology can
been more comfortable in the online courses. be anonymous or of known authorship, face-
This result surprised them somewhat, since they to-face or distance located, and so forth. At this
assumed that their “interactive” Web site would moment, the possibilities of networking should
favour active learners. However, attempting to be considered.
emulate the interactivity of a face-to-face class
on the Web has a high level of difficulty. use of Ict
It is also important to consider, as some authors
highlight (Burd & Buchanan, 2004), that, even ICT is a useful tool to support pedagogical
if individuals are usually strong in one learn- principles. Yet, for each learning strategy we
ing style, in general, they will exhibit multiple must ensure that the right technology is applied.
0
Pedagogical principles remain and, at each time New technologies can also contribute to bet-
and for each situation, we must analyse the exist- ter transfer high expectations to students. For
ing technology and choose the one appropriate example, students may hand in better essays
to support the learning strategy selected for the when they know that these essays are going to
existing scenario, as suggested by Chickering be available in a Web page.
and Ehrmann (1996). Last of all, the great variety of tools based on
First of all, communication technologies can new technologies facilitates a greater range of
facilitate the access to teachers by means of elec- learning methods. Technologies allow students to
tronic mail or videoconference systems. Besides, learn in the most appropriate and effective way,
teachers can provide students with documents and according to their learning styles. For example,
any interesting information of the course through there are tools for collaborative learning and there
a Web page, for example. are also tools for lonely, reflective, and self-as-
A clear advantage of electronic mail, forum or sessable learning.
instant message applications is that students can Summarizing, new technologies seem to be
communicate with their classmates on distance, efficient learning support tools. However, the need
making the cooperation among them easier. More- to familiarize with the new learning tools as well
over, collaborative work tools allow a group of as the possible technical problems, arise as threats
students to work on the same document, to share to the normal learning process. Student motivation
documents, and so forth. may decrease too much if the use of a new tool is
Besides, virtual laboratories, simulation complex or if the computer frequently fails.
software, interactive Web pages, and so forth, Although education technologists promote the
are efficient tools for active learning, as previ- use of interactive systems, we must be careful
ously said. when designing these systems. More reflective
Finally, other tools enable students to know students may not benefit from environments that
their progress in learning. For example, there are provide instantaneous feedback and response; they
software tools to monitor the knowledge develop- may prefer first reading and following links until
ment, interactive activities that provide automatic they develop a fuller representation of the entire
feedback, and monitoring tools that register the learning space, and then acting, inputting, and
activities done by students in databases. In the writing (Mehlenbacher et al., 2000).
same way, these tools can be used by teachers to The above-described telematic systems can
monitor student progress, so that they can take be used outside or inside the classroom, in an
the corrective actions accordingly. isolated way or together with traditional resources,
All these technologies also make possible time such as blackboards and books. Thus, against
saving in different ways. For example, students the learning carried out completely on distance
do not need to go to libraries if they have the through the Internet, an intermediate solution,
documentation available on the Web. Besides, named blended learning, arises. Blended learning
they allow a more efficient management of time as is more appropriate in certain contexts, which are
asynchronous communication tools free students mentioned below in this section. It aims to join
of the limited timetables of face-to-face classes, the best of face-to-face classroom with the best
being able to interact with their classmates and of online learning to promote active independent
teachers at any moment. Moreover, ICT allows learning and reduce class seat time (Reichlmayr,
distance education as neither time nor space is a 2005).
barrier to education (Verdú, Verdú, Regueras, & For example, Reichlmayr (2005) uses blended
de Castro, 2005). learning in order to better take advantage of the

class face time; students are expected to prepare anonymous or of known authorship. An example
for class by reading relevant sections from the of the success of this possibility is shown in the
textbook and online resources. Lectures and following section, where we present QUEST
in-class activities complete and consolidate this (Quest Environment for Self-managed Training),
learning process, rather than being the primary a telematic versatile learning system for active,
source of knowledge. Besides, a chat application competitive, and collaborative learning (Verdú
is used for the communication among students et al., 2006; Verdú et al., 2006b).
and with the teacher outside the classroom. In
this case, most students (72%) liked this mixed
“online-face to face” environment. Quest: A mIxed comPetItIve-
Sonya Symons and Doug Symons (2002) indi- coLLABorAtIve soLutIon
cate that technology helped them to incorporate
valuable aspects of education in more ways than QUEST has been developed in the context of the
it is usually possible in large classes. The tele- interdisciplinary research group Intersemiotics,
matic part of the learning process lays in weekly Translation, and New Technologies (ITNT). It is
discussion groups in order to discuss the week’s accessible from every computer with Internet ac-
material, Web-based assignments, and activities cess and, hence, can be used in the classroom, at
using existing Internet resources to encourage home, or in a cybercafé. This is possible because
interactive learning. QUEST has been implemented as a module that
Godoy (2005) uses a Web-based simulated can be integrated into the e-learning platform
environment as a support tool of the methodol- Moodle.
ogy “learning-by-doing.” At the beginning of As a learning tool, QUEST aids the introduc-
the simulation the learner is placed in a role tion of cooperative and competitive workshops
and learns about a problematic situation to be supported by telematics. The system pursues the
solved. Then, the student interacts with the case development of student inquiry, documentation,
obtaining more information about it through the and critical analysis skills, while raising the level
navigation. The options have to be chosen from of involvement and communication between stu-
a menu and the learner decides on the path to dents and teachers.
follow. Resources about previous cases and other The system presents both individual and
material of general relevance to the understand- group work environments in which a set of in-
ing of the case are given to the student. Besides, tellectual “challenges” that must be solved in a
the student can perform computer simulations time-constrained way are proposed to students by
and interact with other learners and with a tutor other students and/or by teachers and tutors. The
through a synchronous forum. Students can send answers to those challenges are open, with the
the solution to a problem as a free format response possibility of including mathematical equations
to a tutor in order to obtain feedback. They can and any type of attached files.
also choose the response from a limited set of The workshop mainly focuses on competitive-
possibilities and send it to the system, which will ness, collaboration, and social acknowledgment as
automatically respond with positive or negative motivation mechanisms and seeks to strengthen
considerations. these skills through the student’s academic work.
Finally, ICT and, more specifically, network- Hence, workshop sessions are presented as a
ing, provide the support of anonymity, which contest with the corresponding ranking based
allows teachers to design learning strategies on the scores obtained by the students with the
adapted to the above-mentioned different modes: answers submitted by them to the set of chal-
lenges proposed.

To enrich the learning process by means of decreases following an exponential or a

collaboration and involvement, the system allows linear pattern so that the student who is the
the students to submit challenges and to pre-as- first to answer is awarded the maximum
sess the corresponding answers, being rewarded score.
depending on the quality of the tasks done. Each
new challenge proposed by a student must be Some research suggest that it is very useful
validated by the tutor. for students to access to the essays done by other
From when the challenge is created until the students and, besides, they do better essays when
end of the process, the score of a challenge varies they know that their classmates will read their
(as shown in Figure 1): work (Hislop, 1999). Thus, when the time to an-
swer is over, the challenge is closed, and students
1. Stationary phase: During this phase, the can read all submissions from all participants
score remains as proposed by the teacher anonymously, which can help them to understand
during a period of time to allow students and reinforce concepts.
to understand and to take in the task. This The platform has many interesting features
period should be longer when the task is more that permit the implementation of a wide variety
complex and/or needs important previous of learning styles: individual, collaborative, and
documentation work. competitive learning, as well as a combination.
2. Inflationary phase: During this phase, the For example, teachers can propose challenges
score grows to adjust the reward to the dif- that must be solved individually, thus leading to
ficulty. It is assumed that a lack of correct individual and competitive learning. They can
answers means that the difficulty of the task also propose challenges that must be solved in
proposed is higher than the reward offered teams, leading to collaborative (within teams)
at a certain time. and competitive (between different teams) learn-
3. Deflationary phase: This phase starts ex- ing. They can also propose challenges that must
actly when a challenge is correctly answered be solved by teams or individually, eliminating
and, at this moment, the score starts decreas- the awarding of a score and the time-constraint
ing. During this phase the score continuously condition to solve the task, which leads to collab-
Figure 1. Variable scoring of a challenge during its life-cycle

orative or individual, but not competitive learning. innovative active learning strategies, has been
As explained before, teachers must analyse the described. The system is focused on active, com-
scenario in which the learning process will take petitive, and collaborative learning.
place and the profiles of participants. Taking those Many of the existing tools for competitive
elements as a base, they must adapt the available learning are specific, “tailor-made” for a concrete
technological tools to make the best use. subject, totally separated from other tools. This is
As it has been described above, students can the reason why one of the objectives established
access to their classmates’ answers anonymously, when the tool QUEST was designed was that it
that is, the system also permits the identity of should be valid for multiple disciplines and inte-
participants to be hidden, since we opted to grable into Moodle, a widely-used open-source
maintain the privacy of submitting authors. In e-learning platform. This has the advantage of
fact, in the light of the results of our experiences incorporating all the stages of the learning process
with QUEST (Verdú et al., 2006a), we subscribe (contents, evaluation, tutorials, and so forth) into
the idea introduced above by Yu et al. (2002): the the same platform, without having to access to
negative effects of face-to-face competition can be different environments for each stage.
mitigated with the anonymity inherent in ICT. Unlike in other tools, in QUEST students are
content generators, participating in the learning
process in an active way. They can propose chal-
concLusIon lenges and, moreover, they can assess the answers
sent by their classmates, thus being involved in
This chapter gives an overall view of the state of the assessment process. Besides, the answers of
the art of active learning techniques based on ICT, students are made public when challenges finish,
analysing some pedagogical aspects that worry but preserving the anonymity of the authors. This
us, such as the student motivation. An important fact, with no doubt, enriches the learning of stu-
conclusion we can draw from this study is that, dents, who can learn from the answers submitted
when designing and applying different learning by their classmates, and contributes to a better
strategies, we should take into account several quality of these answers.
important elements: the cognitive activity of learn- The variable scoring system, as well as the
ing, the individual learning styles of students, the multiple options provided when generating a
available resources, and the educational, social, contest (individual and group contests, different
or professional context. marking strategies, etc.), makes QUEST a highly
In general, the results obtained when applying flexible system.
active learning based on ICT are satisfactory. ICT The results obtained when using QUEST in
improves the learning process, facilitates motiva- different courses of the University of Valladolid
tion, and allows students to develop their inquiry, have confirmed that QUEST is a tool which pro-
creativity, and critical analysis skills. motes the students’ participation in class. Besides,
However, the starting phase of the introduc- in spite of the possible initial negative reactions to
tion of these methods is complicated, due to the new methods, QUEST has been widely welcomed
rejection of new methods by both students and by the students, who would like this tool to be
teachers, and the difficulties in the design of in- applied in other courses.
teractive contents, as well as in the definition of The focus of the strategies followed when us-
the learning strategy. ing QUEST brings students and teachers closer
In the last section of this chapter, the telematic to new education methods, in which students
system QUEST, designed to be used as a tool for must have a more active role, as they are not

only content receivers but also content genera- results than others (competitive learning), as this
tors. This is something that has to be promoted is also required in the labour world. ICT may be
more and more. used in order to reduce negative effects as those
related to a possible bad atmosphere. There are
not many studies related to this, so it remains to
Future reseArch dIrectIons be seen whether the negative effects of competi-
tion can be mitigated with the anonymity that
Nowadays, it is not easy to find in the literature networking technologies provide.
cases in which a competitive active learning ICT provide the possibility of simultaneously
strategy based on ICT is applied. Actually, we applying different learning strategies through
cannot say that the application of active learning the use of different telematic tools within the
strategies is a widely spread practice, as the old classroom. As students with different learning
model in which teachers give lectures and students styles coexist in a classroom and the learning
have passive roles still predominates. In Europe, strategies must be in accordance with the learning
universities are immersed in the process of con- styles of students, the results of applying different
vergence towards the European Higher Education learning strategies within a classroom should be
Area (EHEA), and the use of active methodologies analysed. In this sense, several questions arise
is an objective in the short term. Therefore, it is such as whether we should use a unique strategy
urgent to analyse the strategies that are adequate for all the students of a classroom, adapted to the
for different educational situations, according to most common learning style of the students, or
the cognitive objectives, the individual learning use different strategies for the different groups
styles and the context, thus providing teachers with of students. Also, it should be analysed how this
some guidelines that allow them to apply these would affect the results of students as well as the
strategies in their classrooms. Although, as it has development of course programmes.
been shown along this chapter, there are different Besides, there are other ways of active partici-
real cases of application of these methodologies pation of students such as peer review that has
with good results, teachers do not count with advantages in terms of reduction of the workload
clear guidelines to decide which methodology or of the teacher. However, at the same time, teach-
technology is adequate in each case. Frequently, ers partially lose control over the process, and
teachers would like to use a methodology or a this is the reason why it can not be applied in
tool but they do not feel able to do it efficiently. any educational context. In this sense, it would
We can affirm that there is a lot of work left in also be important to analyse these techniques
order to establish a set of clear guidelines to be comparatively with competitive strategies.
applied in each academic situation.
As it has been commented on in the chapter,
there are some authors who think that competi- reFerences
tion is not a good learning strategy due to the
fact that it causes stress in the student and it can Brightman, H. J. (2006). GSU master teacher
even create a bad atmosphere in the classroom. program: On critical thinking. Retrieved October
Anyway, although these negative effects exist, it 26, 2007, from http://www2.gsu.edu/~dschjb/ww-
must be taken into account the educational level. wcrit.html
In a university education context, the student must
Brown, E., Cristea, A., Stewart, C., & Brailsford,
not only learn to work in groups (collaborative
T. (2005). Patterns in authoring of adaptive educa-
learning) but also to be competitive offering better

tional hypermedia: A taxonomy of learning styles. Fandos, M., & González, A. P. (2005). Estrategias
Educational Technology & Society, 8(3), 77-90. de Aprendizaje ante las Nuevas Posibilidades
Retrieved October 26, 2007, from http://www. Educativas de las TIC. In A. Méndez-Vilas, B.
ifets.info/journals/8_3/8.pdf Gonzalez, J. Mesa, & J. A. Mesa (Eds.), Proceed-
ings of the Third International Conference on
Bryndum, S., & Montes, J. A. (2005). La moti-
Multimedia and Information & Communication
vación en los entornos telemáticos. RED Revista de
Technologies in Education (pp. 7-10). Cáceres,
Educación a Distancia, V(13). Retrieved October
Spain: Formatex.
26, 2007, from http://www.um.es/ead/red/13/
Frees, S., & Kessler, G. D. (2004). Developing
Burd, B. A., & Buchanan, L. E. (2004). Teaching
collaborative tools to promote communication
the teachers: Teaching and learning online. Refer-
and active learning in academia. In Proceed-
ence Services Review, 32(4), 404-412.
ings of the 34th Annual Conference Frontiers in
Canós, L., & Mauri, J. J. (2005). Metodologías Education (FIE’04) (Vol. 3, pp. S3B/20 - S3B/25).
Activas para la Docencia y Aplicación de las Piscataway, NJ: IEEE.
Nuevas Tecnologías: una Experiencia. In URSI
Godoy, L. A. (2005). Learning-by-doing in a Web-
2005. Retrieved October 26, 2007, from http://
based simulated environment. In Proceedings of
w3.iec.csic.es/ursi/articulos_gandia_2005/articu-
the 6th International Conference on Information
los/otros_articulos/462.pdf
Technology Based Higher Education and Training
Chang, L. J., Yang, J. C., Yu, F. Y., & Chan, T. W. (ITHET 2005) (pp. F4C/7 - F4C/10). Piscataway,
(2003). Development and evaluation of multiple NJ: IEEE.
competitive activities in a synchronous quiz game
Hislop, G. W. (1999). Anytime, anyplace learning
system. Journal of Innovations in Education and
in an online graduate professional degree program.
Training International. 40(1), 16-26.
Group Decision and Negotiation, 8, 385-390.
Chang, S. -B., Wang, H. -Y., Liang, J. -K., Liu,
Hyland, B. (2002). Cone of learning. From the
T. -C., & Chan, T. W. (2004). A contest event in
course “Train the trainer”. Iowa Center for Public
the connected classroom using wireless handheld
Health Preparedness. Retrieved October 26, 2007,
devices. In J. Roschelle, T.-W. Chan, Kinshuk, &
from http://www.public-health.uiowa.edu/icphp/
S. J. H. Yang (Eds.), Proceedings of the 2nd IEEE
ed_training/ttt/archive/2002/2002_course_mate-
International Workshop on Wireless and Mobile
rials/Cone_of_Learning.pdf
Technologies in Education (WMTE 2004) (pp.
207-208). Los Alamitos, CA: IEEE Computer Johnson, R., & Johnson, D. W. (1998). Coopera-
Society. tive learning. Two heads learn better than one.
Transforming Education, 18, 34.
Chickering, A. W., & Ehrmann, S. C. (1996,
October). Implementing the seven principles: Kim, S., & Sonnenwald, D. H. (2002). Investi-
Technology as lever. American Association for gating the relationship between learning style
Higher Education Bulletin, 49(2), 3-6. preferences and teaching collaboration skills
and technology: An exploratory study. In E.
Chu, K., Chang, M., & Hsia, Y. (2004). Stimulating
Toms (Ed.), Proceedings of the American Society
students to learn with accuracy counter based on
of Information Science & Technology Annual
competitive learning. In Proceedings of the IEEE
Conference (pp. 64-73). Medford, NJ: Informa-
International Conference on Advanced Learning
tion Today.
Technologies (ICALT’04) (pp. 786-788). IEEE
Computer Society.

Marqués, P. (2001). Didáctica. Los Procesos de Symons, S., & Symons, D. (2002). Using the
Enseñanza y Aprendizaje. La motivación. Re- Inter- and Intranet in a university introductory
trieved October 26, 2007, from http://dewey.uab. psychology course to promote active learning. In
es/pmarques/actodid.htm Proceedings of the International Conference on
Computers in Education (ICCE’02) (Vol. 2, pp.
Martínez, A., Gómez, E., Dimitriadis, Y., Jorrín,
844- 845). IEEE Computer Society.
I. M., Rubia, B., & Vega, G. (2005). Multiple case
studies to enhance project-based learning in a Timmerman, B., & Lingard, R. (2003). Assess-
computer architecture course. IEEE Transactions ment of active learning with upper division
on Education, 48(3), 482-489. computer science students. In Proceedings of the
33rd Annual Conference Frontiers in Education
McCarthy, J. P., & Anderson, L. (2000). Active
(FIE’03) (Vol. 3, pp. S1D/7 - S1D/12). Piscataway,
learning techniques vs. traditional teaching
NJ: IEEE.
styles: Two experiments from history and politi-
cal science. Innovative Higher Education, 24(4), Titcomb, S. L., Foote, R. M., & Carpenter, H.
279-294. J. (2004). A model for a successful high school
engineering design competition. In Proceedings
Mehlenbacher, B., Miller, C. R., Covington, D.,
of the 34th Annual Conference Frontiers in Edu-
& Larsen, J. S. (2000). Active and interactive
cation (FIE’04) (Vol. 1, pp. 138-141). Piscataway,
learning online: A comparison of Web-based
NJ: IEEE.
and conventional writing classes. IEEE Trans-
actions on Professional Communication, 43(2), Van der Linde, G. (2005). The perception of
166-184. business students at PUCMM of the use of col-
laborative learning using the BSCW as a tool. In
Mickle, M. H., Shuman, L., & Spring, M. (2004).
Proceedings of the 6th International Conference on
Active learning courses on the cutting edge of
Information Technology Based Higher Education
technology. In Proceedings of the 34th Annual
and Training (ITHET 2005) (pp. F2D/10- F2D/15).
Conference Frontiers in Education (FIE’04) (Vol.
Piscataway, NJ: IEEE.
1, pp. T2F/19 – T2F/23). Piscataway, NJ: IEEE.
Verdú, M. J., de Castro, J. P., Pérez, M. A., Verdú,
Morozov, M., Tanakov, A., Gerasimov, A., Bys-
E., & Regueras, L. M. (2006). Application of
trov, D., & Cvirco, E. (2004). Virtual chemistry
TIC-based active methodologies in the framework
laboratory for school education. In Proceedings of
of the new model of university education: The
the IEEE International Conference on Advanced
educational interaction system QUEST. In F. J.
Learning Technologies (ICALT’04) (pp. 605-608).
García, J. Lozano & F. Lamamie de Clairac (Eds.),
IEEE Computer Society.
CEUR Workshop Proceedings, Virtual Campus
Philpot, T. A., Hall, R. H., Hubing, N., & Flori, 2006 Postproceedings. Selected and Extended
R. E. (2005). Using games to teach statics calcu- Papers (Vol. 186, pp. 33-40). CEUR-WS.org.
lation procedures: Application and assessment. Retrieved October 26, 2007, from http://CEUR-
Computer Applications in Engineering Educa- WS.org/Vol-186/
tion, 13(3), 222-232.
Verdú, E., Regueras, L. M., Verdú, M. J., Pérez,
Reichlmayr, T. (2005). Enhancing the student M. A., & de Castro, J. P. (2006a). Improving the
project team experience with blended learning higher education through technology-based active
techniques. In Proceedings of the 35th Annual methodologies: A case study. WSEAS Transac-
Conference Frontiers in Education (FIE’05) (pp. tions on Advances in Engineering Education,
T4F/6- T4F/11). Piscataway, NJ: IEEE. 3(7), 649-656.

Verdú, E., Regueras, L. M., Verdú, M. J., Pérez, M. 207-208). Los Alamitos, CA: IEEE Computer
A., & de Castro, J. P. (2006b). QUEST: A contest- Society.
based approach to technology-enhanced active
Chang, L. J., Yang, J. C., Yu, F. Y., & Chan, T. W.
learning in higher education. In S. Impedovo, D.
(2003). Development and evaluation of multiple
Kalpic & Z. Stjepanovic (Eds.), Proceedings of
competitive activities in a synchronous quiz game
6th WSEAS International Conference on Distance
system. Journal of Innovations in Education and
Learning and Wb Engineering (DIWEB ‘06) (pp.
Training International, 40(1), 16-26.
10-15). Wisconsin: WSEAS.
Chu, K., Chang, M., & Hsia, Y. (2004). Stimulating
Verdú, E., Verdú, M. J., Regueras, L. M., & de
students to learn with accuracy counter based on
Castro, J. P. (2005). Intercultural and multilingual
competitive learning. In Proceedings of the IEEE
e-learning to bridge the digital divide. Lecture
International Conference on Advanced Learning
Notes in Computer Sciences, 3597, 260-269.
Technologies (ICALT’04) (pp. 786-788). IEEE
Wirsig, S. (2002). ¿Cuál es el lugar de la tecnología Computer Society.
en la educación? Retrieved October 26, 2007, from
Covington, M. V. (1998). The will to learn: A
http://www.educoas.com/Portal/xbak2/tempo-
guide for motivating young people. New York:
rario1/latitud/Wirsig_Tic_en_Educacion.doc
Cambridge University Press.
Yu, F. Y., Chang L. J., Liu, Y. H., & Chan, T. W.
Frees, S., & Kessler, G. D. (2004). Developing
(2002). Learning preferences towards computer-
collaborative tools to promote communication
ised competitive modes. Journal of Computer-
and active learning in academia. In Proceed-
Assisted Learning, 18(3), 341-350.
ings of the 34th Annual Conference Frontiers in
Zywno, M. S., & Waalen, J. K. (2002). The effect Education (FIE’04) (Vol. 3, pp. S3B/20 - S3B/25).
of individual learning styles on student outcomes Piscataway, NJ: IEEE.
in technology-enabled education. Global Journal
Godoy, L. A. (2005). Learning-by-doing in a Web-
of Engineering Education, 6(1), 35-44.
based simulated environment. In Proceedings of
the 6th International Conference on Information
Technology Based Higher Education and Training
AddItIonAL reAdIngs (ITHET 2005) (pp. F4C/7 - F4C/10). Piscataway,
NJ: IEEE.
Brown, E., Cristea, A., Stewart, C., & Brailsford,
Hislop, G. W. (1999). Anytime, anyplace learning
T. (2005). Patterns in authoring of adaptive educa-
in an online graduate professional degree program.
tional hypermedia: A taxonomy of learning styles.
Group Decision and Negotiation, 8, 385-390.
Educational Technology & Society, 8(3), 77-90.
Retrieved October 26, 2007, from http://www. Johnson, R., & Johnson, D. W. (1998). Coopera-
ifets.info/journals/8_3/8.pdf tive learning. Two heads learn better than one.
Transforming Education, 18, 34.
Chang, S. -B., Wang, H. -Y., Liang, J. -K., Liu,
T. -C., & Chan, T. W. (2004). A contest event in Kim, S., & Sonnenwald, D. H. (2002). Investi-
the connected classroom using wireless handheld gating the relationship between learning style
devices. In J. Roschelle, T.-W. Chan, Kinshuk & preferences and teaching collaboration skills
S. J. H. Yang (Eds.), Proceedings of the 2nd IEEE and technology: An exploratory study. In E.
International Workshop on Wireless and Mobile Toms (Ed.), Proceedings of the American Society
Technologies in Education (WMTE 2004) (pp. of Information Science & Technology Annual

Conference (pp. 64-73). Medford, NJ: Informa- Titcomb, S. L., Foote, R. M., & Carpenter, H.
tion Today. J. (2004). A model for a successful high school
engineering design competition. In Proceedings
Levy, Y. (2007). Comparing dropouts and per-
of the 34th Annual Conference Frontiers in Edu-
sistence in e-learning courses. Computers &
cation (FIE’04) (Vol. 1, pp. 138-141). Piscataway,
Education 48, 185-204.
NJ: IEEE.
McCarthy, J. P., & Anderson, L. (2000). Active
Van der Linde, G. (2005). The perception of
learning techniques vs. traditional teaching
business students at PUCMM of the use of col-
styles: Two experiments from history and politi-
laborative learning using the BSCW as a tool. In
cal science. Innovative Higher Education, 24(4),
Proceedings of the 6th International Conference on
279-294.
Information Technology Based Higher Education
Mehlenbacher, B., Miller, C. R., Covington, D., & and Training (ITHET 2005) (pp. F2D/10- F2D/15).
Larsen, J.S. (2000). Active and interactive learn- Piscataway, NJ: IEEE.
ing online: A comparison of Web-based and con-
Verdú, E., Regueras, L. M., Verdú, M. J., Pérez
ventional writing classes. IEEE Transactions on
M. A., & de Castro, J. P. (2006). Improving the
Professional Communication, 43(2), 166-184.
higher education through technology-based active
Mickle, M. H., Shuman, L., & Spring, M. (2004). methodologies: A case study. WSEAS Transac-
Active learning courses on the cutting edge of tions on Advances in Engineering Education,
technology. In Proceedings of the 34th Annual 3(7), 649-656.
Conference Frontiers in Education (FIE’04) (Vol.
Verdú, E., Verdú, M. J., García, J., & López,
1, pp. T2F/19 – T2F/23). Piscataway, NJ: IEEE.
R. (Eds.). (2006). Best practices in e-learn-
Reichlmayr, T. (2005). Enhancing the student ing: Towards a tecnology-based and quality
project team experience with blended learning education. Valladolid, Spain: Boecillo Editora
techniques. In Proceedings of the 35th Annual Multimedia.
Conference Frontiers in Education (FIE’05) (pp.
Yu, F. Y., Chang L. J., Liu Y. H., & Chan, T. W.
T4F/6- T4F/11). Piscataway, NJ: IEEE.
(2002). Learning preferences towards computer-
Symons, S., & Symons, D. (2002). Using the ised competitive modes. Journal of Computer-
Inter- and Intranet in a university introductory Assisted Learning, 18(3), 341-350.
psychology course to promote active learning. In
Zywno, M. S., & Waalen, J. K. (2002). The effect
Proceedings of the International Conference on
of individual learning styles on student outcomes
Computers in Education (ICCE’02) (Vol. 2, pp.
in technology-enabled education. Global Journal
844- 845). IEEE Computer Society.
of Engineering Education, 6(1), 35-44.
Timmerman, B., & Lingard, R. (2003). Assess-
ment of active learning with upper division
computer science students. In Proceedings of the
33rd Annual Conference Frontiers in Education
(FIE’03) (Vol. 3, pp. S1D/7 - S1D/12). Piscataway,
NJ: IEEE.

0
Chapter XIV
Open Source LMS
Customization: A Moodle
Statistical Control Application
Miguel Ángel Conde González
Carlos Muñoz Martín

CLAY Formación Internacional, Spain
Alberto Velasco Florines

CLAY Formación Internacional, Spain
ABstrAct
This paper reflects the possibility of doing adaptations on a learning management system (LMS) depend-
ing on the necessities of a company or institution. In this case, ACEM allows the definition of course-
level and platform-level reports and the automatic generation of certificates and diplomas for Moodle
LMS. These adaptations are intended to complement all the different learning platforms by contributing
added-value features like the generation of customizable diplomas and certificates and reports, which
allow the obtaining information about both grades and participation in every activity of a course. All
this necessities are not provided by default.
Open Source LMS Customization
IntroductIon This situation must be resolved without providing

a new platform but developing an independent
Lately both state and private institutions are Web application which can be used on top of the
betting on e-learning solutions so as to satisfy instance of Moodle being used now. In addition,
their formation necessities. Any of the kinds the use of the product can be extended to any
of formation which can be offered—in person, other instance of this LMS.
blended, and online—should rely on a complete In order to resolve this necessity, the creation
and flexible enough technological support, that of ACEM is proposed. This will be an applica-
is, a learning management system (LMS). It can tion independent of Moodle and will provide the
be used as a support for in-person learning and as desired functionality without having to alter the
a base of the virtual part for the other two types. data obtained from any instance of this LMS.
These systems are in charge of the different actions Some of the current LMS in the market
involved in the online formation process, which will be described next. It will be detailed their
include the student management and providing limitations about dealing with grades and their
them with resources and activities. Because of management of certificates and diplomas, with
this, e-learning cannot be implemented without special interest in Moodle as it is the starting
the support of an LMS. point of the problem. After that, we comment on
Nowadays, many different LMS can be the development model in Moodle, the difficulties
found in the market, which must be classified which are raised depending on the distribution
into commercial ones and free distribution ones. of the data in the LMS. Finally, the developed
Commercial LMS generally offer a higher level product is shown and the conclusions and future
of customization, since the companies which de- work lines are listed.
velop them can accomplish additional features by
incrementing the final price. On the other hand,
free software-based LMS have several advantages current PLAtForms And theIr
due to its free software condition. However, these LImItAtIons
platforms are not adequate in some cases.
Among free distribution LMS, we must men- New necessities arise from the experience with the
tion Moodle, which is a free software project usage of knowledge management platforms, which
designed to support a framework for a construc- current systems cannot resolve. It is necessary to
tive social education (Comezaña & Garcia, 2005). understand and realize the limitations which these
Currently, it is one of the most extended LMS in platforms have and also the objectives which are
the world, both in state and private institutions. intended to achieve, so that they do not involve
It also has a large community of users and de- an obstacle when deploying.
velopers. Despite this large community behind Although it is obvious that there is not a pana-
it, Moodle does not adapt to all the necessities cea capable of solving all the limitations which
that can arise. these platforms show, it is simple to observe, by
The problem arises specifically from a going into their features in depth, that some basic
particular necessity of Centro Internacional functions are not supported.
de Tecnologías Avanzadas (CITA), belonging The first limitation which arises when dealing
to Fundación Germán Sánchez Ruipérez. Its with LMS platforms is just its own definition. They
objective is achieving an optimal qualification are systems for managing the learning process
management and a system which allows them to but generally they do not include other interesting
generate diplomas and certificates automatically. aspects, like an adequate content management

which enables the creation of content within the And this satisfaction is a key element for the
platform, that is, the typical features of a content success of any kind of platform, especially in the
management system (CMS) which turns out to be case of a LMS platform which does not pose a
very useful for this kind of platforms. total innovation but a new way of understanding
The so-called learning content management the learning process.
system (LCMS) arise so as to solve this separation, An unsupported concept by LMS platforms
but its extension, apart from scarce, is basically arises as a result of these statistic properties:
reduced to proprietary tools for specific usages the idea of portfolio associated to each student,
(Rengarajan, 2001). seen as a curriculum which shows the student’s
Another limitation shown by LMS platforms progress and achievements or even information
is the absence of implementing standards which related to learning modalities like in-person or
enable an easy migration of learning objects blended (blended learning). Taking the different
between platforms (Maurer, 2004). Here comes phases of the learning process into account, many
the concept of reutilization, which is fundamen- platforms in the market cannot cover the whole
tal when talking about learning platforms. An process due to, basically, the lack of methods for
approximation which represents a solution is the certificating or generating documentation which
implantation of the necessary functionality for proves the acquisition of certain knowledge.
the correct processing of packages, like sharable Finally, setting aside the technical features,
content object reference model (SCORM) (Jones, another handicap which these platforms have to
2002). Thus, learning objects can be transferred face is their necessity to be correctly managed by
between platforms without having to redefine the qualified personnel, not only from the academic
contents on each platform. point of view, but from a technical point of view.
Another lack of functionality in LMS platforms This will help to obtain the desired effect and
is related to the graphical user interface manage- make the learning process be productive and
ment, which is derived from the features of any satisfactory.
Web application. Although the concentration for In general, these limitations are not applicable
this kind of platforms should be in the learning to every single existing platform but each platform
process itself and the quality of contents, important presents a subset of these limitations which forms
interface aspects like Web accessibility should a base for the implementation of possible improve-
never be neglected. ments. The identification of the weakest points
From a statistic point of view, there are usu- of those platforms will depend on the objectives
ally not platforms which have a wide range of or requirements which are intended to fulfil and,
statistics about their usage, their administration, therefore, a research for helping to choose the
or, more specifically, the grades of the students most adequate option should be made.
who use them. What is more, they do not include
any graphical representations which help to take
decisions about, for example, the way of posing the cAse oF moodLe
or presenting contents based on the information
taken from the system (about users, administra- Moodle is a kind of LMS platform aimed basically
tors, teachers, students, and grades, above all). to provide a set of tools and structures which enable
Taking into account the importance of some adapting the learning model to an online one.
aspects like feedback, the features previously As a result of the nature of LMS platforms
mentioned become necessary if a high level of and Moodle in special, the lack of a direct com-
satisfaction of all the involved parts is desired. munication between the teacher and the students,

or even, among the students, makes the system user to obtain the percentage of those who are
responsible for this communication as it is the included in each range. Thus, the quality of the
mediator, whereas in other types of learning knowledge penetration on the students could be
these responsibilities rely on personal interaction. measured in a moderately accurate way, depend-
These refer to the necessity of providing a set of ing on the number of factors taken into account:
functionality which will substitute the exchange the more factors, the more accurate measure.
of information between teacher and student and This measure, along with other aspects, could
among students. be established as highly adequate rates so that
In order to achieve this goal, the techniques the teachers could readapt the contents in order
used are based on statistic methods for compiling to increase the reach of the required knowledge.
information. This information includes important This content customization is very important when
aspects like the measure of the students’ grades dealing with platforms like this or similar.
which are generated by the management mod- Another possible factor to be measured should
ules and for which these systems provide. As for be the presentation rate of those activities, that
Moodle, although it does provide methods for is, the percentage of students who either have
evaluating the different activities which can be not done the activity or have not handed it in. In
done, it does not generate relevant statistics about addition, reports listing those students could be
the evolution of the students based on either their generated.
grades or other measures considered useful for As it can be observed, the possibilities of
these cases. system improvement that Moodle leaves open
Besides these problems, the grade manage- are considerable, especially the grade system, and
ment system of Moodle has its own gaps. Moodle an improvement would contribute positively to
groups the students of a subject of course into achieve the goals of the LMS platforms in general
groups, and each of them can have a disjoint set and of Moodle, in particular. However, there is
of those students. This layout, along with the another relevant limitation shared by most of the
characteristics of the group, enables the interaction LMS platforms, besides those which have already
with the students depending on the definition and been exposed: the inability to cover the whole
permissions associated with that group (Castro, learning process, especially the final phases.
2007). However, the problem arises when estab- More specifically, Moodle does not allow the
lishing the grades. This organization in groups is generation of any kind of certification for each
not extended to the classification of students and student which would identify him as a holder of
therefore no reports of grades are provided taking the knowledge learned in the course once it has
this into account. Because of this, the teacher is finished. This would require some kind of module
forced to do the segregation, which is really tedi- which allow the generation of diplomas, in an
ous, even more when dealing with courses with automatic and systematic way for every student.
a high number of students. These documents should be also fully customiz-
As for the achievement of the results of the able so as to adequate to the characteristics of each
course, grades obviously are the main scale of course, based on its modality or other aspects.
measure, though it is influenced by other aspects Besides all these limitations, Moodle has
like the frequency of visits to the different activi- some positive features which make it one of the
ties suggested. Considering grades as the main most used LMS platforms. The objective of the
element of the metric, some kind of organization description of its limitations is that though the
of the students depending on certain ranges of functionality of these platforms is usually wide
grades is missed in Moodle, which will allow the and flexible, they are not able to cover all the goals
considered relevant by those who use them.

the deveLoPment modeL In tion and taking into account application goals, two
moodLe architectures can be proposed, in order to obtain
the maximum scalability, which allows an easy
The development in Moodle can be tackled in reutilization of the application for later versions
different ways depending on the desired level of of Moodle. It is up to a client’s decision as to the
independence from the platform of the software type of architecture to be implemented since each
to develop: one will have different goals and estimations.
• External applications. An application which suggested Architectures

uses the database of the platform without
having to make any integration with its First, a solution based on a separated application
interface. with access to the database of Moodle is sug-
• Module development. It consists on building gested. A diagram representing this architecture
new modules which can be integrated into the is shown in Figure 1.
platform. That integration involves defining In this solution, the creation of an application
an installation in the database or even an externally from Moodle is suggested. This appli-
inclusion into the interface of Moodle. There cation will access its database directly and it will
is a lot of documentation about the style of be structured in the following layers:
coding for this type of developments and
some libraries of functions for the database • Access and queries to Moodle DB: This
management of Moodle as well (http://docs. layer will contain the necessary functions
moodle.org). to perform queries on a Moodle database.
It is proposed that the utilization of query
Based on the client’s necessities, the first op- description files which enable the queries
tion is chosen. These necessities specify certain to be changed in a simple way depending
requirements whose fulfilment is more adequate on the changes done on Moodle database.
to a nonintegrated solution. Considering this op- This helps to achieve a certain level of
Figure 1. ACEM first architecture proposal

compatibility with later versions of Moodle • GUI: Graphical user interface of the ap-
by doing minimal changes, although the plication.
objective of the application is to support
Moodle version 1.6. This layer also abstracts Here, the database access is performed via the
the statistic control functions layer from the new application programming interface (API)
data searching process. which Moodle provides in version 1.7: Moodle
• Statistic control functions layer: This DML Library (Lafuente & Hunt, 2007) and
layer will use the functions located in the Moodle DDL Library. Access for data extracting
previous layer so as to perform the statistic for Moodle version 1.6 will also be provided.
control. This layer will implement the logic Finally, the client chooses the first option.
of the application.
• GUI: Graphical user interface of the ap- obtaining the data model
plication.
The development can start once its model has
The second architecture suggested is based on been defined. Developing Moodle based software
an application linked to Moodle which uses the involves several difficulties. One of the most rep-
new database access features of version 1.7 with resentatives is the little knowledge about Moodle
full support for version 1.6. This architecture is database because neither its data model nor the
shown in Figure 2. used tables are public. Thus, in order to find out
In this case, the layers to be developed are that information, reverse engineering (Hainaut,
these: Tonneau, Joris, & Chandelon, 1993) methods have
to be used. In this case, a tool called DBDesigner
• Statistic control functions layer: This (http://www.fabforce.net/dbdesigner4/) has been
layer will implement the logic of the ap- used since it enables data model extraction from
plication. the own database currently installed. Considering
the data model obtained, research must be done so
Figure 2. ACEM second architecture proposal

as to determine how Moodle fills the data tables consist of logging in each course automatically
corresponding to the element to be analysed. and accessing the grades Web page (grades.php)
Specifically, one or more tables will be queried in a totally transparent way for the user. This
for each of them because of the data distribution technique is effective but after some testing, it
in Moodle and the necessities of the application was noticed that Moodle might have an alterna-
as well. Some of those tables are: tive login page, so this case must be taken into
account (that piece of information is asked the
• User related tables: They will give informa- user when configuring the application).
tion about the users of the platform, that is, Nowadays, some significant delay is produced
the students of the courses. This information when testing this technique with a large number
is necessary in order to generate reports, of students. Because of that, it might be changed
certificates, and diplomas. for retrieving that information by doing direct
• Resource related tables: They will give queries on the database as it is done when deal-
information about the existing resources ing with grades.
in the platform and the way they can be
accessed. An error in the grade system
• Graded activity related table: These will
allow the calculation of the grades at course- After researching grade calculation methods, a
level and platform-level. possible defect in the grade system of Moodle has
• Not graded activity related tables: These appeared. The platform weights all the grades and
are used for doing a list of activities for the can also weight them a second time if the corre-
reports. spondent option has been enabled by the user. It
• Course related tables: These will give all appears that this might lead to an error, because
the information about the existing courses in the final grade of a student does not correspond
the platform along with an identifier which to the logic expected. It has been observed that
enables the relation to both resource and Moodle does weight grades although it has been
activity tables. asked not to do it (Figure 3) when doing some
testing:
Once the information necessities and the way In Figure 3, the attention must be paid on user
of retrieving it are defined, some research on the “apellido 2, alumno 2.” This user has finished his
possible situations which might involve an update three graded activities in which he has got, respec-
of the data stored in the tables mentioned before tively, 4 out of 10, 16.5 out of 20, and 3.3 out of 10.
needs to be done. Special care is needed when Moodle adds the user’s grades and then divides
generating grades per activity. the result by the sum of the maximum grades of
The table which associates the different activi- each activity he has done: 23.8/90 = 0.264, which
ties with a course is built from the data of many is equivalent to 26.4%. This situation does not
other tables and the information is updated only reflects correctly the grades because, this way,
when the teacher or student access the grades an activity rated in a scale of 100 will always be
section of a certain course. This means that, when more worthy than an activity rated in a scale of
generating a platform-level report, it would be 10, although both have the same weight.
necessary to access the grades of each course The correct calculation is weighting each grade
manually every time a change happens. The op- first and then doing the average so that the global
tion chosen to avoid that is “touching” the courses grade is logical. In the previous example, the
involved in the desired report. This method will results would be these, when normalizing to 10:

Figure 3. Moodle grades GEnERaToR oF REPoRTS,

DiPLomaS, anD CERTiFiCaTES
In the following lines, the functionality obtained

in the generator of reports will be described, its
main elements, and the difficulties found during
the development.
Functionality
ACEM was built as an application for a certain

client but after considering the limitations of
Moodle which have been overcome, we decided
to use it as an add-value component for that learn-
ing platform.
The application implements the following
functionality:
• 3n. This option enables the obtaining of

statistic data related to the whole platform.
The generated report will reflect data related
to all the students in the campus, the users,
some generic information about the courses,
the resources and the activities grouped by
their type, as well as information about the
grades in the platform. The user is allowed
to decide whether to include or not those
courses without graded activities, which in-
volves a change in the final grade since more
courses are used to compute the statistics.
Information of final report can be comple-
mented with data typed by the user means
of a WYSIWYG (What You See Is What
You Get) editor. The data of the generated
document are shown in several ways (lists,
tables, charts, etc.) as it can be observed on
figures 4, 5, 6, and 7.
(4/10)*10 = 4, (16./20)*10 = 8.25 and (3.3/10)*10 =
3.3. So the average value would be: (4+8.25+3.3)/8 Any of the documents generated can be ob-
= 1.94; whereas according to Moodle calculations tained in different formats like PDF, HTML, or
it would be: (23.8/90 )*10 = 2.64. Thus, in the MS-WORD (.doc).
case of ACEM, all the grades will be considered
equally no matter the scale of each element. • Course-level report generation: It allows
a user to obtain statistic data of a certain

Figure 4. Information in list mode
Figure 5. Information in table
Figure 6. Bar chart

Figure 7. Pie chart
course. The information of this report is the graphical user interface (GUI), which is a
about the number of students, resources, and fundamental element of any application. Other
activities of that course and the grades as elements which are not shown in the diagram are
well. The user is allowed to choose the stu- the authentication component and the installation
dents and grades to see. Later in the report, component.
each student’s grades are shown grouped by
subject. Like in the previous case, the user • GUI: It is very important for any applica-
is allowed to enter a head text and footer by tion to have a user interface which enables
means of an editor. It can also be exported a simple and efficient access to its func-
to different formats. tionality (Gándara, 1995). As for ACEM, it
• Diplomas and certificate generation: The has been tried to design a user interface as
application also provides a tool for defining efficient, light, friendly, and functional as
customized diplomas for the students of a possible. From the main page, all the avail-
course. Some of the elements which can be able options are accessible and it is possible
configured are the image logos (which are to come back here from within any of them.
placed automatically), the title, the name of The technologies used for building it are
the institution, and so forth. The diplomas HTML, PHP, and Cascade Styling Sheets
can be obtained one by one or by groups of (CCS).
students. • Authentication: ACEM deals with personal
data which include grades of several courses,
some components of the so the access to this data must be controlled
Application somehow. In order to do this, a component
for authentication will check if the user that
Three different layers are distinguished in the tries to access to the system has logged in
architecture shown in Figure 1. One of those is correctly. If not, he will have the chance to

Figure 8. ACEM GUI
log in but he can never access the data without Difficulties Found During the
having been authenticated. The system uses Development
HTTP sessions (Welling & Thomson, 2003)
to check it. The username and password are The development of any software applications
specified during the installation process. involves some difficulties. In this particular case,
• Installation: As any other application, most of them have appeared from the necessity
ACEM will need a process of installation and of researching on different PHP libraries which
configuration. In this particular case, it will were not known by the developers. Another big
be checked if a file called “config.php” exists problem is the time efficiency of the suggested
and if it does not exist, the Web page for the solutions.
basic configuration of the application will Some research on several libraries had to be
be shown (Figure 9). In this window there done in order to be able to perform the different
are several data which have to be entered kind of actions needed, like a library for generat-
so that the application can work properly: ing charts in PHP, another for creating PDF files,
• The type of the database which will be and so forth. The first option for each of them has
used along with its name, a user, and the always been a free software one.
correspondent password. For the generation of charts, we chose a free
• The Moodle administrator’s username and software PHP library called PHPlot (http://source-
its password. This is required in order to be forge.net/projects/phplot/) which allows develop-
able to generate the grades. ers to generate a wide range of charts like bars
• The URL of the installation of Moodle to be charts or pie charts.
used and the one of the login Web page. As for PDF files, the first choice was using
• The username and password to access FPDF library (http://www.fpdf.org/) for generat-
ACEM. ing the certificates and diplomas and DOM PDF
(http://sourceforge.net/projects/dompdf/) in the
case of reports. FPDF allows the generation of
0
Figure 9. ACEM configuration page
complex documents but it needs every element Another difficulty is the increase of time due
to be positioned within the document, what can to the large number of database queries to be
be really difficult for large documents. This is done. At first, the queries were executed without
the reason why DOM PDF is used for generating control obtaining excessive execution times. Then,
the reports, as this library allows the automatic the number of queries to execute was reduced
conversion from HTML to PDF. Thus, the re- and a cache of query results was implemented.
port is initially formatted in HTML and then, Thanks to this, the execution time was reduced
by invocating one function of the library, the to a fourth of the original time, though we are
final PDF document is obtained. However, when still working on it.
the number of students, and thus the size of the
documents generated grows, the time spent in
generating the reports is unacceptable. Therefore, concLusIon
the generation of reports will be done using FPDF
library as well, although it involves much more An added-value tool for a LMS platform has been
time to develop. developed by using Web technologies, which is
For generating MS-WORD documents currently under exploitation by the client.
(DOC), several libraries researched, but none By observing the different LMS platforms in
of them worked properly so the final choice was the market, it can be noticed that most of them
generating a HTML document and then chang- do not provide enough graphical representation
ing its file extension to .doc because, currently, about the students’ activity in the courses and
MS-WORD converts automatically from HTML they do not allow the generation of certificates
to DOC keeping the structure of the document or diplomas either. What is more, Moodle, which
practically intact.

is one of the most used LMS platforms, does not be searched for in order to avoid doing the
include those features. current conversion. Therefore, some com-
Several options for developing the application mercial library could be used.
have been taken into account and, based on the • Reduce report generation times. This is a
client’s needs, an external application has been really important point to be taken into ac-
defined. ACEM allows users to obtain information count. It can be accomplished by reducing
in many ways, like documents, charts, or diplo- the number of queries executed, improving
mas which enhance the information stored by the the cache of queries or even redefining the
platform. Nowadays, this kind of information is business logic.
fundamental for the management of studies in a • Include new graded activities. It might
learning platform. be taken into account introducing certain
The development of the application has in- graded elements which are not considered
volved doing an exhaustive research on the data now or even other elements which will ap-
model of Moodle, its management, and the way pear in later versions of Moodle.
it deals with grades. After doing it and resolving • Adaptation to new versions of Moodle.
some problems, a set of queries has been defined Since Moodle is continuously developing,
so that the application can use them for obtaining ACEM should adapt to those changes. In
the requested reports. order to do that, new versions of Moodle
Due to the fact that application aim is the have to be analysed, including its database
generation of the most complete, representative and also the new APIs available like Moodle
and useful reports, the use of several libraries, DML Library and Moodle DDL Library.
which enable the inclusion of charts and reports • Build ACEM as a module of Moodle. In
to be exported to the most usual formats, is the near future, the possibility of adapting
necessary. ACEM to an integrated module of Moodle
The current version of the application is the should be considered, with all the work
first, although development will be continued so which it will involve.
as to improve its functionality, thus enhancing
Moodle.
reFerences
Future work LInes Castro, E. (2007). Moodle: Manual del professor.

Retrieved October 28, 2007, from http://moodle.
As for the possible evolutions of the application, org/file.php/11/manual_del_profesor/Manual-
there are several options to work with. profesor.pdf
Comezaña, O., & García, F. J. (2005). Plataformas
• Different methods for graphical represen-
para educación basada en Web: Herramientas,
tation. Visualization of information is an
procesos de evaluación y seguridad (Tech Rep.
area of computer science which is developing
DPTOIA-IT-2005-001). España, Salamanca:
these days (Rohrer & Swing, 1997). A bar
Universidad de Salamanca, Departamento de
char or a pie chart is not enough. Charts with
Informática y Automática.
much richer information are needed and the
technology is improving in that direction. Gándara, M. (1995). User Interface: An introduc-
• Improve MS-WORD document generation for educators. In J. M. Alvarez-Manilla and
tion. A solution for a better generation should A. M. Bañuelos (Eds.), Computer pedagogical
uses. Mexico CISE/UNAM.

Hainaut, J., Tonneau, C., Joris, M., & Chandelon, AddItIonAL reAdIngs
M. (1993). Transformation based database reverse
engineering. In R. Elmasri, V. Kouramajian & B. Castro, E. (2007). Moodle: Manual del professor.
Thalheim (Eds.), Conference on Entity Relation- Retrieved October 28, 2007, from http://moodle.
ship Approach (pp. 364-375). Springer. org/file.php/11/manual_del_profesor/Manual-
profesor.pdf
Jones, E. R., (2002). Implications of SCORM™
and emerging e-learning standards on engineer- Koper, R., & Tattersall, C. (Eds.). (2005). Learning
ing education. In ASEE Gulf-Southwest Annual design: A handbook on modelling and delivering
Conference (pp. 20-22). networked education and training. Springer.
Lafuente, E., & Hunt, T. (2007) Development: Lambropoulos, N., & Zaphiris, P. (Eds.). (2007).
XMLDB documentation. Retrieved October 28, User-centered design of online learning com-
2007, from http://docs.moodle.org/en/Develop- munities. Hershey, PA: IGI Global.
ment:XMLDB_Documentation
Spence, R. (Ed.). (2007). Information visualiza-
Maurer, W. (2004). Estándares e-learning. tion: Design for interaction. Prentice Hall.
SEESCYT. Retrieved October 28, 2007, from
Welling, L., & Thomson, L. (Eds.). (2003). Php and
http://fgsnet.nova.edu/cread2/pdf/Maurer1.pdf
Mysql Web development. Developer’s Library.
Rengarajan, R. (2001). LCMS and LMS: Taking
William, R. (Ed.). (2006). Moodle e-learning
advantage of tight integration. Click 2 Learn.
course development: E-learning course develop-
ment. UK: Packt Publishing.
e-learn.cz/soubory/lcms_and_lms.pdf
Rohrer, R.M., & Swing, E. (1997). Web-based
information visualization. Computer Graphics
and Applications, IEEE, 17(1/4), 52-59.
Welling, L., & Thomson, L. (2003). Using session
control in PHP. In Sams Publishing (Ed.), Php and
mysql Web development. Developer’s Library.

Chapter XV
Evaluation and
Effective Learning:
Strategic Use of E-Portfolio as an
Alternative Assessment at University
Nuria Hernández Nanclares

Universidad de Oviedo, Spain
ABstrAct
This chapter analyses evaluation as a strategic instrument to promote active and significant learning
and how, in that strategy, the use of alternative assessment and technology-aided learning-and-teaching
processes could be of great help. There is an important margin to allow the teachers to design the as-
sessment in a strategic manner and modify the nature of the students’ learning activities. So, the central
question is analysing whether the use of an electronic portfolio as an assessment tool in the subject
“International Economic Relations,” has been used strategically. In other words, is the type of desired
learning really being achieved? Is significant and deep learning being stimulated? If not, what kind of
learning is being stimulated? How should the assessment be modified to achieve the desired results?
To help answer all these questions, we have analysed whether the activities and products which make
up the “International Economic Relations” portfolio fulfil the conditions that characterise a strategic
evaluation.
IntroductIon of alternative assessment and technology-aided

learning-and-teaching processes could be of great
The present chapter analyses evaluation as a help. Therefore, the principal aim of the chapter
strategic instrument to promote active and sig- is to discuss the difficulties that assessment pro-
nificant learning and how, in that strategy, the use cesses have and how the use of digital learning
Evaluation and Effective Learning
platforms to complement teaching could help to of learning we want in the modern university and
use evaluation as a strategic decision to achieve how to use the evaluation as a strategic element
better learning. of the teaching process. Second, to explore the
The method used to raise the question is to possibilities of using the portfolio as an alterna-
expose an educational innovation that has been tive assessment tool, I describe my own experi-
developed for several years in the University ence with an electronic version of this evaluation
of Oviedo. The subject I teach in the faculty of methodology. Finally, I critically review the REI
Economics “International Economic Relations” evaluation innovation and compare it with what
(REI) is the context I have used to develop an is considered by experts as a strategic assessment
electronic portfolio as an assessment tool which tool to enhance effective learning. Hence, the
helps me to evaluate the students and to implicate experience is described in detail trying to find
them in more active learning. This is an optional out if the conditions of a tactical evaluation are
subject in business administration with 150-175 present in it.
students in three groups, morning and evening.
The technological aspect of the portfolio is sup-
ported by the electronic learning platform in aSSESSmEnT aS a STRaTEGy,
University of Oviedo, Aulanet, nowadays ruled sIgnIFIcAnt LeArnIng As A
in Moodle. goAL
The theoretical framework in which this
chapter is inspired has two complementary di- One of the basic objectives of the present-day uni-
mensions. On the one hand, we have to consider versity is to achieve effective teaching: teaching
the crucial aspect of evaluation at university and by transforming the lecture room into a learning
the role teachers have to develop as assessment space, thus endowing the classes with an addi-
agents. To evaluate the learning achieved by tional value that justifies the students’ presence in
students is a big challenge for teachers and for them. This approach means that students must be
institutions, especially in a context where the active subjects of their own learning, they must
learning of competences and abilities are of much experience for themselves the changes in the way
higher importance. As a result, evaluation has of thinking, feeling, and acting that the learning
to be revisited, being used as a strategic tool to process produces in them.
promote effective learning and an alternative to To achieve this, it must be the learners who,
assessment options. with the teacher’s help, build their own learning.
On the other hand, the chapter is concerned This requires the students to become involved
with the use of technology in education, exploring in the process, to be willing to take part in the
the possibilities that virtual educational platforms learning opportunities suggested in the lecture
and ITC’s have in helping the use of alternative room, and to continue the process until they
assessment procedures to enhance active learning. reach a high level of independence in it. It is
The context is the technology-aided learning-and- essential, therefore, that the students should be
teaching processes. I don’t employ the more ex- highly motivated. Some students are capable of
tended expression of “technology based learning” motivating themselves, but the majority need
because I agree with the idea that technology can external stimuli to arouse interest within them in
help learning but it cannot replace the personal the task to be carried out and the knowledge to be
and social dimensions of learning. acquired. One of the aspects that has the greatest
Bearing in mind all these questions, the rest of influence on the students and their involvement
the chapter is devoted first to explore what type in the learning process is the way they expect

to be assessed under the system of assessment etc.) which will help them to characterize the
established by the teachers. Studies carried out structure and organization of the new knowledge,
in the 1970s and 1980s ( Miller & Parlet, 1974; drawing their attention to it and enabling them to
Snyder, 1974) indicate that the assessment in fact link new to existing knowledge.
has more influence on the learning process than The use of active learning techniques is essen-
the actual teaching does. Furthermore, students tial in this kind of learning. Active methodologies
react to changes in the system of assessment by involve the students in their own learning and lead
modifying the way in which they tackle tasks and to assessment systems that go beyond conceptual
by generating learning activities adapted to the contents, stimulating in the student an integral
assessment requirements (Sambell & McDowell, learning process of all the abilities required of
1998). professionals nowadays.
Hence, a relevant question to ask is what kind Moreover, a thorough learning process has
of learning is generated by the different assess- two additional dimensions which I consider to
ment systems we are using and whether it is the be relevant. On the one hand, it has an important
right kind for the requirements associated with social dimension since collaborative learning
the new situation. The evidence seems to suggest helps to achieve more effective and significant
that on many occasions the assessment normally results than does individual learning alone. On
utilized does not stimulate the right kind of learn- the other hand, however, the personal dimension
ing, that is to say, learning of a significant nature. is essential from the moment when the student’s
A margin therefore exists in which the system of processes of reflection on their own learning
assessment, as a basic teaching tool, can be used (metacognition) form an essential element in the
strategically, motivating the student to acquire a degree of thoroughness of this learning.
specific type of learning. From the foregoing and in view of the nature
Clearly, the kind of learning we want to mo- of the learning desired, the final assessment,
tivate is an effective and significant one in which which is normally individual, basically accumu-
the students build up their own knowledge, alone lative, and has a very low formative potential,
or in a group, and in an active manner in order to should be replaced by a continuous assessment
acquire conceptual, procedural, and behavioural process with a significant diagnostic and forma-
knowledge. tive character and with a high level of feedback
Previous knowledge on the part of the stu- for the student. In short, the assessment should
dents is fundamental to this type of learning be designed as a system of incentives aimed at
since the need to connect the new to the old is a achieving a certain type of learning: significant,
basic premise if the students are to acquire really active, and collaborative.
significant learning. According to its constructive Therefore, as mentioned previously, there is an
focussing, the learning process implies the stu- important margin to allow the teachers to design
dents evolve from the structure of their previous the assessment in a strategic manner and modify
knowledge. Constructivists therefore propose the the nature of the students’ learning activities.
use of organized structures of concepts that form The assessment can be conceived as a system of
the framework within which the students can incentives aimed at ensuring that the students
structure new materials related to their previous adopt specific forms of behaviour directed to-
knowledge. Consequently, it is very convenient to wards achieving certain types of learning (Gibbs
make the students aware of their knowledge of a & Simpson, 2005).
specific subject, presenting them with some type From this point of view, the assessment
of material (oral, written, graphic, audiovisual, becomes a central teaching element, closely

related to the learning objectives and decisive As already stated, the students select the topics
in the choice of contents and the establishing of and the way of carrying out tasks depending on
tasks. The assessment is therefore not an isolated what they think will be asked of them in the exam.
process undergone by students and then used to So the first condition the assessment has to meet
evaluate them but rather an activity in which they is to guarantee a certain quantity and quality of
also participate and through which they can learn effort on the part of the student. Hence, a system of
(Brown & Glasner, 1999). assessment designed to make the students dedicate
The following question to consider is which sufficient time and effort to the tasks assigned to
characteristics must the evaluation possess in them, distributing this effort throughout the entire
order to be strategic and contribute to effective process, meets the first condition. Moreover, the
learning. According to Gibbs and Simpson (2005), learning effort made by the student must reach
the assessment systems must fulfil a number of a certain level of quality .For this purpose, the
conditions if they are to motivate the students to learning tasks and activities must be so devised
become involved in the learning tasks, modify- that the student learns in an efficient manner,
ing the way in which they would initially tackle clearly demanding mental and cognitive processes
them. The students are also strategic in the use of of a high level to solve these tasks.
their time and effort and divide them up in order If the assessment system is to stimulate effec-
to obtain the best results in accordance with what tive and significant learning, it must itself become
they perceive as the assessment requirements. We a learning activity whose development is designed
must therefore bear in mind that the signals sent as a learning process. For this to happen, the
by the teachers through their assessment strategy correction and feedback activities must play an
do not always lead to the kind of learning basi- essential part. The feedback from the teacher has
cally desired. To achieve this type of learning, to be of sufficient quantity, quality, and speed. It
the assessment must fulfil several conditions that is important that the corrections and indications
relate to the students’ effort, the feedback from offered should be focussed more on assessing
the teacher, and how the students respond to this learning, particularly in its formative aspect,
feedback (see Table 1). and less on assessing the student. The comments
Table 1. Conditions under which assessments supports student learning (Gibbs, Simpson, & Macdonald,
2003, p. 2)
1.Quantity and distribution of student effort

Assessed tasks capture sufficient study time and effort.
These task distribute student effort evenly across topics and weeks.
2.Quality and level of student effort
These tasks engage students in productive learning activity.
Assessment communicates clear and high expectations to students.
3.Quantity and timing of feedback
Sufficient feedback is provided, both often enough and in enough detail.
The feedback is provided quickly enough to be useful to students.
4.Quality of feedback
Feedback focuses on learning rather than on marks or students themselves.
Feedback is linked to the purpose of the assignment and to criteria.
Feedback is understandable to students, given their sophistication.
5.Student response to feedback
Feedback is received by students and attended to.
Feedback is acted upon by students to improve their work or their learning.

should be focussed on the objective for which and in formation and conviction, for the employ-
the task was designed and limit themselves to ment of active methodologies and the preparation
the quality criteria previously established. They of the feedback necessary to make the correction
should preferably be in written form and compre- of activities effective. New technologies and their
hensible to the students; they need to be adapted application in teaching may alleviate the burden
to the students’ vision of the discipline since, as somewhat in this process. The possibilities of-
learners, the students are not always capable of fered by the Web and virtual teaching platforms
having an overall idea of the subject and of what mean that computers have become an essential
knowledge it implies. work tool and that the means of communication
Finally, in order to be truly effective, the as- between teacher and students and among the latter
sessment system must guarantee that students have multiplied. Thus, technology plays a major
receive the feedback, assimilate it, and react to it, part in various aspects of the teaching-learning
modifying those matters that the correction has process, among which are the use of collaborative
shown can be improved upon. The assessment work and the possibilities of the strategic use of
must therefore be designed in such a way that the assessment, a matter of concern to us. Informa-
feedback is useful, making use of the indications tion and communication technologies (ICT’s) can
for subsequent tasks and activities which thus help in the design and programming of activities,
incorporate the improvements indicated in the encouraging a specific distribution of work time
correction of the previous task. by establishing work procedures directed towards
It is important to point out that all these ele- more productive learning, communicating the
ments, which should be taken into account when results of the correction process to the students,
selecting the assessment strategy, are associated and establishing feedback between the students
with the learner, the student being an active subject and the teacher, briefly by involving the students
of the learning process, and not so much with the more in the tasks and making them aware of the
teacher or the teaching process. As I see it, the level of quality that these should possess.
central idea suggested by Gibbs and Simpson
(2005) is therefore that the assessment is related to
the students’ learning, occupying a central position the eLectronIc PortFoLIo oF
in their involvement in the process, leaving the “InternAtIonAL economIc
teachers the possibility of using it to direct their reLAtIons”
teaching and achieve the right kind of learning.
This reinforces the idea that the learner, with all This section describes the teaching innova-
his internal learning processes and motivation, tion carried out in the Department of Applied
plays a central role in the teaching-learning pro- Economics at the University of Oviedo for the
cess. The teacher, as an expert in the subject and subject “International Economic Relations.” The
in the best way of learning it, thereby becomes a innovation is designed as an alternative assessment
provider of the process mentioned, endowing his that permits evaluation of more than conceptual
teaching and assessment decisions with a strategic knowledge. One of the instruments that can be
character that guarantees the best results. used to carry out this type of alternative assess-
This activity of the teacher as a mediator in the ment is the student’s portfolio.
learning process of his students and a strategist in The portfolio is a technique of collection,
the use of teaching and assessment does not come compilation, and repertoire of evidences and
cheaply (Feuerstein, 1990; Feuerstein et al., 1980). professional competences that qualify a person
A great effort is required, both in terms of time for a satisfactory professional development. So,

this tool is a collection of processes and prod- the use of portfolio in the practicum assessment
ucts which result from the student carrying out for the professional training course for part time
the learning activities designed by the teacher youth workers in the United Kingdom. Johnson-
to achieve certain objectives. These objectives Bogart (1995) explains the use of a portfolio in
should be fixed for the acquisition of abilities the University of Washington’s Interdisciplin-
using suitable contents that have been selected ary Writing Program. In this experience, it is
for this purpose. Originally, the use of portfolios of special importance the opportunity portfolio
appeared in artists disciplines, especially in ar- gives students of being self reflective about their
chitecture and design, as a form to demonstrate own writing. Spillane (1999) explains how port-
professional capacities in the labour market. In folio could be used in Long-Life Learning adult
the educational field, the portfolios are adapted education and Williams, Davis, Metcalf, and
as a teaching and assessment technique used as Covington (2003) apply this assessment system
an alternative method of evaluating the capacities to teacher education programs. All of them find
and abilities the students have acquired during the useful and valuable this alternative method of
learning process (Klenowski, 2002; Klenowski, evaluation, although to make such assessment
Askew, & Carnell, 2006). more widely accepted further research, compar-
Different options exist regarding the contents ing portfolio with conventional method, has to
of the portfolio. They range from compulsory be conducted. In the way of making alternative
contents, where the tasks to be performed are assessment methods more accepted and used the
decided by the teacher, to pure portfolios, where it evolution of new technologies could be of great
is the student or group of students responsible for help. In the case of portfolios, the development
the portfolio who decides which of their learning of electronic portfolios could play a valuable
products are the best. role in higher education (Challis, 2005). In Bar-
This instrument permits continuous assess- ragan (2005) and in Agra, Gewerc, and Montero
ment since the tasks can be carried out one after (2002) experiences in using portfolio in Spanish
the other, throughout the learning period, and the University can be found.
selected products can be corrected by the teacher
to reach the formative goal. At the same time, it Description of the innovation
is possible to ask for the inclusion in the portfolio
of artefacts which show the learning processes The essential innovative teaching aspect in “In-
followed by the student, allowing these processes ternational Economic Relations” is to propose a
to be assessed. This is of particular interest when change in the way of assessing the knowledge
we are dealing with collaborative portfolios and acquired by the students with the purpose of
it is necessary to assess the progress of the work stimulating active learning and encouraging at-
group. Summing up, if an alternative, continuous, tendance in classes by transforming them into a
formative assessment is desired, the student’s space for collaborative activities where interac-
portfolio may be a good option. tions between pairs, both within the collaborative
The use of a different approach to the assess- group and the large one and also between the
ment of the student learning, based in portfolio, students and teachers, should form the focal point
has been used in many different situations. There of activity in the lecture room. This helps give
are many references in literature to experiences the classes a value of their own, a work time and
using portfolio based assessment in any levels place whose products will be assessed by means of
and in higher education, too. For example, and an electronic portfolio linked to the collaborative
among many others, Cooper (1996) describes work both inside and outside of the class.

This experience started during the 2001-02 system, trading relations, etc.), preparing and tak-
academic year and was repeated during the ing part in a debate on globalization, developing
following years, with a number of changes and tasks associated with the theoretical contents of
improvements relating to the learning objectives each topic, providing any complementary work
and activities through which these objectives are which they consider will improve the contents
to be achieved and also certain modifications in of the portfolio, and so forth. Attending classes
the actual development of the classes and in the always counts for a certain percentage in the final
way of assessing attendance, work done, and the assessment of the portfolio.
contents of the file. The thorough renovation of the The assessment system suggested involves
subject was completed in the 2004-05 academic the students in a teaching-learning process
year, with the managing of the portfolio pass- which presents three elements considered to be
ing over to the virtual teaching platform of the essential to achieve high-quality learning: gener-
University of Oviedo, Aulanet, thus converting ate students active learning implicating them in
it into an electronic portfolio. class activities and tasks, collaborative learning
This portfolio is called electronic because of by using the work groups as the focal point for
the use of ITC’s and virtual platform as support activities, and an alternative assessment process
and infrastructure in the design of the course. by gathering the learning products together in an
This is not the typical “on-line” subject where electronically managed portfolio. So the process
the students have to do the work by their own at is active, collaborative, and supported by new
home. On the contrary, the tasks are distributed technologies.
over different times: classroom time, working Currently, the development of this experi-
group time, individual time, and deadline time. ence allows different types of objectives to be
The connection between these times is done elec- achieved. On the one hand, the main objective
tronically, so the materials and written instructions of the innovation is using in practice a procedure
are handed out through the Web page, but the of alternative assessment that stimulates active
activities are presented, explained, and begun in learning by the students, consisting of an elec-
the classroom time. To take advantage of the digital tronic, collaborative portfolio with which they
formats, the submission has to be done through can pass the “International Economic Relations”
the platform working group tools and most of the subject through different levels of requirements
management of the activities and communication and involvement.
between students and teachers is electronically In addition, the following secondary objectives
done. So, in this experience, the technology is of are set: rethinking the university teacher’s role
great help but the initiative is mainly done with as an assessor; revising the assessment process
attendance. The technology cannot substitute the within the context of learning abilities, recogniz-
personal and social dimension of learning. ing its difficulties and the need for alternative
The contents of the portfolio have become forms of assessment, considering and analysing
more sophisticated over the years, progressing the portfolio as one such possible form of alterna-
from exercises in the application of theory to tive assessment.
more elaborate pieces of work. Nowadays, the To assess the experience during different years
portfolio is made up of several different kinds we carried out a survey among the students, both
of processes and products: preparing a piece at the beginning and at the end of the academic
of work on a certain aspect of the international term. This was slightly modified during these
economy (international economic institutions years in order to adapt to changes in the project.
and their operation, the international monetary The initial survey was designed to find out what
0
prior intentions and expectations the students To this end, the way to gain greater insight
had with regard to the subject and the active into this experience is to ask oneself whether this
practical work. The purpose of the final survey objective is really being achieved: is the right kind
was to ascertain whether the level of interest in of learning being stimulated? If not, what kind
the subject had been maintained until it had been of learning is being stimulated? How should the
completed, whether students considered it impor- assessment be modified to achieve the desired
tant for their education, the degree of difficulty results? (Gibbs & Simpson, 2003).
of the subject, the suitability of the bibliography, A first approximation to help answer these
the students’ willingness to take part in further questions is to analyse whether the activities
innovative teaching experiences, and a series of and products which make up the “International
assessments of their collaborative work and its Economic Relations” portfolio, in its present
possibilities. form, fulfil the conditions indicated by Gibbs et
Apart from the questions, the students were al. (2003) and presented in Table 1. So, in this
given the opportunity to indicate those aspects that section, a reflection over the strategic elements
could be improved upon in the future. The main of the electronic portfolio of the IER subject is
observations were: there should be more practical done and summarized in Table 2.
groupwork, debates and greater opportunities to
express opinions, though they are conscious of Quantity and distribution of student
the difficulties they have in participating more effort
in class (particularly orally); the practical work
should be more closely linked to real-life situa- The first condition is related to the amount and
tions and more up-to-date. They also suggested distribution of the students’ effort. To ensure
commenting on newspaper articles in class; better that they dedicate sufficient time to the subject,
assessment of the work done, since it requires a we developed a series of tasks to be spread out
great effort on their part, and better organization over the term. In the first place, they were asked
to reduce class sizes, and so forth. The results of to prepare reports on the international monetary
this assessment were very satisfactory and that is system over a period of twelve weeks. This is a
the reason why the experiment has continued over group project, but each member of the group is
several successive academic years (Hernández responsible for one particular report. One hour of
Nanclares 2004; 2006). class time per week, the hour designated “practical
work,” is entirely devoted to the development and
completion of this work. To prevent the students
the PortFoLIo As A strAtegIc from concentrating all their effort at the end,
Instrument oF Assessment just before the date for handing in the work and
starting the Christmas holidays, we suggested to
The question under consideration at this point them a work plan with a detailed schedule fixing
in the development of the innovation is whether a series of tasks and convenient time periods for
the system described here is being used strategi- completing them. Each week the groups check to
cally, in other words, whether it is encouraging see how the work is progressing. Group meetings
significant and deep learning in my students. As have been arranged for the end of November. In
has been discussed throughout the section, there them, each student has to present to the group
is a margin to allow the assessment to be oriented and to the teacher the scheme, the central ideas
towards the kind of work done by the students and and the planning of the report before dedicating
the way in which they tackle the tasks. the last part of the time to writing it.

Secondly, apart from this “medium-term” class. The basic structure followed in the class is
work, “short-term” tasks are decided upon. These to provide the students with materials that have to
are associated with the different topics being dealt be processed inside and outside the class, consider
with in class and add extra value to the portfolio. question clusters and discuss the materials, work
They are activities that have to be handed in within in a small group and present the results in a large
a short period and normally involve completing one. Within this basic organization, the function
classwork or a short check of knowledge acquired. of the teacher is to coordinate participations in
Some of them are voluntary, so excessive pressure class combined with some lectures. The bulk of
is not put on the students. In general, they are the the work here consists in preparing materials,
responsibility of the whole group but, in some producing the question clusters, and presenting the
cases, such as the knowledge test, they are carried topic in a way that will prove to be of interest and
out by the individual, though the average mark will make the students want to know more about
of the group is entered in the portfolio. Thirdly, a it, thus motivating them to prepare the materials
“long-term” task, presented at the start of the year and consult the bibliography. This kind of work
but not carried out until January, has also been allows the students to investigate their previous
conceived. It involves preparing and taking part in concepts about a particular question, share them
a debate on globalization. With this organization with their group and from there, through discus-
of tasks we believe that the students’ work effort sion and participation, acquire new knowledge
and time will be divided up evenly throughout that permits them to advance. It also forces them
the term and among the contents. to develop new abilities and skills related to com-
munication, teamwork, and comprehension and
Quality and Level of Student Effort analysis. The tasks then allow them to complete
the process and present it in the form of a prod-
The second condition refers to the quality of effort uct. They have therefore been asked to produce
made by the students. To ensure that their learn- a concept map on globalization, present written
ing is of a suitable nature (significant, active, and answers to the question clusters of the material
collaborative), the activities have been devised in relating to international capital markets, answer
a way that means students will become involved a short test on the basic theory of exchange rates
in mental processes of a high cognitive level, with and produce another concept map on the reform of
increasing degrees of difficulty. As an example, the the International Monetary Fund. We believe the
reports must be produced within clearly set limits, tasks to be sufficiently demanding and challeng-
with a well-defined analysis perspective. The key ing to involve the students in productive learning
concepts to be covered are specified and the space activities. The fact that they are performed in a
is limited to three or four sheets so they have to work group means that the knowledge acquired
concentrate in real relevant aspects of the topic. is much richer, the levels of difficulty of the tasks
A very important role is played in this point by increasing as the groups develop their own dy-
the way instructions are given. They are written, namics and become more consolidated.
very clear from the start, and with indications as
to how the elements have to be assesed. Moreover, the role of correction and
the specific assessment criteria that will be taken Feedback
into consideration during the correction work are
at the students’ disposal. The rest of the conditions refer to feedback. On
As already mentioned, the “short-term” tasks the one hand, the quantity and quality of cor-
are related to the day-to-day dynamics of the rections and the opportunity the teacher has of

making them and, on the other, the spirit in which with the criteria explained, the work of all the
the students receive them and their effects on the groups was presented in class, and the students
students’ future learning. were encouraged to look at other students’ work
As regards the amount and opportunity of and compare it with their own. This allowed them
feedback, we, the teachers, try to make correc- to become aware of the different levels of quality
tions as quickly and opportunely as possible. For that existed and of how other groups had gone
this purpose, we utilize the ICT’s and the virtual about planning and presenting the work. They
platform of the subject to speed up communica- therefore gain an idea of what is involved in the
tion. We also carry out a check of the messages search for quality in the work and can begin the
that the students send to their forums in order self-assessment process. As for the knowledge
to detect problems, imbalances and erroneous test, the correction was carried out in pairs. The
concepts which we attempt to rectify. We make teacher resolved the questions in class and, at the
suggestions to them and increase the bibliography same time, each student checked an anonymous
references. The forums allow us to detect gaps colleague’s exam. Different levels of quality
that may have remained, thus providing the op- were fixed, depending on the completeness of the
portunity to complete the information in another answers, and students were asked to indicate this
class or with messages to the general forum to level in the exercise they corrected. This makes
which all the students have access. them aware of what it means to assess somebody
To achieve a quality that endows the cor- else’s work, learn from their own and other people’s
rections with a true formative character, we mistakes and adheres to quality criteria that can
distinguish between two assessment dimensions. reasonably be expected.
Firstly, we assess the quality of the learning pro- The last of the conditions established in Table
cess, maintaining a constant observation of the 1 refers to the spirit in which students accept
development of the groups. Three instruments are the corrections and how these affect their future
basically used: direct observation of the group learning.
activity during attendance hours, a check of the To ensure the students receive the corrections
messages in the group forums, and the opinions and incorporate them in the learning processes,
of the students themselves gathered directly dur- we follow two guidelines: to carry out two or
ing classes and in the group meeting. In this way more short tasks of the same kind and to check
we aim to detect possible malfunctions in the each week the progress of the longer tasks. Two
development of the group and solve problems of concept maps are therefore produced: the first, at
relationships and understanding before the end the start of the academic year, with precise instruc-
of the term. tions, exhaustive corrections, and the exhibition
Second, the intention is to offer the students of work. The second during the Christmas period
useful corrections of the learning products. Nor- to check whether the students are incorporating
mally, the objectives, contents, and assessment the improvements indicated in the previous pro-
criteria of the tasks are clearly established and are duction. In addition, the questions and issues of
known to the students. Thus, the comments refer the voluntary tasks and the knowledge check are
to previously described variable, which enables so conceived that the same models, concepts and
students to assess the corrections. Moreover, the theories are considered from different points of
assessments are qualitative, which as far as pos- view. For their part, the medium-term reports are
sible avoids giving a final, numerical mark. checked weekly. The work plan is checked and the
For example, in the case of the concept map, work is completed at the group meeting.
the students were sent a comment written together

Table 2. Is the REI electronic portfolio strategically used?
Assessment conditions Activities and products of the REI portfolio

1.Quantity and distribution of Design of tasks with different hand-in periods
student effort “Short-term” tasks: Associated with class activities, responsibility of the group;
some are voluntary
“Medium-term” tasks: Preparation of reports; work plan with detailed schedule;
at least one group meeting; weekly check during practical class
“Long-term” tasks: globalization debate; maintaining effort at the end of the term,
examining in greater depth ideas already dealt with
2.Quality and level of student Design of tasks in which conceptual objectives are combined with abilities and
effort skills
“Short-term” tasks: Knowledge and analysis of and relationship between
concepts, preparation of contents starting out from previous personal and group
knowledge, discussion and justification of different opinions, reading comprehen-
sion in different languages
“Medium-term” tasks: Search for, selection and integration of information,
taking decisions on how to organise the different phases, written communication
“Long-term” tasks: Development of oral communication skills; consolidation of
teamwork capacity; defence of postures through technical arguments
3.Quantity and timing of Returning correction within a reasonable period of time in order to maintain both
feedback the teacher’s and the student’s interest in the task and its results
Use of electronic means of communication available on the subject Web page
4.Quality of feedback Preparation of written corrections in accordance with the following ideas:
Start out from clear and familiar assessment criteria
Focus the correction on what the student has done, being positive; highlight
mistakes but also things done well
Place special emphasis on ways of improvement for future tasks Make the students
aware of the minimum quality criteria required in the technical work and in the
quality of their own work
Carrying out individual and group self-assessment and assessment in pairs
5.Student response to feed- A qualitative assessment, without a final mark, to ensure attention is paid to the
back formative aspect.
Repetition of tasks in the same style to incorporate corrections made and past
learning
Incorporation of individual assessments in the group portfolio to encourage
responsibility and a desire for quality
Accumulative assessment of the portfolio with the option of exemption from the
exam, which encourages self-motivation, continuous effort and the capacity to
extend the reward for work done.
The entire foregoing analysis indicates that to improve the part relating to the collaborative
the assessment system conceived is strategic in work assessment and the usefulness of the sub-
character and can help the student to generate suit- ject page on the virtual platform, Aulanet. The
able learning. At all events, the definitive check questionnaire is given to students at the end of the
would be to ask the students. For this purpose, semester, in January, before they get their final
during the 2006-07 academic year, we modified qualifications. So, the answers are reliable because
the questionnaires with which the innovation is they have just finished the term and are not yet
assessed in order to try to detect these aspects. disappointed by the summative assessment. At
In this new form of assessing the experience we this point of the research, we have not yet results
used the “Assessment Experience Questionnaire” from this first evaluation of the innovation. We
developed by Brown, Gibbs, and Glover (2003) cannot compare it with the previous years results
and Gibbs and Simpson (2004). We also tried because the questions and evaluation tool is totally

changed. The previous experience has been useful results? To help answer all this questions we have
for design the innovation but, in same sense, from analysed whether the activities and products which
now on it is completely new one. The previous make up the “International Economic Relations”
experience has been useful in designing the cur- portfolio, in its present form, fulfil the conditions
rent innovation but, the one described here is a that characterised a strategic evaluation.
completely new one. The basic differences are how After the review, we can conclude that the as-
the experience is evaluated and what the analysis sessment system conceived and practiced in the
of the results and their implications are. experience described is strategic in character and
can help the student to generate suitable learning.
concLusIon Of course, the experience could be improved, and
for that, and as a result, the opinion of students
The great challenge university teachers have has to be taken in account. Future versions of
nowadays is to promote adequate learning in the REI electronic portfolio would be modified
students. Clearly, the kind of learning we want in light of the results of the questionnaire with
to motivate is an effective and significant one in which we are evaluating the experience.
which the students build their own knowledge,
alone or in a group, and in an active manner in
order to acquire conceptual, procedural, and Future reseArch And
behavioural knowledge. dIrectIons
One of the most important tools we have to
reach this aim is evaluation. There is an important The research I have begun this course 2006-07
margin to allow the teachers to design the assess- is based in the “Assessment Experience Ques-
ment in a strategic manner and modify the nature tionnaire” developed by Brown et al. (2003) and
of the students’ learning activities. The assessment Gibbs and Simpson (2004) and it has a specific
can be conceived as a system of incentives aimed objective: analysing the assessment tool used in
at ensuring that the students adopt specific forms the view of its strategic capacity to generate ef-
of behaviour directed towards achieving certain ficient learning.
types of learning. The assessment becomes a The future research will be oriented in several
central teaching element, closely related to the directions:
learning objectives and decisive in the choice of
contents and the establishing of tasks. To achieve • First, I want to refine the assessment tool.
this type of learning, the assessment must fulfil This means to gain deeper knowledge about
several conditions that relate to the students’ ef- the portfolio technique to evaluate in the
fort, the feedback from the teacher, and how the university. So, analyzing more the theoretical
students respond to this feedback. aspects and the previous experiences would
Having all this in mind, the central question of be the field of research.
this chapter is whether the innovation developed • Secondly, I want to analyse the results of
in the subject “International Economic Relations,” the questionnaire and change those aspects
already described here, is being used strategi- of the assessment that not contribute to effi-
cally. In other words, is really being achieved cient learning. This implies using again next
the objective of this experience? Is significant course the portfolio approach and collecting
and deep learning being stimulated? If not, what more data about it, so comparisons could
kind of learning is being stimulated? How should be done. Probably, the analysis of results
the assessment be modified to achieve the desired requires us to rethink or redefine the differ-

ent activities which are part of the portfolio. Cooper, T. (1996). Portfolio assessment in higher
Perhaps more effort has to be put in task education. In Proceedings Western Australia
related with capacities and competencies Institute for Educational Research Forum 1996.
than in conceptual questions. Retrieved October 28, 2007, from http://www.
• Third, I would like, if possible, to compare waier.org.au/forums/1996/cooper.html
my results with other experiences using
Feuerstein, R. (1990). The theory of structural
the same questionnaire a different context:
cognitive modifiability. In B.Z. Presseisen (Ed.),
careers, specialities, uiversities, and so
Learning and thinking styles: Classroom appli-
forth.
cations (pp. 68-134). Washington, DC: National
Education Association.
reFerences Feuerstein, R., Rand, Y., Hoffman, M. B, & Miller,
R. (1980). Instrumental enrichment. Baltimore,
Agra, M.J., Gewerc, A., & Montero, M.L. (2002). MD: University Park Press.
El portafolios como herramienta de análisis en
Gibbs, G., & Simpson, C. (2004). Measuring the
experiencias de formación online y presenciales
response of students to assessment: The assess-
II Congreso Europeo de Tecnologías de la In-
ment experience questionnaire. In C. Rust (Ed.),
formación en la Educación y en la Ciudadanía.
Improving student learning: Theory, research and
Barcelona.
scholarship. Oxford: Oxford Centre for Staff and
Barragán Sánchez, R. (2005). El Portafolio, met- Learning Development.
odología de evaluación y aprendizaje de cara al
Gibbs, G., & Simpson, C. (2003). Does your
nuevo Espacio Europeo de Educación Superior.
assessment support your students’ learning?
Una experiencia práctica en la Universidad de
Journal of Learning and Teaching in Higher
Sevilla, Revista Latinoamericana deTecnología
Education, 1(1).
Educativa, 4(1), 121-139. Retrieved October 28,
2007, from http://www.unex.es/didactica/RE- Gibbs, G., & Simpson, C. (2005). Conditions
LATEC/sumario_4_1.htm under which assessment supports students’ learn-
ing. Learning and Teaching in Higher Education
Brown, E., Gibbs, G., & Glover, C. (2003). Evalu-
1, 3-31.
ation tools for investigating the impact of assess-
ment regimes on student learning. Bioscience Gibbs, G., Simpson, C., & Macdonald, R. (2003).
Education E-Journal, 2. Retrieved October 28, Improving student learning through changing
2007, from http://bio.ltsn.ac.uk/journal/vol2/beej- assessment—A conceptual and practical frame-
2-5.htm work. Paper presented at the European Associa-
tion for Research into Learning and Instruction
Brown, S., & Glasner, A. (Ed.). (1999). Assess-
Conference, Padova, Italy.
ment matters in higher education. UK: Open
University Press. Hernández Nanclares, N. (2004). La evaluación
mediante portafolio en Relaciones Económicas
Challis, D. (2005, Fall). Towards the mature e-
Internacionales. In R. Rodríguez, J. Hernández
portfolio: Some implications for higher education.
& S. Fernández (Eds.), Docencia universitaria:
Canadian Journal of Learning and Technology,
Orientaciones para la formación del profesorado
31(3).
(pp. 331-341). Documentos ICE, Instituto de Cien-
cias de la Educación, Universidad de Oviedo.

Hernández Nanclares, N. (2006) El portafolios AddItIonAL reAdIng

electrónico: Una alternativa para evaluar en la
Universidad. Paper presented in I jornadas de Baron, C. (1996). Creating a digital portfolio.
innovación educativa de la Escuela Politécnica Indianapolis, IN: Hayden Books
Superior de Zamora, junio Zamora, España.
Barret, H. (2000). Create your own electronic
Johnson-Bogart, K. (1995). Writing portfolios: portfolio. Learning & Leading with Technology,
What teachers learn from students self-assess- 27(7), 14-21.
ment. In Washington Centre’s Evaluation Com-
Barrett, H. (2000). The electronic portfolio de-
mittee (Ed.), Assessment in and of collaborative
velopment process. Retrieved October 28, 2007,
learning. Washington: Washington Centre for
from http://electronicportfolios.org/portfolios/EP-
Improving the Quality of Undergraduate Edu-
DevProcess.html
cation.
Barrett, H. (2005). Research electronic portfolios
Klenowski, V. (2002). Developing portfolios for
and learner engagement. White Paper. Retrieved
learning and assessment: Processes and princi-
October 28, 2007, from http://www.taskstream.
ples. London: RoutledgeFalmer.
com/reflect/whitepaper.pdf
Klenowski, V., Askew, S., & Carnell, E. (2006).
Bates, A., & Poole, G. (2003). Effective teaching
Portfolios for learning, assessment and profes-
with technology in higher education. San Fran-
sional development. Assessment and Evaluation
cisco: Jossey-Bass.
in Higher Education, 31(3), 267-286.
Belait, L. (2001). La evaluación de la acción.
Miller, C.M.I., & Parlett, M. (1974). Up to the
El dossier progresivo de los alumnos. Diada
mark: A study of the examination game. Guildford:
editoras.
Society for Research into Higher Education.
Benito, A., & Cruz, A. (Eds.). (2005). Nuevas
Sambell, K., & Mcdowell, L. (1998). The con-
claves para la docencia universitaria en el Es-
struction of the hidden curriculum: Messages and
pacio Europeo de Educación Superior. Madrid:
meanings in the assessment of student learning.
Narcea.
Assessment and Evaluation in Higher Education,
23(4), 391-402. Bird, T. (1997). El portafolio del profesor: Un
ensayo sobre las posibilidades. In J. Millman
Snyder, B.R. (1971). The hidden curriculum.
& L. Darling Hammond (Eds.), Manual para la
Cambridge, MA: MIT Press.
evaluación del profesorado (pp. 332-351), Madrid:
Spillane, M.G. (1999, June). Portfolio assessment La Muralla.
in higher education: Seeking credibility on the
Bitter, G., & Pierson, M. (2005). Using technology
campus. Journal of the National Institute of the
in the classroom (6th ed.). Boston: Pearson.
Assessment of Experiential Learning, 17-28.
Brookheart, S.M. (2001) Successful students’
Williams, S.C., Davis, M.L., Metcalf, D., & Cov-
formative and summative uses of assessment in-
ington, V.M. (2003). The evolution of a process
formation. Assessment and Evaluation in Higher
portfolio as an assessment system in a teacher
Education, 8(2), 154-169.
education program. Current Issues in Education,
6(1). Retrieved October 28, 2007, from http://cie. Bullock, A.A., & Hawk, P. (2001). Developing
ed.asu.edu/volume6/number1/ a teaching portfolio: A guide for preservice and
practicing teachers. Upper Saddle River, NJ:
Prentice-Hall.

Campbell, D., Cignetti, P., Melenyzer, B., Nettles, Lyons, N. (Ed.). (1998). With portfolio in hand:
D., & Wyman, R. (1997). How to develop a profes- Validating the new teacher professionalism. New
sional portfolio: A manual for teachers. California York: Teachers College Press.
University of Pennsylvania.
Martin-Kniep, G. (1998). Why am I doing this?
Campbell, Melenyzer, Nettles, & Wyman (2000). Purposeful teaching through portfolio assess-
Portfolio and performance assessment in teacher ment. Portsmouth: Heinemann.
education. Boston: Allyn & Bacon.
McLaughlin, M., & Vogt, M. (1996). Portfolios
Cano, E. (2005). El portafolios del profesorado in teacher education. Newark, NJ: International
universitario. Un instrumento para la evaluación Reading Association.
del desarrollo profesional. Barcelona: Octaedro/
McLaughlin, M., Vogt, M. E., Anderson, J. A.,
ICE-UB.
DuMez, J., Peter, M. G., & Hunter, A. (1998).
Carless, D.M. (2002). The mini-viva as a tool to Professional portfolio models: Reflections across
enhance assessment for learning. Assessment the teaching profession. Norwood, MA: Christo-
and Evaluation in Higher Education, 27(4), pher-Gordon Publishers.
353-363.
Murphy, P. (2003). E-portfolios: Collections of stu-
Carney, J. (2005). What kind of electronic port- dent work move from paper to pixels. TLTC News,
folio research do we need? Retrieved October University of California. Retrieved October 28,
28, 2007, from it.wce.wwu.edu/carney/Presenta- 2007, from http://www.uctltc.org/news/2003/02/
tions/SITE05/ResearchWeNeed.pdf feature.html
Casado Ortiz, R. (2006). Convergencia con Eu- Pozuelos, F. J. (2003). La carpeta de trabajos. Una
ropa y cambio en la universidad. Los profesores propuesta para compartir la evaluación en el aula.
y las nuevas tecnologías como elementos clave Cooperación Educativa. Kikirikí. 71/72, 37-43.
en el nuevo modelo de aprendizaje del Espacio
Tosh, D., & Werdmuller, B. (2004). E-portfolios
Europeo de Educación Superior. Edutec. Revista
and Weblogs: One vision for ePortfolio develop-
Electrónica de Tecnología Educativa.
ment. Retrieved October 28, 2007, from http://
Castillo, S. (2004). Use y proporcione retroacción. eduspaces.net/dtosh/files/7371/16864/ePortfo-
In L.M. Villar (Coord.). (2004). Programa para lio_Weblog.pdf
la Mejora de la Docencia Universitaria. Madrid:
Vandervelde, J.M. (2004). A+ rubric: Rubric for
Pearson/Prentice Hall.
electronic portfolio. Retrieved October 28, 2007,
Greer, L. (2001). Does changing the method of from http://www.uwstout.edu/soe/profdev/eport-
assessment of a module improve the performance foliorubric.html
of a student? Assessment and Evaluation in Higher
Wiske, M.A. (2005). Teaching for understanding
Education, 26(2), 128-138.
with technology. San Francisco: Jossey-Bass.
Hebert, E. (2001). The power of portfolios: What
Yorke, M. (2001). Formative assessment and
children can teach us about learning and assess-
its relevance to retention. Higher Education
ment. Jossey-Bass.
Research and Development, 20(2), 115-126.

Chapter XVI
Formative Online Assessment
in E-Learning
Izaskun Ibabe
University of the Basque Country, Spain
Joana Jauregizar
Quality Evaluation and Certification Agency of the Basque University System, Spain
ABstrAct
This chapter provides an introduction to formative assessment, especially applied within an online or
e-learning environment. The characteristics of four strategies of online formative assessment currently
most widely used—online adaptive assessment, online self-assessment, online collaborative assessment,
and portfolio—are described. References are made throughout recent research about the effectiveness
of online formative assessment for optimizing students’ learning. A case study in which a computer-as-
sisted assessment tool was used to design and apply self-assessment exercises is presented. The chapter
emphasizes the idea that all type of assessment needs to be conceptualized as “assessment for learning.”
Practical advices are detailed for the planning, development, implementation, and review of quality
formative online assessment.
IntroductIon when there is alignment between what teachers

want to teach, how they teach, and how they
Assessment should be the first step in educational assess, teaching is likely to be more effective
design (Stiggins, 1987). A significant body of (Sluijsmans, Prins, & Martens, 2006).
research supports the view that the design of as- E-learning can be defined as the use of digital
sessment is critical in determining the direction technologies and media to deliver, support, and
of student effort, and e-learning is no exception enhance teaching, learning, assessment, and
to this (see Black & William, 1998). Furthermore, evaluation (Armitage & O’Leary, 2003). Never-
Formative Online Assessment in E-Learning
theless, it is not essential for the assessment of • To provide guidance, from a pedagogical
e-learning to be online, even if is often appropriate, point of view, on the design of questions
for example, when rapid feedback is required on or activities for formative assessment in
progress and achievement testing (Macdonald, e-learning environments, and explore a
2004). In any case, in this chapter, we focus on 28-item categorization in order to reveal
the formative power of online assessment. Forma- the broad potential of online assessment.
tive assessment is associated with considerable • To illustrate the four strategies of online
improvement in students’ performance, though formative assessment currently most widely
frequent testing and reporting of scores can be used—online adaptive assessment, online
prejudicial to weaker students. self-assessment, online collaborative assess-
Institutions are increasingly turning to infor- ment, and portfolio—which are not mutually
mation and communication technologies (ICTs) exclusive, and review recent research on
to plan their teaching, learning, and assessment their effectiveness for optimizing students’
tasks. In e-learning environments it is necessary learning.
to develop assessment models appropriate for the • To present a case study about an innovative
object of assessment and the different contexts teaching experience in which a computer-
involved. By comparison with the use of comput- assisted assessment tool (Hot Potatoes) was
ers to aid student learning, computer-assisted as- used to design and apply self-assessment
sessment (CAA) is a relatively new incorporation. exercises in higher education.
In the assessment process, it is vital to provide • To conclude by providing practical advice
a channel of communication between students for the planning, development, implementa-
and their mentors, and modern technologies offer tion, and review of quality formative online
many opportunities for innovation in educational assessment.
assessment, through rich new assessment tasks
and potentially powerful scoring, reporting,
and real-time feedback mechanisms for use by the concePt oF FormAtIve
teachers and students (Scalise & Gifford, 2006). Assessment
Computer-based platforms permit high-quality
formative assessments that can fit closely with Formative vs. Summative
instructional activities and goals, as well as con- Assessment
tributing to e-learning assessment.
Therefore, the purposes of this chapter are: There is considerable debate within the higher
educational community about assessment issues,
• To review the concept of “formative assess- given the proliferation of online classrooms and
ment” and its development over time, in rela- the emphasis on constructivist approaches to
tion to other concepts such as “summative learning. Constructivist learning paradigms are
assessment” or “assessment for learning.” learner-centred, and posit that learning occurs
Computer-provided feedback, as an essential when students are actively engaged in making
part of formative online assessment, will be sense of phenomena as well as constructing and
addressed, considering its advantages and negotiating meanings with others (for an exten-
offering practical advice for its design. sive review and analysis of this literature, see
• To clarify the current confusion of terms in Comeaux, 2002).
relation to “online assessment” and consider Recently, the Organization for Economic
its benefits, limitations, and effectiveness. Cooperation and Development (OECD) lent its
0
support to formative assessment as a powerful terms synonymously. The term “formative” itself
learning tool: is open to a variety of interpretations, and often
means no more than that assessment is carried
Teachers using formative assessment approaches out frequently and is planned at the same time as
guide students toward development of their own teaching. Such assessment does not necessarily
learning to learn skills that are increasingly nec- have all the characteristics just identified as help-
essary as knowledge is quickly outdated in the ing learning. It may be formative in helping the
information society. (OECD, 2005, p. 22) teacher to identify areas where more explanation
or practice is needed, but for the pupils, the marks
Many teachers and researchers seem to have or comments on their work, though indicating their
misunderstood the distinction between the terms degree of success or failure, may not tell them
“evaluation” and “assessment.” Rowntree (1982) how to make progress towards further learning
makes three points in relation to these terms: (Assessment Reform Group, 1999, p. 7).
In the literature on elaborate forms of forma-
1. Evaluation and assessment, though often tive assessment (i.e., “assessment for learning”)
used as synonyms, refer to different levels there emerged a tendency to highlight the positive
of investigation. effects of formative assessment and the negative
2. Evaluation is concerned with the macro or effects of summative assessment. The general
holistic level of the learning event, taking disparagement of summative assessment is per-
into account the context of learning and all haps understandable given what Biggs (1998)
the factors that go with it, while assessment described as its negative “backwash” effects.
can be seen as the measurement of student “Backwash effect” is a vicious cycle referring to
learning, and is one of the elements involved the influence of testing on teaching and learning,
in evaluation, at the micro level. the result of high-stake examinations. People try
3. One aspect of any sound evaluation is al- to minimize the effort required in their activities
lowance for the unexpected. Above all, an by adjusting their working process (learning) to
evaluation is a designed and purposeful maximize the outcome (examination grades, ease
enquiry that is open to discussion. of passing the exam, etc.), rather than focusing
on the quality or intrinsic interest of their work
The concept of formative assessment has (learning). Biggs (1998) suggests that backwash
changed over time. Scriven (1967, pp. 40-43) can have a positive result when assessment tasks
used the terms “formative” and “summative” are deliberately and firmly referenced to learning
evaluation to differentiate between the two roles standards contained in the curriculum.
evaluation may play in education. Using evalua- The distinction between formative and sum-
tion in the development or improvement of some mative assessment has resulted in some excellent
educational process is “formative.” Using evalu- research and development work on formative
ation in decision-making about the end result of assessment. Nowadays, some researchers tend
an educational process is “summative.” Scriven’s towards a more inclusive model of assessment.
(1967) argument was focused on different func- For example, Kennedy, Sang, Wai-ming, and Fok
tions for curriculum evaluation, rather than on (2006) point out that a model of assessment should
the assessment of student learning. have the following characteristics:
The Assessment Reform Group distinguished
between “formative assessment” and “assessment 1. All assessment needs to be conceptualized
for learning,” although some authors used the as “assessment for learning.”

2. Feedback needs to be seen as a key function Verification is the simple judgment as to whether
for all forms of assessment. an answer is correct or incorrect, while elaboration
3. Teachers need to be seen as playing an im- is the informational component providing relevant
portant role not only in relation to formative cues for guiding the learner toward a correct an-
assessment but in all forms of summative swer. Most researchers now share the view that
assessment as well—both internal and ex- successful feedback (feedback that facilitates
ternal. the greatest gains in learning) must include both
4. Decisions about assessment need to be verification and elaboration. However, there are
viewed in a social context, since in the end also considerable differences in types of elabora-
they need to be acceptable to the commu- tion (Mason & Brunning, 2000):
nity.
a. Informational elaboration provides a frame-
At the heart of formative assessment is the work of relevant information from which the
feedback provided to students by the teacher. The correct answer can be drawn.
iterative nature of evaluation should help to make b. Topic-specific elaboration provides more
the learning experience both more efficient and specific information about the target ques-
more effective, as the feedback is used to produce tion or topic and leads the learner through
continuous improvement (Crompton, 1999). the correct answer, though it does not address
incorrect responses.
Computer-Provided Feedback c. Response-specific elaboration addresses
both the correct answer and incorrect
In general terms, feedback is any message response choices; if a learner selects an
generated in response to a learner’s action. As incorrect response, response-specific feed-
explained before, feedback is an essential part of back explains why the selected response is
formative assessment, and “formative feedback” incorrect and provides information about
would constitute additional information used by what the correct answer should be.
students to improve their performance, insofar as
it provides hints and tips, points to the resources Computer-provided feedback has several
needed, reinforces the learning of concepts, and advantages, such as the possibility to repeat
provides information on the following steps to feedback as many times as students want, and the
be taken (Schulze & O’Keefe, 2002). Students unbiased, accurate, and nonjudgemental informa-
must understand their learning goal and be able tion provided by the computer, irrespective of
to compare their current performance with their student characteristics or the nature of the student
desired performance; they must also have the abil- response (Mason & Brunning, 2000).
ity to act in such a way as to close the gap (Gipps, Designers of computer-provided feedback
1994; Sadler, 1998), so that the informational should consider many aspects, including: when
feedback becomes intrinsically motivating for and where to provide feedback, how much feed-
them (Covington, 1992; Pintrich & Schrauben, back to give, what kind of feedback is needed
1992). Teaching students to monitor their own and the level of feedback. Schulze and O’Keefe
performance is the ultimate goal of providing (2002) make interesting recommendations about
feedback (Sadler, 1989). these aspects. With regard to when to provide
According to Kulhavy and Stock (1989), ef- feedback, if it is given at the end of the exercise,
fective feedback provides the learner with two the assessment should be short, because nothing
types of information: verification and elaboration. is more frustrating than completing a long activity

only to find out at the end that everything was or “Web-based assessment,” and such terms are
done incorrectly. indeed often synonymously, even though they do
When the exercises are long, feedback should not all mean the same. Recently, the term “e-as-
be given throughout the assessment, so that stu- sessment” has been used to mean assessment in e-
dents can identify their errors and achievements. learning environments or the (electronic) process
This is directly related to the levels of feedback. by which the learner’s progress and understanding
The concept of feedback levels comes into play is assessed (BECTA, 2006), although the term
when learners attempt part of an activity once has more frequently been applied to the projects
and obtain general feedback that they are not of great scale where abilities, competences, ap-
quite correct and should keep working; they then titudes, and personality are assessed.
receive more and more specific feedback until Computer-based assessment and computer-
they complete the exercise correctly. In complex aided assessment are the most widely used terms,
exercises, learners may not need to be told the and refer to the use of a computer for viewing items
correct strategy at their first attempt, and feedback and responding to them. The software managing
should be given gradually. the assessment may be on the individual computer
However, in less complex exercises, such as or on one connected over a network, local or oth-
multiple-choice tests, learners benefit more from erwise. When the software access comes from a
clear and detailed feedback about their answers network, with current assessment solutions often
at every attempt. As for the kind of feedback in being delivered from a local or distant server,
terms of marks, Black and William (1998) sum- the proper term is online assessment. Web-based
marized research evidence from 250 articles and assessments are online assessments that can be
chapters and noted that: delivered over the World Wide Web or a local
area network. This type of assessment permits
Feedback has been shown to improve learning instructors to receive feedback (answers, results,
when it gives each pupil specific guidance on errors, or comments) from students.
strengths and weaknesses, preferably without any
overall marks. Thus, the way in which test results advantages and Disadvantages of
are reported to pupils so that they can identify online Assessment
their own strengths and weaknesses is critical.
(Black & William, 1998, p. 144) We can identify advantages and disadvantages
of online assessment for the institution, for the
In terms of where to place feedback, designers students and for the teachers. The most important
should take into account where the learner may disadvantage for the institution would be the “cost”
struggle and where the critical “chunks” of content of setting up the system at the beginning, though
are, and then provide feedback in those places. this would be compensated by the subsequent
savings of time and money, and reduction of the
administrative burden if the system is properly
onLIne Assessment used.
The advantages for students are numerous
Definition (Collis, De-Boer, & Slotman, 2001; Lowry, 2005;
Plous, 2000; Sherman, 1998; Ward & Newlands,
There is some confusion among similar terms, 1998): immediate feedback, guided effort, diagno-
such as “computer-based assessment,” “com- sis of problems in learning, a more flexible pace
puter-aided assessment,” “online assessment,” of learning, reaching and motivating a large and

diverse set of respondents, gaining experience every student, a task that would otherwise be too
in assessment methods, freedom from restric- arduous. The time saving advantages of electronic
tions of time and place of assessment, and so on. marking are unquestionable (a wide range of
The advantages increase if online assessment is topics and large groups can be assessed quickly,
adapted to the students’ ability (adaptive online and results can be entered automatically into an
assessment), or if it is adapted to student learning administration system so that students receive
styles (Clariana, 1997). Online assessment, used their marks rapidly), but these advantages need
as self-assessment, can help students monitor to be offset against the time invested in writing
their own progress, making it an important tool challenging and effective questions, meaningful
of formative assessment (Ibabe & Jauregizar, feedback and structuring appropriate tests (Bull,
2005). 1999).
Students should be trained to become ac- Although, as James, McInnis, and Devlin
customed to the online assessment tool, so that (2002, p. 24) point out, the design of online exami-
the assessment methodology does not obstruct nations is likely to require more time and effort
performance. Stress or anxiety caused by inex- than conventional pen and paper examinations,
perience in a computer-based system may be a these authors also recognized that computers
disadvantage of online assessment. However, offer the potential to present students with more
research comparing performance using computer complex scenarios through the use of interactive
and paper-based multiple-choice tests (Lee & resources (images, sound, or simulation).
Weerakon, 2001) has demonstrated that there is Some authors have expressed their fear about
no measurable effect. Even so, Zakrzewski and the “superficial” type learning that online assess-
Bull (1999) suggest that student anxiety can be ment can generate (Ryan, 2000). The concern
reduced if they take formative assessment before is that online assessment would be designed
summative tests. for assessment tasks only involving memoriza-
Teachers should also be trained to master tion and recall. Indeed, using the technology
software so as to enable efficient delivery of the for assessment involving higher-level cognitive
assessment, which requires a “cultural shift” to skills, including the application of analysis and
invest time in designing new assessments rather synthesis, is a great challenge (Hyde, Booth, &
than in traditional “marking” assessments (Bull, Wilson, 2003), but work is already in progress
1999). In any case, universities are facing an im- on the development of these kind of assessment
portant “academic shift” with the development exercises in the online context.
of the European Higher Education Area, and the
use of ICTs will be crucial in adapting to that Effectiveness of online Assessment
challenge.
As Macdonald (2004) notes, online feedback Many studies have indicated that integrating the
can be given not only to individuals, but also e-learning environment with online assessment
to a whole tutorial group, forming the basis for has positive results (Buchanan, 2000; Henly, 2003;
online collaborative assessment. Moreover, com- Velan, Killen, Dziegielewski, & Kumar, 2002).
puter-based assessment provides focus and timely Buchanan (2000) showed that a Web-based forma-
feedback not only to students, but also to teach- tive assessment strategy is able to improve student
ers, who can identify the gaps in their students’ learning interest and student scores. He argued
knowledge or the questions that have not been that the “repeat the test” strategy (giving more
adequately understood in class. Thus, teachers opportunities for becoming familiar with learning
can give constructive and detailed feedback to materials) is an important element in Web-based

formative assessment design. However, he noted 6. “All pass and then reward”: This strategy
that this strategy should be implemented in con- rewards students with a Flash animation
junction with the functions of “provide with no when they pass all the test items.
answer” (pushing them to make clear what they
did not understand) and “instant feedback,” so
that the Web-based formative assessment is more desIgn oF ActIvItIes For
beneficial. For such feedback to be effective it FormAtIve onLIne Assessment
needs to be provided early in the learning process
(Brown & Knight, 1994) and to offer guidance The factors that influence the choice and design of
for improving performance (William & Black, online assessment methods include the learners’
1996, p. 543). needs, their access to technology, the available
In the study by Wang, Wang, Wang, and resources and, to some extent, the discipline or
Huang (2006), performance in the multiple- industry area. Currently, technology offers many
choice Web-based formative assessment group new opportunities for innovation in educational
with six strategies, was significantly better than assessment through rich new assessment tasks
those in both the partial Web-based formative and potentially powerful scoring, reporting,
assessment strategy group and the Paper and and real-time feedback mechanisms. Through
Pencil Test group. This finding suggests that the these and other technological innovations, the
more the different formative assessment strategies computer-based platform offers the potential
incorporated in the e-learning environment, the for high quality formative assessment that can
greater the learning effect obtained by students. closely match instructional activities and goals,
The Web-based formative assessment condition and make meaningful contributions to e-learning
contained six strategies: or summative tests.
It is important to examine what is to be learned
1. “Repeat the test”: This strategy allows and assessed, in order to identify appropriate
students to take the same test item repeatedly methods for demonstrating these skills. A potential
if they make errors on it. However, if they limitation of computer-based assessment resides
pass one test item correctly three times, then in the design of questions and tasks with which
the item will be deleted automatically. computers can effectively interact, including
2. “Provide with no answer”: This strategy scoring and score reporting.
shows students incorrect answers they
made without offering the correct answer. recommendations for creating an
However, it also allows students leave the online Assessment tool
module to find correct answers in their own
way. Davis and Morrow (2004) suggest five questions
3. “Ask questions”: This strategy allows for creating an assessment tool:
students to send questions to the teacher by
e-mail. 1. What is it we want to measure?
4. “Query scores”: This strategy provides an We need to broadly define what it is we
interface for students to make queries about want to measure. Specifically, we need to
peer and personal scores. identify the construct of interest. Most often,
5. “Monitor answering history”: This strat- the topic of interest will fall into one of two
egy provides an interface for students to categories:
check their personal answer history for each
item.

• A type of cognitive achievement, either on an instrument, or performing a science

a knowledge or skill (e.g., maths skills or experiment in front of a group of raters. The
knowledge of Spanish history). use of computers in test administration has
• A type of affective trait (e.g., motivation, led to the development of adaptive tests, as
interest in maths). well as alternative item types with audio or
The concept of achievement can be broken video files, or items that allow for Internet
down into knowledge and skills. Tests of searches.
knowledge measure an individual’s under- 4. Who will be taking the test?
standing and comprehension of facts, con- Defining the target population of a test is
cepts, and principles. Tests of skills involve extremely important at the outset of instru-
testing the application of this knowledge to ment development, and for several reasons.
problems or situations (Haladyna, 1997). An If a test is made available for public use, the
example of this distinction would be: intended set of respondents must be identi-
• Knowledge item: “What is the difference fied. Just as you would not give medicine
between a median and a mean?” for an adult to a young child, it is wrong
• Skill item: “Given the set of test scores of to give a test designed for one population
55, 89, 74, 68, 92, 73, 85, and 66, compute to members of another. Examples of well-
the mean.” defined populations would be:
2. Why are we developing an instrument? • An assessment of Spanish history knowledge
There are several reasons why instruments designed for college students.
measuring achievement are created: • An assessment of self-confidence for high-
• To assess learning from a particular course school seniors.
or subject area. 5. What are the conditions of measurement?
• To assess the effectiveness or outcome of a We need to describe how the test will be
program. used. It is important to consider the time
• To assess level of student knowledge in and effort involved in developing and scor-
relation to a particular competence. ing the test. With online testing there arise
In addition, the primary reason for the other problems, such as security, cheating,
development of an instrument may be its time limits and examinee anxiety. Thus, it
use as a tool for formative assessment. It is is essential to emphasize the need to use a
important to define the instrument’s purpose range of methods for collecting evidence of
in order to justify the time and effort that valid assessment.
will be put into the process.
3. How do we want to measure this con- type of Questions and tasks for
struct? e-Learning Assessment
The commonest type of instrument is a
selected response format. This format is rela- Formative assessment refers to those activities
tively easy to administer and easy to score. that are used to help students learn. These types
Despite the benefits of the selected response of activities include short tests and quizzes, ques-
format, many researchers are exploring the tion, and answers in the lesson, assignments,
option of performance assessment. Some homework, and so on. Questions, tasks, activities,
common examples of performance assess- and other methods of eliciting student responses
ments include having students responding are often called “items” in the assessment process.
to an essay prompt, playing a piece of music One organizational scheme describes innovative

features for computer-administered items, such different media. Thus, by combining intermediate
as the technological enhancements of sound, constraint types and varying the response and me-
graphics, animation, video, or other new media dia inclusion, e-learning instructional designers
incorporated into the item stem, response options, can create a vast array of innovation assessment
or both (Parshall, Davey, & Pashley, 2000). For approaches and can arguably match assessment
some innovative formats, students can, for in- needs and evidence for many instructional design
stance, click on graphics, drag or move objects, objectives. The need to integrate the learning and
re-order a series of statements or pictures, or assessment is considered critical (Hyde et al.,
construct a graph or other representation. In- 2003), as is the assessment of complex skills.
novation may involve not just a single item, but
the way items flow, as in the case of branching
through a changing series of items contingent on strAtegIes For FormAtIve
an examinee’s responses. onLIne Assessment
The question type currently dominating large-
scale computer-based testing and many e-learning A range of assessment methods and tools are
assessments is the standard multiple-choice ques- being used in the online environment, and a
tion, which generally includes a prompt followed body of literature describing these uses is being
by a small set of responses from which students developed. Below, four strategies of formative
are expected to select the best choice. online assessment are described, which are not
Scalise and Gifford (2006) present a taxonomy mutually exclusive, especially in the case, for
or categorization of 28 innovative item types that example, of online adaptive assessment and self-
may be useful in computer-based assessment. assessment. For a complete review, see Booth and
They reviewed 44 papers and book chapters on Berwyn (2003).
item types and item designs—many of them classic
references regarding particular item types—with online self-Assessment with
the intention of consolidating considerations of Feedback
item constraint for use in e-learning assessment
designs. Organized according to the degree of Self-assessment can enrich the learning process,
constraint on the respondent’s options for an- as it helps students to self-monitor their learning,
swering or interacting with the assessment item increases their strategic knowledge of how to go
or task, the proposed taxonomy (shown in Table about improving, and improves their motivation.
1) describes a set of iconic item types termed Using self-assessment information requires con-
“intermediate constraint” items. trol over one’s cognitive activities or metacogni-
Table 1 shows taxonomy based on the level of tion: students must understand what strategies
constraint in the item/task response format. The and skills they should use in each task, and know
most constrained item types, at left in column when and how to use them (Brookhart, 2001); the
1, use fully selected response formats. The least end result is more autonomous and self-directed
constrained item types, at right in column 7, use learners. Furthermore, self-assessment puts the
fully constructed response formats. In between, learner in control of the instruction provided, and
there are “intermediate constraint items,” which information that is irrelevant to students can be
are organized with decreasing degrees of con- eliminated from their learning experience (Quinn
straint from left to right. & Reid, 2003).
All item types in the item taxonomy can While self-assessment has typically been used
involve new response actions and inclusion of after a learning activity as a review, it has an

Scalise & Gifford, 2006)
More constrained Less constrained
Less
3. Reordering/ 7. Portfolio/
complex 1. Multiple Choice 2. Selection/ Identification 4. Correction/ Substitution 5. Completion 6. Construction
Rearrangement Presentation
5.A. 6.A. Open-
1.A. 2.A. 3.A. 4.A. 7.A.
Single Numerical Ended Multiple
True/False Multiple True/ False Matching Interlinear Project
Constructed Choice
5.B. 6.B. 7.B.
1.B. 2.B. 3.B. 4.B. Short-Answer & Figural Demonstration,
Alternative Choice Yes/no with Explanation Categorizing Sore-Finger Sentence Constructed Experiment,
Completion Response Performance
1.C.
3.C. 7.C.
Conventional or 2.C. 4.C. 5.C. 6.C.
Ranking & Se- Discussion,
Standard Multiple Multiple Answer Limited Figural Drawing Cloze-Procedure Concept Map
quencing Interview
Choice
1.D.
5.D. 6.D. 7.D.
Multiple Choice 2.D. 3.D. 4.D.
Matrix Essay & Auto- Diagnosis, Teach-
with Media Complex Multiple Choice Assembling Proof Bug/Fault Correction
Completion mated Editing ing
More Distractors
complex
Table 1. Intermediate constraint taxonomy for e-learning assessment questions and tasks (adapted from
important role as a prelude to a specific area of student’s performance (“testing on demand”). As

study (Challis, 2005). Furthermore, if we add the students answer the items correctly, the complex-
potential of Internet to self-assessment benefits, ity of the exercises grows, so that the level of
the result will be a highly relevant diagnostic and difficulty is constantly revised. As Challis (2005)
assessment tool. points out, the score is derived not from the number
With regard to online self-assessment marks, of correct answers, but rather from the difficulty
Taras (2001) has constantly stressed the im- level of the questions answered correctly.
portance of no grading self-assessment, since The difference between online adaptive assess-
grades can distract and “block” the students. “By ment and other online assessment tools is relevant,
desisting from giving the students grades until since adaptive assessment is more dynamic and
the students have carried out self-assessment, more “student centred” than other assessments.
they are being encouraged to focus on their work Adaptive online testing can be used as a diagnos-
with as little emotional interference as possible” tic tool that contributes to meaningful learning,
(Taras, 2001, p. 609). Students should be free to and thus, come to constitute an important type of
explore their knowledge and areas of weakness formative assessment. Futhermore, adaptive test-
and to make mistakes without fear of this having ing makes student cheating less likely, because,
consequences for their final marks. as items change depending on the answers, it is
In any case, online self-assessment results are quite difficult to copy an exercise and pass it to
interesting for teachers, because they can identify other students for them to memorize the correct
students at risk and monitor student performance answers.
early in the course, so that the appropriate re- A significant advantage for self- or group test-
sources for helping them can be recommended. ing is that students can specify from the outset
Many studies note the importance of relevant the level at which they wish to work (for example,
feedback and self-assessment for efficient learn- entry, intermediate, or advanced), and change
ing (Black & William, 1998; Brookhart, 2001; level, with less need for repetitive drill before
Dearing, 1997; Taras, 2003). Feedback and self- students can be confident they have the requisite
assessment should be part of the same process. level of ability. Online adaptive assessment can
Taras (2003) confirmed that self-assessment reduce testing time by more than 50% while
without tutor feedback cannot help students to be maintaining the same level of reliability. Shorter
aware of their errors, and that therefore they can- testing times also reduce fatigue, a factor that
not understand their cause. Taras’ (2003) version can significantly affect an examinee’s test results
of self-assessment has two distinctive features: (Rudner, 1998).
minimal and integrated tutor feedback (i.e., ac- Another advantage is that examinees at a low
cording to the learning needs of the student), and achievement level are not required to respond to
feedback provided before a grade. items that are very difficult and far beyond their
level, thus reducing potential negative psychologi-
online adaptive assessment cal effects (e.g., examinees becoming despondent
or more test-anxious). Similarly, examinees at a
Online adaptive assessment is an innovative on- high achievement level are not required to answer
line form of assessment in which an examinee is a number of items that are much too simple for
presented items in a sequence dependent on the them, thus reducing the potential for boredom
correctness of the response to the previous item. in that group of examinees (Computer Adaptive
Therefore, adaptive assessment is characterized Assessment Project, 2005).
by the adaptation of the items depending on the

Despite the advantages described, computer The research by McConnell (2002) shows that
adaptive tests also have limitations, such as pos- a positive social climate is necessary in developing
sible inequities because each examinee receives and sustaining collaborative assessment, and that
a different set of questions, or the impossibil- this form of assessment helps students to reduce
ity (nearly always) of going back and changing dependence on lecturers as the only or major
the answers. Rudner (1998) also points out that source of judgment about the quality of learn-
students can cheat to obtain a higher score: a ing. Students develop skill and know-how about
clever examinee could intentionally miss early self- and peer assessment and see themselves as
questions, so that the program assumed low abil- competent in making judgments about their own
ity and selected a series of easy questions. The and each other’s work; this approach also creates a
examinee could then go back and change the certain mindfulness and reflection among students
answers, getting them all right. The result could (Garrison, 2003; Hiltz, 1994; Poole, 2000), which
be 100% correct answers, which would result in are certainly good lifelong learning skills.
the examinee’s estimated ability being the high- Sluijsmans et al. (2006) presented three argu-
est ability level. ments for the implementation of peer assessment
in e-learning. First, students can play a role in the
online Collaborative assessment choice of performance assessment tasks and in
discussing assessment. Second, the student shares
Collaborative learning and assessment can be responsibility and collaborates in a continuous
developed effectively in the online environment. dialogue with the teacher. Third, peer assessment
This means that students can also participate in can decrease the workload of teachers. Peer review
assessment through peer review. Using collabora- is a demanding task for undergraduates, because
tive assessment, learners employ online resources they need the confidence firstly to judge fellow
to work collaboratively on a student’s course work students’ work, and secondly to be able to criticize
assessment. An added benefit of this technique without giving offence (Macdonald, 2004).
is the development of communication and team-
building skills highly valued by instructors. Portfolio
This assessment strategy involves the student,
their peers, and their tutor in thoughtful and criti- Portfolio assessment is defined as any method
cal examination of each student’s course work. by which a student’s work is stored over time so
Students can learn to comment on each other’s that it can be reviewed in relation to both process
work as an integral part of summative assess- and product (Knight, 1994). Portfolios contain a
ment, and this has been shown to be an effective sample of various types of students’ work that can
approach when helping them to develop a critical identify how academically successful they are,
approach to their own written work (see, for exam- and that can allow them to share their work with
ple, Boud & Falchikov, 1989). Their performance peers. As it does not focus exclusively on solu-
will determine whether, as e-learners, they have tions, but also on the intermediate steps and draft
internalized the new skills. In terms of trainer products involved in the performance of the task,
collaboration on assessment, some Web-based the portfolio approach teaches students the value
survey tools (e.g., Zoomerang, SurveyMonkey, of self-assessment and builds self-esteem, help-
and SurveyShare) have incorporated collaborative ing them engage in meaningful learning (Chang,
features to share survey templates, questions, and 2002; ITC, 2003; Reeves, 2000). The portfolio
results (Curtis, 2002). provides authentic assessment opportunities for
reflection and a context in which students actively
0
participate in the evaluation process (Banta, 2003). access to evidence of professional or generic
Therefore, portfolio is more than a type of as- graduate skills (Cooper, 1999; Cooper & Love,
sessment, implying a new consideration of the 2000, 2001a, 2002).
teaching process; most importantly, the portfolio Online portfolios (the process of presenting via
involves a process, rather than a final aim (Agra, Web digital evidence of progress of achievement)
Gewerc, & Montero, 2003). have the added advantage of the interactivity
Trudi Cooper suggests six steps in a portfolio- provided by the Web, and the easier possibility
building process (Cooper, 1997; Cooper & Emden, of organization and updating of the material.
2000; Cooper, Hutchins, & Sims, 1999): Used in conjunction with appropriate software
solutions, online portfolio-based assessment
1. To identify the areas of skills that the student can relieve teachers of some of the more tedious
should develop. aspects of assessment and permit parts of the
2. Taking into account these skill areas, to assessment process to be automated (Cooper &
develop specific learning outcomes to be Love, 2001b).
achieved by the students. As Agra et al. (2003) describe in their experi-
3. To identify appropriate learning strategies ence of implementation of online portfolio in a
so that students can achieve their learning postgraduate degree, students’ portfolios were
outcomes. accessible online for tutors and peers, so that
4. To identify performance indicators that teachers could view students’ process and give
establish whether students have achieved feedback to them, and students could also inter-
their learning outcomes and indicate the change ideas fluidly with their peers.
evidence the students need to collect.
5. To collect evidence that demonstrates the
students have met the performance indica- A cAse study:
tors. seLF-Assessment And
6. To organize this evidence in a portfolio so LeArnIng
that teachers can easily understand how
the evidence relates to each performance This case study illustrates an innovative teach-
indicator. ing experience in the Psychology Faculty at the
University of the Basque Country. In this study a
The benefits of portfolio-based assessment computer-assisted assessment tool (Hot Potatoes)
over other assessment approaches have been well was used to design and apply self-assessment
established (see, for example, Biggs & Tang, 1997; exercises that are automatically corrected online
Brooks & Madda, 1999; Cooper, 1999; Hutchins, (Ibabe, Gómez, & Jauregizar, 2006). Learner
Sims, & Cooper, 1999). Love and Cooper (2004) satisfaction and learning perception was evalu-
point out the main advantages of this tool, such ated. The main aim of this project was to verify
as its capacity to contain many different types of whether interactive self-assessment improved
evidence and from different sources, the active university students’ academic results on their
involvement of students in their processes, equity, Data Analysis course.
and moderation in the assessment process and The procedure employed was as follows. First
its suitability to assessment in lifelong learning of all, we acquired additional service of TexToys
contexts. Moreover, portfolios provide a means Creative Technology program (http://www.hot-
for students to learn to manage their own profes- potatoes.net/help/lw.php) in order to record the
sional development, since they offer them easy results of the Hot Potatoes assessment. Next, we

Figure 1. Students’ final marks depending on the use or nonuse of the self-assessment tool
40
users
non-users
30
% of students
20
10
0
d c B A A+
student's final marks
designed self-assessment exercises (multiple- agreed” or “totally agreed” with the statement,
choice, short-answer questions, fill-in the blanks The exercises were useful for understanding and
exercises, crosswords, etc.) in HTML format. processing the information. In response to the
Twenty items were designed for each unit of the item, In my opinion, self-assessment exercises can
teaching program (100 items in total). Further- be a useful complementary tool for learning, 80%
more, each unit included some revision exercises “agreed” or “totally agreed.” The mean of all of
and a final exercise including content from all the items, on a Likert scale from 1 to 5, was 3.8.
units. It was explained to students that using the The results suggest that students who use
tool would not mean they obtained higher final interactive self-assessment exercises as a comple-
marks. The self-assessment items were published mentary study tool obtain better final marks. There
on the Internet as each unit was finished in class. is a positive correlation between the frequency
At the end of the semester, students completed with which students use these exercises and
a questionnaire about their satisfaction with the academic results, r (81) = .24, p < .05. Obviously,
self-assessment tool, and their learning percep- the correlation between the number of exercises
tion. Finally, summative assessment marks were done and the final mark was also positive, r (81)
awarded. = .25, p < .05. In other words, students who do
A high proportion of students used the self- more self-assessment exercises obtain higher
assessment tool (46% of all students registered), final marks.
considering that the exercises were voluntary, Figure 1 was designed to show the kind of stu-
with no extra incentive, outside of the normal dent that makes use of self-assessment exercises.
timetable and with the requirement of an Inter- The results indicate that the majority of students
net connection. Results show an acceptable rate with higher marks (B, A, or A+) had used self-
of satisfaction among students who used Hot assessment tool.
Potatoes. Sixty-six percent of students “agreed” In short, interactive self-assessment exer-
or “totally agreed” with the statement, The tool cises can act as formative assessment tools for
was useful for revising the content explained in improving students’ learning process and learn-
the classroom. Similarly, 66% of the learners ing satisfaction, thus increasing the quality of
education.

assessment tasks.
concLusIon: key PoInts For 3. Implementation stage
FormAtIve onLIne Assessment • State the assessment criteria in advance to
students; be clear about the criteria used for
On the basis of the research, and by way of conclu- the assessment.
sion, listed below are key points to be taken into • Use pretesting to alleviate student anxiety
account when planning, developing, implement- about ICTs and take account of the access
ing, and reviewing formative online assessment to technology available to students.
of quality (Bull, 1999; Challis, 2005; Hyde et al., • Help eliminate cheating by devising ways
2003; Kendle & Northcote, 2000; Macdonald, of knowing about learners’ abilities and by
2004; Stephens, 2001). gathering a range of evidence of compe-
tence.
1. Planning stage • For ethical reasons, students should be
• Assessment must be clearly related to the aware of how their assessment results will
aims and objectives of the subject. be used.
• Plan in advance the competence to be as- • Try to develop learner-centred assessment,
sessed and the way students can demonstrate using strategies such as self-assessment,
their performance (evidence of achieve- adaptive assessment or collaborative assess-
ment). ment, and allowing students to participate
• Keep in mind that the technology available more in the assessment process.
should not determine the methods used: 4. Review stage
Online assessment should be pedagogically- • Share resources with other experts to help
led, not technology-driven. enhance one’s own materials.
• Make assessment part of the online learn- • Keep up to date with the constantly changing
ing process. The learning strategies and technology.
assessment strategies should be developed • Review and evaluate the assessment strate-
simultaneously. gies used, the evidence collected and judg-
2. Development stage ments of other assessors in order to validate
• Variety: Include a range of methods for col- assessment.
lecting evidence of competence (for example,
online portfolios).
• Authenticity: Use open-ended tasks that Future reseArch dIrectIons
simulate workplace tasks, in order to assess
competence development. Online assessment is seen by many as useful for
• Collaboration: Allow interaction between assessing lower order skills, such as the recall
learners and others, and use appropriate of knowledge, while being not well equipped to
communication technologies. assess higher order skills, such as the ability to
• Feedback: Ensure appropriate feedback apply knowledge in new situations or to evaluate
mechanisms are possible using peer feed- and synthesize information (Ashton, Beevers, Mil-
back and peer tutoring. ligan, Schofield, Thomas & Youngson, 2006).
• Learner responsibility: Provide options The future trends in formative online assess-
and opportunities for accountability within ment research should be directed to the assessment
of complex skills. ICTs are often used to simulate

the context of professional practice in education, as a competency teaching method that arouses
but we know little about how to involve the assess- interest and reflective processes, activates prior
ment of competences (complex skills with their knowledge, clarifies meanings, and provides
underlying knowledge structures and attitudes, information about learners’ progress. Online
van Merriënboer, 1997; for example, designing collaborative assessment can model appropriate
a house in the case of an architect) in e-learning. learning strategies and create online communi-
Currently, institutions of higher education are ties of learners. For instance, online portfolios
confronted with a demand for competence-based could be incorporated to measure complex and
learning (CBL), which is expected to narrow the transferable skills. Anyway, further research
gap between learning in the educational setting is required to better understand the nature of
and future workplace performance (Bastiaens & competency-based performance assessment in
Martens, 2000). e-learning and the strategies and tools needed to
The assessment task is described in terms assess the learners’ competency.
of a certain performance that is perceived as
worthwhile and relevant to the learner, and can
therefore be defined as performance assessment reFerences
(Wiggins, 1989). Performance assessment focuses
on the ability to use combinations of acquired Agra, M. J., Gewerc, A., & Montero, M. L. (2003).
skills and knowledge, and therefore fits in well El portafolios como herramienta de análisis en
with the theory of powerful learning environments experiencias de formación online y presenciales.
(Linn, Baker, & Dunbar, 1991). Because the goals Enseñanza, 23, 101-114.
as well as the methods of instruction are oriented
Armitage, S., & O’Leary, R. (2003). A guide for
towards integrated and complex curricular objec-
learning technologists. Learning and teaching
tives, it is necessary for assessment practices to
support network. York, UK: LTSN Generic
reflect this complexity and to use various kinds of
Center.
assessments in which learners have to interpret,
analyze, and evaluate problems and explain their Ashton, H.S., Beevers, C. E., Milligan, C. D.,
arguments. In CBL, it is important that a number of Schofield, D. K., Thomas, R. C., & Youngson, M.
performance assessments are organized to gather A., (2006). Moving beyond objective testing in
reliable and valid information about a learner’s online assessment. In S.L. Howell & M. Hricko
competence development (Sluijsmans et al., 2006). (Eds.), Online assessment & measurement. Case
Thus, the learner is required to perform similar studies from higher education, K-12 and Corpo-
types of tasks in a variety of situations under the rate (pp. 116-128). Hershey, PA: Idea Group.
same conditions.
Assessment Reform Group. (1999). Assessment
Many efforts have been made to implement
for learning: Beyond the black box. Cambridge,
CBL in face-to-face education, but an electronic
UK: University of Cambridge School of Educa-
learning environment represents a still greater
tion. Retrieved October 29, 2007, from http://arg.
challenge. The integrated simulations can be
educ.cam.ac.uk/AssessInsides.pdf
used to provide answer mechanism, feedback, or
different forms of assessment, and students can Banta, T. W. (Ed.). (2003). Portfolio assessment:
be assessed in the same environment in which Uses, cases, scores and impact. San Francisco:
they learn. Measuring competencies requires Jossey-Bass.
the implementation of new and innovative pro-
cesses. Online formative assessment can be used

Bastiaens, T., & Martens, R. (2000). Conditions Bull, J. (1999). Computer-assisted assessment:
for Web-based learning with real events. In B. Impact on higher-education institutions. Educa-
Abbey (Ed.), Instructional and cognitive impacts tional Technology & Society, 2(3), 123-126.
of web-based education (pp. 1-32). London: Idea
Challis, D. (2005). Committing to quality learn-
Group Publishing.
ing through adaptive online assessment. Assess-
BECTA. (2006). Retrieved October 29, 2007, from ment & Evaluation in Higher Education, 30(5),
www.becta.org.uk 519-527.
Biggs, J. (1998). Assessment and classroom learn- Chang, C. C. (2002, March). Building a Web-based
ing: A role for summative assessment? Assessment learning portfolio for authentic assessment. Paper
in Education, 5(1), 103-110. presented at Proceedings International Confer-
ence on Computers in Education (ICCE’02),
Biggs, J., & Tang, C. (1997). Assessment by
Melbourne, Australia.
portfolio: Constructing learning and designing
teaching. Research and Development in Higher Clariana, R. B. (1997). Considering learning style
Education, 79-87. in computer-assisted learning. British Journal of
Black, P., & William, D. (1998). Assessment and
classroom learning, Assessment in Education, Collis, B., De-Boer, W., & Slotman, K. (2001).
5(1), 7-74. Feedback for Web-based assignments. Journal of
Computer Assisted Learning, 17, 306-313.
Booth, R., & Berwyn, C. (2003). The develop-
ment of quality online assessment in vocational Comeaux, P. (Ed). (2002). Communication and
education and training. Leabrook, Australia: collaboration in the online classroom: Examples
Australian Flexible Learning Framework. Re- and applications. Bolton, MA: Anker.
trieved October 29, 2007, from www.ncver.edu.
Computer Adaptive Assessment Project. (2005).
au/research/proj/nr1F02_1.pdf
What is CAA? Retrieved October 29, 2007, from
Boud, D., & Falchikov, N. (1989). Quantitative http://www.castlerockresearch.com/caa/Whatis-
studies of self-assessment in higher education: A CAA.aspx
critical analysis of findings. Higher Education,
Cooper, T. (1997). Portfolio assessment: A guide
18(5), 529-549.
for students. Perth, WA: Praxis Education.
Brookhart, S. M. (2001). Successful students’
Cooper, T. (1999). Portfolio assessment: A guide
formative and summative use of assessment infor-
for lecturers teachers and course designers. Perth,
mation. Assessment in Education, 8(2), 153-169.
WA: Praxis Education.
Brooks, B. A., & Madda, M. (1999). How to
Cooper, T., & Emden, C. (2000). Portfolio assess-
organize a professional portfolio for staff and
ment: A guide for nurses and midwives. Perth,
career development. Journal for Nurses in Staff
WA: Praxis Education.
Development, 15(1), 5-10.
Cooper, T., Hutchins, T., & Sims, M. (1999).
Brown, S., & Knight, P. T. (1994). Assessing learn-
Developing a portfolio which demonstrates
ers in higher education. London: Kogan Page.
competencies. In M. Sims and T. Hutchins (Eds.),
Buchanan, T. (2000). The efficacy of a world-wide Learning materials: Certificate in children’s
Web mediated formative assessment. Journal of Services; 0-6 years (bilingual support) (pp. 3-
Computer Assisted Learning, 16, 193-200. 29). Perth, WA: Ethnic Childcare Resource Inc.
Western Australia.

Cooper, T., & Love, T. (2000). Portfolios in uni- Garrison, D. R. (2003). Cognitive presence for
versity-based design education. In C. Swann & effective asynchronous online learning: The role
E. Young (Eds.), Re-inventing design education of reflective inquiry, self-direction and metacogni-
in the university (pp. 159-166). Perth, WA: School tion. In J. Bourne & J.C. Moore (Eds.), Elements of
of Design, Curtin University. quality online education: Practice and direction
(pp. 47-58). Needham, MA: Sloan-C.
Cooper, T., & Love, T. (2001a). Online portfolio
assessment in information systems. In S. Stoney Gipps, C. (1994). Beyond testing: Towards a
& J. Burn (Eds.), Working for excellence in the e- theory of educational assessment. London:
conomy (pp. 417-426). Perth, WA: We-B Research Falmer Press.
Centre, Edith Cowan University.
Haladyna, T. (1997). Writing test items to evaluate
Cooper, T., & Love, T. (2001b). Online port- higher order thinking. Needham Heights, MA:
folios: Issues of assessment and pedagogy. In Allyn & Bacon.
International Education Research Conference,
Henly, D. C. (2003). Use of Web-based forma-
Melbourne. Retrieved October 29, 2007, from
tive assessment to support student learning in
http://www.aare.edu.au/01pap/coo01346.htm
a metabolism/nutrition unit. Journal of Dental
Cooper, T., & Love, T. (2002). Online portfolios: Education, 7(3), 116-122.
Issues of assessment and pedagogy. In P. Jef-
Hiltz, S. R. (1994). The virtual classroom:
frey (Ed.), AARE 2001: Crossing borders: New
Learning without limits via computer networks.
frontiers of educational research. Coldstream,
Worwood, NJ: Ablex.
Victoria: AARE Inc.
Hutchins, T., Sims, M., & Cooper, T. (1999).
Covington, M. V. (1992). Making the grade: A
Developing a portfolio which demonstrates
self-worth perspective on motivation and school
competencies. In M. Sims & T. Hutchins (Eds.),
reform. Cambridge, UK: Cambridge University
Learning materials: Certificate in children’s
Press.
services; 0-6 years (bilingual support) (pp. 3-
Crompton, P. (1999). Evaluation: A practical 29). Perth, WA: Ethnic Childcare Resource Inc.
guide to methods. Retrieved October 29, 2007, Western Australia.
from http://www.icbl.hw.ac.uk/ltdi/implement-
Hyde, P., Booth R., & Wilson, P. (2003). The de-
ing-it/eval.pdf
velopment of quality online assessment in VET.
Curtis, J. B. (2002). Collaborative tools for e- In H. Guthrie (Ed.), Online learning: Research
learning. Chief Learning Office. Solutions for En- readings (pp. 87-106). Leabrook, South Australia:
terprise Productivity. Retrieved October 29, 2007, NCVER.
from http://www.clomedia.com/content/tem-
Ibabe, I., Gómez J., & Jauregizar, J. (2006). Apli-
plates/clo_feature.asp?articleid=41&zoneid=30
cación de pruebas de auto-evaluación interactivas
Davis, S. L., & Morrow, A. K. (2004). Creating para potenciar el trabajo autónomo de los estudi-
usable assessment tools: A step-by-step guide to antes conforme al sistema ECTS. In J. Guisasola
instrument design. Retrieved October 29, 2007, & T. Nuño (Eds.), La educación universitaria en
from http://www.jmu.edu/assessment/wm_li- tiempos de cambio (pp. 63-74). San Sebastián,
brary/ID_Davis_Morrow_AAHE2004.pdf Spain: Universidad del País Vasco.
Dearing, R. (1997). Higher education in the learn- Ibabe, I., & Jauregizar, J. (2005). Ejercicios de
ing society. London: HMSO. autoevaluación con Hot Potatoes. In I. Ibabe &

J. Jauregizar (Eds.), Cómo crear una web do- Love, T., & Cooper, T. (2004). Designing online
cente de calidad (pp. 65-100). A Coruña, Spain: information systems for portfolio-based assess-
Netbiblo. ment: Design criteria and heuristics. Journal of
Information Technology Education, 3, 65-81.
ITC. (2003). Online assessment techniques. Re-
trieved October 29, 2007, from http://web.utk. Lowry, R. (2005). Computer-aided self-assess-
edu/~dsuppach/indep/assessment2.htm. ment. An effective tool. Chemistry Education
Research and Practice, 6(4), 198-203.
James, R., McInnis, C., & Devlin, M. (2002). As-
sessing learning in Australian universities. Can- Macdonald, J. (2004). Developing competent
berra, Australia: Center for the Study of Higher e-learners: The role of assessment. Assessment
Education, The University of Melbourne & The & Evaluation in Higher Education, 29(2), 215-
Australian Universities Teaching Committee. 226.
Kendle, A., & Northcote, M. (2000). The strug- Mason, B. J., & Bruning, R. (2000). Providing
gle for balance in the use of quantitative and feedback in computer-based instruction: What
qualitative online assessment tasks. In Proceed- the research tells us. Retrieved October 29, 2007,
ings ASCILITE 2000, Coffs Harbour. Retrieved from http://dwb.unl.edu/Edit/MB/MasonBrun-
October 29, 2007, from http://www.ascilite.org. ing.html
au/conferences/coffs00/papers/amanda_kendle.
McConnell, D. (2002). The experience of col-
pdf
laborative assessment in e-learning. Studies in
Kennedy, J. K., Sang, J. C. K, Wai-ming, F. Y., & Continuing Education, 24(1), 73-92.
Fok, P. K. (2006, May). Assessment for productive
OECD. (2005). Formative assessment: Improv-
learning: Forms of assessment and their potential nd ing learning in secondary classrooms. Paris:
for enhancing learning. Paper presented at the 32
OECD.
Annual Conference of the International Associa-
tion for Educational Assessment, Singapore. Parshall, C. G., Davey, T., & Pashley, P. J. (2000).
Innovative item types for computerized testing.
Knight, M. E. (1994). Portfolio assessment: Ap-
In W. Van der Linden & C. A. W. Glas (Eds.),
plication of portfolio analysis. Lanham, MD:
Computerized adaptive testing: Theory and
University Press of America.
practice (pp. 129-148). Norwell, MA: Kluwer
Kulhavy, R. W., & Stock, W. A. (1989). Feedback Academic Publisher.
in written instruction: The place of response
Pintrich, P. R., & Schrauben, B. (1992). Students’
certitude. Educational Psychology Review, 1(4),
motivational beliefs and their cognitive engage-
279-308.
ment in classroom academic tasks. In D. H. Schunk
Lee, G., & Weerakoon, P. (2001). The role of & J. L. Meece (Eds.), Student perceptions in the
computer-aided assessment in health professional classroom. Hillslade, NJ: Lawrence Erlbaum.
education: A comparison of student performance
Plous, S. (2000). Tips on creating and maintaining
in computer-based and paper-and-pen tests. Medi-
an educational World Wide Web site. Teaching
cal Teacher, 23, 152-157.
of Psychology, 27, 63-70.
Linn, R. L., Baker, E. L., & Dunbar, S. B. (1991).
Poole, D. M. (2000). Student participation in a
Complex, performance-based assessment: Ex-
discussion-oriented online course: A case study.
pectations and validation criteria. Educational
Journal of Research on Computing in Education,
Researcher, 20(8), 15-21.
33(2), 162-177.

Quinn, D., & Reid, I. (2003). Using innovative Perspectives of curriculum evaluation (pp. 39-83).
online quizzes to assist learning. Retrieved Chicago: Rand McNally.
October 29, 2007, from http://ausweb.scu.edu.
Sherman, R. C. (1998). Using the World Wide Web
au/aw03/papers/quinn/paper.html
to teach everyday applications of social psychol-
Reeves, T. C. (2000). Alternative assessment ogy. Teaching of Psychology, 25, 212-216.
approaches for online learning environments in
Sluijsmans, D. M. A., Prins, F. J., & Martens,
higher education. Journal of Educational Com-
R. L. (2006). The design of competency-based
puting Research, 23(1), 101-111.
performance assessment in e-learning. Learning
Rowntree, D. (1982). Educational technology in Environments Research, 9, 45-66.
curriculum development. Newcastle upon Tyne,
Stephens, D. (2001). Use of computer assisted as-
UK: Athenaeum Press Ltd.
sessment: Benefits to students and staff. Education
Rudner, L. M. (1998). An online, interactive, com- for Information, 19, 265-275.
puter adaptive testing tutorial. Retrieved October
Stiggins, R. J. (1987). Design and development of
29, 2007, from http://edres.org/scripts/cat
performance assessment. Educational Measure-
Ryan, Y. (2000). Assessment in online teaching. ment: Issues and Practice, 4, 263-273.
Paper presented at the Australian Universities
Taras, M. (2001). The use of tutor feedback and
Teaching Committee Forum 2000, Canberra,
student self-assessment in summative assessment
Australia. Retrieved October 29, 2007, from
tasks: Towards transparency for students and
http://www.autc.gov.au/forum/papers/online-
for tutors. Assessment and Evaluation in Higher
teaching1.rtf
Education, 26(6), 606-614.
Sadler, D. R. (1989). Formative assessment and
Taras, M. (2003). To feedback or not to feedback
the design of instructional systems. Instructional
in student self-assessment. Assessment and Evalu-
Science, 18, 119-144.
ation in Higher Education, 25(5), 549-565.
Sadler, D. R. (1998). Formative assessment: Re-
Van Merriëboer, J. J. G. (1997). Training complex
visiting the territory. Assessment in Education,
cognitive skills. Englewood Cliffs, NJ: Educational
5(1), 77-84.
Technology Publications.
Scalise, K., & Gifford, B. (2006). Computer-
Velan, G. M., Killen, M. T., Dziegielewski, M.,
based assessment in e-learning: A framework for
& Kumar, R. K. (2002). Development and evalu-
constructing “Intermediate Constraint” questions
ation of a computer-assisted learning module on
and tasks for technology platforms. The Journal
glomerulonephritis for medical students. Medical
of Technology, Learning, and Assessment, 4(6),
Teacher, 24(4), 412-416.
1-44.
Wang, K. H., Wang, T. H., Wang, W. L., & Huang,
Schulze, A., & O’Keefe, A. (2002, August). Ef-
S. C. (2006). Learning styles and formative assess-
fectively using self-assessment in online learning.
ment strategies: Enhancing student achievement
Paper presented at the 18th Annual Conference
in Web-based learning. Journal of Computer
on Distance Teaching Learning, Madison, Wis-
Assisted Learning, 22(3), 207.
consin.
Ward, M., & Newlands, D. (1998). Use of the
Scriven, M. (1967). The methodology of evalua-
Web in undergraduate teaching. Computers and
tion. In R. Tyler, R. Gagne & M. Scriven (Eds.),
Education, 31, 171-184.

Wiggins, G. (1989). A true test: Toward a more Frederiksen, N. (1984). The real test bias: Influ-
authentic and equitable assessment. Phi Delta ences of testing on teaching and learning. Ameri-
Kappan, 70, 703-713. can Psychologist, 39, 193-202.
William, D., & Black, P. (1996). Meanings and con- Hoskins, S. L., & van Hooff, J. C. (2005). Mo-
sequences: A basis for distinguishing formative tivation and ability: Which students use online
and summative functions of assessment. British learning and what influence does it have on their
Educational Research Journal, 22, 537-48. achievement? British Journal of Educational
Technology, 36(2), 177-192.
Zakrzewski, S., & Bull, J. (1999). The mass
implementation and evaluation of computer-based Howell, S. L., & Hricko, M. (2006). Online as-
assessments. Assessment & Evaluation in Higher sessment and measurement. Case studies from
Education, 23(2), 141-152. higher education, K-12 and corporate. London:
Information Science Publishing.
Hricko, M., & Howell, S. L. (2006). Online as-
AddItIonAL reAdIngs sessment and measurement. Foundations and
challenges. London: Information Science Pub-
Barbosa, H., & García, F. (2005, July). Importance lishing.
of online assessment in the e-learning process.
Huba, M. E. (2000). Learner-centred assessment
Paper presented at the ITHET 6th Annual Inter-
on college campus. Shifting the focus from teach-
national Conference, Juan Dolio, Dominican
ing to learning. London: Allyn and Bacon.
Republic.
McDonald, B., & Boud, D. (2003). The effects
Bennett, R. E., Goodman, M., Hessinger, J.,
of self assessment training on performance in
Ligget, J., Marshall, G., Kahn, H., et al. (1999).
external examinations. Assessment in Education,
Using multimedia in large-scale computer-based
10(2), 210-220.
testing programs. Computers in Human Behavior,
15, 283-294. McIntosh, M. E. (1997). Formative assessment in
mathematics. Clearing House, 71(2), 92-97.
Billings, D. (2000). A framework for assessing
outcomes and practices in Web-based courses Peat, M., & Franklin, S. (2002). Supporting student
in nursing. Journal of Nursing Education, 39, learning. The use of computer-based formative
61-67. assessment modules. British Journal of Educa-
tional Technology, 33(5), 515-523.
Boud, D., Cohen, R., & Sampson, J. (1999). Peer
learning and assessment. Assessment and Evalu- Pellegrino, J., Chudowsky, N., & Glaser, R. (2001).
ation in Higher Education, 24(4), 413-426. Knowing what students know: The science and
design of educational assessment. In Center
Bracey, G., & Rudner, L. M. (1992). Person-fit
for Education (Ed.). Washington, DC: National
statistics: High potential and many unanswered
Academy Press.
questions. Practical Assessment, Research &
Evaluation, 3(7). Retrieved October 29, 2007, from Ramaprasad, A. (1983). On the definition of feed-
http://PAREonline.net/getvn.asp?v=3&n=7 back. Behavioral Science, 28(1), 4-13.
Comeaux, P. (2005). Assessing online learning. Ricketts, C., & Zakrzewski, S. (2005). A risk-
Bolton, MA: Anker Publishing Company, Inc. analysis approach to implementing Web-based
assessment. Assessment & Evaluation in Higher
Education, 30(6), 603-620.

Roberts, T. S. (2006). Self, peer and group as- Taras, M. (2002). Using assessment for learning
sessment in e-learning. London: Information and learning from assessment. Assessment &
Science Publishing. Evaluation in Higher Education, 27(2), 501-510.
Silberman, M. (1996). Active learning: 101 strate- Yan, Z., Hao, H., Hobbs, L. J., & Wen N. (2003).
gies to teach any subject. Needham Heights, MA: The psychology of e-learning: A field of study.
Allyn & Bacon. Journal Educational Computing Research, 29(3),
285-296.
Stake, R. E. (1995). The art of case study research.
Thousand Oaks, CA: Sage. Yorke, M. (2003). Formative assessment in higher
education: Moves towards theory and the enhance-
Stiggins, R. (2005). Assessment FOR learning:
ment of pedagogic practice. Higher Education,
Building a culture of confident learners. In R.
45(4), 477-501.
DuFour, R. Eaker & R. DuFour (Eds.), On com-
mon ground: The power of professional learning
communities. Bloomington, IN: Solution Tree
(formerly National Educational Service).
Straus, S., Miles, J., & Levesque, L. (2001).
The effects of videoconference, telephone, and
face-to-face media on interviewer and applicant
judgments in employment interviews. Journal of
Management, 27, 363-381.
00
0
Chapter XVII
Designing an Online
Assessment in E-Learning
María José Rodríguez Conde
ABstrAct
In this chapter we carry out analysis of the term “assessment,” applied over all the elements which
constitute the environment of formation (evaluation), and also particularizing in the assessment of the
learning process, developed in the frame of what we call e-learning. The perspective guiding text is of
a methodological and pedagogical nature. We try to plan the assessment process in online formation
environments dealing in depth with the different elements which constitute it: objectives and functions
of assessment, assessment criteria and indicators, people involved and assessment agents, software
instruments and tools for the collection of data, and analysis of the information and reports. We raise
a discussion about institutional strategies for the incorporation of this e-assessment methodology in
higher educational institutions and come to the final conclusions about the validity and appropriateness
of the e-learning assessment processes.
IntroductIon we have the term learning assessment; the term

program evaluation, however, defines the evalu-
Evaluation covers a wide semantic field which ation of programs.
should be explained from the outset. The terms With regards to the definition of the general
evaluation, assessment, and evaluating research concept of “evaluation,” Stufflebeam (1999, p.
appear to refer, indistinctly in some cases, to the 3), when dealing with evaluation of educational
different types of evaluation processes. Each one programs, defines the term as a study designed and
of these terms alludes to a slight difference in conducted to assist some audience to measure an
the concept. The term assessment is used when object’s merit and worth. Evaluation, on the other
referring to the evaluation of people from there hand, when referred to as assessment, is used to
Designing an Online Assessment in E-Learning
determine the objective level of a variable that is As an example of the importance that com-
of interest (marks in a test, level of interaction, puter-assisted assessment programs are acquiring
time in responding, etc.), constitutes part of the nowadays, and the problem that goes joined to
previous evaluation concept. In our pedagogic them, we want to cite Gibbs (2006, p. 18), when
context evaluation is defined, in a general sense, he asserts that in the United Kingdom,
as the set of systematic processes of collection,
analysis, and interpretation of valid and reliable ...the implementation of institutions-wide “vir-
information, which when compared with a point tual learning environments” (or e-learning) has
of reference or criteria allows us to make a deci- made it much easier to use simple forms of com-
sion which is favourable to the improvement of puter–based assessment and there has been ever
the object being evaluated. more funding, projects, dissemination and staff
We are interested in highlighting three aspects development to support those who would like to
of this concept. First, to evaluate is not to know use such methods. Unlike the USA, much use of
something, nor have an opinion on something and computer–aided assessment in largely formative
express it. Evaluation is a process wherein we, as in nature: to give students practice and feedback
teaching professionals, develop a process which and to highlight where more studying might be
adheres to a methodology, certain techniques appropriate before the “real” assessment at a
(conditions), which is, therefore, far more than later point.
mere incidental knowledge, intuition, or opinion.
From this arises the concept of measurement. Nevertheless, it is a fact that if formation
Without measurement, the very valuation leads through this type of environments is actually
us to a subjective opinion; we would not be car- carried out (e-learning) a process of evaluation of
rying out an objective evaluation. it must be associated. How is it carried out, what
Second, we evaluate precisely when we are computing tools are available, which is the secu-
able to establish a comparison between the in- rity of these systems, how results are transmitted
formation which is available to us and some of to students, and so forth, will be problems to be
the reference frameworks, criteria, or norm-types solved from a technological point of view. What
which govern our actions. In this case, different is the purpose of this evaluation, under which
types of evaluation are usually differentiated: criteria the obtained data are compared, which
normative, with external referent or criteria and will be the standard or reference point established
personalised evaluation. to obtain different levels of performance, and so
And, finally, the evaluation process concludes forth, are problems of a pedagogical type that we
with decision making. This is one of the aspects are going to try to analyse here.
which is acquiring increasing importance within Besides all these considerations, there is a
the current concept of evaluation, above all be- series of peculiarities that have to be taken into
cause it tries to link the evaluation process with account in virtual learning environments, such
the process of improvement, therefore decision as the following:
making should be carried out with the aims of
optimizing the very process/activity which is • In a learning programme which offers the
being evaluated. student some type of recognition of the level
Then, the object of evaluation on which we are achieved, the evaluation of performance
going to centre this chapter will be set on “Learn- will be the first concern when tackling the
ing” in educational spaces of the denominated teaching-learning process. If the student is
e-learning environments. also interested in acquiring new concepts
0
or strategies, the self-evaluation or learning metrodoLogy Assessment In

evaluation processes will be very useful. e-LeArnIng
• In e-learning programmes, it is fundamen-
tal to evaluate participation and contrast The perspective guiding text is of a methodological
whether the students have attained certain and pedagogical nature; therefore, we try to plan
learning goals or not and, hence, whether the online evaluation process dealing in depth
they have reached the objectives of the with the different elements which constitute it:
course. objectives and functions of assessment, assess-
• In online courses it is essential for the student ment criteria and indicators, people involved and
to receive feedback on progress in the course. assessment agents, software instruments and
This also serves as a motivating element tools for the collection of data, and analysis of
• In online teaching, contrary to expecta- the information and reports
tions, there is much material for evaluating
the students, given that a large part of the Aims of the Assessment
communication is written.
• Hence, the assessment process must be A first element to consider in a learning evalu-
planned, the assessment strategies have to be ation process of students in any formative scale
coherent with the material provided online is the aim of assessment: Why do we assess? In
and the criteria or references for assessment this moment, we should reflect on two concepts
must be stated explicitly so that the student associated to evaluation and differentiated by
at a distance will know what the student is their purpose; we are referring to the concepts
going to be evaluated on, how, when, and of “formative assessment” and “summative
under what criteria the work will be as- assessment” (Scriven, 1967). Formative evalua-
sessed. tion is mainly developed during the educational
process because its objective is to improve it as
As García Carrasco et al. (2002) point out, eval- long as the different tasks or learning-teaching
uation is an essential part of the teaching-learning activities are carried out; that is to say, formative
process, as a measurement of the achievement of evaluation centres its focus on the development
the learning objectives on the part of the student phase. Is a type of assessment intimately joined
and also as a control of the quality of this process. to continuous assessment; both are carried out
However, current evaluation instruments have during the learning-teaching process. On the
many limitations in the reading-writing context, other hand, summative assessment tries to verify
which can be partly overcome by the new infor- whether the objectives of a certain programme
mation and communication technologies. New have been achieved or not, whether both have
technologies are emerging which make it possible been both efficient and effective, and, therefore,
to construct more complete models that are closer is planned at the end, once the application of it
to the evaluation criteria. Mostly they are tech- has finished (Rosales, 1990). Nowadays, refer-
nologies oriented towards objects, which provide ences to the concept of “assessment” in a general
greater advantages than traditional technologies. sense try more to stress the idea of formative
Likewise, the Web is evolving towards a modular (Charman, 2005; Robinson & Udall, 2006) than
structure, following this same philosophy. The summative, insisting in the improvement of the
Web is a universal space of information, but it students learning process, when making it easier
is a matter of turning it into a universal space of and promoting a reflexive attitude necessary after
knowledge. the feedback received.
0
Assessment has been, as an element of the Assessment criteria and Indicators

educational process, characterised as the critical
factor in the teaching-learning process (Brown, We evaluate precisely when we are in a position to
Bull, & Pendleburg, 1997; Bull & Mckenna, 2001; establish a comparison between the information
McAlpine, 2002; Warburton & Conole, 2003). If available and one of the reference frameworks,
the term assessment has had a negative connota- criteria, or normotypes that govern our action.
tion, it has been due to its consideration outside In this case, different types of assessment are
the learning improvement process. Pérez Juste usually identified: normative, with an external
(2006, p. 24) states that: or criterial referent, and personalised assessment.
When no explicit assessment criteria exist, the
...to assessment it is assigned, or recognised, the objectives of the learning process become as-
function of improvement; a function which, by the sessment referents.
way, is totally coherent with the essence of educa- When, on assessing students’ learning, for
tive acts: we have to bear in mind that education example, we take as a referent the group the
is a systematical and intentional activity at the subject belongs to, conditioning the individual’s
service of improvement of people. mark by the student’s relative position in it, we
are in assessment contexts with reference to the
This supposes to have analysed education norm, or normative evaluation. If the assessment is
regardless of educational quality, when both con- made in reference to previously specified criteria,
cepts must actually be interconnected. Assessment that is, the passing of educational objectives, we
is the element which contributes in a higher degree will be in assessment situations with reference
to the quality of the teaching-learning process, to a criterion, or criterial assessment (Popham,
and is, therefore, necessary to be planned, useful, 1983). And, if the subject’s results are compared
coherent and ethical.1 Underlying the importance to that subject’s own previous results, we shall
of the relation existent between assessment and be in personalised assessment.
quality, we want to stress the following words: Wise et al. (2006) write a chapter from the
perspective of an engineering teacher at Penn
Nowadays when so much about quality is said, it State University, showing two different online
is important to state that assessment and educa- assessment systems which are used to prepare the
tion are closely bound together. It is possible that professional accreditation in engineering. This
not assessment nor quality have sense, at least in online self-assessment instrument captured on a
education, if they are considered as independent weekly basis three types of class level data from
parts (Zabalza, 2001, p. 270). the faculty: learning goal(s), learning activities
to support each goal, and performance summary.
In brief, assessment is an important part in the The second online assessment system developed
teaching-learning process, acting as a means for focused on three data sources or measurements
the consecution of the objectives of the learning that would provide further evidence to the ac-
process by the student, and also as a means to creditation agency that outcomes were being met
control the quality of the cited process (Pérez (criterial assessment). Those three data targets
Juste, López, Peralta, & Municio, 2004). included student performance on each outcome,
faculty perception of course effectiveness, and
student’s perception of their own degree of mastery
0
at the outcome level. This triangulation of data suited to the types of content to be evaluated.
clearly provides the multilevel, outcomes-based Learning evaluation indicators should be
measurements critical for not just accreditation linked to the goals one is trying to evaluate, which
but, even more importantly, continuous improve- in the case where no other type of criteria or ref-
ment and advancement of student learning ( for- erent exists, this goal becomes one (O’Donovan,
mative assessment). Price, & Rust, 2004). As an example, we have as-
If the object for evaluation is the learning pro- sociated dimensions to be evaluated in the student
cess, we must establish a psychological reference with indicators or more suitable procedures.
frame from where we can plan this assessment
process. An interesting approach is that made by People involved and assessment
Mishra (2002) exposing the contribution of the Agents
three psychological theories which have had more
impact on the design and the institutional practice In this section we refer to whom the people in-
in the online learning environments (behaviour- volved in the assessment process of students are
ism, cognitive psychology, and constructivism). who can provide us with information about the
In respect to the learning assessment indicators acquisition level of the contents explained.
(information to gather), when the conception of Linked to the concepts of evaluation we have
learning is undertaken from the perspective of been analysing, another typology appears, ac-
constructivist psychology, we should consider cording to the agent carrying out the evaluation:
when collecting information for an evaluation, self-evaluation and hetero-evaluation. In the case
different fields of knowledge. In this sense, in of learning, the student can evaluate the effort
accordance with the training program aims and made better than anyone else, as well as the dif-
according to the course level and characteristics, ficulties and satisfaction caused by the learning.
the learning information can refer to three large These cases of self-evaluation processes are more
fields: conceptual (knowledge, comprehension, adequately approached if we are in situations of
application, analysis, synthesis, and valuation), formative evaluation. On the other hand, processes
skills or abilities, and attitudes. In this study we of summative evaluation require systems of het-
will suggest that the information gathering strate- ero-evaluation, or assessment by other agents to
gies offer online training platforms which are more complement the former.
Table 1. Constructivist tasks vs. Web tools (Mishra, 2002, p. 494)
Constructivist tasks Web tools

Establishment of personal and group objectives/goals E-mails, discussion groups, note pads
Discuss and debate ideas and receive feedback E-mails, discussion groups, voice-chat
Seek and collect information Web page, search engines, digital drop, boxes, book marking
Software to analyze data, prepare labels, charts and concept
Organizing information in a coherent framework
maps
Integrate different external information to internal concep-
Note taking, annotations, and so forth.
tions
HTML, editors, Web page creation tools, word processors, and
Generate/construct new information
so forth.
Manipulate external information and variables Simulation and animation on the Web
Understanding real world phenomenon Streaming media technology for audio and video
0
Table 2. Learning evaluation dimensions and indicators
Dimensions Indicators
- Correct answers in open answer objective tests and so forth.
Acquisition of conceptual contents
- Production of work , tasks, projects, and so forth, via online.
Acquisition of procedural contents (skills, - Production of work via online, projects, group assignments, wikis,
etc.) portfolios, and so forth.
Acquisition of attitudes - Forms, online questionnaires, chats, discussion forums, and so forth.
We understand that in distance learning, ad- different procedures for evaluating the level of
dressed to adults seeking a certain qualification, competence acquired by the student and which
the criterial, summative, hetero type of evaluation can be used on the Internet. In the following chart,
would be the most suitable for certifying that they we give a general classification of them, together
have satisfactorily achieved the objectives formu- with the potential use of the technology.
lated in the educational process and that they fit the The use of different evaluation strategies
profile of the course. However, the online system through the Internet mainly depends on the type
of learning, based on the new information and of learning we wish to evaluate and how we wish
communication technologies, is going to favour to use the evaluation. If the objective is merely
or foster formative evaluation systems, based on summative, and the level of learning relates to
self-evaluation with objective marking systems knowledge acquired, we shall deduce that the most
that will help the students to situate themselves in suitable way will be the use of objective tests. On
the level of learning achieved and lead the process the other hand, if we are seeking to evaluate with
back to higher levels of performance. a formative purpose, in a context of constructivist
learning, which allows motivation to be included
Information gathering techniques as an important factor, we shall have to resort to
for e-Assessment in e-Learning some system of self-evaluation, with the neces-
sary immediate feedback.
The following step in a methodological process Within all these evaluation strategies we ob-
valid to assess the learning in e-learning envi- served two categories, those procedures that are
ronments is to select the information gathering now normally used in attendance teaching (tra-
technique suitable for each learning objective. ditional tests) and another group of tests that are
For this purpose, learning management systems lately being incorporated to evaluation (alternative
(LMS) offer us different alternatives. Each tool tests). It seems evident that certain technological
comes with advantages and disadvantages that resources incorporated to the use of computers
we are explaining below. open up new possibilities for these new approaches
One of the computer applications most used to the recording of information. This is obvious
in student evaluation is the software for the de- in the case of the portfolios strategy, now incor-
signing of objective tests (closed answer) with the porated to many educational software packages
possibility of self-correction (Ashton et al., 2006, and whose use is beginning to show a greater
experience of SCROLLA project in Scotland). commitment of the students in self-evaluation
This does not mean that the Internet does not and self-learning (Klenowski, 2002). Electronic
offer other resources of high pedagogical value, mail, data bases, and discussion lists, for their
although the use of the technology may not be part, make it possible to store and exchange the
simple. In this section, we shall briefly describe students’ work in its process and in its products,
0
Table 3. Tools assessment and potential use of the technology (Rodríguez-Conde & Herrera-García,
2005)
Evaluation instruments Potential use of the technology

HIGH: the evaluation system can be completely computerised (from the
a) Objective tests (closed exams)
design of the test to its correction and the preparing of reports).
b) Open answer tests (essay, short LOW: this would require recognition of key words, phrases, and so forth
answer, and so forth, exams) (content analysis procedures).
c) Practical exams (experimental tasks,
simulations, observation, and so MEDIUM: the technology can save data, analyse, use OMR.
forth)
d) Oral exams (before a board) LOW: video-streaming technology could help.
e) Projects or papers (research, case
LOW/MEDIUM/HIGH: depending on the type of content.
studies, diagnoses, and so forth)
HIGH: The process can be completely computerised, with the incorporation
f) Self-evaluations
of immediate feedback.
LOW/HIGH: depending on the organization of the contents to be evaluated
g) Portfolios
and the correction procedure of the type of contents.
as well as to accelerate feedback mechanisms in • Flexibility of access (space and time)

both directions. But, once again, also in the case • Motivation of the students
of evaluation, technology can serve to make a
certain philosophy of learning operative. From all the studies on evaluation through the
We think it is interesting to point out that, Web it can be concluded that this is beneficial, both
among the many studies carried out on the as- for the student and for the teacher. For the stu-
sessment that teachers make of the effectiveness dents, it helps them to improve their competence
of different instruments for evaluation through level, motivates them for study and, in short, is
the Internet, there is a general opinion concern- a useful tool for learning. The teachers value the
ing the importance of feedback and the effect it facility of being able to send the student immedi-
has on the student’s learning. Charman (2005) ate feedback by means of comments or guidance.
points out what the main advantages of the use However, there should be some modification in the
of evaluation through the Internet may be: instruction design. If we only change in the fact
that the assessment will be automated (correction
• Frequency of the evaluation and preparation of reports for the student), there
• Immediate feedback (the students like to will be no change in the students’ learning. What
receive feedback since they connect their is really effective is the quality of the feedback
production with the results, they are offered that these technological tools allow us, that is,
help and guidance and all this motivates using the evaluation in its formative modality in
them to continue studying) which the student is offered detailed information
• Immediate correction of work by the teach- on the student’s performance.
ers (so that mistakes can be detected and, if Among the software applications currently
necessary, certain contents adapted) on the market, after evaluating them through the
• Reliability in measurement (stability, ac- demonstration modality over a period of time, we
curacy) consider that the possibilities offered by Percep-
tion (http://www.questionmark.com), compared
0
with those of other programmes, makes it one of block, according to the score obtained, the
the most suitable for carrying out an evaluation student has the possibility of repeating it or
in a formative context, as we pointed out. This having other questions sent or considering
software was created by a British company and has the exam concluded.
been in use for over 10 years, in its correspond- c. The great variety of types of reports it pres-
ing versions. The facilities of this programme are ents based on exams done by the students
summarised above all in three aspects: of a group. Thus we can obtain through the
Internet both individual and group results,
a. The type of questions that can be designed presenting the appropriate statistics.
using its latest version are: multiple choice,
multiple answer, gap-filling, questions on In short, it is a product with many possibili-
an interactive image, matrix, selection, ties for encouraging processes of self-evaluation
introducing number, explanation, and devel- of learning, according to the competence levels
opment question. It also makes it possible manifested, and in turn it allows teachers to fol-
to offer a type of feedback of elaboration low up each student’s progress. However, in all
in which more exhaustive and personalised these systems there is a clear need for a technical
explanations are offered, according to the support that will facilitate and manage the good
student’s response. functioning of the tool, especially the need for a
b. Furthermore, it makes it possible to adapt server whose maintenance is guaranteed.
the exam in the style of computerised adap-
tive tests, through the concept of “blocks” data Analysis and Assessment
within an exam. This system makes it report
possible to divide the test into parts, with
different questions, and gives the possibility While carrying out the analysis of all the infor-
of creating conditions among the different mation that we can gather from a student in e-
blocks, so that when the student finishes one learning environments, a problem appears when
Figure 1. Preliminary view of the edition of an item. Perception 3.0
0
Figure 2. Preliminary view: Incorrect answer and associated feedback. Perception 3.0
establishing the assessment criteria, in the case of packets as SPSS http://www.spss.com/, etc.) each
summative assessment, over which we are going of the typical indicators of the Classis Test Theory
to judge the execution and the level of accomplish- (difficulty and discrimination indexes), as well
ment of the objectives established. Within these as average difficulty of the test as a whole and
criteria, we find the standards or departing points reliability tests, and so forth.
we should establish for each level. The problem-to-be-solved in the data analysis
To carry out these processes, the computing is found in the moment when the information gath-
tool must contain automatic systems which allow ered from the student is too extensive (elaboration
us to program each of the levels and produce the of a project, essay-form answer, etc.). Nowadays,
result in an automatic form, supplying all the nec- systems based on the analysis of contents under
essary information to fulfil an individual report tools type data base already exist. For example,
for each student. One of the solutions and reports Margerum-Leys and Bass (2006) show us that
that an LMS such as Moodle (http://moodle.org/) a data base is a tool designed to organise and
presents can be observed below. This tool allows manage large amounts of information following
the teacher to owe several parameters over which a general process of content analysis, establish-
to configure different tests of an objective type ing analysis units, category systems (e.g., “very
(items with a unique true answer) and some se- good,” “good,” “needs improvement”) and an
curity parameters. The student is able to answer adjudication system of the units to the categories;
and consult the results obtained item by item and these systems can help to the analysis of a large
the student’s final punctuation. group of information.
On the other hand, it is very interesting to Another alternative being considered is to work
know how each of the tests the students have taken with the possibilities of the so called “Semantic
has functioned, especially when using objective Web,” and natural language processing. Further-
tests. A simple analysis of the items of the test, more, there are programs as Nid.ist, Atlast, Aguad,
where it is reflected in form of a chart exportable Ethnograph, which facilitate this content analysis
to other programs (calculation sheets, statistics through the analysis of contents based in the pre-
0
Figure 3. Report of student results (Moodle)
Figure 4. Report of test or item analysis (Moodle)
0
vious establishment of the analysis categories or evaluation system the student is going to use, to
in the quantitative lexicometric study. the software navigation system, the formulation
of questions, and so forth.
The creation of specific centres for assessment
dIscussIon through Internet is a strategy coming from the
Anglo-Saxon context where there are centres
the Problem of Learning which have worked on the online assessment.
Assessment Based on the use A good example of it is the centre known as
of internet: management alternatives SCROLLA2 (The Scottish Centre for Research
in higher Education Environments into Online Learning and Assessment), gather-
nowadays ing the participation and the work as a whole
of (inner-university) different universities: Ed-
A computer-based assessment system (CBA) inburgh, Glasgow, and Strathdyde. The work
requires group work of students, teachers, and developed in it is concerned about providing and
supplementary people, both technical and peda- carrying out investigations related to the educa-
gogical. By reason of it, much relevance has been tion technologies, including their direct relation
given to the adoption of different roles that all with the assessment processes. In this Centre,
participants, one way or another, must cope with. the work performed has dealt with the existing
We are referring to the implication needed for the topics related to online learning, online assess-
correct development of the CBA so that every ment, computer-assisted assessment, computer
component has a fixed set of activities described assessment, investigation, and information and
in Figure 5. communication technologies.
Any CBA system must offer the possibility In the previously cited centre, a series of confer-
to plan exams with different formats (unique ences and seminars directly related to evaluation
answer, cases study, and so forth); besides this, and the uses of technologies are planned, among
when an exam is designed, aspects related to the which we can stress: the future of computer-as-
novelty that the system involves must be taken sisted assessment and the e-portfolio, among oth-
into consideration; we know it is convenient to ers. They are also working on the implementation
facilitate a previous approach of the student to the of a Masters Degree in e-learning, including in
Figure 5. Role of participants in an e-assessment (own elaboration from Zakrzewske & Steven, 2003)
STUDENTS TEACHERS SUPPLEMENT MANAGER

ARY STAFF
- Change the learning way - Time to develop the tests - Implementation and - Coordination and
- Improve the abilities efficiently development of direction of the team.
- Reflect on the process -Reduction of charges to CBA systems. - Inversion on software
- Work in proper place and liberate teachers and - Deep knowledge and hardware.
time. allow them develop other about the adequate - Integration of the pre-
areas of interest. software. existent evaluation
- Quick feedback systems.

its courses several content blocks such as e-learn- also the access to other secondary components
ing strategies, investigation methods, and online where activities and reports performed by the
evaluation among others. Work proposals in SC- project which contributed to the creation of it
ROLLA are lead to reinforce the incorporation are reflected.
of TICs in the teaching, learning, and formation
processes, including assessment processes. In the
implementation of the online assessment, the prob- concLusIon
lem is not, in the first instance, whether students
possess computers connected to Internet, but the We would like to point out several of the advantages
quality of the access, that is to say, that the access that the so-called e-learning processes can provide
could be simultaneously carried out through, for in the assessment processes of students:
example, wifi, without appearing technical effects
interfering or negatively influencing the develop- • Offer a great temporal flexibility, allowing
ment of assessment exercises. the student to choose the best moment to
Another centre, Computer-Assisted Assess- develop it, that is to say, the student has
ment3 (CAA), was a result of consortium be- more options to decide when to carry out
tween four institutions: The University of Luton, the assessment.
Glasgow, Loguhborough,4 and Oxford Brookes.5 • Flexibility of spaces, the student chooses
Among the objectives and priorities of the cited the place from where to accomplish the
centre (CAA) we can emphasize the following assessment; the only need is a computer
ones: promote the use of CAA in British higher connected to internet.
education, identify and develop a good practice • It is possible to offer the student an im-
integrated in the study plans, develop models, mediate feedback, emphasizing this way
paradigms to the strategic development of CAA the correction of mistakes in the moment
in departments, institutions and any other cen- they are made. In this section we should
tre, promote the use of materials helping people emphasize that the quantity of feedback is
interested in practising any assessment through not as important as its quality.
technologies possible. • The incorporation of information and com-
We must emphasize that this centre provides munication technologies motivated students
information about the use of computer-assisted means a novelty methodology, not previously
assessment in higher education and was obtained used with an assessment purpose, what en-
in a project about this same topic initiated in 1998 tails more interest and a better willingness
and finished in 2001, despite its specific projects of the student to carry out the tests.
and “blueprint” investigations or reports whose • The online evaluation can be easily admin-
content is very varied, form assessment principles istrated.
to item-analysis summaries or any type of con- • Different instruments, strategies or as-
tent related to the issue treated. During those 3 sessment tools can be used, we are not
years, the centre was an important referent which only talking about fulfilling an “objective”
facilitated actualized information about the CAA test through internet, but we can use other
activities planned, the models developed, confer- means such as the participation in different
ences, and so forth. On the centre’s Web site, we activities offered, as forums, chats, discus-
can see that it consists of six sections (Centro, sion groups, or elaboration of tasks of varied
history, outcomes, contact us, blueprint) which types (investigation projects, professional
we can access from the main menu, allowing activities design, etc.)

On the other hand, and as a result of the Orientation and Educative Assessment (GE2O) of
e-learning experience of the enterprise Cisco the University of Salamanca and, especially, by
Networking AcademyTH Program, Behrens, Col- the Investigation Project i+d: “E-Learning Plat-
lison, and Demark (2006) set the seven C’s of the form Based on the Management of Knowledge,
comprehensive assessment model. Learning Objects Libraries and Adaptive Systems
As a final statement, we consider it necessary (KEOPS Project)” (Reference TSI2005-00960),
to design projects of interdisciplinary investigation in which context we are developing the e-assess-
in the field of e-assessment, which join specialists ment investigation.
in the content to evaluate, specialists in formation
processes or pedagogics, and people in charge
of the computer area. We have also observed, Future reseArch dIrectIons
through the contact with various experiences in
international environments, that the formation of This section describes future research directions,
interdisciplinary working teams with institutional opportunities, or additional ideas offered by the
supply will be a decisive development factor, es- author of chapter on the main focus of the chapter or
sential in this subject. related areas. This section will be highly beneficial
to other colleagues and other researchers world-
wide, in particular for doctoral students constantly
AcknowLedgment in search of additional research areas. Research
topics that can open up in this field are:
We want to acknowledge the support received
by the Investigation Group GRIAL (E-learning • To carry out empiric research through ex-
and Interaction Investigation Group) and by the perimental designs to check the effectiveness
of use of ICTs in learning assessment.
Table 4. Summary of seven C’s of Networking Academy comprehensive assessment model (Behrens, et
al., 2006, p. 232)
Area Goal Examples
Develop content following claims implies evidence implies
Claims Clarity of design tasks model. Make delivery system flexible to match different
assessment goals.
Embed development in Involve instructors in content development via online authoring
Collaboration
instructional community tool. Establish online feedback mechanism.
Leverage digital technolo-
Flexible scoring technologies. Use of simulation and automated
Complexity gies to make assessments as
scoring of hands-on tasks. Linking to curricula.
complex as appropriate
Translate and localize con- Authoring and delivery tools that support complex language
Contextualization tent for stakeholders around formats. Localization and review processes to ensure appropri-
the world ate translation and delivery.
Analyze data to improve and Use of classical and IRT models to analyse results. Flexible
Computing
revise content delivery system allows false revision
Empower stakeholders with Multilevel grade book. Item information pages and summary
Communication
data reports.
Develop assessment system
Linking of curricular assessments with external capstone assess-
Coordination in context of other assess-
ments by design and statistics.
ment and learning activities

• To propose formation models through the Charman, D. (2005). Issues and impacts of using
Internet, establishing the contrast between computer-based assessments (cbas) for formative
different tools of learning assessment. assessment. In S. Brown, J. Bull & P. Race (Eds.),
• To compare the effectiveness and efficiency Computer-assisted assessment in higher educa-
of different tools of learning assessment tion (pp. 85-93). Eastbourne: Routledge.
in an experimental way and through the
García Carrasco, J., Pérez, M.A., Rodríguez, B., &
Internet.
Sánchez, M.C. (2002). Evaluar en la red. Teoría de
• To propose evaluation strategies in problem-
la Educación: Educación y Cultura en la Sociedad
solving learning in environments of online
de la Información, 3(5). Retrieved June 11, 2007,
training.
from http://www.usal.es/~teoriaeducacion/
• To design strategies or alternatives to for-
mative assessment in e-learning, adapted to Gibbs, G. (2006). Why assessment is changing.
different learning styles. In C. Bryan & K. Clegg (Eds.), Innovative as-
• To research in the field of qualitative evalu- sessment in higher education (pp. 11-22). New
ation strategies in e-learning. York: Routledge.
Joint Committee on Standards for Educational
Evaluation. (1988). The personnel evaluation
reFerences
standards. Thousand Oaks, CA: Corwin Press.
Ashton, H. S., Beevers, C. E., Milligan, C. D., Klenowski, V. (2002). Developing portfolios for
Schofield, D. V., Thomas, R. C., & Youngson, learning and assessment: Processes and prin-
M. A. (2006). Moving beyond objective testing ciples. London: RoutledgeFalmer.
in online assessment. In S.L Howell & M. Hricko
Margerum-leys, J., & Bass, K.M. (2006). Elec-
(Eds.), Online assessment and measurement.
tronic tools for online assessment: An illustra-
Case studies from higher education, K-12 and
tive case study from teacher education. In S.L.
corporate (pp. 116-128). London: Information
Howell & M. Hricko (Eds.), Online assessment
Science Publishing.
and measurement. Case studies from higher edu-
Behrens, J.T., Collison, T.A., & DeMark, S. cation, K-12 and corporate (pp. 62-81). London:
(2006). The seven C’s of comprehensive online Information Science Publishing.
assessment: Lessons learned from 36 million
Mcalpine, M (2002). Principles of assessment.
classroom assessments in the Cisco networking
Luton: CAA Centre.
academy program. In S.L Howell & M. Hricko
(Eds.), Online assessment and measurement. Mishra, S. (2002). A design framework for online
Case studies from higher education, K-12 and learning environments. British Journal of Edu-
corporate (pp. 229-245). London: Information cational Technology, 33(4), 493-496.
Science Publishing.
O’Donovan, B., Price, M., & Rust, C. (2004). Know
Brown, G., Bull, J., & Pendleburg, M. (1997). what I mean? Enhancing student understanding
Assesing student learning in higher education. of assessment standars and criteria. Teaching in
London: Routledge. Higher Education, 9(3), 325-336.
Bull, J., & Mckenna, C. (2001). Blueprint for CAA. Pérez Juste, R. (2006). Evaluación de programas
Loughborough: University of Loughborough. educativos. Madrid: La Muralla.

Pérez Juste, R., López, F., Peralta, M.D., & Mu- tion enviroment. In S.L. Howell & M. Hricko
nicio, P. (2004). Hacia una educación de calidad. (Eds.), Online assessment and measurement.
Gestión, instrumentos y evaluación. Madrid: Cases Studies from higher education, K-12 and
Narcea. corporate. (pp. 98-115). London: Information
Science Publishing.
Popham, W.J. (1983). Problemas y técnicas de
evaluación educativa. Madrid: Anaya Zabalza, M.A. (2001). Evaluación de los aprendi-
zajes en la Universidad. In A. García-Valcárcel
Robinson, A., & Udall, M. (2006). Using formative
(Ed.), Didáctica Universitaria (pp.261-291).
assessment to improve student learning through
Madrid: La Muralla.
critical reflection. In C. Bryan & K. Clegg (Eds.),
Innovative assessment in higher education (pp. Zakrzewske, S., & Steven, C. (2003). Computer-
92-99). Oxon: Routledge. based assessment. Quality assurance issues, the
hub of the wheel. Assessment & Evaluation in
Rodríguez-Conde, M.J. (2006). Teaching evalua-
tion in an e-learning enviroment. In E. Verdú, M.J.
Verdú, J. García & R. López (Eds.), Best practices
in e-learning. Towards a technology-based and
quality education (pp. 55-70). Valladolid: BEM. AddItIonAL reAdIng
Rodríguez-Conde, M.J., & Herrera-García, M.E.
Books
(2005). Assessment processes in tele-learning
programmes. In F.J. García, J. García, M. López,
Bull, J., & Mckenna, C. (2001). Blueprint for CAA.
R. López & E. Verdú (Eds.), Educational virtual
Loughborough: University of Loughborough.
spaces in practice the Odiseame approach. (pp.
161-178). Barcelona: Ariel. Falchikov, N. (2004). Improving assessment
through student involvement: Practical solutions
Rosales, C. (1990). Evaluar es reflexionar sobre
for aiding learning in higher and further educa-
la enseñanza. Madrid: Narcea.
tion. London: Routledge.
Scriven, M. (1967). The methodology of evalua-
Howell, S.L., & Hricko, M. (2006). Online as-
tion. En perspectives on curriculum evaluation
sessments and measurement: Foundations and
(AERA Monograph Series on Curriculm Evalu-
challenges. Hershey, PA: IGI Global.
ation, n.1). Chicago: Rand McNally.
Stuffebeam, D.L. (1999). Foundational models
for 21st century program evaluation. Kalamazoo,
Journal Articles
MI: Western Michigan University, The Evalua-
Alfonseca, E., Carro, R.M., Freire, M., Ortigosa,
tion Center.
A., Pérez, D., & Rodríguez, P. (2005). Authoring
Warburton, B., & Conole, G. (2003). Key find- of adaptive computer assisted assessment of free-
ings form recent literature on computer-aided text answers. Educational Technology & Society,
assessment (pp. 1-19). ALTC-C University of 8(3), 53-65.
Southampton.
Challis, D. (2006). Committing to quality learn-
Wise, J. C., Lall, D., Shull, P. J., Sathianathan, ing through adaptive online assessment. Assess-
D., & Lee, S. H. (2006). Using Web-enabled ment and Evaluation in Higher Education, 30(5),
technology in a performance-based accredita- 519-527.

Lutticke, R. (2004). Problem solving with adaptive Internet

feedback. Lecture Notes in Computer Science,
3137, 417-420. Examples and Projects about E-assessment. Re-
trieved October 29, 2007, from http://www.elearn.
Mason, R., Pegler, C., & Weller, M. (2004). E-
malts.ed.ac.uk/services/CAA/index.phtml
portfolios: An assessment tool for online courses.
British Journal of Educational Technology, 35(6), Computer Aided Assessment, CAA, University
717-727. of Edinburgh, UK. Retrieved October 29, 2007,
from http://www.heacademy.ac.uk/633.htm
Ricketts, C., & Wilks, S.J. (2002). Improving
student performance through computer-based Retrieved October 29, 2007, from http://www.
assessment: Insights from recent research. As- jiscinfonet.ac.uk/ (a good place to search for
sessment and Evaluation in Higher Education. information on e-learning in the UK)
27(5), 475-479.
Retrieved October 29, 2007, from http://assess-
Rovai, A.P. (2000). Online and traditional assessment.cetis.ac.uk/
ments: What is the difference? The Internet and
caaconference.com/ (International Computer
Russell, J., Elton, L., Swinglehurst, D., & Green- Assisted Assessment Conference: Research into
halgh, T. (2006). Using the online environment e-assessment)
in assessment for learning: A case-study of a
Web-based course in primary care. Assessment
questionmark.com/us/glossary.aspx (Testing and
& Evaluation in Higher Education, 31(4), 465-
Assessment Glossary of Terms)
478.
Sim, G., Holifield, P., & Brown, M. (2004).
Implementation of computer assisted assessment: cases studies
Lessons from the literature. ALT-J, Research in
Learning Technology, 12(3), 216-229. JISC ITT: E-Assessment. Case Studies of effective
and innovative practice in the area of e-assessment.
Underhill, A.F. (2006). Theories of learning and A joint project between the Open University and
their implications for online assessment. Turkish the University of Derby. (September 2005 - March
Online Journal of Distance Education, TOJDE, 2006). Dr Denise Whitelock (Lead), Simon Rae,
7(1), 165-174. Hassan Sheikh (all OU). Professor Don Macken-
zie, Christine Whitehouse, Cornelia Ruedel (all
Valenti, S., Neri, F., & Cucchiarelli, A. (2003).
UoD). Retrieved October 29, 2007, from http://
An overview of current research on automated
kn.open.ac.uk/public/index.cfm?wpid=4927
essay grading. Journal of Information Technology
Education, 2, 319-330 Effective Practice with E-Assessment. An over-
view of technologies, policies, and practice in
Weller, M. (2002). Assessment issues on a Web-
further and higher education. Author: The Joint
based course. Assessment & Evaluation in Higher
Information Systems Committee (JISC) supports
Education, 27(2),109.
UK post-16 and higher education and research by
providing leadership in the use of Information

and Communications Technology in support of Electrical and Electronic Engineering Assess-

learning, teaching, research and administration. ment Network. The project has built a set of peer
JISC is funded by all the UK post-16 and higher reviewed test bank questions for electrical and
education funding councils. www.jisc.ac.uk. Ef- electronic engineering. It has also established and
fective Practice with E-Assessment is the third built a network in order to identify and disseminate
in a series of JISC publications on the skillful good practice in Engineering Assessment to UK
use of e-learning in 21st century practice in a Academics. Retrieved October 29, 2007, from
technology-rich context. In contrast to the pre- http://www.e3an.ac.uk/
ceding guides in the series—Effective Practice
with e-learning and Innovative Practice with
E-Learning—the focus of this publication is on
endnotes
practice in a broader institutional sense, including
the potential impact of e-assessment on learning 1
The Joint Committee on Standard for Edu-
and teaching. Retrieved October 29, 2007, from
cational Evaluation (1988: 38-39) establishes
http://www.jisc.ac.uk/media/documents/themes/
four necessary characteristics to carry out an
elearning/effprac_eassess.pdf
assessment as correct as possible; the cited
The CAMEL Project: Collaborative Approaches characteristics are: Usefulness, Feasibility
to the Management of E-Learning. CAMEL was (viability), Legitimacy (honesty or ethical
a project funded by the HEFCE Leadership, Gov- integrity) and precision.
ernance, and Management programme which set 2
SCROLLA, Scottish Centre for research into
out to explore how institutions who were making online learning and assessment http://www.
effective use of e-learning and who were collabo- scrolla. ac.uk
rating in regional lifelong learning partnerships 3
http://www.caacentre.ac.uk
might be able to learn from each other in a Commu- 4
http://www.lboro.ac.uk/service /ltd/flicaa/
nity of Practice based around study visits to each index.html
of the partner institutions. This short publication 5
http://www.brookes.ac.uk.services/ocsd/
highlights some of the things CAMEL participants
found out about e-learning and about each other.
jiscinfonet.ac.uk/publications/camel

Chapter XVIII
Quality Assessment of
E-Facilitators
Evelyn Gullett
U21Global Graduate School for Global Leaders, Singapore
ABstrAct
Organizations, in particular HR/Training departments, strive to set forth good practices, quality assur-
ance, and improvement on a continuing basis. With the continuous growth of online university programs,
it is crucial for e-learning establishments to include service quality assessments along with mechanisms
to help e-facilitators consistently maintain the highest quality standard when lecturing, teaching, guid-
ing, administering, and supporting the online learner. This chapter discusses the application of an
e-quality assessment matrix (e-QAM) as part of a quality assessment model that promotes continuous
improvement of the e-learning environment. This model will serve as a tool for online universities and
organizations to achieve a base standard of consistent quality that is essential for program accreditation
and satisfaction of global customers.
IntroductIon that service quality assessments help e-facilitators

consistently maintain the highest quality standard
Both for-profit and nonprofit organizations aim when lecturing, teaching, guiding, administering,
to achieve consistent quality in order to maintain and supporting the online learner. E-facilitators
leadership in a competitive global market. Higher include professors, adjunct faculty, and educators,
educational institutes are becoming professional as well as organizational trainers, who conduct
businesses (Holmes & McElwee, 1995). With the teaching and training online.
continuous growth of online university programs, This chapter discusses the application of a
also referred to as e-universities, e-learning estab- quality assessment model for e-facilitators that
lishments, or virtual universities, it is important promotes continuous improvement of the e-learn-
Quality Assessment of E-Facilitators
ing environment and helps to achieve a standard of quality. The delivery of online education also
consistent quality, which is crucial for accredita- includes customer-based aspects of quality, such
tion and quality management. as competence, responsiveness, reliability, access,
courtesy, communication, credibility, security,
understanding, and knowing (Zeithaml, Parasura-
BAckground man, & Berry, 1990).
E-education and e-training, which take place in

a virtual online environment, allow knowledge totAL QuALIty mAnAgement
to be shared, created, and enhanced on a global (tQm)
platform. With the growing popularity of these
programs, universities are under pressure to Although Deming (1989), Juran (1988), and Crosby
maintain the quality of their online curricula (1979) first applied the concept of quality to their
and their delivery methods in order to remain teachings in the 1950s, the concept of total quality
competitive. Assessing the quality of e-professors control (TQC) was introduced a decade later by
and e-facilitators will assist in the advancement Feigenbaum (1951). He defines total quality as a
of online learning on many fronts. control mechanism that impacts the entire organi-
zation. TQM is known for its quality awards, such
as the Deming Prize (Japan), Baldrige (USA), and
QuALIty In hIgher educAtIon the European Foundation of Quality Management
(EFQM). Teamwork, customer focus, leadership,
Quality can be defined as “an essential and dis- training, and continuous improvement tools are
tinguishing attribute of something or someone” essential for a successful TQM program imple-
(Webster’s Online Dictionary, 2006). There are mentation (Sirvanci, 2004).
various ways to measure quality in the academic Some scholars believe that TQM is a concept
sector. Dahlgaard, Kristensen, and Kanji (1995) rather than a method (Burr, 1993); this may be one
define total quality education as an active joint of the reasons why there is no universally defined
effort of both students and employees towards theory of TQM (Sahney, Banwet, & Karunes,
continuous improvement and a high level of cus- 2004). Nevertheless, Evans and Lindsay (1999)
tomer satisfaction. Cheng (1996) uses a system present some common TQM views:
approach to define education quality by adding
input, output, and processes as components that • The goal is to deliver maximum satisfac-
deliver services to exceed internal and external tion to the customer (or what the customer
stakeholders’ satisfaction. Winch (1996) adds perceives to be maximum satisfaction).
user-and value-based quality to the definition. • The entire organization, at all levels and
Other studies focus on organizational inputs and functions, is involved to meet and exceed
outputs (Cave, Hanney, Kogan, & Trevett, 1988; that goal.
Johnes & Taylor, 1990) or on the processes (Green, • The TQM concept remains flexible and
1994) of the same. adaptable toward change and the evolution
This chapter address the following elements of the organization, its industry, and its
of quality: performance, for example, the reli- customers.
ability, timeliness, and competence of e-facilita- • The responses to customer inquiries are
tors; conformance, or the consistency of service; delivered consistent, accurate, and timely,
and the use of appropriate design tools to assess and waste is eliminated at every level.

The goal of the TQM revolution was to meet Universities and their accrediting bodies
and exceed customer expectations by applying provide evidence of a growing interest in TQM
a continuous improvement processes through applications in higher education (Mergen, Grant,
an integrated system of tools, techniques, and & Widrick, 2000). While TQM matters appear to
training. Similar TQM goals have developed in be addressed in productivity and financial areas
the higher educational arena. (Helms et al., 2001), changing attitudes and per-
ceptions of the global customer requires a new
tQm in higher education response to meet and exceed his or her needs.
Thus, a consistent effort to improve teaching
Originally concentrating on production and must be embraced through a continuous quality
operation management, quality today embraces improvement project.
government, service organizations, healthcare in-
stitutions, the entire private sector, and educational Evaluation as Part of TQm in higher
organizations (Helms, Williams, & Nixon, 2001). education
Harvey and Knight (1996) find various ideas of
quality in higher education, namely exceptional One constituent of a TQM project in higher
quality, perfect or consistent quality, quality fit online education is the evaluation of the e-fa-
for purpose, and quality as a value for money, cilitator. When evaluation takes place, quality
and quality that transforms. Because TQM is a is acknowledged (Stake, 1999). In other words,
dynamic theory that encompasses many concepts, we look for excellence and improvement in
it is difficult to find a comprehensive definition of performance by systematically determining the
quality or TQM in higher education (Chadwick, merit and significance of the evaluation taking
1994; McCulloch, 1993; Taylor & Hill, 1993). place (Scriven, 1999). The traditional approach
It is important to realize that the total qual- in the college setting evaluates the performance
ity of education at every level is a continuous of each individual instructor in the classroom
dynamic interaction of all systems, that is, the (Stake & Cohernour, 1999). Student evaluations
university, support staff, administration, policies, are commonly used to evaluate instructors in both
organizational culture, processes and procedures, face-to-face and online settings; they are also
faculty, as well as the interaction of students used for faculty development recommendations
as a system. Unfortunately, empirical evidence (Scriven, 1995). While this evaluation tool may
regarding TQM methods applied in universities be helpful when integrated into the overall faculty
is restricted to administrative functions, such evaluation, its validity would be limited if it were
as financial aid and registration that comprise the only source assessment or evaluative source
the nonacademic side of the university (Koch & considered. Thus, we may consider the presence
Fisher, 1998; Owlia & Aspinwall, 1996). Some of an online e-faculty mentor or supervisor who
researchers suggest that the application of TQM performs periodic quality checks throughout the
methods will “unite campuses, increase employee semester by applying some type of evaluation
satisfaction, and improve nearly any process that tool. Quality assessment of performance and
it touches” (Koch & Fisher, 1998, p. 659). consistency of service appear to be missing in
online universities. The next section will discuss
this in more detail.
0
Issues In QuALIty Assessment to adjust to the latest technology and to enhance

oF e-FAcILItAtors student-centered learning based on cooperation
and collaboration. There appears to be a correla-
the rationale for a Quality tion between what students perceived as quality
Assessment model service and the behavior displayed by faculty and
administrators (Ham, 2003). Addressing quality
The quality assessment model resulted from the and assessment during training and development
authors’ experience as a former HR professional sessions modifies faculty behavior, resulting in
and e-facilitator for different universities. Only an immediate impact on the students’ perception
two of the five universities (both of which were of quality.
true virtual schools) offered some type of training Helms et al. (2001) argue that most university
program for new e-faculty members; these pro- faculty in the U.S. are evaluated on their perfor-
grams varied in length, thoroughness, intensity, mance in teaching, research, and service and
and rigor. It was not a priority to communicate may be a part of the quality problem of higher
the expectations of basic quality standards to new education. For example, some e-facilitators poorly
e-faculty members. apply pedagogical methods, are not present in
In addition to having a high quality training discussion areas, do not provide detailed and
program, which really serves as the interview constructive feedback, or do not reply to questions
for potential e-faculty candidates, one of the uni- in a timely manner.
versities that offered a training program offers a The University of Wisconsin won the first
mentor program in which a full-time e-facilitator Baldrige education award as a higher education
monitors the course of another e-faculty colleague. institution in 2002, making the successful appli-
Nevertheless, there is no quality feedback given cation of TQM methods worthwhile criteria for
during the time that the e-faculty member is teach- consideration on a global level (Daniels, 2002).
ing the course, which would be useful for making Since then, some 160 universities in the U.S.
immediate improvements; only a short evaluation have actively applied TQM methods, and 50% of
is provided at the end of the class. While most those universities have created quality councils
colleges provide surveys in which students rate (Burkhalter, 1996).
the overall course content and the e-facilitator, Total quality in online education requires
this is insufficient for quality improvement as far quality assessment and management tools that
as the e-professor is concerned. develop teaching and learning. There are not
Who monitors consistent quality assessments enough e-TQM methods currently in place to
of product delivery via the e-facilitator? How is address the more significant problems (such as
the quality of the e-facilitator measured? What consistent quality delivery of teaching by e-fa-
methods are in place to maintain the highest cilitators) facing higher education in the delivery
quality standards? There is a need for a quality of distance learning.
assessment tool that can be used to provide support
and to consistently maintain good practices while
the e-facilitator is teaching online. Moreover, such soLutIons And
a tool can be linked to the e-facilitator’s annual recommendAtIons
performance appraisal.
Online teaching requires an adjustment of Deming (1989) argues that continuous qual-
methods. Unlike traditional face-to-face set- ity improvement will ultimately lead to higher
tings, applied online pedagogy requires faculty customer satisfaction and lower costs as a result

of fewer mistakes and delays, and better use of Interacting with students in a respectful manner,
time, materials, and competencies, allowing firms consistently applying a friendly and enthusiastic
to achieve sustainable competitive advantages. tone, being aware and sensitive to cultural dif-
When reviewing candidates for accreditation, ferences, assisting and guiding them to a suc-
the American Assembly of Collegiate Schools of cessful completion of the course are elements
Business (AACSB) focuses on how continuous of assessment in the student interaction criteria.
quality improvement is integrated into the uni- The discussion and feedback criteria assess how
versity culture. The e-quality assessment matrix well e-facilitators respect student diversity and
(e-QAM), as part of the e-quality management how well they provide constructive, detailed, and
(e-QM) model, meets these criteria and bridges meaningful feedback.
the gap between missing TQM methods in online The matrix describes the expected perfor-
education. Evaluation allows us to construct and mance of the e-facilitator; these activities are
to communicate a certain level of quality (Stake, constructed quality standards of practice that
1999). The e-QAM serves as a standard against should be delivered continuously. In addition,
which the e-facilitator’s performance should be target activities for outstanding performance are
evaluated and compared. This matrix can assist presented as a TQM approach to exceed qual-
online universities in their continuous quality as- ity commitment to delivering e-teaching and
surance, improvement, and management efforts e-learning.
in the delivery of e-teaching and e-learning. In contrast to the typical evaluation approach,
this quality assessment tool relies on an already-
the e-Quality Assessment matrix existing category level of good, consistent quality
(e-QAm) that is taken to the highest level by providing
continuous feedback. This tool is more formative
The e-QAM (Table 1) evaluates the e-facilitator and positive to the individual being assessed:
in six areas: presence, classroom organization and by focusing on best quality practices, it gives e-
environment, interaction with students, discus- professors something toward which to strive. All
sion, and feedback. quality criteria in the e-QAM are interrelated to
The e-facilitator’s daily presence on the dis- achieve outstanding quality performance.
cussion board encourages students to engage in Application of the e-QAM provides detailed
conversation; responding quickly in a respectful and applicable feedback to the e-facilitator, allow-
tone and encouraging questions conveys profes- ing him or her to improve continuous quality ef-
sionalism. Preparing the necessary administrative forts. The matrix can also serve as a tool to enhance
components of the course, such as e-mailing and intercultural communication and understanding
posting the e-facilitators introduction, sending between facilitators. The e-QAM respects the
a welcome e-mail to the students inviting them knowledge that each individual can bring to the on-
to ask questions any time throughout the course line classroom while integrating quality criteria.
are examples of creating an open and collabora- The evaluation criteria of the e-QAM (if met and
tive online environment. Furthermore, e-mail- exceeded by the e-facilitator) may also affect how
ing students additional guidelines regarding the online learning environment is perceived by
course expectations, such as late assignments students. As a leadership tool, the e-QAM serves
for example, the discussion board protocol, or sets forth the expectation of certain behavior and
even guiding them to helpful sections within allows for the review, assessment, and planning
the course ware are also elements of assessment of training and development approaches.
for the organization and environment criteria.

Table 1. The e-quality assessment matrix
Expected Quality Perfor- Target Activities: OUTSTANDING CONSISTENT Quality Perfor-

mance & Activities mance
Presence • Actively visible in the • Actively visible and felt in the classroom 7 days a week
classroom 5 days a • Absent from class less than 2 consecutive days
week • Posts 18-20% of the communications in the classroom
• Absent from class no more • Replies to e-mails and questions posted on the discussion board
than 2 consecutive within 24 hours
days
• Posts 10% of the
communications in the
classroom
Classroom • Sets up the entire • Sets up the entire course and is ready for students one week prior to
Organization & course and is ready for session start
Environment students 3 days prior • Posts a welcome announcement/introduction and e-mails the same to
to the session start date all students
• Posts a welcome • Posts reminder announcements regarding assignments and due dates
announcement & • Provides helpful guidelines such as discussion protocol, grading
introduction criteria, and overall expectations
• Responds to e-mails • Creates an environment that encourages questions and participation
and provides appropri- • Creates a supportive, student-centered learning environment
ate guidance for help- • Guides students throughout the course with appropriate e-mails/post-
ing students get started ings
Interaction with • Maintains a friendly • Displays enthusiasm towards e-learning

Students and professional atti- • Encourages student-to-student and student-to-professor dialogue
tude in all communica- • Encourages questions
tions with students at • Respects diverse opinions
all times • Displays cultural sensitivity
• Enthusiastic, encour- • Provides continuous support at all times
aging and supportive • Respectfully communicates the importance of timeliness
when providing • Identifies student problems early on and ensures that students receive
feedback and assisting the appropriate assistance needed to complete the course success-
students with problems fully
and questions
• Addresses the needs
and abilities of indi-
vidual students
• Assists students with
problems that may
impede the success-
ful completion of
the course, directing
students to tutorial
services as appropriate
continued on following page

Table 1 continued
Discussion • Encourages collab- • Reads discussion postings daily and provides feedback within 24
orative learning and hours
active student involve- • Respects diversity in students and different ways of learning
ment • Shows presence and interest by probing students further to deepen
• Fosters a highly discussions
interactive learning • Encourages experienced based learning by asking students to share
environment by show- their experiences when discussing theory
ing his/her presence in • Encourages cooperation, collaboration, and participation among
the discussion area students
• Facilitates a meaning-
ful, ongoing dialog
and applies real-world
professional experi-
ences in the classroom
Feedback • Assesses student • Assesses student performance within 7 days of the assignment due
performance based date
grading criteria and • Provides detailed, constructive meaningful feedback
course competencies • Provides detailed feedback of written analysis on theory and applica-
within 10 days of the tion when appropriate
assignment due date • Advises the student of precise where improvement is expected and
• Provides detailed, how they can achieve it
constructive meaning-
ful, and appropriate
feedback on student
assignments
• Replies to e-mails
within 48 hours
concLusIon and conformance of quality are tied to the qual-

ity of e-training and development. Applying the
Competition in e-education is becoming fierce as e-QAM will allow online educational programs
institutions seek to gain positions in the global to monitor, guide, and support e-facilitators by
marketplace. Consequently, e-universities must assessing their performance on specific criteria
focus on service quality at every level. Accord- important to quality in the online educational
ing to Orsini (2000), colleges really do not make environment and train and develop them in areas
quality a part of their mission statement and are they need to improve in.
unable to agree on who their customers are. Thus, If quality is consistently displayed by every
if e-universities embark on the next logical move e-facilitator throughout the online university,
of applying a rigorous e-quality management then students will realize that quality is a part
evaluation system effectively, then continuous of this organization’s culture, which may lead to
improvement and quality of service to the student higher student satisfaction and higher enrollment
will be embraced as a growing movement. numbers. The e-QAM holds both the e-facilitator
Consequently, the e-quality assessment matrix and the e-institution responsible for providing a
(e-QAM) presented in this chapter would be a high level of quality. The model may be used as
solid starting point to give performance feedback a strategic e-TQM planning element to identify
to the e-facilitator to achieve and maintain high training and development needs, monitor prog-
quality standards consistently. The performance ress, and continually enhance maximum quality
of e-teaching and e-learning.

Future reseArch dIrectIons Continuous review of this model will produce

continuous quality improvement in e-teaching
The e-quality management model serves as a and learning, resulting in personal and profes-
solid starting point for maintaining high quality sional development growth for the student, the
standards. Future research should compare TQM individual, and the entire organization.
model applications to e-educational environments.
Another area of interest is how the e-QM model
aligns with the organizational mission and HR reFerences
recruitment efforts of the university. Considering
that the range of student feedback in face-to-face Burkhalter, B.B. (1996). How can institutions
settings are an element of value during facilitator of higher education achieve quality within the
evaluations (Scriven, 1999), other studies should new economy? Total Quality Management, 7,
examine the relationship between the e-QM model 153-160.
and student survey responses, and the alignment
Burr, J.T. (1993, March). A new name for a not so
of both sources of evaluation. With the question of
new concept. Quality Progress, 87-88.
quality in mind, additional research should explore
how different class dynamics, such as personality, Cave, M., Hanney, S., Kogan, M., & Trevett, G.
for example, influence the e-professor’s perfor- (1988). The use of performance indicators in
mance and the outcome of student evaluation on higher education: A critical analysis of developing
the e-facilitator. practice. London: Jessica Kingslay.
Bearing in mind that every individual has a
Chadwick, P.A. (1994). University’s TQM ini-
different construct of quality (most frequently
tiative. In P. Nightingale & M. O’Neill (Eds.),
defined by personal experience), quality occurs
Achieving quality learning in higher education
the moment the individual recognizes its occur-
(pp. 120-135). London: Kogan Page.
rence (Stake, 1999). Thus, future research should
consider asking how each, the student and the Cheng, Y.C. (1996). The pursuit of school effective-
e-facilitator, construct quality This should then ness: Theory, policy, and research. Hong Kong:
be compared to the quality standards set for in Hong Kong Institute of Educational Research,
the e-facilitators evaluation. The Chinese University of Hong Kong.
The concept of evaluating teaching as a com-
Crosby, P.B. (1979). Quality is free. New York:
munity practice (Stake & Cohernour, 1999) should
McGraw-Hill.
also be investigated in virtual universities. In the
online environment, e-facilitators are geographi- Dahlgaard, J.J., Kristensen, K., & Kanji, G.K.
cally dispersed; thus, both individual and team (1995). TQM and education. Total Quality Man-
contributions must be considered when evaluating agement, 6(5-6).
teaching in the online environment.
Daniels, S.E. (2002). First to the top. Quality
A case study may look at the effectiveness of
Progress, 35(5), 41-53.
e-facilitator development and growth in teach-
ing skills by promoting class visitations by peer Deming, W.E. (1989). Foundation for manage-
facilitators. A self-assessment component may be ment of quality in the western world. New York:
added to the evaluation model. Finally, the psy- Perigee Books.
chological impact of evaluations on e-facilitators
Evans, J.R., & Lindsay, W.M. (1999). The manage-
should also be considered.
ment and control of quality (4th ed.). Cincinnati,
OH: South-Western College Publishing.

Feigenbaum, A.V. (1951). Quality control: Prin- McCulloch, M. (1993). Total quality management:
ciples, practice, and administration. New York: Its relevance for higher education. Quality Assur-
McGraw-Hill. ance, 1(2), 5-11.
Green, D. (Ed.). (1994). What is quality in higher Mergen, E., Grant, D., & Widrick, S.M. (2000).
education? (pp. 3-20). Buckingham, UK: Open Quality management applied to higher education.
University Press. Total Quality Management, 11(3), 345-352.
Ham, C.L. (2003). Service quality, customer Orsini, J.N. (2000). Profound education. Total
satisfaction, and customer behavioral inten- Quality Management, 11(4-6), 762-766.
tions in higher education. Published doctoral
Owlia, M.S., & Aspinwall, E.M. (1996) A frame-
dissertation AAT 3090234. Nova Southeastern
work for the dimensions of quality in higher
University, FL.
education. Total Quality Management, 7(2).
Harvey, L., & Knight, P.T. (1996). Transform-
Ross, J.A., & Bruce, C.D. (2007). Teacher self-
ing higher education. Buckingham, UK: Open
assessment: A mechanism for facilitating profes-
University Press.
sional growth. Teaching & Teacher Education: An
Helms, M.M., Williams, A.B., & Nixon, J.C. International Journal of Research and Studies,
(2001). TQM principles and their relevance to 23(2), 146-159.
higher education: The question of tenure and
Sahney, S., Banwet, D.K., & Karunes, S. (2004).
post-tenure review. The International Journal of
Conceptualizing total quality management in
Education Management, 14(6-7), 322-331.
higher education. The TQM Magazine, 16(2),
Holmes, G., & McElwee, G. (1995). Total quality 145-159.
management in higher education how to approach
Scriven, M. (1995). Student ratings offer useful
human resource management. Total Quality
input to teacher evaluations. Washington, DC:
Management, 7(6), 5.
ERIC Clearinghouse on Assessment and Evalu-
Johnes, J., & Taylor, J. (1990). Performance indi- ation, Catholic University of America.
cators in higher education. Buckingham: Open
Scriven, M. (1999). The nature of evaluation (Pts.
University Press.
I–II). Washington, DC: ERIC Clearinghouse on
Juran, J.M., & Gyrna, F.M. Jr. (1988). Juran’s Assessment and Evaluation.
quality control handbook. New York: McGraw-
Sirvanci, M.B. (2004). TQM implementation:
Hill.
Critical issues for TQM implementation in higher
Kerlin, C.A. (2000). Measuring student satisfac- education. The TQM Magazine, 16(6), 382-386.
tion with the service processes of selected student
Stake, R.E. (1999). Representing quality in evalu-
educational support services at Everett Commu-
ation. Paper presented at the annual meeting of
nity College. Published Dissertation AAT9961458.
the American Educational Research Association,
Oregon State University.
Quebec, Canada.
Koch, J.V., & Fisher, J.L. (1998). Higher educa-
Stake, R.E. & Cohernour, E.J. (1999). Evaluation
tion and total quality management. Total Quality
of college teaching in a community of practice.
Management, 9(8), 659-668.
University of Illinois.

Taylor, A.W., & Hill, F.M. (1993). Issues for imple- Darabi, A.A., Sikorski, E.G., & Harvey, R.B.
menting TQM in further and higher education: (2006). Validated competencies for distance teach-
The moderating influence of contextual variables. ing. Distance Education, 27(1), 105-122.
Quality Assurance in Education, 1(2), 12-21.
Davies, G., & Stacey, E. (2003). Quality education
Webster’s Online Dictionary. (n.d.). Retrieved @ a distance. Boston, MA: Kluwer Academic
October 30, 2007, from http://www.websters- Publishers.
online-dictionary.org/definition/quality
Demirbolat, A.O. (2006). Education faculty stu-
Winch, C. (1996). Quality in education. Oxford: dents’ tendencies and beliefs about the teacher’s
Blackwell. role in education: A case study in a Turkish
university. Teaching & Teacher Education: An
Zeithaml, V.A., Parasuraman, A., & Berry, L.L.
International Journal of Research and Studies,
(1990). Delivering quality service: Balancing
22(8), 1068-1083.
customer perceptions and expectations. New
York: Free Press. Dommeyer, C.J., Baum, P., Hanna, R.W., &
Chapman, K.S. (2004). Gathering faculty teach-
ing evaluations by in-class and online surveys:
Their effects on response rates and evaluations.
AddItIonAL reAdIng
Assessment & Evaluation in Higher Education,
29(5), 611-623.
Attinello, J.R., Lare, D., & Waters, F. (2006).
The value of teacher portfolios for evaluation Donaldson, S.I., Gooler, L.E., & Scriven, M.
and professional growth. NASSP Bulletin, 90(2), (2002). Strategies for managing evaluation
132-152. anxiety: Toward a psychology of program evalu-
ation. American Journal of Evaluation, 23(3),
Aydin, C.H. (2005). Turkish mentors’ perception
261-273.
of roles, competencies and resources for online
teaching, online submission. Turkish Online Easton, S.S. (2003). Clarifying the instructor’s
Journal of Distance Education, 6(3), 8-12. role in online distance learning. Communication
Education, 52(9), 87-105.
Briggs, S. (2005). Changing roles and competen-
cies of academics. Active Learning in Higher Edu- Garcia, P., & Rose, S. (2007). The influence of
cation: The Journal of the Institute for Learning technocentric collaboration on preservice teach-
and Teaching, 6(3), 256-268. ers’ attitudes about technology’s role in powerful
learning and teaching. Journal of Technology and
Bronmann, L., Mittag, S., & Danie, H.-D. (2006).
Teacher Education, 15(2), 247-266.
Quality assurance in higher education-meta-
evaluation of multi-stage evaluation procedures Gultekin, M. (2006). The attitudes of preschool
in Germany. Higher Education: The International teacher candidates studying through distance edu-
Journal of Higher Education and Educational cation approach towards teaching profession and
Planning, 52(4), 687-709. their perception levels of teaching competency,
online submission. Turkish Online Journal of
Coppola, N.W., Hiltzt, S.R., & Rotter, N.G.
Distance Education, 7(3), 184-197.
(2002). Becoming a virtual professor: Pedagogi-
cal roles and asynchronous learning networks. Kyriakides, L., Demetriou, D., & Charalambous,
Journal of Management Information Systems, C. (2006). Generating criteria for evaluating
18(4), 169-189. teachers through teacher effectiveness research.
Educational Research, 48(1), 1-20.

Malm, B., & Lofgren, H. (2006). Teacher compe- Taut, S. (2007). Studying self-evaluation capac-
tence and students’ conflict handling strategies. ity building in a large international development
Abdingdon, UK: Manchester University Press. organization. American Journal of Evaluation,
28(1), 45-59.
Macdonald, J., & Hills, L. (2005). Combining
reflective logs with electronic networks for pro- Turner, J.E., & Reed, P.A. (2004). Creation of a
fessional development among distance education faculty task list for teaching in a televised dis-
tutors. Distance Education, 26(3), 325-339. tance learning environment. Journal of Industrial
Teacher Education, 41(4), 1-13.
Palmer, A., & Collins, R. (2006). Perceptions of
rewarding excellence in teaching: Motivation and Uzunboylu, H. (2007). Teacher attitudes toward
the scholarship of teaching. Journal of Further online education following an online inservice
and Higher Education, 30(2), 193-205. program. International Journal on E-Learning,
6(2), 267-277.
Ross, J.A., & Bruce, C.D. (2007). Teacher self-
assessment: A mechanism for facilitating profes- Villar Angulo, L.M., & Alegre De La Rosa, O.M.
sional growth. Teaching & Teacher Education: An (2006). Online faculty development in the Canary
International Journal of Research and Studies, Islands: A study of e-mentoring. Higher Educa-
23(2), 146-159. tion in Europe, 31(1), 65-81.
Scriven, M. (2001). Evaluation: Future tense. Walker, S.L. (2005). Modifying formative evalu-
American Journal of Evaluation, 22(3), 301- ation techniques for distance education class
307. evaluation, online submission. Turkish Online
Journal of Distance Education, 6(4), 7-11.
Scriven, M. (2002). Out of the frying pan, into the
fire: Comments on Roth/Tobin. Journal of Per- Wells, J. (2007). Key design factors in durable in-
sonnel Evaluation in Education, 16(4), 303-306. structional technology professional development.
Journal of Technology and Teacher Education,
Stake, R. (2004). How far dare an evaluator go
15(1), 101-122.
toward saving the world? American Journal of
Evaluation, 25(1), 103-107. Williams, P.E. (2003). Roles and competencies
for distance education programs in higher educa-
Stufflebeam, D.L. (2001). Evaluation checklists:
tion institutions. American Journal of Distance
Practical tools for guiding and judging evalu-
Education, 17(1), 45-57.
ations. American Journal of Evaluation, 22(1),
71-79. Wilkerson, J.R., & Lang, W.S. (2007). Assessing
teacher competency: Five standards-based steps
Stufflebeam, D.L., & Wingate, L.A. (2005). A
to valid measurement using the CAATS model.
self-assessment procedure for use in evaluation
Thousand Oaks, CA: Sage Publications.
training. American Journal of Evaluation, 26(4),
544-561.

Chapter XIX
E-QUAL:
A Proposal to Measure the Quality of
E-Learning Courses
Célio Gonçalo Marques

Instituto Politécnico de Tomar, Portugal
João Noivo
Universidade do Minho, Portugal
ABstrAct
This chapter presents a method to measure the quality of e-learning courses. An introduction is first
presented on the problematics of quality in e-learning emphasizing the importance of considering the
learners’ needs in all the development and implementation stages. Next several projects are mentioned,
which are related to quality in e-learning, and some of the most important existing models are described.
Finally, a new proposal is presented, the e-Qual model, which is structured into four areas: learning
contents, learning contexts, processes, and results. With this chapter, the authors aim, not only to draw
the attention to this complicated issue but above all to contribute to a higher credibility of e-learning
proposing a new model that stands out for its simplicity and flexibility for analyzing different pedagogi-
cal models.
IntroductIon E-learning is a clear answer to this challenge.

This distance-teaching model is characterised by
In a society where individual skills tend to become its flexibility. Flexibility in terms of time, allowing
rapidly out of date, one of the greatest challenges the students to access the contents any time they
is to discover new ways of learning that allow want at their own rhythm. Flexibility in terms of
the learners not only to choose what and when space as students do not need to move and have
they want to learn, but also the most appropriate the chance to attend courses anywhere in the
learning way for their own case. world. Flexibility in terms of syllabus contents
E-QUAL
because module-based plans, learning contexts, In what concerns general process-oriented

and educational strategies are used according approaches, we refer to International Standard
to the student’s level of knowledge (Marques, ISO 9000 (ISO, 2003) as well as the total quality
2004). management (TQM) approach (Dahlgaard, Kris-
In the European Union, the European Parlia- tensen, & Khanji, 2005). Despite their recognised
ment and the Council created the e-Learning importance in our society, they have been reveal-
Programme, a bi-annual programme (2004-2006) ing several usage constraints in e-learning courses
for the effective integration of information and (ADEIT, 2002; Ehlers, Goertz et al., 2005).
communication technologies in education and As specific approaches, we refer to those di-
training systems in Europe. This programme is rected towards the product and the process. The
the continuation of the e-Learning Action Plan first ones are more centred in industry specifi-
(2001-2004), and has four action lines: promot- cations (standards) for learning objects and its
ing digital literacy, European virtual campuses, main promoters are the Advanced Distributed
e-twinning of schools in Europe and promotion Learning Initiative (ADL), the Aviation Industry
of teacher training, and transversal actions for CBT Committee (AICC), the Institute of Electri-
the promotion of e-learning in Europe (European cal and Electronic Engineers (IEEE), the Global
Commission, 2003). Learning Consortium, Inc. (IMS), the Alliance of
E-learning is also integrated in the objectives of Remote Instructional Authoring and Distribution
the Information Society Technologies Programme Networks for Europe (ARIADNE), and the Dublin
which is part of the European Union Research Core Meta-Data Initiative (DCMI).
Framework Programme. The second ones aim to ensure the quality of
The Final Report Study on The e-Learning the whole process from the analysis of the require-
Suppliers “market” in Europe conducted on behalf ments to its actual operation. Sustainable Environ-
of the European Commission, DG Education and ment for the Evaluation of Quality in e-Learning
Culture, estimates that in 2003/2004 the European (SEEQUEL), Quality On the Line, Methodological
e-learning market is 4.66 to 5.16 billion euros worth Guide for the Analysis of Quality in Open and
(Danish Technological Institute, 2004). Distance Learning (Meca-ODL), Open eQual-
The highest-value sector is workplace learning ity Learning Standards, and InnoeLearning are
with 3.5-4.0 billion euros. The higher education among the most important projects.
sector occupies the third position with 100 million Within these approaches, there is still the need
euros (60 million in e-learning technologies and to analyse quality from the intervening actors’
40 million in e-learning contents and services)—a perspective: learners, producers, and distributors
value that we believe will increase considerably, a (ADEIT, 2002).
tendency resulting from this new phase in higher Another important aspect is quality from the
education governed by the challenges of the Dec- perspective of contents and contexts. According to
laration of Bologna and lifelong learning. Figueiredo (2002), the great enthusiasts of e-learn-
Without quality e-learning there is no suc- ing believe that the future lies in contents. In his
cessful learning. According to the European opinion, a significant part of that future—maybe
Commission (2005), the good/bad/best practices the most significant—will not lie in the contents
need to be identified and systematized. Ehlers, but in the contexts created to materialise those
Goertz, Hildebrandt, and Pawlowski (2005) contents.
corroborate this opinion, making reference to a This chapter aims to make an analysis of quality
lack of actual implementation and information in e-learning, to explore the problem of quality
on e-learning quality, for example, about specific evaluation in e-learning and to present some of the
quality approaches. main evaluation models. Finally, a model (e-Qual)
0
E-QUAL
is proposed to evaluate the quality of e-learning We know that quality covers a variety of areas
courses. Considering the complexity of this sort from the technology implemented to the organi-
of models, which can lead in many cases to its sational processes and competences of the actors
reduced use, the e-Qual model stands out for involved. Only in combining these areas (which
its simplicity and flexibility to analyse different used to be handled independently) will we be able
pedagogical models. to assure not only quality, but also continuous
improvement and innovation, thus increasing
skills level of the intervening actors.
QuALIty In e-LeArnIng Seufert and Euler (2005) identified five dimen-
sions in their study, which were revealed to be
The concept of quality, defined by Juran (1988) as relevant for sustainable e-learning innovations:
fitness for purpose, has gained a huge significance pedagogical, economical, technical, organisa-
in the educational field, particularly, in e-learn- tional, and cultural.
ing. Quality in e-learning has become a slogan According to Ehlers (2007), quality develop-
for educational policies, producers, distributors, ment should occur in every single development
and a principal demand for learners (Ullmo & and delivery process of e-learning courses and
Ehlers, 2007). programmes, not as an isolated evaluation ap-
Although, quality in e-learning be less charac- proach at the end of a course. The author presents
terised by its precise definition than by its positive three concepts that can be used and combined to
connotation (Ullmo & Ehlers, 2007), we can say form a new, comprehensive concept of quality
that it is achieved when an e-learning experience development: (1) “Quality development has to lead
provides just the right content at just the right time, to better learning,” (2) Quality development has to
when it helps learners master needed knowledge take the “learner’s needs” and the “interests and
and skills, in a manner that they are motivated requirements of the e-learning stakeholders” into
to learn and apply their learning to improve in- account, (3) “How existing concepts, approaches
dividual and organizational performance (ASTD and strategies can be used for quality develop-
& NGA, 2001). ment” (Ehlers, 2007, p. 1).
It is obvious that quality will determine the fu- Ehlers, Hildebrand, Tescheler, and Pawlowski
ture of e-learning (ASTD & NGA, 2001; Berlecon (2004) refer to the open nature of quality as both
Research, 2001; Connolly, Jones, & O’Shea, 2005; a normative definition and a relation between
Ehlers, 2004; among others), but in this debate supply and training needs.
about e-learning quality a number of concepts Today there are many different positions on
must be analysed, which are not consensual. the concept of quality e-learning as a resource.
It has become clear that learning in the soci- But it is consensual that a global quality framing
ety and economy of information and knowledge for e-learning resources should be established
requires the development of new competences, that could be used to support developments at
new visions, new approaches, and new models the different levels: supply, users, producers, and
that will certainly differ from the traditional ap- technology.
proaches rooted in industrial society. Many questions and challenges concerning
In this discussion about quality, it is more and quality management and quality assurance in e-
more clear that e-learning has to be centred in learning have been focussed both by international
learners (Husson, 2004; Ruttenbur, Spickler, & organisations, national authorities, institutions,
Lurie, 2000), which means that the learners’ needs and consumers during the last 10 to 15 years
should be determined in a concrete manner. (Rekkedal, 2006). In the next section, some of

E-QUAL
the most important quality approaches will be Quality on the Line presents criteria for the
analysed. evaluation of Internet distance-learning education
success (Phipps & Merisotis, 2000) that will be
analysed later in this chapter.
QuALIty evALuAtIon In Finally, Open eQuality Learning Standards
e-LeArnIng lists a series of requirements that distance-learning
products and services should have in order to be
As it was already mentioned, quality in e-learning effective (LIfIA & EIfEL, 2004). The standards
is difficult to measure because it depends not only proposed are based on the Canadian Recom-
on the contents or the services, but also on the mended e-Learning Guidelines copublished
interaction of the learner with those contents and by the Canadian Association for Community
services, with trainers and the technology. Education (CACE) and the enterprise FuturEd
The perception of the quality level of an e- Inc (Barker, 2002).
learning product or service depends on the role In Europe, e-learning quality has been a topic
played by the evaluator in the process, which is for several projects, most of them under the e-
different to the producer, the distributor, and the Learning Initiative by the European Commission
learner. such as European Quality Observatory (EQO),
According to the Fundación Universidad- Qual-e-Learning, Supporting Excellence in e-
Empresa de Valencia (ADEIT), the quality issue Learning (SEEL), SEEQUEL, European Founda-
can be approached from different perspectives: tion for Quality in e-Learning (EFQUEL), Qual-
standardization, benchmarking,1 certification, ity, Interoperability and Standards in E-learning
and peer review (ADEIT, 2002). (QUIS), Self Evaluation for Quality in e-Learning
This chapter analyses the benchmarking issue (SEVAQ), European Quality in Individualised
presenting different reference frameworks used Pathways in Education (EQUIPE), e-Learning
to compare courses. Project Example (ELEX), and Virtual European
In the USA and Canada, the following refer- Centre in e-Learning (EQUEL).
ence frameworks have been identified: e-Learning EQO is a portal for the promotion of e-learning
Courseware Certification (ECC) by American quality (EQO, 2007). This project aims the cre-
Society for Training & Development (ASTD), ation of a single place where people interested in
Quality On the Line by IHEP (Institute for e-learning (designers, apprentices, and managers)
Higher Education Policy), Open eQuality Learn- will find the best solution for their needs.
ing Standards by Learning Innovations Forum Based on the results of a study about the use and
d’Innovation d’Apprentissage (LIfIA), and by dissemination of quality approaches in European
European Institute for e-Learning (EIfEL). e-learning (Ehlers, Goertz et al., 2005) and in an-
The ECC is a service for the certification of other additional experience from the EQO project,
e-learning courses by ASTD (ASTD, 2006). This a set of guidelines have been defined that:
certification is based on 19 standards that evaluate
interface, compatibility, production quality, and should shape the quality of e-learning by 2010:
instructional design of e-learning courseware (a) learners must play a key part in determining
(ASTD, 2007). To speed the process of submitting the quality of e-learning services; (b) Europe
e-learning courses for review and certification, the must develop a culture of quality in education
ASTD has created the ECC Self-Assessor Tool, and training; (c) quality must play a central
which allows the user to prescreen an asynchro- role in education and training policy; (d) quality
nous learning course against the 19 standards of must not be the preserve of large organisations;
the ECC (ASTD, 2006). (e) support structures must be established to

E-QUAL
provide competent, service-oriented assistance digital and learning literacy, and promote social
for organisations’ quality development; (f) open cohesion (EFQUEL, 2006).
quality standards must be further developed and
widely implemented; (g) interdisciplinary quality One of the projects of EFQUEL is Triangle—a
research must become established in future as an project based on the work of SEEL, EQO and
independent academic discipline; (h) research and SEEQUEL that seeks:
practice must develop new methods of interchange;
(i) quality development must be designed jointly to promote European diversity of quality ap-
by all those involved; (j) appropriate business proaches and services in the field of learning,
models must be developed for services in the field education and training, to connect results and
of quality. (Ehlers, Goertz et al., 2005, p. 11) concepts on European e-Learning quality devel-
oped in three successful projects, to broaden the
Qual-e-Learning is a research project that discussion and discourse on e-Learning qual-
seeks to identify the “good practices” based on ity, and to provide a sustainable infrastructure
the study of different e-learning courses (Qual as a single entry point for e-Learning quality.
e-Learning Project Consortium, 2003). Two of (EFQUEL, 2006)
the most important results of this project are a
Handbook of Best Practices for the Evaluation The QUIS project is a transversal project in
of e-Learning Effectiveness (Qual e-Learning EU e-Learning Programme. Its activities are all
Project Consortium, 2004) and an evaluation directed towards quality in e-learning, interoper-
tool for the evaluation of training effectiveness ability, and reusability of e-learning materials and
and impact measures (Qual e-Learning Project development of standards (TISIP, 2007). Among
Consortium, 2007). the products developed by this project are several
SEEL is a project that focuses on the study reports and disseminations.
of e-learning quality and its impact on local and SEVAQ operates within the framework of
regional development (EIfEL, 2007). It also aims the Leonardo Da Vinci Programme. The project
at defining policies for the creation of quality aims at improving the quality of the vocational
e-learning courses. Through this project were and educational courses offered via open and dis-
developed guidelines based on a series of indica- tance learning, e-learning and blended learning,
tors that should be used both as success indicators providing a number of good practices concern-
and quality assurance measures for new learning ing quality and a multifunctional self-evaluation
region initiatives (SEEL, 2004). questionnaire in order to obtain valuable customer
SEEQUEL was designed to create a European feedback (SEVAQ, 2005).
forum on evaluation of e-learning quality in order The EQUIPE project aims to increase con-
to identify good practices and define guidelines fidence and encourage innovative educational
(MENON Network EEIG, 2004). This project cre- practices in lifelong learning in universities
ated a model for the analysis of e-learning courses by developing, testing, and promoting quality
that will be analysed later in this chapter. assurance and enhancement tools (EQUIPE,
The purpose of EFQUEL: 2007). This project operates under the Socrates
Programme and is coordinated by the University
is to involve actors in a European community of of Porto (Portugal), European Universities Con-
users and experts to share experiences on how tinuing Education Network (EUCEN), University
e-Learning can be used to strengthen individual, of Genève (Switzerland), University of Limerick
organisational, local and regional development, (Ireland), University of Valencia (Spain), Uni-

E-QUAL
versity of Turku (Filand), University of Bergen firm, offers assessment and certification of e-
(Norway), University of Liverpool (UK), Univer- Learning products and services (eQCheck, 2006).
sity of Genova (Italy) and Kaunas University of The quality assessment bases upon the Canadian
Technology (Lithuania). Recommended e-Learning Guidelines.
ELEX main aim was to exploit the practice In France the Association Française de Nor-
potential of the communities within the European malisation (AFNOR), in cooperation with the
Vocational Training Association (EVTA) using Forum Français pour la Formation Ouverte et à
ICT tools to support team work and by trying to Distance (FFFOD), developed the French Code
maximize the dissemination and re-use of selected of Practice in e-Learning (AFNOR, 2004).
best practices of e-learning and ICT use in the In Germany, the Deutsches Institut für Nor-
field of vocational training (ELEX, 2005). mung e. V. (DIN), which represents the country
EQUEL is coordinated by Lancaster Uni- interests in international standardisation activi-
versity (UK) and involves key researchers and ties, is responsible for the creation of the DIN PAS
e-learning practitioners from 14 European Higher 1032-1 Reference Model for Quality Management
Education institutions. EQUEL stands for “e-qual- and Quality Assurance (DIN, 2004).
ity” in e-learning and is a virtual centre of excel- In Norway, the Norsk Forbund for Fjernunder-
lence for innovation and research in networked visning og Fleksibel Utdanning (NFF), also
learning in higher and post-compulsory education known as Norwegian Association for Distance and
(EQUEL, 2004). Flexible Education (NADE) created the Guide-
Under the framework of Socrates Pro- lines for Quality Standards (NADE, 2001).
gramme—Minerva Action, the Meca-ODL proj- In Portugal, the Portuguese Society for In-
ect was held by a partnership led by the ADEIT. novation (SPI) took charge of the InnoeLearning
This project produced a methodological guide project, which has been funded by the European
in order to analyse Internet learning quality de- Union Programme Information Society Technolo-
veloping a support computer application for the gies. This project consisted on the evaluation of
purpose (ADEIT, 2002). e-learning sites based on quality standards criteria
In England the Institute for IT training (IITT) (SPI, 2003).
developed several best practice documents stat- Among the international organizations that
ing that quality of e-learning services should have been promoting the development of quality
be assured and strengthened such as Code of management approaches, is the European Foun-
Practice for e-Learning Providers, Web Site Us- dation for Quality Management (EFQM), the
ability Standards, Competence Framework for International Organisation for Standardisation
e-Learning Designers and Developers, Compe- (ISO), the European Foundation for Manage-
tence Framework for e-Tutors, and a Charter for ment Development (EFMD), the International
Learners to inform learners of what to require Network for Quality Assurance Agencies in
from an e-learning course (IITT, 2005). The Brit- Higher Education (INQAAHE), the European
ish Open and Distance Learning Quality Council Association for Distance Learning (EADL), the
(ODLQC) developed the Open and Distance European Committee for Standardization (CEN),
Learning Quality Council Standards (ODLQC, and the European Centre for the Development of
2006). The Quality Assurance Agency for Higher Vocational Training (CEDEFOP).
Education (QAA) also developed quality guide- We can also find a huge variety of recom-
lines for quality assurance in distance learning mendations (e.g., AFT, 2000; Hollands, 2000),
(QAA, 2004). Finally, the Quality E-Learning guidelines (e.g., EADL, 2003), criteria catalogues
Assurance Services Ltd. (eQCheck), a British (e.g., Wright, 2003), checklists (e.g., Bellinger,

E-QUAL
2004; Scalan, 2003) and quality approaches for quality of three factors: the learning sources and
quality certification and accreditation (e.g., DECT, resources committed, the process designed and
2007; eduQua; 2005; EFMD, 2007). implemented to generate learning results, and the
Current developments in standardisation pro- coherence and meaningfulness of the experience
vide an International Standard for harmonizing with the context in which the learner is working
the various approaches used around the world and living. In each of these factors different topics
for assessing the quality of e-learning initiatives. are analyzed (MENON Network EEIG, 2004).
This common approach has been developed by The learning sources factor analyses the sup-
the ISO/IEC Joint Technical Committee JTC 1, porting staff, teaching staff, learning materials,
Information Technology, Subcommittee SC 36, and learning infrastructures (MENON Network
Information Technology for Learning, Educa- EEIG, 2004).
tion and Training. This standard, called ISO/IEC The topics for core learning processes are
19796, comprises four parts: Part 1- General ap- guidance/training needs analysis, recruitment,
proach; Part 2: Quality model; Part 3: Reference learning design, learning delivery, course evalua-
methods and metrics; and Part 4: Best practice tion, and learners’ assessment (MENON Network
and implementation guide. The first part is already EEIG, 2004).
complete and next parts will be complete up to Finally, the learning context section analy-
2007 (ISO, 2005). The ISO/IEC 19796 – Part 1: ses the institutional setting, the cultural setting
General approach, “provides an overall frame- (organisational, professional, and general), the
work which can be used for introducing quality learning environment, the legislation, the financial
approaches in all provider and user organizations setting, and values systems (MENON Network
of e-Learning” (ISO, 2005). EEIG, 2004).
This project enhances the subjectivity of as-
sessment of e-learning quality resulting from the
modeLs For the evALuAtIon evaluator’s environment (university, secondary
oF e-LeArnIng QuALIty school, and industry), his/her role (teacher, stu-
dent) and his/her worldview.
This section describes some of the main specific The model requires that for each topic a
evaluation models of process-oriented e-learn- great number of aspects have to be classified
ing courses, namely SEEQUEL, Open eQuality and it does not give the learning context enough
Learning Standards, Meca-ODL, Quality On the relevance (communication with other students
Line, and InnoeLearning. and the staff).
Sustainable Environment for the open e-Quality Learning standards

Evaluation of Quality in e-Learning
(seeQueL) The Canadian and American LIfIA and European
EIfEL have created a joint committee, which rec-
The SEEQUEL project integrated in the European ommended “Open eQuality Learning Standards”
Commission e-Learning Initiative (MENON for the analysis of e-learning courses (LIfIA &
Network EEIG, 2004) was conducted by a group EIfEL, 2004).
of enterprises and institutions called MENON This evaluation grid serves as a model
Network. This project created a model for the for everyone who wishes to plan, carry out,
analysis of e-learning courses based on the idea assess, and take e-learning courses (LIfIA &
that learning experience depends on the inherent EIfEL, 2004).

E-QUAL
The main distinctive features of this guide are: methodological guide for the
consumer-oriented (developed with particular at- Analysis of Quality in open and
tention to return on investment in e-learning for distance Learning (meca-odL)
learners), consensus-based (developed through
consultation with a balance of provider and This reference framework proposal derived from
consumer groups), comprehensive (inclusive of a European Socrates-Minerva project. The project
all elements of the learning system: outcomes called Meca-ODL has been conducted by a part-
and outputs, processes and practices, inputs and nership (English, Spanish, and German universi-
resources), futuristic (describing a preferred future ties and Italian and Greek training organizations)
rather than the present circumstances for design led by ADEIT (ADEIT, 2002). The reference
and delivery), adaptable (with modifications, framework stands out because it involves the entire
appropriate to all levels of learning services), process from conception to evaluation. The seven
and flexible (not all guidelines will apply in all stages and respective topics for a distance-learn-
circumstances) (LIfIA & EIfEL, 2004). ing course included in this evaluation grid are:
This guide includes three areas: quality out- conception, analysis, design, content, production,
comes, quality processes and practices, quality delivery and evaluation (ADEIT, 2002).
inputs, and resources for e-learning products and This project has also developed an online
services (LIfIA & EIfEL, 2004). evaluation tool with the methodology already
Within the outcomes section skills and described. With this tool, the evaluator’s profile
knowledge, learning skills, and course credits (developer/user/reseller) has to be given in order
are analyzed (LIfIA & EIfEL, 2004). to select between 140 criteria available. Items
The quality processes and practices section have a 1-5 weight and their evaluation can also
includes student management, the delivery and range from 1 to 5 (ADEIT, 2002).
management of learning, duly used technologies This evaluation tool is quite complete and at-
(computers and other ICT), communications tempts to ensure quality in every stage of course
facilities, and the digital archive and e-portfolio development (from design to evaluation). This
service/system (LIfIA & EIfEL, 2004). evaluation, therefore, requires complete knowl-
As to the quality inputs and resources for e- edge of courses under analysis, which is not the
Learning products and services the evaluation case when the evaluator’ analysis is based upon
includes: intended learning outcomes, curriculum public information.
content, teaching/learning materials, product/
service information for potential students, learn- Quality on the Line
ing technologies and materials, appropriate and
necessary staff, the comprehensive course pack- The grid “Quality On the Line” has been developed
age (all materials and technologies), evidence of by Phipps and Merisotis (2000) and published by
program success, program plans and budget and the IHEP. It is divided into 7 areas comprising
advertising, recruiting and admissions informa- 45 topics.
tion (LIfIA & EIfEL, 2004). The benchmarks in the category Institutional
The model under analysis enhances e-learning Support include those activities by the institution
results which requires a previous deep knowledge that help to ensure an environment conducive
of the courses and also does not take enough ac- to maintaining quality distance education, as
count of learning context. well as policies that encourage the development
of Internet-based teaching and learning. These

E-QUAL
benchmarks address technological infrastructure InnoeLearning

issues, a technology plan, and professional incen-
tives for faculty (Phipps & Merisotis, 2000). InnoeLearning derived from the identification
The category course development includes of good practices in e-learning in three specific
benchmarks for the development of courseware, areas as identified by the European Commis-
which is produced largely either by faculty sion: learning models, interpersonal skills, and
members on campus, subject experts in organiza- informal learning, and learning communities
tions, and/or commercial enterprises (Phipps & (SPI, 2003).
Merisotis, 2000). In the learning models area the topics to be
The category teaching/learning process ad- considered are: previous skills analysis, tutor
dresses the array of activities related to pedagogy, availability, trainee flexibility, information display
the art of teaching. Included in this category control, goal definition, self-evaluation, content
are process benchmarks involving interactivity, integration, document management, and help
collaboration, and modular learning (Phipps & easiness (SPI, 2003).
Merisotis, 2000). Interpersonal skills and informal learning,
The benchmarks in the category course struc- on their turn, deal with the activities, recognis-
ture address those policies and procedures that ing, transmissibility, environment, communica-
support and relate to the teaching/learning proc- tional interactivity, and motivation, and should
ess. They include course objectives, availability of be trainee-centred (SPI, 2003).
library resources, types of materials provided to Learning communities are characterised by
students, response time to students, and student clearly defined goals (community thematic):
expectations (Phipps & Merisotis, 2000). experienced moderators, registration system,
The category student support includes the synchronous tools, asynchronous tools, group
array of student services normally found on a conflict solving, calendar, and interaction level
college campus including admissions, financial (SPI, 2003).
aid, and so forth, as well as student training and The present model is particularly oriented
assistance while using the Internet (Phipps & towards learning communities disregarding the
Merisotis, 2000). processes and contents, which may as well hamper
The benchmarks of category faculty support the success of the course.
address activities that assist faculty in teaching
online, including policies for faculty transition
help as well as continuing assistance throughout e-QuAL modeL
the teaching period (Phipps & Merisotis, 2000).
The benchmarks in the category evaluation Unlike Meca-ODL or Quality on the Line, this
and assessment relate to policies and procedures new model, e-Qual, aims at analysing learn-
that address how, or if, the institution evaluates ing contents, learning contexts, processes, and
Internet-based distance learning. They include results in a more balanced way without putting
outcomes assessment and data collection (Phipps so much emphasis on quality assurance of the
& Merisotis, 2000). whole development and implementation process
Like Meca-ODL, this model presents a great of e-learning courses.
deal of topics aiming at making a complete analysis The e-Qual model derives from the analysis of
of all course details. This characteristic adjusts reference frameworks presented through projects
to a possible quality certification but hampers its such as Open eQuality Learning Standards (LIfIA
use by the learner applying to courses. & EIfEL, 2004), SEEQUEL (MENON Network

E-QUAL
EEIG, 2004), InnoeLearning (SPI, 2003), Meca- not apply to course analysis. The three remaining
ODL (ADEIT, 2002), Quality On the Line (Phipps values (1, 2, 3) allow to distinguish between the
& Merisotis, 2000) and the several lists of good significance of the items used.
practices and is based on structural simplicity and Course analysis can be done on a global basis
flexibility for the analysis of different pedagogical or by area.
models. Table 1 compares e-Qual with previously The learning contents area comprises four
analysed models. items: written contents, multimedia contents,
The structure of our model includes the four complementary bibliographical sources, and
areas previously mentioned: learning contents, content management system.
learning contexts, processes, and results. In this area all the content-related aspects are
The first two areas relate to the necessary analysed. The first two items refer to the materi-
resources for the implementation of an e-learn- als made available while the third item addresses
ing course (learning contents, learning contexts). the content markers used to deepen topics. The
The third area deals with the processes ensured last item, in turn, deals with the way contents
by the staff (administrative, technical and peda- are acceded.
gogical). The last area relates to the outcomes, in In the second area, the learning contexts, a
particular to the learners’ satisfaction, which is fundamental infrastructure for learning per-
the main aspect to be considered as far as quality formance, is evaluated in three items: common
is concerned. space of the learning community, asynchronous
Within the four areas 16 items have been iden- communication tools, and synchronous com-
tified, which are to be classified according to a munication tools.
0-10 scale. This scale allows the evaluator greater Aspects included in these items are deter-
detail in the classification of each item than the minant in the implementation of a real learning
1-5 scale used, for instance, in Meca-ODL. This community which characterises quality e-learn-
requires a greater accuracy in evaluation but also ing courses.
helps distinguish courses under analysis using a The process area focuses on the staff fostering
relatively small number of items. learning processes. The five items to be classified
The flexibility of the classification chart lies are: administrative management (administrative
in the assignment of a weight to each item, which staff), technical management (technical staff),
varies from 0 to 3 according to its importance to management of contents transfer (trainers/tutors),
the evaluator. Weight 0 indicates that the item does management of learning communities (trainers/
moderators), and learners evaluation.
Table 1. Comparison between quality evaluation models analysed
Open eQual-
Quality
ity Meca- Inno-eL-
e-Qual SEEQUEL On the
Learning ODL earning
Line
Standards
Learning X X X X X
Contents
Learning X X X X
Contexts
Processes X X X X X X
Results X X X X

E-QUAL
E-learning added two different actors and The following tables present the e-Qual model
their performance affected learning quality. The with relevant items and the characteristics to be
significance of tutors and/or moderators depends considered within each of them.
on the pedagogical model adopted. In the case of The aspects to be considered in items classi-
individual-centred contents tutors acquire a more fication are as follows: the evaluator’s sensitivity
important role. As for the learning community, to any of the topics mentioned, the evaluator’s
on the other hand, moderators become the most perspective (learners, producers and distributors),
important actors. and the course objectives determine its signifi-
In the results area, the last to be analysed, cance within each item.
four items were identified: knowledge and skills In order to test the e-Qual model, several
acquired, training recognition, learners satisfac- e-learning courses were evaluated. In the last
tion (clients), and business. evaluation, the e-Qual model has been applied to
Knowledge and acquired skills are the most evaluate three Microsoft Excel courses promoted
quantifiable and measurable result providing an by accredited international organizations.
objective idea of training quality. In this study the choice of courses derived
Training recognition is an important aspect from no pre-defined criteria but the possibility
for the client and must be regarded as so in this of “attending” them.
evaluation. The weight given to each item is the same (1)
Learner’s satisfaction is no doubt the most and the results show a balance between several
sensitive item in quality terms because true sat- courses, though course C stands out slightly
isfaction is only achieved when it is not perceived (Table 6).
by the client. The e-Qual model allows comparison between
The last item relates to the sustainability of the courses either on a whole basis or by area: learn-
e-learning activity and reveals determinant since ing contents, learning contexts, processes, and
a “non-profitable” business ends up by failing, no results.
matter the quality achieved. The model however allows privileging of
certain aspects of the courses, through specific
Table 2. e-Qual model, learning contents
1. LEARNING CONTENTS
Credibility (recognised authors in the field), updating, respect for
author rights, adjustment to trainee cultural background and his
1.1 Written Contents
learning needs, introduction of real experiences, modularity and
reusability.2
Credibility, updating, respect for author rights, technical and
aesthetic quality, compliance with standards and usage guides,
adequacy to technical specifications (e.g., bandwidth3) of trainees,
1.2 Multimedia Contents
interactivity, adjustment of cultural background of trainee and
training needs, incorporation of real experiences, modularity and
reusability.
1.3 Complementary
Web addresses, virtual libraries, and books.
Bibliographical Sources
Structuring coherence according to a learning theory, autonomy of
1.4 Content Management browsers and plug-ins, system personalization, facility of content
System (CMS) location/visualization/download and electronic safety to ensure
integrity and validity of contents provided.

E-QUAL
Table 3. e-Qual model, learning contexts
2. LEARNING CONTEXTS
2.1 Common Space of the Relevant administrative and pedagogic information, trainee intro-
Learning Community duction, activity schedule and published news
2.2 Asynchronous E-mail, mailing lists, newsgroups, forums, blogs, and wikis are
Communication Tools evaluated as to their usage facility and access speed.
Chat, document transfer, whiteboard, document share, audioconfer-
ence and videoconference are evaluated as to their usage facility
2.3 Synchronous
and access speed, possession by trainees of appropriate techni-
Communication Tools
cal conditions (e.g., bandwidth) and necessary equipments (e.g.,
microphone).
Table 4. e-Qual model, processes
3. PROCESSES
Course dissemination/advertising, presentation of duration, goals
and pedagogic methods, guarantee of the fulfilment of pre-re-
quirements for formal and informal learning, the existence of a
3.1 Administrative Man-
reasonable trainee/staff ratio, enrollment procedures (online or in
agement
presence), different payment processes (bank transfer, credit card,
check, money, etc.) and administrative support through different
means (e-mail, chat, and phone).
Usage of CMS and LMS through different means (learning sessions,
3.2 Technical Manage-
e-mail, chat, phone, FAQ’s-Frequently Asked Questions) and glos-
ment
sary is relevant.
Trainer/tutor availability through the tracking of synchronous and
3.3 Management of Con- asynchronous communications between trainees and tutors as well
tent Transfer as effective access to the contents made available, which is meas-
ured through the number of downloads and accesses.
Trainer intervention to dynamise and guide communities for prob-
3.4 Management of Learn- lem solving and effective use of synchronous and asynchronous
ing Communities communication tools between trainees and between the trainers and
the tutors.
Quantitative (self-assessment tests, written exams and simulations)
3.5 Learning Evaluation or qualitative (learning contracts, presentations, written assign-
ments, projects, portfolios and peer evaluation).
Table 5. e-Qual model, results
4. RESULTS
4.1 Knowledge and Skills Consistent with the established syllabus, relevant for the educational
Acquired background and the profession, and improve the trainee’s skills.
Certification/accreditation by professional organizations, the credits
recognised by teaching institutions leading to academic degrees,
4.2 Training Recognition
recognising of similar courses taught under the presential system
and national and international recognising of training performed.
Learning goals achieved (efficacy), the money, time and effort
4.3 Learners Satisfaction spent (effectiveness), the contents made available, and the contexts
created
4.4 Business ROI – Return on Investment.
0
E-QUAL
Table 6. Application of the e-Qual model
Course Course
Course A
Weight B C
Mark Mark Mark
1. LEARNING CONTENTS
1.1 Written Contents 1 7 6 8
1.2 Multimedia Contents 1 6 5 7
1.3 Complementary Bibliographical Sources 1 6 6 7
1.4 Content Management System (CMS) 1 7 7 7
2. LEARNING CONTEXTS
2.1 Common Space of the Learning Community 1 7 8 7
2.2 Asynchronous Communication Tools 1 7 8 6
2.3 Synchronous Communication Tools 1 7 8 6
3. PROCESSES
3.1 Administrative Management 1 8 8 8
3.2 Technical Management 1 7 7 6
3.3 Management of Content Transfer 1 7 6 8
3.4 Management of Learning Communities 1 7 8 6
3.5 Learning Evaluation 1 7 6 8
4. RESULTS
4.1 Knowledge and Skills Acquired 1 7 7 8
4.2 Training Recognition 1 8 6 6
4.3 Learners Satisfaction 1 7 7 8
4.4 Business 1 7 7 7
Global Average 7.0 6.9 7.1
weights assigned to their correspondent items. Figure 1 presents for three courses under
An evaluator who is more concerned with the analysis the initial results (equal weight) together
contents (contents perspective) will attribute a with the results obtained with two different weight
higher weight (3) to items such as written contents assignment (contents perspective and context
(1.1), multimedia contents (1.2), complementary perspective).
bibliographical sources (1.3), content manage- An evaluator who is looking for quality con-
ment systems (1.4), and management of contents tents will choose course C (7.2). An evaluator
transfer (3.3). who is looking for experiences and exchange of
On the other hand, an evaluator that is context- information will choose course B (7.3).
centered (contexts perspective) will attribute a
higher weight (3) to such items as common space
of the learning community (2.1), asynchronous concLusIon
communication tools (2.2), synchronous commu-
nication tools (2.3), and management of learning Quality approaches have different perspectives
communities (3.4). and interpretations depending on its methodology
and implementation (they differ as to the goal:

E-QUAL
Figure 1. Application of e-Qual model with different perspectives (context perspective, contents perspec-
tive and equal weight)
quality policies, quality management, quality The e-Qual model has been applied to analyse
assessment, and so forth; as to the target group: several distance-learning courses.
trainees, designers, decision-makers; and as to A relevant aspect of this application was the
the method: process, product, skill, guidance). model’s flexibility in adapting to the evaluator’s
Therefore, any quality approach must be open to perspective (learners, producers, and distributors)
different values, goals and interests. and to the contents and contexts perspective. This
From the models analysed it can be concluded difference of perspective reveals in the weight
that some are unbalanced putting too much focus attributed to the different items under analysis.
on one of the e-learning components such as In fact, the results obtained by the courses were
SEEQUEL, Open eQuality, and Innolearning. very different when the evaluator’s perspective
Meca-ODL and Quality On the Line concern with was changed.
all course aspects from its design to evaluation, Another characteristic of this model is the
therefore requiring a deep knowledge of the whole possibility of being applied with some lack of
development process. This deep analysis is not information by assigning null weight to those
feasible when an applicant wishes to choose the items.
best course based on information made available In future, the distinctive characteristics in
to the public. each item must be clarified in order to provide a
Therefore, the e-Qual model appears as an more detailed referential, thus reducing the evalu-
alternative to the models proposed to analyse the ator’s subjectivity. The evaluator will thus have a
different areas judged to be relevant: learning guide, which will help him with the classification
contents, learning contexts, processes, and results. of different items.
This model prevents exhaustive analyses and also With the e-Qual model we hope to give a
allows the evaluator to distinguish between items contribution for the improvement of e-learning
through the weights assigned. quality and believe that this is the only way it can
develop and grow in a sustainable manner.

E-QUAL
Future trends Referring to relevant technical standards is

another approach for analysing e-learning quality:
The future of e-learning relies upon learner- ISO/IEC 19796 Standard on Information Technol-
centered learning in different environments and ogy—Learning, education, and training—quality
contexts with a focus on informal learning. New management, assurance, and metrics.
information and communication technologies The quality of learning objects, in turn, should
(synchronous and asynchronous) are essential to comply with relevant industry specifications such
this kind of learning, but pedagogic issues are as Sharable Content Object Reference Model
also determining for its success. Learners are (SCORM), Learning Object Metadata (LOM) or
more independent but also more responsible being AICC-CMI Guidelines for Interoperability.
capable of producing contents. A clear tendency for an integration and har-
In future, contents should also be taken into monization of both quality approaches (process
account since improvement of their quality, mainly and product) also seems to exist.
multimedia contents, requires huge investment.
Considering its reuse possibility in different con-
texts, learning objects are particularly suitable to note
solve this problem. Apart from significantly reduc-
ing development costs and promoting personal- All trademarks and registered trademarks are
ized teaching, learning objects may contribute the property of their respective owners. FuturEd,
to the consolidation of e-learning by improving Portuguese Society for Innovation, Quality E-
instruction quality without the need for teachers Learning Assurance Services Ltd. (eQCheck) are
to become technology experts. company names and property of their respective
The use of intelligent learning systems based owners. Other company, product, brand and ser-
on Artificial Intelligence to provide the learners vice names may be trademarks or service marks
with guidance on their learning process is an of others.
aspect to be considered. As technology develops
and becomes more accessible games and complex
simulations will be able to be used in the learn- reFerences
ing process.
In an Era of constant change, Rapid E-Learn- ADEIT. (2002). Meca-ODL -methodological
ing is another subject to keep an eye on because guide for the analysis of quality in open and
it reduces the lapse of time between creation and distance learning delivered via Internet. Project
content availability. Blended learning, or mixed Socrates-Minerva, European Commission.
learning, and e-learning embedded in the learner’s
AFNOR. (2004). Code of practice: Information
workflow are other contemporary trends.
technologies—E-learning guidelines. Retrieved
In what concerns quality, and following the
October 30, 2007, from http://www.fffod.org/fr/
trend of e-learning itself, there is a current trend
doc/RBPZ76001-EN.doc
that claims a leading role for the student in the
characterization of learning systems that may AFT. (2000). Distance education: Guidelines for
contribute for his/her success. This quality ap- good practice. American Federation of Teachers.
proach has deep consequences since evaluation Retrieved October 30, 2007, from http://www.aft.
must be done with basis on student’s motivation org/higher_ed/pubs-reports/reportslist.htm
and context, not on external, universal, objective
ASTD. (2006). E-learning courseware certifi-
criteria.
cation. American Society for Training & De-

E-QUAL
velopment. Retrieved October 30, 2007, from Directorate-General for Education and Culture,
http://www.astd.org/astd/marketplace/ecc European Commission. (2004). Final Report.
Study of the e-learning suppliers’ market in Eu-
ASTD. (2007). The ASTD institute e-learning
rope. Danish Technological Institute, Massy, J.,
courseware certification (ECC) standards.
Alphametrics Ltd, & Heriot-Watt University.
American Society for Training & Develop-
ment. Retrieved October 30, 2007, from EADL. (2003). Quality guide. Quality guidelines
http://workf low.ecc-astdinstitute.org/index. to improve the quality of distance learning insti-
cfm?sc=help&screen_name=cert_view tutes in Europe (2nd ed.). European Association
Distance Learning.
ASTD & NGA. (2001). A vision of e-learning
for America’s workforce. American Society for eduQua. (2005). Schweizerisches Qualitätszerti-
Training & Development. Retrieved October 30, fikat für Weiterbildungsinstitutionen. Retrieved
2007, from http://www.astd.org/NR/rdonlyres/ October 30, 2007, from http://www.eduqua.ch
8C76F61D-15FD-4C57-8554-D7E940A59009/0/
EFMD. (2007). European quality improvement
pp_ jh_ver.pdf
system. European Foundation for Management
Barker, K. (2002). Canadian recommended e- Development. Retrieved October 30, 2007, from
learning guidelines (CanREGs). FuturEd and Ca- http://www.efmd.be/equis
nadian Association for Community Education.
EFQUEL. (2006). European foundation for qual-
Bellinger, A. (2004). Good course, bad course. ity in e-learning. Retrieved October 30, 2007,
Retrieved October 30, 2007, from http:// from http://www.qualityfoundation.org
www.trainingfoundation.com/articles/default.
Ehlers, U. -D. (2004). Quality in e-learning. The
asp?PageID=1844
learner as a key quality assurance category. Eu-
Berlecon Research. (2001). Wachstumsmarkt ropean Journal Vocational Training, 29, 3-15.
e-learning: Anforderungen, akteure und per-
Ehlers, U. -D. (2007). Towards greater quality lit-
spektiven im Deutschen markt. Berlin: Berlecon
eracy in a e-learning Europe. e-Learning Papers,
Research. Retrieved October 30, 2007, from
2. Retrieved October 30, 2007, from http://www.
http://www.berlecon.de/studien/e-Learning/in-
e-Learningpapers.eu/index.php?page=doc&doc_
dex.html
id=8549&doclng=6
Connolly, M., Jones, N., & O’Shea, J. (2005).
Ehlers, U. -D., Goertz, L., Hildebrandt, B., &
Quality assurance and e-learning: Reflections
Pawlowski, J. (2005). Quality in e-learning. Use
from the front line. Quality in Higher Education,
and dissemination of quality approaches in Euro-
11(1), 59-67.
pean e-learning. A study by the European quality
Dahlgaard, J., Kristensen, K., & Khanji, G. K. observatory (CEDEFOP Panorama Series, 116).
(2005). Fundamentals of total quality manage- Luxembourg: Office for Official Publications of
ment. London: Routledge. the European Communities.
DECT. (2007). Distance education and train- Ehlers, U. -D., Hildebrand, B., Tescheler, S., &
ing council. Retrieved October 30, 2007, from Pawlowski, J. (2004). Designing tools and frame-
http://www.dect.org works for tomorrows quality development. In
Quality in European E-Learning Designing Tools
DIN. (2004). PAS 1032-1, Deutsches institut für
and Frameworks for Tomorrows Quality Develop-
normung e. V. Retrieved October 30, 2007, from
ment, Workshop European Quality Observatory
http://www.din.de

E-QUAL
(EQO) co-located to the 4th IEEE International Hollands, N. (2000). Online testing: Best practices
Conference on Advanced Learning Technologies. from the field. Retrieved October 30, 2007, from
Joensuu: European Quality Observatory. http://198.85.71.76/english/blackboard/testingad-
vice.html
EIfEL. (2007). About SEEL, Supporting ex-
cellence in e-learning. European Institute for Husson, A. (2004). Comparing quality models
e-Learning. Retrieved October 30, 2007, from adequacy to the needs of clients in e-learning. In
http://www.eife-l.org/activities/projects/seel Quality in European E-Learning Designing Tools
and Frameworks for Tomorrows Quality Develop-
ELEX. (20 05). E X EM PLO e - le a r ning
ment, Workshop European Quality Observatory
project. Retrieved October 30, 2007, from
(EQO) co-located to the 4th IEEE International
http://217.222.182.72/html/iess/pt.htm
Conference on Advanced Learning Technologies.
eQCheck. (2006). Qualite-learning assurance Joensuu: European Quality Observatory.
services ltd. Retrieved October 30, 2007, from
IITT. (2005). Institute of IT training. Retrieved
http://www.eqcheck.co.uk
October 30, 2007, from http://www.iitt.org.uk
EQO. (2007). European quality observatory.
ISO. (2003). ISO standards compendium ISO
9000 – quality management (10th ed.). Geneva:
eqo.info
International Organization for Standardization.
EQUEL. (2004). Virtual European centre in
ISO. (2005). ISO/IEC standard benchmarks
e-learning. Retrieved October 30, 2007, from
quality of e-learning. International Standard
http://equel.net
Organization. Retrieved October 30, 2007, from
EQUIPE. (2007). Aims & objectives. European http://www.iso.org/iso/en/commcentre/pressre-
Quality in Individualised Pathways in Education. leases/2006/ref992.html
Juran, J. M. (1988). Quality control handbook.
EQUIPE.up.pt/equipe/objectives.htm
New York: McGraw Hill.
European Commission. (2003). E-learning pro-
LIfIA & EIfEL. (2004). Open e-quality learning
gramme, Education. Programmes, Europe. Re-
standards. Joint e-quality committee of LIfIA
trieved October 30, 2007, from http://ec.europa.
(Learning Innovations Forum d’Innovation
eu/education/programmes/e-Learning/pro-
d’Apprentissage) and EIfEL (European Institute
gramme_en.html#
for e-Learning).
European Commission. (2005). E-learning,
Marques, C. (2004). E-learning: Uma nova forma
designing education tomorrow. Report on the
de aprender. Revista e-Ciência, 1(1), 23.
consultation workshop. The “e” for our univer-
sities—Virtual campus. Organisational changes MENON Network EEIG. (2004). Sustainable
and economics models. DRAFT. Brussels: Eu- environment for the evaluation of quality in e-
ropean Commission, Directorate-General for learning. SEEQUEL core quality framework.
Education and Culture. E-Learning Initiative, European Commission.
Figueiredo, A. D. (2002). Redes e Educação: NADE. (2001). Quality standards for distance
A Surpreendente Riqueza de um Conceito. In education. Norwegian Association for Distance
Conselho Nacional de Educação (Ed.), Redes de and Flexible Education. Retrieved October 30,
Aprendizagem, Redes de Conhecimento. Lisboa: 2007, from http://www.nade-nff.no/nff2/filer/
Conselho Nacional de Educação. Kvalitet/Kvalitetsnormer%20for%20fjernunde
rvisning.pdf

E-QUAL
ODLQC. (2006). Open and distance learning SEEL. (2004). Quality guidelines for learning
quality council standards. Open and Distance strategy and innovation, version 3. Supporting
Learning Quality Council. Retrieved October Excellence in E-Learning.
30, 2007, from http://www.odlqc.org.uk/standard.
Seufert, S., & Euler, D. (2005). Nachhaltigkeit
doc
von e-Learning-Innovationen: Fallstudien zu
Phipps, R., & Merisotis, J. (2000). Quality on the Implementierungsstrategien von e-Learning als
line: Benchmarks for success in Internet-based Innovationen an Hochschulen. St. Gallen: Swiss
distance education. Washington: IHEP - Institute Centre for Innovations in Learning. Retrieved
for Higher Education Policy. October 30, 2007, from http://www.scil.ch/pub-
lications/docs/2005-01-seufert-euler-nachhaltig-
QAA. (2004). Code of practice for the assurance
keit-e-Learning.pdf
of academic quality and standards in higher
education (2nd ed.). Mansfield: Quality Assurance SEVAQ. (2005). Self evaluation for quality in
Agency for Higher Education. e-learning. Retrieved October 30, 2007, from
http://www.sevaq.com
Qual E-Learning Project Consortium. (2003).
Qual-e-learning project. Qual E-Learning Project SPI. (2003). Empre-learning: Promoção de Es-
Consortium. Retrieved October 30, 2007, from truturas de e-Learning Inovadores, em Língua
http://www.qual-e-Learning.net/cgi/index. Portuguesa, que Permitam o Aumento de Com-
php?wpage=overview petências e Aumentem a Empregabilidade. Porto:
Sociedade Portuguesa de Inovação.
Qual E-Learning Project Consortium. (2004).
Handbook of best practices for the evaluation of TISIP. (2007). QUIS—Quality, interoperability
e-learning effectiveness. Qual e-Learning Project and standards in e-learning. Trondheim: TISIP
Consortium. Research Foundation.
Qual E-Learning Project Consortium. (2007) Ullmo, P. -A., & Ehlers, U. -D. (2007). Quality
Qual e-learning evaluation tool. Qual-e-Learn- in e-learning. E-Learning Papers, 2. Retrieved
ing Project. Retrieved October 30, 2007, from October 30, 2007, from http://www.e-Learning-
http://www.qual-e-Learning.net/cgi/index.php papers.eu/index.php?page=volume
Rekkedal, T. (2006). Distance learning and e- Wright, C. R. (2003). Criteria for evaluating the
learning quality for SMEs—State of the art. Paper quality of online courses. Retrieved October 30,
prepared for the EU Leonardo Project, E-learning 2007, from http://www.imd.macewan.ca/imd/con-
Quality for SMEs: Guidance and Counselling. tent.php?contentid=36
Ruttenbur, B., Spickler, G., & Lurie, S. (2000). E-
learning—The engine of the knowledge economy.
New York: Morgan Keegan & Co. AddItIonAL reAdIng
Scalan, C. L. (2003). Reliability and validity of a
ACE. (2001). Distance learning evaluation
student scale for assessing the quality of Inter-
guide. Washington, DC: American Council on
net-based distance learning. Distance Learning
Education.
Administration, VII (III). Retrieved October 30,
2007, from http://www.westga.edu/~distance/ Artur, A. C., Coll, J., Fagerberg, T., Leeuwen, R.
ojdla/fall63/scanlan63.html J. M. V., Liikane, K., Liivrand, A. et al. (2006).
State of the art report on e-learning quality for

E-QUAL
SMEs: An analysis of e-learning experiences in Chao, T., Saj, T., & Tessier, F. (2006). Establish-
European small and medium sized enterprises. ing a quality review for online courses. Educause
Bekkestua, Norway: ELQ-SME Project 2006. Quarterly, 29(3). Retrieved October 30, 2007,
from http://www.educause.edu/apps/eq/eqm06/
Barbera, E. (2004). Quality in virtual education
eqm0635.asp?bhcp=1
environments. British Journal of Educational
Technology, 35(1), 13-20. Concannon, F., Flynn, A., & Campbell, M. (2005).
What campus-based students think about the qual-
Barron, T. (2003). LoD survey: Quality and ef-
ity and benefits of e-learning. British Journal of
fectiveness. Learning Circuits. Retrieved Octo-
ber 30, 2007, from http://www.learningcircuits.
org/2003/may2003/qualitysurvey.htm Dam, N. V. (2004). E-quality in e-learning. Chief
Learning Officer Magazine. Retrieved October 30,
Belanger, F., & Jordan, D. (2000). Evaluation
2007, from http://clomedia.com/content/templates/
and implementation of distance learning: Tech-
nologies, tools and techniques. Hershey, PA: Idea
Group Publishing. Dondi, C., Moretti, M., Husson, A.-M., & Paw-
lowski, J. M. (2005). Providing good practice for
Bertzeletou, T. (2002). Presentation of the work
e-learning quality approaches. Interim Report:
of the European forum on quality in VET: Main
CWA 1. Project Team Quality Development.
outcomes and list of possible quality dimensions,
CEN/ISSS WSLT.
criteria and indicators on quality management
approaches (QMA), self assessment, examina- Ehlers, U.-D. (2004). Quality in e-learning from
tion and certification arrangements, quality a learner’s perspective. Paper presented at the
indicators. Retrieved October 30, 2007, from Third EDEN Research Workshop 2004, Olden-
http://www2.trainingvillage.gr/etv/quality/Sum- burg, Germany.
mary_of_the_main_results.doc
Ehlers, U.-D. (2005). What do you need for quality
Booth, A., Levy, P., Bath, P. A., Lacey, T., Sander- in e-learning? e-Learningeuropa.info. Retrieved
son, M., & Diercks-O’Brien, G. (2005). Studying October 30, 2007, from http://www.e-Learningeu-
health information from a distance: Refining an ropa.info/directory/index.php?page=doc&doc_
e-learning case study in the crucible of student id=6068&doclng=6
evaluation. Health Information and Libraries
Ehlers, U.-D. (2007). The E- empowering learn-
Journal, 22(Suppl. 2), 8-19.
ers: Myths and realities in learner-orientated
Bourne, J., & Moore, J. C. (2004). Elements of e-learning quality. E-Learning Papers, 2. Re-
quality online education into the mainstream. trieved October 30, 2007, from http://www.
Needham, MA: Sloan-C. e-Learningpapers.eu/index.php?page=doc&doc_
id=8550&doclng=6
Brittain, S., & Liber, O. (2004). A framework
for the pedagogical evaluation of e-learning Ehlers, U.-D., & Goertz, L. (2006). Handbook on
environments. Retrieved October 30, 2007, from quality and standardization in e-learning. Berlin-
http://www.cetis.ac.uk/members/pedagogy/files/ Heidelberg, Germany: Springer Verlag.
4thMeet_framework/VLEfullReport
Ehlers, U.-D., & Pawlowski, J. M. (2004). E-
Carr-Chellman, A., & Duschatel, P. (2000). The learning-quality: A decision support model for
ideal online course. British Journal of Educational European quality approaches. In G. Fietz, C. Godio
Technology, 31(3), 229-241. & R. Mason (Eds.), E-learning for international

E-QUAL
markets. Development and use of e-learning in Khan, B. H. (2005). Managing e-learning strate-
Europa. Bielefeld. gies: Design, delivery, implementation and evalu-
ation. Information Science Publishing.
EVTA. (2005). Selected e-learning in vocational
training—Good practices collection. Final re- Kidney, G., Cummings, L., & Boehm, A. (2007).
port. Brussels: European Vocational Training Toward a quality assurance approach to e-learning
Association. courses. International Journal on E-Learning,
6(1), 17-30.
Giannini-Gachago, D., Lee, M., & Thurab-Nk-
hosi, D. (2005). Towards development of best Littlejohn, A. (2005). Key issues in the design
practice guidelines for e-learning courses at the and delivery of technology-enhanced
University of Botswana. In Proceedings of the
learning. In P. Levy & S. Roberts (Eds.), Develop-
IASTED International Conference on Computers
ing the new learning environment: The chang-
and Advanced Technology for Education (CATE
ing role of the academic librarian (pp. 70-90).
2005), Aruba.
London: Facet.
Govindasamy, T. (2001). Successful implementa-
Lockee, B., Moore, M., & Burton, J. (2002). Mea-
tion of e-learning: Pedagogical considerations.
suring success: Evaluation strategies for distance
The Internet and Higher Education, 4(3), 287-
education. Educause Quarterly, 1. Retrieved
299.
October 30, 2007, from http://www.educause.
Heldberg, J. G. (2003). Ensuring quality e-learn- edu/ir/library/pdf/EQM0213.pdf
ing creating engaging tasks. Educational Media
Macpherson, A., Elliot, M., Harris, I., & Homan,
International, 40(3-4), 175-186.
G. (2004). E-learning: Reflections and evaluation
Herrington, A., Herrington, J., Oliver, R., Stoney, of corporate programmes. Human Resource De-
S., & Willis, J. (2001). Quality guidelines for on- velopment International, 7(3), 295-313.
line courses: The development of an instrument
Mandinach, E. B. (2005). The development of
to audit online units. In G. Kennedy, M. Keppell,
effective evaluation methods for e-learning: A
C. McNaught & T. Petrovic (Eds.), Meeting at
concept paper and action plan. Teachers College
the Crossroads: Proceedings of ASCILITE 2001
Record, 107(8), 1814-1835.
(pp. 263-270). Melbourne: The University of
Melbourne. Massy, J. (2002). Quality and e-learning in Eu-
rope. Reading: BizMedia Ltd.
Hicks, S. (2000, December). Evaluating e-learn-
ing. Training and Development, 77-79. McGorry, S. Y. (2003). Measuring quality in online
programs. The Internet and Higher Education,
Hughes, J., & Attwell, G. (2002). A framework
6, 159-177.
for the evaluation of e-learning. Paper presented
at European Seminars—Exploring Models and McNaught, C., & Lam, P. (2005). Building an eval-
Partnerships for e-Learning in SMEs, Scotland uation culture and evidence base for e-learning in
and Brussels, Belgium. three Hong Kong universities. British Journal of
Keppell, C. McNaught & T. Petrovic (Eds.), Meet-
ing at the crossroads: Proceedings of ASCILITE Melton, R. (2002). Planning and developing open
2001 (pp. 263-270). Melbourne, Australia: The and distance learning: A framework for quality.
University of Melbourne. London: RoutledgeFalmer.

E-QUAL
Meyer, K. A. (2002). Quality in distance education: Tulloch, J. B., & Sneed, J. R. (2000). Quality
Focus on online learning. ASHE-ERIC Higher enhancing practices in distance education: Teach-
Education Report, 29(4), 1-121. ing and learning. Washington, DC: Instructional
Telecommunications Council.
Nicolaou, C. T., Nicolaidou, I. A., & Constan-
tinou, C. P. (2005). The e-learning movement Welber, M. (2002). How AT&T adapted Kirk-
as a process of quality improvement in higher patrick’s evaluation tools to e-learning then
education. Educational Research and Evaluation, applied the same rigor to selecting vendors. E-
11(6), 605-622. Learning, 3(6), 1-3.
Pawlowski, J. M. (2006). Adopting quality stand- Wild, R. H., & Hope, B. (2003). DATQUAL: A
ards for education. CEDEFOP. prototype e-learning application to support quality
management practices in service industries. TQM
Pond, W. K. (2002). Twenty-first century education
& Business Excellence, 14(6), 695-713.
and training: Implications for quality assurance.
The Internet and Higher Education, 4, 185-192. Willging, P. A. (2004). Factors that influence
students’ decision to drop out of online courses.
Rovai, A. F. (2003). A practical framework for
The Journal of Asynchronous Learning Networks,
evaluating online distance education programs.
8(4). Retrieved October 30, 2007, from http://www.
The Internet and Higher Education, 6(2), 109-
sloan-c.org/publications/jaln/v8n4/v8n4_will-
124.
ging_member.asp
Rumble, G. (2000). The globalisation of open and
Wirth, M. A. (2005). Quality management in e-
flexible learning: Considerations for planners and
learning: Different paths, similar pursuits. Paper
managers. Online Journal of Distance Learning
presented at 2nd International SCIL Congress.
Administration, 3(3). Retrieved October 30, 2007,
from http://www.westga.edu/~distance/ojdla/
scil.ch/congress-2005/programme-10-11/docs/
fall33/rumble33.html
workshop-1-wirth-text.pdf
Schifter, C., Greenwood, L., & Monolescu, D.
(2004). The distance education evolution: Issues
and case studies. Information Science Publish-
endnotes
ing.
1
Shifrin, T. (2006, March 7). International stand- A test used to measure performance.
2
ard aimed at improving quality of e-learning. Content can be used in different learning
Computer Weekly, 76. contexts with different goals.
3
Data transfer capacity or speed of transmis-
Sonwalkar, N. (2002). A new methodology for
sion of a digital communications system.
evaluation: The pedagogical rating of online
courses. Syllabus, 15(6), 18-21.

0
Compilation of References
ADEIT. (2002). Meca-ODL -methodological guide for Albano, G. (2006). A case study about mathematics and
the analysis of quality in open and distance learning e-learning: First investigations. In International Com-
delivered via Internet. Project Socrates-Minerva, Eu- mission for the Study and Improvement of Mathematics
ropean Commission. Education 58 Congress, Changes in Society: A Chal-
lenge for Mathematics Education (pp. 146-151). Plezeň:
AFNOR. (2004). Code of practice: Information technolo-
University of West Bohemia Press.
gies—E-learning guidelines. Retrieved October 30, 2007,
from http://www.fffod.org/fr/doc/RBPZ76001-EN.doc Albano, G., Bardelle, C., & Ferrari, P. L. (2007). The
impact of e-learning on mathematics education: Some
AFT. (2000). Distance education: Guidelines for good
experiences at university level. La matematica e la sua
practice. American Federation of Teachers. Retrieved
didattica, 21(1), 61-66.
October 30, 2007, from http://www.aft.org/higher_ed/
pubs-reports/reportslist.htm Albano, G., Gaeta, M., & Salerno, S. (2006). E-learning:
A model and process proposal. International Journal of
Agra, M. J., Gewerc, A., & Montero, M. L. (2003). El
Knowledge and Learning, 2(1/2), 73-88.
portafolios como herramienta de análisis en experien-
cias de formación online y presenciales. Enseñanza, Alexander, B. (2006). Web 2.0: A new wave of innova-
23, 101-114. tion for teaching and learning? EDUCAUSE Review,
41(2), 32-44.
Agra, M.J., Gewerc, A., & Montero, M.L. (2002). El
portafolios como herramienta de análisis en experiencias Allen, I. E., & Seaman, J. (2004). Sizing the opportunity:
de formación online y presenciales II Congreso Europeo The quality and extent of online education in the US,
de Tecnologías de la Información en la Educación y en 2002 and 2003. Needham, MA: Sloan-C.
la Ciudadanía. Barcelona.
Allen, T. D., McManus, S. E., & Russell, J. E. A. (1999).
Ahmad, A., Basir, O., & Hassanein, K. (2004). Adaptive Newcomer socialization and stress: Formal peer rela-
user interfaces for intelligent e-learning: Issues and tren- tionships as a source of support. Journal of Vocational
ds. In Proceedings of the 4th International Conference on Behaviour, 54(3), 453-470.
Electronic Business (ICEB2004) (pp. 925-934).
Anderson, E.M., & Shannon, A.L. (1995). Towards a
Albano, G. (2005). Mathematics and e-learning: Stu- conceptualization of mentoring. In T. Kerry & A.S. Mayes
dents’ beliefs and waits. In International Commission (Eds.), Issues in mentoring. London: A.S. Routledge.
for the Study and Improvement of Mathematics Educa-
Anderson, G. (2006). E-learning 2.0 is about people.
tion 57 Congress, Changes in Society: A Challenge for
Konferenz Professionelles Wissensmanagement - Erfah-
Mathematics Education (pp. 153-157). Piazza Armerina:
rungen und Visionen Live von der ICL 2006». Retrieved
Università di Palermo Press.
October 25, 2007, from http://elearningblog.tugraz.
at/archives/130
Argyris, C. (2004). Double-loop learning and imple- Augar, N., Raitman, R., & Zhou, W. (2004). Teaching and
mentable validity. In H. Tsoukas & N. Mylonopoulos learning online with wikis. Paper presented at the ASCI-
(Eds.), Organizations as knowledge systems: Knowledge, LITE Australasian Society for Computers in Learning in
learning, and dynamic capabilities (pp. 29-45). New Tertiary Education 2004 Conference. Perth, WA.
York: Palgrave Macmillan.
Avgerou, C. (2001). The significance of context in infor-
Argyris, C., & Schon, D.A. (1996). Organizational mation systems and organizational change. Information
learning II. Reading, MA: Addison-Wesley Publishing Systems Journal, 11, 43-63.
Company.
Avgerou, C., & Madon, S. (2004). Framing IS studies:
Armitage, S., & O’Leary, R. (2003). A guide for learning Understanding the social context of IS innovation. In C.
technologists. Learning and teaching support network. Avgerou, C. U. Cibbora & F. F. Land (Eds.), The social
York, UK: LTSN Generic Center. study of ICT (pp. 162-182). Oxford: Oxford University
Press.
Asgari, M., & O’Neill, K. (2004). What do they mean by
success? Contributors to perceived success in a telemen- Avison, D., & Fitzgerald, G. (2003). Information systems
toring program for adolescents. Paper presented at the development: Methodologies, techniques and tools (3rd
Annual Meeting of the American Educational Research ed.). Maidenhead: McGraw-Hill.
Association, San Diego, CA.
Avison, D.E., & Wood-Harper, A.T. (1990). Multiview:
Ashton, H.S., Beevers, C. E., Milligan, C. D., Schofield, D. An exploration in information systems development.
K., Thomas, R. C., & Youngson, M. A., (2006). Moving Henley-on-Thames: Alfred Waller.
beyond objective testing in online assessment. In S.L.
Ayersman, D. J. (1996). Reviewing the research on
Howell & M. Hricko (Eds.), Online assessment & meas-
hypermedia-based learning. Journal of Research on
urement. Case studies from higher education, K-12 and
Corporate (pp. 116-128). Hershey, PA: Idea Group.
Balacheff, N. (2000). Teaching, an emergent property
Assessment Reform Group. (1999). Assessment for learn-
of e-learning environments. The Information Society
ing: Beyond the black box. Cambridge, UK: University
for All. (IST 2000). Retrieved October 21, 2007, from
of Cambridge School of Education. Retrieved October
http://www-didactique.imag.fr/Balacheff/TextesDiv-
29, 2007, from http://arg.educ.cam.ac.uk/AssessInsides.
ers/IST2000.html
pdf
Balacheff, N., & Sutherland, R. (1999). Didactical com-
ASTD & NGA. (2001). A vision of e-learning for
plexity of computational environments for the learning
America’s workforce. American Society for Training
of mathematics. International Journal of Computers for
& Development. Retrieved October 30, 2007, from
Mathematical Learning, 4, 1-26.
http://www.astd.org/NR/rdonlyres/8C76F61D-15FD-
4C57-8554-D7E940A59009/0/pp_ jh_ver.pdf Baldacci, M. (1999). L’individualizzazione. Basi psi-
copedagogiche e didattiche. Bologna: Pitagora.
ASTD. (2006). E-learning courseware certification.
American Society for Training & Development. Retrieved Banathy, B.H. (1996). Systems inquiry and its applica-
October 30, 2007, from http://www.astd.org/astd/mar- tion in education. In D.H. Jonassen (Ed.), Handbook of
ketplace/ecc research for educational communications and technol-
ogy (pp. 74-92). New York: Prentice Hall.
ASTD. (2007). The ASTD institute e-learning course-
ware certification (ECC) standards. American Society Banathy, B.H. (1999). Systems thinking in higher educa-
for Training & Development. Retrieved October 30, tion: Learning comes to focus. Systems Research and
2007, from http://workflow.ecc-astdinstitute.org/index. Behavioral Science, 16, 133-145.
cfm?sc=help&screen_name=cert_view

Banta, T. W. (Ed.). (2003). Portfolio assessment: Uses, for reading. In 9th International Conference on User
cases, scores and impact. San Francisco: Jossey-Bass. Modelling (pp. 303-312).
Barberà, E. (2006). Los fundamentos teóricos de la BECTA. (2006). Retrieved October 29, 2007, from www.
tutoría presencial y en línea: Una perspectiva socio- becta.org.uk
constructivista. In J.A. Jerónimo Montes & E. Aguilar
Behrens, J.T., Collison, T.A., & DeMark, S. (2006). The
Rodríguez (Eds.), Educación en red y tutoría en línea
seven C’s of comprehensive online assessment: Lessons
(pp. 161-180). Mexico: UNAM FES-Z.
learned from 36 million classroom assessments in the
Barberà, E., & Badia, A. (2004). Educar con aulas vir- Cisco networking academy program. In S.L Howell & M.
tuales. Orientaciones para la innovación en el proceso Hricko (Eds.), Online assessment and measurement. Case
de enseñanza y aprendizaje. Barcelona: A. Machado studies from higher education, K-12 and corporate (pp.
Libros. 229-245). London: Information Science Publishing.
Barker, K. (2002). Canadian recommended e-learning Beitler, M. A., & Frady, D. A. (2002). E-learning and
guidelines. CACE. Retrieved October 19, 2007, http:// e-support for expatriate managers. In H. B. Long &
www.futured.com/pdf/CanREGs%20Eng.pdf Associates (Eds.), Twenty-first century advances in self-
directed learning (CD). Boynton Beach, FL: Motorola
Barker, K., Trafalis, T., & Rhoads, T.R. (2004). Learn-
University.
ing from student model. In System and Information
Engineering Design Symposium (pp. 79-86). Bellinger, A. (2004). Good course, bad course. Retrieved
October 30, 2007, from http://www.trainingfoundation.
Barnes, K., Marateo, R., & Ferris, S. (2007). Teaching
com/articles/default.asp?PageID=1844
and learning with the net generation. Innovate, 3(4).
Retrieved October 18, 2007, from http://www.innova- Berlecon Research. (2001). Wachstumsmarkt e-learning:
teonline.info/index.php?view=article&id=382 Anforderungen, akteure und perspektiven im Deutschen
markt. Berlin: Berlecon Research. Retrieved October 30,
Barragán Sánchez, R. (2005). El Portafolio, metodología
2007, from http://www.berlecon.de/studien/e-Learn-
de evaluación y aprendizaje de cara al nuevo Espacio
ing/index.html
Europeo de Educación Superior. Una experiencia práctica
en la Universidad de Sevilla, Revista Latinoamericana Berman, S. H., & Pape, E. (2001). A consumer’s guide to
deTecnología Educativa, 4(1), 121-139. Retrieved October online courses. School Administrator, 58(9), 14.
28, 2007, from http://www.unex.es/didactica/RELATEC/
Bernabé, A. (2004). Blended learning. Conceptos
sumario_4_1.htm
básicos. Pixel-Bit. Revista de Medios y Educación, 23,
Bastiaens, T., & Martens, R. (2000). Conditions for 7-20. Retrieved October 18, 2007, from www.lmi.ub.es/
Web-based learning with real events. In B. Abbey (Ed.), personal/bartolome/articuloshtml/04_blended_learn-
Instructional and cognitive impacts of web-based educa- ing/documentacion/1_bartolome.pdf
tion (pp. 1-32). London: Idea Group Publishing.
Bezdek, J.C., & Pal, S.K (1992). Fuzzy models for pattern
Bayne, R. (1995). MBTI: A critical review. London: recognition: Methods that search for structures in data.
Chapman & Hall. New York: IEEE Press.
Beck, J.E., & Woolf, B.P. (1998). Using a learning agent Bierema, L. L., & Merriam, S. B. (2002). E-mentoring:
with a student model. Lecture Notes in Computer Sci- Using computer mediated communication to enhance
ence, 1452, 6-15. the mentoring process. Innovative Higher Education,
26(3), 211-227.
Beck, J.E., Jia, P., Sison, J., & Mostow, J. (2003). Predict-
ing student help-request behavior in an intelligent tutor

Biggs, J. (1996). Enhancing teaching through constructive Blurton, C. (2000). New directions of ICT-use in educa-
alignment. Higher Education, 32, 347-364. tion. UNESCO World Communication and Information
Report. Paris: UNESCO.
Biggs, J. (1998). Assessment and classroom learning: A
role for summative assessment? Assessment in Educa- Boley, H. (2003). RACOFI: A rule-applying collaborative
tion, 5(1), 103-110. filtering system. In 2003 IEEE/WIC International Confer-
ence on Web Intelligence/Intelligent Agent Technology.
Biggs, J. (1999). Teaching for quality learning at uni-
versity: What the student does. Buckingham: Open Bonabeau, E., Dorigo, M., & Theraulaz, G. (1999).
University Press. Swarm intelligence: From natural to artificial Systems.
NY: Oxford University Press
Biggs, J., & Tang, C. (1997). Assessment by portfolio:
Constructing learning and designing teaching. Research Booth, R., & Berwyn, C. (2003). The development of
and Development in Higher Education, 79-87. quality online assessment in vocational education and
training. Leabrook, Australia: Australian Flexible Learn-
Biggs, J.B. (2003). Teaching for quality learning at
ing Framework. Retrieved October 29, 2007, from www.
university (2nd ed.). Buckingham: SRHE & Open Uni-
ncver.edu.au/research/proj/nr1F02_1.pdf
versity Press.
Borst, W.N. (1997). Construction of engineering on-
Bishop, J. (2006, February). Training talk newsletter.
tologies for knowledge sharing and reuse. University
Retrieved October 16, 2007, from http://www.dest.gov.
of Twenty, NL-Centre for Telemática and Information
au/sectors/training_skills/publications_resources/train-
Technology.
ingtalk/issue_20/
Boud, D., & Falchikov, N. (1989). Quantitative studies of
Black, P., & William, D. (1998). Assessment and class-
self-assessment in higher education: A critical analysis
room learning, Assessment in Education, 5(1), 7-74.
of findings. Higher Education, 18(5), 529-549.
Blackboard. (2006). Blackboard unveils blackboard
Boud, D., Cohen, R., & Sampson, J. (1999). Peer learning
beyond initiative. Four bold inaugural projects will
and assessment. Assessment and Evaluation in Higher
advance e-learning 2.0 vision. Retrieved October 25,
Education, 24(4), 413-426.
2007, from http://www.blackboard.com/company/press/
release.aspx?id=823603 Boyd-Barrett, O. (2000). Distance education provision
by universities: How institutional context affect choices.
Blair, A. (2004, May 3). Speech to NAHT conference.
Information Communication & Society, 3(4), 474-493.
Retrieved October 16, 2007, from http://www.number10.
gov.uk/output/Page5730.asp Brahm, C., & Kleiner, B. H. (1996). Advantages and
disadvantages of group decision-making approaches.
Blamire, R. (2006). Insight blog. The online diary of
Team Performance Management, 2(1), 30-35.
European schoolnet’s insight team. Retrieved October
25, 2007, from http://blog.eun.org/insightblog/2006/06/ Bransford, J.D., Brown, A.L., & Cocking, R.R. (Eds.).
elearning_20.html (1999). How people learn: Brain, mind, experience and
school. Washington: National Academic Press.
Blinco, K., Mason, J., McLean, N., & Wilson, S. (2004,
July 19). Trends and issues in e-learning infrastructure Brian, L. (2004) Taking a walk on the wiki side. Campus
development. A White paper for alt-i-lab 2004, pre- Technology. Retrieved October 25, 2007, from http://
pared on behalf of DEST (Australia) and JISC-CETIS www.campustechnology.com/article.asp?id=9200
(UK) (Version 2). Retrieved October 19, 2007, from
Briggs, J. (2003). Rich pictures of UK education. Re-
http://www.jisc.ac.uk/uploaded_documents/Altilab04-
trieved October 16, 2007, from http://www.reengage.
infrastructureV2.pdf
org/go/Article_111.html

Brightman, H. J. (2006). GSU master teacher program: Bruner, J. (1998). Desarrollo cognitivo y educación.
On critical thinking. Retrieved October 26, 2007, from Madrid, Spain: Morata.
http://www2.gsu.edu/~dschjb/wwwcrit.html
Bryndum, S., & Montes, J. A. (2005). La motivación en
Britain, S. (2004, May). A review of learning design: los entornos telemáticos. RED Revista de Educación
Concept, specifications and tools. JISC. Retrieved a Distancia, V(13). Retrieved October 26, 2007, from
October 19, 2007, from http://www.jisc.ac.uk/uploa- http://www.um.es/ead/red/13/
ded_documents/ACF1ABB.doc
Buchanan, T. (2000). The efficacy of a world-wide Web
Brookhart, S. M. (2001). Successful students’ formative mediated formative assessment. Journal of Computer
and summative use of assessment information. Assess- Assisted Learning, 16, 193-200.
ment in Education, 8(2), 153-169.
Bull, J. (1999). Computer-assisted assessment: Impact on
Brooks, B. A., & Madda, M. (1999). How to organize a higher-education institutions. Educational Technology
professional portfolio for staff and career development. & Society, 2(3), 123-126.
Journal for Nurses in Staff Development, 15(1), 5-10.
Bull, J., & Mckenna, C. (2001). Blueprint for CAA.
Brousseau, G. (1997). Theory of didactical situations in Loughborough: University of Loughborough.
mathematics. Kluwer Academics Publisher.
Burd, B. A., & Buchanan, L. E. (2004). Teaching the
Brown, E., Cristea, A., Stewart, C., & Brailsford, T. teachers: Teaching and learning online. Reference Serv-
(2005). Patterns in authoring of adaptive educational ices Review, 32(4), 404-412.
hypermedia: A taxonomy of learning styles. Educational
Burkhalter, B.B. (1996). How can institutions of higher
Technology & Society, 8(3), 77-90. Retrieved October 26,
education achieve quality within the new economy? Total
2007, from http://www.ifets.info/journals/8_3/8.pdf
Quality Management, 7, 153-160.
Brown, E., Gibbs, G., & Glover, C. (2003). Evaluation
Burr, J.T. (1993, March). A new name for a not so new
tools for investigating the impact of assessment regimes
concept. Quality Progress, 87-88.
on student learning. Bioscience Education E-Journal,
2. Retrieved October 28, 2007, from http://bio.ltsn. Burr, L., & Spennemann, D. H. R. (2004). Patterns of
ac.uk/journal/vol2/beej-2-5.htm user behaviour in university online forums. Internatio-
nal Journal of Instructional Technology and Distance
Brown, G., Bull, J., & Pendleburg, M. (1997). Assesing stu-
Learning, 1(10), 11-28.
dent learning in higher education. London: Routledge.
Buzzetto-More, N. A., & Pinhey, K. (2006). Guidelines
Brown, J. S., & Duguid, P. (1998). Universities in the
and standards for the development of fully online lear-
digital age. In B. L. Hawkins & P. Battin (Eds.), The mi-
ning objects. Interdisciplinary Journal of Knowledge
rage of continuity: Reconfiguring academic information
and Learning Objects, 2, 95-104.
resources for the 21st century (pp. 39-60). Washington,
DC: Council on Library and Information Resources. Cabena, P., Hadjnian, P., Stadler, R., Verhees, J., & Zanasi,
A. (1997). Discovering data mining: From concept to
Brown, S., & Glasner, A. (Ed.). (1999). Assessment matters
implementation (IBM Books). Pearson Education.
in higher education. UK: Open University Press.
Cabero, J. (2004). Bases pedagógicas del e-Learning.
Brown, S., & Knight, P. T. (1994). Assessing learners in
Revista de Universidad y Sociedad del conocimiento,
higher education. London: Kogan Page.
3. Retreived October 18, 2007, from http://www.uoc.
Bruner, J. (1997). La educación, puerta de la cultura. edu/rusc
Madrid, Spain: Visor.

Cabero, J. (2006). Comunidades virtuales para el apre- Carruthers, J. (1993). The principles and practices of
ndizaje. Su utilización en la enseñanza. Edutec, 20. mentoring. In B.J. Caldwell & E.M.A. Carter (Eds.), The
Retrived October 18, 2007, from http://www.uib.es/de- return of the mentor: Strategies for workplace learning.
part/gte/gte/edutec-e/revelec20/cabero20.htm London: Falmer Press.
Calder, A. (2004). Online learning support: An action Cashion, J., & Palmieri, P. (2000). Quality in online
research project. James Cook University. Paper presented learning: Learners views. Retrieved October 19, 2007,
at 4th Pacific Rim First Year Experience Conference from http://flexiblelearning.net.au/nw2000/talkback/
at Queensland University of Technology. Brisbane, p14-3.htm
Australia.
Castro, E. (2007). Moodle: Manual del professor. Re-
Camacho, D., & R-Moreno, M.D. (2007). Towards and trieved October 28, 2007, from http://moodle.org/file.
automatic monitoring for higher education learning php/11/manual_del_profesor/Manual-profesor.pdf
design. International Journal of Metadata, Semantics,
Cavanaugh, C. (2002). Distance education quality: The
and Ontologies, 2(1), 1-10.
resources-practices-results cycle and the standards.
Canós, L., & Mauri, J. J. (2005). Metodologías Activas Retrieved October 19, 2007, from http://www.unf.
para la Docencia y Aplicación de las Nuevas Tecnologías: edu/~caavanau/2569.htm
una Experiencia. In URSI 2005. Retrieved October 26,
Cave, M., Hanney, S., Kogan, M., & Trevett, G. (1988).
2007, from http://w3.iec.csic.es/ursi/articulos_gan-
The use of performance indicators in higher education:
dia_2005/articulos/otros_articulos/462.pdf
A critical analysis of developing practice. London: Jes-
Caragea, D., Pathak, J., & Honavar, V. (2004). Learning sica Kingslay.
classifiers from semantically heterogeneous data. Lecture
Cecez-Kecmanovic, D., & Webb, C. (2000). Towards a
Notes in Computer Science, 3291, 963-980.
communicative model of collaborative Web-mediatic
Carlsen, W., & Single, P. B. (2000). Factors related learning. Australian Journal of Educational Technol-
to success in electronic mentoring of female college ogy, 16(1), 73-85.
engineering students by mentors working in industry.
Chadwick, P.A. (1994). University’s TQM initiative. In
Paper presented at the Annual Meeting of the National
P. Nightingale & M. O’Neill (Eds.), Achieving quality
Association for Research in Science Teaching, New
learning in higher education (pp. 120-135). London:
Orleans, LA.
Kogan Page.
Carro, R.M, Ortigosa, A., & Schlichter, J. (2003). A
Challis, D. (2005). Committing to quality learning
rule-based formalism for describing collaborative
through adaptive online assessment. Assessment &
adaptive courses, KES2003. Lecture Notes in Artificial
Evaluation in Higher Education, 30(5), 519-527.
Intelligence, 2774, 147-178.
Challis, D. (2005, Fall). Towards the mature e-portfo-
Carro, R.M, Pulido, E., & Rodríguez, P. (1999). Design-
lio: Some implications for higher education. Canadian
ing adaptive Web-based courses with TANGOW. In G.
Journal of Learning and Technology, 31(3).
Cumming, T. Okamoto & L. Gómez (Eds), Advanced
research in computers and communications in education Chang, C. C. (2002, March). Building a Web-based learn-
(pp. 147-178). Amsterdam: IOS Press. ing portfolio for authentic assessment. Paper presented
at Proceedings International Conference on Computers
Carro, R.M., Pulido, E., & Rodríguez, P. (1999). Dynamic
in Education (ICCE’02), Melbourne, Australia.
generation of adaptive Internet-based courses. Journal of
Network and Computer Applications, 22, 249-257. Chang, L. J., Yang, J. C., Yu, F. Y., & Chan, T. W. (2003).
Development and evaluation of multiple competitive

activities in a synchronous quiz game system. Journal Chiero, T. C. (1997). Teachers’ perspectives on factors that
of Innovations in Education and Training International. affect computer use. Journal of Research on Computing
40(1), 16-26. in Education, 30(2), 133-145.
Chang, S. -B., Wang, H. -Y., Liang, J. -K., Liu, T. -C., Christensen, E. W., Anakwe, U. P., & Kessler, E. H.
& Chan, T. W. (2004). A contest event in the connected (2001). Receptivity to distance learning: The effect of
classroom using wireless handheld devices. In J. Ro- technology, reputation, constraints, and learning prefer-
schelle, T.-W. Chan, Kinshuk, & S. J. H. Yang (Eds.), ences. Journal of Research on Computing in Education,
Proceedings of the 2nd IEEE International Workshop 33(3), 263-370.
on Wireless and Mobile Technologies in Education
Chu, K., Chang, M., & Hsia, Y. (2004). Stimulating stu-
(WMTE 2004) (pp. 207-208). Los Alamitos, CA: IEEE
dents to learn with accuracy counter based on competitive
Computer Society.
learning. In Proceedings of the IEEE International Con-
Charman, D. (2005). Issues and impacts of using com- ference on Advanced Learning Technologies (ICALT’04)
puter-based assessments (cbas) for formative assessment. (pp. 786-788). IEEE Computer Society.
In S. Brown, J. Bull & P. Race (Eds.), Computer-assisted
Cichocki, A., & Amari, S. (2001). Adaptive blind signal
assessment in higher education (pp. 85-93). Eastbourne:
and image processing: Learning algorithms and applica-
Routledge.
tions. New York: John Wiley & Sons.
Checkland, P. (1981). SystemsThinking, Systems practice.
Clariana, R. B. (1997). Considering learning style in
Chichester: John Wiley.
computer-assisted learning. British Journal of Educa-
Checkland, P. (2000). Soft systems methodology: A 30- tional Technology, 28(1), 66-68.
year retrospective. Systems Research and Behavioral
Clark, R.E. (2001). New directions: Evaluating distance
Science, 17, S11-S58.
education technologies. In R.E. Clark (Ed.), Learning
Checkland, P., & Holwell, S. (1998). Information, systems from media: Arguments, analysis, and evidence (pp. 125-
and information systems. John Wiley and Sons. 136). Greenwich, CT: Information Age Publishing.
Checkland, P., & Scholes, J. (1990). Soft systems meth- Coffield, F., Moseley, D., Hall, E., & Ecclestone, K.
odology in action. John Wiley & Sons. (2004). Learning styles and pedagogy in post-16 learn-
ing: A systematic and critical review. London: Learning
Cheng, Y. C. (2002). Linkage between innovative man-
and Skills Research Centre.
agement and student-centred approach: Platform theory
for effective learning. Paper presented at the Second Cohen, M. D., & March, J. D. (1974). Leadership and
International Forum on Education Reform: Key Factors ambiguity: The American college president. New York:
in Effective Implementation, Bangkok, Thailand. McGraw-Hill.
Cheng, Y.C. (1996). The pursuit of school effectiveness: Coll, C., Martín, E., Mauri, T., Miras, M., Onrubia, J.,
Theory, policy, and research. Hong Kong: Hong Kong Solé, I., et al. (2005). El constructivismo en el aula, Vol.
Institute of Educational Research, The Chinese Univer- 111 (15th ed.). Barcelona: Graó.
sity of Hong Kong.
Collis, B., De-Boer, W., & Slotman, K. (2001). Feedback
Chickering, A. W., & Ehrmann, S. C. (1996, October). for Web-based assignments. Journal of Computer As-
Implementing the seven principles: Technology as lever. sisted Learning, 17, 306-313.
American Association for Higher Education Bulletin,
Comeaux, P. (Ed.). (2002). Communication and col-
49(2), 3-6.
laboration in the online classroom: Examples and ap-
plications. Bolton, MA: Anker.

Comezaña, O., & García, F. J. (2005). Plataformas para (Eds.), Working for excellence in the e-conomy (pp.
educación basada en Web: Herramientas, procesos de 417-426). Perth, WA: We-B Research Centre, Edith
evaluación y seguridad (Tech Rep. DPTOIA-IT-2005- Cowan University.
001). España, Salamanca: Universidad de Salamanca,
Cooper, T., & Love, T. (2001). Online portfolios: Issues
Departamento de Informática y Automática.
of assessment and pedagogy. In International Education
Computer Adaptive Assessment Project. (2005). What Research Conference, Melbourne. Retrieved October
is CAA? Retrieved October 29, 2007, from http://www. 29, 2007, from http://www.aare.edu.au/01pap/coo01346.
castlerockresearch.com/caa/WhatisCAA.aspx htm
Computing: Work-Life Balance. (2007). The Economist, Cooper, T., & Love, T. (2002). Online portfolios: Issues
23/12/06-5/1/07, 99-100. of assessment and pedagogy. In P. Jeffrey (Ed.), AARE
2001: Crossing borders: New frontiers of educational
Connolly, M., Jones, N., & O’Shea, J. (2005). Quality
research. Coldstream, Victoria: AARE Inc.
assurance and e-learning: Reflections from the front line.
Quality in Higher Education, 11(1), 59-67. Cooper, T., Hutchins, T., & Sims, M. (1999). Developing
a portfolio which demonstrates competencies. In M. Sims
Conway, C. (1998). Strategies for mentoring: A blueprint
and T. Hutchins (Eds.), Learning materials: Certificate
for successful organizational development. New York:
in children’s Services; 0-6 years (bilingual support)
John Wiley and Sons.
(pp. 3-29). Perth, WA: Ethnic Childcare Resource Inc.
Cookson, P. (2002). The hybridization of higher education. Western Australia.
International Review of Research in Open and Distance
Costa, D., Hertz, A., & Dubious, O. (1995). Embed-
Learning, 2(2), 1-4.
ding of a sequential algorithm within an evolutionary
Cooper, T. (1996). Portfolio assessment in higher edu- algorithm for coloring problems in graphs. Journal of
cation. In Proceedings Western Australia Institute for Heuristics, 1, 105-128.
Educational Research Forum 1996. Retrieved October
Council for Higher Education Accreditation. (2002).
28, 2007, from http://www.waier.org.au/forums/1996/
International quality review. Retrieved October 19, 2007,
cooper.html
from http://www.chea.org/international/inter_summa-
Cooper, T. (1997). Portfolio assessment: A guide for ry02.html
students. Perth, WA: Praxis Education.
Covington, M. V. (1992). Making the grade: A self-worth
Cooper, T. (1999). Portfolio assessment: A guide for perspective on motivation and school reform. Cambridge,
lecturers teachers and course designers. Perth, WA: UK: Cambridge University Press.
Praxis Education.
Crompton, P. (1999). Evaluation: A practical guide to
Cooper, T., & Emden, C. (2000). Portfolio assessment: methods. Retrieved October 29, 2007, from http://www.
A guide for nurses and midwives. Perth, WA: Praxis icbl.hw.ac.uk/ltdi/implementing-it/eval.pdf
Education.
Cronbach, L., & Snow, R. (1977). Aptitudes and instruc-
Cooper, T., & Love, T. (2000). Portfolios in university- tional methods: A handbook for research on interactions.
based design education. In C. Swann & E. Young (Eds.), New York: Irvington.
Re-inventing design education in the university (pp. 159-
Crosby, P.B. (1979). Quality is free. New York: Mc-
166). Perth, WA: School of Design, Curtin University.
Graw-Hill.
Cooper, T., & Love, T. (2001). Online portfolio assess-
Curtis, J. B. (2002). Collaborative tools for e-learn-
ment in information systems. In S. Stoney & J. Burn
ing. Chief Learning Office. Solutions for Enterprise

Productivity. Retrieved October 29, 2007, from http:// Dewart, H., Drees, D., Hixenbaugh, P., & Williams, D.
www.clomedia.com/content/templates/clo_feature. (2004, April 5-7). Electronic peer mentoring: A scheme to
asp?articleid=41&zoneid=30 enhance support and guidance and the student learning
experience. Paper presented at the Psychology Learning
Dahlgaard, J., Kristensen, K., & Khanji, G. K. (2005).
and Teaching Conference, University of Strathclyde,
Fundamentals of total quality management. London:
Glasgow, UK.
Routledge.
Dewey, J. (1933). How we think. Boston, MA: Heath.
Dahlgaard, J.J., Kristensen, K., & Kanji, G.K. (1995).
TQM and education. Total Quality Management, 6(5- Dewey, J. (1938). Experience and education. New York:
6). Macmillan.
Daniels, S.E. (2002). First to the top. Quality Progress, DfES. (2003). Widening participation in higher educa-
35(5), 41-53. tion. London: Department for Education and Skills.
Davies, M. (2001). Adaptive AHP: A review of marke- Di Martino, P., & Zan, R. (2002). An attempt to describe
ting applications with extensions. European Journal of a negative attitude toward mathematics. In P. Di Mar-
Marketing, 35(7), 872-893. tino (Ed.), Proceedings of the Mathematics Views—XI
European Workshop: Research on Mathematical Beliefs
Davis, F. D. (1989). Perceived usefulness, perceived
(pp. 22-29). Pisa: Università di Pisa Press.
ease of use and user acceptance of information tech-
nology. Management Information Systems Quarterly, Diaz, D., & Cartnal, R. (2006). Term length as an indicator
13, 319-340. of attrition in online learning. Innovate, 2 (5). Retrieved
October 18, 2007, from http://www.innovateonline.
Davis, S. L., & Morrow, A. K. (2004). Creating usable
info/index.php?view=article&id=196
assessment tools: A step-by-step guide to instrument
design. Retrieved October 29, 2007, from http://www. DIN. (2004). PAS 1032-1, Deutsches institut für normung
jmu.edu/assessment/wm_library/ID_Davis_Mor- e. V. Retrieved October 30, 2007, from http://www.din.
row_AAHE2004.pdf de
De Smet, M., Van Keer, H., & Valcke, M. (2008). Blend- Directorate-General for Education and Culture, European
ing asynchronous discussion groups and peer tutoring Commission. (2004). Final Report. Study of the e-learn-
in higher education: An exploratory study of online ing suppliers’ market in Europe. Danish Technological
peer tutoring behaviour. Computers and Education, Institute, Massy, J., Alphametrics Ltd, & Heriot-Watt
50(1), 207-223. University.
Dearing, R. (1997). Higher education in the learning Distance Education and Training Council. (2002). DETC
society. London: HMSO. accreditation overview. Retrieved October 19, 2007, from
http://www.detc.org/content/freePublications.html
DECT. (2007). Distance education and training council.
Retrieved October 30, 2007, from http://www.dect.org Dorigo, M., & Stützle, T. (2004). Ant colony optimiza-
tion. MIT Press.
Deming, W.E. (1989). Foundation for management
of quality in the western world. New York: Perigee Dougiamas, M., & Taylor, P.C. (2003). Moodle: Using
Books. learning communities to create an open source course
management system. Paper presented at the Proceed-
Deshpande, M., & Karypis, G. (2004). Selective Markov
ings of the EDMEDIA 2003 Conference, Honolulu,
models for predicting Web page accesses. ACM Transac-
Hawaii.
tions on Internet Technology (TOIT), 4(2), 163-184.

Downes, S. (2005, October 17). E-learning 2.0. Elearn Educational Testing Service. (2006). ICT literacy assess-
Magazine. Retrieved October 18, 2007, from http://elearn- ment preliminary findings. Retrieved October 18, 2007,
mag.org/subpage.cfm?section=articles&article=29-1 from http://www.ets.org/Media/Products/ICT_Literacy/
pdf/2006_Preliminary_Findings.pdf
Downes, S. (2005). What e-learning 2.0 means to you.
Paper presented at the meeting of the Transitions in eduQua. (2005). Schweizerisches Qualitätszertifikat für
Advanced Learning Conference, Ottawa. Weiterbildungsinstitutionen. Retrieved October 30, 2007,
from http://www.eduqua.ch
Downes, S. (2005). E-learning 2.0. eLearn Magazine.
Retrieved October 25, 2007, from http://elearnmag.org/ EFMD. (2007). European quality improvement system.
subpage.cfm?section=articles&article=29-1 European Foundation for Management Development.
Retrieved October 30, 2007, from http://www.efmd.
Driscoll, M. (2002). Blended learning: Let’s get beyond
be/equis
the hype. Learning and Training Innovations Newsline.
Retrieved October 18, 2007, from http://www.ltimagazine. EFQUEL. (2006). European foundation for quality in
com/ltimagazine/article/articleDetail.jsp?id=11755 e-learning. Retrieved October 30, 2007, from http://www.
qualityfoundation.org
Driscoll, M.P. (2002). How people learn (and what tech-
nology might have to do with it). ERIC Digest, Syracuse Ehlers, U. -D. (2004). Quality in e-learning. The learner
University. Retrieved October 16, 2007, from http://www. as a key quality assurance category. European Journal
ericdigests.org/2003-3/learn.htm Vocational Training, 29, 3-15.
Dron, J. (2002). Achieving self-organisation in net- Ehlers, U. -D. (2007). Towards greater quality literacy
work-based learning environments. PhD doctoral dis- in a e-learning Europe. e-Learning Papers, 2. Retrieved
sertation. October 30, 2007, from http://www.e-Learningpapers.
eu/index.php?page=doc&doc_id=8549&doclng=6
Dron, J., Mitchell, R., Siviter, P., & Boyne, C (1999).
CoFIND: Experiment in n-dimensional collaborative Ehlers, U. -D., Goertz, L., Hildebrandt, B., & Pawlowski,
filtering. In World Conference on the WWW and Internet J. (2005). Quality in e-learning. Use and dissemination of
(pp. 301-306). quality approaches in European e-learning. A study by
the European quality observatory (CEDEFOP Panorama
Duda, R., Hart, P.E., & Stork, D.G. (2000). Pattern clas-
Series, 116). Luxembourg: Office for Official Publications
sification (2nd ed.). Wiley-Interscience.
of the European Communities.
Dusick, D. (1998). What social cognitive factors influ-
Ehlers, U. -D., Hildebrand, B., Tescheler, S., & Pawlowski,
ence faculty members’ use of computers for teaching.
J. (2004). Designing tools and frameworks for tomorrows
A literature review. Journal of Research on Computing
quality development. In Quality in European E-Learning
in Education, 31(2), 123-137.
Designing Tools and Frameworks for Tomorrows Quality
Duval, R. (1995). Sémiosis et pensée humaine. Peter Development, Workshop European Quality Observatory
Lang. (EQO) co-located to the 4th IEEE International Confer-
ence on Advanced Learning Technologies. Joensuu:
EADL. (2003). Quality guide. Quality guidelines to im-
European Quality Observatory.
prove the quality of distance learning institutes in Europe
(2nd ed.). European Association Distance Learning. Ehrmann, S. C. (1995). Asking the right question: What
does research tell us about technology and higher learn-
Eby, L. T. (1997). Alternative forms of mentoring in
ing? Change, 17(2), 20-27.
changing organizational environments: A conceptual
extension of the mentoring literature. Journal of Voca-
tional Behaviour, 51, 125-144.

EIfEL. (2007). About SEEL, Supporting excellence in EQUEL. (2004). Virtual European centre in e-learning.
e-learning. European Institute for e-Learning. Retrieved Retrieved October 30, 2007, from http://equel.net
October 30, 2007, from http://www.eife-l.org/activi-
EQUIPE. (2007). Aims & objectives. European Quality
ties/projects/seel
in Individualised Pathways in Education. Retrieved
El Louadi, M., Galletta, D.F., & Sampler, J.L. (1998). October 30, 2007, from http://www.EQUIPE.up.pt/
An empirical validation of a contingency model for equipe/objectives.htm
information requirements determination. ACM SIGMIS
European Commission. (2003). E-learning programme,
Database archive, 29(3), 31-51.
Education. Programmes, Europe. Retrieved October 30,
ELEX. (2005). EXEMPLO e-learning project. Retrieved 2007, from http://ec.europa.eu/education/programmes/e-
October 30, 2007, from http://217.222.182.72/html/iess/ Learning/programme_en.html#
pt.htm
European Commission. (2005). Common European
Elton, L.R.B., & Laurillard, D.M. (1979). Trends in principles for teacher competences and qualifications.
research on student learning. Studies in Higher Educa- Retrieved October 18, 2007, from http://europa.eu.int/
tion, 4(1), 87-102. comm/education/policies/2010/doc/principles_en.pdf
Ensher, E. A., Heun, C., & Blanchard, A. (2003). Online European Commission. (2005). E-learning, designing
mentoring and computer-meadiated communication: education tomorrow. Report on the consultation work-
New directions in research. Journal of vocational be- shop. The “e” for our universities—Virtual campus.
haviour, 63, 264 - 288. Organisational changes and economics models. DRAFT.
Brussels: European Commission, Directorate-General
Entwistle, N. (2003). University teaching-learning
for Education and Culture.
environments and their influences on student learning:
An introduction to the ETL project. In Proceedings of European Commission. (2006, September 29). Bench-
the 10th Conference of the European Association for marking access and use of ICT in European schools
Research on Learning and Instruction (EARLI). Padova, 2006. Retrieved October 18, 2007, from http://ec.europa.
Italy: EARLI. eu/information_society/eeurope/i2010/docs/studies/fi-
nal_report_3.pdf
Entwistle, N., McCune, V., & Hounsell, J. (2002). Ap-
proaches to studying and perceptions of university European Schoolnet. (2005, July 15). Insight special
teaching-learning environments: Concepts, measures report on assessment schemes for teachers’ ICT com-
and preliminary findings. Edinburgh: University of petence—A policy analysis. Retrieved October 18,
Edinburgh. 2007, from http://www.e-Learningeuropa.info/index.
php?page=doc&doc_id=6578&doclng=6
Entwistle, N., Tait, H., & McCune, V. (2000). Patterns
of response to an approach to studying inventory across Evans, J.R., & Lindsay, W.M. (1999). The management
contrasting groups and contexts. Paper presented at the and control of quality (4th ed.). Cincinnati, OH: South-
European Journal of the Psychology of Education. Western College Publishing.
eQCheck. (2006). Qualite-learning assurance services Evans, M. (2006). Goodbye Web 2.0, long live Web 3.0.
ltd. Retrieved October 30, 2007, from http://www.eq- Retrieved October 25, 2007, from http://evans.blogware.
check.co.uk com/blog/_archives/2006/11/12/2493546.html
EQO. (2007). European quality observatory. Retrieved Falchikov, N. (2001). Learning together: Peer tutoring
October 30, 2007, from http://www.eqo.info in higher education. Falmer Press.
0
Fandos, M., & González, A. P. (2005). Estrategias de FitzGerald, S. (2006, June 8). Social networking: Phi-
Aprendizaje ante las Nuevas Posibilidades Educativas losophy and pedagogy. 2006 Networks Community
de las TIC. In A. Méndez-Vilas, B. Gonzalez, J. Mesa, Forum. Edna, Australia. Retrieved October 25, 2007,
& J. A. Mesa (Eds.), Proceedings of the Third Interna- from http://www.groups.edna.edu.au/mod/forum/dis-
tional Conference on Multimedia and Information & cuss.php?d=6615
Communication Technologies in Education (pp. 7-10).
Flynn, A., Concannon, F., & Bheachain, C. N. (2005).
Cáceres, Spain: Formatex.
Undergraduate students’ perceptions of technology-
Farrell, R., Liburd, S.D., & Thomas, J.C. (2004). Dynamic supported learning: The case of an accounting class.
assembly of learning objects. In Proceedings of 13th International Journal on E-Learning, 4(4), 427-444.
International World Wide Web Conference, NY.
Ford, P., Goodyear, P., Heseltine, R., Lewis, R., Darby,
Fayyad, U., Piatetsky-Shapiro, G., Smyth, P., & Uthuru- J., Graves, J., et al. (1996). Managing change in higher
samy, R. (1996). Advances in knowledge discovery and education: A learning environment architecture. Society
data mining. New York: The MIT Press. for Research in Higher Education and Open University
Press.
Feigenbaum, A.V. (1951). Quality control: Principles,
practice, and administration. New York: McGraw- Frees, S., & Kessler, G. D. (2004). Developing col-
Hill. laborative tools to promote communication and active
learning in academia. In Proceedings of the 34th Annual
Felder, R., & Silverman, L. (1988). Learning and teach-
Conference Frontiers in Education (FIE’04) (Vol. 3, pp.
ing styles. Journal of Engineering Education, 78(7),
S3B/20 - S3B/25). Piscataway, NJ: IEEE.
674-681.
Freund, R. J., & Piotrowski, M. (2003). Mass customiza-
Ferrari, P. L. (2004). Mathematical language and ad-
tion and personalization in adult education and train-
vanced mathematics learning. In M. Johnsen Høines &
ing. Paper presented at the 2nd Interdisciplinary World
A. Berit Fuglestad (Eds.), Proceedings of the 28th Con-
Congress on Mass Customization and Personalization
ference of the International Group for the Psychology
MCPC2003, Munich, Germany.
of Mathematics Education (Vol. 2, pp. 383-390). Bergen,
Norway: Bergen University College Press. Fridman, N., & McGuinness, D. (2001). Ontology de-
velopment: A guide to creating your first ontology (Rep.
Feuerstein, R. (1990). The theory of structural cognitive
No. KSL-01-05, SMI-2001).
modifiability. In B.Z. Presseisen (Ed.), Learning and
thinking styles: Classroom applications (pp. 68-134). FuturEd. (2002). Consumers guide to e-Learning.
Washington, DC: National Education Association. Retrieved October 19, 2007, from http://www.futured.
com/pdf/ConGuide%20Eng%20CD.pdf
Feuerstein, R., Rand, Y., Hoffman, M. B, & Miller, R.
(1980). Instrumental enrichment. Baltimore, MD: Uni- Gambardella, L.M, Taillard E., & Agazzi G. (1999).
versity Park Press. MACS-VRPTW: A multiple ant colony system for ve-
hicle routing problems with time windows. New Ideas
Figueiredo, A. D. (2002). Redes e Educação: A Surpreen-
in Optimization, 63-76.
dente Riqueza de um Conceito. In Conselho Nacional de
Educação (Ed.), Redes de Aprendizagem, Redes de Con- Gándara, M. (1995). La interfaz con el usuario: Una
hecimento. Lisboa: Conselho Nacional de Educação. introducción para educadores. In Alvarez-Manilla and
Bañuelos (Eds.), Usos educativos de la computadora.
Fikes, R., & Nilsson, N. (1971). Strips: A new approach
México: CISE/UNAM.
to the application of theorem proving to problem solving.
Artificial Intelligence, 2, 189-208.

García Aretio, L. (2003). Comunidades de aprendizaje en Gibbs, G., & Simpson, C. (2003). Does your assessment
entornos virtuales. La comunidad iberoamericana de la support your students’ learning? Journal of Learning
CUED. In M. Barajas (Ed.), La tecnología educativa en and Teaching in Higher Education, 1(1).
la enseñanza superior. Madrid, Spain: McGrawHill.
Gibbs, G., & Simpson, C. (2004). Measuring the response
García Carrasco, J., Pérez, M.A., Rodríguez, B., & of students to assessment: The assessment experience
Sánchez, M.C. (2002). Evaluar en la red. Teoría de la questionnaire. In C. Rust (Ed.), Improving student learn-
Educación: Educación y Cultura en la Sociedad de la ing: Theory, research and scholarship. Oxford: Oxford
Información, 3(5). Retrieved June 11, 2007, from http:// Centre for Staff and Learning Development.
www.usal.es/~teoriaeducacion/
Gibbs, G., & Simpson, C. (2005). Conditions under which
Garcia, P., Amandi, A., Schiaffino, S., & Campo, M. assessment supports students’ learning. Learning and
(2007). Evaluting Bayesian networks’ precision for detect- Teaching in Higher Education 1, 3-31.
ing students’ learning styles. Computers & Education,
Gibbs, G., Simpson, C., & Macdonald, R. (2003). Improv-
49(3), 794-808.
ing student learning through changing assessment—A
Garrison, D. R. (2003). Cognitive presence for effective conceptual and practical framework. Paper presented
asynchronous online learning: The role of reflective at the European Association for Research into Learning
inquiry, self-direction and metacognition. In J. Bourne and Instruction Conference, Padova, Italy.
& J.C. Moore (Eds.), Elements of quality online educa-
Gifford, B.R., & Enyedy, N. (1999). Activity centered
tion: Practice and direction (pp. 47-58). Needham, MA:
design: Towards a theoretical framework for CSCL. In
Sloan-C.
Proceedings of the Third International Conference on
Garrison, D. R., & Kanuka, H. (2004). Blended learning: Computer Support for Collaborative Learning. Retrieved
Uncovering its transformative potential in higher educa- October 18, 2007, from http://www.gseis.ucla.edu/fac-
tion. Internet and Higher Education, 7(2), 95-105. ulty/enyedy/pubs/Gifford&Enyedy_CSCL2000.pdf
Garrison, D.R., & Anderson, T. (2003). E-learning in Ginns, P., & Ellis, R. (2007). Quality in blended learn-
the 21st century: A framework for research and practice. ing: Exploring the relationships between online and
London, New York: RoutledgeFalmer. face-to-face teaching and learning. Internet and Higher
Education, 10(1), 53-64.
Gasar, S., Bohanec, M., & Rajkovic, V. (2002). Com-
bined data mining and decision support approach to the Gipps, C. (1994). Beyond testing: Towards a theory of
prediction of academic achievement. In Workshop on educational assessment. London: Falmer Press.
Integrating Aspects of Data Mining (pp. 41-52).
Gisbert, M. (2004). Las TIC como motor de innovación
Gerbic, P. (2002). Learning in asynchronous environ- de la Universidad. En SANGRÀ, A. Y GONZÁLEZ,
ments for on campus students. In R. Kinshuk, K. Lewis, M. (coord.): Barcelona. Ed. UOC. In A. Sangrà & M.
R. Akahori, T. Kemp, L. Okamoto, C. Henderson & H. Gonz’alez (Eds.), La transformación de las universidades
Lee (Eds.), Proceedings of the 9th International Confer- a través de las TIC: Discursos y prácticas (pp. 193-197).
ence on Computers in Education (Vol. 2, pp. 1492-1493), Barcelona: Ed. UOC.
Auckland, New Zealand: Asia-Pacific Society for Com-
Global University Alliance. (2000). About GUA. Retrieved
puters in Education.
October 19, 2007, from http://www.gua.com/shell/gua/
Gibbs, G. (2006). Why assessment is changing. In C. index.asp
Bryan & K. Clegg (Eds.), Innovative assessment in higher
Godoy, L. A. (2005). Learning-by-doing in a Web-based
education (pp. 11-22). New York: Routledge.
simulated environment. In Proceedings of the 6th Inter-

national Conference on Information Technology Based Guthrie, W.K.C. (1971). The sophists. London: Cambridge
Higher Education and Training (ITHET 2005) (pp. F4C/7 University Press.
- F4C/10). Piscataway, NJ: IEEE.
Gutiérrez, S., Pardo, A., & Delgado Kloos, C. (2006).
Gomez-Perez, A., & Manzano-Macho, D. (2004). An A modular architecture for intelligent Web resource
overview of methods and tools for ontology learning from based tutoring systems. Intelligent Tutoring Systems,
texts. Knowledge Engineering Review, 19(3), 187-212. 753-755.
Goode, V. L. (2003). Lifestyle in the balance. Chartered Gutiérrez, S., Pardo, A., & Delgado Kloos, C. (2006).
Accountants Journal, 82(3), 22-24. Some ideas for the collaborative search of the optimal
learning path. In Adaptive Hypermedia 2006 (pp. 430-
Goodyear, P. (2001). Effective networked learning in
434).
higher education: Notes and guidelines (Deliverable 9)
(Vol. 3). Lancaster: CSALT, Lancaster University. Gutiérrez, S., Valigiani, G., Jamont, Y., Collet, P., &
Delgado Kloos, C. (2007). A swarm appoach for auto-
Goodyear, P. (2002). Online learning and teaching in the
matic auditing of pedagogical planning. In Proceedings
arts and humanities: Reflecting on purposes and design.
of IEEE ICALT 2007 (pp. 136-138).
In E.A. Chambers & K. Lack (Eds.), Online conferencing
in the arts and humanities (pp. 1-15). Milton Keynes: Hainaut, J., Tonneau, C., Joris, M., & Chandelon, M.
Institute of Educational Technology, Open University. (1993). Transformation based database reverse engi-
neering. In R. Elmasri, V. Kouramajian & B. Thalheim
Goodyear, P. (2005). Educational design and networked
(Eds.), Conference on Entity Relationship Approach (pp.
learning: Patterns, pattern languages and design prac-
364-375). Springer.
tice. Australasian Journal of Educational Technology,
21(1), 82-101. Haladyna, T. (1997). Writing test items to evaluate
higher order thinking. Needham Heights, MA: Allyn
Gould, S. J. (1978). Ever since Darwin - reflections in
& Bacon.
natural history. Burnett.
Haldiki, M., Batistakis, Y., & Vazirgiannis, M. (2001). On
Grant, L. (2006). Using wikis in schools: A case study.
clustering validation techniques. Journal of Intelligent
Retrieved October 25, 2007, from http://www.futurelab.
Information Systems, 17(2-3), 107-145.
org.uk/research/discuss/05discuss01.htm
Halperin, R. (2005). Learning technology in higher
Gray, M. M., & Gray, W. A. (1990). Planned mentoring:
education: A structurational perspective on technol-
Aiding key transitions in career development. Career
ogy-mediated learning practices (Doctoral dissertation).
Planning and Adult Development Journal, 6(3), 27-32.
London: London School of Economics.
Green, D. (Ed.). (1994). What is quality in higher edu-
Ham, C.L. (2003). Service quality, customer satisfaction,
cation? (pp. 3-20). Buckingham, UK: Open University
and customer behavioral intentions in higher education.
Press.
Published doctoral dissertation AAT 3090234. Nova
Gruber, T.R. (1995). Towards principles for the design Southeastern University, FL.
of ontologies used for knowledge sharing. International
Hamilton, B. A., & T.A., S. (2003). E-mentoring: Im-
Journal of Human-Computer Studies, 43, 907-928.
plications for organizational learning and development
Guldberg, K., & Pilkington, R. (2007). Tutor roles in fa- in a wired world. Organizational Dynamics, 31(4),
cilitating reflection on practice through online discussion. 388-402.
Educational Technology and Society, 10(1), 61-72.

Hara, N., & Kling, R. (2000). Student distress in a Hernández Nanclares, N. (2004). La evaluación mediante
Web-based distance education course. Information, portafolio en Relaciones Económicas Internacionales.
Communication and Society, 3(4), 556-579. In R. Rodríguez, J. Hernández & S. Fernández (Eds.),
Docencia universitaria: Orientaciones para la formación
Hardle, W., & Simar, L. (2006). Applied multivariante
del profesorado (pp. 331-341). Documentos ICE, Instituto
statical analysis. New York: Springer.
de Ciencias de la Educación, Universidad de Oviedo.
Harrington, A. (1999). E-mentoring: The advantages
Hernández Nanclares, N. (2006) El portafolios elec-
and disadvantages of using e-mail to support distant
trónico: Una alternativa para evaluar en la Universidad.
mentoring. The Coaching and Mentoring Network
Paper presented in I jornadas de innovación educativa
Articles. Retrieved October 17, 2007, from http://www.
de la Escuela Politécnica Superior de Zamora, junio
coachingnetwork.org.uk/ResourceCentre/Articles/
Zamora, España.
ViewArticlePF.asp?artId=63
Hiltz, S. R. (1994). The virtual classroom: Learning
Harris, J., O’Bryan, E., & Rotenberg, L. (1996). It’s a
without limits via computer networks. Worwood, NJ:
simple idea, but it’s not easy to do! Practical lessons in
Ablex.
telementoring. Learning and Leading with Technology.
Retrieved October 17, 2007, from http://emissary.wm.edu/ Hisham, N., Campton, P., & FitzGerald, F. (2004). A tale
templates/content/publications/October96LLT.pdf of two cities: A study on the satisfaction of asynchronous
e-learning systems in two Australian universities. In R.
Hartley, J., & Sleeman, D. (1973). Towards more intel-
Atkinson, C. McBeath, D. Jonas-Dwyer, & R. Phillips
ligent teaching systems. International Journal of Man-
(Eds.), Beyond the comfort zone: Proceedings of the 21st
Machine Studies, 2, 215-336.
ASCILITE Conference (pp. 395-402). Perth, Australia:
Harvey, L., & Knight, P.T. (1996). Transforming higher ASCILITE. Retrieved October 19, 2007, from http://www.
education. Buckingham, UK: Open University Press. ascilite.org.au/conferences/perth04/procs/hisham.html
Headlam-Wells, J., Gosland, J., & Craig, J. (2005). Hislop, G. W. (1999). Anytime, anyplace learning in an
There’s magic in the Web: E-mentoring for women’s online graduate professional degree program. Group
career development. Career Development International, Decision and Negotiation, 8, 385-390.
10(6-7), 444-459.
Hofmann, T. (2003). Collaborative filtering via Gauss-
Headlam-Wells. (2004). E-mentoring for aspiring women ian probabilistic latent semantic analysis. In 26th ACM
managers. Women in Management Review, 19(4), 212- SIGIR Conference on Research in Information Retrieval
218. (pp. 259-266).
Heerema, D. L., & Rogers, R. L. (2001). Avoiding the qua- Hollands, N. (2000). Online testing: Best practices
lity/quantity trade-off. T.H.E. Journal, 29(5), 14-21. from the field. Retrieved October 30, 2007, from
http://198.85.71.76/english/blackboard/testingadvice.
Helms, M.M., Williams, A.B., & Nixon, J.C. (2001).
html
TQM principles and their relevance to higher educa-
tion: The question of tenure and post-tenure review. Holmes, G., & McElwee, G. (1995). Total quality manage-
The International Journal of Education Management, ment in higher education how to approach human resource
14(6-7), 322-331. management. Total Quality Management, 7(6), 5.
Henly, D. C. (2003). Use of Web-based formative assess- Hope, A. (2001) Quality assurance. In G. Farrell (Ed.),
ment to support student learning in a metabolism/nutrition The changing faces of virtual education (pp. 125-140).
unit. Journal of Dental Education, 7(3), 116-122. London: The Commonwealth of Learning.

Hudson, B. (2005). Conditions for achieving com- Hyvärinen, A., & Oja, E. (1998). A fast fixed-point
munication, interaction and collaboration in e-learn- algorithm for independent component analysis. Neural
ing environments. E-Learningeuropa.info. Retrieved Computation, 9(7), 1483-1492.
October 18, 2007, from http://www.e-Learningeuropa.
Hyvärinen, A., Karhunen, J., & Oja, E. (2001). Inde-
info/index.php?page=doc&doc_id=6494&doclng=7&
pendent Component Analysis. New York: John Wiley
menuzone=1
& Sons.
Hunt, L. M., Thomas, M. J. W., & Eagle, L. (2002). Stu-
Ibabe, I., & Jauregizar, J. (2005). Ejercicios de autoevalu-
dent resistance to ICT in education. In R. Kinshuk, K.
ación con Hot Potatoes. In I. Ibabe & J. Jauregizar (Eds.),
Lewis, R. Akahori, T. Kemp, L. Okamoto, C. Henderson
Cómo crear una web docente de calidad (pp. 65-100).
& H. Lee (Eds.), Proceedings of the 9th International
A Coruña, Spain: Netbiblo.
Conference on Computers in Education (Vol. 2, pp.
964-968), Auckland, New Zealand: Asia-Pacific Society Ibabe, I., Gómez J., & Jauregizar, J. (2006). Aplicación de
for Computers in Education. pruebas de auto-evaluación interactivas para potenciar el
trabajo autónomo de los estudiantes conforme al sistema
HUSAT. (1990). The HUFIT planning, analysis and
ECTS. In J. Guisasola & T. Nuño (Eds.), La educación
specification toolset. Loughborough: HUSAT Research
universitaria en tiempos de cambio (pp. 63-74). San
Institute, Loughborough University.
Sebastián, Spain: Universidad del País Vasco.
Husson, A. (2004). Comparing quality models adequacy
IITT. (2005). Institute of IT training. Retrieved October
to the needs of clients in e-learning. In Quality in Euro-
30, 2007, from http://www.iitt.org.uk
pean E-Learning Designing Tools and Frameworks for
Tomorrows Quality Development, Workshop European Ilghami, O., & Nau, D.S. (2003). A general approach
Quality Observatory (EQO) co-located to the 4th IEEE to synthesize problem-specific planners (Tech. Rep.
International Conference on Advanced Learning Tech- CS-TR-4597). University of Maryland: Department of
nologies. Joensuu: European Quality Observatory. Computer Science.
Hutchins, T., Sims, M., & Cooper, T. (1999). Developing IMS CP. (2006). Retrieved October 22, 2007, from
a portfolio which demonstrates competencies. In M. Sims http://www.imsglobal.org/content/packaging/
& T. Hutchins (Eds.), Learning materials: Certificate
IMS LD, IMS Learning Design. (2006). IMS Global
in children’s services; 0-6 years (bilingual support)
Learning Consortium. Retrieved October 22, 2007, from
(pp. 3-29). Perth, WA: Ethnic Childcare Resource Inc.
http://www.imsglobal.org/learningdesign/index.html
Western Australia.
IMSSS, IMS Simple Sequencing. (2006). Retrieved
Hyde, P., Booth R., & Wilson, P. (2003). The develop-
October 22, 2007, from http://www.imsglobal.org/simple-
ment of quality online assessment in VET. In H. Guthrie
sequencing/index.html
(Ed.), Online learning: Research readings (pp. 87-106).
Leabrook, South Australia: NCVER. Inglis, A. (1999). Is online delivery less costly than
print and is it meaningful to ask? Distance Education,
Hyland, B. (2002). Cone of learning. From the course
20(2), 220-232.
“Train the trainer”. Iowa Center for Public Health
Preparedness. Retrieved October 26, 2007, from http:// Institute for Higher Education Policy. (2000). Quality on
www.public-health.uiowa.edu/icphp/ed_training/ttt/ar- the line. Retrieved October 19, 2007, from http://www.
chive/2002/2002_course_materials/Cone_of_Learning. ihep.com/Pubs/PDF/Quality.pdf/
pdf
Intelligent Web Teacher. (2006). Retrieved October 21,
2007, from http://www.momanet.it/english/iwt_eng.
html

ISO. (2003). ISO standards compendium ISO 9000–qual- ton Centre for Improving the Quality of Undergraduate
ity management (10th ed.). Geneva: International Organi- Education.
zation for Standardization.
Joint Committee on Standards for Educational Evalu-
ISO. (2005). ISO/IEC standard benchmarks quality of e- ation. (1988). The personnel evaluation standards.
learning. International Standard Organization. Retrieved Thousand Oaks, CA: Corwin Press.
October 30, 2007, from http://www.iso.org/iso/en/com-
Jonassen, D. H., & Grabowski, B. L. (1993). Handbook
mcentre/pressreleases/2006/ref992.html
of individual differences, learning and instruction.
ITC. (2003). Online assessment techniques. Retrieved Erlbaum, Hillsdale.
October 29, 2007, from http://web.utk.edu/~dsuppach/
Jonassen, D.H., & Grabowski, B.L. (1993). Handbook
indep/assessment2.htm.
of individual differences, learning, and instruction.
Jaeger, W. (1945). Paideia: The ideals of Greek culture (G. Hillsdale, NJ: Lawrence Erlbaum Associates.
Highet, Trans.). New York: Oxford University Press.
Jones, E. R., (2002). Implications of SCORM™ and
James, R., McInnis, C., & Devlin, M. (2002). Assessing emerging e-learning standards on engineering educa-
learning in Australian universities. Canberra, Australia: tion. In ASEE Gulf-Southwest Annual Conference (pp.
Center for the Study of Higher Education, The University 20-22).
of Melbourne & The Australian Universities Teaching
Juran, J. M. (1988). Quality control handbook. New
Committee.
York: McGraw Hill.
Jennings, D. (2005). E-learning 2.0, whatever that is.
Juran, J.M., & Gyrna, F.M. Jr. (1988). Juran’s quality
Retrieved October 18, 2007, from http://alchemi.co.uk/
control handbook. New York: McGraw-Hill.
archives/ele/e-Learning_20_wh.html
Karrer, T. (2006, February 10). What is e-learning 2.0.
Johnes, J., & Taylor, J. (1990). Performance indicators
E-Learning Technology. Retrieved October 18, 2007,
in higher education. Buckingham: Open University
from http://e-Learningtech.blogspot.com/2006/02/what-
Press.
is-e-Learning-20.html
Johnson, D. W., & Johnson, R. T. (1987). Learning
Kasprisin, C. A., Single, P. B., Single, R. M., & Muller,
together and alone: Cooperative, competitive, and indi-
C. B. (2003). Building a better bridge: Testing e-training
vidualistic. Englewood Cliffs, NJ: Prentice Hall.
to improve e-mentoring programmes in higher education.
Johnson, D., & Johnson, R. (1999). Learning together Mentoring and Tutoring, 11(1).
and alone: Cooperative, competitive, and individualistic
Kassabova, D., & Trounson, R. (2000). Applying soft
learning. Boston: Allyn and Bacon.
systems methodology for user centred design. In Proceed-
Johnson, D., Johnson, R., & Holubec, E. (1998). Coopera- ings of the NACCQ 2000 (pp. 159-165). Wellington.
tion in the classroom. Boston: Allyn and Bacon.
Kauffman, S. (1996). At home in the universe: The search
Johnson, R., & Johnson, D. W. (1998). Cooperative for the laws of self-organization and complexity. Oxford
learning. Two heads learn better than one. Transforming University Press.
Education, 18, 34.
Kearsley, G. (2000). Online education: Learning and
Johnson-Bogart, K. (1995). Writing portfolios: What teaching in cyberspace. Belmont, CA: Wadsworth.
teachers learn from students self-assessment. In Wash-
Kendle, A., & Northcote, M. (2000). The struggle for
ington Centre’s Evaluation Committee (Ed.), Assessment
balance in the use of quantitative and qualitative online
in and of collaborative learning. Washington: Washing-
assessment tasks. In Proceedings ASCILITE 2000, Coffs

Harbour. Retrieved October 29, 2007, from http://www. Klenowski, V. (2002). Developing portfolios for learn-
ascilite.org.au/conferences/coffs00/papers/amanda_ken- ing and assessment: Processes and principles. London:
dle.pdf RoutledgeFalmer.
Kennedy, G.A. (1963). The art of persuasion in Greece. Klenowski, V. (2002). Developing portfolios for learn-
Princeton, NJ: Princeton University Press. ing and assessment: Processes and principles. London:
RoutledgeFalmer.
Kennedy, J. K., Sang, J. C. K, Wai-ming, F. Y., & Fok,
P. K. (2006, May). Assessment for productive learning: Klenowski, V., Askew, S., & Carnell, E. (2006). Portfolios
Forms of assessment and their potential for enhancing for learning, assessment and professional development.
nd
learning. Paper presented at the 32 Annual Conference Assessment and Evaluation in Higher Education, 31(3),
of the International Association for Educational Assess- 267-286.
ment, Singapore.
Knight, M. E. (1994). Portfolio assessment: Application
Kennedy, R., & Eberhart, R. (2001). Swarm intelligence. of portfolio analysis. Lanham, MD: University Press
CA: Morgan Kaufmann/Academic Press. of America.
Kerlin, C.A. (2000). Measuring student satisfaction with Knight, P.T. (2001). Complexity and curriculum: A
the service processes of selected student educational sup- process approach to curriculum-making. Teaching in
port services at Everett Community College. Published Higher Education, 6(3), 369-381.
Dissertation AAT9961458. Oregon State University.
Knight, P.T., & Trowler, P. (2001). Departmental leader-
Kerr, M. S., & Rynearson, R. (2006). Student character- ship in higher education. Buckingham: Society for Re-
istics for online learning success. Internet and Higher search in Higher Education & Open University Press.
Education, 9, 91-105.
Koch, J.V., & Fisher, J.L. (1998). Higher education and
Kickul, G., & Kickul, J. (2006). Closing the gap: Impact total quality management. Total Quality Management,
of student proactivity and learning goal orientation on 9(8), 659-668.
e-learning outcomes. International Journal on E-Learn-
Kolisch, R., & Hartmann, S. (1999). Heuristic algorithms
ing, 5(3), 361-372.
for solving the resource-constrained project scheduling
Kieran, C., Forman, E., & Sfard, A. (2001). Learning problem: Classification and computational analysis.
discourse: Sociocultural approaches to research in Project scheduling: Recent Models, Algorithms and
mathematics education. Educational Studies in Math- Applications, 147-178.
ematics, 46, 1-12.
Koper, R. (2005). Designing learning networks for life-
Kim, S., & Sonnenwald, D. H. (2002). Investigating long learners. In R. Koper & C. Tattersall (Eds.), Learn-
the relationship between learning style preferences ing design: A handbook on modelling and delivering
and teaching collaboration skills and technology: An networked education and training (pp. 239-252).
exploratory study. In E. Toms (Ed.), Proceedings of the
Koper, R., & Olivier, B. (2004). Representing the learn-
American Society of Information Science & Technology
ing design of units of learning. Educational Technology
Annual Conference (pp. 64-73). Medford, NJ: Informa-
& Society, 7(3), 97-111.
tion Today.
Kotsiantis, S.B., Pierrakeas, C.J., & Pintelas, P.E. (2003).
Kimber, K., Pillay, H., & Richards, C. (2007). Tech-
Preventing student dropout in distance learning using
noliteracy and learning: An analysis of the quality of
machine learning techniques. In Proceedings of 7th In-
knowledge in electronic representations of understanding.
ternational Conference on Knowledge-Base Intelligent
Information an Engineering Systems.

Kozma, R. (1991). Learning with media. Review of Lee, G., & Weerakoon, P. (2001). The role of computer-
Educational Research, 61(2), 179-212. aided assessment in health professional education: A
comparison of student performance in computer-based
Kram, K. E. (1985). Mentoring at work: Developmental
and paper-and-pen tests. Medical Teacher, 23, 152-157.
relationships in organizational life. New York: University
Press of America. Lee, T., Girolami, M., & Sejnowski, T. (1999). Inde-
pendent component analysis using an extend infomax
Kristofic, A., & Bielikova, M. (2005). Improving adapta-
algorithm for mixed sub-Gaussian and super-Gaussian
tion in Web-based educational hypermedia by means of
sources. Neural Computation, 11, 417-441.
knowledge discovery. In ACM Conference on Hypertext
and Hypermedia (pp. 184-192). Lee, Y. L., & Nguyen, H. (2005). So are you online yet?!
Distance and online education today. In M. Khosrow-Pour
Kulhavy, R. W., & Stock, W. A. (1989). Feedback in
(Ed.), Managing modern organizations with informa-
written instruction: The place of response certitude.
tion technology. Proceedings of the 2005 Information
Educational Psychology Review, 1(4), 279-308.
Resources Management Association International
Lafuente, E., & Hunt, T. (2007) Development: XMLDB Conference (pp. 1035-1036). San Diego, CA: Information
documentation. Retrieved October 28, 2007, from http:// Resource Management Association.
docs.moodle.org/en/Development:XMLDB_Documen-
Leidner, D., & Jarvenpaa, S. (1995). The use of in-
tation
formation technology to enhance management school
Lambert, M. P. (1996). The distance education and train- education: A theoretical view. MIS Quarterly, 19(3),
ing council: At the cutting edge. Quality Assurance in 265-291.
Education, 4(4), 26-28.
Leonard, J., & Guha, S. (2001). Students’ perspectives
LAMS. (2006). Learning Activity Management System. on distance learning. Journal of Research on Technology
Retrieved October 22, 2007, from http://lamsfoundation. in Education, 34(1).
org/
Levy, Y. (2007). Comparing dropouts and persistence
Larsen, J., Hansen, L.K., Szymkowiak, A., Christiansen, in e-learning courses. Computers & Education, 48(2),
T., & Kolenda, T. (2002). Web mining: Learning from 185-204.
the world wide Web (Special Issue of Computational
Liaw, S. S. (2002). Understanding user perceptions of
Statistics and Data Analysis). Computational Statistics
WWW environments. Journal of Computer Assisted
and Data Analysis, 38, 517-532.
Learning, 18, 1-12.
Laurillard, D. (1993). Rethinking University Teaching: a
Liaw, S., Chen, G., & Huang, H. (in press). Users’ attitudes
framework for the effective use of educational technol-
toward Web-based collaborative learning systems for
ogy. London: Routledge.
knowledge management. Computers & Education.
Laurillard, D. (1999). A conversational framework for
LIfIA & EIfEL. (2004). Open e-quality learning stand-
individual learning applied to the ‘learning organisa-
ards. Joint e-quality committee of LIfIA (Learning
tion’ and the ‘learning society’, systems research and
Innovations Forum d’Innovation d’Apprentissage) and
behavioral science (vol. 16, pp. 113-122).
EIfEL (European Institute for e-Learning).
Laurillard, D. (2002). Rethinking university teaching: A
Linn, R. L., Baker, E. L., & Dunbar, S. B. (1991). Complex,
conversational framework for the effective use of learning
performance-based assessment: Expectations and valida-
technologies (2nd ed.). London: Routledge.
tion criteria. Educational Researcher, 20(8), 15-21.

Little, J.W. (1990). The mentor phenomenon and the social Margerum-leys, J., & Bass, K.M. (2006). Electronic tools
organisation of teaching. In C. B. Courtney (Ed.), Review for online assessment: An illustrative case study from
of Research in Education, 16, 297-351. Washington, DC: teacher education. In S.L. Howell & M. Hricko (Eds.),
American Educational Research Association. Online assessment and measurement. Case studies
from higher education, K-12 and corporate (pp. 62-81).
Liu, H., & Yang, M. (2005). QoL guaranteed adapta-
London: Information Science Publishing.
tion and personalization in e-learning systems. IEEE
Transactions on Education, 48(4), 676-687. Marques, C. (2004). E-learning: Uma nova forma de
aprender. Revista e-Ciência, 1(1), 23.
Lizzio, A., Wilson, K., & Simons, R. (2002).University
students’ perceptions of the learning environment and Marqués, P. (2001). Didáctica. Los Procesos de Ense-
academic outcomes: Implications for theory and practice. ñanza y Aprendizaje. La motivación. Retrieved October
Studies in Higher Education, 27(1), 27-52. 26, 2007, from http://dewey.uab.es/pmarques/actodid.
htm
LOM. (2006). Retrieved October 22, 2007, from http://
ltsc.ieee.org/wg12/ Martínez, A., Gómez, E., Dimitriadis, Y., Jorrín, I. M.,
Rubia, B., & Vega, G. (2005). Multiple case studies to
Love, T., & Cooper, T. (2004). Designing online infor-
enhance project-based learning in a computer architec-
mation systems for portfolio-based assessment: Design
ture course. IEEE Transactions on Education, 48(3),
criteria and heuristics. Journal of Information Technology
482-489.
Education, 3, 65-81.
Marton, F., & Ramsden, P. (1988). What does it take to
Lowry, R. (2005). Computer-aided self-assessment.
improve learning? In P. Ramsden (Ed.), Improving learn-
An effective tool. Chemistry Education Research and
ing: New perspectives. London: Kogan Page.
Practice, 6(4), 198-203.
Masie, E. (2006). Nano-learning: Miniaturization of
LTSC. (2006). Retrieved October 22, 2007, from http://
design. Chief Learning Officer (CLO) Magazine. Re-
ieeeltsc.org/
trieved October 25, 2007, from http://www.clomedia.
M’tir, R.H., Jeribi, I., Rumpler, B., & Ghazala, H.H.B. com/content/templates/clo_article.asp?articleid=1221
(2004). Reuse and cooperation in e-learning systems. In &zoneid=173
Proceedings of the Fifth International Conference on
Mason, B. J., & Bruning, R. (2000). Providing feedback
Information Technology Based Higher Education and
in computer-based instruction: What the research tells
Training, ITHET (pp. 131-137).
us. Retrieved October 29, 2007, from http://dwb.unl.
Ma, Y., Liu, B., Wong, C.K., Yu, P.S., & Lee, S.M. edu/Edit/MB/MasonBruning.html
(2000). Targeting the right students using data mining.
Maurer, W. (2004). Estándares e-learning. SEESCYT.
In KDD’00: Sixth ACM SIGKDD International Confer-
Retrieved October 28, 2007, from http://fgsnet.nova.
ence on Knowledge Discovery and Data Mining (pp.
edu/cread2/pdf/Maurer1.pdf
457-464).
Mazza, R., & Dimitrova, V. (2003, July 20-24). CourseVis:
Macdonald, J. (2004). Developing competent e-learners:
Externalising student information to facilitate instructors
The role of assessment. Assessment & Evaluation in
in distance learning. In U. Hoppe, F. Verdejo & J. Kay
(Eds.), Proceedings of the International conference in
Maimon, O., & Rokach, L. (2005). Data mining and Artificial Intelligence in Education, Sydney, Australia.
knowledge discovery handbook (1st ed.). Springer.
Mazzolini, M., & Maddison, S. (2003). Sage, guide or
Maragliano, R. (2000). Nuovo manuale di didattica ghost? The effects of instructor intervention on student
multimediale. Editori Laterza.

participation in online discussion forums. Computers & T. Petrovic (Eds.), Meeting at the crossroads. Proceed-
and Education, 40, 237-253. ings of the 18th Annual Conference of the Australasian
Society for Computers in Learning in Tertiary Educa-
Mcalpine, M (2002). Principles of assessment. Luton:
tion (pp. 417-426). Melbourne, Australia: Australasian
CAA Centre.
Society for Computers in Learning in Tertiary Education.
McCarthy, J. P., & Anderson, L. (2000). Active learn- Retrieved October 19, 2007, from http://www.ascilite.
ing techniques vs. traditional teaching styles: Two ex- org.au/conferences/melbourne01/pdf/papers/mclough-
periments from history and political science. Innovative linc2.pdf
McMahon, M., Limerick, B., & Gillies, J. (2002). Struc-
McConnell, D. (2002). The experience of collaborative tured mentoring: A career transition support service
assessment in e-learning. Studies in Continuing Educa- for girls. Australian Journal of Career Development,
tion, 24(1), 73-92. 11(2), 7-12.
McCormick, N., & Leonard, J. (1996). Gender and McPherson, K. (2006). Wikis and student writing.
sexuality in the cyberspace frontier. Women & Therapy, ProQuest Information and Learning. Teacher Librar-
19, 109-119. ian. Retrieved October 25, 2007, from http://redorbit.
com/news/education/761377/wikis_and_student_writ-
McCulloch, M. (1993). Total quality management: Its
ing/index.html
relevance for higher education. Quality Assurance,
1(2), 5-11. Mehlenbacher, B., Miller, C. R., Covington, D., & Larsen,
J. S. (2000). Active and interactive learning online: A
McCune, V. (2003). Promoting high-quality learning:
comparison of Web-based and conventional writing
Perspectives from the ETL project. In Proceedings:
classes. IEEE Transactions on Professional Communi-
14th Conference on University and College Pedagogy.
cation, 43(2), 166-184.
Fredrikstad: Norwegian Network in Higher Education.
MENON Network EEIG. (2004). Sustainable envi-
McDonald, J., & Mcateer, E. (2003). New approaches
ronment for the evaluation of quality in e-learning.
to supporting students: Strategies for blended learning
SEEQUEL core quality framework. E-Learning Initia-
in distance and campus based environments. Journal of
tive, European Commission.
Educational Media, 28(2-3), 129-146.
MENTOR. (2001). US quality standards for e-mentor-
McKnight, R., & Demers, N. (2002). Evaluating course
ing: Elements of effective practice for e-mentoring.
Web site utilization by students using Web tracking
E-Mentoring Clearinghouse. Retrieved October, 2005,
software: A constructivist approach. In Proceedings of
from www.mentoring.org/emc
the Technology, Colleges and Community Worldwide
Online Conference 2002, Kapio’lani, Hi: University of Merceron, A., & Yacef, K. (2003). A Web-based tutor-
Hawaii. Retrieved October 19, 2007, http://kolea.kcc. ing tool with mining facilities to improve learning and
hawaii.edu/tcc/tcon02/presentations/mcknight.html teaching. In 11th International Conference on Artificial
Intelligence in Education (pp. 41-52).
McLean, M. (2004). Does the curriculum matter in peer
mentoring? From mentee to mentor in problem-based Mergen, E., Grant, D., & Widrick, S.M. (2000). Quality
learning: A unique case study. Mentoring and Tutoring, management applied to higher education. Total Quality
12(2), 173-186. Management, 11(3), 345-352.
McLoughlin, C., & Luca, J. (2001). Quality in online Mertens, R., Farzan, R., & Brusilovsky, P. (2006).
delivery: What does it mean for assessment in e-learning Social navigation in Web lectures. In U. K. Wiil, P. J.
environments? In G. Kennedy, M. Keppell, C. McNaught Nürnberg & J. Rubart (Eds.), Proceedings of Hypertext
Conference 2006.
0
Metros, S.E., & Bennett, K. (2002). Learning objects Monk, A., & Howard, S. (1998, March-April). The
in higher education. Educause Research Bulletin, 19, rich picture: A tool for reasoning about work context.
2. Retrieved October 16, 2007, from www.educause. Interactions, 21-30.
edu/ir/library/pdf/ERB0219.pdf
Moodle. (2006). Retrieved October 21, 2007, from
Meyen, E.L., Aust, R., Gauch, J.M., Hinton, H.S., http://moodle.org/doc/
Isaacson, R.E., Smith, S.J., et al. (2002 ). E-learning: A
Moodle. (2006). Retrieved October 22, 2007, from
programmatic research construct for the future. Journal
http://demo.moodle.com/
of Special Education Technology, 17(3), 37-46.
Moodle. (2007). Philosophy. Retrieved October 17, 2007,
Michalsky, R.S., & Stepp, R.E. (1983). Automated con-
from http://docs.moodle.org/en/Philosophy
struction of classifications: Conceptual clustering versus
numerical taxonomy. IEEE Transactions on Pattern Moore, G. R. (1991). Computer to computer: Mentoring
Analysis and Machine Intelligence, 5(4), 396-410. possibilities. Educational Leardership, 49(3), 40.
Mickle, M. H., Shuman, L., & Spring, M. (2004). Active Mor, E., & Minguillón, J. (2004). E-learning personal-
learning courses on the cutting edge of technology. In ization based on itineraries and long-term navigational
Proceedings of the 34th Annual Conference Frontiers behavior. In Thirteenth World Web Conference (pp.
in Education (FIE’04) (Vol. 1, pp. T2F/19 – T2F/23). 264-265).
Morley, R. (1996). Painting trucks at general motors:
Miller, A. (2004). E-mentoring: An overview. Paper The effectiveness of a complexity-based approach. In
presented at the First Aimhigher Networking Meeting, Ernst and Young Center for Business Innovation, (Ed.),
Aston University. Embracing Complexity: Exploring the Application of
Complex Adaptive Systems to Business (pp. 53-58).
Miller, C.M.I., & Parlett, M. (1974). Up to the mark: A
Cambridge, MA.
study of the examination game. Guildford: Society for
Research into Higher Education. Morozov, M., Tanakov, A., Gerasimov, A., Bystrov, D.,
& Cvirco, E. (2004). Virtual chemistry laboratory for
Minaei, B., Kashy, D.A., Kortemeyer, G., & Punch, W.
school education. In Proceedings of the IEEE Interna-
(2003). Predicting student performance: An application
tional Conference on Advanced Learning Technologies
of data mining methods with an educational Web-based
(ICALT’04) (pp. 605-608). IEEE Computer Society.
system. In Proceedings of 33rd Frontiers in Education
Conference. Mortera-Gutierrez, F. (2006). Faculty best practices us-
ing blended learning in e-learning and in face-to-face
Mishra, S. (2002). A design framework for online learning
instruction. International Journal on E-Learning, 5(3),
environments. British Journal of Educational Technol-
313-337.
ogy, 33(4), 493-496.
Murray, T. (1999). Authoring intelligent tutoring systems:
Mitra, A., & Steffensmeier, T. (2000). Change in student
An analysis for the state of the art. International Journal
attitudes and student computer use in a computer-enriched
of Artificial Intelligence in Education, 10, 98-129.
environment. Journal of Research on Computing in
Education, 32(3), 417-431. Muscettola, N., Dorais, G.A., Fry, C., Levinson, R., &
Plaunt, C. (2002). IDEA: Planning at the core of autono-
Miyahara, K., & Pazzani, M. (2000). Collaborative
mous reactive agents. In Proceedings of the Workshop
filtering with the simple Bayesian classifier. In Pacific
Online Planning and Scheduling, AIPS 2002 (pp. 49-55).
Rim International Conference on Artificial Intelligence
Toulouse, France.
(pp. 679-689).

Myers, I.B., McCaulley, M.H., Quenk, N.I., & Hammer, O’Neill, K. (2004). Building social capital in a knowl-
A.L. (1999). MBTI manual: A guide to the development edge-building community: Telementoring as a catalyst.
and use of the Myers-Briggs Type Indicator. Paolo Alto, Interactive Learning Environments, 12(3), 179-208.
CA: Consulting Psychologist Press.
O’Neill, K., & Harris, J. (2005). Bridging the perspectives
Nachmias, R. (2002). A research framework for the study and developmental needs of all participants in curricu-
of a campus-wide Web-based academic instruction proj- lum-based telementoring programs. Journal of Research
ect. Internet and Higher Education, 5(3), 213-229. on Technology in Education, 37(2), 111-128.
NADE. (2001). Quality standards for distance educa- O’Neill, K., Harris, J., Cravens, J., & Neils, D. (2002).
tion. Norwegian Association for Distance and Flexible Perspectives on e-mentoring: A virtual panel holds an
Education. Retrieved October 30, 2007, from http://www. online dialogue. National Mentoring Center Newslet-
nade-nff.no/nff2/filer/Kvalitet/Kvalitetsnormer%20for ter, 9, 5-12.
%20fjernundervisning.pdf
O’Regan. (2006). MOLIE: Mentoring online in Europe.
Ndubisi, N. O. (2006). Factors of online learning adoption: Salford: University of Salford.
A comparative juxtaposition of the theory of planned
O’Reilly, T. (2005). Web 2.0: Compact definition? Re-
behaviour and the technology acceptance model. Inter-
trieved October 18, 2007, from http://radar.oreilly.com/
national Journal on E-Learning, 5(4), 571-591.
archives/2005/10/web_20_compact_definition.html
Neal, L. (2006, January 19). Predictions for 2006: E-
O’Reilly, T. (2005). What Is Web 2.0. design patterns
learning experts map the road ahead. eLearn Magazine.
and business models for the next generation of software.
Retrieved October 19, 2007, from http://www.elearnmag.
O’Reilly Web. Retrieved October 25, 2007, from http://
org/subpage.cfm?section=articles&article=31-1
www.oreillynet.com/pub/a/oreilly/tim/news/2005/09/30/
Ngai, E., & Poon, J. (2007). Empirical examination of what-is-web-20.html
the adoption of WebCT using TAM. Computers and
ODLQC. (2006). Open and distance learning quality
Education, 42(2), 250-267.
council standards. Open and Distance Learning Quality
Noe, R. A. (1998). An investigation of the determinants of Council. Retrieved October 30, 2007, from http://www.
successful assigned mentoring relationships. Personnel odlqc.org.uk/standard.doc
Psychology, 41, 457-479.
OECD. (2005). Formative assessment: Improving learn-
Nowack, K. (1996). Is the Myers Briggs Type Indicator ing in secondary classrooms. Paris: OECD.
the right tool to use? Performance in Practice, 6.
Oliver, M., & Shaw, G. P. (2003). Asynchronous discus-
O’Donovan, B., Price, M., & Rust, C. (2004). Know what sion in support of medical education. Journal of Asyn-
I mean? Enhancing student understanding of assessment chronous Learning Networks, 7(1), 56-67.
standars and criteria. Teaching in Higher Education,
Oliver, M., & Trigwell, K. (2005). Can blended learning
9(3), 325-336.
be redeemed? E-Learning, 2(1). Retrieved October 18,
O’Hear, S. (2005). Seconds out, round two. The Guardian. 2007, from http://www.wwwords.co.uk/pdf/viewpdf.
Retrieved October 25, 2007, from http://education.guard- asp?j=elea&vol=2&issue=1&year=2005&article=3_Oli-
ian.co.uk/elearning/story/0,10577,1642281,00.html ver_ELEA_2_1_web&id=83.104.158.140
O’Hear, S. (2006). E-learning 2.0 - how Web technologies Omalley, J., & McCraw, H. (1999). Student perceptions
are shaping education. In R. MacManus (Ed.). Retrieved of distance learning, online learning and the traditional
October 25, 2007, from http://www.readwriteweb.com/ classroom. Online Journal of Distance Learning Admin-
archives/e-learning_20.php istration, 2(4), 1-16.

ONeill, K., & Harris, J. (2000, April 24-28). Is everybody Pearce, J. (2006) Using wiki in education. The Science
happy? Bridging the perspectives and perspectives and of Spectroscopy. Retrieved October 25, 2007, from
developmental needs of participants in telementoring http://www.scienceofspectroscopy.info/edit/index.
programs. Paper presented at the Annual Meeting of php?title=Using_wiki_in_education
the American Educational Research Association, New
Pejtersen, A.M. (1989). The BOOKHOUSE: An icon
Orleans. LA.
based database system for fiction retrieval in public
Onions, C.T. (Ed.). (1983). The shorter Oxford English libraries. In Proceedings of 7th Nordic Information and
dictionary (3rd ed.). Oxford: Oxford University Press. Documentation Conference, Århus, Denmark.
Orlikowski, W. J. (2000). Using technology and constitut- Pennock, D., Horvitz, E., Lawrence, S., & Lee Giles, C.
ing structure: A practice lens for studying technology in (2000). Collaborative filtering by personality diagnosis:
organizations. Organizational Science, 11(4), 404-428. A hybrid memory- and model-based approach. In 16th
Conference on Uncertainty in Artificial Intelligence
Orsini, J.N. (2000). Profound education. Total Quality
(pp. 481-488).
Management, 11(4-6), 762-766.
Penuel, B., & Roschelle, J. (1999). Designing learning:
Ortigosa, A., & Carro, R. (2003). The continuous empiri-
Cognitive science principles for the innovative organiza-
cal evaluation approach: Evaluating adaptive Web-based
tion. Stanford Research Institute International, 1-26.
courses. User modeling. Lecture Notes in Computer
Science, 2702, 163-167. Pérez Juste, R. (2006). Evaluación de programas edu-
cativos. Madrid: La Muralla.
Owlia, M.S., & Aspinwall, E.M. (1996) A framework
for the dimensions of quality in higher education. Total Pérez Juste, R., López, F., Peralta, M.D., & Municio,
Quality Management, 7(2). P. (2004). Hacia una educación de calidad. Gestión,
instrumentos y evaluación. Madrid: Narcea.
Paloff, R.M., & Pratt, K. (1999). Building communities in
cyberspace: Effective strategies for the online classroom. Perren, L. (2003). The role of e-mentoring in entrepreneur-
San Francisco: Jossey-Bass. ial education and support: A meta-review of academic
literature. Education and Training, 45(8-9), 517-525.
Paredes, P., & Rodríguez, P. (2002). Considering sens-
ing-intuitive dimension to exposition-exemplification Perrin Glorian, M. J. (1994). Théorie des situations di-
in adaptive sequencing. In P. De Bra, P. Brusilovsky & dactiques: Naissance, développement, perspectives. In
R. Conejo (Eds.), Adaptive hypermedia and adaptive M. Artigue, R. Gras, C. Laborde & P. Tavignot (Eds.),
Web-based systems. Lecture Notes in Computer Sci- Vingt ans de didactique des mathématiques en France
ence, 2347, 556-559. (pp. 97-147). Paris: La Pensée Sauvage.
Parshall, C. G., Davey, T., & Pashley, P. J. (2000). Innova- Peterson, E. R., Deary, I. J., & Austin, E. J. (2003). The
tive item types for computerized testing. In W. Van der reliability of Riding’s Cognitive Style Analysis test.
Linden & C. A. W. Glas (Eds.), Computerized adaptive Personality and Individual Differences, 34, 881-891.
testing: Theory and practice (pp. 129-148). Norwell,
Petrova, K. (2001).Teaching differently: A hybrid delivery
MA: Kluwer Academic Publisher.
model. In N. Delener & C. N. Chao (Eds.), Proceedings
Patel, N.V. (1995). Application of soft systems methodol- of the 2001 Global Business and Technology Association
ogy to the real world process of teaching and learning. International Conference (pp. 717-727). Istanbul, Turkey:
International Journal of Educational Management, Global Business and Technology Association.
9(1), 13-23.

Petrova, K. (2002). Course design for flexible learning. Pillay, H. (1998). An investigation of the effect of
New Zealand Journal of Applied Computing and Infor- individual cognitive preferences on learning through
mation Technology, 6(1), 45-50. computer-based instruction. Educational Psychology,
18(2), 171-182.
Petrova, K. (2007). Mobile learning as a mobile busi-
ness application. International Journal of Innovation in Pils, C., Roussaki, L., & Strimpakou, M. (2006). Loca-
Learning, 4(1), 1-13. tion-based context retrieval and filtering. Lecture Notes
in Computer Science, 3987, 256-273.
Petrova, K., & Sinclair, R. (2005). Business under-
graduates learning online: A one semester snapshot. Pintrich, P. R., & Schrauben, B. (1992). Students’ mo-
International Journal of Education and Development tivational beliefs and their cognitive engagement in
using Information and Communication Technology, 1(4), classroom academic tasks. In D. H. Schunk & J. L. Meece
69-88. Retrieved October 19, 2007, from http://ijedict. (Eds.), Student perceptions in the classroom. Hillslade,
dec.uwi.edu/viewissue.php?id=6 NJ: Lawrence Erlbaum.
Pettigrew, A. (1990). Longitudinal field research on Piramuthu, S. (2005). Knowledge-based Web-enabled

change: Theory and practice. Organization Science, agents and intelligent tutoring systems. IEEE Transac-
1(3), 267-291. tions on Education, 48(4), 750-756.
Phillimore, R. (2002). Face to face lectures or econtent: Pituchs, K.A., & Lee, Y.-K. (2006). The influence of
Student and staff perspectives. In R. Kinshuk, K. Lewis, system characteristics on e-learning use. Computers &
R. Akahori, T. Kemp, L. Okamoto, C. Henderson & H. Education, 47(2), 222-244.
Lee (Eds.), Proceedings of the 9th International Confer-
Plous, S. (2000). Tips on creating and maintaining an
ence on Computers in Education (Vol. 1, pp. 211-212).
educational World Wide Web site. Teaching of Psychol-
Auckland, New Zealand: Asia-pacific Society for Com-
ogy, 27, 63-70.
puters in Education.
Polo, F. (2006). Marketing 2.0 New way to old things.
Philpot, T. A., Hall, R. H., Hubing, N., & Flori, R. E.
Jornada Internet de Nueva Generación: Web 2.0, Internet
(2005). Using games to teach statics calculation proce-
2.0. Spain. Retrieved October 25, 2007, from http://inter-
dures: Application and assessment. Computer Applica-
netng.dit.upm.es/ponencias-jing/2006/polo.pdf
tions in Engineering Education, 13(3), 222-232.
Ponzurick, T. G., France, K.R., & Logar, C.M. (2000).
Phipps, R., & Merisotis, J. (1999). What’s the difference?
Delivering graduate marketing education: An analysis
A review of contemporary research on the effectiveness
of face-to-face versus distance education. Journal of
of distance learning in higher education. Washington,
Marketing Education, 22(3), 180-187.
DC: The Institute for Higher Education Policy.
Poole, D. M. (2000). Student participation in a discussion-
Phipps, R., & Merisotis, J. (2000). Quality on the line:
oriented online course: A case study. Journal of Research
Benchmarks for success in Internet-based distance
on Computing in Education, 33(2), 162-177.
education. Washington: IHEP - Institute for Higher
Education Policy. Popham, W.J. (1983). Problemas y técnicas de evaluación
educativa. Madrid: Anaya
Picciano, A. (2002). Beyond student perceptions: Issues
of interaction, presence, and performance in online Potelle, H., & Rouet, J. F. (2003). Effects of content
course. Journal of Asynchronous Learning Networks, representation and readers’ prior knowledge on the
6(1), 21-40. comprehension of hypertext. International Journal of
Human-Computer Studies, 58, 327-345.

Preece, J., Rogers, Y., & Sharp, H. (2002). Interaction Recommendation of the European Parliament and the
design: Beyond human computer interaction. Wiley. Council of 18 December 2006 on Key Competences for
Lifelong Learning. (2006, December 12). Official Jour-
Prensky, M. (2001). Digital natives digital immigrants.
nal of the European Union Retrieved October 18, 2007,
On the Horizon NCB University Press, 9(5).
from http://www.cimo.fi/dman/Document.phx/~public/
Prinz, W. (2006). Social Web applications. Jornada Inter- Sokrates/Comenius/keycompetences06.pdf
net de Nueva Generación: Web 2.0, Internet 2.0. Spain.
Reeves, T. C. (2000). Alternative assessment approaches
Retrieved October 25, 2007, from http://internetng.dit.
for online learning environments in higher education.
upm.es/ponencias-jing/2006/prinz.pdf
Journal of Educational Computing Research, 23(1),
Puntambekar, S. (2006). Analyzing collaborative 101-111.
interactions: Divergence, shared understanding and
Reeves, W. (1999). Learner-centered design: A cognitive
construction of knowledge. Computers & Education,
view of managing complexity in product, information,
47(3), 332-351.
and environmental design. Thousand Oaks, CA: Sage.
QAA. (2004). Code of practice for the assurance of
Reichlmayr, T. (2005). Enhancing the student project
academic quality and standards in higher education (2nd
team experience with blended learning techniques. In
ed.). Mansfield: Quality Assurance Agency for Higher
Proceedings of the 35th Annual Conference Frontiers
Education.
in Education (FIE’05) (pp. T4F/6- T4F/11). Piscataway,
Qual E-Learning Project Consortium. (2003). Qual-e- NJ: IEEE.
learning project. Qual E-Learning Project Consortium.
Reilly, R. (2005). Guest editorial Web-based instruc-
Retrieved October 30, 2007, from http://www.qual-e-
tion: Doing things better and doing better things. IEEE
Learning.net/cgi/index.php?wpage=overview
Transactions on Education, 48(4), 565-566.
Qual E-Learning Project Consortium. (2004). Handbook
Rekkedal, T. (2006). Distance learning and e-learning
of best practices for the evaluation of e-learning effec-
quality for SMEs—State of the art. Paper prepared for
tiveness. Qual e-Learning Project Consortium.
the EU Leonardo Project, E-learning Quality for SMEs:
Qual E-Learning Project Consortium. (2007) Qual Guidance and Counselling.
e-learning evaluation tool. Qual-e-Learning Project.
Rengarajan, R. (2001). LCMS and LMS: Taking ad-
Retrieved October 30, 2007, from http://www.qual-e-
vantage of tight integration. Click 2 Learn. Retrieved
Learning.net/cgi/index.php
October 28, 2007, from http://www.e-learn.cz/soubory/
Quality Assurance Agency for Higher Education. (2002). lcms_and_lms.pdf
Distance learning guidelines. Retrieved October 19,
Rhodes, J. E. (2002). A critical view of youth mentor-
2007, from http://www.qaa.ac.uk/public/dlg/dlg_tex-
ing. Boston.
tonly.htm
Riding, R., & Rayner, S. (1998). Cognitive styles and
Quinlan, R.J (1992). C4.5: Programs form machine
learning strategies: Understanding style differences
learning. San Mateo, CA: Morgan Kaufmann.
in learning and behaviour. London: David Fulton Pub-
Quinn, D., & Reid, I. (2003). Using innovative online lishers.
quizzes to assist learning. Retrieved October 29, 2007,
R-Moreno, M.D. (2003). Representing and planning tasks
from http://ausweb.scu.edu.au/aw03/papers/quinn/paper.
with time and resources. Ph.D. Thesis, Universidad de
html
Alcalá.

R-Moreno, M.D., & Camacho, D. (2007). AI techniques Romilly, J.D. (1992). The great sophists in Periclean
for automatic learning design. In Proceedings of the Athens. Oxford, UK, New York: Clarendon Press, Oxford
International e-Conference of Computer Science (IeCCS University Press.
2006), Lecture Series on Computer and Computational
Rosales, C. (1990). Evaluar es reflexionar sobre la en-
Sciences (LSCCS) (vol. 8, pp. 193-197). VSP/Brill Aca-
señanza. Madrid: Narcea.
demic Publishers.
Rosen, A. (2006). Technology trends: E-learning 2.0.
R-Moreno, M.D., Oddi, A., Borrajo, D., & Cesta, A.
Learning Solutions E-Magazine. Retrieved October
(2006). IPSS: A hybrid approach to planning and sched-
25, 2007, from http://www.readygo.com/e-learning-
uling integration. IEEE Transactions on Knowledge and
2.0.pdf
Data Engineering, 18(12), 1681-1695.
Ross, B. (2004). First aimhigher e-mentoring networking
Robey, D., & Bourdreau, M. (1999). Accounting for the
meeting. Birmingham: Middlesex University.
contradictory organizational consequences of informa-
tion technology: Theoretical directions and method- Ross, J.A., & Bruce, C.D. (2007). Teacher self-assess-
ological implications. Information Systems Research, ment: A mechanism for facilitating professional growth.
10(2), 167-185. Teaching & Teacher Education: An International Journal
of Research and Studies, 23(2), 146-159.
Robinson, A., & Udall, M. (2006). Using formative as-
sessment to improve student learning through critical Rowntree, D. (1982). Educational technology in cur-
reflection. In C. Bryan & K. Clegg (Eds.), Innovative riculum development. Newcastle upon Tyne, UK: Ath-
assessment in higher education (pp. 92-99). Oxon: enaeum Press Ltd.
Routledge.
Rudner, L. M. (1998). An online, interactive, computer
Rodríguez-Conde, M.J. (2006). Teaching evaluation in adaptive testing tutorial. Retrieved October 29, 2007,
an e-learning enviroment. In E. Verdú, M.J. Verdú, J. from http://edres.org/scripts/cat
García & R. López (Eds.), Best practices in e-learning.
Ruttenbur, B., Spickler, G., & Lurie, S. (2000). E-learn-
Towards a technology-based and quality education (pp.
ing—The engine of the knowledge economy. New York:
55-70). Valladolid: BEM.
Morgan Keegan & Co.
Rodríguez-Conde, M.J., & Herrera-García, M.E. (2005).
Ryan, Y. (2000). Assessment in online teaching. Paper
Assessment processes in tele-learning programmes. In
presented at the Australian Universities Teaching Com-
F.J. García, J. García, M. López, R. López & E. Verdú
mittee Forum 2000, Canberra, Australia. Retrieved
(Eds.), Educational virtual spaces in practice the Odis-
October 29, 2007, from http://www.autc.gov.au/forum/
eame approach. (pp. 161-178). Barcelona: Ariel.
papers/onlineteaching1.rtf
Rogers, E. M. (1995). Diffusion of innovations. New
Saaty, T. L. (1994). How to make a decision: The analytic
York: Free Press.
hierarchy process. Interfaces, 24(6), 19-43.
Rohrer, R.M., & Swing, E. (1997). Web-based informa-
Saba, F. (1999). Is distance education comparable to
tion visualization. Computer Graphics and Applications,
traditional Education? Retrieved October 19, 2007,
IEEE, 17(1/4), 52-59.
from http://www.distance-educator.com/der/comparable.
Romero, C., Ventura, S., De Bra, P., & Castro, C. (2003). html
Discovering prediction rules in AHA! courses. In 9th In-
Sadler, D. R. (1989). Formative assessment and the
ternational Conference on User Modeling (pp. 25-34).
design of instructional systems. Instructional Science,
18, 119-144.

Sadler, D. R. (1998). Formative assessment: Revisiting Schmitz, C., Staab, S., Studer, R., Stumme, G., & Tane
the territory. Assessment in Education, 5(1), 77-84. J. (2002). Accessing distributed learning repositories
through a courseware watchdog. In Proceedings of the
Sahney, S., Banwet, D.K., & Karunes, S. (2004). Concep-
E-Learn 2002-World Conference on E-learning in Corpo-
tualizing total quality management in higher education.
rate, Government, Healthcare for Higher Education.
The TQM Magazine, 16(2), 145-159.
Schön, D.A. (1971). Beyond the stable state: Public and
Salazar, A., Gosalbez, J., Bosch, I., Miralles, R., & Ver-
private learning in a changing society. Temple Smith.
gara, L. (2004). A case study of knowledge discovery
on academic achievement, student desertion and student Schön, D.A. (1983). The reflective practitioner. NewYork:
retention. In IEEE 2th International Conference on Basic Books.
Information Technology: Research and Education (pp.
Schön, D.A. (1987). Educating the reflective practitioner:
150-154).
Toward a new design for teaching and learning in the
Salmon, G. (2000). E-moderating. The key to teaching professions. San Francisco: Jossey-Bass Publishers.
and learning online. London: Kogan Page.
Schön, D.A. (1991). The reflective turn: Case studies in
Sambell, K., & Mcdowell, L. (1998). The construction of and on educational practice. New York: Teachers Press,
the hidden curriculum: Messages and meanings in the Columbia University.
assessment of student learning. Assessment and Evalu-
Schulze, A., & O’Keefe, A. (2002, August). Effectively
ation in Higher Education, 23(4), 391-402.
using self-assessment in online learning. Paper presented
Sanders, D., & Morrison-Shetlar, A. I. (2001). Student at the 18th Annual Conference on Distance Teaching
attitudes towards Web-enhanced instruction in an Learning, Madison, Wisconsin.
introductory biology course. Journal of Research on
SCORM. (2006). Sharable Courseware Object Reference
Model. Retrieved October 22, 2007, from http://www.
Scalan, C. L. (2003). Reliability and validity of a student academiccolab.org/projects/scorm.html
scale for assessing the quality of Internet-based distance
Scott, D.W., & Sain, S.R. (2004). Multi-dimensional
learning. Distance Learning Administration, VII (III).
density estimation. In C. R. Rao, E. J. Wegman & J.
Retrieved October 30, 2007, from http://www.westga.
L. Solka (Eds.), Handbook of Statistics, Data Mining
edu/~distance/ojdla/fall63/scanlan63.html
and Computational Statistics, Vol. 24, (pp. 229-261).
Scalise, K., & Gifford, B. (2006). Computer-based as- Elsevier.
sessment in e-learning: A framework for constructing
Scott, G. (2001). Assuring quality for online learning.
“Intermediate Constraint” questions and tasks for tech-
Retrieved October 19, 2007, from http://www.qdu.uts.
nology platforms. The Journal of Technology, Learning,
edu.au/pdf%20document/QA%20for%200
and Assessment, 4(6), 1-44.
Scriven, M. (1967). The methodology of evaluation. In
Schacht, P. (2006). The collaborative writing project. Re-
R. Tyler, R. Gagne & M. Scriven (Eds.), Perspectives
trieved October 25, 2007, from http://node51.cit.geneseo.
of curriculum evaluation (pp. 39-83). Chicago: Rand
edu/WIKKI_TEST/mediawiki/index.php/Main_Page
McNally.
Schellens, T., & Valcke, M. (2006). Fostering knowledge
Scriven, M. (1995). Student ratings offer useful input to
construction in university students through asynchro-
teacher evaluations. Washington, DC: ERIC Clearing-
nous discussion groups. Computers & Education, 46(4),
house on Assessment and Evaluation, Catholic University
349-370.
of America.

Scriven, M. (1999). The nature of evaluation (Pts. I–II). lifelong learning. Definition, modalities, methodology,
Washington, DC: ERIC Clearinghouse on Assessment competences and skills (CEUR Workshop Proceedings).
and Evaluation. Virtual Campus 2006. Selected and Extended Papers,
186, 41-55.
SEEL. (2004). Quality guidelines for learning strategy
and innovation, version 3. Supporting Excellence in Seoane, A.M., García, F.J., Bosom, Á., Fernández, E., &
E-Learning. Hernández, M. J. (2007). Online tutoring methodology
approach. International Journal of Continuing Engi-
Selim, H. (2007). Critical success factors for e-learning
neering Education and Life-Long Learning (IJCEELL),
acceptance: Confirmatory factor models. Computers &
17(6), 479-492.
Education, 49(2), 396-413.
Seufert, S., & Euler, D. (2005). Nachhaltigkeit von
Selim, H. M. (2003). An empirical investigation of stu-
e-Learning-Innovationen: Fallstudien zu Implemen-
dent acceptance of course Web sites. Computers and
tierungsstrategien von e-Learning als Innovationen an
Education, 40, 343-360.
Hochschulen. St. Gallen: Swiss Centre for Innovations in
Selim, H. M. (2005). Elearning critical success factors: Learning. Retrieved October 30, 2007, from http://www.
an exploratory investigation of student perceptions. In scil.ch/publications/docs/2005-01-seufert-euler-nach-
M. Khosrow-Pour (Ed.), Managing modern organiza- haltigkeit-e-Learning.pdf
tions with information technology. Proceedings of the
SEVAQ. (2005). Self evaluation for quality in e-learn-
2005 Information Resources Management Association
ing. Retrieved October 30, 2007, from http://www.
International Conference (pp. 340-346). San Diego, CA:
sevaq.com
Information Resources management Association.
Sfard, A. (2000). Symbolizing mathematical reality into
Selwyn, N. (1999). Students’ attitudes towards comput-
being—or how mathematical discourse and mathematical
ers in sixteen to nineteen education. Education and
objects create each other. In P. Cobb, E.Yackel & K.
Information Technologies, 4(2), 129-141.
McClain (Eds.), Symbolizing and Communicating in
Semet, Y., Lutton, E., & Collet, P. (2003). Ant colony Mathematics Classrooms. Mahwah, NJ: Lawrence
optimization for e-learning: Observing the emergence Erlbaum Associates.
of pedagogic suggestions. In Proceedings of the 2003 Shank, R. C. (2002). Designing world-class e-learning.
IEEE Swarm Intelligence Symposium. McGraw Hill.
Seoane Pardo, A.M , & García Peñalvo, F.J. (in press). Sharpe, R., & Benfield, G. (2005). The student experience
Tutoring & mentoring online. Definition, roles, skills of e-learning in higher education. Brookes eJournal of
and case studies. In G.D. Putnik & M.M. Cunha (Eds.), Learning and Teaching, 3(1), 1-9. Retrieved October 19,
Encyclopedia of networked and virtual organizations. 2007, from http://www.brookes.ac.uk/publications/bejlt/
Hershey, PA: Idea Group Inc. volume1issue3/academic/sharpe_benfield.pdf
Seoane Pardo, A.M., & García Peñalvo, F.J. (2006). Shee, D., & Wang, Y. (in press). Multi-criteria evaluation
Determining quality for online Activities. Methodology of the Web-based e-learning system: A methodology
and training of online tutors as a challenge for achiev- based on learner satisfaction and its applications. Com-
ing the excellence. WSEAS Transactions on Advances puters & Education.
in Engineering Education, 3(9), 823-830.
Sherman, R. C. (1998). Using the World Wide Web to
Seoane Pardo, A.M., García Peñalvo, F.J., Bosom Nieto, teach everyday applications of social psychology. Teach-
Á., Fernández Recio, E., & Hernández Tovar, M.J. (2006). ing of Psychology, 25, 212-216.
Tutoring online as quality guarantee on e-learning-based

Shin, N., & Kim, J. (1999). An exploration of learner Single, P. B., & Muller, C. B. (2000, April 24-28). Elec-
progress and dropout in Korea National Open Univer- tronic mentoring: Quantifying the programmatic effort.
sity. Distance Education an International Journal, 20, Paper presented at the Annual meeting of the American
81-97. Educational Research Association, New Orleans, LA.
Shuell, T. (1992). Designing instructional computing Single, P. B., & Muller, C. B. (2001). When e-mail and
systems for meaningful learning. In M. Jones & P. mentoring unite: The implementation of a nationwide
Winne (Eds.), Adaptive learning environments. New electronic mentoring program--MentorNet, the national
York: Springer Verlag. electronic industrial mentoring network for women in
engineering and science. American Society for Training
Sicilia, M.A., Sánchez-Alonso, S., & García-Barriocanal,
and Development.
E. (2006, March 23-25). In Proceedings on Supporting
the Process of Learning Design Through Planners. Vir- Single, P. B., & Muller, C. B. (2005). Electronic mentor-
tual Campus 2006 Post-Proceedings, CEUR Workshop ing programs: A model to guide practice and research.
Proceedings (vol. 186). Barcelona, Spain. Mentoring and Tutoring, 13(2), 305-320.
Siemens, G. (2005). Connectivism: A learning theory Single, P. B., & Muller, C. B. (forthcoming). Electronic
for the digital age. Retrieved October 25, 2007, from mentoring programs: A model to guide practice and
http://www.elearnspace.org/Articles/connectivism.htm research. Retrieved January 2008 from www.apesma.
asu.au/mentorsonline/reference/pdfs/muller_and_boy-
Silander, P., & Rytkohen, A. (2005). An intelligent mo-
le_single.pdf
bile tutoring tool enabling individualization of students’
learning processes. In Proceedings of the 4th World Con- Single, P. B., & Single, R. M. (2005). E-mentoring for
ference on mLearning (paper 59), Cape Town, Republic social equity: Review of research to inform program
of South Africa. development. Mentoring and Tutoring, 13(2), 301-320.
Silvescu, A., Reinoso-Castillo, J., & Honavar, V. (2001). Single, P. B., & Single, R. M. (2005). Mentoring and the
Ontology-driven information extraction and knowledge technology revolution: How face-to-face mentoring sets
acquisition from heterogeneous, distributed, autonomous the stage for e-mentoring. In F. K. Kochan & J. T. Pascarelli
biological data sources. In International Joint Confer- (Eds.), Creating successful telementoring programs (pp.
ences on Artificial Intelligence (IJCAI) (pp. 1-10). 7-27). Greenwich: Information Age Press.
Simons, R. J., van der Linden, J., & Duffy, T. (Eds.). Single, R. M., Muller, C. B., Cunningham, C. M., &
(2000). New learning. Dordrecht: Kluwer Academic. Single, R. M. (2000). Electronic communities: A forum
for supporting women professionals and students in
Sinclair, R. M. S. (2003). Components of quality in
scientific fields. Journal of Women and Minority? En-
distance education. In G. Davies & E. Stacey (Eds.),
gineering, 6, 115-129.
Quality education @ a distance (pp. 257-264). Boston:
Kluwer Academic Publishers. Sirvanci, M.B. (2004). TQM implementation: Critical
issues for TQM implementation in higher education.
Sinclair, R. M. S. (2003). Stakeholders’ views of quality in
The TQM Magazine, 16(6), 382-386.
flexibly delivered courses. Unpublished Masters Research
Paper. Deakin University, Australia: Geelong. Sluijsmans, D. M. A., Prins, F. J., & Martens, R. L.
(2006). The design of competency-based performance
Single, P. B., & Muller, C. B. (1999). Electronic men-
assessment in e-learning. Learning Environments Re-
toring: Issues to advance research and practice. Paper
search, 9, 45-66.
presented at the Annual Meeting of the International
Mentoring Association, Atlanta, GA.

Small, M., & Lohrasbi, A. (2003). Student perspectives Stephens, D. (2001). Use of computer assisted assessment:
on online degrees and courses: An empirical analysis. Benefits to students and staff. Education for Informa-
International Journal on E-learning, 2(2), 15-28. tion, 19, 265-275.
Smith, T., & Ingersoll, R. (2004). What are the effects Stephenson, J.E., Brown, C., & Griffin, D.K. (in press).
of induction and mentoring on beginning teacher turn- Electronic delivery of lectures in the university environ-
over? American Educational Research Journal, 41(3), ment: An empirical comparasion of three delivery styles.
681-714. Computers & Education.
Snyder, B.R. (1971). The hidden curriculum. Cambridge, Stiggins, R. J. (1987). Design and development of per-
MA: MIT Press. formance assessment. Educational Measurement: Issues
and Practice, 4, 263-273.
Song, L., Singleton, E. S., Hill, J. R., & Koh, M. H.
(2004). Improving online learning: Students perceptions Stockley, D. (2003). E-learning definition. Retrieved
of useful and challenging characteristics. Internet and October 18, 2007, from http://derekstockley.com.au/el-
Higher Education, 7(1), 59-70. earning-definition.html
Southern Regional Education Board. (2000). Principles Stokes, A. (2001). Using telementoring to deliver train-
of good practice. Retrieved October 19, 2007, from ing to SMEs: A pilot study. Education + Training, 43(6),
http://ww.electroniccampus.org/student/srecinfor/pu- 317-324.
blicaitons/Principles_2000.pdf.
Stokes, P., Garrett-Harris, R., & Hunt, K. (2003). An
Sowa, J.F. (2000). Knowledge representation: Logical, evaluation of electronic mentoring (e-mentoring). Paper
philosophical and computational foundations. Pacific presented at the 10th European Mentoring & Coaching
Grove, CA: Brooks Cole. Conference.
SPI. (2003). Empre-learning: Promoção de Estruturas Stuffebeam, D.L. (1999). Foundational models for 21st
de e-Learning Inovadores, em Língua Portuguesa, que century program evaluation. Kalamazoo, MI: Western
Permitam o Aumento de Competências e Aumentem Michigan University, The Evaluation Center.
a Empregabilidade. Porto: Sociedade Portuguesa de
Summerville, J. (1999). Role of awareness of cognitive
Inovação.
style in hypermedia. International Journal of Educa-
Spillane, M.G. (1999, June). Portfolio assessment in tional Technology, 1.
higher education: Seeking credibility on the campus.
Sun, P., Tsai, R., Finger, G., Chen, Y., & Yeh, D. (in
Journal of the National Institute of the Assessment of
press). What drives a successful e-learning? An empirical
Experiential Learning, 17-28.
investigation of the critical factors influencing learner
Srivastava, J., Cooley, R., Deshpande, M., & Tan, P. satisfaction. Computers & Education.
(2000). Web usage mining: Discovery and applications
Swan, M. K. (1995). Effectiveness of distance learning
of usage patterns from web data. In SIGKDD Explora-
courses—Students’ perceptions. In Proceedings of the
tions (pp. 12-23).
22nd Annual National Agricultural Education Research
Stake, R.E. & Cohernour, E.J. (1999). Evaluation of Meeting (pp.34-38), Denver, CO. Retrieved October 19,
college teaching in a community of practice. University 2007, from http://www.ssu.missouri.edu/SSU/AgEd/
of Illinois. NAERM/s-a-4.htm
Stake, R.E. (1999). Representing quality in evaluation. Symons, S., & Symons, D. (2002). Using the Inter- and
Paper presented at the annual meeting of the American Intranet in a university introductory psychology course
Educational Research Association, Quebec, Canada. to promote active learning. In Proceedings of the Interna-
0
tional Conference on Computers in Education (ICCE’02) TISIP. (2007). QUIS—Quality, interoperability and
(Vol. 2, pp. 844- 845). IEEE Computer Society. standards in e-learning. Trondheim: TISIP Research
Foundation.
Tait, H., Entwistle, N.J., & McCune, V. (1998). ASSIST:
A reconceptualisation of the approaches to studying Titcomb, S. L., Foote, R. M., & Carpenter, H. J. (2004).
inventory. In C. Rust (Ed.), Improving student learn- A model for a successful high school engineering design
ing (pp. 262-271). Oxford: Oxford Centre for Staff and competition. In Proceedings of the 34th Annual Confer-
Learning Development. ence Frontiers in Education (FIE’04) (Vol. 1, pp. 138-141).
Taras, M. (2001). The use of tutor feedback and student
self-assessment in summative assessment tasks: Towards Trow, M. (1973). Problems in the transition from elite
transparency for students and for tutors. Assessment and to mass higher education. Berkley, CA: Carnegie Com-
Evaluation in Higher Education, 26(6), 606-614. mission on Higher Education.
Taras, M. (2003). To feedback or not to feedback in Trowler, P., & Knight, P.T. (2000). Coming to know in
student self-assessment. Assessment and Evaluation in higher education: Theorising faculty entry to new work
Higher Education, 25(5), 549-565. contexts. Higher Education Research & Development,
19(1).
Tastle, W. J., White, B. A., & Shackleton, P. (2005).
E-learning in higher education: The challenge, effort, Trowler, P., Saunders, M., & Knight, P.T. (2003). Change
and return on investment. International Journal on E- thinking, change practices: A guide to change for heads
Learning, 4(2), 241-250. of department, programme leaders and other change
agents in higher education. Learning and Teaching
Tattersall, C., Manderveld, J., Van den Berg, B., Van
Support Network, Generic Centre.
Es, R., Janssen, J., & Koper, R. (2005). Swarm-based
wayfinding support in open and distance learning. In Tu, C. (2000). Critical examination of factors affecting
E. M. Alkhalifa (Ed), Cognitively informed systems: interaction on CMC. Journal of Network and Computer
Utilizing practical approaches to enrich information Applications, 23, 39-58.
presentation and transfer (pp. 166-183).
Tullous, R., & Utecht, R. L. (1994). A decision support
Taylor, A.W., & Hill, F.M. (1993). Issues for implementing system for integration of vendor selection tasks. Journal
TQM in further and higher education: The moderating of Applied Business Research, 10(1), 132-144.
influence of contextual variables. Quality Assurance in
Twigg, C.A. (1994). The changing definition of learning.
Education, 1(2), 12-21.
Educom Review, 29(4).
Tesone, D. V., & Gibson, J. W. (2001, October). E-mentor-
Ullmo, P. -A., & Ehlers, U. -D. (2007). Quality in e-
ing for professional growth. Paper presented at the IEEE
learning. E-Learning Papers, 2. Retrieved October
International Professional Communication Conference,
30, 2007, from http://www.e-Learningpapers.eu/index.
Santa Fe, NM.
php?page=volume
Thorson, D. (2006). Marketing 2.0: The constellation.
Ullrich, C. (2005). Course generation based on HTN
Retrieved October 25, 2007, from http://donthorson.ty-
planning. In Proceedings of 13th Annual Workshop of
pepad.com/don_thorson/2006/04/the_constellati.html
the SIG Adaptivity and User Modeling in Interactive
Timmerman, B., & Lingard, R. (2003). Assessment of Systems (pp. 74-79).
active learning with upper division computer science Untersteiner, M. (1954). The sophists. New York: Philo-
students. In Proceedings of the 33rd Annual Conference sophical Library.
Frontiers in Education (FIE’03) (Vol. 3, pp. S1D/7 -
S1D/12). Piscataway, NJ: IEEE.

Uribe, C. L., Schweikhart, S. B., Pathak, D. S., Marsh, G. ter-assisted learning module on glomerulonephritis for
B., & Fraley, R .R. (2002). Perceived barriers to medical- medical students. Medical Teacher, 24(4), 412-416.
error reporting: An exploratory investigation. Journal
Verdú, E., Regueras, L. M., Verdú, M. J., Pérez, M. A.,
of Healthcare Management, 47(4), 263-280.
& de Castro, J. P. (2006). Improving the higher education
Uschold, M., King, M., Morales, S., & Zorgios, Y. through technology-based active methodologies: A case
(1998). The enterprise ontology. Knowledge Engineering study. WSEAS Transactions on Advances in Engineering
Review, 13, 32-89. Education, 3(7), 649-656.
Valenta, A., Theriault, D., Dieter, M., & Mrtek, R. (2001). Verdú, E., Regueras, L. M., Verdú, M. J., Pérez, M. A., &
Identifying student attitudes and learning styles in de Castro, J. P. (2006). QUEST: A contest-based approach
distance education. Journal of Asynchronous Learning to technology-enhanced active learning in higher educa-
Networks, 5(2), 111-127. tion. In S. Impedovo, D. Kalpic & Z. Stjepanovic (Eds.),
Proceedings of 6th WSEAS International Conference on
Valiathan, P. (2002). Blended learning models. Learning
Distance Learning and Wb Engineering (DIWEB ‘06)
Circuits. Retrieved October 18, 2007, from http://www.
(pp. 10-15). Wisconsin: WSEAS.
learningcircuits.org/2002/aug2002/valiathan.html
Verdú, E., Regueras, L. M., Verdú, M. J., Pérez, M. A.,
Valigiani, G., Jamont, Y., Biojout, R., Lutton E., & Col-
& de Castro, J. P. (2006). Improving the higher education
let, P. (2005). Experimenting with a real-size man-hill to
through technology-based active methodologies: A case
optimize pedagogical paths. In H. Haddad, L. Liebrock,
study. WSEAS Transactions on Advances in Engineering
A. Omicini & R. Wainwright (Eds.), ACM Symposium
Education, 3(7), 649-656.
on Applied Computing (pp. p4-8).
Verdú, E., Verdú, M. J., Regueras, L. M., & de Castro,
Valigiani, G., Lutton, E., Jamont, Y., Biojout, R., & Col-
J. P. (2005). Intercultural and multilingual e-learning
let, P. (2006). Automatic rating process to audit a man-
to bridge the digital divide. Lecture Notes in Computer
hill. WSEAS Transactions on Advances in Engineering
Sciences, 3597, 260-269.
Education, 3(1), 1-7.
Verdú, M. J., de Castro, J. P., Pérez, M. A., Verdú, E., &
van Bruggen, J., Sloep, P., Van Rosmalen, P., Brouns, F.,
Regueras, L. M. (2006). Application of TIC-based ac-
Vogten, H., & Koper, R. (2004). Latent semantic analysis
tive methodologies in the framework of the new model
as a tool for learner positioning in learning networks
of university education: The educational interaction
for lifelong learning. British Journal of Educational
system QUEST. In F. J. García, J. Lozano & F. Lamamie
Technology, 35(6), 729-738.
de Clairac (Eds.), CEUR Workshop Proceedings, Virtual
Van der Linde, G. (2005). The perception of business Campus 2006 Postproceedings. Selected and Extended
students at PUCMM of the use of collaborative learn- Papers (Vol. 186, pp. 33-40). CEUR-WS.org. Retrieved
ing using the BSCW as a tool. In Proceedings of the 6th October 26, 2007, from http://CEUR-WS.org/Vol-186/
International Conference on Information Technology
Verhaart, M., & Kinshuk, C-K., (2004). Adding seman-
Based Higher Education and Training (ITHET 2005)
tics and context to media resources to enable efficient
(pp. F2D/10- F2D/15). Piscataway, NJ: IEEE.
construction to learning objects. In C. Kinshuk, K.
Van Merriëboer, J. J. G. (1997). Training complex cogni- Looi, E. Sutinen, D. G. Sampson, I. Aedo, L. Uden, &
tive skills. Englewood Cliffs, NJ: Educational Technology E. Kähkönen (Eds.), Proceedings of the 4th International
Publications. Conference on Advanced Learning Technologies (pp.
651-653). Joensuu, Finland: IEEE Computer Society.
Velan, G. M., Killen, M. T., Dziegielewski, M., & Kumar,
R. K. (2002). Development and evaluation of a compu-

Vermetten, Y.J., Vermunt, J.D., & Lodewijks, H.G. (2002). Webster, W.R. (2002, July). Metacognition and the
Powerful learning environments? How university stu- autonomous learner: Student reflections on cognitive
dents differ in their response to instructional measures. profiles and learning environment development. In A.
Learning and Instruction, 12, 263-284. Goody (Ed.), Spheres of influence: Ventures and visions
in educational development. Proceedings of ICED
Vermunt, J.D. (1998). The regulation of constructive
2002, UWA, Perth, Australia: University of Western
learning processes. British Journal of Educational
Australia.
Psychology, 67, 149-171.
Webster, W.R. (2003). Cognitive styles, metacogni-
Vinicini, P. (2001). The use of participatory design
tion and the design of e-learning environments. In F.
methods in a learner-centered design process. ITFORUM
Albalooshi (Ed.), Virtual education: Cases in teaching
54. Retrieved October 18, 2007, from http://it.coe.uga.
and learning (pp. 225-240). Hershey, PA: Idea Group
edu/itforum/paper54/paper54.html
Publishing.
Vora, P. (1998). Human factors methodology for designing
Webster, W.R. (2004, November 2-3). A learner-centred
Web sites. In C. Forsythe, E. Grose & J. Ratner (Eds.),
methodology for learning environment design and de-
Human factors and Web development. Hillsdale, NJ:
velopment. In Exploring integrated learning environ-
Lawrence Erlbaum.
ments. Proceedings, Online Learning and Training 2004,
Vygotsky, L.S., & Cole, M. (1978). Mind in society: Brisbane. Brisbane, Australia: Queensland University
The development of higher psychological processes. of Technology.
Cambridge, MA: Harvard University Press.
Webster, W.R. (2005). A reflective and participatory
Wadia-Fascetti, S., & Leventman, P. G. (2000). E- approach to the design of personalised learning environ-
mentoring: A longitudinal approach to mentoring ments. Unpublished PhD Thesis, Lancaster, Lancaster
relationships for women pursuing technical careers. University.
Journal of Engineering Education, 89(3), 295-300.
Webster’s Online Dictionary. (n.d.). Retrieved October
Wang, K. H., Wang, T. H., Wang, W. L., & Huang, S.
30, 2007, from http://www.websters-online-dictionary.
C. (2006). Learning styles and formative assessment
org/definition/quality
strategies: Enhancing student achievement in Web-
based learning. Journal of Computer Assisted Learning, Weigand, H. (1997). Multilingual ontology-based lexicon
22(3), 207. for news filtering. In IJCAI Workshop on Multilingual
Ontologies (pp. 138-159).
Warburton, B., & Conole, G. (2003). Key findings form
recent literature on computer-aided assessment (pp. Weil, S. (1999). Re-creating universities for beyond the
1-19). ALTC-C University of Southampton. stable state: From dearingesque systematic control to
post-dearing systemic learning and inquiry. Systems
Ward, M., & Newlands, D. (1998). Use of the Web in
Research and Behavioral Science, 16, 170-190.
undergraduate teaching. Computers and Education,
31, 171-184. Welling, L., & Thomson, L. (2003). Using session control
in PHP. In Sams Publishing (Ed.), Php and mysql Web
Waxman, H. C., Lin, M., & Michko, G. M. (2003). A
development. Developer’s Library.
meta-analysis of the effectiveness of teaching and learn-
ing with technology on students outcomes. Naperville, Wells, P., Fieger, P., & de Lange, P. (2005, July). Integra-
IL: Learning Point Associates. ting a virtual learning environment into a second year
accounting course: Determinants of overall student
Webster, J., & Hackley, P. (1997). Teaching effectiveness
perception. Paper presented at the 2005 Accounting and
in technology mediated distance learning. Academy of
Finance Association of Australia and New Zealand Con-
Management Journal, 40(6), 1282-1309.

ference, Melbourne, Australia: Accounting and Finance and implementation, curriculum and instruction. In M.
Association of Australia and New Zealand. Simonson (Ed.), Proceedings of selected research and
development presentations. Washington, DC: Associa-
Wenger, E. (1998). Communities of practice learning,
tion for Educational Communications and Technology.
meaning, and identity. Cambridge University Press.
Retrieved October 18, 2007, from http://carbon.cudenver.
Wenger, E. (1998). Communities of practice. Learning edu/~bwilson/sitid.html
as a social system. Retrieved October 17, 2007, from
Wilson, B.G. (1996). What is a constructivist learning
http://www.co-i-l.com/coil/knowledge-garden/cop/lss.
environment? In B.G. Wilson (Ed.), Constructivist
shtml
learning environments: Case studies in instructional
Wenger, E. (1998). Communities of practice. Learning, design (pp. 3-8). Educational Technology Publications.
meaning and identity. Cambridge: Cambridge Univer- Winch, C. (1996). Quality in education. Oxford: Black-
sity Press. well.
Wentling, T. L., Waight, C., Gallaher, J., La Fleur, J., Wirsig, S. (2002). ¿Cuál es el lugar de la tecnología en
Wang, C., & Kanfer, A. (2000). E-learning - a review la educación? Retrieved October 26, 2007, from http://
of literature. Knowledge and Learning Systems Group, www.educoas.com/Portal/xbak2/temporario1/latitud/
University of Illinois at Urbana-Champaign. Retrieved Wirsig_Tic_en_Educacion.doc
October 19, 2007, from http://learning.ncsa.uiuc.edu/pa-
Wise, J. C., Lall, D., Shull, P. J., Sathianathan, D., &
pers/elearnlit.pdf .
Lee, S. H. (2006). Using Web-enabled technology in a
Western Interstate Commission for Higher Education. performance-based accreditation enviroment. In S.L.
(2002). Best practice for electronically offered degree Howell & M. Hricko (Eds.), Online assessment and
and certificate programs. Retrieved October 19, 2007, measurement. Cases Studies from higher education,
from http://www.wiche.edu/telecom/Article1.htm K-12 and corporate. (pp. 98-115). London: Information
Science Publishing.
Wiggins, G. (1989). A true test: Toward a more authen-
tic and equitable assessment. Phi Delta Kappan, 70, Woolfolk, A. (2001). Educational psychology. Boston:
703-713. Allyn and Bacon.
William, D., & Black, P. (1996). Meanings and con- World Alliance in Distance Education, (2002). World al-
sequences: A basis for distinguishing formative and liance in distance education. Retrieved October 19, 2007,
summative functions of assessment. British Educational from http://www.wade-universities.org/index.htm
Research Journal, 22, 537-48.
Wright, C. R. (2003). Criteria for evaluating the quality of
Williams, P. E. (2003). Roles and competencies for online courses. Retrieved October 30, 2007, from http://
distance education programs in higher education insti- www.imd.macewan.ca/imd/content.php?contentid=36
tutions. The American Journal of Distance Education, Wu, D., & Hiltz, S. R. (2004). Predicting learning from
17(1), 45-57. asynchronous online discussions. Journal of Asynchro-
nous Learning Networks, 8(2), 139-152.
Williams, S.C., Davis, M.L., Metcalf, D., & Covington,
V.M. (2003). The evolution of a process portfolio as Xenos, M. (2004). Prediction and assessment of student
an assessment system in a teacher education program. behaviour in open and distance education in computers
Current Issues in Education, 6(1). Retrieved October 28, using Bayesian networks. Computers & Education,
2007, from http://cie.ed.asu.edu/volume6/number1/ 43(4), 345-359.
Wilson, B.G. (1995). Situated instructional design: Blur- Yin, R. K. (1984). Case study research: Design and
ring the distinctions between theory and practice, design methods. Thousand Oaks, CA: Sage.

Yu, F. Y., Chang L. J., Liu, Y. H., & Chan, T. W. (2002). Zang, W., & Lin, F. (2003). Investigation of Web-based
Learning preferences towards computerised competi- teaching and learning by boosting algorithms. In IEEE
tive modes. Journal of Computer-Assisted Learning, International Conference on Information Technology:
18(3), 341-350. Research and Education (pp. 445-449).
Zabalza, M.A. (2001). Evaluación de los aprendizajes en Zeithaml, V.A., Parasuraman, A., & Berry, L.L. (1990).
la Universidad. In A. García-Valcárcel (Ed.), Didáctica Delivering quality service: Balancing customer percep-
Universitaria (pp.261-291). Madrid: La Muralla. tions and expectations. New York: Free Press.
Zakrzewske, S., & Steven, C. (2003). Computer-based Ziehe, A., & Müller, K.R. (1998). TDSEP-an efficient
assessment. Quality assurance issues, the hub of the algorithm for blind separation using time structure.
wheel. Assessment & Evaluation in Higher Education, In 8th International Conference on Artificial Neural
28(6), 609-623. Networks (pp. 675-680).
Zakrzewski, S., & Bull, J. (1999). The mass implemen- Zywno, M. S., & Waalen, J. K. (2002). The effect
tation and evaluation of computer-based assessments. of individual learning styles on student outcomes
Assessment & Evaluation in Higher Education, 23(2), in technology-enabled education. Global Journal of
141-152. Engineering Education, 6(1), 35-44.

About the Contributors
Francisco José García-Peñalvo (1971) graduated from University of Valladolid with a degree in
Computer Science, later to obtain a PhD at the University of Salamanca, and is currently professor
Titular de Universidad (a senior lecturer) in the Computer Science Department of the University of
Salamanca. He leads the GRoup on InterAction & eLearning (GRIAL), a research group whose main
lines of work are human-computer interaction, Web engineering, software engineering, educative com-
puting and communications theory. He has published in excess of 100 papers in international publica-
tions and conferences, and has participated in more than 20 research projects. He currently teaches in
the Programa de Doctorado del Departamento de Informática (Computer Science Doctorate Program,
which has been awarded the Quality Mention of ANECA since Academic Course 2003-2004), as well
as in the Máster Oficial en Sistemas Inteligentes (Intelligent Systems Master), in the Máster Oficial en
TIC Aplicadas a la Educación (ICT applied to Education Master), and in the Programa de Doctorado
Procesos de Formación en Espacios Virtuales (Educational Virtual Spaces Doctorate Program) at the
University of Salamanca. Dr. García-Peñalvo is currently Director of the Experto/Máster en eLearning:
Tecnologías y Métodos de Formación en Red (eLearning Master). Lastly, as concerns administrative
tasks, he is currently Vice-Director of Technology Innovation at his University.
***
Giovannina Albano has been assistant professor of Geometry at the University of Salerno (Italy),
since 1997, and for a long time she has been a teacher in several mathematics courses at the Faculty of
Engineering. Her research interests are in mathematics education and in educational models for e-learn-
ing environments. She has contributed to the educational theoretical framework implemented in the
e-learning platform IWT. Her research topics include knowledge domain representation, cooperative
learning, and computer-based mathematics learning. She has also investigated the affective impact of
the use of e-learning platform.
David Camacho has been a lecturer in the Department of Computer Science at Universidad Autonoma
de Madrid since 2005. He holds a PhD in Computer Science from Universidad Carlos III de Madrid
(Spain), obtained in 2001, and a BS in Physics from Universidad Complutense de Madrid (1994). He
has worked in several areas, including multi-agent systems, automated planning, Web intelligence and
e-learning. He has also participated in several projects about automatic machine translation, optimising
industry processes, multi-agent technologies, intelligent systems and virtual education.
Juan Pablo de Castro was awarded his master’s degree in Telecommunications Engineering (1996)
and his PhD (2000). He has been working as a lecturer at the University of Valladolid (Spain) since
November 1996. He was the research director of a technological centre from February 2001 to June
2003, overseeing several projects involving a staff of 40. He has participated in projects in the fields of
telemedicine, e-learning and the Information Society. In addition, he has published papers in interna-
tional journals and in relevant conference proceedings related to these fields. He currently acts as R&D
consultant for various enterprises.
Miguel Ángel Conde was born in 1980 in Salamanca (Spain). He has been studying computer sci-
ence in Salamanca (Spain) since 2006. From 2002 to 2004, he worked in the educational environment
teaching in several courses related to computers. In 2004, he decided to begin working in software
development environment and worked in GPM. In 2005, he began working in the Clay Formación
Internacional R&D department where he was involved in different e-learning projects. He works there
still and also works as a teacher in Salamanca University. Miguel Ángel belongs to GRIAL (Group for
Research on InterAction and eLearning) Research Group where he specializes in e-learning and Web
development.
Olga Díez Fernández holds a PhD in Classical Philology at the Universidad de Sevilla (1995). She
belongs to the Educational Department at the Govern of the Canary Islands - Spain, and has taught
at a secondary school specializing in distance learning since 1992. She was the coordinator of adult
education and distance learning for the government of the Canary Islands, Spain during 2005-2006,
was a tutor online and coordinator of the teaching training for adult and distance learning program of
the Educational Department of the government of the Canary Islands from 2006-2007, and was the
coordinator of Content Development for Upper Secondary Online Courses during 2005-2006, an online
teacher for Upper Secondary Online at the CEAD, and a member of the Grupo de Investigación en
Interacción y e-Learning (GRIAL).
Pier Luigi Ferrari is full professor of mathematics education at the University of East Piedmont
(Italy). He earned his degree in 1975 at the University of Genova. His earlier research fields were math-
ematical logic and categorical algebra. As regards his current research interests, he has investigated
mathematical language and the interactions between semiotic systems and mathematics learning. He
widely uses technology in his mathematics classes and has carried out a number of studies on the use
of ICTs at undergraduate level.
Raised in Germany, Evelyn Gullett’s 23 years of professional experience embrace a variety of

responsibilities ranging from strategic business, to HRM, corporate training and development, and
organizational behaviour in both national and international settings. She worked on projects in various
industries ranging from international hotel and tourism management, airline industry, retail, hospital
administration, and education, as well as the federal government. Evelyn Gullett holds an MBA, an MA
in Organizational Development, and a PhD in Human and Organizational Systems. She has been teach-
ing for 10 years and is currently an assistant professor and faculty manager at U21Global, a consortium
of 20 research-intensive universities worldwide.

Sergio Gutiérrez got his MEng in Telecommunication Engineering at University Carlos III of
Madrid (Spain) in 2002. He got his PhD in Communication Technologies from the same university in
2007, where he works now as a research and teaching assistant. In 2005, he was
selected by the Spanish Agency of Research Quality Evalution (ANECA) as one of the best 100 Ph.D.
students in Spain. His research interests include autoorganizative systems, the application of swarm
intelligence techniques to social systems, elearning.
Ruth Halperin holds a PhD in Information Systems from the London School of Economics and
Political Science, where currently she is a research fellow in the Information Systems and Innovation
Group of the Department of Management. Her research interests include design and implementation
of learning technologies and technology-mediated learning practices. Her current research focuses on
the social aspects of identity in the Information Society. She is a member of the EU funded Network of
Excellence FIDIS working in areas such as interoperability and profiling. Prior to joining the LSE in
2002, she was a project manager of a leading software development company specializing in e-learning
and KM technologies.
Nuria Hernández Nanclares is currently a teacher in the economics faculty, Applied Economics
Department, from University of Oviedo, Spain (Profesora Titular de Economía Aplicada). She has fin-
ished her graduate degree in Economics, with a Special Award in 1992 (Premio Fin de Carrera “Valentín
Andrés Alvarez). She earned her PhD in Economics in University of Oviedo getting the Special Prize
(Premio Extraordinario de Doctorado) in the course 1999. She works in the International Economics
field, both in research and teaching. She is very interested in everything related with high university
teaching, especially in teaching economics. She has done several courses related with effective tea-
ching, alternative assessment techniques, and teaching methodologies. Specially, she has visited the
University of Maastricht (NL) to follow a course about “Problem Based Learning” methodology. She
has presented papers in different national and international meetings about teaching and learning at
university. She has also coordinated several “Innovative Teaching Projects” developed in University
of Oviedo, some of them related to new technologies and its use in teaching, and others associated to
new assessment alternatives. She is also collaborative teacher in the Education Institute (Instituto de
Ciencias de la Educación) in her University.
Izaskun Ibabe Erostarbe, a doctor of psychology and professor at the University of the Basque
Country, has taught the subject “New Technologies and Simulation in Psychology Research” for 8 years.
She designed the Web site of the faculty of psychology and the Department of Social Psychology and
Methodologies of Behaviour Sciences.
Joana Jauregizar Albonigamayor holds a doctoral degree in psychology and baccalaureate degree
in Psychopedagogy. Jauregizar has been lecturer at the University of the Basque Country, and at present
works at the Quality Evaluation and Certification Agency of the Basque University System, which has
recently published a book: Ibabe, I., & Jauregizar, J. (2005). Cómo crear una Web docente de calidad
(How to design a quality teaching Web site). A Coruña: Netbiblo. The book tries to guide university
teachers in the use of didactic resources available on the Internet, so as to facilitate the autonomous
work of students. Izaskun Ibabe coordinated two innovative projects, in which Joana Jauregizar took

part: “An Interactive Self-Assessment Tool Application” and “Continuous Assessment of Teaching and
Learning Process.” They have also many publications about formative assessment.
Célio Gonçalo Marques holds a 5-year degree in Computer Science and Management from Man-
agement School of Santarém (Portugal), a masters’ degree in Multimedia Educational Communication
from Open University (Portugal), a post-graduation course in E-Learning Techniques and Contexts
from Coimbra University (Portugal) and is presently taking a PhD in Education, specialty of Educa-
tional Technology at University of Minho (Portugal). He has been involved in projects related with
education-oriented computing including the Portuguese Program “Internet at School” and worked as a
computer consultant for several enterprises. Presently he is an assistant professor in the Department of
Information and Communication Technologies at the Management School of the Polytechnic Institute
of Tomar. He has authored various publications with emphasis to the book “Os Hipermédia no Ensino
Superior” (“Hypermedia in Higher Education”). His research has been oriented towards the design,
development, and evaluation of interactive learning environments, the application of cognitive flexibility
theory, and e-learning.
Carlos Muñoz Martín was born in Salamanca (Spain). D. Carlos Muñoz Martín studied Computer
Science at the University of Salamanca (Spain) an is an expert in e-commerce at the University of
Salamanca. From June to September of 2005, he worked for M2C developing a traceability program for
agrarian products. In 2005, D. Carlos Muñoz joined the I+D+i team of Clay Formación Internacional
and began work on proyects related with e-learning, e-learning platforms and Web development.
João Noivo holds a 5-year degree in Telecommunications and Electronics Engineering from the
University of Aveiro, Portugal, a specialization in Industrial Engineering from LNETI, Portugal and a
masters’ degree in Operations Research and Statistics from the University of Lisbon, Portugal. He was
an assistant professor in Industrial Electronics Department of University of Minho. In this department,
he participated in management activities and organized its informatics area. He was also responsible
of department that implemented the e-learning in University of Minho. He frequented post-graduate
courses in e-learning and public administration and his current research interests are e-learning and
organizational architecture.
Alvaro Ortigosa is associate professor of Computer Science at Universidad Autonoma de Madrid

and member of the research group GHIA. He received his BA degree in Computer Science from the
Universidad Nacional del Centro de la Provincia de Buenos Aires, Argentina, his MS in Computer Sci-
ence from the Universidade Federal de Rio Grande do Sul, Brazil, and his PhD in Computer Science
from the Universidad Autonoma de Madrid. Dr. Ortigosa has worked in software engineering support
environments, software reuse and, more recently, in adaptive systems, collaborative systems, user
modelling, mobile environments, and authoring and evaluation of adaptive systems.
Abelardo Pardo is an associate professor of Telematic Engineering at theCarlos III University of

Madrid. He received his PhD in Computer Science from theUniversity of Colorado at Boulder. His
research interests are in the area of computer supported learning. He is the principal investigator of
the mosaicLearning2 project on e-learning platforms, tutoring systems, user modeling, and adaptive
hypermedia.

María Ángeles Pérez was awarded her master’s degree in Telecommunications Engineering (1996)
and her PhD (1999). She has been working as a lecturer at the University of Valladolid since October
1996. She has experience in coordinating projects related to telematic applications for the Information
Society, mainly concerning the application of information and communication technologies (ICT) to the
learning process. She also has experience in the evaluation of preproposals, proposals, and final reports
of projects cofunded by the European Commission. She is author or co-author of various publications
and contributions to conferences.
Krassie Petrova is a senior lecturer at the School of Computing and Mathematical Sciences at
Auckland University of Technology, Auckland, New Zealand. She has over 10 years of international
experience of consulting in information systems development and management, and over 15 years of
university teaching and research in information systems, programming, data communications, and
networks. Krassie has published and presented in New Zealand and internationally. Her research areas
are e- and m-learning, IS/IT curriculum development and student skills and capabilities building, and
mobile business (including mobile applications, adoption, and business models). Currently, Krassie is
the programme leader of the Master of Computer and Information Sciences (MCIS), and the co-editor
of the New Zealand Bulletin of Applied Computing and Information Technology (BACIT). See also
http://www.aut.ac.nz/schools/computing_and_mathematical_sciences/our_staff/krassie_petrova/
Estrella Pulido received a degree in Computer Science (1989) from the Facultad de Informática of
the Universidad Politécnica de Madrid. She worked in industry from 1987 until 1991, in R&D depart-
ments. She obtained an MSc in Artificial Intelligence with honours from the University of Bristol (1992)
and was granted with a faculty fellowship from the same University to carry out a PhD (1996). She has
worked at the Escuela Politécnica Superior of the Universidad Autónoma de Madrid since October 1996,
where she has held an associate professor position since July 2000. She has worked on several projects
related to Web-based education and user adaptation.
Luisa M. Regueras was awarded her master’s degree in Telecommunications Engineering in 1998
and her PhD (2003). She has worked as a lecturer at the University of Valladolid in Spain since 1999,
and is actively involved in developing projects related to the application of ICT to the learning process
(e-learning) and telecommunication networks. Her present work involves research into new e-learn-
ing technologies. She is author or co-author of various publications and contributions to national and
international conferences and congresses.
Angélica Rísquez BA (psych), MBS. Angélica has been a researcher at the Centre for Teaching and
Learning at the University of Limerick since 2003. She is committed to the use of innovative teacher
and student support mechanisms at third level. From 2004 to 2006, she was responsible for designing,
coordinating, and evaluating a program of peer e-mentoring to facilitate first year students’ adjustment
to university. She is currently involved on the implementation of the learning management system at
her institution as a technology enhanced learning facilitator. She also supports the use of plagiarism
prevention software, and is involved in initiatives involving using technology for intercultural educa-
tion, research skills, and so forth.
0
María D. R-Moreno earned a PhD (2004) in Computer Sciences by the Universidad de Alcalá
(Spain) with the distinction of the European Doctorate. During her PhD she performed several visits
to different international centers such as British Telecom Adastral Park in UK and the CNR in Rome.
In 2006, she was at NASA Ames Research Center as a Postdoc, working at the Autonomous Systems
and Robotics group. During the summer of 2007, she was a Visiting Researcher at ESA in the ExoMars
mission. Her research interests are in the area of planning, scheduling, monitoring, and execution ap-
plied to real domains as satellites, workflow, and e-learning.
María José Rodríguez-Conde earned her PhD in Education (University of Salamanca, Spain, 1994).
She has been an associate professor of Research Methods in Education at the Faculty of Education of
the University of Salamanca since 1999, an expert in evaluation methodology and statistical analysis of
data in social sciences. She leads the Research Group in Educational Evaluation and Guidance (GE2O),
is a collaborator in the Group of Research in Interaction and E-learning (GRIAL) of the University of
Salamanca, and is the director of several projects on evaluation in education. GRIAL’S last published
work dealt with the evaluation processes in e-learning and it has directed several doctoral dissertations
centered in the processes of evaluation of programmes in education.
Addisson Salazar is working towards a doctorate degree in Telecommunications at Universidad

Politécnica de Valencia (UPV). He has received the BSc and MSc in Informatics from the Universidad
Industrial de Santander and a D.E.A. in Telecommunications from UPV in 2003. He is a researcher
of the Signal Processing Group of the Institute of Telecommunication and Multimedia Applications
at UPV. His research interest is focused on statistical signal processing, pattern recognition, and data
mining and knowledge discovery, where he has worked in different theoretical an applied problems,
many of them under contract with the industry. His theoretical aspects of interest are signal classifica-
tion, time-frequency analysis, independent component analysis, and algorithms for data mining. He
has participated in different Programmes of the European Community. He has published more than 70
papers including journals and conference contributions.
Antonio Seoane (1971) holds a 5-year degree in Philosophy from the University of Salamanca
(Spain) and has worked as high-school professor at the Department of Philosophy since 1998; he also
teaches for E-Learning masters degrees at both the University of Alcalá de Henares and the University
of Salamanca (Spain). Currently, he is finishing two doctoral theses on ancient greek rhetoric and online
training methodology, respectively. His researching interest fields are the ancient rhetoric, the modern
communication theory, e-learning, and online training methodology. He is an active member of the Re-
searching GRoup on InterAction & eLearning (GRIAL) at the University of Salamanca, where he acts as
Academic Coordinator of Training Activities, standing out the E-Learning Master and the Online Tutor
Lifelong Learning Diploma, both from the University of Salamanca. He is author of several articles and
chapters regarding his researching fields and published in recognized reviews and volumes.
Rowena Sinclair is a senior lecturer in the Accounting Department of the School of Business at
Auckland University of Technology, Auckland, New Zealand. As a chartered accountant, Rowena has
over 10 years of extensive accounting and auditing experience in both the private and public sector
working with both small enterprises through to large multinational corporations. Her teaching areas
currently focus on auditing and contextualizing accounting within the workplace. Her research is cur-

rently focusing on the transparency of the financial reports of charities for which she is completing a
PhD. Previously, Rowena was involved in teaching in the banking field reflecting her years of service in
the banking industry for which she was recognized by being given the award of senior associate of the
Financial Services Institute of Australasia. Further recognition of this expertise was reflected by being
asked to appear on television where she was interviewed on various banking issues.
Sergio Vásquez Bronfman is professor of Information Systems at ESCP-EAP (European School of

Management), in Paris. Since 1983, he has been involved in research and practice of ICT-based learning.
In the 1990s, he worked mainly in university settings in France developing Internet-based educational
innovations, while, since 2000, he has been doing e-learning projects in Spain, in corporate settings.
His work focus is on (a) on the design of learning systems aimed at bridging the gap between “know-
ing” and “doing,” hence increasing the return on investment of professional and corporate education,
and (b) on the political questions and power games that exist in e-learning and knowledge management
projects and which are key in order to ensure implementation success.
Alberto Velasco Florines was born in Salamanca (Spain) in 1981. He studied computer science
engineering in the University of Salamanca (Spain) from 1999 to 2007. He worked as a system admin-
istrator and Web developer in 2006. He has been working as a developer for the R&D department of
Clay Formación Internacional since November 2006. He has been developing a mobility system for an
own e-learning platform and has collaborated in the development of an application for statistic control
in Moodle.
Elena Verdú, telecommunications engineer, has been project manager at CEDETEL (Centre for the
Development of Telecommunications of “Castilla y León”) since December 2000, coordinating research
projects in the fields of new telematic applications for the Information Society, communication networks
and software engineering, at different scopes (regional, national, and international). She has published
papers in international journals and participated in international conferences and congresses. She is also
an associate lecturer at the “Escuela Técnica Superior de Ingenieros de Telecomunicación” (the School
of Telecommunications Engineering), at the University of Valladolid, Spain.
María Jesús Verdú received both an MS and PhD in Telecommunications Engineering from the
University of Valladolid, Spain, in 1996 and 1999, respectively. She has been working as a lecturer at the
University of Valladolid since November 1996. She has experience in coordinating projects in the fields
of new telematic applications for the Information Society and telecommunications networks, especially
related to e-learning. She has published papers in international journals and in relevant conference
proceedings related to these fields.
Luis Vergara was born in Madrid (Spain) in 1956. He received the Ingeniero de Telecomunicación
and the Doctor Ingeniero de Telecomunicación degrees from the Universidad Politécnica de Madrid
(UPM), in 1980 and 1983, respectively. Until 1992 he worked at the Departamento de Señales, Siste-
mas y Radiocomunicaciones (UPM) as an associate professor. In 1992, he joined the Departamento de
Comunicaciones (Universidad Politécnica de Valencia (UPV) as Department Head until April 2004.
From April 2004 to April 2005 he was vice-director of New Technologies at the UPV. Currently he
is responsible of the Signal Processing Group of the Institute of Telecommunication and Multimedia

Applications at UPV. His research concentrates in the statistical signal processing area, where he has
worked in different theoretical an applied problems, many of them under contract with the industry.
He has participated in different international actions, particularly in NATO projects and in different
Programmes of the European Community. He has published more than 150 papers including journals
and conference contributions.
Ray Webster is currently associate professor (adjunct) in the School of Information Technology,
Murdoch University, Australia. He has a PhD in Educational Research from Lancaster University, an
MSc in Cognitive Science (Brunel) and a BA (Hons) in Economics and Geography (Middlesex) as well
as graduate qualifications in information systems and education. In a long career combining research,
teaching and consultancy, Ray has worked in countries such as Australia, the UK, Turkey, and Malaysia.
His current research interests include the use of cognitive and learning profiles in virtual environment
design and to enhance student autonomy and self actualization in the learning process.

Index
A course-level report generation 259
a-didactical situations 135 D

active learning, ICT-based 235
data mining scheme 180
active learning approach 232–249
data preprocessing 177
activity nodes 202
diplomas and certificate generation 259
analytic hierarchy process (AHP) 117
distance mode 238
anonymity mode 238
domain theory 153
ant colony optimization (ACO) 200
artificial intelligence techniques 149–172 E
asynchronous Javascript and XML (AJAX),
new look 216 e-assessment, information gathering techniques
305
B e-facilitators, quality assessment 317–327
e-facilitators quality assessment, issues 320
blended learning 89, 134
e-learning, accessibility 119
blended learning course on communication 40
e-learning, components 120
C e-learning, designing an online assessment
300–316
clustering analysis 186 e-learning, failure w/o a method 47
CoFIND 207 e-learning, formative online assessment 279–
cognitive profiles 3 299
collaborative filtering 206 e-learning, Heideggerian view 30–45
collaborative sequencing 200 e-learning, online interaction 123
competitive learning 236 e-learning, quality evaluation 331
computer-assisted assessment (CAA) 280, 311 e-learning, quality in 330
computer-based assessment system (CBA) 310 e-learning, student satisfaction 119
computer-provided feedback 282 e-learning, training teachers 83–95
computer aided instruction (CAI) 86 E-Learning 2.0 213–231
computer attitude scale (CAS) 99 E-Learning 2.0, best practices 223
computer supported collaborative learning E-Learning 2.0, reaching classrooms 219
(CSCL) 234 E-Learning 2.0, tools and challenges 220
constraint programming (CP) 155 e-learning practices, contextual factors 102
constructivism, as a goal 50 e-learning practices, institutional factors
contingent plan 154 96–111
cooperative learning 136 e-learning research, institutional context 101
Index
e-learning tools, automated processes 152 I

e-learning value, stakeholder perspectives 114
e-learning value, student experiences 112–131 ICT-based active learning 235
e-mentoring 61–82 ICT limits, different learning 38
e-mentoring, a European perspective 63 ICT use 240
e-mentoring, best practice 69 independent component analysis (ICA)
e-mentoring, differences from face-to-face 174, 180
67–76 individual learning 134
e-mentoring, issues 64 InnoeLearning 336
e-portfolio, strategic instrument of assessment institutional conventions 103
271 institutional factors, formation of e-learning
e-portfolio, strategic use 264–278 practices 96–111
e-portfolio of "international economic rela- institutional standards 103
tions" 268 instructional style 98
e-Qual 328–348 IPSS final solution 165
e-quality assessment matrix (e-QAM) 321 IPSS integration 159
e-quality learning standards 334 K
e-Qual model 336
education of man as a citizen 55 knowledge discovery, from e-learning activities
effective learning 264–278 173–198
knowledge discovery in databases (KDD) 175
F knowledge presenting 104
face-to-face mode 238 L
flexible lifelong learning 85
flexible student alignment (FSA) 10 Learning at University unit 20
formative assessment concept 280 learning complexity 7
formative online assessment, design of activi- learning design 90
ties 285 learning design (LD) 150
formative online assessment, strategies 288 learning design (LD) information model 151
formative vs. summative assessment 280 learning domains, representation formalisms
151
G learning environment, definition 3
General Agreement on Trade in Services learning management system (LMS) x–
(GATS) 116 xii, 250–263
graphical user interface (GUI) 259 learning management systems (LMS)
Graphplan planner 154 86, 96, 305
Greek Paideia 53–56 learning networks 202
Growltiger 40 learning opportunities 138
learning outcomes 90
H learning strategy, choosing 239
learning styles 240
Heideggerian view on e-learning 30–45 learning styles, classifications 234
Heidegger philosophy 31 learning technology (LT) 98
heuristic search planner (HSP) 154 learning technology use, influencing factors 97
Homeric hero 54 linear vector quantization (LVQ) 185
HTN planner 154

Index
M personalized e-learning environments (PELE),

a systems perspective 8
Markov decision process (MDP) 154 personal teaching 134
mathematical language, pragmatic view 138 phenomenology of learning a skill 33
mathematics education, integrating technology pheromone 200
132–148 planning techniques 153
mathematics education, research 133 platform-level report generation 257
mentoring, multifaceted/elusive concept 62 portfolio assessment 290
MentorNet 65 principal component analysis (PCA) 182
metrodology assessment, aims 302 professional, continuous, and corporate educa-
metrodology assessment, criteria and indicators tion (PCCE) 30
303
metrodology assessment, in e-learning 302 Q
mining decision rules 188
Moodle, development model 254 QUEST, a mixed competitive-collaborative
Moodle, error in the grade system 256 solution 242
Moodle, learning management system platform Quest Environment for Self-managed Training
252 (QUEST) 234
motivation, learning strategy 239 R
N reflective and participatory approach to design
neural networks 185 (RAPAD) 1–29
reflective and participatory approach to design
O (RAPAD), a systems perspective 8
reflective and participatory approach to design
Odyssey 54 (RAPAD), definition 2
online adaptive assessment 288 reflective and participatory approach to design
online assessment 283 (RAPAD), development 15
online assessment, effectiveness 284 reflective and participatory approach to design
online assessment, pros and cons 283 (RAPAD), student engagement 14
online assessment tool, creating 285 regression analysis 185
online collaborative assessment 289 representational state transfer (REST) 216
online training methodology, building 46–60
open and distance learning (Meca-ODL), guide S
335
open source LMS customization 250–263 SAT-based planner 154
scheduling techniques 154
P Scottish centre for research into online learning
and assessment (SCROLLA) 310
paraschool system 201 self-assessment and learning, case study 291
participatory design 12 semiosis 141
participatory methodology 19 semiotic representation systems 137
personalized e-learning environment (PELE) 4 shareable courseware object reference model
personalized e-learning environment (PELE), (SCORM) 152
the need for 6 SIT 204
personalized e-learning environments (PELE) situation learning (SL) 152
1–29 social learning 51

Socrates 55
Sophists 55
student-focused model, the excuse 51
student as learning system (SLS) 8, 10
student learning process, monitoring 149–172
sustainable environment for the evaluation
of quality in e-learning (SEEQUEL)
329, 334
swarm-based techniques, in e-learning 199–
212
T
TANGOW 156
TANGOW, authoring a course 157
TANGOW courses, case study 159
TANGOW courses, monitoring 163
TANGOW logs 158
teacher training programs, developing 87
teaching opportunities 138
telematic environments 232–249
total order (TO) planner 154
total quality control (TQC) 318
total quality management (TQM) 318
total quality management (TQM) in higher
education 319
training teachers, a case study 88
training teachers in evolving educational con-
texts 87
tri-dimensional relationship 135
V
virtual learning environments (VLE) 150
W
Web 2.0 214
Web 2.0, key technologies 215
Web mining 175
Web revolution 214

Advances in E-learning-Experiences and Methodologies

Diunggah oleh

Informasi Dokumen

Hak Cipta

Format Tersedia

Bagikan dokumen Ini

Bagikan atau Tanam Dokumen

Opsi Berbagi

Apakah menurut Anda dokumen ini bermanfaat?

Apakah konten ini tidak pantas?

Hak Cipta:

Format Tersedia

Advances in E-learning-Experiences and Methodologies

Diunggah oleh

Hak Cipta:

Format Tersedia

Advances in E-Learning:

Experiences and Methodologies

Francisco J. García Peñalvo

InformatIon scIence reference

Published in the United States of America by

and in the United Kingdom by

Library of Congress Cataloging-in-Publication Data

Advances in e-learning : experiences and methodologies / Francisco José García-Peñalvo, editor.

Includes bibliographical references and index.

ISBN 978-1-59904-756-0 (hardcover) -- ISBN 978-1-59904-758-4 (ebook)

1. Internet in education. 2. Continuing education--Computer-assisted instruction. I. García-Peñalvo, Francisco José.

British Cataloguing in Publication Data

If a library purchased a print copy of this publication, please go to http://www.igi-global.com/reference/assets/IGR-eAccess-agreement.

Preface . ............................................................................................................................................... xiv

Acknowledgment . ............................................................................................................................. xxii

Compilation of References ............................................................................................................... 350

About the Contributors .................................................................................................................... 386

Preface . ............................................................................................................................................... xiv

Acknowledgment . ............................................................................................................................. xxii

This chapter provides an overview of technology-based competitive active learning. It discusses

Compilation of References ............................................................................................................... 350

About the Contributors .................................................................................................................... 386

A FrAmework to AvoId e-LeArnIng PItFALLs

there is no real Intent in Institutions (“use the Platform or suffer”)

users are Alone

Matching these ideas, we can define quality in e-learning as:

orgAnIzAtIon oF the Book

Francisco José García Peñalvo

IntroductIon ing, discuss it, and apply their thoughts to the

Figure 1. Overview: reflective and participatory approach to design (Webster, 2005)

Figure 2. Student as reflective and participatory system designer for PELE

technology or human-centred is intended to demonstrate an acute case of the

Cognitive Profile: PELE:

then asked to reflect on and write about their emergent Issues

Figure 5. Conceptual model of “Learning at University” as a learning system

Table 1. RAPAD implemented as the “Learning at University” unit

AddItIonAL reAdIngs Sternberg, R.J., & Zhang, L.F. (Eds.). (2001).

IntroductIon one can observe; actual PCCE systems produce

(negotiators, communicators, historians, doctors, An example of the above is a series of speeches

should move to a different kind of situations and • What am I trying to do?

ing to think on them. Often, we are not aware of expert

What happens Schön’s interpretation Heidegger’s interpretation

Future reseArch dIrectIons social skills. Massive Multiplayer Online Games

Wierinck, E. et al. (2005). Effect of augmented when technology is “transparent,” one is

Antonio Miguel Seoane Pardo

Francisco José García Peñalvo

IntroductIon sources are devoted to perfecting these elements

underwent a significant variation. In this age interaction of citizens in a political context in

concLusIon make sense if put into practice and therefore are

IntroductIon of learning possibilities (Gisbert, 2004). During

However, what is understood for “mentoring” There can be components of mentoring in

Table 1. What is mentoring?

Mentoring is… Mentoring is not…

Figure 1. Homepage of MentorNet (www.mentornet.net)

Figure 2. MentorNet e-Forum discussion groups

Figure 3. Mentor profile (to be filled before participating in one-to-one mentoring)

Table 2. Contact. Differences between face-to-face and e-mentoring (a literature review)

Table 3. Timing. Differences between face-to-face and e-mentoring (a literature review)

• Immediate. Pressure on immediately responding • Asynchronous tools (discussion forums, e-mail):

Table 14. Contextualization. Best practice on e-mentoring DESIGN (a literature review)

• Consideration of the information needs of the program’s board, founders, communi-

Table 29.“Don’ts” on e-mentoring (a literature review)

On planning and running an e-mentoring program, don’t…

IntroductIon The point of view of the teacher staff may lead

‘Operative System role = ‘End user’ ‘Services’ , AND

task1: task., task.,… task2: task., task.,… task3 taskn-1 taskn