Available online at www.sciencedirect.com Tersedia online di www.sciencedirect.

com

 
 
 
 
Computers & Education 51 (2008) 559–583 Komputer & Pendidikan 51 (2008) 559-583
 
www.elsevier.com/locate/compedu www.elsevier.com/locate/compedu
 
 
 
Alternatives in evaluating multimedia in secondary school science
teaching Alternatif dalam mengevaluasi multimedia dalam
pengajaran sains sekolah menengah
 
Johannes C Cronje´ a, * , Johann Fouche b Johannes C Cronje ' a, * , Johann Fouche b
 
a Sebuah
Faculty of Informatics and Design, Cape Peninsula University of Technology, South Africa
b
Faculty of Education, University of Pretoria, South Africa Fakultas
Informatika dan Desain, Universitas Teknologi Cape Peninsula, Afrika Selatan b.
Fakultas Pendidikan, Universitas Pretoria, Afrika Selatan
 
Received 25 November 2006; Diterima 25 November 2006; received in revised form 20 June 2007; diterima dalam bentuk revisi 20 Juni 2007;
accepted 20 June 2007 diterima 20 Juni 2007
 

 
 
 
 
Abstract Abstrak
 
While di erent approaches to evaluation will yield di erent results, depending on the purpose of the evaluation this article
describes an evaluation approach that was aimed at investigation the mental models of users of the programme. Sementara
pendekatan yang berbeda untuk evaluasi akan menghasilkan hasil yang berbeda, tergantung pada tujuan evaluasi artikel ini
menjelaskan pendekatan evaluasi yang ditujukan untuk penyelidikan model mental pengguna program. The study was driven by
questions about the di erences in mental models of the instructional designers and the learners, the time learners spent working
through the program and the observable changes in their mental models. Penelitian ini didorong oleh pertanyaan tentang
perbedaan dalam model mental dari perancang instruksional dan peserta didik, waktu yang dihabiskan para pembelajar bekerja
melalui program dan perubahan yang dapat diamati dalam model mental mereka. In this design experiment, a program was
developed to teach basic principles of electricity. Dalam eksperimen desain ini, sebuah program dikembangkan untuk
mengajarkan prinsip dasar listrik. Three boys and three girls one each of high, medium and low achievement in science were
selected from an advantaged urban school, and a similar sample was taken from a disadvantaged rural school. Tiga anak laki-laki
dan tiga perempuan masing-masing satu dari prestasi tinggi, sedang dan rendah dalam sains dipilih dari sekolah perkotaan yang
diuntungkan, dan sampel serupa diambil dari sekolah pedesaan yang kurang beruntung. They were asked to draw their
impressions of various concepts of electricity and then allowed free access to the program, where they could visit any section
even if they had not completed a previous one. Mereka diminta untuk menggambar kesan mereka tentang berbagai konsep
kelistrikan dan kemudian mengizinkan akses gratis ke program, di mana mereka dapat mengunjungi bagian manapun bahkan jika
mereka belum menyelesaikan bagian sebelumnya. After-wards they were asked to draw sketches again. Setelah itu mereka
diminta untuk menggambar sketsa lagi. Other instruments included an opinion questionnaire and observation of the learners
working with a ''think aloud'' protocol. Instrumen lain termasuk kuesioner opini dan pengamatan peserta didik yang bekerja
dengan protokol 'berpikir keras'. It was found that there were considerable di erences in the mental models of the learners and
designers about what to expect from computer-based learning. Ditemukan bahwa ada perbedaan yang cukup besar dalam model
mental para pembelajar dan desainer tentang apa yang diharapkan dari pembelajaran berbasis komputer. While navigational
freedom allowed fast learners to move through work that they knew already, weaker learners tended to get lost. Sementara
kebebasan navigasi memungkinkan pembelajar cepat untuk bergerak melalui pekerjaan yang sudah mereka ketahui, pembelajar
yang lebih lemah cenderung tersesat. The sketches that learners made before and after exposure to the program provided valuable
insights into the growth of their understanding of the concepts. Sketsa-sketsa yang dibuat oleh para siswa sebelum dan sesudah
terpapar pada program ini memberikan wawasan berharga ke dalam pertumbuhan pemahaman mereka tentang konsep-konsep
tersebut.
 
2007 Elsevier Ltd. All rights reserved. 2007 Elsevier Ltd. Semua hak dilindungi undang-undang.
 
Keywords: Mental models; Kata Kunci: Model mental; Computer-mediated learning; Pembelajaran yang dimediasi komputer; Evaluation;
Evaluasi; Media in education; Media dalam pendidikan; Navigation Navigasi
 

 
 
 
1. Introduction 1. Perkenalan
 
This article reflects upon the formative evaluation of a multimedia science program teaching basic princi-ples of
electricity. Artikel ini mencerminkan evaluasi formatif dari program ilmu multimedia yang mengajarkan prinsip
dasar listrik. We focus on the extent to which the learners' view of the program corresponded with that of the
designers, we consider the time spent, and the navigation through the program; Kami fokus pada sejauh mana
pandangan peserta didik dari program sesuai dengan yang dari para desainer, kami mempertimbangkan waktu yang
dihabiskan, dan navigasi melalui program; and finally learners' acqui-sition of certain learning concepts. dan
akhirnya akuisisi siswa dari konsep pembelajaran tertentu. This was a small, qualitative study with a purposive
sample of strong, aver- Ini adalah penelitian kualitatif kecil dengan sampel purposive yang kuat, rata-rata.
 
 
 
* *
Corresponding author. Penulis yang sesuai. Tel.: +27 82 558 5311; Tel .: +27 82 558 5311; fax: +27 21 460 3553. faks: +27 21 460 3553.
 

E-mail address: johannes.cronje@gmail.com (JC Cronje´). Alamat e-mail: johannes.cronje@gmail.com (JC Cronje´).
 
0360-1315/$ - see front matter 0360-1315 / $ - lihat materi depan               2007 Elsevier Ltd. All rights reserved. 2007 Elsevier Ltd. Semua
hak dilindungi undang-undang.
 
doi:10.1016/j.compedu.2007.06.012 doi: 10.1016 / j.compedu.2007.06.012
560 560               JC Cronje´, J. Fouche / Computers & Education 51 (2008) 559–583 JC Cronje´, J. Fouche / Komputer & Pendidikan 51
(2008) 559-583
 
age and weak learners from a uent and disadvantaged schools. usia dan peserta didik yang lemah dari sekolah yang
tidak mampu dan kurang beruntung. We do not aim at transferable results, but present our findings here in the hope
that they may resonate with the experiences of others. Kami tidak bertujuan pada hasil yang dapat ditransfer, tetapi
menyajikan temuan kami di sini dengan harapan bahwa mereka dapat beresonansi dengan pengalaman orang lain. In
a sense our research augments an earlier article in this journal that concentrates on the support that teachers can give
learners in a multimedia-rich environment ( Hennessy et al., 2007 ). Dalam arti penelitian kami menambah artikel
sebelumnya dalam jurnal ini yang berkonsentrasi pada dukungan yang dapat diberikan guru kepada peserta didik
dalam lingkungan multimedia yang kaya ( Hennessy et al., 2007 ).
 
The principal motivation for this study comes from Greca and Moreira (2000, p. 8) who call for better
understanding of mental models both for teachers and researchers, because ''We, so far, know neither how to identify
what kinds of mental models the learners have in a given domain, nor what specific mental models they construct.
Motivasi utama untuk penelitian ini berasal dari Greca dan Moreira (2000, hal. 8) yang menyerukan pemahaman
yang lebih baik tentang model mental baik untuk guru dan peneliti, karena '' Kami, sejauh ini, tidak tahu cara
mengidentifikasi jenis model mental apa pelajar memiliki domain tertentu, atau model mental khusus apa yang
mereka bangun. For researchers, the main di culty here seems to be a methodological one: how to seize these
incomplete, unstable, and personal representations?'' Bagi para peneliti, kekecewaan utama di sini tampaknya
bersifat metodologis: bagaimana cara menangkap representasi yang tidak lengkap, tidak stabil, dan personal ini? ''
 
In this article, we describe a design experiment in which we developed a multimedia program and investi-gated
how users responded to it. Dalam artikel ini, kami mendeskripsikan eksperimen desain di mana kami
mengembangkan program multimedia dan menyelidiki bagaimana pengguna menanggapinya. We present our
findings as they answer our three research questions, and o er a number of conclusions and recommendations. Kami
menyajikan temuan kami saat mereka menjawab tiga pertanyaan penelitian kami, dan memberikan sejumlah
kesimpulan dan rekomendasi.
 
2. The problem 2. Masalahnya
 
The problem underlying this research was to find a simple, multi-faceted way to understand how learners make
sense of multimedia learning programs, because ''(W)hen interactive multimedia programs are designed
intentionally to support learning, some level of pedagogy is required'' ( Reeves, 1993, p. 81 ). Masalah yang
mendasari penelitian ini adalah untuk menemukan cara yang sederhana, multi-faceted untuk memahami bagaimana
peserta didik memahami program pembelajaran multimedia, karena '' (W) hen program multimedia interaktif
dirancang sengaja untuk mendukung pembelajaran, beberapa tingkat pedagogi diperlukan ' '( Reeves, 1993, hlm.
81 ). We were specif-ically interested in an evaluation from a learner-perspective, since evaluation of educational
software is strongly dependent on the perspective of the evaluator. Kami secara khusus tertarik pada evaluasi dari
sudut pandang pembelajar, karena evaluasi perangkat lunak pendidikan sangat bergantung pada perspektif evaluator.
A teacher would look for di erent qualities than would a leaner, or a school administrator. Seorang guru akan
mencari kualitas yang berbeda daripada yang lebih ramping, atau administrator sekolah. Even ratings from software
review services are not valid indicators of the educational value of software. Bahkan peringkat dari layanan ulasan
perangkat lunak tidak merupakan indikator yang valid untuk nilai pendidikan perangkat lunak. Since software is
judged subjectively, as is the case with many software evalu-ation services, one cannot assume learners will or will
not learn from it ( Borton & Rossett, 1989; Burger, 1991; Karena perangkat lunak dinilai secara subyektif, seperti
halnya dengan banyak layanan evaluasi perangkat lunak, seseorang tidak dapat menganggap pelajar akan atau tidak
akan belajar darinya ( Borton & Rossett, 1989; Burger, 1991; Jolicoeur & Berger, 1988; Jolicoeur & Berger, 1988;
Owston & Dudley-Marling, 1988; Owston & Dudley-Marling, 1988; Preece & Jones, 1985; Preece & Jones, 1985;
Zahner, Reiser, Dick, & Gill, 1992 ). Zahner, Reiser, Dick, & Gill, 1992 ).
 
There is still a strong emphasis on large-scale comparative research studies investigating the e ectiveness of
computer based approaches to other approaches to teaching ( Janniro, 1993; Lanza & Roselli, 1991 ). Masih ada
penekanan kuat pada studi penelitian komparatif skala besar yang menyelidiki keefektifan pendekatan berbasis
komputer terhadap pendekatan lain untuk mengajar ( Janniro, 1993; Lanza & Roselli, 1991 ). Many of these findings
have revealed no significant di erences ( Clark, 1991, 1994; Russell, 1999 ). Banyak dari temuan ini telah
mengungkapkan tidak ada perbedaan yang signifikan ( Clark, 1991, 1994; Russell, 1999 ). Reeves (1992) describes
numerous theoretical and methodological flaws in existing media replication studies, and suggests implementing a
multi-faceted approach to research that includes the conduct of intensive case studies and the application of mental
modelling. Reeves (1992) menjelaskan banyak kelemahan teoritis dan metodologis dalam studi replikasi media yang
ada, dan menyarankan menerapkan pendekatan multi-aspek untuk penelitian yang mencakup pelaksanaan studi
kasus intensif dan penerapan pemodelan mental. Investigations of interactive multimedia should include both
observa-tional and regression methods. Investigasi multimedia interaktif harus mencakup metode observasi dan
regresi. Our approach, therefore, was to use a variety of simple methods, using a small sample. Oleh karena itu,
pendekatan kami adalah menggunakan berbagai metode sederhana, menggunakan sampel kecil. We wanted to see
what our learners were conceptualising, rather than to see if the program was successful. Kami ingin melihat apa
yang dipelajari oleh para pembelajar kami, daripada melihat apakah program itu berhasil.
 
Mental model research is based on the assumption that knowledge of how users represent systems and how users
should represent systems will lead to a better understanding of usable systems ( Ackermann & Tauber, Penelitian
model mental didasarkan pada asumsi bahwa pengetahuan tentang bagaimana pengguna mewakili sistem dan
bagaimana pengguna harus mewakili sistem akan mengarah pada pemahaman yang lebih baik dari sistem yang
dapat digunakan ( Ackermann & Tauber, 1990 ). 1990 ). Jih and Reeves (1992) state that: ''Since our understanding
of human perception does play a crucial role Jih dan Reeves (1992) menyatakan bahwa: '' Karena pemahaman kita
tentang persepsi manusia memang memainkan peran penting in the design of interfaces, research on mental models
is a promising approach to analysing human–computer interaction and improving interface designs'' ( Jih & Reeves,
1992, p. 44 ). dalam desain antarmuka, penelitian pada model mental adalah pendekatan yang menjanjikan untuk
menganalisis interaksi manusia-komputer dan meningkatkan desain antarmuka '' ( Jih & Reeves, 1992, hlm. 44 ).
Johnson, O'Connor, Pirnay-Drum-mer, Ifenthaler, Spector and Seel (2006, p. 177) argue that ''researchers can only
learn of conceptual systems if individuals communicate their systems''. Johnson, O'Connor, Pirnay-Drum-mer,
Ifenthaler, Spector, dan Seel (2006, p. 177) berpendapat bahwa '' peneliti hanya dapat mempelajari sistem
konseptual jika individu mengomunikasikan sistem mereka ''. They suggest mental model research as a useful way
for ''the com-parison of a group or an individual with themselves over time''. Mereka menyarankan penelitian model
mental sebagai cara yang berguna untuk '' perbandingan kelompok atau individu dengan diri mereka sendiri dari
waktu ke waktu ''. Coll and Treagust (2001) found that learners prefer simple mental models, while Duit and
Treagust (2003) argue that learners' understanding of concepts can a ect the teaching of science. Coll dan Treagust
(2001) menemukan bahwa peserta didik lebih menyukai model mental sederhana, sementara Duit dan Treagust
(2003) berpendapat bahwa pemahaman konsep peserta didik dapat mempengaruhi pengajaran sains.
 
568 568               JC Cronje´, J. Fouche / Computers & Education 51 (2008) 559–583 JC Cronje´, J. Fouche / Komputer & Pendidikan 51
(2008) 559-583

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Fig. 7.
Gambar
  7. Verify. Memeriksa.  

Table 1 Tabel 1        
Target group identification matrix Matriks
identifikasi kelompok sasaran
 
 
 
 
 
   
 

 
 
 
 
Academic ability Kemampuan akademik
   
 
 

Low High
 
 
 
 
Rendah
   
Medium Medium  
Tinggi
Female, disadvantaged school Sekolah   fdl fdl fdm fdm fdh
perempuan, yang dirugikan fdh
Male, disadvantaged school Sekolah laki-laki mdl mdh
yang dirugikan   mdl mdm mdm mdh
Female, technology-enabled school Sekolah
yang mendukung teknologi wanita   ftl ftl ftm ftm fth fth
Male, technology-enabled school Laki-laki, mth
sekolah yang mendukung teknologi
 
 
 
mtl mtl
   
mtm mtm  
mth
 
Code. Kode. mtl = male, technology-enabled, low ability; mtl = laki-laki, teknologi-diaktifkan, kemampuan rendah; fdh = female,
disadvantaged, high ability, etc. fdh = perempuan, kurang beruntung, berkemampuan tinggi, dll.

 
Table 2 Meja 2
 
Computer experience of the test group Pengalaman komputer dari kelompok uji
 

 
Academic ability
 
 
 
 
Kemampuan akademik  
   
 
 

 
 
 
 
Low Rendah  
Medium Medium  
High Tinggi
Female, resource-disadvantaged school
Perempuan, sekolah yang kekurangan sumber 5 sessions CAI 5 sesi 5 sessions CAI 5
daya CAI 5 sessions CAI 5 sesi CAI sesi CAI
Male, resource-disadvantaged school Laki-laki, 5 sessions CAI 5 sesi 5 sessions CAI 5
sekolah yang kekurangan sumber daya CAI 5 sessions CAI 5 sesi CAI sesi CAI
Female, resource-advantaged school Sekolah Computer Studies
Perempuan, sumber daya yang diuntungkan Games Pertandingan Games Pertandingan Studi komputer
Male, resource-advantaged school Laki-laki, Computer Studies Studi Computer Studies
sekolah yang diuntungkan sumber daya
   
Games Pertandingan
   
komputer  
Studi komputer
 
5.4. 5.4. Description of research instruments Deskripsi instrumen penelitian
 
A variety of methods were used to obtain insight into learners' mental models, and their reaction to the program.
Berbagai metode digunakan untuk mendapatkan wawasan model mental peserta didik, dan reaksi mereka terhadap
program.
 
 
5.4.1. 5.4.1. Observation of users using the system Pengamatan pengguna menggunakan sistem
 
Observation was selected as one of the methods for data acquisition because it gave us the opportunity to obtain
firsthand information concerning the problems experienced with the execution and learning from the program.
Observasi dipilih sebagai salah satu metode untuk akuisisi data karena memberi kami kesempatan untuk
memperoleh informasi langsung mengenai masalah yang dialami dengan pelaksanaan dan pembelajaran dari
program.
 
Observation allows the researcher to collect a variety of information that provides depth to the analysis.
Observasi memungkinkan peneliti untuk mengumpulkan berbagai informasi yang memberikan kedalaman pada
analisis. The observer can view a situation firsthand as it develops. Pengamat dapat melihat situasi secara langsung
saat berkembang. Recording of the observation takes place immedi-ately, thus reducing the possibility of biased
recall. Rekaman pengamatan berlangsung segera, sehingga mengurangi kemungkinan ingatan bias. According to
Mouton and Marais (1990) , observation Menurut Mouton dan Marais (1990) , observasi
 JC Cronje´, J. Fouche / Computers & Education 51 (2008) 559–583 JC Cronje´, J.
Fouche / Komputer & Pendidikan 51 (2008) 559-583 569 569
 
should be considered when investigating the activities or behaviour of people; harus dipertimbangkan ketika
menyelidiki kegiatan atau perilaku orang-orang; and when needing to corrobo-rate opinions about a particular
intervention. dan ketika membutuhkan pendapat korosif tentang intervensi tertentu.
 
Observed events could be subject to the subjectivity and bias of the observer. Peristiwa yang diamati dapat
dikenakan subjektivitas dan bias pengamat. The observations were recorded on video because events happened so
quickly that it became impossible to record every detail. Pengamatan direkam pada video karena peristiwa terjadi
begitu cepat sehingga tidak mungkin untuk merekam setiap detail. Users of the system were observed while a video
camera captured their reactions. Pengguna sistem diamati ketika kamera video menangkap reaksi mereka.
Photographs were taken of users working on the system. Foto-foto diambil dari pengguna yang bekerja pada sistem.
Users were asked to explain the system using a teach-back ''show me how to'' proce-dure. Pengguna diminta untuk
menjelaskan sistem menggunakan mengajar-kembali '' tunjukkan saya bagaimana '' prosedur-dure. They were also
asked to provide reasons for specific program responses. Mereka juga diminta untuk memberikan alasan untuk
tanggapan program tertentu. The observations and responses from the users were duly recorded. Pengamatan dan
tanggapan dari pengguna dicatat secara benar.
 
5.4.2. 5.4.2. Sketches Sketsa
 
Users were asked to sketch the concept ''electricity'' and the concept ''atom'' before they started to work with the
program. Pengguna diminta untuk membuat sketsa konsep '' listrik '' dan konsep '' atom '' sebelum mereka mulai
bekerja dengan program ini. After working with the program for 90 min, they were asked to sketch both concepts
again. Setelah bekerja dengan program selama 90 menit, mereka diminta untuk membuat sketsa kedua konsep itu
lagi.
 
5.4.3. 5.4.3. Performance tests Tes kinerja
 
A performance test was integrated into the program. Tes kinerja diintegrasikan ke dalam program. Because the
amount of user control was one of the questions the researcher tried to answer, maximum user control was allowed.
Karena jumlah kontrol pengguna adalah salah satu pertanyaan yang coba dijawab oleh peneliti, kontrol pengguna
maksimum diizinkan. Learners had the option either to do the test or to skip it. Peserta didik memiliki pilihan untuk
melakukan tes atau melewatkannya.
 
5.4.4. 5.4.4. Navigational pathways Jalur navigasi
 
A navigational pathway tracker was built into the program. Pelacak jalur navigasi dibangun ke dalam program.
This tracker sequentially recorded the path taken by the user, and the time spent at an interaction. Pelacak ini secara
berurutan mencatat jalur yang diambil oleh pengguna, dan waktu yang dihabiskan untuk interaksi.
 
5.4.5. 5.4.5. Questionnaires Kuisioner
 
Learners were briefed about the aim of the study. Peserta didik mendapat penjelasan tentang tujuan penelitian.
Then pre-program questionnaires were distributed for them to complete before they started working with the
program. Kemudian kuesioner pra-program didistribusikan untuk mereka selesaikan sebelum mereka mulai bekerja
dengan program ini. After completion of the program, then they were asked to complete the post-program
questionnaire (See Appendix A and B ). Setelah menyelesaikan program, mereka diminta untuk melengkapi
kuesioner pasca-program (Lihat Lampiran A dan B ). The main aim of the ques-tionnaires was to obtain the learners'
opinions about certain design features of the program, as well as to assess a possible change in the mental models
they may have formed. Tujuan utama dari kuesioner adalah untuk memperoleh pendapat peserta didik tentang fitur
desain tertentu dari program, serta untuk menilai kemungkinan perubahan dalam model mental yang mungkin
mereka telah terbentuk.
 
Both the pre-program and the post-program questionnaires can be grouped into the following 5 sections: Baik
kuesioner pra-program dan pasca-program dapat dikelompokkan ke dalam 5 bagian berikut:
 
Interaction, Interface, Involvement, Motivation, and Rate. Interaksi, Antarmuka, Keterlibatan, Motivasi, dan Nilai.
 
A nine point checklist format was used to assess sections 1 to 5. The reason for using a nine point checklist was
that the researcher wanted to allow for a neutral or a ''not sure'' response to highlight users' uncertainties. Sebuah
format daftar periksa sembilan titik digunakan untuk menilai bagian 1 hingga 5. Alasan untuk menggunakan daftar
periksa sembilan titik adalah bahwa peneliti ingin memungkinkan respons netral atau 'tidak pasti' untuk menyoroti
ketidakpastian pengguna. In reporting the data, a response of 1, 2 or 3 was taken as ''disagree'', a response of 4, 5 or
6 was taken as ''not sure'', while a response of 7, 8 or 9 was reported as ''agree''. Dalam melaporkan data, tanggapan
1, 2 atau 3 diambil sebagai '' tidak setuju '', respons 4, 5 atau 6 dianggap '' tidak yakin '', sementara respons 7, 8 atau
9 adalah dilaporkan sebagai '' setuju ''.
 
5.4.6. 5.4.6. Exclusion of other methods to obtain mental models Pengecualian metode lain untuk mendapatkan
model mental
 
We could also have asked the learners to draw mind maps or concept diagrams of the system, and we could have
used heuristic if/then/else statements, as well as a number of other methods of harvesting mental models. Kita juga
dapat meminta para peserta untuk menggambar peta pikiran atau diagram konsep dari sistem, dan kita dapat
menggunakan pernyataan heuristik jika / kemudian / lain, serta sejumlah metode lain untuk memanen model mental.
Practical considerations, and data saturation, led to the exclusion of these alternatives. Pertimbangan praktis, dan
kejenuhan data, menyebabkan pengecualian dari alternatif ini.
 
6. Discussion of results 6. Diskusi hasil
 
The ultimate aim of this research was to use the mental models developed by the users to refine and improve the
program Introduction to Basic Electricity. Tujuan utama dari penelitian ini adalah untuk menggunakan model mental
yang dikembangkan oleh pengguna untuk memperbaiki dan meningkatkan program Pengantar Dasar Listrik. The
precise goal of the research reported here was to determine what we could learn from obtaining some insight into the
mental models of the learners so that we can better decide upon the integration of generative and supplantive
elements of a multimedia program. Tujuan yang tepat dari penelitian yang dilaporkan di sini adalah untuk
menentukan apa yang dapat kita pelajari dari memperoleh beberapa wawasan ke dalam model mental para
pembelajar sehingga kita dapat dengan lebih baik memutuskan integrasi elemen generatif dan tambahan dari
program multimedia. We chose to con-sider the research questions from various qualitative perspectives rather than
to obtain and quantify the data. Kami memilih untuk mempertimbangkan pertanyaan-pertanyaan penelitian dari
berbagai perspektif kualitatif daripada untuk mendapatkan dan mengukur data. This was an exploratory study with a
very small sample, and we make no claim to the transferability of our results. Ini adalah penelitian eksplorasi dengan
sampel yang sangat kecil, dan kami tidak mengklaim hasil transfer dari hasil kami.
 
The study set out to determine learner views on the program, compared with those of the designers; Studi ini
bertujuan untuk menentukan pandangan pembelajar pada program, dibandingkan dengan para perancang; learner use
of the system – time spent, and navigation through the program; pelajar menggunakan sistem - waktu yang
dihabiskan, dan navigasi melalui program; and learners' acquisition of certain learn-ing concepts. dan akuisisi siswa
terhadap konsep belajar tertentu. The questions driving this study will be used as headings in discussing the findings.
Pertanyaan-pertanyaan yang mendorong penelitian ini akan digunakan sebagai judul dalam mendiskusikan temuan.
570 570               JC Cronje´, J. Fouche / Computers & Education 51 (2008) 559–583 JC Cronje´, J. Fouche / Komputer & Pendidikan 51
(2008) 559-583
 
6.1. 6.1. How did learners' mental models of educational multimedia programs di er from those of the designers?
Bagaimana model mental peserta didik program multimedia pendidikan berbeda dengan yang dimiliki oleh para
desainer?
 
6.1.1. 6.1.1. Interaction Interaksi
 
In the pre-program questionnaire learner mth indicated that he thought that computer based programs were
boring. Dalam kuisioner pra-program, peserta mth mengindikasikan bahwa dia berpikir bahwa program berbasis
komputer membosankan. In the post-program questionnaire, learners mtl and mtm joined the views of learner mth
and indicated that the feedback was boring. Dalam kuesioner pasca-program, peserta didik mtl dan mtm bergabung
dengan pandangan peserta didik dan menunjukkan bahwa umpan balik itu membosankan. During interviews, they
indicated that they would like to see more of a ''bells and whistles'' type of feedback. Selama wawancara, mereka
mengindikasikan bahwa mereka ingin melihat lebih banyak jenis umpan 'lonceng dan peluit'. Learner mth also
wanted audio/video feedback. Learner mth juga menginginkan umpan balik audio / video.
 
The wording: I felt as if someone was engaged in conversation with me on the post-program questionnaire was
not understood by everyone. Kata-kata: Saya merasa seolah-olah seseorang terlibat dalam percakapan dengan saya
di kuesioner pasca-program tidak dipahami oleh semua orang. All the learners from the disadvantaged school
indicated that they disagreed with the statement. Semua pembelajar dari sekolah yang kurang beruntung
mengindikasikan bahwa mereka tidak setuju dengan pernyataan tersebut. When they were asked why they disagreed
with the statement, they responded that they ''heard nothing''. Ketika ditanya mengapa mereka tidak setuju dengan
pernyataan itu, mereka menjawab bahwa mereka "tidak mendengar apa-apa".
 
Two of the three male learners from the technology-advantaged school also indicated that they disagreed with the
statement: I was encouraged by the responses given to my answers of questions, while one of them was unsure. Dua
dari tiga pelajar pria dari sekolah yang diuntungkan oleh teknologi juga menunjukkan bahwa mereka tidak setuju
dengan pernyataan: Saya terdorong oleh tanggapan yang diberikan pada jawaban pertanyaan saya, sementara salah
satu dari mereka tidak yakin. Learners mtl, ftl, mdl and fdm indicated that they did not understand all the questions,
although they were given answers. Peserta didik mtl, ftl, mdl dan fdm mengindikasikan bahwa mereka tidak
memahami semua pertanyaan, meskipun mereka diberi jawaban.
 
If only the specific opinions of the test group outlined above are taken into account, without considering the rest
of the data, one might be inclined to deduce that this program could be too easy for the high achiever, and at the
same time too di cult to be of any use to the low achiever. Jika hanya pendapat spesifik dari kelompok uji yang
diuraikan di atas diperhitungkan, tanpa mempertimbangkan sisa data, orang mungkin cenderung untuk
menyimpulkan bahwa program ini bisa terlalu mudah untuk berprestasi tinggi, dan pada saat yang sama terlalu sulit
untuk berguna bagi orang berprestasi rendah. As will be pointed out in later paragraphs, other data might indicate di
erently. Seperti yang akan ditunjukkan dalam paragraf-paragraf selanjutnya, data lain mungkin menunjukkan secara
berbeda.
 
6.1.2. 6.1.2. Interface Antarmuka
 
Except for learner mth, who expected video, sound and animation, all the learners responded positively on the
interface and interface design. Kecuali untuk pelajar, yang mengharapkan video, suara, dan animasi, semua
pembelajar merespon positif pada antarmuka dan desain antarmuka.
 
In the pre-programme questionnaire, learners mdl and fdh indicated that computers usually frustrated them.
Dalam kuesioner pra-program, peserta didik mdl dan fdh mengindikasikan bahwa komputer biasanya membuat
mereka frustasi. In the post-programme questionnaire, these learners indicated that the program as such did not frus-
trate them. Dalam kuesioner pasca-program, para pembelajar ini menunjukkan bahwa program seperti itu tidak
membetulkannya.
 
 Although 50% of the learners indicated that they were organised learners, they liked the freedom provided by the
program to jump from one topic directly to another topic. Meskipun 50% dari peserta menunjukkan bahwa mereka
adalah pelajar yang terorganisir, mereka menyukai kebebasan yang diberikan oleh program untuk melompat dari
satu topik langsung ke topik yang lain. These learners' mental models on learner control and navigation could be that
maximum learner control will ensure maximum learning. Model mental peserta didik pada kontrol dan navigasi
peserta didik bisa menjadi kontrol pelajar maksimum akan memastikan pembelajaran yang maksimal.
 
In the post-program questionnaire all, except learner mth, indicated that they loved the colours used in the
program. Dalam kuesioner pasca-program, kecuali pelajar, menunjukkan bahwa mereka menyukai warna yang
digunakan dalam program. Six learners indicated in the pre-program questionnaire that a red and orange screen
would look nice. Enam siswa diindikasikan dalam kuesioner pra-program bahwa layar merah dan oranye akan
terlihat bagus. During the interview, learner mth indicated that he preferred brighter colours to the autumn colours
used in the program. Selama wawancara, pelajar menunjukkan bahwa dia lebih suka warna yang lebih cerah dengan
warna musim gugur yang digunakan dalam program. Although he did not like the animations in the program, he
acknowledged that the animations in the program had simplified the content. Meskipun ia tidak menyukai animasi
dalam program, ia mengakui bahwa animasi dalam program telah menyederhanakan konten.
 
Two learners (fdl and mdl) indicated that, at times, they felt completely lost. Dua orang pelajar (fdl dan mdl)
mengindikasikan bahwa, kadang-kadang, mereka merasa benar-benar hilang. In the pre-program question-naire,
these learners also indicated that they gave up easily if they did not succeed with something. Dalam kuesioner-
pertanyaan pra-program, para pembelajar ini juga mengindikasikan bahwa mereka mudah menyerah jika mereka
tidak berhasil dengan sesuatu.
 
Table 3 shows that seven of the twelve learners had problems using the mouse in a drag-and-drop interac-tion.
Tabel 3 menunjukkan bahwa tujuh dari dua belas peserta didik memiliki masalah menggunakan mouse dalam
interaksi drag-and-drop. Two of the seven learners who had problems indicated that they had had ample computer
experience. Dua dari tujuh peserta yang memiliki masalah menunjukkan bahwa mereka telah memiliki pengalaman
komputer yang cukup.
 
The two learners from the technology-advantaged school who experienced mouse problems indicated that they
had previously used the mouse to play games. Dua pelajar dari sekolah yang diuntungkan teknologi yang mengalami
masalah pada tikus mengindikasikan bahwa mereka sebelumnya menggunakan mouse untuk bermain game. When
the researcher asked them about this, they said that Ketika peneliti bertanya tentang hal ini, mereka mengatakan itu
 
Table 3 Tabel 3
 
Learners having problems with drag-and-drop mouse actions Peserta didik mengalami masalah dengan aksi mouse drag-and-drop
 

 
Academic ability
 
 
Kemampuan akademik  
   
 
 

High
 
 
Low Rendah
   
Medium Medium  
Tinggi
Female, resource-disadvantaged school Perempuan, sekolah yang
kekurangan sumber daya ¤¤ ¤¤ ¤¤
Male, resource-disadvantaged school Laki-laki, sekolah yang
kekurangan sumber daya ¤¤ ¤¤ **
Female, resource-advantaged school Sekolah Perempuan, sumber
daya yang diuntungkan     **
Male, resource-advantaged school Laki-laki, sekolah yang
diuntungkan sumber daya ** ** **
 
Previous computer experience; Pengalaman komputer
sebelumnya; experienced mouse problems. masalah mouse yang
dialami. ¤ Limited computer experience; ¤ Pengalaman komputer
terbatas; experienced mouse problems. masalah mouse yang
dialami. * Did not experience any mouse problems. * Tidak
mengalami masalah pada mouse.
 JC Cronje´, J. Fouche / Computers & Education 51 (2008) 559–583 JC Cronje´, J.
Fouche / Komputer & Pendidikan 51 (2008) 559-583 571 571
 
they were used to pointing-and-clicking, but that they had seldom used the mouse for clicking-and-dragging. mereka
digunakan untuk menunjuk-dan-mengklik, tetapi bahwa mereka jarang menggunakan mouse untuk mengklik-dan-
menyeret. They had never used the computer as an electronic tool to simplify daily tasks, such as using the computer
as a word-processor or calculation aid. Mereka tidak pernah menggunakan komputer sebagai alat elektronik untuk
menyederhanakan tugas sehari-hari, seperti menggunakan komputer sebagai pengolah kata atau bantuan
perhitungan.
 
It is obvious that the mental models that they had about the computer system were limited to that of a machine
allowing them to play games. Jelas bahwa model mental yang mereka miliki tentang sistem komputer terbatas pada
mesin yang memungkinkan mereka untuk bermain game. Although they were aware that a computer is much more
than a gaming machine, this knowledge had not been transferred to that of a working model allowing them to use the
com-puter as a tool. Meskipun mereka sadar bahwa komputer lebih dari sekadar mesin judi, pengetahuan ini belum
ditransfer ke model kerja yang memungkinkan mereka menggunakan komputer sebagai alat.
 
Five of the twelve learners had problems remembering the passwords they had selected. Lima dari dua belas
pelajar memiliki masalah mengingat kata sandi yang mereka pilih. When the program asked them for a password,
they just typed a keyboard sequence and when it asked them to retype the pass-word, they were unable to retype it.
Ketika program meminta kata sandi, mereka hanya mengetik urutan keyboard dan ketika diminta untuk mengetik
ulang kata sandi, mereka tidak dapat mengetik ulang.
 
It is clear from the observations illustrated in Table 4 that most of the learners who had limited computer
experience, experienced password problems. Jelas dari pengamatan yang diilustrasikan pada Tabel 4 bahwa
sebagian besar pembelajar yang memiliki pengalaman komputer terbatas, mengalami masalah kata sandi. The
reason for this could be that these users lacked the necessary keyboard experience to enable them to type faultlessly.
Alasannya adalah karena pengguna ini tidak memiliki pengalaman keyboard yang diperlukan untuk memungkinkan
mereka mengetik tanpa kesalahan. The mental model that the researcher had of the user of the system was that all
users will be able to type without any typing error. Model mental yang peneliti miliki dari pengguna sistem adalah
bahwa semua pengguna akan dapat mengetik tanpa kesalahan pengetikan. The password log on procedure could
thus present a problem to users and should be redesigned. Prosedur log masuk kata sandi dapat menimbulkan
masalah bagi pengguna dan harus dirancang ulang.
 
6.1.3. 6.1.3. Involvement Keterlibatan
 
After working with the program, more learners favoured the computer based instruction. Setelah bekerja dengan
program, lebih banyak peserta didik menyukai instruksi berbasis komputer. Learner mth was the only learner whose
neutral views on computer-based instruction remained unchanged. Learner mth adalah satu-satunya pembelajar yang
pandangannya netral pada instruksi berbasis komputer tetap tidak berubah.
 
Learner mth indicated in the pre-program questionnaire that he agreed that the lessons in class were mostly dull.
Learner mth ditunjukkan dalam kuesioner pra-program bahwa ia setuju bahwa pelajaran di kelas kebanyakan
membosankan. In the post-program questionnaire, he indicated that the multimedia lessons were di cult to follow
and that they were dull. Dalam kuesioner pasca-program, ia menunjukkan bahwa pelajaran multimedia sulit diikuti
dan bahwa mereka membosankan. During the interview, he admitted that the lessons were not di cult to follow, but
they were dull in that he had expected more of an action-type of feedback. Selama wawancara, ia mengakui bahwa
pelajaran tidak sulit untuk diikuti, tetapi mereka membosankan karena ia mengharapkan lebih banyak jenis tindakan
umpan balik.
 
6.1.4. 6.1.4. Motivation Motivasi
 
All the learners acknowledged that working through the program was a stimulating and motivating expe-rience,
although learners mth and fth indicated that the program did not challenge them to do their best. Semua peserta
mengakui bahwa bekerja melalui program adalah pengalaman yang merangsang dan memotivasi, meskipun peserta
didik mengatakan bahwa program tersebut tidak menantang mereka untuk melakukan yang terbaik. In the pre-
questionnaire, both learners indicated that even the teachers did not always challenge them to do their best. Dalam
pra-kuesioner, kedua pembelajar menunjukkan bahwa bahkan para guru tidak selalu menantang mereka untuk
melakukan yang terbaik. Learner mth also indicated that the program did not really keep him involved. Learner mth
juga mengindikasikan bahwa program itu tidak benar-benar membuatnya terlibat.
 
In the pre-program questionnaire, six learners had indicated that they would not really like to learn more about
atoms and electricity. Dalam kuesioner pra-program, enam peserta telah menunjukkan bahwa mereka tidak akan
benar-benar ingin belajar lebih banyak tentang atom dan listrik. After working with the program, these two learners
showed more interest in this topic. Setelah bekerja dengan program ini, kedua pembelajar ini menunjukkan minat
yang lebih dalam topik ini.
 
In the pre-program questionnaire, learner mth indicated that he thought that a computer program will not make
the normal classwork easier. Dalam kuesioner pra-program, peserta didik mengatakan bahwa dia berpikir bahwa
program komputer tidak akan membuat pekerjaan kelas yang normal menjadi lebih mudah. After working through
the program, his views on the usefulness of the com-puter in a classroom changed completely. Setelah bekerja
melalui program, pandangannya tentang kegunaan komputer di ruang kelas berubah sepenuhnya.
 
In the pre-program questionnaire, four learners indicated that to them, extra classes were a waste of time. Dalam
kuesioner pra-program, empat peserta menunjukkan bahwa bagi mereka, kelas tambahan adalah pemborosan waktu.
None of these learners thought that working through the program was a waste of time, although learner mth was
relatively neutral on this aspect. Tak satu pun dari para pelajar ini berpikir bahwa bekerja melalui program itu adalah
buang-buang waktu, meskipun pelajar relatif netral pada aspek ini.
 
If one compares the pre-program and post-program responses, one can see that most of the learners were
positively motivated through the use of the program. Jika seseorang membandingkan respon pra-program dan pasca-
program, orang dapat melihat bahwa sebagian besar peserta didik termotivasi positif melalui penggunaan program.
 
Table 4 Tabel 4
 
Learners who experienced password problems Peserta didik yang mengalami masalah kata sandi
 

 
Academic ability
 
 
Kemampuan akademik  
   
 
 

High
 
 
Low Rendah
   
Medium Medium  
Tinggi
Female, resource-disadvantaged school Perempuan, sekolah yang
kekurangan sumber daya ¤¤ ¤¤ **
Male, resource-disadvantaged school Laki-laki, sekolah yang
kekurangan sumber daya ¤¤ ¤¤ **
Female, resource-advantaged school Sekolah Perempuan, sumber
daya yang diuntungkan   ** **
Male, resource-advantaged school Laki-laki, sekolah yang
diuntungkan sumber daya ** ** **
 
Previous computer experience; Pengalaman komputer sebelumnya;
experienced password problems. masalah kata sandi yang
berpengalaman. ¤ Limited computer experience; ¤ Pengalaman
komputer terbatas; experienced password problems. masalah kata
sandi yang berpengalaman. * Did not experience any password
problems. * Tidak mengalami masalah kata sandi.
572 572               JC Cronje´, J. Fouche / Computers & Education 51 (2008) 559–583 JC Cronje´, J. Fouche / Komputer & Pendidikan 51
(2008) 559-583
 
6.1.5. 6.1.5. Rate Menilai
 
All the learners of the technology-advantaged school indicated that they were pressed for time while work-ing
with the program. Semua pembelajar dari sekolah yang diuntungkan oleh teknologi mengindikasikan bahwa mereka
terdesak waktu saat bekerja dengan program tersebut. This is true, because they had a limited time available to work
with the program. Ini benar, karena mereka memiliki waktu terbatas untuk bekerja dengan program ini. All the
learners indicated that despite the limited time available, they felt that they could work at their own pace. Semua
peserta menunjukkan bahwa meskipun waktu yang tersedia terbatas, mereka merasa bahwa mereka dapat bekerja
dengan kecepatan mereka sendiri.
 
For learners mth, fth and mdl the course material was presented too slowly, although learner mdl also indi-cated
that the program ran too quickly. Untuk peserta didik mth, fth dan mdl materi kursus disajikan terlalu lambat,
meskipun pembelajar mdl juga menunjukkan bahwa program berjalan terlalu cepat.
 
6.1.6. 6.1.6. Summary of mental models of designers and users Ringkasan model mental perancang dan pengguna
 
The various instruments allowed us to derive an understanding of how the users viewed the program. Berbagai
instrumen memungkinkan kami untuk memperoleh pemahaman tentang bagaimana pengguna melihat program.
Com-paring their view to the initial design specifications we were able to determine the di erences and similarities
between the mental models of the researcher/designer and the users as shown in Table 5 . Sesuaikan pandangan
mereka dengan spesifikasi desain awal kami dapat menentukan perbedaan dan persamaan antara model mental
peneliti / desainer dan pengguna seperti yang ditunjukkan pada Tabel 5 .
 
Table 5 shows that there was a reasonably high level of correlation between the mental models of the Tabel 5
menunjukkan bahwa ada tingkat korelasi yang cukup tinggi antara model mental designers and the users in terms of
the didactic usefulness of graphics and animation, a preference for this pro-gram over classroom instruction about
the same topic, and the possibilities of learners working at their own pace. desainer dan pengguna dalam hal
kegunaan didaktik dari grafik dan animasi, preferensi untuk pengajaran ini di atas instruksi kelas tentang topik yang
sama, dan kemungkinan peserta didik bekerja dengan kecepatan mereka sendiri. In one instance, the e ect of
maximum learner control, the learners were sure that learner control would ensure maximum learning, while the
designers thought that it might. Dalam satu contoh, pengaruh kontrol pembelajar maksimum, para pembelajar yakin
bahwa kendali pembelajar akan memastikan pembelajaran maksimum, sementara para perancang berpikir bahwa itu
mungkin. Of course it turns out that maximum learner control was not useful for all learners. Tentu saja, kontrol
pelajar maksimum tidak berguna untuk semua siswa. The strongest disagreement between the mental models of
designers and users came with the use of the help facility. Ketidaksepakatan terkuat antara model mental perancang
dan pengguna datang dengan menggunakan fasilitas bantuan. Here it was clear that users did not even expect there
to be a help facility, while the designers expected all learners to use it. Di sini jelas bahwa pengguna bahkan tidak
mengharapkan ada fasilitas bantuan, sementara para perancang mengharapkan semua pelajar untuk
menggunakannya. It is our opinion that here we, the design-ers, were probably naı¨ve in our expectations. Ini adalah
pendapat kami bahwa di sini kita, para perancang, mungkin naïf dalam harapan kita.
 
6.2. 6.2. What could we learn from learners' navigation through the program? Apa yang bisa kita pelajari dari
navigasi pelajar melalui program ini?
 
The following observations were made while the learners were working on the system: Observasi berikut dibuat
ketika para siswa sedang mengerjakan sistem:
 
6.2.1. 6.2.1. Using help Menggunakan bantuan
 
Although a full on-line hypertext help facility was available, it was only accessed by one learner. Meskipun
fasilitas bantuan hypertext on-line lengkap tersedia, itu hanya diakses oleh satu pelajar. This spe-cific learner
accessed help at the end of the session, after she had already worked through the whole program. Pelajar khusus ini
mengakses bantuan di akhir sesi, setelah dia menyelesaikan seluruh program. When asked why she chose the help-
facility, she answered ''I am just curious what the program can do''. Ketika ditanya mengapa dia memilih fasilitas
bantuan, dia menjawab, "Saya hanya ingin tahu apa yang bisa dilakukan oleh program".
Table 5 Tabel 5
Description of the mental models of the researcher and program users Deskripsi model mental dari peneliti dan pengguna program
#. #.
Designer's model Model perancang
The computer is a didactic tool Komputer adalah alat
1. 1 didaktik
Everybody will type without any typing errors Semua
2. 2 orang akan mengetik tanpa kesalahan pengetikan
   
All users will access the help pull-down menu when
Semua pengguna akan mengakses menu bantuan pull-
3. 3 down kapan
  experiencing problems mengalami masalah
Everybody will understand figurative speech and Semua Wordings of questions should be taken
4. 4. orang akan mengerti bahasa kiasan dan
  expressions ekspresi
Too many audio and video e ects could shift the Terlalu
5. 5. banyak efek audio dan video bisa menggeser
focus from important concepts towards cosmetic fokus
  dari konsep penting menuju kosmetik
  extras ekstra
All users will always understand all the questions if
Semua pengguna akan selalu memahami semua
6. 6 pertanyaan jika
  answers are given jawaban diberikan
Graphics and animations simplify contents Grafik dan
7. 7 animasi menyederhanakan konten
Maximum learner control might ensure maximum
Kontrol pelajar maksimum mungkin memastikan
8. 8. maksimum
  learning belajar
Constant, visually pleasing colours should be used
Konstan, warna visual yang menyenangkan harus
9. 9. digunakan
  in a presentation dalam presentasi
10. A computer-based lesson might be preferable to
10. Pelajaran berbasis komputer mungkin lebih baik
  normal class instruction instruksi kelas normal
11. The program Introduction to Basic Electricity might
11. Program Pengantar Listrik Dasar mungkin
  motivate learners memotivasi peserta didik
A computer-based lesson will allow users to work at
12. Pelajaran berbasis komputer akan memungkinkan
12. pengguna untuk bekerja
their own pace kecepatan mereka sendiri
Users' model held by
Model pengguna dipegang
 
Users' model Model pengguna  
oleh  
The computer is a game machine Komputer
adalah mesin permainan 2 learners 2 peserta didik
The computer will automatically correct
Komputer akan secara otomatis benar 5 learners 5 peserta didik
typing errors kesalahan pengetikan    
The help-facility is not part of the program
Fasilitas bantuan bukan bagian dari program 11 users 11 pengguna
     
5 users from the resource-
5 pengguna dari sumber
Wordings pertanyaan harus diambil daya-
disadvantaged school
sekolah yang kurang
literally secara harfiah beruntung
Multimedia programs should contain ''bells
Program multimedia harus mengandung '' 3 male users from the 3
lonceng pengguna laki-laki dari
and whistles'', video and much more audio dan resource-advantaged
peluit, video, dan banyak lagi audio diuntungkan sumber daya
  school sekolah
A computer program should always explain
Program komputer harus selalu menjelaskan 6 users 6 pengguna
everything in detail semuanya secara detail    
Graphics and animations simplify contents
Grafik dan animasi menyederhanakan konten All users Semua pengguna
Maximum learner control will ensure Kontrol
pelajar maksimum akan memastikan All users Semua pengguna
maximum learning pembelajaran maksimal    
Bright colours should be used in a Warna-
warna cerah harus digunakan dalam One user Satu pengguna
presentation presentasi    
A computer-based lesson is preferable to
Pelajaran berbasis komputer lebih baik All users Semua pengguna
normal class instruction instruksi kelas normal    
The use of program Introduction to Basic
Penggunaan program Pengantar Dasar All users Semua pengguna
Electricity was a motivating experience Listrik
adalah pengalaman yang memotivasi    
A computer-based lesson will allow the user
Pelajaran berbasis komputer akan
memungkinkan pengguna All users Semua pengguna
to work at his own pace untuk bekerja dengan
 
langkahnya sendiri  
 JC Cronje´, J. Fouche / Computers & Education 51 (2008) 559–583 JC Cronje´, J. 573
Fouche / Komputer & Pendidikan 51 (2008) 559-583 573

Table 6 Tabel 6      
Learners accessing the help facility Peserta didik mengakses fasilitas
bantuan
 
   
 
 
 
 

Academic ability
 
 
Kemampuan akademik  
   
 
 

High
 
 
Low Rendah
   
Medium Medium  
Tinggi
Female, resource-disadvantaged school Perempuan, sekolah yang
kekurangan sumber daya ** ** **
Male, resource-disadvantaged school Laki-laki, sekolah yang
kekurangan sumber daya ** ** **
Female, resource-advantaged school Sekolah Perempuan, sumber
daya yang diuntungkan ** ** ee
Male, resource-advantaged school Laki-laki, sekolah yang
diuntungkan sumber daya ** ** **
 
e Accessed the help facility. e Mengakses fasilitas bantuan.
* Did not access the help facility. * Tidak mengakses fasilitas bantuan.
 
The results illustrated in Table 6 might indicate that the users' mental models of the system did not include the
tools available to assist them when working with the system. Hasil yang diilustrasikan pada Tabel 6 dapat
menunjukkan bahwa model mental pengguna dari sistem tidak termasuk alat yang tersedia untuk membantu mereka
ketika bekerja dengan sistem. The focus of the users could have been so task-oriented that they did not see the help-
facility as being part of the system. Fokus pengguna bisa jadi sangat berorientasi pada tugas sehingga mereka tidak
melihat fasilitas bantuan sebagai bagian dari sistem.
 
6.2.2. 6.2.2. Time spent with the program Waktu yang dihabiskan dengan program ini
 
No indication of maximum time allowed or any other indication concerning time was given to the learners,
although the researcher has decided beforehand to allow a maximum of 90 min per learner. Tidak ada indikasi waktu
maksimum yang diizinkan atau indikasi lain tentang waktu diberikan kepada peserta didik, meskipun peneliti telah
memutuskan sebelumnya untuk memungkinkan maksimum 90 menit per pelajar. The learners were not briefed on
how long the program would take, in order to allow them to proceed at their own pace. Para peserta tidak diberitahu
tentang berapa lama program akan mengambil, untuk memungkinkan mereka untuk melanjutkan dengan langkah
mereka sendiri.
 
Unfortunately, the group from the technology-advantaged school indicated that they had limited time to help
evaluate the program. Sayangnya, kelompok dari sekolah yang diuntungkan oleh teknologi menunjukkan bahwa
mereka memiliki waktu terbatas untuk membantu mengevaluasi program. Therefore, they spent less time working
on the program than the group from the disadvantaged school, although more content was covered by the first group
than the second group. Oleh karena itu, mereka menghabiskan lebih sedikit waktu untuk mengerjakan program
daripada kelompok dari sekolah yang kurang beruntung, meskipun lebih banyak konten yang dicakup oleh
kelompok pertama daripada kelompok kedua.
 
Fig. 8 graphically represents the time taken by the learners working with the program. Gambar. 8 grafis mewakili
waktu yang diambil oleh peserta didik yang bekerja dengan program ini.
 
6.2.3. 6.2.3. Navigation Navigasi
 
6.2.3.1. 6.2.3.1. Navigation through the program. Navigasi melalui program. The built-in user tracking option of
Authorware was used to deter-mine which sections (or levels) of the program were accessed and when. Opsi
pelacakan pengguna yang sudah ada dari Authorware digunakan untuk menentukan bagian mana (atau level) dari
program yang diakses dan kapan. Some learners followed a linear pro-gression through the program, visiting each
section, and doing so in numerical order. Beberapa pelajar mengikuti progresi linear melalui program, mengunjungi
setiap bagian, dan melakukannya dalam urutan numerik. Others accessed sections seemingly at random. Bagian lain
yang diakses tampaknya secara acak. Although they accessed almost all the sections they missed important
information, while other learners worked slowly and systematically, but sequentially completing only about 50 per
cent of the program. Meskipun mereka mengakses hampir semua bagian, mereka kehilangan informasi penting,
sementara para pembelajar lain bekerja secara perlahan dan sistematis, tetapi secara berurutan menyelesaikan hanya
sekitar 50 persen dari program.
 
Fig. 9 represents the number of sections visited by the learners while using the program. Gambar. 9 mewakili
jumlah bagian yang dikunjungi oleh peserta saat menggunakan program. ''Deepness'' refers 'Deepness' 'merujuk
simply to the number of sections visited, and how far into the section the learner went. cukup dengan jumlah bagian
yang dikunjungi, dan seberapa jauh ke bagian yang didatangi pelajar.
 
6.2.3.2. 6.2.3.2. Navigational pathways. Jalur navigasi. Although all the candidates were allowed to interact with the
program for a maximum of 90 min, many candidates indicated that they had worked through the program using less
time. Meskipun semua kandidat diizinkan untuk berinteraksi dengan program untuk maksimal 90 menit, banyak
kandidat menunjukkan bahwa mereka telah bekerja melalui program menggunakan lebih sedikit waktu.
 
Timespent withthe program. Jangka waktu dengan
program ini.
 
1:30
  1:30                        

1:20
  1:20                        

1:10
1:10                        

(h:m 1:00
m) 1:00                        

(h: 0:50
mm)
0:50                        

0:40
                       

Time 0:40                        

Wakt 0:30
u 0:30                        

0:20
                       

  0:20                        

0:10
  0:10                        

0:00
  0:00      
md
 

mt mt ft md fd
m h m h fd
m m h
mt mt ftl ft fth mdl md md fdl fd fd
  mtl mtl m h ftl m fth mdl m h fdl m h  

Student
Mahasis
            wa            

 
 
Fig. 8. Time spent with the program. Gambar. 8. Waktu yang dihabiskan dengan program.
574 574               JC Cronje´, J. Fouche / Computers & Education 51 (2008) 559–583 JC Cronje´, J. Fouche / Komputer & Pendidikan 51
(2008) 559-583
 
Deepness level reached Tingkat kedalaman tercapai
 
1200
  1200                        

1000
level 1000                        

tingkat 800                      
800
 

                       

600
600                      
Deepness  

400
Kedalam 400                      
an
 

200                        

  200                        

  00                        

mt mt ft md md fd fd
m h m m h m h
mt mt ftl ft fth mdl md md fdl fd fd
  mtl mtl m h ftl m fth mdl m h fdl m h  

Student
Mahasis
            wa            

 
 
Fig. 9. Program deepness level reached by learners. Gambar 9. Tingkat kedalaman program yang dicapai oleh peserta didik.
 
 
Examples of the navigational pathways through the program for some learners are presented visually and
discussed in Figs. Contoh jalur navigasi melalui program untuk beberapa peserta didik disajikan secara visual dan
didiskusikan dalam Gambar. 10–12 . 10–12 . The vertical axis denotes the level reached inside the program and is
calibrated in terms of interactions. Sumbu vertikal menunjukkan tingkat yang dicapai di dalam program dan
dikalibrasi dalam hal interaksi. The horizontal axis shows time. Sumbu horizontal menunjukkan waktu. The dotted
diagonal line represents the hypothetical track of a learner taking a linear path through the program and completing
it exactly on time in one hour. Garis diagonal bertitik menunjukkan lintasan hipotetis seorang pembelajar yang
mengambil jalur linear melalui program dan menyelesaikannya tepat waktu dalam satu jam.
 
The navigational record of learner mth seems to confirm Waern (1990) view in that experienced users first try a
top-down approach. Catatan navigasi pelajar nampaknya mengkonfirmasi pandangan Waern (1990) bahwa
pengguna berpengalaman pertama mencoba pendekatan top-down. Fig. 10 shows that he started the Introduction to
Static Electricity module, but two interactions before the end of the module, he jumped to Electric Fields and
completed all the sections. Gambar. 10 menunjukkan bahwa ia memulai modul Pengantar Listrik Statis, tetapi dua
interaksi sebelum akhir modul, ia melompat ke Medan Listrik dan menyelesaikan semua bagian. He then jumped
back to Coulomb's Law and completed the first section. Dia kemudian melompat kembali ke Hukum Coulomb dan
menyelesaikan bagian pertama. He jumped to Electrical Fields and completed all the sections of this module. Dia
melompat ke Bidang Listrik dan menyelesaikan semua bagian dari modul ini. In the interview, he mentioned that he
jumped to this section because they had not yet done this topic in class. Dalam wawancara, dia menyebutkan bahwa
dia melompat ke bagian ini karena mereka belum melakukan topik ini di kelas. This was also the reason why he
skipped Atoms, because he thought that he knew enough of the topic. Ini juga alasan mengapa dia melewatkan
Atom, karena dia berpikir bahwa dia cukup tahu topiknya.
 
Fig. 11 shows that learner mtl spent 12 min working on the first module Gambar. 11 menunjukkan bahwa pelajar
mtl menghabiskan 12 menit mengerjakan modul pertama Introduction. Pengantar. He jumped back to Dia melompat
kembali the menu and once again worked through the first module. menu dan sekali lagi bekerja melalui modul
pertama. He jumped to module Atoms and did the test. Dia melompat ke modul Atom dan melakukan tes. Although
the program advised him to revise the first section again, he ignored the advice, skipped the next section of the
module Atoms and completed the last section of this module. Meskipun program menyarankannya untuk merevisi
bagian pertama lagi, ia mengabaikan saran tersebut, melewatkan bagian selanjutnya dari modul Atom dan
menyelesaikan bagian terakhir dari modul ini. After starting o linearly, the lear-ner used the top-down approach (
Waern, 1990 ) for the rest of the program. Setelah memulai o linear, lear-ner menggunakan pendekatan top-down (
Waern, 1990 ) untuk sisa program. It would seem that allowing free access to low achievers is not a good idea.
Sepertinya mengizinkan akses gratis ke yang berprestasi rendah bukanlah ide yang bagus. This is confirmed by the
tracks of the other low achievers, such as learner fdl. Hal ini dikonfirmasi oleh trek dari orang-orang berprestasi
rendah lainnya, seperti pelajar fdl. From some of their graphs one could deduce that they got lost completely in the
program and spent most of their time returning to the menu and then trying another section. Dari beberapa grafik
mereka dapat disimpulkan bahwa mereka tersesat sepenuhnya dalam program dan menghabiskan sebagian besar
waktu mereka kembali ke menu dan kemudian mencoba bagian lain.
 
 
 
 
 
 
 

Program
leve l
Program
leve l
 
 
Linear path Jalur linier
 
  mth mth  

1200
1200    

ElectricFields
1000
ElectricFields
1000  
 

800 Coulomb's Law

800 Hukum  

  Coulomb  

600
600    

400
400 Atoms Atom  

   

200
200 Introduction  

pengantar
0 0
 

   

 
 
000
:
000 0:10 200: 300: 0:40 500: 001:
: 0:10 200: 300: 0:40 500: 001:
 
Time Waktu
 
Fig. 10. Navigation tracking record of male, technology-advantaged, high achiever. Gambar. 10. Rekam jejak navigasi pria, teknologi yang
diuntungkan, berprestasi tinggi.
 JC Cronje´, J. Fouche / Computers & Education 51 (2008) 559–583 JC Cronje´, J.
Fouche / Komputer & Pendidikan 51 (2008) 559-583 575 575
 
Linear path Jalur linier
mtl mtl

 
 
1200 1200
 
ElectricFields ElectricFields
1000 1000
 
l l             Coulomb's Law  

leve 800 Hukum

tanggul 800  
 
 
 
 
 
 
 
 
   
Coulomb  

600
 
Program
Program 600  
 
 
 
 
 
 
 
 
 
 
 
 

400
 
   
400  
 
 
 
 
 
 
 
 
  Atoms Atom  

200
  200           Introduction  

            pengantar
0 0
 

               

000: 0:10 0:20 0:30 0:40 0:50 001:

  000: 0:10 0:20 0:30 0:40 0:50 001:  

 
Time Waktu
 
Fig. 11. Navigation tracking record of male, technology-advantaged, low achiever. Gambar. 11. Rekam jejak navigasi laki-laki, teknologi yang
diuntungkan, berprestasi rendah.
 
 
Linear path Jalur linier
 
fdl
            fdl    

1200
  1200              

ElectricFields
  1000           ElectricFields
  1000            
 

l l                

800 Coulomb's Law

leve 800           Hukum  

tanggul             Coulomb  

600
Progra 600            
m
 

400
Progra 400           Atoms Atom
m
 

             

           
200  

  200           Introduction  

            pengantar
0 0
 

               

0:00 100: 0:20 0:30 400:500:

  0:00 100: 0:20 0:30 400:500: 1:00 1:00  

Time
        Waktu        
 
 
Fig. 12. Navigation tracking record of female, disadvantaged, low achiever. Gambar. 12. Rekam jejak navigasi perempuan, kurang beruntung,
berprestasi rendah.

 
Fig. 12 shows that learner fdl jumped to the second module, Gambar. 12 menunjukkan bahwa pelajar fdl
melompat ke modul kedua, Atoms, after working through the first 2 inter-actions of the first module. Atom, setelah
bekerja melalui 2 inter-aksi pertama dari modul pertama. She decided not to take the on-line test, and jumped to the
second section where she spent about 30 min working on it. Dia memutuskan untuk tidak mengikuti tes on-line, dan
melompat ke bagian kedua di mana dia menghabiskan sekitar 30 menit untuk mengerjakannya. After completing
section 3, she returned to section 2 of the same module, and, after working through the first 3 interactions, started to
jump randomly to di erent sections in the program. Setelah menyelesaikan bagian 3, ia kembali ke bagian 2 dari
modul yang sama, dan, setelah bekerja melalui 3 interaksi pertama, mulai melompat secara acak ke bagian yang
berbeda dalam program.
 
After completing the module on atoms, learner fdl jumped back to the menu, and then she started to jump,
apparently randomly, to di erent sections in the program. Setelah menyelesaikan modul pada atom, pelajar fdl
melompat kembali ke menu, dan kemudian dia mulai melompat, tampaknya secara acak, ke bagian yang berbeda
dalam program. The facilitator o ered some help, and she told him that she was once again trying to find the Atoms
and electricity module. Fasilitator memberikan bantuan, dan dia memberi tahu dia bahwa dia sekali lagi mencoba
menemukan atom dan modul listrik. It appears as if she got ''lost in hyper-space'' for a short time. Tampaknya
seolah-olah dia "tersesat dalam ruang-luar" untuk waktu yang singkat. These unintentional jumps might possibly be
ascribed to limited computer experience. Lompatan yang tidak disengaja ini mungkin dianggap berasal dari
pengalaman komputer yang terbatas.
 
In considering the navigational paths of the high and low achievers it would seem that low achievers do not
benefit from free access – they get lost. Dalam mempertimbangkan jalur navigasi dari orang yang berprestasi tinggi
dan rendah, tampaknya orang yang berprestasi rendah tidak mendapatkan manfaat dari akses gratis - mereka
tersesat. High achievers, on the other hand, benefit from free access as it allows them to save time by skipping
sections that they already know. Mereka yang berprestasi tinggi, di sisi lain, mendapat manfaat dari akses gratis
karena memungkinkan mereka untuk menghemat waktu dengan melewatkan bagian yang sudah mereka ketahui. It
might be useful to develop an intelligent tracking system that would grant linear access to low achievers and random
access to high achievers – based on their results as they work through the program. Mungkin berguna untuk
mengembangkan sistem pelacakan cerdas yang akan memberikan akses linier kepada orang yang berprestasi rendah
dan akses acak ke yang berprestasi tinggi - berdasarkan hasil mereka saat mereka bekerja melalui program.
 
6.3. 6.3. What visible growth could we see in terms of learners' mental models of the subject matter? Pertumbuhan
nyata apa yang dapat kita lihat dalam hal model mental peserta didik dari materi pelajaran?
 
In an attempt to measure learners' growth in understanding of various concepts we asked them to draw sketches
of such concepts as ''electricity'' and ''atom''. Dalam upaya untuk mengukur pertumbuhan peserta didik dalam
memahami berbagai konsep kami meminta mereka untuk menggambar sketsa konsep-konsep seperti '' listrik '' dan ''
atom ''. We discuss here the growth we saw in the learners' understanding of the concept ''atom''. Di sini kita
membahas pertumbuhan yang kita lihat dalam pemahaman siswa tentang konsep '' atom ''. Similar results were
found with the concept ''electricity'' and it would be repetitive to relate these here. Hasil serupa ditemukan dengan
konsep '' listrik '' dan akan berulang untuk menghubungkan ini di sini.
576 576               JC Cronje´, J. Fouche / Computers & Education 51 (2008) 559–583 JC Cronje´, J. Fouche / Komputer & Pendidikan 51
(2008) 559-583
 
To be able to categorise the mental model of the concept ''atom'', we considered the historical development of the
model of an atom and the mental models that scientists have formed of the atom over the years ( Table 7 ). Untuk
dapat mengkategorikan model mental dari konsep '' atom '', kami mempertimbangkan perkembangan historis dari
model atom dan model mental yang telah terbentuk oleh para ilmuwan dari atom selama bertahun-tahun ( Tabel 7 ).
 
 
Table 7 Tabel 7
 
Historical development of the atomic model
 

 
Year Tahun
   
Model developed by  
Model also known as
1803–1807 Dalton Dalton Billiard ball model

 
 
 
 
 
1898 1898               Thomson               Plum pudding model

 
 
 
 
1910 1910               Rutherford

 
 
 
 
1913 1913               Bohr               Planetary model

 
 
 
 
1925 1925               Schro¨dinger               Wave mechanical model
 

 
 
 
 
 
Table 8 Tabel 8
 
Sketches of atoms
 

 
 
 
Pre-program sketches  
   
 
   
Post-program sketches
   
 
 

 
 
Academic ability
 
 
 
 
 
 
 
Academic ability
 
 
 
 
 

Medium Medium
 
 
Low Rendah
 
Medium
   
High Tinggi  
 
Low Rendah
   
Medium  
High Tinggi

 
 
Mt Mt

 
 
 
 
 
Ft

 
 
 
 
Md Md

 
 
 
 
Fd
 

 
 
Code mt = male, ex-TED school; ft = female, ex-TED school; md = male, ex DET school; fd = female, ex-DET school.
  JC Cronje´, J. Fouche / Computers & Education 51 (2008) 559–583 577

Table 9 Tabel 9      
Categories of atom concepts
 
 
 
 
 
 
 

Category Kategori
   
Pre-program  
Post-program  
 

Other Lain Mdl    

Billiard ball model mtl, mtm, ftl, fth, mdh    
Plum pudding model/Rutherford mth, mdm, fdl, fdm, fdh mtl, ftl, mdl, mdh, fdl  
Planetary model
   
ftm  
mtm, mth, ftm, fth, mdm, fdm, fdh
 

 
Code. mdl = male, Disadvantaged school, low academic ability; ftm = female, technology enabled school, medium academic ability.
 
 
 
In our research participants were asked to sketch the concept ''atom'' both before and after using the pro-gram.
The results are summarised in Table 8 .
 
Table 9 shows a classification of the results from table 8 using the categories of table 7 as a basis, while Table 10
graphically illustrates the changes in the concepts that took place.
 
 
 
 
Table 10 Tabel 10
 
Changes in users' models of the concept ''atom''
 
578               JC Cronje´, J. Fouche / Computers & Education 51 (2008) 559–583
 
It is clear from the data illustrated in table 9 , and table 10 that, except for learners fdl and ftm, the pre-pro-gram
mental model that users had of the atom had changed for all the learners after working with the program.
 
If one assumes that the mental models which scientists held of the atom had cognitively reached higher levels
through history, it can also be assumed that the mental models that the learners had of the atom had cognitively
reached higher levels through the interaction with the program. The program, of course, pre-sented the learning
material in a linear fashion, moving from Dalton's model to that of Schro¨dinger. Learn-ers who did not spend
enough time going through the last section, did not shift their mental models far enough.
 
Putting together what was learnt from the navigational pathways and the sketches, it becomes clear that the
program was good for the faster learners, but had limited value for the slow ones. Once again this would indi-cate
that a more traditional approach would be better for low achievers.
 
 
7. Conclusions and recommendations
 
Three questions were asked in this study: How did learners' mental models of educational multimedia pro-grams
di er from those of the designers? What could we learn from learners' time spent with the program and their
navigation through it? What visible growth could we see in terms of learners' mental models of the sub-ject matter?
We now present the conclusions and recommendations in the same sequence.
 
 
7.1. Alignment of mental models of designers and users
 
As has been shown, the user's ideal working mental model of a program is one that is consistent with the
conceptual model of the program developed by the designer. A user's accurate mental model shows a model
functional to the program. Weak or inaccurate mental models lack key components or features of the actual
program.
 
In order to evaluate the program, using the mental models users had of it, one should determine if users had
certain mental models that contained concepts that did not exist in the program, and further, if there were subsets of
the program of which users were unaware. Once these mental models were determined and described, suggestions
for program improvement could be made.
 
One of the aims of this research was to use the mental models developed by the users to refine and improve the
program. Although some of the recommendations below were initially peculiar to our program, we believe that they
may be generalised to open-ended learning environments generally.
 
Mental models need to be adjusted from both sides. Our analysis of the di erence of mental models of learners
and designers showed, for instance, that learners do not believe that the help function exists as an integral part of the
programme, while designers take it for granted that learners would use the help. In this case, designers need to take
note that there are two types of help, system help and learning material help, while they also need to realise that
learners may have to be taught and even prompted to access the help function. Many users do not view the help-
facility as being a part of the program. Although the help pull-down menu was always available, users did not focus
on the pull down menus, because they were busy with an interaction that took place somewhere else on the screen.
A solution could be to show a perpetual push-button, icon or hot spot on every screen, allowing the user free access
to the help sys-tem, or to allow a help prompt to appear when it becomes evident to the program that the user needs
help.
 
Games are a powerful instructional tool that can provide an environment that facilitates learning or the
acquisition of skills ( Alessi & Trollip, 2001 ). Because many users do associate a computer to a gaming machine,
extensive use of this mode could be made in future versions of the program. In teaching school sub-jects to
adolescents using a game metaphor may well enhance their motivation.
 
The existing log-on interface presented problems for users. Learners also had problems with fill-in answers.
 
Spelling checkers may go some way towards helping here.
 JC Cronje´, J. Fouche / Computers & Education 51 (2008) 559–583 579
 
In keeping with Johnson-Laird (1983) , we found many instances of incomplete or even false mental models.
Continuous formative evaluation is necessary to ensure that the language can be understood by the target population.
Here again the mental model of the designers need to be adjusted to ensure a higher level of user-centred design.
Some questions, both in the program and in future questionnaires, should be reworded to ask for information
precisely and directly. Many more questions could be built into the pro-gram. Automatic remediation paths could be
incorporated into the program to ensure that learning did take place.
 
The pre- and post-program questionnaires show that the program could be a motivational factor in the learning of
users. Owing to the small test group, further research is necessary to confirm this deduction. How-ever, many
learners had previously used a wide range of multimedia programs and expected extensive audio and video features
to be integrated into the program. Although extensive use has been made of graphics and animation in the present
version of the program, this aspect should not be neglected in future versions of the program.
 
Where possible, the user could be allowed to change the colour palette of certain screens to allow for indi-vidual
preferences. Further research to investigate the relationship between colour and successful learning should be
conducted.
 
7.2. Navigation and time
 
The navigational path records of the users revealed that maximum learner control was not advisable in a
multimedia tutorial. Navigational freedom should be allowed in relation to the computer skills and the exist-ing
knowledge of the user. Therefore, jumps to specific modules should not be allowed if the user does not possess the
knowledge base necessary to successfully complete the module.
 
Although the program successfully allowed users to work at their own pace, this prevented some users from
completing all the modules. We found three clear tendencies. At least one fast learner found the pro-gram too easy,
dipped in and out, and then, having realised that he knew all the work, terminated his use. The fast female learner,
however, worked systematically and methodically through the program although she knew all the work – thus e
ectively wasting her time. More than one slow learner got lost in the pro-gram and learnt noting – also a waste of
time. A tracking program could be incorporated within the pro-gram that records the progress of the users. This data
should be made accessible to the teacher. This implies that an administrative module should be added to the program
to monitor and manage the progress of every learner.
 
 
 
7.3. Acquisition of learning concepts such as ''atom''
 
Table 10 highlights the changes that occurred in the mental models that users held of the concept ''atom''.
Significant and fundamental changes did occur in the mental models users had of the atom before working with the
program, and the models users had of this concept after working with the program. Of particular interest here was
the value of using pre-and post-program sketches of the concept and comparing those.
 
 
 
7.4. Recommendations for further research
 
Over all our results presented and discussed here concur with those of Hennessy et al. (2007) who report ''a shift
away from the educational legacy of 'exemplary scientific practice' within the school curriculum as char-acterised by
real experiments'' (p. 149). We support their argument for the teacher's role ''in selecting appro-priate resources,
sequencing and structuring learning activities, adapting to particular learners' needs'' (2007, p. 149). We believe that
this ''selection, sequencing and structuring'' is the pre-requisite for learning in the inte-gration quadrant ( Cronje´,
2006 ) and we suggest that qualitative research into the mental models of learners would be a useful way for
teachers to determine those needs.
580 580 JC Cronje´, J. Fouche / Computers & Education 51 (2008) 559–583                

Appendix A. The pre-program questionnaire                  

Pre-program questionnaire
   
 
 
 
 
         
         
 
 
 
 

Name Nama Date of Birth                  

Male/female
Pria / wanita Previous computer experience?              
Very Very
    Sangat         Sangat  
strongly strongly
dengan dengan
    kuat       kuat
disagree
tidak
 
 
 
   
setuju
   
 
   
   
 
agree  
     

1 3 4 6 7 9
Interaction I would love it if the computer could speak back to me. 1 22 3 45 5 6 7 88 9
1 3 4 6 7 9
Interaction I always guess the answer for multiple choice questions. 1 22 3 45 5 6 7 88 9
1 3 4 6 7 9
Interaction I hate it if a teacher marks my problems wrong and writes a long 1 22 3 45 5 6 7 88 9
  story about it.                  
1 3 4 6 7 9
Interaction In class, the teacher provides me with answers, but I still do not 1 22 3 45 5 6 7 88 9
  understand. memahami.                  
1 3 4 6 7 9
Interaction I think computer based education programs are boring. 1 22 3 45 5 6 7 88 9
Interface 1 3 4 6 7 9
Antarmuka I love animations! 1 22 3 45 5 6 7 88 9
Interface 1 3 4 6 7 9
Antarmuka I consider myself as being ''artistically inclined''. 1 22 3 45 5 6 7 88 9
Interface 1 3 4 6 7 9
Antarmuka I always organise my learning: eg I will complete my 1 22 3 45 5 6 7 88 9
  Mathematics first, before starting my Science.                  
Interface 1 3 4 6 7 9
Antarmuka A red and orange screen would look nice. 1 22 3 45 5 6 7 88 9
Interface 1 3 4 6 7 9
Antarmuka I always use sketches when I learn. 1 22 3 45 5 6 7 88 9
Interface 1 3 4 6 7 9
Antarmuka Computers normally frustrate me. 1 22 3 45 5 6 7 88 9
Interface 1 3 4 6 7 9
Antarmuka When trying something, I give up easily if I do not succeed. 1 22 3 45 5 6 7 88 9
Interface 1 3 4 6 7 9
Antarmuka It is normally easy to work with a computer. 1 22 3 45 5 6 7 88 9
1 3 4 6 7 9
Involvement I prefer the computer based type of lesson to traditional 1 22 3 45 5 6 7 88 9
  instruction. petunjuk.                  
1 3 4 6 7 9
Involvement When starting a new year at school, I am concerned that I may 1 22 3 45 5 6 7 88 9
  not be able to cope with the work.                  
1 3 4 6 7 9
Involvement I hate science! 1 22 3 45 5 6 7 88 9
 1 3 4 6 7 9
Involvement The lessons in class are mostly dull. 1 22 3 45 5 6 7 88 9
Motivation 1 3 4 6 7 9
Motivasi I really think I would like to learn more about atoms and 1 22 3 45 5 6 7 88 9
  electricity. listrik.                  
Motivation 1 3 4 6 7 9
Motivasi I tense easily. 1 22 3 45 5 6 7 88 9
Motivation 1 3 4 6 7 9
Motivasi I think a computer program will make the normal classwork 1 22 3 45 5 6 7 88 9
  much easier.                  
Motivation 1 3 4 6 7 9
Motivasi I think extra classes are mostly a waste of time. 1 22 3 45 5 6 7 88 9
Motivation 1 3 4 6 7 9
Motivasi I am always totally involved in class. 1 22 3 45 5 6 7 88 9
Motivation 1 3 4 6 7 9
Motivasi My teachers always challenge me to do my best.. 1 22 3 45 5 6 7 88 9
1 3 4 6 7 9
Rate Menilai I learn best when I feel I am pressed for time. 1 22 3 45 5 6 7 88 9
1 3 4 6 7 9
Rate Menilai I love to work at my own pace, without being pushed. 1 22 3 45 5 6 7 88 9
1 3 4 6 7 9
Rate Menilai Sometimes the teacher goes too slow because he keeps on 1 22 3 45 5 6 7 88 9
  explaining things to the rest of the class.                  
1 3 4 6 7 9
Rate Menilai I consider myself to be a patient person.
     
1 22
   
3 
45
 
5 6 7
     
88 9
 
  JC Cronje´, J. Fouche / Computers & Education 51 (2008) 559–583               581

Appendix B. Post-program questionnaire                  

Post-program questionnaire
   
 
 
 
 
         
         
   
   

Name Nama Date of Birth                  

Male/female
Pria / wanita Previous computer experience?              
Very
Very Sanga
    Sangat         t  
strongly strongly
dengan dengan
    kuat       kuat
disagree
tidak
 
 
 
   
setuju
   
 
 
 
 
 
   
agree  
   

1 3 4 6 9
Interaction I felt as if someone was engaged in conversation with me. 1 22 3 45 5 67 78 8 9
1 3 4 6 9
Interaction I guessed the answers to some questions. 1 22 3 45 5 67 78 8 9
1 3 4 6 9
Interaction I was encouraged by the responses given to my answers of 1 22 3 45 5 67 78 8 9
  questions. pertanyaan.                  
1 3 4 6 9
Interaction I was given answers, but still do not understand the questions. 1 22 3 45 5 67 78 8 9
1 3 4 6 9
Interaction The feedback was boring. 1 22 3 45 5 67 78 8 9
Interface 1 3 4 6 9
Antarmuka I did not like the animations in the program. 1 22 3 45 5 67 78 8 9
Interface 1 3 4 6 9
Antarmuka I did not like the screen layout at all. 1 22 3 45 5 67 78 8 9
Interface 1 3 4 6 9
Antarmuka I like the fact that I can jump from one topic directly to another 1 22 3 45 5 67 78 8 9
  topic.                  
Interface 1 3 4 6 9
Antarmuka I loved the colours. 1 22 3 45 5 67 78 8 9
Interface 1 3 4 6 9
Antarmuka The animations in the program made the contents easy to 1 22 3 45 5 67 78 8 9
  understand. memahami.                  
Interface 1 3 4 6 9
Antarmuka The program frustrated me. 1 22 3 45 5 67 78 8 9
Interface 1 3 4 6 9
Antarmuka Sometimes I felt completely lost 1 22 3 45 5 67 78 8 9
Interface 1 3 4 6 9
Antarmuka The program is very easy to work with. 1 22 3 45 5 67 78 8 9
1 3 4 6 9
Involvement I prefer the computer based type of lesson to traditional 1 22 3 45 5 67 78 8 9
  instruction. petunjuk.                  
1 3 4 6 9
Involvement I was concerned that I might not be able to understand the 1 22 3 45 5 67 78 8 9
  material. bahan.                  
1 3 4 6 9
Involvement My feeling towards the course material after I had completed 1 22 3 45 5 67 78 8 9
  the program was favourable.                  
1 3 4 6 9
Involvement The lessons in the program were dull and di cult to follow. 1 22 3 45 5 67 78 8 9
Motivation 1 3 4 6 9
Motivasi As a result of having studied by this method, I am interested in 1 22 3 45 5 67 78 8 9
  learning more about the subject matter.                  
Motivation 1 3 4 6 9
Motivasi I felt quite tense when I worked through the program 1 22 3 45 5 67 78 8 9
Motivation 1 3 4 6 9
Motivasi I think that what I have learned from the program, should 1 22 3 45 5 67 78 8 9
  make the normal classroom and laboratory work easier to                  
  understand. memahami.                  
Motivation 1 3 4 6 9
Motivasi I think working through the program was a waste of time. 1 22 3 45 5 67 78 8 9
Motivation 1 3 4 6 9
Motivasi The lessons were interesting and really kept me involved. 1 22 3 45 5 67 78 8 9
Motivation 1 3 4 6 9
Motivasi The program challenged me to try my best. 1 22 3 45 5 67 78 8 9
1 3 4 6 9
Rate Menilai I could have learned more if I had not felt pushed. 1 22 3 45 5 67 78 8 9
1 3 4 6 9
Rate Menilai I felt that I could work at my own pace. 1 22 3 45 5 67 78 8 9
1 3 4 6 9
Rate Menilai The course material was presented too slowly. 1 22 3 45 5 67 78 8 9
Rate 1 3 4 6 9
Menilai
   
The program ran much too fast.  
1 2 2 3 45 5 6 7 7 8 8 9
               
 582              JC Cronje´, J. Fouche / Computers & Education 51 (2008) 559–583
 
References Referensi
 
Ackermann, D., & Tauber, MJ (1990). Mental Models and Human–computer Interaction 1. Human Factors in Information Technology (vol. 3).
Amsterdam: Elsevier.
 
Alessi, SM, & Trollip, SR (2001). Multimedia for learning. Third ed.
 
Alexander, S. & Hedberg, JG (1994). Evaluating technology-based learning: Which model? In Proceedings of the IFIP TC3/WG3.2 working
conference on the Seign, implementation and evaluation of interactive multimedia in university settings: Designing for change in teaching and
learning, July 06–08 (pp. 233–244).
Authorware Professional . <http://www.macromedia.com> .
 
Cronje´, JC (2006). Paradigms regained: Toward integrating objectivism and constructivism in instructional design and the learning sciences.
Educational Technology Research and Development, 54(4), 387–416.
 
Ball, LJ, Phillips, P., Wade, CN, & Quayle, JD (2006). E ects of belief and logic on syllogistic reasoning: Eye-movement evidence for selective
processing models. Experimental Psychology, 53(1), 77–86.
 
Bauer, MI, & Johnson-Laird, PN (1993). How diagrams can improve reasoning. Psychological Science, 4(6), 372–378.
 
Borton, W., & Rossett, A. (1989). Educational Software And Published Reviews: Congruence of Teacher. Developer and Evaluator Perceptions.
Education, 109(4), 434–444.
 
Bucciarelli, M. (2007). How the construction of mental models improves learning. Mind and Society, 6(1), 67–89.
 
Burger, M. (1991). Kriteria en prosedures vir die evaluering van opvoedkundige rekenaarprogramme. Informedia, Mei, 4–6.
 
Carley, K., & Palmquist, M. (1992). Extracting, representing and analyzing mental models. Social Forces, 70(3), 601–636.
 
Carney, RN, & Levin, JR (2002). Pictorial illustrations still improve students' learning from text. Educational Psychological Review, 14(1), 5–26.
ChemLab . <http://modelscience.com> .
 
Clark, RE (1991). When researchers swim upstream: Reflections on an unpopular argument about learning from media. Educational Technology,
31(2).
 
Clark, RE (1994). Media Will Never Influence Learning. Educational Technology Research and Development, 42(2), 21–29.
 
Coll, RK, & Treagust, DF (2001). Learners' Mental Models of Chemical Bonding Research in Science. Education, 31, 357–382.
 
Craik, K. (1943). The Nature of Explanation. Cambridge: Cambridge University Press. Cambridge: Cambridge University Press.
 
de Kleer, J., & Brown, JS (1985). A qualitative physics based on confluences. In DG Bobrow (Ed.), Qualitative reasoning about physical systems
(pp. 7–83). Cambridge, MA: MIT Press.
 
Duit, R., & Treagust, DF (2003). Conceptual change: A powerful framework for improving science teaching and learning. International Journal of
Science Education, 25, 671–688.
 
Ferguson, RW, & Forbus, KD (2002). A cognitive approach to sketch understanding. In Proceedings of IAAI-2002.
 
Fisher, G. (1991). The importance of models in making complex systems comprehensible. In MJ Tauber & D. Ackermannn (Eds.).
 
Mental models and human–computer interaction 2. Human factors in information technology (vol. 7). Amsterdam: Elsevier.
Geometer's Sketchpad . <www.dynamicgeometry.com> .
 
Gentner, D., & Stevens, AL (1983). Mental Models. Hillsdale, NJ: Lawrence Erlbaum. Hillsdale, NJ: Lawrence Erlbaum.
 
Greca, I., & Moreira, M. (2000). Mental models, conceptual models, and modelling. International Journal of Science Education, 22(1), 1–11.
 
Hennessy, S., Wishart, J., Whitelock, D., Deaney, R., Brawn, R., la Velle, L., et al. (2007). (2007). Pedagogical approaches for technology-integrated
science teaching. Computers and Education, 48, 137–152.
Interactive Physics . <http://www.design-simulation.com/> .
 
Janniro, MJ (1993). E ects of computer-based instruction on student learning of psycho physiological detection of deception test question
formulation. Journal of Computer-Based Instruction, 20(2), 58–62.
 
Jih, HJ (1991). The relationship among the structure of interfaces, users' mental models, and performance in computer-based interactive courseware.
Unpublished doctoral dissertation, The University of Georgia.
 
Jih, HJ, & Reeves, TC (1992). Mental models: A research focus on interactive learning systems. Educational Technology Research and
Development, 40(3), 39–53.
 
Johnson-Laird, PN (1983). Mental models: Toward a cognitive science of language, influence, and consciousness. Cambridge, England:
 
Cambridge University Press. Cambridge University Press.
 
Johnson, TE, O'Connor, DL, Pirnay-Drummer, PN, Ifenthaler, D., Spector, JM, & Seel, N. (2006). Comparative study of mental model research
methods: Relationships Among Acsmm, Smd, Mitocar & deep methodologies. In AJ Can˜as, & JD Novak (Eds.), Proceedings of the Second
International Conference on Concept Mapping, San Jose´, Costa Rica. Available from: <http://cmc.ihmc.us/ cmc2006Papers/cmc2006-p177.pdf>
.
 
Jolicoeur, K., & Berger, DE (1988). Implementing educational software and evaluating its academic e ectiveness: Part II. Education Technology,
28(10), 13–19.
 
Kyllonen, PC, & Shute, VJ (1989). A Taxonomy of Learning Skills. In PL Ackerman, RH Sternberg, & R. Glaser (Eds.), Learning and individual di
erences: Advances in theory and research. New York: Freeman.
 
Lanza, A., & Roselli, T. (1991). E ects of the hypertexual approach versus the structured approach on students' achievement. Journal of Computer-
Based Instruction(18), 48–50.
 
Lucas, EJ, & Ball, LJ (2005). Think-aloud protocols and the selection task: Evidence for relevance e ects and rationalization processes.
 
Thinking and Reasoning, 11(1), 35–66.
 
Malamed, C. (1991). CBT Meets the Magic Kingdom. CBT Directions, September, 10–15.
 JC Cronje´, J. Fouche / Computers & Education 51 (2008) 559–583 583
 
Mavers, D., Somekh, B., & Restorick, J. (2002). Interpreting the externalised images of pupils' conceptions of ICT: Methods for the analysis of
concept maps. Computers and Education, 38(3), 187–207.
 
Mayer, RE (1989). Models for Understanding. Review of Educational Research, 59(1), 43–46.
 
Mayer, RE, & Moreno, R. (2003). Nine ways to reduce cognitive load in multimedia learning. Educational Psychologist, 38(1), 43–52. Merrill,
MD (2005). Hypothesized performance on complex tasks as a function of scaled instructional strategies. <http://it.coe.uga.edu/
 
itforum/paper84/DrMerrillPDF.pdf> Retrieved 21.10.06.
 
Merrill, MD, Li, S., & Jones, MK (1990). Second Generation Instructional Design (ID2). Educational Technology, 30(2), 7–14.
 
Mouton, J., & Marais, HC (1990). Basiese begrippe: Metodologie van die geesteswetenskappe. Pretoria: RGN.
 
Norman, DA (1983). Some Observations on Mental Models. In D. Gentner & AL Stevens (Eds.), Mental models. Hillsdale, NJ: Hillsdale, NJ:
 
Lawrence Erlbaum. Lawrence Erlbaum.
 
Owston, RD, & Dudley-Marling, C. (1988). A criterion-based approach to software evaluation. Journal of Research in Computing in Education,
20(3).
 
Pol, H., Harskamp, E., & Suhre, C. (2005). Solving physics problems with the help of computers-assisted instruction. International Journal of
Science Education, 27, 51–469.
 
Preece, J., & Jones, A. (1985). Training teachers to select educational computer software: Results of a formative evaluation of an open university
pack. British Journal of Educational Technology, 16(1).
 
Reeves, TC (1992). Computer modelling: A research tool for computer-based instruction. Interactive Learning International, 8, 3–13.
 
Reeves, TC (1993). Research support for interactive multimedia: Existing foundations and new directions. In C. Latchem, J. Williamson,
 
& Henderson-Lancett (Eds.), Interactive multimedia: Practice and promise. London: Kogan Page. London: Kogan Page.
 
Russell, TL (1999). The no significant di erence phenomenon as reported in 355 Research Reports, summaries and papers: A comparative research
annotated bibliography on technology for distance education, North Carolina State University, Raleigh.
 
Rouse, WB, & Morris, NM (1985). On looking into the black box: Prospects and limits in the search for mental models. Psychological Bulletin, 100,
349–363.
 
Smith, PL, & Ragan, TJ (1999). Instructional design. Hoboken, NJ: Wiley. Hoboken, NJ: Wiley.
 
Sasse, Martina-Angela (1991). How to T(r)ap Users' Mental Models. In MJ Tauber & D. Ackermann (Eds.). Mental models and human– computer
interaction 2. Human factors in information technology (vol. 7). Amsterdam: Elsevier.
 
Van der Veer, GC (1989). Individual Di erences and the User Interface. Ergonomics, 32(11), 1431–1449.
 
Waern, Y. (1990). On the Dynamics of Mental Models. In D. Ackermann & MJ Tauber (Eds.). Mental models and human–computer interaction 1.
Human factors in information technology (vol. 3). Amsterdam: Elsevier.
 
Zahner, JE, Reiser, RA, Dick, W., & Gill, B. (1992). Evaluating instructional software: A simplified model. Educational Technology, 22(3), 39–44.
 
Johannes Cronje´ is the Dean of the Faculty of Informatics and Design at Cape Peninsula University of Technology in Cape Town, South Africa.
He obtained a Doctorate in South African Literature in 1990 and a Masters' Degree in Computer-Assited Education in 1994. He started his career
as schoolmaster teaching English and then as lecturer teaching communication in English at a Technikon before being appointed associate
professor in computer-assisted education at the University of Pretoria. Following this he was appointed Professor of Internet Studies in the School
of Information Technology at the University and then Professor of Instructional Technology in the Faculty of Education.
 
 
Johann Fouche is a consultant in the field of e-learning and owner of JFMultimedia. He holds a Masters' Degree in Computer-Integrated
Education and specializes in developing educational multimedia and teaching instructional design. The program reported on here was designed
and evaluated as part of his Masters studies at the University of Pretoria.