Anda di halaman 1dari 11

Journal of Marketing Education

Volume 31 Number 2
August 2009 120-130
2009 Sage Publications

Viva Voce (Oral Examination) 10.1177/0273475309334050


http://jmd.sagepub.com
hosted at
as an Assessment Method http://online.sagepub.com

Insights From Marketing Students


Glenn Pearce
Geoffrey Lee
University of Western Sydney, Australia

Viva voce (viva) or oral examinations are widely used in medical education, clinical examinations, and doctoral defenses, yet the
assessment method is seldom adopted by university marketing departments. Correspondingly, the marketing education literature
makes no reference to vivas as an alternative academic assessment technique. This research discusses the use of viva as a sum-
mative assessment method in university marketing education. Final-year marketing students undertaking an elective unit in ser-
vices marketing were given an end-of-unit viva instead of a written examination. Fifty-four cartoon completion tests were
analyzed to gain insight into student perceptions of the viva assessment task. Interpretive findings from this study suggest that
viva is a valid and novel method of assessing learning outcomes such as application of deep learning, application of theory to
practice, and problem-solving skills. The vivas enabled dialectic communication between the examiner and student and provided
invaluable experience for career interviews. Although some students were anxious prior to the viva examination, on reflection they
conceded that the process was user friendly. Based on these findings, implications for the adoption of vivas as an assessment
method are discussed.

Keywords: marketing education; viva voce; assessment; technique

D erived from Medieval Latin, the term viva voce (viva)


was recorded in The Shorter Oxford English Dictionary
in 1815 as an examination conducted by speech (Douglas
assessment was defined by Joughin (1998, p. 367) as
assessment in which a students response to the assess-
ment task is verbal, in the sense of being expressed or con-
& Swift, 1997). Vivas have been of interest to education veyed by speech instead of writing.
researchers since the early 20th century. Pressey, Pressey, Among its multiplicity of applications, vivas have been
and Barnes (1932) questioned the underlying assumption used to assess such programs as doctoral degrees (Douglas
that written exams comprehensively test a candidates edu- & Swift, 1997; Jackson & Tinkler, 2001; Morley, Leonard,
cation learning outcomes and requisite professional life & David, 2002), clinical dentistry (Yip, Smales, Newsome,
skills, and suggested integrating systematic and objective Chu, & Chow, 2001) medical studies (Bashook, 2000), and
assessment tools by including both written exams and vivas. various university degrees including natural sciences, engi-
Similarly, todays educators are concerned that assessment is neering, and the social sciences (Kehm, 2001).
often inadequately conceived (Challis, 2008). Assessment
methods should be an integral part of the constructive
alignment of curricular design to ensure complementary Literature Review
learning outcomes, teaching and learning approaches, and
evaluation methods (Biggs, 1999; Chadwick, 2004). Business schools have been criticized for producing
Understanding the role, value, and application of vivas will graduates without effective problem-solving and communi-
assist educators in designing reliable and valid assessment cation (written and oral) skills (Lunsford, 1991). Some
policy and practice (Jackson & Tinkler, 2001). suggest that traditional marketing curricula focus on knowl-
Although the viva voce method of assessment has been edge acquisition, with less emphasis on skill development
called by various namesviva, oral(s), oral test, oral inter- (Lamb, Shannon, & Moncrief, 1995). However, marketing
view, and oral examinationthe term viva is used in this educators are charged with the responsibility of providing
article. Irrespective of nomenclature, the viva method of students with concepts and principles, tools, and experiences

120
Lee, Pearce / Viva Voce as an Assessment Method 121

in the practice of marketing to prepare students to enter the problem-based learning (PBL) provide the theoretical
workforce (Floyd & Gordon, 1998; Haffer & Hoth, 1981; underpinning for using vivas as an assessment tool. PBL
Lamb et al., 1995; Morrison, Sweeney, & Heffernan, starts with the premise of a problem, query, or puzzle to
2003). Floyd and Gordon (1998) and Morrison, Sweeney, & be solved and assists student learning by integrating theory
Heffernan (2006) purport that educators are responsible for and practice. Prior to solving the problem, students need
developing well-rounded graduates who fulfill employers to acquire knowledge and professional and academic skills,
demands regarding knowledge and skills. and understand client requirements and contextual factors
Employers seek graduates with more than technical of the situation (Pinto Pereira, Telang, Butler, & Joseph,
skills and knowledge; graduates need effective communica- 1993). Furthermore, Borin, Metcalf, and Tietje (2008)
tion skills (Crosling, 2000; R. Davis, Misra, & van Auken, suggest PBL builds on coursework, such that students are
2002; Haffer & Hoth, 1981), leadership, team-building and exposed to increasingly sophisticated and complex issues.
decision-making skills (Lamb et al., 1995), and an ability to The policy and practice of assessing student-learning
cope with change (Morrison et al., 2003). Thus, it is argued outcomes in business schools is of increasing interest in
that the challenge for marketing educators is how to most higher education (Borin et al., 2008). It is suggested that
effectively teach students the desired skills, knowledge, and assessment drives learning for many students (Boud, Cohen,
competencies required for career-ready graduates who & Samson, 1999; Wass, Van der Vleuten, Shatzer, & Jones,
match employers needs (Haffer & Hoth, 1981; Lamb et al., 2001), and so there is a need to align student-learning objec-
1995). tives, content, learning pedagogy, and assessment techniques.
Oral communication skills are often cited by Western Direct, embedded assessment techniques such as assign-
universities as attributes that graduates should possess. ments, exams, and presentations are considered the stron-
These skills are recognized as important because they gest measures of learning outcomes. A critical component
underpin a students development of social networks, of outcome-based education is students ability to demon-
they are positively linked to academic achievement, and strate their success in mastery of content, concepts, and
they enhance the ability of the student to think on their feet, skills by focusing on what they have learned, rather than on
express themselves meaningfully, and relate to others, thus what they have been taught (Borin et al., 2008).
improving their future work opportunities and life chances Blooms hierarchal six-level taxonomy (Bloom,
(Crosling, 2000). Crosling (2000) found that many univer- Englehart, Furst, Hill, & Krathwohl, 1956) can be used as
sities were sporadic and unsystematic in their approach to a framework to understand the ability of students to demon-
the developmental curriculum of undergraduate oral com- strate their mastery of the subject through the assessment of
munication skills as students progress through their higher order skills. Traditional marketing education has
degree. been criticized for its focus on lower order skills such as the
The literature is divided on how students individual dif- retention of knowledge and understanding of content
ferences and preferences in student-learning styles can influ- (Lamb et al., 1995), often referred to as surface learning
ence learning outcomes and the overall effectiveness of (Sayce, 2007). Graduate attributes require higher order
teaching (see Karns, 2006; Kolb, 1984; Morrison et al., 2003, skills of analysis, synthesis, and evaluation that take exist-
2006; Solomon & Felder, 2004). Morrison et al. (2003) ing knowledge and apply it to new situations not previously
recommend targeting teaching styles to learner preferences to encounteredthat is, the ability to apply critical thinking
enhance students retention of information, improve teach- skills (Ponder et al., 2004). Higher order skills are regarded
ing efficiency, and facilitate students positive attitude to as more reflective of a deeper approach to learning, in
the subject. Others suggest that marketing educators should which the learner takes control (Sayce, 2007).
stretch students beyond their preferred learning style Vivas provide the opportunity to examine a students
(Frontczak, 1990). Karns (2006) recommends employing a knowledge and understanding of the subject matter, applied
variety of learning activities to encompass multiple learn- problem-solving ability, and interpersonal competence and
ing modalities, with only modest tailoring of course design qualities (Joughin, 1998). Despite a long history of oral
to meet individual learning style preferences, a strategy that examination techniques, educators use a range of tech-
is most effective for active-oriented learners, such as mar- niques to assess a students competencies, with written
keting students. Similarly, Ponder, Beatty and Foxx (2004) assessment methods dominating higher education institu-
recommend that a range of teaching activities and assess- tions (Joughin, 1998). Chadwick (2004) asserts that multiple-
ment methods be incorporated into marketing course design. choice short-answer questions and essays test knowledge,
Active learning pedagogies are effective in matching and constructed response questions assess application of
learning activities with student-learning styles (Karns, knowledge, whereas student portfolios, clinical examina-
2006). Experiential learning (see Jarvis, 1987; Kolb, 1984) tions, and vivas assess learning outcomes not easily assessed
as an active learning pedagogy and, more specifically, by other methods. Vivas also have the potential to measure
122 Journal of Marketing Education

the students achievement in course outcomes not restricted potential challenges when using vivas as sociophysiological
to knowledge, but related to professional tasks and the indi- (unequal power distance between the candidate and the
viduals professionalism (Harden, Crosby, & Davis, 1999). assessors), psychoanalytical (ritual aspects of the process),
Kehm (2001) argues that oral examinations provide evidence sociological (differences in cultural and social norms
of a students ability for reflective and critical thinking, and affecting oral performance), and methodological issues of
vivas serve as a complementary part of a total examination measurement (low degree of objectivity, reliability, and
regime. Examples of disciplines using vivas to assess appli- validity). Vivas reliability and validity are affected by many
cation of knowledge in undergraduate degrees include factors, including examiner judgments, cases presented, the
clinical medicine (Paul, 1994), sports (Oakley & Hencken, students nervousness, and examination conditions (Wass et al.,
2005), and physiotherapy (Kuisma, 2007). Clinical medical 2001). Issues of bias and poor interexaminer reliability
faculties have used vivas in objective structured clinical were cited by Birley (2001) as reason for the method to be
examinations, to provide objective, formative, and summa- de-emphasized in British pharmacy education. M. H. Davis
tive assessment that examines factual recall, applied knowl- and Karunathilake (2005) and Ebel (1972) echoed concerns
edge, and communication skills (Wass et al., 2001). Vivas have for vivas low reliability and validity, citing the additional
been used in sport degrees to examine learning outcomes from problems of cost in professional time and energy. Paul (1994)
undergraduate units, where students answered six randomized claimed vivas revolve around theory and are subjective,
questions relating to the learning objectives during a 30-min with Park (2003) identifying problems associated with the
assessment period. In physiotherapy, vivas have been used to socially constructed nature of the assessment technique.
delineate an individuals contribution to group projects that Wakeford, Southgate, and Wass (1995) warned that
form part of his or her overall mark (Kuisma, 2007). examiners of vivas in medicine must be carefully selected,
The use of the viva is defended on the grounds of giv- trained, and monitored to avoid allocating marks that have
ing students the opportunity to show their strengths and pro- no bearing on competence. Schwarzer and Buchwald
vide the examiners with as clear and complete an impression (2000) argued that the student should trust the examiner,
as possible of candidates knowledge and abilities (Kehm, otherwise defensive or aggressive behaviors can emerge.
2001, p. 26). Examination through viva enables interactive Arndt, Guly, and McManus (1986) and Sayce (2007) noted
dialogue between candidate and assessor, allowing the exam- the stress associated with a viva; however, all exams are stress-
inee to demonstrate strengths and letting examiners discrimi- ful and there is no evidence to suggest that vivas are more
nate between superficial and real knowledge via in-depth stressful than other assessment techniques (M. H. Davis &
questioning. Karunathilake, 2005).
Vivas are often used as an assessment component of a Recommended preparatory techniques such as guidance
Doctor of Philosophy defense (see Jackson & Tinkler, 2001; from the supervisor, clearly defined guidelines, and mock
Morley, Leonard, & David, 2003; Ponder et al., 2004; Tinkler vivas can reduce students stress levels (Tinkler & Jackson,
& Jackson, 2002; Trafford, 2003), demonstrating that the 2002). Although 60% of surveyed students described their
material is the candidates own work through revealing mas- experience defending their PhD dissertation through vivas
tery of the material and its literature (Douglas & Swift, 1997). as positive, 20% described their experience as hostile, sar-
Similarly, Jackson and Tinklers (2001) study of PhD asses- castic, and insulting (Jackson & Tinkler, 2001), findings
sors identified the importance of vivas to check candidates similar to Parks (2003) review of the literature on students
understanding and ability to produce research, clarifying areas perceptions of their viva experience. Tinkler and Jackson
of weakness and testing their knowledge of the literature and (2002) and Park (2003) highlighted the need for educators
their positioning within the literature. It can be argued that to maintain a vivas consistency, openness and fairness, and
vivas employ an adaptive assessment methodology (a concept reliability and validity to avoid problematic situations.
used in computer-assisted adaptive testing; see Challis, 2008) Despite concerns about the subjectivity of vivas (Birley,
that enables educators to differentiate higher order cognitive 2001; M. H. Davis & Karunathilake, 2005), a longitudinal
skills and probe a students knowledge, understanding, and study of oral practice examination within medical programs
application of complex and abstract concepts (Ponder et al., revealed substantial internal consistency and reliability of
2004) for decision making and problem solving (Wass et al., vivas, identifying a positive correlation to in-training exam-
2001). Joughin (1998) pointed out that viva examinations ination scores and faculty evaluation scores (Schubert,
enable two qualities to be measured: student command of the Tetzlaff, Tan, Ryckman, & Mascha, 1999). Other studies in
oral medium, and a students command of content. Some sug- different disciplines report similar findings (e.g., Allison &
gest that vivas are the best way to assess recall and synthesis Katona, 1992; Levine & McGuire, 1970; Oakley & Hencken,
of information (Wass et al., 2001). 2005; Wakeford et al., 1995), providing the viva is well
No one assessment technique is ideal and viva has its structured, the assessment criteria are clearly defined, and
critics. Kehms (2001) review of the literature highlights examiners are carefully selected, trained, and monitored.
Lee, Pearce / Viva Voce as an Assessment Method 123

Indeed, the use of mini-vivas (Carless, 2002) as a form of to develop students into more complete marketers capable
summative assessment in assignments is reported to have of operating in service marketing environments; and to
improved student learning through providing timely feed- develop generic competencies in students such as learning
back for students. to learn, problem solving, creativity, and public speaking
The literature suggests quality assurance, standards, bench- through a process of peer tutoring. Independent learning
marks, and performance indicators need to be established was identified as a key feature of the subject.
for effective vivas (Morley et al., 2003), with M. H. Davis Assessment tasks and weighting for the unit were:
and Karunathilake (2005) recommending six techniques to Lucky Draw Services Marketing Topic Seminar (30%),
increase reliability: use several vivas, use several examiners, Services Reflective Diary (30%), and The Service Interview
ask candidates the same questions, use descriptors, employ Viva Voce (Oral Exam; 40%). The Lucky Draw seminar
rubrics and criteria for answers, and train examiners. A component required student pairs to randomly select a ser-
meta-analysis of teacher perspectives on oral assessment in vice marketing topic (following the chapters in the assigned
higher education (Joughin, 1998) identified six dimensions textbook) as the basis of a 65-min interactive seminar
of oral assessment: primary content type (objectives), inter- where the team taught the class through peer tutoring that
action (reciprocity), authenticity (simulation), structure incorporated the textbook with published research and real-
(agenda), examiners (audience), and orality (tasks). These world applications. The Services Reflective Diary required
dimensions guided the design and implementation of the individuals to reflect each week on their perceptions of
viva for the marketing students in this study. Two assess- their learning outcomes, their experiences with services,
ment frameworks also underpinned the study of the viva and how service marketing concepts are operationalized in
examination reported here. The first is Biggss (1999) industry. Related to this assessment mix, seven stated student-
notion of constructive alignment, in which learning out- learning objectives were: (1) be more aware of key services
comes (relating to graduate attributes), teaching activities, marketing concepts and developments, (2) be better informed
and assessment methods are synchronized. The second is about/possess knowledge on a special services topic, (3)
Rowntrees (1994) five dimensions of assessment, relating have acquired skills as autonomous learners, (4) possess
to mental activity undertaken by assessment designers, advanced communication competencies, (5) have had the
which correspond to key activities in the process of assess- opportunity to be orally examined as distinct from being
ment (p. 11): (1) Why assess? (2) What to assess? (3) How examined using written means, (6) be aware of what it takes
to assess? (4) How to interpret? and (5) How to respond? In to perform in emerging service marketing areas, and (7)
a review of the literature, Park (2003) develops a frame- have developed skills as reflective practitioners. Although
work for best practice in doctoral assessment using vivas, learning outcome (5) related directly to the viva task, it
which includes definition and status of a viva, role of the could be argued that the other objectives were also relevant.
participants, student preparation, membership of the review Learning objectives aside, the viva was implemented to
panel, record keeping, timing and arrangements, examina- expose students to the assessment method as part of their
tion guidelines, and strategies for conducting the viva. undergraduate learning experience.
Despite extensive debate on the use of vivas in medical
education and doctoral defense, relatively few business
How the Viva Was Conducted
studies on vivas, apart from some research on student anxi-
ety, reflect the students perspective. Moreover, there Services classes in the last 2 weeks of semester (weeks
appears to be a paucity of accounts on the use of viva 12 and 13) were allocated to in-class viva preparation,
examinations in marketing education literature. This study rehearsal, and revision. Vivas were conducted in the study
addresses these two gaps in the literature. week prior to the universitys 2-week formal exam period,
and were scheduled as half-hour appointments across 4 con-
Method secutive days with up to 15 vivas per day. Students nominated
their preferred appointment time between 9:00 a.m. and
5:00 p.m. So all vivas could incorporate a uniform set of
Subject Context
questions, students were asked to sign a confidentiality
At the time the study was conducted, Services Marketing agreement forbidding them from disclosing information
was a final-year marketing elective offered in a Bachelor of about their viva or the process. Any students uncomfortable
Business (Marketing) program. The subject was taught in a with this agreement were given the option of being inter-
3-hr workshop format with a curricular orientation toward viewed using an alternative viva format. No students elected
service gap analysis rather than a 4Ps (i.e., product, pricing, such an option.
placement, and promotion) approach. The course had sev- Two panelists assessed the vivas: the teacher/principal
eral aims: to expose students to relevant theory and practice; researcher and a consultant from industry who was familiar
124 Journal of Marketing Education

with service marketing concepts and practice and who was Figure 1
employed to assist with viva assessment. The teacher/ Cartoon Completion Test
principal researcher has more than 20 years experience in
teaching undergraduate and postgraduate marketing in
higher education, has received numerous awards for teach-
ing proficiency, and has extensive publications in marketing
education. The consultant, with 15 years of industry experi-
ence, was briefed prior to the vivas to ensure awareness of
the vivas procedure and assessment criteria, and to answer
any questions. The assessment items were derived from the
units espoused student-learning outcomes, which, in turn,
align to graduate attributes. Each students 20-min viva per-
formance was assessed by the panelists using a rating sheet
comprising a Likert-type scale from 1 (very poor) to 5 (very
good) across 12 criteria: appearance, knowledge of subject,
confidence, conciseness of responses, quality of responses,
thinking on the spot, communication skills, application of
theory to practice, ability to handle questions, body language, conducting marketing research (see Donoghue, 2000) and
professional manner, and clarity of responses. Panelists rat- have been successfully deployed in educational studies (see
ings of each student were summated to give a score out of Catterall & Ibbotson, 2000; Lee & Holland, 2008; Pearce
60, which was then converted to a percentage contribu- & Lee, 2006, 2007). Projective testing was considered an
tion reflecting his or her performance for this assessment appropriate method to elicit responses that gave students
component. the freedom to express their perceptions and thoughts with-
On arriving for the interview, the student was seated and out the potential biases associated with more structured
informed about the process, particularly that the viva was methods, such as student response to a series of statements
trying to determine how much they knew rather than what or sentence completion.
they didnt know. The student then selected 1 of 14 enve- Participation in the projective test research was both
lopes containing four questions relating to chapter topics voluntary and anonymous. Without any forewarning about
such as Service Delivery, Building Customer Relationships, the line of enquiry, students were instructed by the lec-
and so forth. Students were given up to 6 min to think over turer (also the researcher) to project themselves into a
their questions before answering. The student then had to cartoon and complete the empty bubble, in response to the
interpret a photographic exhibit using knowledge and skills following statement: A viva is used as an assessment
gained in the subject. The photographic exhibit depicted a method in services marketing . . . (see Figure 1). The
salesperson serving a customer in a fast-food restaurant research objectives in this exercise were twofold: to deter-
environment. Before leaving, students were asked some mine what students studying the services unit would say
general questions including what they took away from the about the viva experience; and to ascertain the contribu-
subject, how they had developed, what grade they expected tion, if any, of vivas as a method for assessing student
from the viva, and how they thought they had performed in learning in the unit.
the viva. Finally, they were reminded about the confidenti- Students were encouraged to be frank and were given as
ality clause they had signed and were bid farewell. much time as they needed to complete the cartoon test. Fifty-
four responses were collected across the two classes, with all
but three students participating in the study. Written responses
Projective Test were transcribed into a Word document for cross-sectional
Immediately after the viva, students were ushered into indexing and thematic labeling. Data were analyzed using
another area to complete a cartoon projective test. Projective QSR NVivo according to recommendations by Richards
tests generate valuable insights and understanding that can- (1999) and Bazeley and Richards (2000). QSR NVivo takes
not be obtained by more direct methods (Malhotra, Hall, a formal approach to qualitative data analysis (Hussey &
Shaw, & Oppenheim, 2002), through the participants Hussey, 1997) and is suitable for making sense of a mass
projecting their personality, opinions, and self-concept to of open-ended material (Mostyn, 1985). Emergent themes
give the situation structure (Donoghue, 2000). The projec- were identified, using a grounded theory approach (Glaser
tive tests used a cartoon drawing for which the respondents & Strauss, 1967) because the co-researcher was unfamiliar
suggest a dialogue between two characters (Zikmund, with viva methodology and had not been exposed to the data
2003). Cartoon tests are commonly used by practitioners set or the objectives of the research prior to undertaking data
Lee, Pearce / Viva Voce as an Assessment Method 125

Table 1
Self-Reported Attribute Identification
Construct Identified From Attribute Identified From Frequency of
Response Themes NVivo Coding Construct NVivo Coding Reporting Attribute

Positive Total positive responses See below 47a


Assessment Total positive assessment comments 42a
More effective and efficient assessment method 28a
Dialectic communication process 23a
Novel assessment method 21a
Encourages self-reflection 9c
Real-life application 6c
Communication NA 31a
Understanding NA 16b
Career NA 15b
Theory application NA 10c
Problem solving NA 10c
Negative Total negative responses See below 17d
Preactivity anxiety NA 14d
Assessment Total negative assessment comments 9e
Limited in scope 4e
Too comprehensive 4e
Suitability NA 5e
Future recommendations 12

a. Strong cluster for positive attributes.


b. Medium cluster for positive attributes.
c. Weak cluster for positive attributes.
d. Medium cluster for negative attributes.
e. Weak cluster for negative attributes.

analysis. Constructs and, where appropriate, attributes of Findings


constructs were identified within each theme using pattern
matching to explore student perceptions and opinions of the Three broad themes emerged from the data: positive
technique. The relative importance of attributes was also comments (n = 47), negative comments (n = 17), and rec-
assessed by clustering both positive and negative responses ommendations for future use (n = 12; see Table 1).
into strong, medium, or weak clusters on the basis of how Six main positive attributes of the viva technique were
frequently themes were mentioned. identified: being preferable to written exams (n = 42);
Despite its widespread acceptance, qualitative research, facilitating oral communication, which was superior to
as an interpretive methodology, has potential limitations written exams in examining student knowledge (n = 31);
in the analysis of evidence, objective reporting, and lack studying for the viva encouraged understanding of subject
of generalizability (McGuiggan & Lee, 2008). Yin (1994) matter as opposed to traditional exams, which rely on regur-
and Denzin and Lincoln (1998) identified external valid- gitation of content (n = 16); preparing students for job inter-
ity and reliability as constructs to judge research design. views and therefore enhancing career prospects (n = 15);
Validity reflects the accuracy with which the findings rep- aiding theory application (n = 10); and enabling demonstra-
resent what is happening in the field. In this case, valid- tion of problem-solving skills (n = 10). The following
ity was enhanced through the use of NVivo for thematic quotation reflects the feelings of the majority of students
coding of data (Hussey & Hussey, 1997). Clustering of toward the use of vivas as an assessment technique.
themes (Miles & Huberman, 1994) that emerged from
the data identified salient constructs that were reported
in this article. Reliability refers to the generalizability of I think its a unique but practical method of assessment.
the findings to other situations. As this research exam- The viva allows you to comprehensively convey aspects
ined one cohort of students in a services marketing unit, that would not be as evident in a written exam. The exam-
the replication of the study using different cohorts and iners also allow you to be relaxed so as to enable you to
across marketing units would improve reliability of the answer the questions more thoroughly. Good practice
findings. especially if youre going into the workforce next year.
126 Journal of Marketing Education

Student sentiment in the use of vivas to facilitate oral Within the construct of assessment, respondents reported
communication is illustrated in the quote below. This quote opposing attributes: some considered the technique too nar-
demonstrates students perceptions that oral examinations row to assess the units learning outcomes (n = 4) and, in
are a novel way to examine subject knowledge, they develop contrast, others considered the technique too comprehen-
student communication skills, and they are valuable prepa- sive in its examination scope (n = 4). The following quota-
ration for job interviews. tion reflects the anxiety of students prior to the viva and
their opinion after the experience that the technique is not
Viva is a good way to test your knowledge on the as difficult as first imagined.
issues of services marketing. It is a good way to pre-
pare yourself for the real industry. It is very similar to The viva process is at first very nerve racking and
a job interview and you have to prepare yourself in unexpected. Having now done a viva I can say that
the same manner you would for an interview. It is the process is very easy going. For future reference
important to know your stuff otherwise the viva could I think letting people know what to expect would be
be your worst enemy! much appreciated.

Evaluation of positive assessment themes revealed: stu- Projective tests uncovered several student recommenda-
dents saw a viva as being a more efficient and effective tions: improved briefing on the process and more frequent
assessment technique than written exams (n = 28); the dia- practice through the semester; continued use in services
lectic communication process was beneficial in examining marketing as an appropriate and preferred assessment
in-depth knowledge because it allowed examiners to ask method; and adoption of vivas in other marketing units.
probing questions in assessing overall student competency Students thought greater understanding of the viva process
(n = 23); the technique was regarded as a novel assessment and more practice in class would help reduce anxiety prior
method (n = 21); and vivas encouraged self-reflection and to the viva.
simulated real-life situations (n = 9 and n = 6, respec- Overall, students also regarded the viva as a better form
tively). The main recommendation offered by respondents of assessment than exams because of the ability to engage
(n = 12) was that the viva method should be more widely with the examiners and demonstrate knowledge and skills.
adopted throughout their university program, as it is better The positive opinion of vivas was reflected by several stu-
able to assess a students knowledge. The following quota- dents who considered that vivas should be more widely
tion identifies the benefits of vivas by engaging students adopted in other marketing units. The quotation below is
with assessment technique by applying theory to practice. indicative of the students recommendations for use of
vivas in the future.
Vivas are a good idea. Throughout viva exam you learn
a lot of things, not only you reflect and revise back all
The very nature of services doesnt allow for it to be
theory you learn, but also it gives you an opportunity to
confined in a set frameworkit is interactive and the
answers spontaneously, communicating skills and
viva allows you to better elaborate and expand on dif-
apply theory to the practical. It is not going to be like
ferent service experiences, within a service environ-
those boring written exams like we experience during
ment. It is an interactive and highly educational experience
years we spent in university; it gives you a new era and
that not only serves to assist you in University but
meaning of the fun exams. It seems hard, but it is not
throughout your day-to-day life. I definitely recom-
because it is interactive and interesting. Definitely
mend it over traditional assessments.
worthwhile and useful for future marketers.

The novel nature of vivas was expressed by many stu- Discussion and Conclusion
dents who regarded the process as valuable preparation for
future employment interviews through experiential learn- Many universities espouse generic graduate attributes such
ing as illustrated in the following quotation: This is an as effective communication, critical thinking, problem solving,
unorthodox and practical way of assessing your knowledge thinking on their feet, decision making, and leadership.
on the subject and is immeasurable in preparing students Similarly, employers also seek marketing graduates with these
for the workforce and job interviews. generic skills, as well as discipline-specific marketing knowl-
An exploration of negative themes identified the follow- edge, concepts, tools, and experiences. Oral communication
ing issues: previva anxiety concerns (n = 14); disagreement skills are frequently mentioned in the literature as a graduate
that viva is an appropriate assessment technique (n = 9); attribute desired by employers and universities alike (Crosling,
and vivas being suitable only for confident students (n = 5). 2000; R. Davis et al., 2002; Haffer & Hoth, 1981).
Lee, Pearce / Viva Voce as an Assessment Method 127

Clearly, marketing graduates require both discipline- students the ability to develop skills regarded as essential
specific knowledge and generic skills. However, some for graduates entering a marketing career.
suggest that many marketing educators concentrate on the Despite Kehm (2001) and others identifying sociophys-
discipline-specific knowledge at the expense of the generic iological, psychoanalytical, sociological, and methodologi-
skills (Lamb et al., 1995; Lunsford, 1991). Graduates deci- cal problems with conducting vivas, the literature notes
sion making, problem solving, and communication skills frameworks or guidelines to improve reliability (Birley,
equip students with career-ready attributes with the poten- 2001), validity (M. H. Davis & Karunathilake, 2005), objec-
tial to differentiate graduates and increase their marketabil- tivity (M. H. Davis & Karunathilake, 2005; Ebel, 1972;
ity through improved competence in communication skills Paul, 1994), and fairness of the procedure (Park, 2003). The
(Lamb et al., 1995). A challenge for marketing educators is use of trained examination staff, defined assessment criteria
to design curricula to align to learning objectives with mul- and procedures, and student preparation can assist in ensur-
tiple methods of learning and assessment. ing reliability, validity, and practicality of the vivas. Studies
Ramsden (1992) suggests assessment should be about of viva use in medical practice examination find viva results
learning, not just grades, and that students should be able to positively correlate with overall grades (Birley, 2001;
display their knowledge and creativity using a variety of M. H. Davis & Karunathilake, 2005). Thus, these findings
assessment techniques to take responsibility for their learn- suggest that when vivas are carefully implemented, the
ing. Correspondingly, the results of this study support the results may provide a reliable and valid assessment of can-
use of vivas as an alternative and complementary form of didates learning outcomes.
assessment. It offer educators more than another summative Results from this study indicate that examination by oral
assessment method, by assisting students in communication defense is regarded by many students as a superior method
skill development (Joughin, 1998), facilitating deep learn- to traditional university techniques. These findings concur
ing (Sayce, 2007), and preparing graduates for their careers with those of Borin et al. (2008), who suggest that learning
(Lamb et al., 1995). As a summative assessment method, outcomes are best demonstrated in practice; indeed, the
vivas are able to examine higher order complex and abstract majority of marketing students in this study reported vivas
learning objectives (Kehm, 2001). Oral communication offer a more engaging and comprehensive assessment method
skills are developed as students prepare for and participate that allows them to demonstrate application of their skills
in their vivas. As an active learning pedagogy, vivas facili- and knowledge. Thus, it can be argued that vivas, as a com-
tate deep learning by students applying theory to practice plementary component of schools assessment policies, have
(Floyd & Gordon, 1998) through PBL activities to demon- the potential to improve quality assurance standards. In
strate higher order skills such as analysis, synthesis, and addition, student responses during vivas provide in-depth
evaluation. Vivas allow students to demonstrate their appli- qualitative feedback to assess student-learning outcomes;
cation of theory to practice and their mastery of what they this feedback can be used to modify and improve curricu-
have learned, rather than what they have been taught (Borin lum design. The use of qualitative feedback through dialec-
et al., 2008). tic discussions with students will complement traditional
Vivas offer marketing educators an assessment method unit evaluations that rely primarily on Likert-type scale
that aligns to student-learning outcomes, which, in turn, (or similar) ratings and short comments.
support graduate attributes by teaching oral communica- The study has two main implications for marketing edu-
tion, problem solving, and decision-making skills. The cators. First, it provides evidence of an alternative form of
results of this study concur with previous findings in the assessment that students believe is superior to traditional
literature that indicate some students consider that vivas written examinations in allowing them to communicate and
allow a more effective examination of their learning out- demonstrate what they have learned. Despite vivas being an
comes (Chadwick, 2004), test their problem-solving ability and established method of assessing doctoral defenses and clini-
application of theory to practice (Joughin, 1998). Vivas also cal examinations, the majority of marketing educators do not
assist in the examination of areas that may not be easily use this technique, even though it may offer significant ben-
assessed by written work or presentations, through probing efits. The authors experience within their marketing school
and questioning, interpretation of the items, and the need to aligns with that of Crosling (2000), who found that many
demonstrate critical reasoning. The dialectic communica- universities were sporadic and unsystematic in their approach
tion process between examiners and the student provides an to the developmental curriculum incorporating oral commu-
effective method to assess students understanding of theory nication skills. However, the study highlights the opportu-
and examines deep knowledge (Kehm, 2001). Furthermore, nity to incorporate vivas to assess competencies that are
the use of vivas may assist in assessing professional skills traditionally difficult to assess using traditional summative
that are harder to examine by traditional methods (Harden assessment techniques such as single authored manuscripts,
et al., 1999), with the process and experience offering examinations, and group projects.
128 Journal of Marketing Education

Second, student anxiety may be overemphasized. All period between undertaking the viva and completing the
students who identified anxiety as an issue conceded, after cartoon test, researcher interpretations, and students report-
the viva process, that the technique was student friendly. ing what they expect the researcher (lecturer) wants to hear.
Furthermore, anxiety may have been exacerbated because Furthermore, the use of frequency counts is not intended to
few, if any, students had previous experience with the viva infer statistical rigor, but rather to demonstrate a level of
method. To reduce unnecessary anxiety, educators must importance reported by the sample group. Finally, it must be
ensure that students who lack confidence in their interper- acknowledged that because of human nature, an individuals
sonal communication skills are thoroughly briefed on the assessment may be influenced by an examiners potential
process and, if possible, are able to practice prior to the bias for attractive and/or like-minded individuals and thus
examination. Perhaps if more use were made of vivas, stu- the interview process is not value free. Future research might
dents confidence and career-enabling interview skills include broadening the application of the assessment tech-
would be enhanced. nique for use in other subjects and disciplines, investigating
This study makes a contribution to marketing pedagogy opinions of students for whom English is a second language,
by offering insight into marketing students perceptions of exploring marketing educator attitudes to vivas, and using
vivas. In doing so, it reminds us there are alternative, potent quantitative methods to gauge and validate findings.
ways of assessing student learning that are underutilized.
Notwithstanding difficulties with planning and staging vivas
(such as selecting panel members, assembling panels, and
References
scheduling interviews), additional workload and resources Allison, R., & Katona, C. (1992). Reliability and validity of oral examina-
implications (Park, 2003), and reliability and validity con- tions, Medical Teacher, 14, 383-389.
cerns, the method has distinct virtues. Crosling (2000) Arndt, C. B., Guly, U. M. V., & McManus, I. C. (1986). Preclinical anxiety:
purport that the costs of developing oral skills are out- The stress associated with a viva voce examination. Medical Education,
20, 274-280.
weighed by potential benefits of well planned and imple- Bashook, P. (2000, April). Assessing clinical judgment using standardized
mented programs. oral examinations. Paper presented at the Annual Meeting of the
The call for the inclusion of a variety of learning activi- American Educational Research Association, New Orleans, LA.
ties (Karns, 2006) and assessment modalities (Floyd & Bazeley, P., & Richards, L. (2000). The NVivo qualitative project book.
Gordon, 1998) supports the notion to incorporate vivas in London: Sage.
Biggs, J. (1999). Teaching for quality learning at university. Buckingham,
marketing curricula. As assessment drives learning (Boud
UK: Society for Research Into Higher Education and Open University
et al., 1999; Wass et al., 2001), vivas may assist in facilitat- Press.
ing deep learning and problem-solving skills. Although the Birley, H. (2001). The Society of Apothecaries diploma examination in
literature and this studys results identify that not all stu- genitourinary medicine: Death of the viva voce? Sexually Transmitted
dents prefer vivas as an assessment method, the use of vivas Infections, 77, 223-224.
Bloom, B. S., Engelhart, M. D., Furst, E. J., Hill, W. H., & Krathwohl, D. R.
aligns with the recommendation of Karns (2006) to stretch
(Eds.). (1956). Taxonomy of educational objectives. The classification
students beyond their preferred learning style. of educational goals, Handbook I: Cognitive domain. New York: David
The incorporation of vivas into marketing course curri- McKay.
cula through in-class activities should be phased in to allow Borin, N., Metcalf, L. E., & Tietje, B. C. (2008). Implementing assess-
students the opportunity to develop skills in a building-block ment in an outcome-based marketing curriculum. Journal of Marketing
approach (Lamb et al., 1995). Initially, student-learning acti Education, 30, 150-159.
Boud, D., Cohen, R., & Sampson, J. (1999). Peer learning and assessment.
vities may include in-class debates, role-plays, and short Assessment & Evaluation in Higher Education, 24, 413-426.
presentations (individually and as part of a group). In the Carless, D. R. (2002). The mini-viva as a tool to enhance assessment
later stages of their course, students should practice mock for learning. Assessment & Evaluation in Higher Education, 27,
vivas with their peers and formative vivas in preparation for 353-363.
summative assessment vivas. Future research in the adop- Catterall, M., & Ibbotson, P. (2000). Using projective techniques in educa-
tion research. British Educational Research Journal, 26, 245-256.
tion of vivas across marketing cohorts, subjects, and disci- Chadwick, S. M. (2004). Curriculum development in orthodontic special-
plines would assist in understanding the reliability and ist registrar training: Can orthodontics achieve constructive align-
validity of the method. In addition, further investigation on ment? Journal of Orthodontics, 31, 267-274.
the impact of student-learning outcomes and development Challis, D. (2008). Committing to quality learning through adaptive
of graduate attributes based on individual preferred learning online assessment (Unpublished Report). Melbourne, Australia:
Deakin University. Retrieved May 4, 2008, from http://www.unisa
styles and cultural nuances and as a way of facilitating deep
.edu.au/evaluations/Full-papers/ChallisFull.doc
learning is warranted. Crosling, G. (2000). Transition to university: The role of oral communi-
Limitations of this exploratory study included potential cation in the undergraduate curriculum. Journal of Institutional
bias associated with students not being given a cooling-off Research, 9(1), 69-77.
Lee, Pearce / Viva Voce as an Assessment Method 129

Davis, M. H., & Karunathilake, I. (2005). The place of the oral examina- Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis: An
tion in todays assessment system. Medical Teacher, 27, 294-297. expanded sourcebook (2nd ed.). Thousand Oaks, CA: Sage.
Davis, R., Misra, S., & van Auken, S. (2002). A gap analysis approach to Morley, L., Leonard, D., & David, M. (2002). Variations in vivas: Quality
marketing curriculum assessment: A study of skills and knowledge. and equality in British PhD assessments. Studies in Higher Education,
Journal of Marketing Education, 24, 218-224. 27, 263-273.
Denzin, N. K., & Lincoln, Y. S. (1998). Collecting and interpreting quali- Morley, L., Leonard, D., & David, M. (2003). Quality and equality in
tative materials. Thousand Oaks, CA: Sage. British PhD assessment. Quality Assurance in Education, 11,
Donoghue, S. (2000). Projective techniques in consumer research. Journal 64-72.
of Family Ecology and Consumer Sciences, 28, 47-53. Morrison, M. A., Sweeney A., & Heffernan, T., (2003). Leaning styles
Douglas, A., & Swift, J. (1997). The viva voice (Report). Birmingham, of on-campus and off-campus marketing students: The challenge
UK: University of Central England, Research Training Initiative. for marketing educators. Journal of Marketing Education, 25,
Ebel, R. L. (1972). Essentials of educational measurement. Englewood 208-217.
Cliffs, NJ: Prentice Hall. Morrison, M. A., Sweeney A., & Heffernan, T., (2006). Karnss learning
Floyd, C. J., & Gordon, M. E. (1998). What skills are important? A com- styles and learning effectiveness: A rejoinder. Journal of Marketing
parison of employer, student, and staff perceptions. Journal of Marketing Education, 28, 64-68.
Education, 20, 103-109. Mostyn, . (1985). The content analysis of qualitative research data: A
Frontczak, N. (1990). The role of learning styles in marketing education. dynamic approach. In M. Brenner, J. Brown, & D. Cauter (Eds.), The
In J. Doubt & G. McKinnon (Eds.), Proceedings of the Western Marketing research interview: Uses and approaches (pp. 115-145). London:
Educators Association (pp. 57-62). Las Vegas, NV: Western Marketing Academic Press.
Educators Association. Oakley, B., & Hencken, C. (2005). Oral examination assessment prac-
Glaser, B. G., & Strauss, A. L. (1967). The discovery of grounded theory. tices: Effectiveness and change with a first year undergraduate cohort.
New York: Aldine de Gruyter. Journal of Hospitality, Leisure, Sport and Tourism, 4(1), 3-14.
Haffer, J. C., & Hoth, C. C. (1981). Grooming your marketing students to Park, C. (2003). Levelling the playing field: Towards best practice in the
match the employers ideal job candidate. Journal of Marketing doctoral viva. Higher Education Review, 36(1), 47-67.
Education, 3, 15-19. Paul, V. K. (1994). Assessment of clinical competence of undergraduate
Harden, R. M., Crosby, J. R., & Davis, M. H. (1999). AMEE Guide No. medical students. Indian Journal of Pediatrics, 61, 145-151.
14: Outcome-based education. Part 1: An introduction to outcome- Pearce, G., & Lee, G. (2006, December). Your place or mine: New tech-
based education. Medical Teacher, 21, 7-14. niques for team formation. Paper presented at the Australian and New
Hussey, J., & Hussey, R. (1997). Business research. London: Macmillan. Zealand Marketing Academy Conference, Brisbane, Australia.
Jackson, C., & Tinkler, P. (2001). Back to basics: A consideration of the Pearce, G., & Lee, G. (2007, December). Marketing student perceptions
purposes of the PhD viva. Assessment & Evaluation in Higher of viva voce (oral examination) as an assessment. Paper presented at
Education, 26, 355-366. the Australian and New Zealand Marketing Academy Conference,
Jarvis, P. (1987). Adult learning in the social context. London: Croom Helm. Dunedin, New Zealand.
Joughin, G. (1998). Dimensions of oral assessment. Assessment & Evaluation Pinto Pereira, L. M., Telang, B. V., Butler, K. A., & Joseph, S. M. (1993).
in Higher Education, 23, 367-378. Preliminary evaluation of a new curriculumIncorporation of problem
Karns, G., L. (2006). Learning style differences in the perceived effec- based learning (PBL) into the traditional format. Medical Teacher, 15,
tiveness of learning activities. Journal of Marketing Education, 28, 352-365.
56-63. Ponder, N., Beatty, S., & Foxx, W. (2004). Doctoral comprehensive
Kehm, B. M. (2001). Oral examinations in German universities. Assessment exams in marketing: Current practices and emerging perspectives.
in Education, 8(1), 25-31. Journal of Marketing Education, 26, 226-235.
Kolb, D. A. (1984). Experiential learning. Englewood Cliffs, NJ: Prentice Pressey, S. L., Pressey, L. C., & Barnes, E. J. (1932). The final ordeal. The
Hall. Journal of Higher Education, 3(5), 261-264.
Kuisma, R. (2007). Portfolio assessment of an undergraduate group proj- Ramsden, P. (1992). Learning to teach in higher education. London:
ect. Assessment & Evaluation in Higher Education, 32, 557-569. Routledge.
Lamb, C. W., Shannon, S. H., & Moncrief, W. C. (1995). Integrating Richards, L. (1999). Using NVivo in qualitative research. Thousand Oaks,
skills and content knowledge in the marketing curriculum. Journal of CA: Sage.
Marketing Education, 17, 10-19. Rowntree, D. (1994). Assessing students: How shall we know them?
Lee, G., & Holland, B. (2008, July). Using creativity to measure impacts London: Kogan Page.
of community-based learning experiences on business students. Paper Sayce, S. (2007). Managing the fear factor (or how a mini-viva assess-
presented at the Australian Universities Community Engagement ment can improve the process of learning for international students).
Alliance National Conference, Fraser Island, Queensland, Australia. Electronic Journal of Business Research Methods, 5, 83-92.
Levine, H. G., & McGuire, C. H. (1970). The validity and reliability of Schubert, A., Tetzlaff, J. E., Tan, M., Ryckman J. V., & Mascha, E.
oral examinations in assessing cognitive skills in medicine. Journal of (1999). Reliability and validity of oral examination. Anaesthesiology,
Educational Measures, 7(2), 63-74. 91(1), 288-298.
Lunsford, D. A. (1991). Developing creative problem solving skills in Schwarzer, C., & Buchwald, P. (2000, August). Dyadic coping and per-
marketing case analysis. Marketing Education Review, 1(3), 62-69. sonal trust in studentteacher interactions. Poster session presented at
Malhotra, N., Hall, J., Shaw, M., & Oppenheim, P. (2002). Marketing the 108th Convention of the American Psychology Association,
research: An applied orientation. Frenchs Forest: Pearson Education Washington, DC.
Australia. Solomon, B., & Felder, R. (2004). Index of learning styles. Retrieved on
McGuiggan, R., & Lee, G. (2008, December). Cross-case analysis: May 4, 2008, from http://www.engr.ncsu.edu/learningstyles/ilsweb
An alternative methodology. Paper presented at the Australian .html
and New Zealand Marketing Academy Conference, Sydney, Tinkler, P., & Jackson, C. (2002). In the dark? Preparing for the PhD viva.
Australia. Quality Assurance in Education, 10, 86-97.
130 Journal of Marketing Education

Trafford, V. (2003). Questions in doctoral vivas: Views from the inside. Glenn Pearce, a senior lecturer in the School of Marketing at the
Quality Assurance in Education, 11, 114-122. University of Western Sydney, is an innovative educator receiving several
Wakeford, R., Southgate, L., & Wass, V. (1995). Improving oral examina- external awards for teaching excellence. He has pioneered the somewhat
tions: Selecting, training, and monitoring examiners for the MRCGP unorthodox technique of "applied theatre" is recognized by the wider
(Royal College of General Practitioners). British Medical Journal, marketing education community as a leader in his field. He has extensive
311, 931-935. publications in educational drama and services marketing.
Wass, V., Van der Vleuten, C., Shatzer, J., & Jones, R. (2001). Assessment
of clinical competence. Lancet, 357, 945-949.
Yin, R. K., 1994. Case study research: Design and methods. Thousand Geoffrey Lee is the Associate Dean Engagement at the University of
Oaks, CA: Sage. Western Sydney, Australia. His research focuses on the strategic applica-
Yip, H. K., Smales, R. J., Newsome, P. R., Chu, F. C., & Chow, T. W. tions of ICT for SMEs, industry sectors, and education. He has completed
(2001). Competency-based education in a clinical course in conserva- numerous industry-funded studies and advises government on innovation
tive dentistry. British Dental Journal, 191, 517-522. and knowledge management. Dr. Lee publishes and works at conferences
Zikmund, W. G. (2003). Exploring marketing research. Mason, OH: and in journals in the areas of e-commerce, education, and post-modern
Thomson Learning. constructivism.