The Use of International Examinations in The English Language Program at Uqroo

MEMORIAS DEL V FORO DE ESTUDIOS EN LENGUAS INTERNACIONAL (FEL 2009)
ISBN 978-607-9015-05-3
THE USE OF INTERNATIONAL EXAMINATIONS IN THE ENGLISH

LANGUAGE PROGRAM AT UQROO
M.C. Jos Luis Borges Ucan
M.C. Deon Victoria Heffington
Dr. Alfredo Marn Marn
Dr. Caridad Macola Rojo
Universidad de Quintana Roo
Abstract
According to Cambridge ESOL in order to reach the B1 level of the Common
European Framework of Reference students may need approximately 350
400 hours of guided study. In the first phase of this work, the main aim was
to find out whether students in the English language program would pass the
PET exam after 384 hours of guided study and identify their strengths and
weaknesses in language skills. In a second phase of the study, the TOEFL
examination was applied to the same group of students. The results in both
examinations were carefully analyzed in order to define the level of the
development of the skills.
1. INTRODUCTION
Evaluation plays an important role in students achievement while learning a
foreign language. Furthermore, when it comes to comparing examinations, it is not
easy since most of them have different emphases (Simmonds, 1985). Despite this
fact, the need for examining students performance in different language skills still
remains.
It is assumed that a significant number of students enrolled in the English
Language Major at Universidad de Quintana Roo do not develop their language
skills at the same level of competence, and therefore, are more likely to fail at
achieving a high score on an international examination, such as Preliminary
English Test (PET) or Test of English as a Foreign Language (TOEFL) after
receiving 384 hours of guided study.
The main aim of this study is not only to describe students English level
according to different international sample tests, but identify in which skills they
Universidad de Quintana Roo Departamento de Lengua y Educacin

http://fel.uqroo.mx - fonael@yahoo.com
648

ISBN 978-607-9015-05-3
have more strengths and weaknesses. Additionally, a comparison of students

scores among two international sample tests will be presented.
2. BACKGROUND
2.1 Common European Framework of Reference for Languages (CEFR)

The Common European Framework of Reference for Languages (CEFR)
provides a common basis for the elaboration of language syllabuses, curriculum
guidelines, examinations, textbooks, etc. It describes in a comprehensive way what
language learners have to learn to do in order to use a language for
communication and what knowledge and skills they have to develop so as to be
able to act effectively.
The Framework also defines levels of proficiency which allow learners
progress to be measured at each stage of learning and on a life-long basis. It
provides the means for educational administrators, course designers, teachers,
teacher trainers, examining bodies, etc., to reflect on their current practice, with a
view to situating and co-ordinating their efforts and to ensuring that they meet the
real needs of the learners for whom they are responsible.
2.1.1 Cambride ESOL PET examination

Cambridge ESOL Exams are aligned to the CEFR, which makes it easier for
both learners and teachers to see the level of different qualifications, identify which
exam corresponds to each level, and measure the progress they are achieving.
According to Figure 1.1, a Threshold user (B1) should be able to pass the
Preliminary English Test (PET).

649

ISBN 978-607-9015-05-3
Figure 1.1 Cambridge ESOL exams aligned to CEFR
According to Cambridge ESOL, students who are at level B1 have been

exposed to approximately 350-400 hours of guided study and are capable of
performing the following tasks:
ALTE Can Do Statements: overall general ability

CEFR
Listening/Speaking
Reading
Writing
LEVEL
B1
CAN express opinions on
CAN understand
CAN write
abstract/cultural matters in a
routine information
letters or make
limited way or offer advice
and articles, and the
notes on
within a known area, and
general meaning of
familiar or
understand instructions or
non-routine
predictable
public announcements.
information within a
matters.
familiar area.
Figure 1.2 ALTE Can Do Statements for B1 (Threshold) level

650

ISBN 978-607-9015-05-3
2.1.2 ELT Internet-based Test of English as a Foreign Language (TOEFL

iBT )
The Internet-based TOEFL iBT (Test of English as a Foreign Language)
measures the ability to use and combine listening, reading, speaking and writing
skills to communicate in an academic environment. ELT offers the following
minimum scores on the TOEFL iBT corresponding to each level of the CEFR.
Minimum scores were identified separately for speaking, listening, writing and
reading:
Test section
Total Score
Minimum score
Range Scale
B1 Level
TOEFL iBT total
0-120
57-86
TOEFL iBT Reading
0-30
TOEFL iBT Listening
0-30
13
TOEFL iBT Speaking
0-30
19
TOEFL iBT Writing
0-30
17
Source: ETS TOEFL
Likewise to Bachman, et al (1995) this is a comparative study between two

sample international tests which are used worldwide to assess students general
English.
3. METHODOLOGY
3.1 Participants
In phase one of the study we decided to apply the PET (Preliminary English
Test) to 33 students who had finished the English III course in the program. These
students had already had 384 hours of guided instruction, and therefore, were
supposed to reach the PET, i.e. level B1 according to the Common European
Framework for Languages.
The participants were all Mexican and Spanish native speakers and also
enrolled in the English Language Program at the University of Quintana Roo.
651

ISBN 978-607-9015-05-3
3.2 Instruments
Two instruments were used in this study: the Preliminary English Test from
Cambridge ESOL examinations and the TOEFL iBT from Educational Testing
Services (ETS). A comparison of the results in both tests will be provided in the
following paragraphs.
3.2.1 PET
The PET test used can be found at the official web page of Cambridge
ESOL examinations. They are sample tests provided by Cambridge and can be
considered as valid and reliable as an official PET examination.
The test contains four sections or papers: reading, writing, listening and
speaking. They all have the same weight which is 25%.
3.2.2 TOEFL
The Test of English as a Foreign Language (TOEFL) is used worldwide to
measure English Language Proficiency. We used the companion CD-ROM of the
Official Guide to the New TOEFL iBT which includes authentic sections.
The TOEFL examination also gives the same weight to all four skills. The
score scale is 0-30 for each skill making a total score of 0-120.
3.3 DATA COLLECTION
3.3.1 Applying the PET

The examiners had to become familiar with the test format of the Preliminary
English Test (PET) especially for the evaluation of speaking and writing in which
both examiners were asked to provide a score.
One of the main problems encountered during the application of the PET
was that the participants were hard to locate and finally only 22 out of 33
completed the four sections of the test. Therefore, only data obtained from 22 was
analyzed.

652

ISBN 978-607-9015-05-3
3.3.2 Applying the TOEFL

We decided to used the TOEFL iBT because it measures the same four
abilities: reading, listening, speaking, and writing. Other versions of the TOEFL
examination omit the evaluation of writing and/or speaking skills.
The main problem encountered during the application of the TOEFL was
that some students had busy schedules which made it difficult for them to take an
examination that lasted approximately four hours. In the end, only 11 students out
of 22 were able to complete the reading and listening sections.
1. RESULTS AND DISCUSSION
4.1 PET scores

On a PET exam, a student may achieve any of the following scores:
Pass with Merit =>85%
Pass 70-84%
Narrow Fail 65-69%
Fail =<64
After grading the 22 examinations we obtained the results explained in the

following table.
Frequency
Percent
Pass with merit
4.5
Pass
12
54.5
Narrow fail
13.6
Fail
27.3
Total
22
100.0
Figure 1.3 Results from the PET
As it can be observed in figure 1.3, 59.1% of the total number of participants

passed the examination and 40.9% failed.

653

ISBN 978-607-9015-05-3
4.2 TOEFL scores

As mentioned above both tests use different scales, and in the case of
TOEFL it was necessary to have a reference that would allow us to find out
whether a student reached or did not reach the B1 level of the Common European
Framework for Languages.
In a study conducted by ETS researchers Richard J. Tannenbaum, Ph.D.,
and E. Caroline Wylie, Ph.D., in October 2006, a panel of 23 experts was able to
make connections between the TOEFL and the CEFR.
The table below presents the recommended score ranges and the minimum
score for each modality at each CEFR proficiency level according to that study.
By using the table above we found out the following results for each
participant and we compared them with the overall result from the PET.
iBT
Reading
iBT
Subject
Result PET
Result
Listening
Result
B1
16-18
B1
21-23
B2
Fail
15-18
B1
10-15
B1
B1
18-21
B1
16-21
B1
B1
15-18
B1
16-18
B1
Fail
14-16
B1
9-13
B1
B1
22-24
B2
17-22
B1
B1
25-27
B2
26-29
C1
B1
16-18
B1
14-18
B1
B1
19-21
B1
6-12
Fail
10
B1
12-16
B1
14-18
B1
11
B1
15-18
B1
10-15
B1
Figure 1.4 Results from the PET compared with results from the TOEFL

654

ISBN 978-607-9015-05-3
Results show that some participants did not reach B1 level according to the
PET but they did reach it according to the TOEFL in reading and listening. Case
number one, six and eight reached a higher level in this examination. Case nine
failed to pass the listening section despite having reached level B1 in the first
examination.
5. CONCLUSIONS AND IMPLICATIONS

In the first phase of the study we expected students to perform better in
receptive skills. However, the results show that students achieved a lower skill in
listening than in any other skill.
The reasons may be many but we assumed that the fact that students are
not exposed to the British accent during the first three courses could lead to poor
comprehension in the listening section. Therefore, it is suggested that students be
exposed to different varieties of English throughout their courses of English.
Another reason may be that listening requires more development of short-term
memory than other skills.
The results show that skills are not developed at the same level. This is
probably due to the lack of equal distribution of time to reinforce each skill in the
classroom. Further research about time used to develop different skills can be
made by using class observations. It is recommended that students be given an
individual and overall grade. This will help both the instructors and students find a
more balanced development of skills.
Students performed significantly better in the speaking section (18.6%). This
result shows a possitive washback effect, in terms of the communicative approach
that is used in English classes at UQROO.
In the second phase of the study, we found that students tend to perform
better in the TOEFL than in the PET. Further research can be done in order to find
out a more accurate table of equivalences than the one currently used in the
department of language and educational. A limitation of this study was the number
of participants. A much larger number will be necessary in order to improve the
current table of equivalences.
655

ISBN 978-607-9015-05-3
Finally, it is important to notice that the students level of English needs to be

improved in the first semesters and that the proficiency of the students needs to be
measured not only by using achievement tests but also proficiency tests. Using the
Cambridge or the ETS examinations after certain semesters may be an excellent
tool to measure the progress of the students and might even help them prepare to
reach the C1 level, which is a requirement to obtain their degree.

656

ISBN 978-607-9015-05-3
References
1. Bachman, L. et al (1995). An Investigation into the Comparability of Two Tests of English as
a Foreign Language: The Cambridge-TOEFL Comparability Study. Cambridge: Cambridge
University Press
2. CEFR
and
Can
Do.
Retrieved
October
5,
2009,
from
http://www.cambridgeesol.org/exams/exams-info/cefr.html
3. Common European Framework of Reference for Languages. Retrieved October 3, 2009,

from http://www.coe.int/t/dg4/linguistic/Source/Framework_EN.pdf
4. Mapping TOEFL iBT on the Common European Framework of Reference . Retrieved
October
3,
2009,
from
http://www.ets.org/Media/Research/pdf/CEF_Mapping_Study_Interim_Report.pdf
5. Simmonds, P. (1985). A Survey of English Language Examinations. ELT Journal, 39 (1),

33-42. Retrieved October 1, 2009, from ELT Journal on CD-ROM
6. The TOEFL Test - Test of English as a Foreign Language. Retrieved October 3, 2009,
from http://www.ets.org/portal/site/ets/menuitem.fab2360b1645a1de9b3a0779f1751509/?
vgnextoid=69c0197a484f4010VgnVCM10000022f95190RCRD

657

ISBN 978-607-9015-05-3
Biodata
Jos Luis Borges Ucn is a teacher and researcher at Universidad de Quintana
Roo. He holds a BA in English Language from Universidad de Quintana Roo and
an MA in Curriculum and Instruction from University of Nebraska.
Contact: jlborges@uqroo.mx
Deon Victoria Heffington is a teacher and researcher at Universidad de Quintana
Roo. She holds a BA in English Language and MA in Education. Her research
interests include individual differences in language learning. She is currently head
of the Department of Language and Education.
Contact: deon@uqroo.mx
Alfredo Marn Marn is a teacher and researcher at the Universidad de Quintana

Roo. He holds a BA in English Language and a especialidad from the
Universidad Veracruzana, an MA in Education from the Universidad de La Habana,
and a Ph.D. in English Language Teaching from the University of Essex, UK. His
research
interests
include
individual
differences
in
language
learning,
teachers/learners beliefs, L2 vocabulary acquisition, and uses of corpora in ELT.

Contact: alfmarin@uqroo.mx
Caridad Macola Rojo holds a BA in the Teaching of English from the Havana
Business University and Doctor in Education from the University of Havana. Fulltime professor in the Language and Education Department in the University of
Quintana Roo. Has written in co-authorship textbooks for the teaching of English,
Translation and English Grammar. Is recently involved in research concerning the
development of skills in language learning, especially the skill of reading.
Contact: macola@uqroo.mx

658

The Use of International Examinations in The English Language Program at Uqroo

Diunggah oleh

Informasi Dokumen

Judul Asli

Hak Cipta

Format Tersedia

Bagikan dokumen Ini

Bagikan atau Tanam Dokumen

Opsi Berbagi

Apakah menurut Anda dokumen ini bermanfaat?

Apakah konten ini tidak pantas?

Hak Cipta:

Format Tersedia

The Use of International Examinations in The English Language Program at Uqroo

Diunggah oleh

Hak Cipta:

Format Tersedia

MEMORIAS DEL V FORO DE ESTUDIOS EN LENGUAS INTERNACIONAL (FEL 2009)