Assessing productive skills

With reference to PT3

What is speaking?
What do we actually do?
 Self-
monitoring
PT3 Speaking
 Paper 3: Speaking
Candidates The Speaking test is taken by all Form 3 students.
Paper Format Three sections, all aimed at the A2-B1 CEFR range, with an increase in level as the
candidate progresses through the test. The test is taken as a paired test.
Number of Tasks 3
Task types Interview, individual prompt-card story and discussion based on mind map input.
Input Word limits in place for each test part. All input checked for compliance with stated
CEFR levels.
Instructions Instructions for the tasks are given by the interlocutor, some prompting may be
present on materials presented to the test takers to scaffold responses
appropriately.
Time Allowed 11 minutes
Marking The paper is examiner marked, in situ, by two trained examiners. There will be an
interlocutor who manages the interaction and provides an overall score for spoken
production and an assessor who provides scores for Grammar, Vocabulary and
Communicative Competence.
Weighting Candidates are assessed according to four criteria: Overall Spoken Production,
Grammar, Vocabulary and Communicative Competence.
Task focus Tasks cover a range of language-relevant foci. It should be assumed that the
relevant CEFR descriptors for the A2-B1 levels will be referenced throughout the
test material production process. The descriptors referred to are derived from the
curriculum learning standards.
 Let’s look at PT3 Speaking

• Understanding the tasks

• What is expected from the candidates?
Writing
So what skills and knowledge are involved in
creating a written message (a text)?
 Cognitive processes
• Macro-planning
• Organisation

cognitive •
•
Micro-planning
Translation
• Monitoring
• Revising

socio-
• Adopting an
appropriate

linguistic register

linguistic • Performing
language
functions in an
appropriate
• Lexical and structural way/
resources awareness of
• Functional and politeness
pragmatic knowledge
conventions
• Textual knowledge -
coherence, cohesion • Genre
knowledge
PT3 Writing
Paper 2: Writing

Candidates The Writing test is taken by all Form 3 students.

Paper Format Two sections, both aimed at the A2-B1 CEFR range.
Number of Tasks 2
Task types Short communicative message and notes expansion.
Input Word limits in place for each test part. All input checked for compliance with
stated CEFR levels.
Instructions Instructions for the tasks are given on the question paper.
Answer Format Candidates write their answers on the question paper.
Time Allowed 1 hour
Weighting Candidates are assessed according to four criteria: Content, Communicative
Achievement, Organisation and Language.
Marking The paper is examiner marked.
Task focus Tasks cover a range of language-relevant foci. It should be assumed that the
relevant CEFR descriptors for the A2-B1 levels will be referenced throughout the
test material production process. The descriptors referred to are derived from the
curriculum learning standards.
 Let’s look at PT3 Writing

• Understanding the tasks

• What is expected from the candidates?
Assessment scales
Productive skills in PT3
 Assessment scales

• Types of rating scales

– Holistic
• General impression
• 1 overall mark

– Analytic
• Broken down into components
• Several marks - 1 for each assessment category
Let’s look at some Writing sample
scripts.
 Rater issues …

‘Leniency/severity’ effect
• Raters’ relative likelihood of assigning higher and lower scores either compared to other
raters or to a benchmark
‘Halo’ effect
• Rater tendency to award similar ratings than justified on different criteria of an analytic
rating scale
‘Extreme/central-tendency’ effect
• Rater tendency to award scores in either the extremes or middle categories of a rating
scale
‘Bias’ effect
• Raters may show a pattern of harshness/leniency with regard to certain aspects of the
rating process
‘Randomness’ effect
• Idiosyncratic and unpredictable patterns of rater behaviour which cannot be explained in a
systematic fashion