Proceedings of the Redesigning Pedagogy: Culture, Knowledge and Understanding

Conference, Singapore, May 2007

QUALITATIVE ASSESSMENT IN THE GENERAL PAPER

Yang Siwei June

Hwa Chong Institution (College)

Cheong Su-Wei Audrey

Hwa Chong Institution (College)

ABSTRACT

It is a common assessment practice amongst teachers to assess student work by

providing feedback as well as a letter or numerical grade. This indicates to teachers how

students have fared for a particular piece of work as well as to serve as a definitive

measurement of learning on the student's part. However, this practice may "emphasise

competition [among students] rather than personal improvement" (Black et al, 2002). This

study explores how the omission of grades can positively impact a student's General Paper

(GP) essay grades. Over a period of three months and two assignments, 49 students from two

classes with differing linguistic abilities, taught by two different GP tutors, participated in the

study. Another 52 students taught by the same teachers were taken as the control group.

Essays for all students were marked with the same level of rigour and the same quality in

feedback, except that those involved in the study were not given letter or numerical grades.

All students were also familiarised with the Cambridge grade descriptors typically used by

their examiners. Results suggest that there is an increase by 12 percentage points in the final

-1-
grades of students involved in the study over those in the control group. The results of this

study highlights the potential improvement students can achieve through such grading

practices and might encourage English teachers to review common assessment practices.

INTRODUCTION

Assessment has to serve the primary purpose of facilitating and improving student

learning. It may also be designed for other purposes as well, namely, as a means of

accountability to stakeholders in education, for ranking or to certify the competence of a

student. This paper is interested in the former, which sees assessment as a means to further

student learning, rather than to provide a definitive rank or judgement to a student’s ability.

Assessing for learning rather than for ranking is an attempt to harness all possible

opportunities of the curriculum to reinforce a student’s learning. When a student is faced with

either numerical or letter grades, the basis for comparison between his classmates and a sense

of competition overrides any opportunity of improvement present in that assignment. This is

because the grade or mark given provides a final and summative conclusion of the student’s

ability. The focus thus changes from improving one’s work to finding out how one stands

amongst one’s peers. Students reported feeling “lost” or “insecure” when they were not given

any grades and only written feedback, shoring up precisely the very emotional and

psychological impact of numerical and letter grades on them. The grade becomes a marker

for attendant feelings of insecurity, competition and one’s self esteem, moving an exercise

from one that was academic to one that is emotional. Through this, excellent opportunities for

a student to reflect and improve on his learning are lost, as they are subordinated to an

emotional instinct of focusing purely on one’s grade. Withholding grades from a student and

providing, on the other hand more feedback on their work assists in creating a less

threatening and competitive classroom environment. Upon gaining feedback after this pilot

-2-
project, students have said that they would “pay very close attention to the written feedback

since it [was] the only indication of how well [they] had done.” Similarly, Black and William

(1998) have reported that students read feedback more carefully in the absence of marks.

Students are thus encouraged to focus on the given feedback rather than on comparing

grades.

The benefits of giving qualitative as opposed to quantitative feedback are beneficial in

many aspects. The onus on the teacher to provide quality feedback that is understood and can

be worked upon by the student becomes more crucial here. In an interview with students on

the way they interpret their teachers’ comments, Spandell and Stiggins (1990) found that

students had difficulty understanding the expectations of their teachers as feedback was

vague. Teacher comments such as “needs to be more concise” and “be more specific” only

resulted in negative and confused student responses, which did not aid in the improvement of

their writing. Rather than giving generic feedback on “poorly written work” or an

“unbalanced argument” the focus of giving feedback could be further detailed to explain how

it may be so. The teacher should be able to justify his/her feedback rather than providing a

generic comment on the student’s work. Consequently, the student may use such directed,

and detailed feedback to improve on his/her writing, because the feedback is now directed

towards improvement and reflection on one’s errors. Other than giving more detailed and

directed feedback and justifying comments, giving feedback according to Cambridge grade

descriptors gives both teachers and students a common platform to work upon. This ensures

that the marking language is one that is both easily accessible to student and teacher, while it

also familiarises students with Cambridge marking rubrics.

Keeping this in mind, in the four main sections that follow, we outline the main body

of literature on assessment, followed by an overview of what the term “qualitative

assessment” entails. We then go on to describe the work we did with our respective classes in

-3-
Hwa Chong Institution and our subsequent findings from this action research. A final section

recommends modifying teaching practices based on our findings.

LITERATURE REVIEW

Assessment plays a big role in an academic setting, so much so that what influences

students most is not the teaching but rather the assessment (Snyder, 1971; Miller and Parlett,

1974). Indeed, assessment is such a major facet of an educational system that “if we wish to

discover the truth about an educational system, we must first look to its assessment

procedures” (Rowntree, 1987, p1). Indeed, “assessment has become a public and educational

issue, not solely a technical one. We no longer seem to be content to be told that assessments

meet certain psychometric and statistical requirements. We want to know whether they help

or hurt learning and teaching” (Broad 2003, p9). Consequently, it would not be an

exaggeration to say that types of assessment, how it is carried out and how students react to

these modes of assessment is an integral part of any educational system. Studies such as that

of Sambell and McDowell (1998), Snyder, Miller and Parlett (1974) have gone on to

demarcate a research area and provided a lever into understanding the way students respond

to assessment practices. Similarly, Yorke (2001) has outlined the positive and negative ways

in which formative assessment can affect student retention and emphasised its role in

‘academic integration’ (Tinto, 1993).

Assessment has different functions, to establish academic credentials, to rank, to

distinguish between various student abilities or to further a student’s learning. Black,

Harrison, Lee, Marshall and Wiliam (2002) have identified three main problems in

assessment methods. Firstly, that assessment methods used by teachers are not effective in

promoting good learning. Comments from student include frequently highlight how

comments are too “generic” and thus of little help. Secondly, that marking and grading

-4-
practices emphasise competition rather than personal improvement. A student is quoted as

saying that “the atmosphere when receiving the returned scripts was less tense and less

stressful. [As] people normally tend to compare their grades and benchmark themselves in the

class hierarchy.” Thirdly, that assessment feedback often has a negative impact, particularly

on students with low attainments who are led to believe that they lack academic abilities and

are not able to learn. Students who were usually ranked on a lower band for GP felt “relieved

that [they] did not get a bad grade because there wasn’t a grade.”

The strategy of giving detailed feedback and withholding grades helps to direct

student learning by focusing on what they can improve. It similarly reduces competition

because the basis for comparison is removed, consequently preserving students’ self esteem.

Giving grades in this case serve to impair a student’s ability to learn at that moment, because

a grade is likely to be perceived by the student as indicating their personal ability or worth as

a person in relation to others (Gibbs and Simpson, p.8). In response to a question about

whether written feedback sensitizes them to their mistakes, a student agreed that “the

comparison of grades among students occur less often, allowing me to improve by focusing

on the comments, instead of trying to measure my grades against the others.” Feedback given

on its own is more likely to be perceived as a comment on what has been learnt rather than a

reflection on what the student has not yet mastered. Obtaining a fail grade only serves to

accentuate and highlight what the student has failed to learn or understand. It is reported by a

student that: “the education opposes this [using written feedback to improve work] as at the

end of the day, all that counts is the grade you get and not how much you improved.” Indeed,

the educational system has to exist to “encourage learning” rather than to “measure failure”

(Wooten 2002). It is with the basic aim of setting out an assessment practice that fulfils the

most fundamental premise of any education system – to facilitate learning, that this paper

seeks to achieve.

-5-
WHAT IS QUALITATIVE ASSESSMENT?

Qualitative assessment involves giving detailed feedback on a students’ work, by

focusing on specific areas of improvement. Also known as “formative assessment”, the

effects of such assessment laid out by Crooks (1988) include:

a) Reactivating or consolidating prerequisite skills or knowledge prior to introducing

new material.
b) Focusing attention on important aspects of the subject
c) Encouraging active learning strategies
d) Giving students opportunities to practice skills and consolidate learning
e) Providing knowledge of results and corrective feedback
f) Helping students to monitor their own progress and develop skills of self evaluation
g) Guiding the choice of further instructional or learning activities to increase mastery
h) Helping students to feel a sense of accomplishment.

The focus lies in engaging students in the given feedback, thereby expanding the influence

and role of feedback on learning, rather than just ranking.

Qualitative feedback should aim to motivate students, or to help them “to feel a sense

of accomplishment” (Crooks, 1988). Feedback given in the absence of grades helps to

motivate students as “a poor grade may damage a students’ ‘self efficacy’, or sense of ability

to be effective (Gibbs and Simpson, 2002). Ruth Butler (1988) argues against “ego

involving” activities, particularly assessment that directly impinges upon a students’ sense of

self worth. There is after all a psychological mechanism which results in pupils being more

interested in their own and their friends’ grades than in learning (Butler, 1988). Numerical or

letter grades are thus “ego involving” in that they can either motivate or de-motivate a

student. In both scenarios the grade is seen as a summative judgment, thereby closing off

opportunities for learning.

In accordance with the effects of assessment suggested by Crooks, qualitative

feedback should thus open up opportunities for learning by firstly, consolidating knowledge

that was previously taught. Feedback should focus on what the student has managed to

master and deliver effectively in his GP essays, before it can be further expanded to introduce

-6-
new points or content to further a student’s learning. Secondly, a teacher's feedback can focus

attention on important aspects of a GP essay when a marker delineates very clearly, for

example, the student’s use of topic sentences and thesis statements, or specifically pointing

out certain flaws in an argument. Thirdly, an active learning strategy can be implemented by

asking students to rewrite certain paragraphs or restructure arguments in follow up pieces of

work. Subsequently, teachers are also guided to see how they can more effectively plan

“further instructional or learning activities” to assist a student in mastering essential aspects

of the General Paper. It may be true that the formulation of lessons based on a student’s

needs, will always take place in a GP classroom, notwithstanding the exercise of omitting

grades and giving more detailed feedback. In this case, striving to give more detailed on

specific areas for improvement also foregrounds for the marker or teacher specific blind spots

in their students’ learning, facilitating the creation of lessons better suited to the varying

needs of the class. When students are attenuated to their areas for improvement, they too are

able to seek out opportunities for learning on their own. Fourthly, students can develop

evaluative skills that help them to match comments given by the marker to assess their own

work. In short, the foregrounding of specific and directed feedback empowers the student to

evaluate his own mistakes, correct them, and improve on his writing by working based on the

feedback given.

METHODOLOGY

Sample

The authors, tutors of College 1 (C1) General Paper (GP) classes, each had two

classes of differing abilities participating in this project – A1, A2, B1, B2 – where A1 and A2

were taught by one tutor, and B1 and B2 taught by the other tutor. A1 and B1 were the classes

which were stronger in their linguistic abilities, whereas A2 and B2 were classes which were

-7-
weaker. For a more representative study, one stronger class (A1) and one weaker class (B2)

from each tutor were chosen for the project, whereas the other two classes – A2 and B1 – were

taken as the control. All students had about the same amount of experience of around five

months in a GP classroom. The test group consisted of 49 students, whereas the control group

consisted of 52 students.

Procedure

Students have to sit for 2 GP papers in the G.C.E. ‘A’ Level examinations – Paper 1

consists of 12 questions and students are required to write a 500 to 800-word essay of an

argumentative or expository nature, while Paper 2 is a comprehension paper where students

are tested on their understanding of either one or two passages. As the GP essay reflects

sustained effort on the students’ part, it was chosen as the basis for qualitative assessment.

Over a 3-month period, all four classes were given 2 standardised essay assignments,

with the same quality of comments. The only difference was that the test group – A1 and B2 –

were not shown their numerical or letter grades, whereas the control group was given both

quality comments and grades on their marked scripts. However, the numerical grades of the

test group were recorded by the tutors for school administrative purposes. Essays for all

students were marked with the same level of rigour and the same quality in feedback. All

students were also familiarised with the Cambridge grade descriptors typically used by their

examiners.

After the 3-month period, students had to sit for their end-of-year examinations in

October, marked by other GP tutors, who provided an unbiased, objective grading of all four

classes involved. The results from the promotional examinations were then compared to that

of the mid-year examinations held in July, before the start of the project, and the percentage

change in grades was regressed against the results of the mid-year examinations and its

-8-
square in percentage terms, along with dummy variables for each class and for foreign

students.

The equation for the regression is as follows:

Percentage Change = β0 + β1(Grade Base) + β2(Grade Base)2 + β3(Foreign)

+ β4(Class A1) + β5(Class A2) + β6 (Class B2)

The grade base refers to the results students received in their mid-year examinations. The

square of the results in percentage terms was added to check the intuition that the project

would have a smaller effect at a higher base grade. Class B1 was held as the basis of

comparison for all three other classes. This means that the grade improvement of other

classes involved in the project will be benchmarked against B1, and it is reflected in the

dummy variables, Class A1, A2 and B2. The grades of foreign students are held constant to

ensure that any marked improvement in their grades will not skew the average improvement

of the class.

If the project has effected any change in the results of students, classes A1 and B2

should reflect a significantly higher coefficient after the regression analysis.

RESULTS AND FINDINGS

The results of the regression reflected that the test group registered a significant

improvement, with a 12 to 13% increase in percentage points over the control classes. Table

1 presents the results of the regression.

Table 1

Regression of Percentage Change in Students’ Results

Variable Coefficient t-Statistic

Class A1 0.133628 3.466323

Class A2 0.071649 1.877673

-9-
Class B2 0.126083 3.089265

Foreign Students 0.018870 0.339566

Grade Base 0.132590 3.131901

(Grade Base)2 -0.001824 -2.608105

The ‘coefficient’ reflects the percentage increase in the students’ marks after the project. The

‘T-Statistic’ reflects the significance of the results – the higher the value of the t-statistic, the

more significant the results of the regression. The threshold for significance is usually pegged

at 1.645. A highly significant value for the t-statistic is 1.96. High t-statistic values of 3.47

and 3.09 for Class A1 and B2 respectively reflects that the improvement in the students’

grades is highly significant.

Impact on test group – classes A1 and B2

Class A1 and B2 show percentage increases of 13.4 and 12.6 percentage points above

B1, and the result is highly significant. If, for example, students in B1 have a percentage

increase by 1% on average, those in B2 would be expected to do so by 13.6%. This is due to

the characteristics of the two classes, of which the common characteristic is that of the

project.

Other characteristics of the classes, for example different tutors and class

environment, caused the divergence in the coefficients for A1, at 13.36 percentage points, and

B2, at 12.61 percentage points, of about 0.75 percentage points. While these characteristics

might affect the percentage change, these characteristics in A2 pushed grades up by about 7

percentage points as compared to B1, which is lower compared to increases of 13.4 and 12.6

percentage points for A1 and B2 respectively. This was despite the fact that all four classes

were given similar teaching materials and assignments, as well as the same quality in the

- 10 -
grading and commenting on their essays, over the 3-month test period. This lends strong

support that the project does positively affect students’ grades.

As observed in the findings, the coefficient for the square of the base grade is a

negative value, which indicates that the higher the base grade, the smaller the impact of this

project. Students with better initial grades saw lesser score improvements than those with

poorer initial grades, which is intuitively correct because it becomes increasingly difficult to

improve by a large margin if a student already has a strong foundation and is doing well in

the subject. This corroborates the validity of the regression analysis, lending more support to

the value of this project.

Translation into real mark increase

Table 2 presents an example of how this increase in percentage points is translated

into real marks in the GP examination. Taking two students, one from B1 and the other from

B2, and assuming they receive the same grade in the mid-year examinations, the student from

B2 will perform around 12 percentage points better than that from B1. That is to say, if the

student from B1 were to obtain 25 marks in his mid-year examinations and improves by 1 real

mark in the end-of-year examination, he has improved by 4 percentage points, but his peer

from B2 will obtain a 16 percentage point increase. In real marks, this could mean an increase

of 1 real mark in the former, but 4 real marks in the latter. This is almost an entire jump in the

letter grade of a student in the test group.

Table 2

Translation of Percentage Point Increase into Real Marks

Class B1 Class B2

Mid-Year Examination 25 25

End-of-Year Examination 26 29

- 11 -
Absolute Increase in Marks 1
4

Percentage Increase (1/25)
4% + 12% = 16%

4% 16%

Limitations

As this was a small-scale study, the sample size was restricted to a total of 102

students in four different classes, and while they had differing linguistic abilities, it was not a

marked difference. Thus, it is difficult to project the findings to ascertain the efficacy of

qualitative feedback to students with a more diverse linguistic background.

Also, although the time period of three months was sufficient in reflecting a change in

the quality of the work produced by students, it may not reflect the true extent of the impact

of having qualitative assessment in GP.

There is also the issue of having different markers, not only between the two sets of

classes, but also between the two major (mid-year and end-of-year) examinations. With

different markers, it would be difficult to ensure uniformity in the standards in marking, and

hence it would not be possible to make a clear and unquestionable comparison. Nevertheless,

the initial marking standards of both tutors were standardised by a third party, minimising

any differences in standards within the 3-month test period.

RECOMMENDATIONS

GP Assessment Practices

This study has provided insights into the possible ways our teaching practices could

be modified to better benefit our students. Some of these possible changes would include

familiarising students with grade descriptors, giving feedback geared towards a constructive

- 12 -
improvement of a piece of work, as well as clear and structured opportunities for students to

follow up on the comments they have received.

Ensuring that all our students are familiarised with the grade descriptors for GP, as

provided by Cambridge, would allow them to better understand the demands of the

examination, and if they are meeting the expectations. We need to "make [our] criteria as

explicit as possible [and] explain clearly to [ourselves], [our] fellow markers and [our]

students exactly what standard is expected and rewarded" (Haines, 2004, p. 63). These clear

guidelines will alert students to what they should demonstrate before they are duly rewarded.

Other than examination issues, the grade descriptors are useful in itself, as it documents

concisely what a piece of good writing entails. Hence, students stand only to gain, whether in

terms of their examination skills or writing skills.

When providing feedback on an essay that is meant to teach, and not test, it is

important that comments are constructive and meaningful. The teacher’s feedback should

avoid being imprecise; but instead, identify what has been done well and what still needs

improvement. Instead of feedback which students would find difficult to follow up on, for

example “not clear” or “vague”, suggestions on specific action could be provided, such as

instructing students to provide an explanation of a certain aspect of the issue at hand. Haines

(2004) explains that students "need feedback on whatever they are doing, saying or writing to

help them understand whether it is right or wrong, conforms to expected standards, is

acceptable or exemplary" (p. 19).

Lastly, while there probably already exist opportunities for students to follow up on a

piece of essay that has been marked, such as corrections that have to be handed in, a clearer

reinforcement of the importance follow-up action is encouraged. This will ensure that a piece

of work is not written in vain, but rather, is taken as a learning process. Black et al (2002)

suggested that teachers should specially plan for students to rewrite selected pieces of work

- 13 -
during lesson time, so that "emphasis can be put on feedback for improvement within a

supportive environment [as this can also potentially] change pupils' expectations about the

purposes of classwork and homework" (p.9).

Suggestions for Future Research

There is tremendous potential for future research in the area of qualitative assessment.

Other than improving on the limitations outlined in the previous section by having a longer-

term study, as well as standardising markers, there can also be a larger-scale study across

different pre-university institutions. It would also be advisable for a longer-term, larger-scale

study to collect more data on the background and characteristics of the students involved, so

as to determine other possible factors that could affect the students’ results.

Also, the usefulness of qualitative assessment could be tested in other subject areas

other than for the General Paper.

CONCLUSION

This paper has outlined what qualitative assessment is, and has provided evidence of

the potential improvement students can achieve through adjustments in grading practices. As

Kubiszyn & Borich (1993) expresses,

"Marks have become an accepted and expected aspect of our culture. Students come

to realise very early in their educational careers that they will be graded or marked

depending on their performance in school. Parents of students, having been graded

themselves, realise that the marks a student receives may well affect their child’s

educational, occupational and financial status by opening or closing various

opportunities" (p. 151).

- 14 -
With considerable meaning and such high premiums placed on marks and grades in our

society by parents, students and educators alike, it is no surprise that the scope for

improvement in a student's work is eclipsed.

Marks no longer simply perform their purpose of "[providing] feedback about student

achievement" (ibid, p. 151), and they do cause some amounts of stress to students, which

could be unnecessary. Some of the students involved in the project reflected that "the marks

would only be [a] comparison of grades with friends" and simply "adds to stress". On the

contrary, having only written feedback without numerical grades "makes [the students] less

marks-oriented" and allows some students to "[feel] less stress [and] less comparison with

peers". However, it should be noted that many still feel uncomfortable without any concrete

and unequivocal indication of their abilities, yet this discomfiture can result in a positive

outcome as students have agreed that they are more sensitised to their mistakes because of the

lack of grades. Students reflected that "[not including grades] is a good way to distract

[students] from focusing on the importance of marks" and instead, they "tend to take the

comments more seriously".

Hence, with the students' learning interests in minds, it is time for educators to reflect

if assessment practices can be improved on. The purpose of this project is, ultimately, to

determine if there are opportunities for teachers to "sharpen and refine their methods of

assessing and responding to children’s efforts as they practise their writing skills [in the hope

that they] write better" (Haines, 2004, pp. 18-19). The highly significant improvement in the

students' results is indeed an encouragement for any GP tutor or educator to explore a

different assessment practice in the classroom.

- 15 -
ACKNOWLEDGEMENTS

The authors would like to thank Mr. Jeffrey Low, formerly of the Curriculum Planning &

Development Division (CPDD) in the Ministry of Education (MOE), for spearheading this

project, and without whom this study would have not been possible. His kind guidance and

encouragement is a source of inspiration to us. We would also like to thank Mr. Tan Wah

Jiam, the Principal Consultant for English in Hwa Chong Institution (College), for his

relentless support for our cause. Lastly, the unwavering support and cooperation of our

students who undertook this study alongside with us, and whose honest and frank feedback

have contributed in no small way to this research paper.

- 16 -
REFERENCES

Black, P., Harrison, C., Lee, C., Marshall, B., & Wiliam, D. (2002). Working inside the black

box: Assessment for learning in the classroom. London: nferNelson.

Broad, B. (2003). What we really value: Beyond rubrics in teaching and assessing writing.

Utah: Utah State University Press.

Butler, R. (1988). Enhancing and undermining intrinsic motivation: The effects of task-

involving and ego-involving evaluation on interest and performance. British Journal

of Education Psychology, 58, 1-14.

Crooks, T.J. (1988) The impact of classroom evaluation practices on students. Review of

Educational Research, 58, 4, pp438-481.

Gannon, P. (1985). Assessing writing: Principles and practice of marking written English.

London: Edward Arnold.

Gibbs, G., & Simpson, C. (2002). Does your assessment support your students' learning?

Journal of Learning and Teaching in Higher Education.

Haines, C. (2004). Assessing students’ written work: Marking essays and reports. London:

RoutledgeFalmer

Kubiszyn, T., & Borich, G. (1993). Educational testing and measurement: Classroom

application and practice (4th ed.). New York: HarperCollins College Publishers.

Miller, C.M.I., & Parlett, M. (1974). Up to the mark: A study of the examination game.

Guildford: Society for Research into Higher Education.

Sambell, K., and McDowell, L. (1998) The construction of the hidden curriculum: messages

and meanings in the assessment of student learning. Assessment and Evaluation in

Higher Education, 23, 4, pp391-402.

- 17 -
Snyder, B.R. (1971) The Hidden Curriculum. Cambridge, MA:MIT Press

Spandell, V., & Stiggins, R. J. (1990). Creating writers: Linking assessment and writing

instruction. Harlow: Longman.

Tinto, V. (1993) Leaving College: rethinking the causes and cures of student attrition (2nd

Edn.) Chicago: University of Chicago Press.

Wooten, S. (2002) Encouraging learning or measuring failure? Teaching in Higher

Education, 7, 3, pp353-357.

Yorke, M. (2001) Formative assessment and its relevance to retention. Higher Education

Research and Development, 20, 2, pp115-126.

- 18 -