NOTICE

Equalisation of scores in SSC Examinations

Staff Selection Commission has been conducting various examinations in
multiple batches because of large number of candidates and difficulties in
getting adequate educational institutions for holding the examinations in a
single batch. For perhaps the first time in its history, the number of applicants
in a single examination exceeded one million when the Combined Higher
Secondary Level Examination, 2010 for the recruitment of Lower Division
Clerks and Data Entry Operators, elicited response from over 16 lakh
candidates, with app. 21% of them applying online. This would require the
Examination, rescheduled on 27

& 28.11.2010 (in view of Common Wealth
Games), to be held in at least three batches. The Commission, with the help of
experts, has striven to construct question papers of comparable difficulty level.
While such an exercise is theoretically possible, in practice it is impossible to
have two or more question papers of identical difficulty levels. Even if the
difficulty levels of question papers vary slightly, candidates taking more
difficult papers may be at a disadvantage viz-a-vis others. Therefore, there is a
need for equating of the marks in examinations involving multiple batches and
question papers.

2. The Commission had examined the views of an Expert Group,
constituted by it with the approval of Government of India in 2009, on this
issue. The Commission had placed before the Expert Group that the technique
to be followed for equating should be transparent, easily comprehensible to
the candidates, acceptable to experts and prove itself in Courts of Law if and
when challenged. This was accepted by the Expert Group which further
advised the Commission to place a paper on the technique on its website for
adequate time, give publicity to such placement through the media, invite
comments, observations and suggestions and decide on adopting the
technique thereafter.

3. Accordingly, the Commission places this paper inviting views, comments
and suggestions from academicians, parents, candidates and also stakeholders
on or before 31.10.2010. Such views, comments and suggestions, if any, may
be addressed to Smt. Vandana Sethi, Deputy Secretary, Staff Selection
Commission, Block-12, CGO Complex, Lodhi Road, New Delhi-110 003 (e-mail:
vsethi.ssc@gmail.com). The Commission proposes to adopt the
Equipercentile Equating Technique for examinations involving multiple batches
from Combined Higher Secondary Level Examination, 2010 onwards.




















PAPER ON EQUATING TECHNIQUES

1. Introduction:
Building Question Papers based on the same test specification either on
blueprint or content template will not ensure that these tests are equivalent,
specifically based on the difficulty level, content and ability tested. To ensure
fairness in the scoring system when candidates are administered different
Question Papers of the test, an appropriate statistical procedure is needed to
equate the test scores based on the different Question Papers. In other words,
Equating is needed.

2. Large-scale testing programs often require multiple Question Papers to
maintain test security over time and to enable equivalent tests to be
administered without repeating identical items. The outcomes after equating are
well get the same scores in tests in which different question papers were
administered.

3. Equating is a statistical process that is used to adjust scores on
multiple question papers so that scores on the forms can be used
interchangeably. It adjusts for differences in difficulty among Question
Papers that are built to be similar in difficulty and content.

4. Example:
Here is an example of Test of English language in 2 Question Papers (VA1 and
VA2) which consists of 50 items each that would assess the usage of articles
amongst other things.
VA1
1. ___ book that I bought yesterday is interesting.
2. I saw ___ man wearing a red suit.

VA2
1. ___ goat was crossing ___ river.
2. I have been waiting for ___ auto.

After evaluating the Answer Sheets the overall performance of the students in
two different Question Papers from similar background students is found to be
varying with the following mean scores.
Mean scores are:
VA1 14.5
VA2 12.5

The number of right answers in VA1 is found to be 2 units more than those
of VA2. Similar variation in mean scores may be found in the case of all
the questions included in the test. Therefore, there could be a need for
adjustment of scores of each test so that scores from different question
papers can be used interchangeably

5. Process followed in Equating

The purpose of equating should be clear.
The Examination should have multiple question papers on the same
subject.
Alternate question papers are set in accordance with items of the same
content, difficulty level and ability tested.
Data of responses to items/questions should be collected.
The responses should be in the form of (A, B, .., X) or (Yes/No)
etc. for objective type tests.
Equating requires that a choice be made about what type of relationships
between different question papers is to be estimated. For example, this
may involve decision on whether to use linear/non linear methods of
equating.
Descriptive statistics to obtain mean, standard deviation and/or the
percentile rank of scores may need to be generated after evaluation of the
papers.
A statistical equating method should be carefully selected for adoption.
The results of equating should be evaluated carefully. .

6. Different Methods of Equating
a) Median/Mean Equating
b) Linear Equating (Based on mean and S.D.)
c) Equipercentile Equating
d) Equating using Item Response Theory

a) Median/Mean Equating
Procedure:
In the given example, two question papers of Verbal Ability Test are
administered on two similar groups of student(VA1 and VA2)
Mean score of VA1 = 14.5 & Mean score of VA2 = 12.5
Difference between mean scores = 14.5 12.5 = 2
Mean equating involves the addition of a constant (2 = difference in the
mean scores) to all the raw scores on VA2 to find equated scores on VA1

b) Linear equating (Based on Mean and S.D.)

This involves equating the two tests using the mean and standard deviation
of each question paper. If x represents a score on the VA2 (in the given
example) and y represents a score on the reference form (VA1), then x and y
are equivalent in a group of examinees if:




Procedure:
For adjusting the new scores according to the reference form the formula is:


X(VA2) Y(VA1)
Mean 12.5 14.5
S.D. 4.5 4

Y adjusted to X:



c) Equipercentile Equating

The method of equi-percentile equating involves fixing up a common percentile
for all forms (it is observed that same test score in each of the question papers
gives rise to different percentiles) and then finding the different scores in these
forms in terms of number right scores. Thus a score of 17.2 in series -1 score is
equivalent to 14.5 in series-2.This in essence is equipercentile equating.

Procedure: Given below are two sets of scores, series 1 and series 2, which are
multiple question papers of the same test. As far as possible, the no. of
examiners in either series should be equal. Their percentile ranks are calculated
from the grouped data using SPSS or Excel.

y mean(y) x mean(x)
SD(y) SD(x)

=
(x mean(x))
y SD(y) mean(y)
SD(x)
(
= +
(

10 12.5
4 14.5 12.28
4.5

(
= + =
(




series 2
score percentile rank
1 0.00%
2 5.50%
3 11.10%
4 16.60%
5 22.20%
6 27.70%
7 33.30%
8 38.80%
9 44.40%
10 50.00%
11 55.50%
12 61.10%
13 66.60%
14 72.20%
15 77.70%
16 83.30%
17 88.80%
18 94.40%
19 100.00%
series 1
score percentile rank
3 0.00%
4 5.20%
5 10.50%
6 15.70%
7 21.00%
8 26.30%
9 31.50%
10 36.80%
11 42.10%
12 47.30%
13 52.60%
14 57.80%
15 63.10%
16 68.40%
17 73.60%
18 78.90%
19 84.20%
20 89.40%
21 94.70%
22 100.00%


The above graph shows both percentile ranks of series 1 and 2 are put together
in order to enable the percentile score of one raw mark of series-1 to be slightly
different from the raw mark score of series-2. The percentile rank of 17 in
series -1 is 88.8% and in series -2 the same score of 17 has a percentile of
93.6%.
In the method of equi-percentile equating, a cut-off of a percentile rank to
qualify for selection is to be fixed, say 75 percentile and this common 75
percentile has a raw score of 14.5 in series-1 and 15.26 in series-2. Therefore,
anyone who gets more than 14.5 in series -1 and 15.26 in series 2 will qualify
and the equi-percentile is 75.
SSC PROPOSES TO USE THE EQUIPERCENTILE METHOD IN VIEW OF ITS
SIMPLICITY


d) Equating using Item Response Theory (IRT)

Basic Concepts

Ability (): Ability is measured on the scale (-3 to +3)
Item difficulty (b): The item difficulty of an item and the ability of the
test taker are on the same scale. it is invariably taken as the middle point
of the item characteristic curve where the curve shows a tendency of
contra flexure that is bending in the opposite directions (-3 to +3)
Both of these are on the same scale and along the x-axis
Probability of getting the correct answer (0 to 1) on the y-axis
This is a fraction like 0.75which means if a person of particular ability
say ( any ability between -3 and +3) attempts 100 times the same item
75 times he will get it right and 25 times he will get it wrong.

Fred Lords Model

- +
-3 + 3
- 4 + 4

tan = a Item
discrimination
Item Difficulty b
Guessing c
Pi ()
(probabilit
y of
getting
answer
right on
any item I
with ability
)
0.5
+1.0
1-c

Types of equating in IRT
i) Horizontal equating
ii) Vertical equating
Horizontal equating is equating scales of about the same ability
Vertical equating is equating scales across completely different levels of ability
THE COMMISSION DOES NOT USE IRT IN ITS EXAMINATIONS.