ANALYZING AND USING TEST ITEM DATA

Item-analysis techniques are among the most valuable tools classroom teachers use in improving the quality of their classroom tests. Purpose and Elements of Item Analysis As pointed out by Payne (1992 ! item analysis is conducted for the follo"ing purposes# $o select the best available items for the final form of the test % $o identify structural or content defects in the items % $o detect learning difficulties of the class as a "hole% and $o identify the areas of "ea&nesses of students in need of remediation

3 main elements in an item analysis ('o"nie ( )ealth! 19*+ ,-amination of the difficulty level of the items 'etermination of the discriminating po"er of each item ,-amination of the effectiveness of distractors in a multiple choice or matching items $he difficulty level of an item is &no"n as inde- of difficulty. .y index of difficulty is meant the percentage of students answering correctly each item in the test. /n the other hand ! index of discrimination refers to the percentage of high scoring individuals responding correctly versus the number of low scoring individuals responding correctly to an item(Payne! 1992 .$his numerical inde- indicates ho" effectively an item differentiates bet"een the students "ho did "ell and those "ho did poorly on the test(0redo!19*1 Preparing Data for Item Analysis Preparing data for item analysis of a classroom test involves counting the number of students in high and lo"-scoring groups responding correctly. Steps that should be followed in preparing the data for item analysis ( Payne1992% 0almorin! 199+% /rionodo ( Antonio! 19*+ 1. Arrange test scores from highest to lo"est. $his ran&ing is based on the student2s total score on the test. 2. 3et one third of the papers from the highest scores and the other third from the lo"est scores. $hese t"o e-tremes sets of e-amination papers are the criterion groups. $he first is the upper group and the latter is the lo"er group. $he middle group is not used in the analysis of test items. 1. 4ecord separately the number of times each alternative "as chosen by the students in both groups. +. Add the number of correct ans"ers to each item made by the combined upper and lo"er groups. 5. 0ompute the inde- of difficulty for each time !follo"ing the formula#

6here I'7 8 Inde- of difficulty 940 I'7 8 $: 940 8 no. of students responding correctly to an item $: 8 total no. of students in the upper and lo"er groups

;. 0ompute the inde- of discrimination! based on the formula#

0<-0= I' 8 9:3

group

6here I'9 8 Inde- of discrimination 0< 8 no. of correct responses of the upper group 0= 8 no. of correct responses of the lo"er 9:3 8 no. of students per group

ITEM 1 2 1 + 5

G !"P A ) 2> =2> ) 2> =2> ) 2> =2> ) 2> =2> ) 2> =2> 1 1> > > + 1> 1 2 15 1

A#S$E S % & D 1+ ? > 1 * 2 1 + 2 1> 2 1 1* 9 + + + 1> 2 + 1 > 2 * + + 1> + 1 5

T!TA' #!( !) &! E&T A#S$E S 21 2? 1> 1+ 1;

DI))I&"'T* I#DE+ 52.5 ;?.5 25.> 15.> +>.>

,-' ? 9 ; ; 1+

DIS& IMI#ATI!# I#DE+ >.15 >.+5 >.1> >.1> >.?>

"sing Information about Inde. of Diffi/ulty $he difficulty inde- of a test item is important because it tells a teacher something meaningful about the comprehension of or performance on! material or tas& contained in an item. $he foregoing points to one thing --- that the higher the value of the difficulty inde-! the easier is the item. $his is because difficulty inde- represents the percentage of the total no. of students ans"ering an item correctly. $hus! there is an inverse relationship bet"een the magnitude of the inde- and "hat it purports to represent. 7or an item to be considered a good item! its difficulty inde- should 5>@. An item "ith 5>@ difficulty inde- is neither easy nor difficult. 'ifficulty inde- of a test can be interpreted "ith the use of the follo"ing table of equivalents
ange 2>( belo" 21-+> +1-;> ;1-*> *1( above Diffi/ulty le0el Cery 'ifficult 'ifficult Average ,asy Cery easy

"sing Information About Inde. of Dis/rimination $he inde- of discrimination! as already pointed out! tells a teacher the degree to "hich a test item differentiates the high achievers from the lo" achievers in hisAher class. 6ith regard to item discrimination! a test item then may have positive or negative discriminating po"er (0redo! 19*1%Payne!1992 . An Item has a positive discriminating po"er "hen more students from the upper group got the right ans"er than those from the lo"er group. 0onversely! "hen more students from the lo"er group got the correct ans"er of an item than those from of the upper group! the item has a negative discriminating po"er. $here are! ho"ever! instances "hen an item has a Bero discriminating po"er. $his happens "hen there is an equal number of students from both the upper and lo"er groups "ho got the right ans"er to a test item. Items "ith negative and Bero discriminating po"ers have to be discarded.

Padua and :antos (199* recommended the use of the follo"ing table of equivalents in interpreting obtained indeof discrimination.
ange .+> ( above .1> - .19 .2> - .29 .>9 - 19 1erbal Des/ription Cery 3ood Item 3ood Item 7air Item Poor Item

E.amining Distra/tor Effe/ti0eness An ideal item is one that all students in the upper group ans"er correctly and all students in the lo" group ans"er "rongly. Doreover! the responses of the lo"er group have to be evenly distributed among the incorrect alternatives. $here are t"o procedures in e-amining the effectiveness of the distractors in a multiple-choice item. $he first procedure requires that the ans"ers to the different distractors be counted for the upper and lo"er groups. 3ood distractors are those chosen more frequently by students from the lo"er group. 6hen a particular distractor is selected more frequently by those from the upper group! the teacher has to revise it. $he analysis of distractors may also reveal some chosen by no one in either of the t"o groups of students. :uch distractors should therefore be revised to ma&e them more useful. An illustration of the foregoing analysis is sho"n belo".
G !"P <pper =o"er A > > % 1 ? esponses & D 1 > 12 ? E 5 ;

Item 1

' 8 correct ans"er

An e-amination of item 1 reveals the no one from the upper and lo"er groups chose distractor A. It has to be revised or changed. 'istractor 0 needs to be revised to ma&e it more popular "ith those in the lo"er group. Another method of analysing the effectiveness of the distractors is by determining the mean score of the students "ho respond to each distractor! as "ell as that of those "ho choose the correct ans"er. As pointed out by 'o"nie and )eath(19*+ ! good distractors have lo"er mean scores than those related to the correct response. De0eloping an Item Data )ile $eachers need to develop a file of test items. 4ecording items on 5E - *E inde- cards and accumulating data on their difficulty and discrimination over several administrations can contribute so much to the improvement of classroom tests. Payne(1992 identifies the follo"ing advantages associated "ith development of an item data file# ,ncouraging teachers to underta&e an item analysis as often as practical% Allo"ing for accumulated data to be used to ma&e item analysis more reliable% Providing for a "ider choice of item format and obFectives%

 7acilitating the revision of items% 7acilitating the physical construction and reproduction of the test% and Accumulating a large pool of items as to allo" for some items to be shared "ith the students for study purposes. 'imitations of Item Analysis 6hile item analysis is underta&e for the purpose of improving test items! its techniques cannot be used "ith essay items. $he scores obtained by students from the upper group can be compared "ith those of belonging to the lo"er group "ithout using the inde- of discrimination and difficulty. $he type of item! format! and reading level are among the many variables "hich can influence test and item difficulty. $eachers should be cautious about "hat damage may be due to the table of specifications "hen items not meeting the criteria are deleted from the test. $hese items have to be re"ritten or replaced.