Test Data Analysis

Compiled by

P.J. Smit

Computer-based Assessment

CIA 722

Department of Curriculum Studies

Faculty of Education

University of Pretoria
Test Data Analysis

P.J. Smit
Acknowledgements

Many thanks to Prof. JG Knoetze for supplying the needed raw data to
complete the report. A big thank you to Prof. Tinus Kuhn for guiding the
report compiler in completing the report.
Table of Contents
Chapter Page
1.1 Abstract 5
1.2 Introduction 6
1.3 Nature of report 6
1.4 Statement of 6
purpose
1.5 Aims and objectives 6
1.6 Test Analysis 7
1.7 Central Tendency 9
1.8 Results and findings 10
1.9 References 11
2.1 Appendix 12
Abstract

In this report the researcher tried to manipulate data into item analysis and
descriptive statistics in order to asses his objective test item namely multi-
choice. The researcher measured the difficulty index, discrimination value
as well as reliability coefficient. In the process the also calculated the Mode,
established the mean as well as median. The completed literature study
helped to narrow focus as well as exposed the researcher to new ways to
convert data to visual representation. The report contains findings of the
analysis as well as a summery of all the data.
Introduction
Before we can use the data provided it is crucial that the data is ordered
and grouped accordingly. The data is read into Microsoft Excel where
formulas make it possible for us to make the most of the data. The data is
managed much easier for it is now grouped and is manipulated to such an
extend that the researcher can draw conclusions and graphs from what was
once just numbers.

Nature of report
Descriptive statistics are used to describe the basic features of the data in
a completed study. They provide simple summaries about the sample and
the measures. Together with simple graphics analysis, they form the basis
of virtually every quantitative analysis of data.
Descriptive Statistics are used to present quantitative descriptions in a
manageable form. Descriptive statistics help us to simplify large amounts
of data in a sensible way. Each descriptive statistic reduces lots of data into
a simpler summary.http://www.socialresearchmethods.net/kb/statdesc.php)

Statement of purpose
Why would we compile ‘n report like this? An analytic report like this would
be great for a teacher or instructor to have, if that person would like to go
forward in life and improve their questions. To have an objective look at
your work is critical for improving the quality of your work. An item analysis
provides information useful for improving the quality and accuracy of
multiple-choice tests. This is extremely valuable in the classroom and
practice. It is always good to know the quality of your questions. Your
questionnaires can always be assessed to be perfected.
http://www.socialresearchmethods.net/kb/statdesc.php)

Aims and Purpose/ aims and objectives

The aim now is to find out witch questions are too easy or to difficult. Lets
look at the table below.
Amount Correct

25
Amount of Learners correct

20

15
Correct questions
10

0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Questions 1 - 20

Figure 1.1

It is very easy to identify right away the very easy questions. Question 1
and question 2 goes hand in hand. More than 20 learners answered the
questions correct. Questions 11 and question 16 is also very easy, because
most of the learners answered correct. Question 14 and question 15 to a
large extend answered correctly. All data used in this report used to draw
up this charts are listed in the appendix.

From figure it is clear that question 4, question7, question 10 and question

18 are amongst the ones that were answered the most incorrectly.
Wrong answers

25
Amount of Learners wrong

20

15
Wrong answers
10

0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Questions 1 - 20

Figure 1.2

A Conclusion based on the fact and info of Figure 1.1 and Figure 1.2 it is
clear that there are some questions that are very easy and very difficult
respectively.

Tests Analysis
Let us look at the data more analytically. The data can be converted to a
difficulty index p(Figure 1.3). The difficulty index takes into consideration
the amount of learners that got a question right. It also works out the
percentage of learners who got it right. It becomes more clear witch
questions was answered correctly the most. Questions 11 & 16 have a
relatively low difficulty index because more or lest 90% of the class got the
answer correct. The questions need to be replaced.

Question dificulty index

100

80
Percentage

60
question dificulty index
40

20

0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Learners
Taking the analysis a step further.
To get to a conclusion the researcher make use of a discrimination index
(Figure 1.4). The discrimination index tells the test compiler whether he
setup a good or a bad test. He looks at the data results and put them into a
formula. That is determined by subtracting the amount of learners who
answered correctly in the upper group from those in the lower group. Take
that answer and divide that by the amount learners in the upper and lower
group.
Discrimination Index

1.2

0.8
D value

0.6 Discrimination Index

0.4

0.2

0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Question 1-20

In our case the reason for the discrimination index can be one of the
following factors: Miskeying, Guessing or the ambiguity factor.
Central Tendency.

0.02
0.018
0.016
0.014
0.012
0.01 Series1
0.008
0.006
0.004
0.002
0
0 5 10 15 20

Figure 1.5

From figure 1.5we got the following information regarding central tendency.
What is central tendency? The central tendency of a distribution is an
estimate of the "center" of a distribution of values. There are three major
types of estimates of central tendency namely Mean, Mode and Median.
(http://www.robertniles.com/stats/dataanly.shtml)

Mean Mode Median

13.04 13 13

The mean is the total average of the data where you add up all the
numbers and then divide by the number of results. The mean for this
specific report looking at the data was 13.04.

The Median is the score found at the exact middle of the set of values. One
way to compute the median is to list all scores in numerical order, and then
locate the score in the center of the sample. In the case of the table below
the median will be 13.

The mode is the most frequently occurring value in the set of scores. To
determine the mode, you might again order the scores as shown above,
and then count each one. The most frequently occurring value is the mode.
In our test data the mode will be 13.
Results and findings
When looking at the given data and the results based on the test data we
can make some necessary findings. Test items compiled in the given test
was not of a very good standard. It is clear that there were no consistency
on the difficulty level of the questions. Some questions were very easy and
some were totally to complex and difficult.

Conclusion & Recommendations

One should take into consideration that not all learners are on the same
cognitive level. Some might be on a level of brilliance and some barely
understanding the new concept or language.

The process of creating the perfect Test item questionnaire would be the
ideal. Although this might sound impossible, this could be a timorously
process and time consuming not the least, but with all the resources to our
advantage any good educator must take the time to do this task.
References

Research methods knowledge base; Accessed on 5 April 2008;

http://www.socialresearchmethods.net/kb/statdesc.php

RobertNiles.com Statistics Help for Journalists; Accessed on 4 April 2008;

http://www.robertniles.com/stats/dataanly.shtml

Tushar Mehta Copyright © 2000-2006; Accessed on 5 April 2008;

http://www.tushar-mehta.com/excel/charts/normal_distribution/
Appendix