1-1
System Requirements
Personal computer running
Microsoft Windows 95/98/NT/ME/or XP
2 Mb free RAM
Overview
ITEMAN is one of the analysis programs that comprise Assessment Systems Corporation’s
Item and Test Analysis Package. ITEMAN analyzes test and survey item response data and
provides conventional item analysis statistics (e.g., proportion/percentage endorsing and
item-total correlations) for each item, in order to assist in determining the extent to which
items are contributing to the reliability of a test and which response alternatives are
functioning well for each item. In addition to item-level statistics the ITEMAN program
also provides statistical indicators on the performance of the test as a whole (e.g., mean,
standard deviation, reliability, median p-value). See Chapter 3 for a complete description of
the ITEMAN program and information on the specific statistical indicators available with
ITEMAN.
This manual will tell you how to install and run the ITEMAN program. Although Chapter 3
describes each statistic that is computed, it assumes some knowledge of the proper use and
interpretation of these statistics. You might wish to refer to a basic textbook on test theory
and methods such as Crocker and Algina (1986) for a brief review if the terms in Chapter 3
seem unfamiliar to you.
Table of Contents
To view the Table of Contents for this manual, select the Bookmark tab on the left side of
the Acrobat window. The bookmark entries are hyperlinks that will take you directly to any
section of the manual that you select.
References
Crocker, L., and Algina, J. (1986). Introduction to classical and modern test theory. New York:
Holt, Rinehart and Winston.
1-1
Chapter 2:
Formatting Examinee Data
Overview
The ITEMAN program requires that input data be formatted in ASCII (text-only) files. This
can be accomplished through the use of the ITEMAN for Windows text editor, Notepad, a
word-processing editor that produces true ASCII output, or a program written specifically to
format your data. All the data to be included in the analysis must be contained in a single
input file. Up to 750 items can be included in a single analysis, while the number of
examinees is virtually unlimited.
2-1
The Control Line
The first line of the data file must contain the following data in the columns specified:
Column Data
1-3 Number of items for which responses are recorded for each examinee
(maximum is 750)
4 Blank
5 Alphanumeric code for omitted responses
6 Blank
7 Alphanumeric code for items not reached by the examinee
8 Blank
9-10 Number of characters of identification data recorded for each examinee
(maximum is 80)
In columns 1-3, you must enter the number of items that are included in the file. This number
must be right-justified: The “units” go into column 3, the “tens” in column 2, and the
“hundreds” in column 1. A maximum of 750 items can be handled by ITEMAN Figure 2-1
shows a data file with 30 items to be analyzed.
Column 5 must contain the alphanumeric code for items that the examinee has omitted.
This can be a digit larger than the number of alternatives, a letter, or some other character
including a “blank.” For example, it might be “9” for a five-alternative item, an “O” for
omitted, or a period. Column 7 must contain the alphanumeric code for items that the
examinee did not reach and therefore did not have a chance to answer. Like the omission
code, it can be a digit larger than the number of alternatives or any other character. In
Figure 2-1, the letter “O” indicates an omitted item, and “N” indicates a not-reached item.
For dichotomously scored items, ITEMAN makes no distinction between items that are
omitted and items that are not reached; both are scored as incorrect. For rating-scale-type
items, ITEMAN will allow you to specify one of three ways for dealing with missing data
(these options are presented on the Options screen). Both the responses recorded as
omitted and those recorded as not reached will be considered “missing.”
Columns 9 and 10 contain the number of characters at the beginning of each examinee’s
data record used for identification. As with the number of items, these digits must be right
justified — the “tens” must be in column 9 and the “units” in column 10. The maximum
number of identification characters is 80. If columns 9 and 10 are left blank or if zero
identification characters are specified, examinee identification will not be expected and the
examinees’ responses must begin in column 1 on the data lines. The example in Figure 2-
1 indicates that there are 5 characters of identification for each examinee; in the data lines
(beginning on line 5 of the input file in Figure 2-1), you will note that the examinees are
identified by “EX001” through “EX005”.
2-2
Dichotomously Scored Items
For dichotomously scored items, each keyed response is coded as the character that will be
scored as correct for the item. The key can be specified using the numerals 1 through 9 or
the letters A through I. For example, a “1” means that all responses of “1” will be counted
as correct. For convenience, “A” and “a” have been defined to be equivalent to “1”.
Similarly, “B” = “b” = “2”. This equivalence continues through “I” = “i” = “9”. There is
no letter equivalent to zero. Note, however, that ITEMAN distinguishes between
alphabetic (e.g., “A”) and numeric (e.g., “1”) keyed responses for purposes of determining
how the alternatives for the item will be labeled in the ITEMAN output file. If you want the
alternative/option labels to appear as alphabetic then you should provide keyed responses
for your items that are alphabetic. (See Figure 2-3.)
Multipoint Items
ITEMAN allows you to analyze multipoint as well as dichotomously scored items.
Multipoint items are not dichotomously scored. Instead, two or more possible response
categories are ordered along some scale. Examples are five-point Likert-type rating scales
and the Like/Indifferent/Dislike scales on interest inventories. The response can be
specified using the numerals 1 through 9 or the letters A through I. For convenience, “A”
and “a” have been defined to be equivalent to “1”. Similarly, “B” = “b” = “2”. This
equivalence continues through “I” = “i” = “9”. There is no letter equivalent to zero.
For multipoint items, the permissible values of the key are “+” and “–” (see Figure 2-2).
When an item is keyed “+”, the item score is equal to the number of the response (for
example, “1” or “A” = 1). When an item is keyed “–”, the item score is equal to the
complement of the number of alternatives (for example, “1” or “A” = 5 for a five-alternative
item).
2-3
The Scale Inclusion Codes
The fourth line contains scale inclusion codes, which indicate whether an item should be
included in the analysis. Items coded “Y” are included in the analysis; those coded “N” are
not. In the example shown in Figure 2-1, all the items will be included in the analysis.
Items on the first scale are assigned inclusion codes of “0”, items on the second scale are
assigned inclusion codes of “1”, and so forth. Up to ten scales can be specified using the
digits 0 through 9 as inclusion codes. You can still use “N” to exclude items altogether. A
“Y” will assign the item to scale zero.
Comments
Comments can be placed to the right of the data on any line. There must be at least one
space between the data on any given line and a comment on that line.
2-4
In order for scanned data files to be compatible with ITEMAN you must ensure that the
scanner output file is written in an ASCII format with one line for each examinee and that all
scanned demographic and identification information (e.g., name, student ID, date of birth,
etc.) is located at the beginning of each examinee’s record. You must also ensure that the
scanner output file writes each item response in a single character-width field and that, once
begun, the item responses are continuous.
To use ITEMAN with a scanner-produced (ASCII) output file you must either append the
first four control lines of information to the top of your scan data file, or use the option for
running ITEMAN in batch mode and include the four control lines in the header file used
for batch operation (see Chapter 3). If you decide to append the four control lines to the
top of your scanner data file, the easiest method to accomplish this is to create a small file
containing only the four control lines using your text editor (Notepad, Wordpad) and paste it
onto the top of the scanner file. Note that if you use a word processor, such as Microsoft
Word to create or combine these file, you must save the file as a text (.TXT) file; ITEMAN
will not function properly with a .DOC file saved from Word or other word processor.
2-5
Chapter 3:
Analyzing Items and Tests
Introduction
This chapter describes how to use the ITEMAN for Windows Item and Test Analysis
Program. The program analyzes items and tests composed of from one to ten scales (or
subtests). ITEMAN can also be used to compute and record test scores for individual
examinees.
ITEMAN will allow you to analyze scales containing either dichotomously scored or
multipoint items. Dichotomously scored items are those that can be scored in a binary (0 or
1) fashion. In psychological tests, this usually refers to correct/incorrect scoring. True-false
items are, by their nature, dichotomously scored. Most multiple-choice items are scored
dichotomously. Multipoint items are not dichotomously scored. Instead, two or more
possible response categories are ordered along some scale. Examples are seven-point Likert-
type rating scales and Like/Indifferent/Dislike interest-inventory scales. Multipoint items
differ from dichotomously scored items in that the item responses are not reduced to one of
two possible values.
Running ITEMAN
The ITEMAN Interface
When you exit the ITEMAN installation program, a new group/folder called “Assessment
Systems” will appear in the ProgramManager/Start Menu. Select the ITEMAN for
Windows icon within the folder to start the program. The ITEMAN copyright screen will
appear briefly.
Five pull-down menus and three buttons are immediately available (see Figure 3-1 and
Figure 3-2). The File menu contains the commands for printing and exiting the program.
The Edit menu permits access to the ITEMAN file browser/editor, with which you can
browse any of the ASCII files ITEMAN uses and edit the input file. The Configure menu
opens the Analysis Options Configuration Window, also accessible by clicking on the pencil
icon button below the menus. ITEMAN performs the analysis when you select Go under
the Analyze menu or click on the computer button. The Help menu accesses copyright
information about ITEMAN.
Figure 3-1. The Three Initial Buttons and Their Commands
3-1
click the folder icon to select an existing file without typing. You may also double click in
the space provided or press the space bar. Three sets of radio buttons allow you to specify
whether you want ITEMAN to create a score file or external statistics file or to use an
exceptions file. If you select Yes for any of these, a folder icon appears, allowing selection
or creation of a file.
Figure 3-2. Main ITEMAN Program Window
3-2
Figure 3-3. ITEMAN Configuration Window (File Names)
3-3
response will be considered correct and the item will have no influence on the total score.
For whatever reason, ITEMAN allows you to analyze a data file which contains such
“exceptions.” If you need to use the exceptions capability, you must first create an
“exceptions” file. The exceptions file is an ASCII (text) file that contains the following
information:
Line 1: The header line must start with ITEMAN3.5 (beginning in column 1), which
must be in capital letters.
Line 2 (and following): The Item Sequence Number (from the input file) followed by at
least one space and then the numerical value(s) of any alternative(s) that you want to
have considered as correct, each separated by one space. The alternatives must be
declared as numerical values (A=1, B=2, C=3, D=4, and so on). An attempt to read
alphabetic characters in the exceptions file will result in an error.
You can create an exceptions file by either using a text editor such as NOTEPAD or
WORDPAD or a word processor(such as Microsoft Word) that can save a file as an ASCII
(text) file with line breaks.
The data for the alternatives defined as “exceptions” will then be combined with the data
from the keyed response for purposes of the item analysis. The alternative(s) that you
declared in the exceptions file will be denoted by an ‘E’ under the “Key” column header in
the analysis output file.
An example of an ITEMAN exceptions file for the SAMPLE1.DAT data file is found in the
file SAMPLE1.EXP on your original ITEMAN Program Diskette. A listing of the
SAMPLE1.EXP exceptions file is shown in Figure 3-5.
Figure 3-5. Sample Exceptions File SAMPLE1.EXP
ITEMAN3.5 Exceptions File
2 4
6 1 2 3 4
20 1 2
In this example exceptions file, alternative 4 in item 2 is defined as an exception to be
counted as correct. For Item 6, alternatives 1, 2, 3, and 4 will all be counted as correct. For
item 20 the alternatives numbered 1 and 2 are defined as correct (note that although option 2
is the “keyed” response, this has no effect on the operation of the exceptions file).
If you want ITEMAN to read exception information from a file, click on the Yes radio
button in the Exceptions File box and select the exceptions file by clicking on the folder
icon. ITEMAN expects .EXP files by default. If your exceptions file has a different
extension, you must specify it rather than simply clicking on the appropriate file name.
3-4
statistics file. Again, if a file with that name already exists, the program will ask whether or
not you want to erase and replace the old file. If you respond No, you will be asked to enter
a different filename. A file extension of .STT will be applied to the file name you provide
unless you specifically enter another extension.
Type of Correlations
This option allows you to define the type of item-total score correlation that you want
ITEMAN to compute and report for dichotomous scales (note that multipoint scales always
use product-moment correlations). You can select either a point-biserial correlation or a
biserial correlation. The point-biserial correlation is a Pearson product-moment correlation
between the 0-1 item scores and the number-correct (total) score for that test (scale). Most
item analysis work will use the point-biserial correlation as the preferred method and it
should probably be selected if you do not understand the distinction between the two types
of correlations. The biserial correlation is a correlation between dichotomized item scores
3-5
(based on the assumption that the dichotomous item score—correct vs. incorrect—has an
underlying normal distribution) and the number-correct (total) score. It should be noted
that biserial correlations, since they are not product-moment correlations, can exceed 1.0.
Ability Grouping
This option allows ITEMAN to group the examinees into the highest and lowest 27% based
on the number-correct (total) scores for each scale (test). When this option is selected, the
proportion/percentage endorsements for the high and low score (ability) groups are
reported in addition to the total (entire) group. It should be noted, however, that in order to
perform the ability grouping based on number-correct scores, ITEMAN requires a second
“pass” through the data file and thus will take somewhat longer to complete the analysis than
when ability grouping is not selected.
When ability grouping is in effect the classical index of discrimination is automatically computed
for each item in dichotomous test scales. The classical index of discrimination measures how
well the item can differentiate between high scoring and low scoring examinees. Items that do
not differentiate well between high and low scoring examinees might not be providing useful
psychometric information. The index is the difference between the proportion correct in the
high ability (score) group and the low ability (score) group (Crocker & Algina, 1986). This index
has a possible range of -1.0 to +1.0, but negative values and low (e.g., less than 0.20) values
might be indicative of poor item performance. In most cases the classical index of
discrimination will identify the same “poor” performing items as does the item-total correlation
(point-biserial or biserial).
Subgroup Analysis
This option tells ITEMAN whether to perform the analysis on a subset of the data file. If you
select Yes, a button with a pencil icon appears, which accesses the subgroup analysis window.
The subgroup target code can be up to 4 characters in length and specifies the code in the
examinee’s identification field that you want to include in the subgroup analysis. You will also
be required to enter the beginning column and ending column for the code (which will be the
same for a single-character target code). For example, Figure 3-7 shows a subgroup analysis of
cases with a “target” code of “F” in column 6 of the data file.
3-6
Figure 3-7. Subgroup Analysis Window
Spuriousness Correction
This option allows you to implement a correction on the item-total score correlations to
remove the influence of each item being included in the total score. The inclusion of an
item in the total (number-correct) score will have a larger effect on item-total correlations for
short tests, but its impact diminishes as test length increases. The correction is computed
based on formulas provided by Guilford (1963).
Endorsements
Endorsement rates can be expressed as either proportions or percentages, depending on
your preferences.
3-7
Line 1: Header Line — can contain any identifying information
Line 2: P for point-biserial correlations, B for biserial correlations
Line 3: Y for ability grouping
Line 4: Y for subgroup analysis
Line 5: Y to correct correlations for spuriousness
Line 6: P to express endorsements as proportions, C to express endorsements as
percentages
Line 7: Integer specifying width of score intervals on distributions (remember “1”
means no score grouping)
Lines 8 through 10 are optional (invoked only if Line 4 is “Y”):
Line 8: Subgroup analysis target code (up to 4 characters).
Line 9: Beginning column of subgroup analysis target code.
Line 10: Ending column of subgroup analysis target code.
You should examine the ITEMAN.CFG file that came with ITEMAN (see Figure 3-8) to
see how this file is constructed. You can create or modify the ITEMAN.CFG file using
either a text editor (such as the DOS EDIT program) or by using a word processor and
saving the file as a text file with line breaks.
Figure 3-8. Sample Configuration File ITEMAN.CFG
**** ITEMAN Version 3.5 Configuration File ****
P - Type of Correlation: P = Pt.Biserial B = Biserial
Y - Ability Grouping: Y = Yes N = No
N - SubGroup Analysis: Y = Yes N = No
N - Correct Correlations for Spuriousness: Y = Yes N = No
P - Express Endorsements as: P = Proportions C = Percentages
1 - Score Distribution Interval Width (1-3 digit integer)
3-8
Figure 3-9. Missing Data Options
Itemwise Deletion
When itemwise deletion is selected, ITEMAN will compute statistics for each item using all
the available data and will compute scores for each examinee using data from those items to
which the examinee responded. Item statistics, therefore, might be based on varying
numbers of examinees within a single scale.
Scalewise Deletion
When scalewise deletion is selected, item statistics are based on the set of examinees who
had a complete set of valid responses to all items in the scale. Thus the number of
examinees responding to each item will be constant within a scale but might vary across
scales. For scoring purposes, an examinee with missing data on a particular (multipoint)
scale is excluded from the analysis of that scale and a score of -9.00 is recorded for the
examinee in the score file (if a score output file was specified.).
Casewise Deletion
When casewise deletion is selected, an examinee with missing data on any multipoint scale is
deleted from all analyses, not just those involving the multipoint scales. This will result in
item statistics where the number of examinees is consistent not only within a scale but across
all scales in the analysis. For scoring purposes, a score of -9.00 is recorded in the examinee
score file (if score output is selected) for all scales (including any containing dichotomously
scored items). Casewise deletion is the only missing data option that will affect the analysis
of dichotomously scored items included in the same data file.
3-9
“3” (or neutral) category. Although this approach is preferable for many psychological as
well as other survey situations, a summed (total) score has been implemented as an optional
feature. Following entry of the multipoint missing data option, another box (Figure 3-10)
will appear allowing you to indicate the type of multipoint scores that ITEMAN should
compute (averaged or summed).
Figure 3-10. Multipoint Score Type Selection Box
Analysis
Once you have selected your file names and analysis options, click on the computer icon (or
select Go under the Analyze menu) to start the analysis. Once the analysis has begun, a
small window will appear displaying a percentage-complete bar. As the bar fills, the
ITEMAN program reads the data file and computes all the requested item and test statistics.
The duration of the analysis depends on the speed of the computer you are using, the size of
the data set that you are analyzing, and the options you select in the configuration window
(ability grouping will require more time). All things considered, ITEMAN is a quickly
executing program on most computers for most data sets; most analyses are completed in a
few seconds. When the analysis is complete, Figure 3-11 will be displayed.
3-10
Figure 3-11. Analysis Completed Screen
3-11
The header file is an ASCII file either eight or 12 lines long. The first eight lines primarily
contain information normally entered from the keyboard. The four optional lines contain
the key, number of alternatives, and other information normally contained on the first four
lines of the data file for ITEMAN. The required lines in the header file are described below.
See Figure 3-13 for an example header file.
1. The name ITEMAN3.5 in all capital letters. This is to ensure that you do not use a
header file intended for one of the other analysis programs in the Item and Test Analysis
Package for ITEMAN.
2. The name of the input file.
3. The name of the output file (including file extension). Note: Unlike running in the
interactive mode, ITEMAN will not ask if it is okay to write over an existing file with this
name— it will just do it.
4. The name of the score file into which you want examinee scores written. If you leave
this line blank, ITEMAN will not compute scores.
5. The name of the Key Exceptions file. If you leave this line blank, ITEMAN assumes
there are no exceptions in the analysis.
6. The name of the external statistics file. If you leave this line blank, ITEMAN will not
create an external statistics file.
7. The missing data option. Note that although this is used only if the analyses include
multipoint items, some numerical value (1, 2, or 3) must be included on this line for
batch mode to work properly. The missing data option must be a single digit (1, 2, or 3)
corresponding to the options that ITEMAN normally prompts you for in that window.
“1” is for itemwise deletion, “2” for scalewise deletion, and “3” for casewise deletion.
8. Enter “Y” if multipoint scores are to be averaged across items, “N” if multipoint scores
should be summed (totals) across items.
Figure 3-13. Example Header File for Batch Mode
ITEMAN3.5
math24.dat
math24.out
math24.scr
math24.exp
math24.stt
2
Y
30 o n 0
423142242113424141324213411334
444444444444444444444444444444
YYYYYYYYYYYYYYYYYYYYYYYYYYYYYY
The four optional lines consist of the information that normally constitutes the first four
lines of an ITEMAN input file (see Chapter 2). If they are not included in the header file,
ITEMAN will assume that this information is in the first four lines of the input file. If they
are included in the header file, ITEMAN will assume that the input file does not contain this
information. When using a header file, it is very important that the four lines of information occur in
only one place. If they are included in both the input file and the header file, they will be
analyzed as data in the input file.
3-12
Interpreting the Analyses
The first page of the analysis output file contains summary information about the analysis,
including file names and configuration options that were in effect. This summary “header” page
can prove useful if you must, at some point in the future, re-run an analysis on the same data.
The actual results of the item and test analyses performed by ITEMAN are then printed in four
sections: item statistics for dichotomously scored items (if any), item statistics for multipoint
items (if any), summary statistics for each scale, and score frequency distributions for
dichotomously scored scales (if any). The four sections will appear in the above order in the
output following the analysis summary “header” page.
3-13
keyed response. An asterisk appear to the right of the data for the alternative keyed as
correct.
Because the example in Figure 3-14 uses ability grouping the proportion/percentage
endorsing each alternative (response) is provided for the total group, the low ability (lowest
27% scoring) group, and the high ability (upper 27% scoring) group.
Figure 3-14. Item and Alternative Statistics for Dichotomously Scored Items
Item Statistics Alternative Statistics
----------------------- -----------------------------------
Seq. Scale Prop. Disc. Point Prop. Endorsing Point
No. -Item Correct Index Biser. Alt. Total Low High Biser. Key
---- ----- ------- ------ ------ ----- ----- ---- ---- ------ ---
As mentioned above, when a statistic cannot be calculated for the response alternative the
value will not be printed. This occurs most frequently when no examinees endorse an
alternative.
3-14
Figure 3-15. Item and Response Alternative Statistics for Mutipoint Items
Item Statistics Alternative Statistics
--------------------------------- -----------------------
Seq. Scale Item Item Item-Scale N per Alter- Proportion
No. -Item Mean Var. Correlation Item native Endorsing Key
---- ----- ------ ------ ----------- ----- ------ ---------- ---
Item Var. This is the variance of the responses to the item. Its computation can also be
affected by the missing data option selected.
Item-Scale Correlation. This is the Pearson product-moment correlation between
responses to the item and average scores for examinees.
N per Item. This is the number of examinees on which the item statistics are based.
The alternative statistics for multipoint items describe the proportion/percentage (depending
on your configuration) of examinees endorsing each response within the specified range for
the item. In the column labeled “Key,” a plus or minus sign indicates the direction in which
the multipoint item was scored when examinee scores and other statistics were computed.
Once again, a blank value field indicates that the statistic could not be calculated for the
response alternative.
3-15
N of Items. This is the number of items in the scale that are included in the analysis. For
scales composed of dichotomously scored items, this will be the total number of items
on the scale.
N of Examinees. This is the number of examinees that are included in the analysis for the
scale. For scales composed of dichotomously scored items, this will be the total number
of examinees in the data file; it will also be the total number of examinees for multipoint
scales in the same data file unless the casewise deletion option was selected for these
scales.
For scales containing multipoint items, the number of examinees can differ as a result of
the missing data option selected. When itemwise deletion is selected, the number
printed is the total number of examinees in the data file; the actual number used to
compute the item statistics can vary from item to item. When scalewise deletion is
selected, the number will be the total number of examinees with complete data for that
scale. When casewise deletion is selected, the number will be the total number of
examinees with complete data on all the multipoint scales.
Mean. For scales with dichotomously scored items, this is the average number of items on
each scale that were answered correctly. For scales with multipoint items, it is the
average examinee score for examinees included in the analysis. An examinee’s score can
be his or her average response or the sum of the weights for their responses. When
itemwise deletion of missing data is selected, each examinee’s score can be based on a
different subset of items.
Variance. This is the variance of the distribution of examinee scores on each scale. It gives
an indication of the spread of the scores around the mean score and, for multipoint
items, can be affected by the missing data option selected.
Std. Dev. This is the standard deviation of the distribution of examinee scores for each
scale. It is the square root of the variance.
Skew. This is the skewness of the distribution of examinee scores for each scale. The
skewness gives an indication of the shape of the score distribution. A negative skewness
indicates that there is a relative abundance of scores at the high end of the scale
distribution. A positive skewness means that there is a relative abundance of scores at
the low end of the distribution. A skewness of zero means that the scores are
symmetrically distributed about the mean.
Kurtosis. This is the kurtosis of the distribution of examinee scores for each scale. The
kurtosis indicates the peakedness/flatness of the distribution relative to that of a normal
distribution. A positive value indicates a more peaked distribution; a negative value
indicates a flatter distribution. The kurtosis of a normal distribution is zero.
Minimum. This is the lowest score on each scale for any examinee.
Maximum. This is the highest score on each scale for any examinee.
Median. This is the examinee score at the fiftieth percentile for each scale. It is thus the
score that half of the examinees scored at or below. Interpolation is not used in the
calculation of this value.
3-16
Figure 3-16. Scale Summary Statistics
Missing-data option: Compute statistics on all available item responses
Scale Statistics
----------------
Scale: 1 2 3
------- ------- -------
N of Items 10 10 10
N of Examinees 50 50 50
Mean 4.800 7.180 3.935
Variance 5.160 6.988 2.145
Std. Dev. 2.272 2.643 1.465
Skew 0.217 -0.909 0.057
Kurtosis -0.981 0.018 -1.423
Minimum 1.000 0.000 1.667
Maximum 9.000 10.000 6.333
Median 5.000 8.000 4.100
Alpha 0.636 0.821 0.943
SEM 1.370 1.120 0.351
Mean P 0.480 0.718 N/A
Mean Item-Tot. 0.301 0.509 0.809
Mean Biserial 0.387 0.679 N/A
Max Score (Low) 3 6 N/A
N (Low Group) 18 16 N/A
Min Score (High) 6 9 N/A
N (High Group) 20 19 N/A
Scale Intercorrelations
-----------------------
1 2 3
3-17
scores for items in the scale. Only those items for which the correlation could be
computed are included in the average.
Mean Biserial. For scales containing dichotomously scored items, this is the average
biserial correlation across all of the items on the scale. This statistic is not applicable
(N/A) for scales composed of multipoint items.
Max Score (Low). When ability grouping is selected from the configuration options, this is
the maximum score an examinee could attain on the scale and still be considered a
member of the low ability (score) group.
N (Low). When ability grouping is selected, this is the number of examinees in the lowest
score grouping. Ideally this number would represent exactly 27% of the total number of
examinees for the scale (test). The score that defines the maximum score for
membership in the low ability group (above) is selected based on that score interval
containing the 27th percentile. Depending on the numbers of examinees at that score
point, however, the actual percentage of examinees in the lower ability (score) group can,
therefore, be slightly larger than the target 27%.
Min Score (High). When ability grouping is selected from the configuration options, this
is the minimum score an examinee could attain on the scale and still be considered a
member of the high (upper) ability (scoring) group.
N (High). When ability grouping is selected, this is the number of examinees in the highest
score grouping. Ideally this number would represent exactly 27% of the total number of
examinees for the scale (test). The score that defines the minimum score for
membership in the high ability group is selected based on that score interval containing
the 73rd percentile. Depending on the numbers of examinees at that score point,
however, the actual percentage of examinees in the high (upper) ability (score) group can,
therefore, be slightly larger than the target 27%.
Scale Intercorrelations. These are the Pearson product-moment correlations among the
examinee scores on the individual scales. When some or all of the scales contain multipoint
items, the intercorrelations can be affected by the missing data option selected. When
itemwise deletion is selected, the correlations will be based on all examinees in the data file;
however, each examinee’s score might be based on a different subset of items. When
scalewise deletion is selected, the correlation between any two scales will be based on the
subset of examinees having scores computed on both scales. When casewise deletion is
selected, the correlations will be based on the subset of examinees who have complete data
on all the multipoint scales.
3-18
PR. The percentile rank for a given row.
PCT. The percent of the examinees obtaining a certain row’s number correct. This figure is
rounded, so the column might not sum to exactly 100 percent. The percentage is also
shown in graphical format in the last column. Each pound sign (#) signifies one percent
of the examinees.
Figure 3-17. The Score Distribution Table from SAMPLE2.DAT
SCALE # 1 Score Distribution Table
The missing data option you select will affect the scores output for scales composed of
multipoint items. When itemwise deletion is selected, an examinee’s scale score is based on
the available item responses. When scalewise deletion is selected, an examinee with missing
data on a particular scale is excluded from the analysis of that scale and a score of -9.00 is
recorded in the score file. When casewise deletion is selected, an examinee with missing data
on any multipoint scale is deleted and a score of -9.00 is recorded in the score file for all
scales (including any containing dichotomously scored items).
3-19
Figure 3-18. Sample Examinee Score Output
10 3 Scores for examinees from file C:\ITEMANW\SAMPLE2.DAT
EX9013371D 4.00 7.00 2.00
EX2024438D 5.00 10.00 3.30
EX1224466W 4.00 6.00 4.80
EX2036874U 4.00 8.00 5.70
EX2034163S 5.00 9.00 5.00
EX3021364L 6.00 8.00 2.40
EX2041546B 7.00 7.00 2.40
References
Guilford, J. P. (1963). Psychometric methods. (2nd ed.) New York: McGraw-Hill.
Crocker, L., and Algina, J. (1986) Introduction to classical and modern test theory. New York:
Holt, Rinehart and Winston.
3-20
Appendix A
A-1
Dichotomous Item Scales (Continued)
Field Contents Columns Field Type
Alternative 3 (C) Response Block
Total Grp Prop/Pct Correct 142 - 148 F7.3 ( XXX.XXX )
Low Grp Prop/Pct Correct 150 - 156 F7.3 ( XXX.XXX )
Top Grp Prop/Pct Correct 158 - 164 F7.3 ( XXX.XXX )
Point-Biserial (r) 166 - 171 F6.3 ( XX.XXX )
Biserial (r) 173 - 178 F6.3 ( XX.XXX )
Alternative 4 (D) Response Block
Total Grp Prop/Pct Correct 180 - 186 F7.3 ( XXX.XXX )
Low Grp Prop/Pct Correct 188 - 194 F7.3 ( XXX.XXX )
Top Grp Prop/Pct Correct 196 - 202 F7.3 ( XXX.XXX )
Point-Biserial (r) 204 - 209 F6.3 ( XX.XXX )
Biserial (r) 211 - 216 F6.3 ( XX.XXX )
Alternative 5 (E) Response Block
Total Grp Prop/Pct Correct 218 - 224 F7.3 ( XXX.XXX )
Low Grp Prop/Pct Correct 226 - 232 F7.3 ( XXX.XXX )
Top Grp Prop/Pct Correct 234 - 240 F7.3 ( XXX.XXX )
Point-Biserial (r) 242 - 247 F6.3 ( XX.XXX )
Biserial (r) 249 - 254 F6.3 ( XX.XXX )
Alternative 6 (F) Response Block
Total Grp Prop/Pct Correct 256 - 262 F7.3 ( XXX.XXX )
Low Grp Prop/Pct Correct 264 - 270 F7.3 ( XXX.XXX )
Top Grp Prop/Pct Correct 272 - 278 F7.3 ( XXX.XXX )
Point-Biserial (r) 280 - 285 F6.3 ( XX.XXX )
Biserial (r) 287 - 292 F6.3 ( XX.XXX )
Alternative 7 (G) Response Block
Total Grp Prop/Pct Correct 294 - 300 F7.3 ( XXX.XXX )
Low Grp Prop/Pct Correct 302 - 308 F7.3 ( XXX.XXX )
Top Grp Prop/Pct Correct 310 - 316 F7.3 ( XXX.XXX )
Point-Biserial (r) 318 - 323 F6.3 ( XX.XXX )
Biserial (r) 325 - 330 F6.3 ( XX.XXX )
A-2
Dichotomous Item Scales (Continued)
Field Contents Columns Field Type
Alternative 8 (H) Response Block
Total Grp Prop/Pct Correct 332 - 338 F7.3 ( XXX.XXX )
Low Grp Prop/Pct Correct 340 - 346 F7.3 ( XXX.XXX )
Top Grp Prop/Pct Correct 348 - 354 F7.3 ( XXX.XXX )
Point-Biserial (r) 356 - 361 F6.3 ( XX.XXX )
Biserial (r) 363 - 368 F6.3 ( XX.XXX )
Alternative 9 (I) Response Block
Total Grp Prop/Pct Correct 370 - 376 F7.3 ( XXX.XXX )
Low Grp Prop/Pct Correct 378 - 384 F7.3 ( XXX.XXX )
Top Grp Prop/Pct Correct 386 - 392 F7.3 ( XXX.XXX )
Point-Biserial (r) 394 - 399 F6.3 ( XX.XXX )
Biserial (r) 401 - 406 F6.3 ( XX.XXX )
Other Response Block
Total Grp Prop/Pct Correct 408 - 414 F7.3 ( XXX.XXX )
Low Grp Prop/Pct Correct 416 - 422 F7.3 ( XXX.XXX )
Top Grp Prop/Pct Correct 424 - 430 F7.3 ( XXX.XXX )
Point-Biserial (r) 432 - 437 F6.3 ( XX.XXX )
Biserial (r) 439 - 454 F6.3 ( XX.XXX )
A-3
Multipoint Item Scales
Field Contents Columns Field Type
Item Sequence Number 1-3 I3 ( XXX )
Scale Number 5 I1 ( X )
Scale Sequence Number 7-9 I3 ( XXX )
Keyed Response 11 A1 ( + OR - )
Number of Categories (Alternatives) 13 I1 ( X )
Number of Examinees (Attempting Item) 15 - 19 I5 ( XXXXX )
(Blank) 21 - 26 A6 ( XXXXXX )
Keyed Response Block
Item Mean 28 - 34 F7.3 ( XXX.XXX )
Item Variance 36 - 42 F7.3 ( XXX.XXX )
(Blank) 44 - 50 A7 ( XXXXXXX )
Item-Scale Correlation 52 - 57 F6.3 ( XX.XXX )
(Blank) 59 - 64 A6 ( XXXXXX )
Prop/Pct Endorsing Category 1 66 - 72 F7.3 ( XXX.XXX )
Prop/Pct Endorsing Category 2 104 - 110 F7.3 ( XXX.XXX )
Prop/Pct Endorsing Category 3 142 - 148 F7.3 ( XXX.XXX )
Prop/Pct Endorsing Category 4 180 - 186 F7.3 ( XXX.XXX )
Prop/Pct Endorsing Category 5 218 - 224 F7.3 ( XXX.XXX )
Prop/Pct Endorsing Category 6 256 - 262 F7.3 ( XXX.XXX )
Prop/Pct Endorsing Category 7 294 - 300 F7.3 ( XXX.XXX )
Prop/Pct Endorsing Category 8 332 - 338 F7.3 ( XXX.XXX )
Prop/Pct Endorsing Category 9 370 - 376 F7.3 ( XXX.XXX )
Prop/Pct Endorsing Category (Other) 408 - 414 F7.3 ( XXX.XXX )
A-4