Grattan

Interchange Level One Placement Test

Introduction
Assessment has undergone many changes in the past century, from the pre-
scientific era to the psychometric-structuralist era starting in the 1950s until the 1990s. At
this point, the communicative approach became the common methodology and is still in
use today. When assessing students, it is important to investigate the language areas and
test techniques. In order to effectively assess a student, one must consider all the
components that go into test design and administration. Matching the assessment design to
its purpose is but one of these components. However, as history has shown us, assessment
evolves. This evolution is a result of continuous research in the field of language
assessment.
Method of Organization (Abstract)
This report will begin by describing why we created this test and going into detail
about the purpose and use in terms of its impact and the inferences that will be made from
it. Next, how the scores are interpreted, the target language use (TLU) domain, and the
construct definition will be clearly stated. Following will be an overview of the task types in
each of the six sections in the test, including how each will be scored. The report will then
explain the pilot test procedures in terms of participants and administration, and share the
results. This data will be statistically analyzed by item and description to determine the
success and effectiveness of the tasks and items 0chosen. The standard error of
measurement (SEM) and description of mastery will also be provided from the results of
this pilot test. Lastly, conclusions and discussion regarding the tests usefulness and impact
will be shared, along with personal reflections on the test development process.
Description of the Test
Purpose
Our assessment scenario is in an EFL environment located in Kurdistan, Iraq at The
English Club for adult men and women. The specific class we are developing this
assessment for meets 2 hours a day, 4 days a week, for 3 months. This program is for
general English purposes, and the purpose of this test is to evaluate which level learners
should be placed in. More specifically, this test assesses whether or not learners should
enter into level 1 of the program, which is the level after the introductory course. This test
will be used before the start of the course to determine students placement. It is integrated
and covers all six language areas so administration can infer learners abilities in all areas
within the TLU domain. Test familiarity is dependent on individuals, and student placement
will depend on individual percentages in regards to a cut score of 40%. It will be criterion
referenced, meaning the students scores will be compared to a standard and not one
another.
This test incorporates items/tasks to assess all areas of proficiency.
Listening/speaking, reading/writing, and vocabulary/grammar are split into six different
sections. This test is administered in an hour, with additional time allotted for the speaking
 Grattan 2

sections following the examination. There are 53 items in total and 69 points possible,
because some items are worth more than one point. The listening section is first, followed
by vocabulary and grammar. Reading and writing are last so students do not spend too
much time on these sections and are unable to finish the others in time. The speaking
sections include a dictation and an oral performance interview and students sit down with
the interviewer/rater once they have completed the other sections.
TLU Domain
The TLU domain is level 1 in the general English program in Kurdistan. This course
includes various tasks for the six different language areas. For listening, comprehension is
the main focus where students must provide information regarding listening activities (e.g.
fill-in-the-blank, sentence completion). For vocabulary, words are taught through thematic
sections, where students are exposed to topical knowledge while using new words in
various ways. For grammar, verb tenses, aspects, parts of speech, and
sentence/question/negation formation are all covered. For reading, skimming and
scanning are strategies taught to students, as well as inferencing. For writing, informal
tones are the focus (e.g. writing postcards and emails) to educate students on
practical/general writing abilities and not academic ones. For speaking, students engage in
role-plays and partner work as well as interact with their peers in authentic tasks that
integrate more than one skill. A TLU domain description can be located in Appendix A.
Type/Design of Test
This is a proficiency test to determine the placement of the learners depending on
their scores in relation to the standard (criterion-referenced). A score of 40% or above
displays enough mastery of the material to proceed into level 1 of the program.
Construct
Test takers of this general English placement test should be able to use grammar
and textual knowledge to complete tasks in all six language skill areas (i.e. listening,
vocabulary, grammar, reading, writing and speaking), as a result, nothing is excluded.
Receptive and productive abilities will be explicitly assessed through various tasks. The
items formulated for this test are created according to the Common European Frame of
References (CEFR) because the Interchange textbook is based off of these standards. A
beginner proficiency level and basic understanding of these language areas at the
introductory level of Interchange is assumed.
Sub Skills
The following sections include each language areas tasks types, instructions,
expected responses, scoring procedures, and time allotted for each task.
Listening
For listening, there are two audio recordings. Both of the listening recordings are
approximately 45 seconds in length and will be played only one time. For both listening
sections, students have to answer four questions. These are comprehension questions in
the format of multiple choice, and true or false. The format was chosen because the
 Grattan 3

learning outcome that this section is trying to measure their ability to understand spoken
English. The reasoning is because at advanced beginner level the students are not tested on
their higher learning outcomes. Scoring for both listening sections are the same; each
question is worth one point and an incorrect answer is scored as a zero.
The first listening section includes a monologue of a male in his mid-twenties
speaking about his favorite place to spend time. The recording consists of 107 words
spoken at a natural pace. The topic of the recording is about the speakers favorite place
(the park), which corresponds to the communicative function of describing places. The
function of describing people, places and things is common in the Interchange level one
course book. The communicative function, expressing opinions, is somewhat accomplished
at a basic level but is not the primary focus of the listening sections. This recording includes
pronunciation features that are presented in the textbook such as the reduction of to, there
is/there are, and linked sounds. In addition, the different verb tenses that are covered in the
coursebook are used in the appropriate context (e.g. the simple present, present
continuous, simple past, and present perfect tense). Following the recording, four multiple-
choice questions are presented with three possible solutions listed below, only one of
which is correct. The questions are presented in the same chronological order as the
information in the recordings necessary to answer the questions. One reason for this
format to be used is to prevent the possibility of a response set that would decrease
reliability (Miller, Gronlund & Linn, 2009).
Section 2 of the listening is a dialogue between a native English speaking female and
a non-native English speaking male; both are in their mid twenties. The duration of this
recording is also approximately forty-five seconds long and consists of 111 words spoken
at a natural pace. The communicative function, taken from the textbook, focuses on asking
for and giving advice. Similar to the first recording, pronunciation features from the
textbook are included, such as the reduction of would you and dont you, question and
statement intonation, and linking sounds. Verb tenses that were focused on in the textbook
are also included (e.g. present simple and continuous, simple past and present perfect).
After the students have listened to the recording, they will complete four true/false
questions that target listening for specific information.
Vocabulary
For vocabulary, there are two sections given for a duration of approximately five
minutes divided evenly between the two sections. The topics were chosen from different
chapters in the Interchange textbook, so the content is what test takers are expected to be
able to recognize and use in Level One of the textbook. Test takers need to be able to have
the basic skills of reading and writing to be able to answer the questions successfully. In
both sections, the student must write the correct vocabulary for the definition or fill in the
blank with the correct given word to receive a point. If unable to provide the correct
vocabulary for the definition or fill in the blank with the correct word, they will receive a
zero.
 Grattan 4

The first section has 5 definitions and students are instructed to write the
vocabulary that is defined in the space provided next to each definition. All questions for
the first section are to determine if students are able to recall vocabulary that they might
encounter in Interchange Level One. The expected response for the questions in the first
section requires test takers to provide vocabulary to definitions provided for them. The
purpose of having students to recall and give vocabulary to definitions in the first section
excludes the possibility of test takers answering correctly by guessing or chance.
The second section consists of 5 fill-in-the-blank items below a word bank. Students
need to fill in blanks with words given to them in a word bank that collocate with the
immediate following word. The questions target whether test takers have knowledge about
verb collocations. The questions are in accordance with the Common European Framework
of References. The expected response for the second section requires test takers to fill in
blanks with the verbs provided in the box (collocations).
Grammar
For grammar, there are two sections that require approximately 10 minutes to
complete. The grammar points are chosen from the Interchange textbook. For both
sections, if the test taker does not provide the correct response, they will receive no points.
If they answer correctly, there will be a point for each one they get correct. Test takers need
to have basic reading and writing skills to be able to complete the tasks successfully.
The first section asks test takers to complete ten blanks that requires students to
conjugate the verbs provided in their base form within an exerpt. The purpose of having
test takers conjugate the verbs will assess their ability in form, use and meaning of the
English tense system. The instructions for the first task type will require test takers to read
a bio and conjugate the verbs given to them in the base form. The expected response for the
questions in the first section will be that the verbs will be conjugated correctly using the
correct tense and aspect.
The second section consists of 5 multiple-choice questions. This task assesses
students knowledge/understanding of grammar elements (e.g. articles, pronouns).
Purpura (2004) states, Selected response tasks (SR) allow us to make inferences about
the learners receptive knowledge of the learning point (p. 113). The instructions for this
second task type will require test takers to read the sentences given to them and choose
the correct answer. The expected response for the questions in the second section is that
test takers will choose the correct response in a multiple-choice task.
Reading
There will be two reading sections, each with their own reading passage. For the
first reading section, there is a reading passage provided in the form of an email where the
writer is talking about a traveling trip she is on. This task assesses students reading
comprehension strategies in the form of skimming and scanning to find relevant
information and grasp main ideas. This passage is short and concise, using vocabulary
students moving into level one should know. The instructions for the first task type tell the
 Grattan 5

test taker to answer five questions regarding the contents of the email, such as who the
person in the email is visiting or where they are going. The topic its a very exciting place!
is unit 11 in the Interchange textbook, so this content is something students will have to
work with and be able to understand/use in level one. The expected response for these
questions will be short answer, with only one correct answer for each question. The use of
short answer items in this section eliminates the possibility of test takers answering
correctly by guessing, and also requires readers to scan the text for specific information
(Alderson, 2000). For example, if the question asks Where did Sarah go after she left
Barcelona? and the email said I went to Madrid after spending a week in Barcelona., the
expected response to this question is Madrid. The answers to these four questions are
found in the email and can be directly copied from the text. If the test taker does not write
the correct response they will not receive a point for that question, and if they answer
correctly they will receive one point. Full sentence responses are not expected and this is
explicitly stated in the instructions.
The second reading section includes a short reading passage in the form of a short
article about going out to eat at a restaurant, which is covered in unit 13 in the Interchange
textbook. There will be two parts with part A having four true/false questions and part B
having seven step arranging questions about the reading. The purpose is to assess the test
takers ability to identify the correctness of statements. For example, if the article says,
First, the waiter will ask you if you want anything to drink..., the true/false statement will
say At a restaurant the waiter will take your drink order, and the answer will be true.
These questions will only have one correct answer (true or false) so they will be scored
either one point or no points depending on the students response. The answers regarding
these four statements are found in the article, but unlike the first sections questions, test
takers must make inferences regarding the statements of fact based on evidence from the
text.
Part B of the second reading section requires students to understand sentence
organization and cohesion in order to arrange the steps for what to expect when eating at a
sit-down restaurant described in the reading passage. The steps are arranged in
random/incorrect order and test takers must put them in chronological order to show the
sequence of actions. These responses will be scored one point per step with no partial
credit possible, so the whole section is out of seven points.
These sections were created in accordance with the CEFR for reading at a high-
elementary to low-intermediate level in which test takers should be able to understand
keywords, cognates, and formulaic phrases. Students are expected to be able to interpret
language used to describe familiar places, things, and time to answer direct comprehension
questions where the information can be found within the text. The time for this reading
section with twelve questions will be approximately 12 to 15 minutes, depending on how
quickly students can read the passages and begin answering questions.

 Grattan 6

Writing
For the writing section test takers will be given a list of activities and told to pick
two and write six sentences, three sentences per activity. This task will assess students
grammar production as well as their vocabulary and cohesion rhetoric. For example, if the
student chooses watching TV and swimming then they should write three sentences
about each activity. These responses will be simple sentences, such as I like to watch TV
shows about cooking. The chef shows you each step of how to make a dish. The food always
looks so good!.... Asking students to talk about personal information is a simple task that
learners entering into level one should be able to complete. The relationship between the
list of activities and the expected responses guides the test taker to write about their
personal preferences using familiar content covered in the introductory level of the
program. There should be an importance placed on the content as well as the language
being used (Plakans & Gebril, 2017, p. 100). These responses will be scored using a rubric
(see Appendix F) that looks at test takers abilities to produce simple, connected text with
accurate sentence structure, word order, grammar, and coherence, as well as demonstrate
productive understanding of the two activities they chose from the list.
The expected responses will be scored in accordance with the CEFR proficiency
guidelines for writing at a high-elementary to low-intermediate level in which these
students are expected to be at to enter into level one. The responses should show that the
student can create simple sentences with appropriate content, language use, and
coherence. The information will most likely be practical and reproduced from knowledge
and topics that have been covered in the classroom prior to the test. The sentences may
predominantly utilize the present tense, be repetitive in structure, and can be understood
by native speakers who have experience in reading non-native speaker writing.
Speaking
The speaking assessment consists of two sections. The first includes a task has the
students read a script at a natural pace out loud, assessing their pronunciation. The passage
consists of 120 words that should take the reader between 1-2 minutes to complete. The
passage contains different aspects of pronunciation including multi-syllabic words, sound
linking, reduction of to and there is/there are, vowel sounds /ou/ and //, and intonation in
statements. The topic of the monologue is discussing a daily routine, which is covered in
the textbook. The score of this assignment is out of a total of four points with partial points
possible. The speaker is measured against a rubric highlighting the specific features being
assessed (see Appendix F). The main focus is the proper pronunciation of the different
aspects divided into four categories, each worth one point. The expected response is
reflected in the four categories of the criteria: fluency, vowel and consonant pronunciation,
linking sounds, and reduction of sounds. A detailed explanation of each category is given to
help determine if full, half or no credit should be given. This also helps to increase the
reliability of the raters score.
 Grattan 7

The second task is a one-on-one interview, with questions increasing in difficulty as

the interview progresses. Scoring for the speaking section differentiates from the other test
parts because of its communicative importance in an ESL classroom. Any student who
wishes to continue to Level 2 must receive at least 50 percent on this section. This
interview assesses the test takers pronunciation, fluency, range of lexical production, and
discourse competence. The interview consists of the rater asking questions which increase
in difficulty as the interview progresses. The questions were chosen to reflect some of the
main vocabulary topics (e.g. daily routines, family, hobbies, favorites and goals). To best
cover what is expected of their speaking ability, the following communicative functions
were taken from the Interchange textbook to be assessed: basic information exchange,
describing people/events, expressing likes/dislikes and comparing ideas. The number of
questions asked will range between 5-8, depending on the speed of response and students
performance in the interview. Scoring for this section will also be out of a total of four
points against a set rubric. These criteria for the rubric include fluency, pronunciation,
vocabulary range and grammar. An explanation of what constitutes either full or half credit
is given in the rubric to increase reliability (see Appendix F).
Pilot Test Procedure
Participants
We had 12 students take this pilot test in Kurdistan. The students taking this test are
learners looking to develop their general English abilities in all language areas. Students
motivation for learning varies between travel, business, and academic ventures. The age
ranges from 16-35 and both males and females may be assessed. All students are fluent and
literate in their native language of Kurdish, and are all citizens of Kurdistan. The majority of
students will not have familiarity with a placement test prior to this one.
Administration
The pilot test was administered to each student at The English Club in Kurdistan as
they arrived to the school on Thursday April 20th. Each test taker had an hour to complete
the listening, vocabulary, grammar, reading, and writing sections. The speaking portion
was taken individually following the completion of the other sections, and involved each
student engaging in a dictation and interview while being audio recorded for evaluation at
another time. The interviewer was in charge of recording the speaking responses, but the
rating was done by the test developers in America.
Test Results
The results from the pilot test were analyzed using Microsoft Excel to compute the
descriptive statistics. Only one version of the test was administered, so there could be no
correlation coefficient. The data results are provided in Appendix J with the first table
showing student scores for the overall test, broken down by language area. According to
our cut-score, 100% of the students who took this test would pass and be placed in level 1.
In Appendix K are tables first summarizing the descriptive statistics for the overall
student scores, and then breaking it down by sections. The mean score was 48.375 points
 Grattan 8

or 74% of the total, with the minimum score being 38.5 points and the highest being 61
points. The standard deviation for the test was 6.876 or 1 SD, which according to Brown
(2003) should include approximately 68% of the students. However in our pilot test, only
58% of the students fell into the acceptable standard deviation range, which means our test
would need revisions before being administered again. Cronbachs alpha was calculated at
0.60001 which falls under the minimum of 0.7 for acceptable reliability.
Inter-rater reliability, shown in tables in Appendix L, was determined by correlating
the scores of the writing and speaking sections of the test from two separate raters. One
rater was a native English speaker and the other was a non-native with Kurdish as his first
language. For the writing section, there was a positive correlation of 0.87 when comparing
the two sets of scores. A perfect score would be 1, indicating a perfect correlation. Our
results showed that in the writing section there were some significant differences between
the scores assigned. The reasoning for this could vary because there was no identifiable
pattern in the score sets. The speaking section had a stronger correlation of 0.998,
indicating inter-rater reliability. The reason for the differences could be because although
the design team worked together in developing the test, we did not go through training on
how to rate the students performance.
It should be noted that for the writing section, one student did not make an attempt.
While that student technically pass the cut score, they would not be admitted into level 1
until they demonstrated their writing ability. Similar in the speaking section, due to an
administrative error, three of the students did not complete the first speaking task. Again,
these students would need to complete this task before admission into level 1.
Discussion
Item Performance
The item results for this pilot test have helped the test designers to remove or alter
certain items that were either too easy or difficult. An example would be for the listening
task 2, item 8, 100% of the students answered correctly. This could be for a variety of
different reasons, such as the question being too easy or common knowledge, or another
item may provide a clue to the answer. Overall, each section had a variety of scores ranging
from below 50% to 100% amongst the test takers. This is an indication that the summary
of all the items in each language area are not severely inappropriate in terms of difficulty
for these learners. If the results showed that in one section every student got every
question right, then that section would have to be rethought entirely, but this is not the
case. Additionally, if there were to be one section where all students did very poorly, this
would need to be reconsidered as well. Looking at the results, some modifications are
required to make sure that the standard deviation of future test results is consistent and
within the appropriate range. One alteration to be made is in the first reading section,
where the majority of test takers got all five questions correct. These results suggest that
the task type is too easy and some changes must be made. Since this is a fill-in-the-blank
task, rewording the questions to differentiate them from the exact wording in the reading
 Grattan 9

passage may make these items more successful in measuring students reading
comprehension skills. Analysis of test results and what they say about the test items in each
section is necessary to ensure the test is assessing the skills it was created to.
Test Usefulness
When thinking about this test in terms of reliability, there are a few components to
consider. The statistical analysis discussed above covers the internal consistency reliability
within the tasks and items. Since this test has only one form, that aspect is consistent as
well. This test was designed to be administered in an hour excluding the speaking section,
and this time frame will stay consistent in future administrations of this test. However,
inter-rater reliability is something to evaluate for future use of this test since the test
developers graded the pilot tests, but will not be in charge of grading future exams. Answer
keys will be provided for future test administrators and raters as well as detailed rubrics
for how to evaluate the writing and speaking portions.
This tests purpose is to assess learners abilities in terms of whether or not they
should be placed in level 1 of a general English program. This required all language areas to
be included in order to make sure students could participate in all classroom activities and
assignments. The test results were useful in providing test developers with enough
information to where they could make placement decisions for each learner. Since all test
tasks and items were based off of those found in the textbook being used in level 1 of the
program, the test tasks were authentic for the target language use domain. The tasks were
similar to those learners will see in level 1, so their ability to complete these tasks
demonstrated their likelihood of success in that class.
Validity
The validity of this test correlates with the authenticity of the tasks in relation to the
English program it is being administered in. The results have been and will be interpreted
to make placement decisions, so therefore a cut score was necessary to ensure meaningful
and appropriate conclusions were being made based on test takers results. Since this is a
beginner proficiency level test, the skills we looked to assess were mostly
knowledge/comprehension and some application. The various tasks provide evidence that
these language skills have been assessed, with writing and speaking targeting application,
and the other four areas largely targeting knowledge/comprehension.
Interactiveness
This placement test involves a variety of communicative language abilities
throughout the different sections. This test incorporates grammatical knowledge with
syntax and lexis in the vocabulary and grammar sections. Topical knowledge regarding
common people, places, and things is required for the first vocabulary section. Textual
knowledge, such as cohesion and organization, are required in the reading and writing
sections. Pragmatic knowledge is incorporated into the reading, listening, and speaking
sections through functional (ideational) and sociolinguistic (naturalness and cultural
reference) means. Lastly, strategic competence is necessary for test takers to complete the
 Grattan 10

second speaking section, requiring them to interact and answer questions with an
interlocutor.
Impact
The scope of impact for this assessment is relatively narrow, when compared to
large standardized tests like the IELTS or TOEFL which can both be used for similar
purposes of admissions and placement within a course. The impact of this assessment in
regards to the student is dependent on their performance on the test. Impact that will be
immediate would be the added time necessary to administer the test to the students and to
grade the tests accurately. Some of the impact on instruction will be over a period of time,
once the students who have passed the test are placed into Level 1. Whether the student is
allowed to enter Level 1 or not will affect how their peers, family and friends view them,
and how they view themselves. Their placement will also determine how quickly they
could possible move through the Interchange series. Lastly, if the students are put into
Level 1 before they are ready, it could negatively affect their opinion of the book, the
program, and English in general. When students start to interact and learn with other
students in class, adjustments may have to be made in order to compensate for the
students not being properly prepared for the course.
Overall Achievement of Test Purpose
Based on the information received by conducting the pilot test, evidence supports
our tests purpose. The purpose of this assessment was to measure the prospective
students proficiency to place them into the appropriate level in the general English
program. To concretely say that our test is a reliable measure of the minimum level of
proficiency would be premature. More data collection is needed throughout the student's
time in Level 1. Assessing the students throughout the course using formative and norm-
referenced criteria will give more evidence for both validity and reliability.
Personal Reflections
Sharayah
The process of going through all the steps of developing an assessment was a very
valuable one. I was particularly happy with the fact that our test was not for a hypothetical
situation, and we were able to email it to Halos brother who administered it at The English
Club in Kurdistan. With actual test results to analyze, we could evaluate how test takers
responded to the items we developed, which gave us a better understanding of what
needed to be modified or changed to create a more valid and reliable test. The three step
process with feedback allowed for our group to work through our ideas and revise our test
before piloting it. This helped our test be more efficient in achieving its purpose than if we
had implemented it early on in the semester. Determining task types and creating test
items for all language areas were very useful practices for future teaching situations where
I may need to develop formative or summative assessments. All of the components of this
project showed the various components that need to be considered when producing an
assessment of any sort, such as the TLU domain, the construct, the purpose, and the
 Grattan 11

scoring. Creating a rubric for a writing assessment I constructed was a difficult task, but it
taught me the importance of defining criteria explicitly and using measurable terms, so
inter-rater reliability can be upheld. Overall this project was very hands-on and useful in
terms of developing and evaluating assessments.
Halo
Developing a placement test was one type of assessment I desperately needed in
order to assess students language knowledge in all six areas of listening, speaking, reading,
writing and grammar for a language school I have worked at for many years. This test will
help the school to make reliable decisions about accommodating learner level since the
school has always used only speaking and listening tests for placing learners in their levels.
Developing this test was valuable for me because it made me think about aspects of
tests I had not considered before as I had mostly used existing quizzes and tests to assess
students level of proficiency. For example, one aspect of test development I learned was
how to make sure I create distractors in multiple-choice type questions that are not
randomly chosen. If distractors are not close enough to the correct answer in terms of
categories (e.g., part of speech), it may make it easier for test takers to guess the correct
answer rather than choosing the correct answer based on what they actually know to be
the correct answer.
Other aspects of tests I realized were clarity of instructions and length of a test. As
for clarity, I discovered that test developers should be very explicit about instructions for a
particular test like a placement test.This clarity can determine whether the test taker
actually knows/does not know the material or if he/she was misled by vague/confusing
instructions. In terms of length of a test, I believe if a test is thought to be too long for a test
taker, it may end up not precisely measuring a test takers ability as the test taker may
become exhausted and as a result lose concentration.
Broc
In the development of this English proficiency test for the purpose of student
placement, I have had the opportunity to gain a fuller understanding of assessment design,
implementation of the pilot, analysis and reflection of results. For myself, there are several
aspects of this project that I valued highly in terms of usefulness to my career goals and will
utilize throughout my career. The creation of the test items wasnt, on its own, difficult.
However, creating items while keeping in mind the specific learning outcomes that I
wanted to asses was somewhat difficult. In my experience, previous to my enrollment in
CSU, I had created several medium-stakes assessments while teaching in an IEP in Brazil.
While there I was charged with developing a summative achievement test for an
intermediate course. Without critiquing myself too harshly, I will simply state that in
hindsight, my test probably relied too heavily on cloze-item and selected-response item
types. Determining inter-rater reliability has been a process that has not been easy to
accomplish, but we were able to use two raters for the speaking and the writing portions of
test that I was able to compute the standard error of measurement (SEM). This showed that
 Grattan 12

for the areas of the test that we felt were susceptible to issues with reliability, that our
rubrics were adequate to ensure reliable results across raters. The area of the test that I
felt was my greatest success in terms of (face) validity was the second item in the speaking
section, which was an interview. Questions asked in this section elicited the vocabulary and
language structures targeted in the construct.
One of the more challenging aspect of in developing this test was the speaking
rubric for the rater to use. When scoring all of the speaking tests, I found that I weighed
some of the criteria more heavily than I should of. Before this test is used for actual
placement, the speaking rubric needs to be revised to have the scoring criteria reflect the
learning outcomes more fairly. When analyzing the results of the test, I found that there
were several items in the first part of the listening section that need to be revised before
the implementation of this test. The two listening items are numbers 2 and 3, both of which
had distractors that I believe were too difficult for the students to eliminate. This is because
when analyzing the test results, the only time these two questions were missed, was when
the students selected the same wrong answer.
In summary, this project has greatly expanded my understanding of what it takes to
develop a test. One aspect of this project that I have used for other projects both personal
and professional and has continued to show its value is feedback. Receiving feedback on
our test from our professor after we created our first draft helped to highlight test items
and design issues that we were bias to. Lastly, although the purpose of this test is to place
students in an existing program, I believe that the tools and strategies that I have acquired
through the process of creating this test and working with Halo and Sharayah can be
applied to many different areas within assessment.

















 Grattan 13

References
Alderson, J. C. (2000). Assessing reading. Cambridge language assessment series.
Alderson, J. C., Figueras, N., Kuijper, H., Nold, G., Takala, S., & Tardieu, C. (2004). The
development of specifications for item development and classification within The
Common European Framework of Reference for Languages: Learning, Teaching,
Assessment: Reading and Listening: Final report of The Dutch CEF Construct
Project.
Brown, J. D. (2003). Criterion-referenced item analysis (item facility and item
discrimination).
Shiken: JALT Testing & Evaluation SIG Newsletter, 7(2), 16-19.
Miller, M. D., Gronlund, N. E., & Linn, R. L. (2009). Measurement and Assessment in Teaching
(10th ed.). Upper Saddle River, NJ: Merrill / Pearson education international.
Plakans, L., & Gebril, A. (2017). Exploring the relationship of organization and connection
with
scores in integrated writing assessment. Assessing Writing, 31, 98-112.
Purpura, J. E. (2004). Assessing grammar. John Wiley & Sons, Inc.: Cambridge
University Press.
Richards, J. C., Hull. J., Proctor. S. (2013). Interchange (4th Ed). New York, NY: Cambridge
University Press.





















 Grattan 14

Appendix A: TLU Domain Description

Students in a General English program living in Kurdistan (EFL) take integrated exam based on curriculum
from textbook used in course to determine placement in level 1.

Characteristics of the setting
physical characteristics Classroom
participants Adult student
time of task daytime
Characteristics of the test rubric
instructions
language English
channel Written
specification of procedures and tasks Briefly explained in writing
structure 53 items
time allotment 60 minutes
scoring method
criteria for correctness Varying between sections
procedures for scoring the response Partial, full, or none
explicitness of criteria and procedures Vary between section
Characteristics of the input
format
channel Aural, visual, written
form language
language target language
length Vary between section
type Written passages, audio recordings/scripts
degree of speededness Slowed speed
vehicle Reproduced or live
language of the input
language characteristics
organizational characteristics
grammatical Verb tenses, articles, demonstratives, adverbs of frequency,
question formation, parts of speech, comparative and
superlatives, conjunctions, full set of phonological features,
simple & frequent vocab, statements & questions, but often
phrasal
textual Dialogue exchange, receptive
pragmatic characteristics
functional ideational
sociolinguistic casual, colloquial, natural
topical characteristics daily activities/life
Characteristics of the expected response
format
channel oral, written
form language
language target
length short conversational exchanges (oral), short passages
(written), short/one word (grammar/vocab)
type Spoken, written, select response
 Grattan 15

degree of speededness normal speed

language of the expected response
language characteristics
organizational characteristics
grammatical full set of phonological features, simple & frequent vocab,
statements & questions, but often phrasal, verb tenses,
articles, demonstratives, adverbs of frequency, question
formation, parts of speech, comparative and superlatives,
conjunctions
textual short series of exchangesconversation, paragraph
pragmatic characteristics
functional ideational
sociolinguistic informal, colloquial, natural
topical characteristics daily activities/life
Relationship between input and response
reactivity Reciprocal, non-reciprocal
scope of relationship Narrow or broad (depending on section)
directness of the relationship direct

























 Grattan 16

Appendix B: Table of Specifications

Task Comprehensio Application # Items # Points %
n s Points

Listening 8 (L1-8) 8 8 12%

Vocabulary 10 (V1-10) 10 10 14.5%

Grammar 5 (G11-15) 10 (G1-10) 15 16 23%

Reading 9 (R1-9) 7 (R10-16) 16 17 24%

Writing 2 (W1-W2) 2 10 14.5%

Speaking 2 (S1-2) 2 8 12%

# Items 53
items

# Points 66
points

% Points 100%

 Grattan 17

Appendix C: Specific Task Types for Six Language Areas

Language Area Sub Skills

Listening Main idea, specific details

Vocabulary Collocations, vocabulary precision

Grammar Tense and aspect mastery, precision in

using articles, pronouns and prepositions

Reading Skimming, scanning, inferencing

Writing Coherence, cohesion, sentence completion,

language use, grammar, vocabulary

Speaking Pronunciation, fluency, grammar,

vocabulary range























 Grattan 18

Appendix D: Copy of Test

Name:_________________
Date:__________________

Interchange Placement Test: Level 1

The purpose of this test is to determine if you are ready for level 1 in The English
Club program.

Directions:
You will have one hour to complete the Listening, Vocabulary, Grammar,
Reading and Writing portions of the test.

The speaking section will be taken immediately after the other sections.

Read all instructions carefully.

Listening: Section 1 (4 pts)

Directions: Listen to the person talk about their favorite place. After you listen, read
the questions and circle the best answer. For each question, choose only one answer.

1. What did John do at the park with his family and friends?
a. Played soccer and had a picnic
b. Fished
c. Played soccer and Swam.

2. John always goes to the park with___________?
a. His Family and Friends
b. His girlfriend
c. His dog Tony

3. What is the main idea of the story?
a. Johns day at the park
b. Finding a good place to exercise
c. Spending time outside

4. What is Johns favorite time to go the the park?
a. In the morning
 Grattan 19

b. At sunset
c. In the afternoon


Listening: Section 2 (4 pts)

Directions: Listen to the conversation between Bruce and Anna about giving advice.
After you listen, read each sentence and circle (T) if it is true and (F) if it is false.

5. Anna is late for class. T F
6. Bruce missed the bus. T F
7. All bikes are expensive. T F
8. Anna tells Bruce to visit a bike shop. T F

Vocabulary: Section 1 (5 pts)

Directions: Write the correct word (person, place or object) in the blank next to each
statement. More than one word is possible for some sentences (synonyms). An
example is given to you.
Example: This person works in a school. Teacher

1. This person serves food and drinks to passengers on a plane. ___________________

2. People exercise in this place to keep fit and become healthier. ___________________

3. People wear these on their hands when it is very cold outside. ___________________

4. This person bring you food and drinks in a restaurant. _______________

5. People can borrow books from this place. __________________


Vocabulary: Section 2 (5 pts)

Directions: Fill in the blanks with the words in the box below. Words can be used
more than once. One word will not be used.


visit have ride watch take go
 Grattan 20

6. Morgan and I usually ___________ breakfast around 8:00 am.
7. My friends do not __________ a vacation every year.
8. I didnt___________ a movie yesterday.
9. I am planning to __________ Norway next year if I have enough money.
10. Marie and Kelsea usually ___________ their bikes to school.

Grammar: Section 1 (10 pts)

Biography about Halo

Directions: Read the bio below. Write the correct form of the verbs in the blanks next
to the verbs given. For some, more than one answer is possible. For example,
(Amanda lives/is living in the US now).

Hello! My name is Halo. I come from Iraqi Kurdistan. Let me tell you a bit about my family. I
have one brother. His name is Mizhda. Currently he (1) _________(live) in Kurdistan and he
(2)__________(work) in a language school now. My mother (3)__________(visit) with me right
now here in the US. She (4) __________ (like) it here a lot. That is all about my family.

I have (5)_______ (be) an English teacher for about 7 years now. I (6)_________ (work) at The
English Club before I (7)_________ (come) to the US. Now I (8)________ (study) Teaching
English as a Second/Foreign Language at an American university in Colorado. I (9)__________
(like) it a lot and I have (10)__________ (make) great friends here in the last year and a half. I
am hoping to make some more.

Grammar: Section 2 (6 pts)

Directions: Read the sentences below and choose the correct answer.

11. A: There is _______ man at the door looking for his dog. I have never seen him before.
a. a
b. an
c. the
B: What is _______ wearing?
a. he
b. his
c. him

12. We have not seen Adele _______ two weeks now
 Grattan 21

a. for
b. from
c. since

13. My brother didnt _______ to the party last night. He just stayed home and slept.
a. came
b. come
c. coming

14. A: I have two sisters and a brother.
B: Id love to meet _______
a. theirs
b. them
c. they

15. Sharayah is _______ student in my class.
a. taller
b. tallest
c. the tallest


Reading: Section 1 (6 pts)

Directions: Read the email below. Write your answers to the questions based on the
information provided in the email on the blanks next to the questions. The responses
do not have to be in full sentences.

Hello Carol,

I finally found a computer and can tell you about my trip so far! Barcelona is a beautiful city
with lots of restaurants, shops, and things to do. I spent three days in the city, and I met
many friendly people from all over the world. Next, I am heading to Madrid to visit Thomas,
and go to some museums and parks. I will show you all the pictures I took when I get home
in one week.
-Jessie

1. How many days did Jessie spend in Barcelona? ___________________
2. Where is Jessie going after she leaves Barcelona? ____________________
3. Who is Jessie visiting? ____________________
4. What two places will Jessie go to in Madrid? ______________ _______________
 Grattan 22

5. When will Jessie return from her trip? ____________________

Reading: Section 2 (11 pts)

Read the article about going out to eat at a restaurant. Complete parts A and B below,
based on information provided in the article.

In America, going out to eat involves a series of steps that provide an enjoyable
dining experience for the customer. When you walk into a restaurant the host or hostess
will ask you how many people will be eating, take you to your seats, and give you menus.
Next, the waiter or waitress will ask you if you want anything to drink. After the waiter or
waitress brings you the drinks he/she will take your food order. He/She will bring you the
food and stop by at least one or two more times to make sure everything tastes good and
ask if you would like dessert. Last, the waiter or waitress will bring you the check.

Part A: True or False (4pts)

Directions: Answer the following four true/false questions by circling either T for
true of F for false.

6. T F The host or hostess takes your drink order.
7. T F The drinks and food are brought out at the same time.
8. T F Food is delivered before dessert orders are taken.
9. T F At the end of the meal, the waiter or waitress brings the check.

Part B: Complete the Steps (7pts)

Directions: Read the steps about two people going out to eat at a restaurant. Then,
number each step according to the order in which they should take place.

10. ______ The host gives Jeff and Sam menus and leaves two more menus on the table.
11. ______ Jeff and Sam tell the host they are expecting two more people.
12. ______ The waiter takes the food orders, and brings them out.
13. ______ The waiter brings the check.
14. ______ The host shows Jeff and Sam to their table.
15. ______ The waiter takes the drink orders and brings them out.
16. ______ The waiter checks on the table and asks if anyone wants dessert.

Writing Section (10pts)

 Grattan 23

Directions: Choose 2 of the activities from the list below that you have done before.
Write 3 sentences about each (6 sentences total). Describe each activity and why you
like to do it. Make sure your responses are in full sentences.

List of activities (choose only 2!)
Playing sports
Watching TV
Talking with friends
Driving a car
Swimming
Going to the movies
Reading
Going to the gym

Activity 1: (5pts)
______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

Activity 2: (5pts)
______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________

Speaking: Section 1 (4 pts)

Directions: Read the following passage out loud at a normal speed. Please make sure
to stop or pause when punctuation requires it.

Normally I wake up around six, but I always want to sleep in. For breakfast I usually have
eggs and tea. Then I prepare my lunch and dinner and Im off to work. Im a photographer,
there are people who come into my studio all day because they want their pictures taken.
 Grattan 24

Sometimes families come in to get a family photo, and they can be pretty loud. Other times,
people have me go to their wedding to photograph them, theyre always so happy to see the
photos of them. I try to leave my studio around seven to make it home by eight. Most days
Im too tired to do anything, so I just watch TV and go to sleep.


Speaking: Section 2 (4 pts)

Directions: You are now going to be asked several questions on different topics from
the Interchange Level 1 textbook. Please answer them in complete sentences to the
best of your ability.





























 Grattan 25

Appendix E: Answer Key

Listening
Seciton 1:
1. A
2. C
3. A
4. B

Section 2:
1. F
2. T
3. F
4. T

Vocabulary:
Section 1: (5 pts) (articles a/an or the and spelling mistakes do not influence the
correctness). Answers should be plausible.
1. Flight attendant/steward(-ess)
2. Gym/garage/rec center
3. Gloves/mittens
4. Waiter/Waitress/server
5. Library

Section 2: (5 pts) (number 10 can be worth half credit if one answer is correct)
6. Have
7. Take/have
8. Watch
9. Visit
10. Ride/take


Grammar:
Section 1: (10 pts)
Lives/is living
Works/is working
Is visiting
Likes

Have been
Worked
 Grattan 26

Came
Am studying/study
Like
Have made

Section 2: (6 pts)
11. B
12. C
13. A
14. C
15. C


Reading:
Section 1: (6pts)
1. Three (3)
2. Madrid
3. Thomas
4. Museums and parks
5. One (1) week

Section 2 Part A: (4pts)
6. F
7. F
8. T
9. T

Section 2 Part B: (7pts)
10. 3
11. 1
12. 5
13. 7
14. 2
15. 4
16. 6

Writing and Speaking answer keys will be in the form of separate rubrics.



 Grattan 27

Appendix F: Rubrics
Student Name:__________________ Rater Name:_______________________ Score:_____/10
Writing Rubric
Assignment Definition: Test takers write 6 sentences about two activities provided on a list.
They will be graded on length, coherence, organization, content and language use.
Scores 10 8 6 4 2
Length 2pts 5-6 complete 4-5 complete 3-4 complete 2-3 complete 1-2 complete
sentences sentences sentences sentences sentences
Content/Coh *Easily *Identifiable *Representation of *Unclear *Ideas are not
erence 4pts identifiable representation relevant representation of representative of
representation of of relevant information needs relevant relevant
relevant information. clarification, but information, hard information.
information. *Generally clear can be understood. to understand. *No *Words and
*Clear and and logical *Followable logical progression phrases instead of
unambiguous progression of progression of of ideas across text complete sentences
progression of ideas across the ideas across the and connection to related to activities
ideas across all sentences about sentences about activities is lost. *No
sentences about each activity. each activity. *Little to no demonstration of
each activity. *Some *Possible familiarity with familiarity with
*Obvious understanding familiarity with activities is shown activities is
understanding of of activities activities through through content. shown through
activities through through ability with little One or no content. No
ability to explain. to explain. explanation. explanations. explanations.
Language Use *A variety of *Mostly well *Use of a few words *One or two words *No words that
4pts words that chosen words that acceptably that somewhat describe the two
accurately that that describe the two describe the two activities.
describe the two accurately activities. activities. *No grammatical
activities. describe the two *Grammatical *Frequent issues consistency or
*Consistently activities. inconsistency and a with grammatical correct use of
accurate *Generally few word use consistency and grammatical
grammatical accurate use of errors. grammatical functions.
functions. grammatical *Some spelling functions. *Several spelling
*Correct form functions. mistakes and *Multiple spelling errors and
and meaning of *Correct form distorted errors and incorrect word
vocabulary and meaning of meanings. incorrect word choice/meanings.
words with no vocabulary choice/meanings.
spelling errors. words with few
spelling errors.
Comments:
 Grattan 28

Speaking Task 1: Reading Out Loud Rubric

Criteria 1 point .5 points 0 Score
Fluency Can say some sentences Speech is unintelligible, Did not make an
at a natural pace without must pause to sound out attempt
pausing or hesitating most words
often
Vowel/Consonant Can speak with limited Consistent mistakes in Did not make an
Pronunciation errors in sentence stress stress or vowel/consonant attempt
or vowel/consonant pronunciation,
pronunciation, does not
pauses/stops for pause/stop for
punctuation punctuation
Linking Sounds Links some sounds Does not link sounds, Did not make an
together like want to speaks each word attempt
wanto individually
Reduction of Sounds Reads one of, or both Does not read the words Did not make an
theyre and/or Im theyre and Im correctly attempt
Comments:

Speaking Task 2: Interview Rubric
Criteria 1 point .5 points 0 Score
Fluency Speech is intelligible, can Speech is limited, pauses Did not make an
string together simple and hesitation, not attempt
sentences, connects speech connected to topic, heavy
with topic use of fixed expressions
Pronunciation Can speak without errors Consistently mistakes in Did not make an
for small segments (at a stress, vowel/consonant attempt
simple sentence level) pronunciation
Vocabulary Range Shows limited development Consistently repeats Did not make an
of ideas, can use several vocabulary, cannot attempt
different to describe express basic ideas
Grammar Can speak without common Consistently misuses Did not make an
errors for small segments articles and verb tenses, attempt
(at a simple sentence level) no subject/verb
agreement
Comments:

 Grattan 29

Appendix G: Reading Passage and Listening Section Scripts

Speaking Passage
Directions: Please read the following passage at a normal pace.

Normally I wake up around six, but I always want to sleep in. For breakfast I usually have
eggs and tea. Then I prepare my lunch and dinner and Im off to work. Im a photographer,
there are people who come into my studio all day because they want their pictures taken.
Sometimes families come in to get a family photo, and they can be pretty loud. Other times,
people have me go to their wedding to photograph them, theyre always so happy to see the
photos of them. I try to leave my studio around seven to make it home by eight. Most days
Im too tired to do anything, so I just watch TV and go to sleep.

Listening Section Scripts

Script 1
Im John and my favorite place to spend time in is the park, I have spent time outside since
I was a boy, even if the weather isnt good. Yesterday I went there with my friends and
family to have a picnic. We get to spend the whole day playing soccer and eating. I always
bring my dog, Tony, he loves running and playing with the other dogs in the park, which is
huge and even has a small lake where some people go swimming or fishing. My favorite
time to be in the park is sunset, so I can see the stars before it gets too cold and I go home.

Script 2
A: Hey Anna! How have you been?
B: Hey Bruce! Ive been alright, you?
A: Im late again.
B: Again?
A: Yeah, I missed the bus and had to walk here.
B: Thats awful, why dont you ride a bike?
A: Id like to, but arent they expensive?
B: Some of them are, but you can find cheap ones too.
A: Really? What would you do, any suggestions?
B: Hmm, well Im not an expert, so if I were you, I would go to a bike shop and ask.
A: Okay, that sounds like a good idea, but would it really be faster?
B: I think so, and you wouldnt need to wait for the bus.
A: Thanks, oh geez, Im still late! See ya!
B: Bye!



 Grattan 30

Appendix H: Separate Directions for Speaking

Task 1

Directions for student: Please read the following passage at a normal pace.

Normally I wake up around six, but I always want to sleep in. For breakfast I usually have
eggs and tea. Then I prepare my lunch and dinner and Im off to work. Im a photographer,
there are people who come into my studio all day because they want their pictures taken.
Sometimes families come in to get a family photo, and they can be pretty loud. Other times,
people have me go to their wedding to photograph them, theyre always so happy to see the
photos of them. I try to leave my studio around seven to make it home by eight. Most days
Im too tired to do anything, so I just watch TV and go to sleep.

Task 2

Directions for the Rater/Teacher: Start with the first question in the list and move down.
If the conversation deviates from the questions but covers the same communicative
functions, continue with the conversation.

Directions for Student: We are going to have a short interview, where I am going to ask
you some questions and you try to answer the best you can, if you do not understand a
question, ask and it I will repeat it.

Questions:
1. Tell me about yourself? Which city are you from? Or, why you are studying English?
2. Can you talk about your daily routine? What you usually do every day?
3. What is your family like? Do you have many brothers or sisters?
4. What do you like to do in your free time?
5. Whats your favorite movie? Singer? Author?
6. What was the most important event/thing that happened to you?
7. Which do you think is better, going two a movie or going to a park? Why?
8. Why do you want to learn English? What will you do after you complete this course?








 Grattan 31

Appendix I: Score Report

Total Score for Listening Section: ______/8

Total Score for Vocabulary Section: ______/11

Total Score for Grammar Section: ______/16

Total Score for Reading Section: ______/16

Total Score for Writing Section: ______/10

Total Score for Speaking Section: ______/8


Total Points: _______/66


Percentage: _______%



Cut Score for Level 1 is 40% or 26 Points

















 Grattan 32

Appendix J: Item Statistics

Overall Scores
Name Listening Vocabular Gramma Reading Writing Speaking Total
___/8 y r ___/17 ___/10 ___/8 ___/66
__/10 ___/13
Ahmed 7 4 7 15 9.5 6 48.5
Banaz 8 6 9 13 9 2 47
Gulshan 4 8 7 16 0 6.5 41.5
Maria 7 6 8 11 8.5 7 47.5
Muhammed 7 8 10 14 7.5 7.5 46.5
Nawa 8 9 11 17 8 8 61
Przha 7 7 7 13 6.5 6.5 47
Shaima 7 9 10 15 10 8 60
Shaniar 8 8 8 12 6 7.5 49.5
Shara 8 7 7 9 6.5 3.5 41
Shelan 4 7 7 11 7.5 2 38.5
Stormy 6 8 9 12 9.5 8 52.5


Listening Section 1
Name Q1 Q2 Q3 Q4 Total
Ahmed 1 1 0 C 1 3
Banaz 1 1 1 1 4
Gulshan 1 0 A 1 1 3
Maria 1 1 0 C 1 3
Muhammed 1 0 A 1 1 3
Nawa 1 1 1 1 4
Przha 1 0 A 1 1 3
Shaima 1 0 A 1 1 3
Shaniar 1 1 1 1 4
Shara 1 1 1 1 4
Shelan 0 B 0 A 1 1 2
Stormy 1 0 A 0 C 1 2


Listening Section 2
Name Q5 Q6 Q7 Q8 Total
Ahmed 1 1 1 1 4
Banaz 1 1 1 1 4
 Grattan 33

Gulshan 0 T 0 F 0 T 1 1
Maria 1 1 1 1 4
Muhammed 1 1 1 1 4
Nawa 1 1 1 1 4
Przha 1 1 1 1 4
Shaima 1 1 1 1 4
Shaniar 1 1 1 1 4
Shara 1 1 1 1 4
Shelan 1 1 0 T 0 F 2
Stormy 1 1 1 1 4

Vocabulary Section 1
Students Q1 Q2 Q3 Q4 Q5 Total
Ahmad 0 0 1 0 1 2
Gulshan 0 1 1 1 1 4
Maria 0 1 1 0 1 3
Muhammed 0 1 1 0 1 3
Nawa 1 1 1 1 1 4
Przha 0 1 1 0 1 4
Shaima 0 1 1 1 1 4
Shaniar 0 1 1 1 1 4
Shara 1 1 0 1 1 4
Shelan 0 0 1 1 1 4
Stormy 0 1 1 1 1 4

Vocabulary Section 2
Students Q1 Q2 Q3 Q4 Q5 Total
Ahmad 1 0 1 0 1 3
Gulshan 1 1 1 0 1 4
Maria 1 0 1 0 1 3
Muhammed 1 1 1 1 1 5
Nawa 1 1 1 0 1 4
Przha 1 0 1 1 1 4
Shaima 1 1 1 1 1 5
Shaniar 1 0 1 1 1 4
Shara 1 1 1 0 1 4
Shelan 1 1 1 1 1 5
Stormy 1 1 1 0 1 4
 Grattan 34

Grammar Section 1
Students Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 Total
Ahmad 0 0 0 0 0 0 1 0 1 1 3
Banaz 1 1 1 1 1 1 1 0 0 1 8
Gulshan 0 1 0 0 1 1 1 0 0 1 5
Maria 1 1 0 1 1 0 0 1 1 0 6
Muhammed 1 1 1 0 1 1 1 1 0 1 8
Nawa 1 1 1 1 1 1 0 1 1 1 9
Przha 1 0 0 0 1 1 1 0 1 0 5
Shaima 1 1 0 1 0 0 1 1 1 1 7
Shaniar 1 1 0 1 0 1 0 0 0 1 5
Shara 0 1 0 0 1 1 0 0 1 0 4
Shelan 1 1 0 1 0 0 0 1 1 0 5
Stormy 1 0 0 1 1 1 1 0 1 1 7


Grammar Section 2
Students Q1 Q2 Q3 Total
Ahmad 1 0 1 2
Banaz 0 1 1 2
Gulshan 1 1 0 2
Maria 1 1 0 2
Muhammed 1 0 1 2
Nawa 1 1 0 2
Przha 1 1 0 2
Shaima 1 1 0 2
Shaniar 1 1 0 2
Shara 1 1 1 3
Shelan 0 1 1 2
Stormy 1 1 1 3


Reading Section 1
Students Q1 Q2 Q3 Q4 Q5 Total
Ahmed 1 1 1 1 1 5
Banaz 1 0 1 1 1 4
 Grattan 35

Gulshan 1 1 1 1 0 4
Maria 1 1 1 1 0 4
Muhammed 1 1 1 1 1 5
Nawa 1 1 1 1 1 5
Przha 1 1 0 1 1 4
Shaima 1 1 1 1 1 5
Shaniar 1 1 1 1 1 5
Shara 1 0 1 1 1 4
Shelan 1 1 1 1 1 5
Stormy 1 1 1 1 1 5


Reading Section 2 Part A
Students Q1 Q2 Q3 Q4 Total
Ahmed 1 0 (T) 0 (F) 1 2
Banaz 1 1 1 1 4
Gulshan 1 1 1 1 4
Maria 1 1 1 1 4
Muhammed 0 (T) 1 1 1 3
Nawa 1 1 1 1 4
Przha 1 1 1 1 4
Shaima 1 1 1 1 4
Shaniar 1 1 1 1 4
Shara 1 0 (T) 0 (F) 1 2
Shelan 1 0 (T) 0 (F) 1 2
Stormy 1 0 (T) 1 1 3

Reading Section 2 Part B
Students Q1 Q2 Q3 Q4 Q5 Q6 Q7 Total
Ahmed 1 1 1 1 1 1 1 7
Banaz 0 (4) 1 1 1 1 0 (6) 0 (3) 4
Gulshan 1 1 1 1 1 1 1 7
Maria 0 (2) 0 (3) 0 (4) 1 0 (1) 0 (5) 1 2
Muhammed 1 0 (2) 1 1 0 (1) 1 1 5
Nawa 1 1 1 1 1 1 1 7
Przha 0 (2) 0 (3) 1 1 0 (1) 1 1 4
Shaima 1 0 (2) 1 1 0 (1) 1 1 5
Shaniar 1 0 (7) 0 (2) 0 (4) 0 (1) 0 (5) 1 2
 Grattan 36

Shara 0 (7) 0 (2) 1 0 (3) 0 (1) 1 1 3

Shelan 0 (1) 0 (4) 1 1 1 0 (6) 0 (3) 3
Stormy 1 0 (2) 0 (4) 1 1 0 (5) 1 4

Writing Section
Content/Coherenc
Students Length e Language Use Total
Ahmed 2 4 3.5 9.5
Banaz 2 4 3 9
Gulshan 0 0 0 0
Maria 2 3 3.5 8.5
Muhammed 2 2.5 3 7.5
Nawa 1.5 3.5 3 8
Przha 1 2.5 3 6.5
Shaima 2 4 4 10
Shaniar 1 2 3 6
Shara 1 2.5 3 6.5
Shelan 2 3 2.5 7.5
Stormy 1.5 4 4 9.5

Speaking Section 1
Students Fluency Pronunciatio Linking Reduction Total
n
Ahmed 1 0.5 1 0.5 3
Banaz 0
Gulshan 0.5 0.5 1 1 3
Maria 1 1 1 1 4
Muhammed 1 0.5 1 1 3.5
Nawa 1 1 1 1 4
Przha 0.5 0.5 1 1 3
Shaima 1 1 1 1 4
Shaniar 1 0.5 1 1 4
Shara
Shelan
Stormy 1 1 1 1 4
 Grattan 37

Speaking Section 2
Students Fluency Pronunciatio Vocabulary Grammar Total
n

Ahmed 1 0.5 1 0.5 3

Banaz 0.5 0.5 0.5 0.5 2

Gulshan 1 1 0.5 1 3.5

Maria 0.5 1 1 0.5 3

Muhammed 1 1 1 1 4

Nawa 1 1 1 1 4

Przha 1 0.5 1 1 3.5

Shaima 1 1 1 1 4

Shaniar 1 1 1 1 4

Shara 1 1 1 0.5 3.5

Shelan 0.5 0.5 0.5 0.5 2

Stormy 1 1 1 1 4















 Grattan 38

Appendix K: Descriptive Statistics

Overall
Mean 48.375
Standard Error 1.984909928
Median 47.25
Mode 47
Standard
Deviation 6.875929689
Range 22.5
Minimum 38.5
Maximum 61

Listening Section
Mean 6.75
Standard Error 0.41056133
Median 7
Mode 7
Standard
Deviation 1.422226168
Range 4
Minimum 4
Maximum 8

Reading Section
Mean 12.33333
Standard Error 0.619547
Median 12
Mode 12
Standard
Deviation 2.146173
Range 7
Minimum 9
Maximum 16

 Grattan 39

Writing Section
Mean 7.375
Standard Error 0.769014
Median 7.75
Mode 9.5
Standard Deviation 2.663943
Range 10
Minimum 0
Maximum 10

Vocabulary Section
Mean 7.25
Standard Error 0.4105613304
Median 7.5
Mode 8
Standard
Deviation 1.422226168
Range 5
Minimum 4
Maximum 9
Sum 87
Count 12

Grammar Section
Mean 8.333333333
Standard Error 0.414387707
Median 8
Mode 7
Standard
Deviation 1.435481125
Range 4
Minimum 7
Maximum 11
Sum 100
Count 12


 Grattan 40

Speaking Section
Mean 6.083333333
Standard Error 0.662391596
Median 6.75
Mode 8
Standard
Deviation 2.294591797
Range 6
Minimum 2
Maximum 8



























 Grattan 41

Appendix L: Inter-rater Reliability

Speaking Rater Scores
Student R1 R1
Ahmed 6 6
Banaz 2 2
Gulshan 6.5 6.5
Maria 7 7.5
Muhammed 7.5 7.5
Nawa 8 8
Przha 6.5 6.5
Shaima 8 8
Shaniar 7.5 7.5
Shara 3.5 3.5
Shelan 2 2
Stormy 8 8

Speaking Inter-rater Correlation

Column 1 Column 2
Column 1 1
Column 2 0.998037988 1












 Grattan 42

Writing Rater Scores
Student R1 R1

Ahmed 9.5 6
Banaz 9 9

Gulshan 0 0

Maria 8.5 8

Muhammed 7.5 4.5

Nawa 8 9
Przha 6.5 7
Shaima 10 9.5
Shaniar 6 3.5
Shara 6.5 6
Shelan 7.5 6.5
Stormy 9.5 9

Writing Inter-rater Correlation

Column 1 Column 2
Column 1 1
Column 2 0.869589756 1