E-rater, ETSs proprietary scoring engine, is used in

Criterion to automatically evaluate essay quality. The

system targets aspects of writing fluency, including
genre-independent features (such as grammar, usage,
mechanics, spelling error detection, transition words),
and genre-dependent features (such as discourse
structure elements). In Criterion, e-rater is used to
provide a holistic score and diagnostic feedback related
to aspects in writing. The genre-independent features
are applicable to any genre of writing; the genredependent features, on the other hand, were designed
for expository and persuasive genres of writing that have
specific discourse structure (including thesis statement,
main points, supporting ideas, and conclusions).
As Criterion has been designed for expository and
persuasive writing, teachers may choose to write a
prompt other than expository or persuasive (such as
narrative). In this case, the genre-independent feedback
(including, grammar and spelling errors) will be relevant
and accurate, and the teacher should plan to provide
feedback for the content and discourse aspects. Genredependent discourse structure features are not relevant
for narrative (story) writing such as the BamBam
submission, and should be disregarded for this genre of
writing. The discourse structure of a story is very
different from that of an expository or persuasive writing
sample. Figure 1 illustrates how Criterions discourse
structure analysis applies to a persuasive essay.
Figure 1, below, illustrates an example output of a
persuasive essay where the discourse elements are
highlighted.
[To illustrate how e-rater can work well for discourse
structure, insert an image of an acceptable discourse
structure output of the kind of essay that e-rater is

intended to evaluate to show them how it is intended to

work. If there is a non-narrative example from 5 th grade,
try that, and if not, just use a 6 th grade example. There
should be examples of 6th grade/middle school
expository writing]
Figure 1. Persuasive essay labeled with discourse
structure elements
Below in Figures 2 and 3 are relevant, genreindependent analysis and score that e-rater produced.
As illustrated in Figure 2, below, usage errors detected
were accurately detected.
[insert usage feedback screenshot to show what was
found]
Figure 2. Usage error feedback
[insert transition words screenshot]
Figure 3. Transition words showing organizational
development
Criterion was designed to be a support resource to
teachers, intended to offer writing practice and facilitate
student-teacher dialogue. The Criterion best practice
philosophy was developed around the notion that
teachers would be present and working with students as
they write using Criterion. When teachers take the time
to write their own prompts, such as in the case of the
BamBamsubmission example where genre-dependent
features are unavailable, teachers will need to engage in
dialogue with students about aspects of writing, such as
discourse structure.

E-rater uses special methods for assigning scores to

Instructor Topics. The essay written by Tess A. titled
BamBam is a story (or narrative) genre. It was
submitted to Criterion as a Grade 5 Scored Instructor
Topic. Instructor topics allow teachers to write their own
prompts. E-rater uses its full feature set to assign a
score; the full feature set includes many features, some
of which are used as feedback, and others which are
used only for scoring purposes, and address aspects of
an essay such as vocabulary usage. In the case of the
BamBam essay, e-rater assigned a score of 5 out of 6
which indicates that for a 5th grade writing sample, this is
a pretty good piece of writing for the following reasons:
(1) it has few grammar and spelling errors; (2) it has
good use of vocabulary, including some words and terms
that support the coherence of the essay; (3) general
vocabulary choice is reasonable.
There are many different genres of writing, and this is
more recently emphasized by the demands of the
Common Core. E-rater was developed to serve
expository and persuasive modes of writing, though the
system is being continuously updated to expand the
genres and range of content that can be evaluated by
the system.