RELC Journal

http://rel.sagepub.com/

Verbal-Report data and Introspective Methods in Second Language

Research : State of the Art
Kazuko Matsumoto
RELC Journal 1993 24: 32
DOI: 10.1177/003368829302400103

The online version of this article can be found at:

http://rel.sagepub.com/content/24/1/32

Published by:

http://www.sagepublications.com

Additional services and information for RELC Journal can be found at:

Email Alerts: http://rel.sagepub.com/cgi/alerts

Subscriptions: http://rel.sagepub.com/subscriptions

Reprints: http://www.sagepub.com/journalsReprints.nav

Permissions: http://www.sagepub.com/journalsPermissions.nav

Citations: http://rel.sagepub.com/content/24/1/32.refs.html

>> Version of Record - Jun 1, 1993

What is This?

Downloaded from rel.sagepub.com by guest on September 23, 2013

Verbal-Report data and Introspective Methods in Second Language Research :
State of the Art

Kazuko Matsumoto
Gifu University, Japan

Abstract

This state-of-the-art article describes two types of verbal-report data

(i.e., concurrent and
retrospective) and four major introspective methods
used in second language (L2) research (i.e., thinking-aloud, questionnaires,
interviews, and diary-keeping). It then summarizes major introspective L2
research conducted to date, followed by a methodological review from ten
perspectives which serve as criteria for classifying and evaluating these
introspective L2 studies. It concludes with a discussion of presently
controversial issues amongst researchers over the use of introspection as a
research method for tapping L2 learners inner cognitive processes,
suggesting ways of minimizing concerns raised about verbal reports. It is
argued that although verbal reports from informant-learners may be an
incomplete reflection of actual internal processing, and enough care must
always be exercised in stages of data collection and interpretation, they
nevertheless contain useful information concerning learners mental
processes including their use of learning strategies which will not be
satisfactorily accessed through extrospective observational studies.

1. Introduction

Recent years have seen renewed respect for introspectionl in the

field of psychology as a method of uncovering informants cognitive
states in processing information. The current shift towards the use of
informants introspective reports has been motivated especially by
Ericsson and Simon (1980), who explicitly proposed within their
framework of human information processing that verbal reports are data,
and elicited and interpreted with care, are a valuable and reliable source
of evidence about human mental processes. In the field of second language
(L2) research, Cohen and Hosenfeld (1981), stressing the limitations of
inferring thought processes from extrospective observations of the learning
process, similarly called for investigating L2 learning by collecting
&dquo;mentalistic&dquo; verbal-report data from learners. Although their proposal
has been attacked by some L2 researchers (e.g., Seliger, 1983) mainly
with respect to the issue of the veridicality of verbal reports with actual
internal processing, L2 researchers have nevertheless been increasingly
interested in verbal-report methodology as a way of tapping learners

32
cognitive processes involved in L2 use or L2 learning/acquisition. Such
tendency among L2 researchers has been reflected most explicitly in
recently flourishing studies of learner strategies (Cohen, 1990; OMalley
& Chamot, 1990; Oxford, 1990; Wenden, 1991; Wenden & Rubin, 1987).

The purposes of this paper following:(a) to classify verbal-

are the
report data into two types (i.e., concurrent and
retrospective) and to
describe major verbal-report research methods currently being used in
L2 research (i.e., thinking-aloud, questionnaires, interviews, and diary-
keeping), (b) to summarize major introspective L2 studies which employed
these verbal-report techniques, (c) to methodologically review
introspective studies in terms of ten classification criteria, and finally,
(d) to discuss issues of controversy amongst researchers over the use of
verbal-report data in L2 research, focusing on the cons of the introspective
techniques that have been raised to date.
2. Two Forms of Verbal Reports and Four Major Introspective
Techniques Used in L2 Research
Verbal reports in L2 research can be broadly classified into two
forms, concurrent and retrospective, on the basis of two dimensions:
time of verbalization (i.e., during or after task) and the relation between
heeded and verbalized information (Ericsson & Simon, 1984, p. 12).
First, in concurrent verbal reporting, the verbalization is done during a
specific task given; thus the informant provides verbal self-reports while
information is heeded, that is, while information is still stored in short-
term memory (STM). The Ericsson and Simons (1984) model, in which
cognitive processes are seen as successive states.of heeded information,
assumes that the heeded information kept in STM is directly accessible
for producing verbal reports. In retrospective reporting, on the other
hand, the verbalization is given after the completion of the task-directed
processes, or it may sometimes be unrelated to any specific task.
Retrospective verbalizations thus involve retrieval of information from
long-term memory (LTM), which must first be transferred to STM before
it can be reported. In sum, concurrent reports are tied to a specific task
given by the researcher, whereas retrospective reports may or may not
be so. Second, the relationship between heeded and verbalized information
is direct in concurrent reporting, whereas retrospective reporting involves
mediating processes between attention to the information and its
verbalization, which may modify the stored information. In retrospective
verbalization, put another way, the heeded information becomes input to

33
intermediate processes, and thus the verbalized information is a product
of such intermediate processing. It is predicted that retrospective self-
reports that are not task-based will involve more mediating processes
than task-based retrospection.

The taxonomy of introspective verbal reports described above is

summarized in Table l.

Table 1
Characteristics of Concurrent and Retrospective Verbal Reports

2.1 Concurrent Verbalizations: Thinking-Aloud

As a verbal-report method of producing concurrent verbalization,
think-aloud procedures2 ask subjects/informants to tell researchers what
they are thinking and doing (i.e., everything that comes to mind) while
performing a task. The informants are usually instructed to keep thinking
aloud, acting as if they are alone in the room speaking to themselves.
They are sometimes prompted to talk when a long period of silence
occurs; they are also asked not to try to plan out what they say or try to
explain what they are saying. Think-aloud verbalizations are tape- and/
or video-recorded and then transcribed. The think-aloud protocols are

content-analyzed, and in many cases, coded for specific categories which

have previously been developed by the researchers.

2.2 Retrospective Verbalizations : Questionnaires, Intervievvs, and

Diary-Keeping
In most retrospective verbalizations subjects/informants are asked
to tell researchers what they have thought and done while performing a

particular task that has already been completed. In some cases,

retrospective verbal reports are provided by the informants immediately
after the task completion, even while the information is still stored in
STM, and thus directly available for verbalization. In other cases,

34
retrospective reports are given quite some time after a specific language

learning task is given or after several or many tasks have taken place. In
still other cases, retrospective reports are unrelated to any specific task,
being elucidated based on the learners past learning experiences in general
(as in the case of self-reports about learners beliefs and perceptions
about L2 learning). Retrospective verbal-report data can be elicited by a
number of techniques. Interviews, discussions, speeches, conversations,
and so on produce subject responses in oral form, while questionnaires,
diary-keeping, note-taking, and so on produce retrospective self-report
data in the written mode. The three major techniques for eliciting
retrospective verbal reports have been successfully utilized in L2 research,
especially in recent studies of the past decade: they are questionnaires,
interviews, and diary-keepln,a-

The structured questionnaire is one of the most restricted forms of

retrospective reporting, which asks informant-learners to agree or disagree,
or to answer yes or no to a series of statements or questions, or to

choose one out of a set of fixed alternatives (e.g., a number from 1 to

5). The structured interview, which is roughly equivalent to a face-to-
face administration of the structured questionnaire, involves a set of
fairly straightforward, prearranged questions often to be answered by a
simple yes or no. Formalized structured interviewing, like structured
questionnairing, thus exposes all informants to identical stimuli, and, in
pursuit of generalized statements, is often conducted with a large number
of people. This is regarded as the most systematic, most objective type
of interviewing, with the possibility of researcher/interviewer bias being
greatly reduced through the use of a predetermined interview schedule.
The unstructured interview, on the other hand, allows the greatest scope
for the interviewer to probe, introduce unprepared, new material into the
discussion, negotiate, and expand the interviewees responses, thus
creating a more equal relationship between the researcher and the
informant. Unstructured interviewing, however, involves the highest risk
of researcher/interviewer effect or bias, and is most sensitive to the
context of verbal interaction between the interviewer and the respondent.
The semi-structured interview, involving use of an interview guide, lies
between the two interviewing techniques in the degree of structuring, the
extent of objectivity and reliability, the degree of negotiation allowed
between the interviewer and interviewee, and the degree of equality
developed in the interviewer-respondent relationship (Bernard, 1988;
Fowler, 1988). Interviews are usually tape-recorded with note-taking
concurrently done, transcribed, and then content-analyzed by interviewers/

35
researchers, followed by, in some cases, quantification of the collected
data.

The keeping of a diary or a journal as a technique for eliciting

retrospective verbal reports is also ethnographic in nature. In the field of
SLA, those studies which employed this technique to tap learners mental
states and cognitive processes involved in L2 learning/acquisition
processes have been known as &dquo;diary studies&dquo; (Allwright & Bailey, 1991,
pp. 169-193; Bailey, 1991), which usually involve five major steps: (a)
provision of an account of the diarists personal L2 learning history, (b)
systematic recording of events, feelings, and so on about the current L2
learning experience by the diarist, (c) revision of journal entries for
public perusal, (d) analysis of the diary data for significant patterns and
events by the researcher, and (e) interpretation and discussion of the
factors identified as important to L2 learning experience (Bailey &
Ochsner, 1983). The diary studies may be categorized as &dquo;introspective&dquo;
(i.e., a researcher-as-diarist analyzes his/her own L2 learning processes)
or &dquo;non-introspective&dquo; (i.e., a researcher analyzes another diarist-learnerss
L2 learning processes) (Matsumoto, 1987, 1989); most of the early studies
conducted in the 1970s are categorized as &dquo;introspective&dquo; in the sense
defined above.

3. Introspective L2 Studies: A Survey

3.1 7?~~~-A/oM<~ .

Think-aloud procedures have been employed to investigate learners

ongoing cognitive processes and strategies in four major L2 areas, i.e.,
translation, reading, writing, and testing. In translation research, Hblscher
and Mbhle (1987) and Krings (1987) used think-aloud techniques with
German-speaking learners of French to elucidate cognitive processes
involved in written translations between Ll and L2. Gerloff (1987) also
collected concurrent think-aloud protocols from English-speaking learners
of French during performance of a written translation task. In L2 reading,
a series of Hosenfelds (1976, 1977, 1979, 1984) studies of reading

strategies involved think-aloud verbal data elicited individually in

combination with interview techniques. Block (1986), using think-aloud
procedures with both native and nonnative English speakers enrolled in
college-level remedial reading courses, investigated the comprehension
strategies used by these nonproficient readers as they read textbook
material in English. This descriptive study combined think-aloud data
with learners performance data elicited from two tasks designed to

36
measure memory (i.e., retelling) and comprehension (i.e., multiple-choice

tests) of the given passage. Haastrups (1987) study combined pair

thinking-aloud and retrospective interviews to explore Danish-speaking
ESL learners lexical inferencing procedures in reading comprehension.
In addition, Cavalcantis (1987) investigation of the L2 reading process
with Portuguese-speaking ESL learners employed what she terms &dquo;pause
protocols&dquo; (i.e., asking informants to read silently and to think aloud
whenever they noticed a pause in the reading process). In the field of L2
writing, Lay (1982), in combination with interviews, examined the
composing processes of four Chinese ESL students by having them
compose aloud (i.e., think aloud while composing). Raimes (1985) also
employed the composing-aloud method in her exploration of composing
processes of eight unskilled ESL writers in the classroom situation; in
this study think-aloud data were supplemented by questionnaire data and
proficiency test scores. The think-aloud composing technique was also
used in Raimes (1987) study, which investigated eight ESL college
students writing behaviors and strategies in two writing tasks dealing
with different topics; the think-aloud protocol data were analyzed in
relation to the subjects performance data collected from proficiency as
well as placement tests and holistic evaluation of their written products.
Similarly Jones and Tetroes (1987) analysis of six Spanish-speaking
ESL writers planning behaviours over a period of six months involved
the subjects think-aloud protocol as well as performance data. In addition,
a current study by Cumming (1989), exploring the relationship between

Ll writing expertise, L2 proficiency, and L2 writing performance, used

the compose-aloud procedures with 23 French-speaking ESL students.
Finally, use of thinking aloud in L2 testing research is best exemplified
by Feldman and Stemmers (1987) study which, in combination with
retrospective interviews, investigated German L2 learners mental
processes during C-test taking.

3.2 Questionnaire
Survey questionnaires in L2 research have been used mainly in
studies of learning styles and strategies, and in research on learners
beliefs, attitudes, and perceptions about L2 learning/acquisition. Reid
(1987), for example, administered a structured questionnaire to 1,388
students learning in intensive ESL programs in the U.S. to identify their
perceptual learning style preferences. Cohen (1987c) used a questionnaire
consisting of both multiple-choice and open-ended questions to investigate
strategies ESL learners employ for processing teacher feedback on their
written compositions. Cohen and Cavalcanti (1990), in their investigation
of EFL students attitudes toward, perceptions of, and strategies in

37
handling teachers comments on their written compositions, also employed
questionnaires in combination with individual interviews. Politzer and
McGroarty (1985), using a self-report questionnaire including 51 yes/no
questions related to assumed good learning behaviors with 37 Asian and
Hispanic ESL students, investigated the relationship of the self-reported
learning strategies and behaviors to gains on four ESL proficiency
measures (i.e., tests of linguistic competence, auditory comprehension,

overall oral proficiency, and communicative competence). Huangs (1985)

study, in which a three-part questionnaire was administered to 60 Chinese
EFL students, attempted to identify the general strategies and specific
techniques employed by Chinese learners of English for oral
communication. The questionnaire data of this study were supplemented
by interview data. Cohen and Aphek (1981), in their exploration of how
L2 learners take in teachers explanations of vocabulary and grammatical
rules during class lessons, employed prepared open questionnaires.
Furthermore, Banerjee and Carrells (1988) study, using a discourse
completion questionnaire involving 60 hypothetical situations, explored
differences in frequency, directness, type of suggestions made, and
politeness strategies used in these suggestions, between ESL students
and native speakers of American English.

Horwitz (1987), stressing close ties between student beliefs about

language learning and their language learning strategies, has developed
a language learning belief inventory, a multiple-choice-type questionnaire

to assess ESL students beliefs about language learning in five major

areas: L2 aptitude, difficulty of L2 learning, nature of L2 learning, learning
and communication strategies, and motivations. Kraemer and Zisenwiness
( 1989) cross-sectional study employed a self-rating questionnaire with
South African informants (i.e., school children from grades 4 to 12) to
investigate changes in their attitudes towards L2 learning. Ostlers (1980)
needs survey, using a questionnaire consisting of multiple-choice questions
of a self-evaluating nature and two sections for assessing students writing
skills, aimed at determining what academic skills (e.g., taking notes in
class) should be addressed in advanced ESL programs for possible syllabus
improvement. Also, Robertsons (1984) survey of English language use,
needs, and proficiency among nonnative English speakers studying at a
U.S. university in different disciplines, academic levels, and teaching
assistantship status employed a 72-item mailed questionnaire in
combination with telephone interviews. Survey questionnaires have also
been employed in recent CALL-related research, e.g., to investigate ESL
learners perceptions of computer-assisted writing (Neu & Scarcella, 1991;
Phinney, 1991).

38
 3.3 Interviews
Interviews have been employed extensively in recent descriptive,
interpretative L2 research, especially in studies of learning strategies,
studies of learner perceptions, beliefs, and attitudes, and research on the
processes (including strategies) of reading, writing and listening. In
learning strategy research, OMalley et al. (1985), for example, in
combination with classroom observations, interviewed high-school ESL
students and their teachers to elucidate the types, range, and frequency
of L2 learning strategies used by the learners with a number of language
learning tasks. In Abraham and Vanns (1987) case study, student
interviews and think-aloud procedures were combined to examine learning
strategies of a &dquo;good&dquo; and a &dquo;poor&dquo; ESL student. Holecs (1987) study,
in which a teacher-as-counsellor interviewed ESL students over a period
of three months, focused on examining how they manage their L2 learning
process and how they change as learners over time. Also, Portes (1988)
structured interviews with 15 under-achieving EFL learners centered on
uncovering learning strategies used when dealing with new vocabulary,
while Pearsons (1988) unstructured interviews with 12 Japanese
businessmen working in South East Asia investigated their ESL learning
styles and strategies in general. Knight et al. (1985) conducted individual
interviews with 23 Spanish-speaking ESL students and 15 third-and fifth-
grade English monolingual students in order to examine whether there
were differences between the two groups in the type and frequency of

self-reported cognitive strategies used as they read a given passage in

English. Cohen and Robbins (1976) conducted interviews as part of their
error analysis: their use of retrospective structured interviews with three
Chinese ESL students aimed at investigating the interrelationship between
the learners L2 background (including learning strategies), errors they
had produced in written work, and their explanations of the errors.
Gillettes (1987) introspective analysis of learner strategies, cognitive,
socio-cultural, and motivational variables of successful L2 learners also
involved unstructured interviews. The interview data of this study were
supplemented by questionnaires, classroom observations, and informal
discussions with the informants. Lennons (1989) retrospective verbal-
report data were elicited from four German-speaking college-level ESL
learners at the stage of initial natural exposure to the English-speaking
community by the techniques of interviews and written self-reports. The
analysis of this introspective study, like that of L2 diary research, focused
on psychological variables involved in the L2 learning process in general,

which includes learner strategies, perceptions, motivation, self-evaluation,

and self-awareness.

39
 In research on learners perceptions, beliefs, and attitudes, Wenden
(1987), for example, using semi-structured interviews supplemented by
a questionnaire, had adult ESL learners retrospect on various aspects of

their learning in order to capture their explicit prescriptive beliefs about

how best to approach L2 learning. Christison and Krahnke (1986)
conducted structured interviews, using a set of open-ended questions,
with 80 nonnative English speakers studying in U.S. colleges and
universities for the purpose of exploring students perceptions about their
L2 learning experiences, their attitude towards teacher behavior and
quality, and their use of English in academic settings.

In L2 reading Cohen et al. (1979) used retrospective structured

interviews with 7 Hebrew-speaking ESL readers to identify problematical
areas as they read ESP-related materials. Devine (1988) also employed

interviewing in her case study of two ESL learners reading processes,

with the informants performance data (i.e., oral reading and retelling)
being additionally analyzed. The use of interview techniques in L2 writing
research is best exemplified by Zamels (1983) study, which, investigating
the composing processes of 6 advanced-level ESL students, employed
individual interviews as well as observations of the informants writing
behaviors while composing and their written products (see also
Matsumoto, 1992a). Further, a current ethnographic case study by Benson
(1989), in exploring the process of an ESL students listening activities
during an academic course, employed extensive interviews not only with
the informant but also with his teacher and classmates in combination
with participant observation, recording of lectures, and examination of
the informant-learners notebooks.

3.4 Diary-Keeping
The diary-keeping technique has been employed by L2 researchers
over the past two decades, especially with the aim of exploring the

psychological and effective dimensions of L2 learning acquisition

processes. Schumann and Schumann (1977) and Schumann (1980), for
example, used the journal-keeping technique in their introspective
investigations of their own L2 learning/acquisition processes, identifying
personal variables affecting (promoting or inhibiting) the learning
processes. Their introspection involved three different L2 learning
situations: one with formal instruction, one with natural exposure to the
target language, and one with a mixture of both. Jones (1977)
introspective study of her own L2 learning experiences in the target
culture also employed diary-keeping during an 11-week intensive study

40
in the classroom, focusing on social and psychological factors influencing
the processes of L2 learning. Bailey (1980, 1983), in her introspective
study of her own L2 learning in a university classroom situation, used
the technique of journal-keeping, identifying and discussing several
variables affecting L2 learning processes, with special emphasis on two
affective factors, competitiveness and anxiety. In addition, Schmidt and
Frotas (1986) introspective case study of the development of
conversational ability in L? by the first author during a 5-month stay in
the target language country both with instruction and exposure also
involved diary-keeping. Grandcolas and Soul6-Susbielles (1986)
introspective study combined diary-keeping with informal discussion
among the informants/learners (i.e., French-speaking learners of EFL)
for the purpose of exploring variables importantly involved in the
classroom foreign language learning process. Lowe (1987) discusses the
findings of individually kept diaries of a group of teachers serving as L2
learners which were expressed in a public seminar; this diary study
includes both &dquo;introspective&dquo; and &dquo;non-introspective&dquo; analyses of L2
learning processes. Furthermore, Brown (1983, 1985b), employing
journal-keeping with English-speaking older and younger adult learners
of Spanish during an 8-week intensive classroom study, explored possible
differences in self-perceived language learning factors between the two
groups of learners. The diary data of this &dquo;non-introspective&dquo; study were
supplemented by participant observations and the subjects performance
data in the form of oral interview test scores. Another &dquo;non-introspective&dquo;
study by Ellis (1989), in its analysis of learning styles and their effects
on classroom language learning, utilized information from diaries kept

by two adult learners of L2 German in addition to data from

questionnaires, cognitive style testing, language aptitude tests, close tests,
and attendance and participation records. In addition, Matsumoto (1989),
employing diary-keeping techniques with a Japanese college-level ESL
student over an 8-week period of intensive classroom study in the U.S.,
explored personal factors, both emotional and non-emotional, which were
influential to, or deeply involved in, the informants process of classroom
L2 learning in the target culture environment. The diary data of this
study were supplemented by self-report data from a retrospective semi-
structured interview and an open-ended questionnaire.

4. Introspective L2 Studies : A Methodological Review

This section provides a methodological review of the introspective
L2 studies summarized above from ten perspectives, which are based on
the classification criteria proposed by Cohen (1984, 1987a, 1987b) and

41
Fxrch and Kasper (1987) by which diversified verbal-report research
can be classified and evaluated.

4.1 Characteristics of Informants

This first factor characterizing introspective L2 studies refers to
how many and what kind of informant-learners serve as verbal reporters
in the data collection stage. The number of informants may be just one,
as in case studies
(e.g., Benson, 1989; Jones, 1977), and that informant
may be the researcher himself/herself as in introspective diary studies

(e.g., Bailey, 1980; Schumann, 1980). Introspective self-reports can also

be collected with a group of learners at one time, as in Raimes (1985)
having had the whole class think aloud while writing. The informants
may differ in their age (e.g., adults or children), their level of L2
proficiency (e.g., elementary- or advanced-level), their native language
(e.g., Chinese- or Spanish-speaking), or their L2 learning background
(e.g., length of stay in the target culture).

4.2 Objectives of Introspection

The objectives of introspection in L2 research can broadly be
categorized into three: (a) investigation of learners cognitive process
(including strategies) involved in their L2 use in a specific task (such as
writing, reading, and translation) given by the researcher, (b) investigation
of learners beliefs, attitudes, and perceptions about language learning
which they have acquired based on past experiences, and (c) exploration
of the overall psychological dimension of the L2 learning/acquisition
process. The think-aloud protocol-based studies have the objective of the
first type, while the questionnaire and interview studies not tied to a
specific task (e.g., Politzer & McGroarty, 1985; Pearson, 1988) have the
purpose of the second type. L2 diary research can be characterized
typically by the research objective of the last type, where a holistic
exploratory investigation of the learning/acquisition process is pursued.
4.3 Characteristics of Data Collection
The characteristics of data collection refer to when, where and how
the informants verbal reports are collected, the how of data collection
including such features as duration, mode, and language of the
verbalization. The data may be collected during task as in concurrent
thinking-aloud or after task as in retrospective questionnairing,
interviewing, and diary-keeping. The non-task-based data may be collected
by questionnaires and interviews quite some time after the processing in
question has taken place. The verbalization may be done during class
lessons (e.g., Cohen & Aphek, 1981), in the researchers office (e.g.,

42
Block, 1986), in the language laboratory (e.g., Raimes, 1985), or elsewhere
(e.g., the informants working place as in Pearsons (1988) interview
research orthe informants residence as would usually be the case with
diary studies). Verbal reports may be collected cross-sectionally or
longitudinally (e.g., Holec, 1987, and diary studies). The mode of
verbalization is either oral as in thinking-aloud and interview studies, or
written as in questionnaire and diary research. Finally, verbal reports can
be collected in the informants Ll as in diary studies like Bailey (1980),
and Schmidt and Frota (1986), or in the target language (e.g., Cohen,
1987c), or in both L and L2 as in most of the studies involving think-
aloud protocols (e.g., Cumming, 1989; Gerloff, 1987). The informants
verbal facility in the target language should always be considered in the
data collection process so that verbalization difficulties will not mask
out emergence of some important mentalistic information (Garner, 1988).

4.4 Characteristics of Verbalized Information

As discussed in Section 2 based on Ericsson and Simon (1984), the
verbalized information may be directly tied to a specific task, which is
always the case with thinking-aloud, or it may not be so as in retrospective
self-reports of learners metacognitive knowledge about language learning.
The relationship between the heeded information and the verbalized
information may be direct as in concurrent thinking-aloud and immediate
retrospection, which are considered to involve STM, or indirect as in
delayed retrospection. If the processes intervening between the heeded
and verbalized information have the potential of changing the originally-
stored information in the case of delayed retrospection involving LTM,
this evidently constitutes a threat to the validity. It is recommended,
therefore, that care should always be taken to minimize the processing-
verbalization time lag and thereby decrease the number of intervening
intermediate processes (see, for example, Cohen and Apheks (1981)
questionnaire research which used immediate retrospection).
4.5 Degree of Structuring
The degree of structuring refers to the extent to which the
researcher controls the form and content of the informants verbalization.
Specifically, such structuring is done by the instructions given to the
informants in the case of studies involving thinking-aloud and diary-
keeping, and by the type of questions given to the respondents in the
case of interviews and questionnaires. According to this criterion, each
of the L2 introspective studies summarized above can be placed
somewhere on a continuum between the most highly structured to the
least structured. For example, as explained above, the questionnaire studies

43
can range from highly controlled closed-type to the least structured open-
ended type; the interviews can range from the structured to semi-structured
and to the unstructured ones. Diary studies can also range from the least
structured types, which provide the learners with freedom of writing on
any aspect of L2 learning experiences, to the structured ones, where the
informants are directed through instructions to focus on certain aspects
of the learning process which are of interest to the researcher. For example,
in Matsumotos (1989) case study, the diarist-learner, who was under
conditions of both formal instruction and exposure, was instructed to
limit her introspection to L2 learning experiences within the classroom.
Another example of structuring in Ll diary research is Schallert et al.
(1988), in which the informants were requested to keep a daily journal
record only when the instructor alluded to the textbook during classtime.

4.6 Researcher-Informant and Informant-Informant Relationship

This is one of the factors which have considerable influence on the
informants verbal reporting process and are closely related to the validity
of self-report data obtained. The relationship between the researcher and
the informant is an influential factor not only in introspection involving
direct interaction between them as in retrospective face-to-face interviews
but also in verbal reporting without direct interaction as in answering
mailed questionnaires or keeping individual diaries. Grotjahn (1987, p.65),
referring to the evaluation criteria of exploratory-interpretative
methodology, points out that the validity of the verbal-report data depends
on an appropriate communicative relationship which has been established

between the researcher and the informant. The importance of such

researcher-informant rapport has also been emphasized by Cavalcanti
(1982, p.76) in relation to thinking-aloud, and by Matsumoto (1987,
p.31) with reference to diary-keeping. Although few L2 studies have
involved informant-informant interaction to date (e.g., pair thinking-aloud
in Haastrup, 1987), the informant-informant relationship should also be
considered an influential factor on the participants verbal reports.

4.7 Informant Straining

Very few of the above-presented L2 introspective studies involved
training of the informants with the exception of think-aloud concurrent
verbalizations, which usually has a practice phase preceding the real
data collection sessions (e.g., Block, 1986; Cavalcanti, 1987; Cumming,
1989; Feldman & Stemmer, 1987; Hosenfeld, 1976). The possible effects
of such informant training on the subjects introspection have been
controversial; the important thing to be considered for obtaining valid
and reliable data seems to be a balance between the informants

44
knowledge of specific research purposes and the informants ignorance
of research techniques (Mann, 1982, p.91). It seems in general that when
the study is highly structured, requiring the informant to focus on a
specific aspect of L2 use or learning, informant training (e.g., giving
practice think-aloud tasks) will be useful, enabling the researcher to
check whether the instructions have been rightly followed. When the
study is least structured, on the other hand, the warming-up phase might
exert a negative effect, biasing the data largely toward the researchers
expected or desired direction. For instance, the informants introspection
might be influenced by the content of the diary entries presented as
sample reports in the practice session.

4.8 Data Quantification and Metlzod of Data Analysis

The informants verbal-report protocols can be analyzed
qualitatively (i.e., interpretatively without data quantification), or
statistically. The data can also be quantified and subjected to statistical
analysis, or analyzed interpretatively. In Grotjahns (1987) terms, in other
words, introspective L2 research can be categorized into either one of
the following methodological paradigms: the pure exploratory-
interpretative (i.e., qualitative data and interpretative analysis), the
exploratory-qualitative-statistical (i.e., qualitative data and statistical
analysis), exploratory-quantitative-statistical (i.e., quantitative data and
statistical analysis) and the exploratory-quantitative-interpretative (i.e.,
quantitative data and interpretative analysis) paradigms. For example,
Schumann and Schumanns (1977) and Baileys (1980) diary studies,
involving qualitative journal data and interpretative analysis without data
quantification, belong to the pure qualitative research paradigm. Brownss
(1983) diary research, on the other hand, involves statistical analysis as
well as data quantification, i.e., belongs to the exploratory-quantitative-
statistical paradigm given above. The exploratory-quantitative-
interpretative type of L2 diary research is exemplified by Matsumoto
(1989).

4.9 Degree of Benefit to the Informant-Learner

The degree to which the technique used is beneficial to the informant
as a learner varies. It seems in general that the methods eliciting verbal-

report data of written mode (i.e., diary-keeping and questionnairing) are

higher in the degree than those involving oral self-report data (i.e.,
thinking-aloud and interviewing) (cf. Matsumoto, 1992b). So are in
general the retrospective self-report techniques than the concurrent ones.

45
 Especially, it is the diary-keeping technique that past research found to
be of animmediate aid to the informants L2 learning. Such studies as
Bailey (1983, 1991), Lowe (1987), Matsumoto (1989), and Rubin and
Henze (1981) have shown that journal-keeping facilitates the L2 learning
process by serving as a tool of self-reflection, self-analysis, and self-
evaluation, by helping the informants become alert to their own learning
styles and strategies, thus, as a result, by raising the learners
consciousness and awareness of language learning. It is also possible
that the informants benefit from other introspective techniques. For
example, they will be the beneficiary of the structured questionnaire
asking them to select their preferred ways of learning, particularly, in
this case, by helping them become aware of alternative learning strategies
and thus expand their repertoire, and ultimately to be more flexible in
their approaches to learning (Nunan, 1991). Given that introspection
undoubtedly places the L2 performer rather than L2 performance who is
actively engaged in cognitive processing in the very central position of
research (Lennon, 1989, p.392), this criterion of the benefit to the
informant-learner must be underscored. It is also desirable that other
verbal-report techniques be revised so as to serve somehow as an
immediate aid to the informants L2 learning.

4.10 Combination of Methods

Data triangulation (i.e., collecting learners mentalistic data from
multiple sources) and methodological triangulation (i.e., combining
introspective verbal-report methods with extrospective techniques) are
strongly encouraged, if we are to obtain more accurate, valid data on
learners cognitive processes as well as compensate for the problems
inherent in each method (Garner, 1988; Jamieson & Chapelle, 1987).
The use of multimethod assessment in the above-presented literature
may be categorized into four groups: (a) combination of concurrent and
retrospective self-report data (e.g., Abraham & Vann, 1987; Cohen &
Cavalcanti, 1990; Feldman & Stemmer, 1987; Haastrup, 1987; Lay, 1982),
(b) combination of retrospective verbal-report data from different sources
(e.g., Gillette, 1987; Huang, 1985; Matsumoto, 1989; Robertson, 1984;
Wenden, 1987), (c) combination of verbal-report data with performance
data (e.g., Block, 1986; Cavalcanti, 1987; Ellis, 1989; Knight et al.,
1985; Ostler, 1980; Politzer & McGroarty, 1985; Raimes, 1985, 1987),
and (d) combination of verbal-report data with researcher observation
(e.g., Benson, 1989; Brown, 1983, 1985; Omalley et al., 1985; Zamel,
1983).

46
5.Verbal-Report Data in L2 Research: Issues of Controversy and
Suggestions for Future Research
The use of introspection as a research tool has been one of the
controversial issues among L2 researchers since what Seliger (1983)
calls the &dquo;psychoanalytic school&dquo; of SLA studies started around in the
mid-1970s. Cohen and Hosenfeld (1981) are among the first researchers
who argued for the usefulness of self-observation of mental processes by
L2 learners, emphasizing that the learners themselves have important
insights and intuitions into the internal processing involved in language
use and learning. The value of collecting mentalistic data through verbal

reporting has also been advanced out of the realization of the limitations
of conventional extrospective observations (i.e., a researcher observes
learners verbal and nonverbal behaviours as they perform a task and
infers from their overt behaviors their cognitive processes). For one
thing, the scope of such classroom observational data, focusing on the
physical dimension of classroom activity, is usually confined to L2
learners who actively participate in verbal classroom interaction, without
providing access to quiet classroom learners (Cohen, 1984; Gaies, 1983).
The observational studies have actually proved unsuccessful in providing
satisfactory data of learners mental processes including strategic
processing. For example, OMalley et al. (1985, pp. 36-37) report that in
their attempt to identify learning strategies student interviews were more
successful than teacher interviews, whereas researchers observations
were &dquo;exceedingly nonproductive and proved highly unreliable.&dquo;

Several concerns, on the other hand, have also been raised both by
cognitive psychologists and L2 researchers over introspective verbal

reports on a number of accounts. Some criticisms are targeted at self-

reports in general; others pertain only to either concurrent verbal-reporting
or retrospection. The criticisms to be discussed below are concerned
with two important issues: the validity and reliability of verbal reports.
Suggestions for future research to diminish those concerns will be
provided wherever appropriate.

Perhaps the most basic concern is the veridicality of verbal-report

data with actual mental processes. As Nisbett and Wilson (1977) argued,
one can doubt peoples capacity for observing directly the workings of
their own minds. Similarly, Seliger (1983), reacting to Cohen and
Hosenfelds (1981) endorsement for introspection, contended firmly that
it is not possible for the language learner to become a linguist and
provide reliable information about the mental processes involved in L2

47
use or L2 acquisition/learning. His main criticism is directed towards
whether we can infer from the conscious verbal reports the internal
processing. That is, he claims that &dquo;introspections are conscious
verbalizations of what we think we know (Seliger, 1983, p.183); therefore,
&dquo;the conscious verbal reports of learners about their own internal device
cannot be taken as a direct representation of internal processing&dquo; (Seliger,
1983, p. 189). Given Ericsson and Simons (1984) taxonomy discussed
earlier, it seems that Seliger is referring here to retrospection, and not
thinking aloud. We must recall, in this regard, that their model assumes
that although retrospectively-verbalized information does not directly
reflect the heeded information because of the intermediate processes
involved, concurrently-verbalized information does bear a direct relation
to the activated information kept in STM. Thus think-aloud verbal reports
can actually be seen as a direct representation of learners internal

processing.
Garner (1988), summarizing criticisms made of introspection by
cognitive psychologists, discusses several factors which will influence
the veridicality of verbal reports. One important factor concerned
specifically with retrospective, not concurrent, reporting is the relationship
between thinking/doing and reporting, which has two dimensions: (a)
temporality, and (b) directness/specificity. In cases where the verbal
reporting involves LTM, that is, reports are provided not immediately
(i.e., some time or long after task completion), informants tend to forget
some information which was originally available while being engaged in
the task. Unwanted extensive interference by recently-stored cognitive
events, in this case, will result in incomplete, less accurate, especially
quantitatively skimpy reporting. As evidence for the processing-reporting
interval effects, Garner (1988) reports that in his experiment involving
retrospection of strategic activity immediately after task completion and
two days later, delayed report informants protocols contained significantly
less cognitive events than those given by same-day report informants. In
the case of eliciting retrospective reports on situations not based on a
specific task (i.e., reports of hypothetical or general information) (Ericsson
& Simon, 1984, p.37), on the other hand, no direct or specific relation
can be established between processing and reporting. The resulting reports

will similarly be vulnerable to inappropriate inferences and therefore

invalidity.
Another potential factor which will affect the veridicality of verbal
reports is the automation of processes. Ericsson and Simon (1980, p.
225) argue that as particular processes become automated, &dquo;intermediate

48
steps are carried out without being interpreted, and without their inputs
and outputs using STM.&dquo; Such automation thus greatly speeds up the
process, preventing, as a consequence, the intermediate products from
being stored in STM, hence from being verbally reported. The result is
incomplete data which are quantitatively poor reflections of cognitive
processing. The informants distraction during task performance will also
be a source of incomplete reporting. It is possible that not only
environmental factors intruding the visual or auditory system (e.g., loud
noises) but also the informants emotional states will interfere with his/
her control of attention during processing, causing incomplete, invalid
data to be obtained. The informants verbal facility should also be
considered as an additional potential source of invalidity of verbal reports.
Garner (1988) cautions us, in this regard, to treat protocols collected
from informants with limited verbal skills such as young children as
incomplete records of processing which should alternatively be tapped
by some nonverbal assessment. We also have to be aware that even
individuals with normally developed language skills will differ in their
tendency to produce verbal reports.
In addition to these cases in which informants know more than
they can tell, there also exist cases where informants-learners tell more
than they can know (Nisbett & Wilson, 1977). The result is faulty
reporting. For example, informants may report what they perceive they
ought to know or do as an ideal learner, instead of what they in fact
know or do. It would also be possible that the subjects, being conscious
of the researchers judgments of their verbal reports, report those things
socially desirable or not detrimental to the teacher, institution, classmates,
and so on (Matsumoto, 1987). Another possibility in this regard is that
the cueing unintentionally offered by instructions or by the researcher-
interviewer can provide a hint of the most desirable response. In these
cases, the resulting reports only poorly reflect the informants actual
cognitive processing.

Further, a major concern raised specifically about think-aloud data

is that the task of generating concurrent verbal reports itself may change
the nature of actual cognitive processes (OMalley & Chamot, 1990). As
Ericsson and Simon (1984) note, thinking-aloud differs from normal
silent thinking in that the rate of thinking has to be slowed down in order
to allow for the additional time required for verbalization of the thought
(cf. Note 2). The possible alternation and disruption of the processes
have also been claimed to result especially in cases where the verbalization
task involves frequent interruptions, or puts informants under a heavy

49
 cognitive load (Ericsson and Simon, 1980). The outcome, again, is less
complete verbalization. To solve this problem, White (1980) advocates
an easy, not complicated, reporting task with reduced verbalization
demands (for example, reading studies involving frequent pause during
verbalization may be problematic in this regard).

Although, as has been shown above, researchers have considered

the factors influencing the veridicality or validity of verbal reports to
data, it seems that the reliability of self-reports has not generally been
examined (Garner, 1988). It may be, as in the case of behavioral research,
verbal reports from an informant exhibit instability over time, or it may
be that the researchers interpretation of an informants protocols shows
inconsistency. It is suggested, in this regard, that we should consider, for
example, collecting data not from one-time but from multiple
administration of the verbalization task, or conducting data-analysis by
multiple researchers to avoid possible researcher bias.
In sum, the above-presented discussion suggests that while verbal
reports may be an incomplete representation of informants underlying
cognitive processes under certain conditions, they still contain useful
information about internal processing given that what remains to be
reported will not invalidate what has been reported, and moreover, we
can in fact minimize many of these concerns. To be added to the ways
to enhance the veridicality of verbal reports discussed above (e.g.,
reduction of processing-reporting time intervals; use of task-based
information; use of a less complicated verbalization task) is the collection
of data from multiple sources. For instance, to compensate for the
purportedly existing invalidity, using multiple introspective techniques,
combining introspection with observation (observable nonverbal behaviors
like eye movements may provide important clues to underlying cognition)
as well as performance data (informants proficiency test scores may

provide a useful hint for their use of certain strategies) may be advocated
(cf. Section 4.10).

Lieberman (1979), in arguing for a wider use of introspection from

a cognitive-psychological perspective, argues as follows in order to
minimize the problems of introspection concerning its scope and accuracy:

Whenever possible, we need to supplement verbal reports with

other circumstantial or behavioral evidence, and the ultimate
criterion for evaluating any form of introspective data must
be their usefulness in predicting future behavior. Assuming

50
 that this criterion can be satisfied in particular instances,
however, it would seem foolish to continue to ignore such
data. The overriding concern... then, has been with how best
to advance our ability to predict and control behavior, and

greater acceptance of introspection and the mind has been

advocated in the belief that it can make a significant, albeit
limited, contribution to this goal (p. 332).

To conclude this section, let me quote Ericsson and Simon (1980):

For more than half a century, and as the result of

unjustified
an

extrapolation ofjustified challenge

a particular mode of
to a
verbal reporting (introspection), the verbal reports of human
subjects have been thought suspect as a source of evidence
about cognitive processes..... verbal reports, elicited with care
and interpreted with full understanding of the circumstances
under which they were obtained, are a valuable and thoroughly
reliable source of information about cognitive processes. It is
time to abandon the careless charge of introspection&dquo; as a
means for disparaging such data. They describe human
behavior that is as readily interpreted as any other human
behavior. To omit them when we are carrying the &dquo;chain and
transit of objective measurement&dquo; is only to mark as terra
incognita large areas on the map of human cognition that we
know perfectly well how to survey (p. 247).

6. Conclusion

Most representatively, with the current increasing attention being

paid by L2 researchers to conscious strategies learners employ in the
learning process, verbal reporting has again come to the fore as a vital
methodology for obtaining invisible insights from learners. Although
enough care seems to be needed to obtain valid mentalities data, it is
expected that introspective verbal reports from L2 learners-as-L2-
performers actively being engaged in L2 learning will continue to provide
us with useful information concerning the inner workings of the mind

unobtainable from extrospective observational studies alone.

Acknowledgements
I would like to thank Russell Campbell, Bob Jacobs, and John
Schumann for their helpful comments on earlier versions of this paper.

51
Notes

l. I will use the terms

introspective and verbal-report data/methods
interchangeably throughout this paper. Following the current general
practice in the literature (e.g., Faerch & Kasper 1987), I will also use the
term introspection as a cover term for all types of verbal reporting to
investigate mental processes. However, there has been a great amount of
variability among researchers in the classification of introspective data/
techniques as well as usage of related terminologies. For example, Radford
and Burton (1974) classify introspective techniques into three types and
define them as follows: (a) self-observation (the subject-researcher
analyzes his/her own reports on mental processes), (b) self-perception
or self-reports (the subject reports on his/her mental activities to the
researcher/analyst), and (c) thinking-aloud (the subject verbalizes while
performing a task). Their distinction between self-observation and self-
perception seems to be analogous to Matsumotos (1987) usage of the
terms introspective and non-introspective, respectively, with reference
to diary studies. Cohen (1984, 1987a, 1987b), on the other hand, classifies

types of verbal-report data into the following: (a) self-report (the learnerss
descriptions or statements of his/her general learning behaviors), (b)
self observation (the learners reports based on inspection of specific
language behavior, either while the information is still in STM
(introspection), or after the event (retrospection); retrospection can be
immediate or clelayecl , and (c) self-revelation (learner reports belonging
to neither (a) nor (b), i.e., thinking-aloud). Cohens introspection and

retrospection, it is noted, are subsumed under retrospective verbalization

in Ericsson and Simon (1984), the usage of which I have adopted here
in this paper. For a historical overview of introspection in psychology,
see Anderson (1990, pp. 5-10), Cavalcanti (1982, pp. 74-75), Ericsson

and Simon (1984, pp.48-61), Lieberman (1979, pp. 319-325), or Radford

and Burton (1974, pp. 389-395).

2. In cognitive psychology a strict distinction is sometimes made

between two types of concurrent verbalizations, talk-aloud and think-
aloud. In talk-aloud reporting &dquo;a subject utters thoughts that are already
encoded in verbal form,&dquo; whereas in think-aloud reporting &dquo;the subject
recodes verbally and utters thoughts that may have been held in memory
in some other form (e.g., visually)&dquo; (Ericsson & Simon, 1984, p. 222).
In other words, in talking aloud the subject simply vocalizes normal
silent thinking, while in thinking aloud the subject must convert the

52
heeded thought into a verbalizable form in order to vocalize it. Ericsson
and Simons (1984) model assumes that the same sequence of states of
heeded thoughts in a cognitive process is maintained in three different
conditions, silent thinking, talk-aloud, and think-aloud reporting, but
predicts that, in the case of thinking aloud, the rate of thinking would be
slowed down because of the additional time required for verbal encoding
of the heeded information.

3. In this paper the term retrospective verbalization is used in a

broad sense to refer to three types of reporting: (a) reporting of what
informants have thought and done while performing a particular task
which has already been completed, (b) reporting of what informants
would do and think in a hypothetical situation, and (c) reporting of
informants perceptions, views, beliefs, and attitudes based on, and
abstracted from, their past experience and knowledge in general, not
directly related to a specific task as in the case of (a). The last category
roughly corresponds to Cohens (1984, 1987) self-reporting.
4. Garner (1988), referring to verbal-report data on cognitive and
metacognitive strategies in LI, discusses three alternative methods for
eliciting retrospective verbal reports: (a) stimulated-recall techniques using
videotapes of informants task performance as a retrieval cue (cf. Poulisse
et al., 1987); (b) peer tutoring to examine strategic repertoires; and (c)
optimal-nonoptimal production activities (e.g., writing &dquo;good&dquo; and &dquo;bad&dquo;
summaries for a passage).

5. Wenden (1991), following Fravell (1979), refers to learners

beliefs, insights, and concepts that they have acquired about language
learning and the learning process as &dquo;metacognitive knowledge&dquo;, which
she distinguishes from learning strategies. Learner strategies and learners
knowledge about language learning, however, are so closely related that
a clear borderline cannot usually be drawn between them. In most learning

strategy research, therefore, learners perceptions, beliefs, views, and

attitudes have been inferred from their reported use of strategies.

References

Abraham, R. G., & Vann, R.J. (1987). Strategies of two language learners:
A case study. In A.L. Wenden & J. Rubin (Eds.) Learner strategies
in language learning (pp. 85-102). Englewood Cliffs, NJ: Prentice
Hall.

53
Allwright, D., & Bailey, K.M. (1991). Focus on the language classroom:
An introduction to classroom research for language teachers.
Cambridge: Cambridge University Press.
Anderson, J.R. (1990). Cognitive psychology and its implications (3rd
Edition). New York: W. H. Freeman and Company.
Bailey, K. M. (1980). An introspective analysis of an individuals language
learning experience. In R. C. Scarcella & S. D. Krashen (Eds.),
Research in second language acquisition: Selected papers of the
Los Angeles Second Language Research Forum (pp. 58-65).
Rowley, MA: Newbury House.
Bailey, K. M. (1983). Competitiveness and anxiety in adult second
language learning: Looking at and through the diary studies. In H.
W. Seliger & M. H. Long (Eds.), Classroom oriented research in
second language acquisition (pp. 67-102). Rowley, MA: Newbury
House.
Bailey, K. M. (1990). The use of diary studies in teacher education
programs. In J. C. Richards & D. Nunan (Eds.), Second language
teacher education (pp. 215-226). Cambridge: Cambridge University
Press.
Bailey, K. M. (1991, April). Diary studies of classroom language learning:
The doubting game and the believing game. Paper presented at the
1991 SEAMEO RELC Conference, Regional Language Center,
Singapore.
Bailey, K. M., & Ochsner, R. (1983). A methodological review of the
diary studies: Windmill tilting or social science? In K. M. Bailey,
M. H. Long, & S. Peck (Eds.), Second language acquisition studies
(pp. 188-198). Rowley, MA: Newbury House.
Banerjee, J. & Carrell, P. L. (1988). Tuck in your shirt, you squid:
Suggestions in ESL. Language Learning, 38, 313-336.
Benson, M. J. (1989). The academic listening task: A case study. TESOL
Quarterly, 23, 421-445.
Bernard, H. R. (1988). Research methods in cultural anthropology.
Beverly Hills, CA: Sage Publications.
Block, E. (1986). The comprehension strategies of second language
readers. TESOL Quarterly, 20, 463-494.
Brown, C. (1983). The distinguishing characteristics of the older adult
language learner. Unpublished doctoral dissertation. University of
California, Los Angeles.
Brown,C. (1985a) Two windows on the classroom world: Diary studies
and participant observation differences. In P. Larsen, E.L. Judd, &
D.S. Messerschmitt (Eds.), On TESOL 84: A brave new world for
TESOL (pp.121-134). Washington, D.C.: TESOL.

54
Brown C. (1985b). Requests for specific language input: Differences
between older and younger adult language learners. In S.M. Gass,
& C.G. Madden (Eds.), Input in second language acquisition (pp.
272-281). Rowley, MA: Newbury House.
Casanave, C.P. (1988). Comprehension monitoring in ESL reading: A
neglected essential TESOL Quarterly, 22, 283-302.
Cavalcanti, M. (1982). Using the unorthodox, unmeasurable verbal
protocol technique: Qualitative data in foreign language reading
research. In S.S. Dingwall, S. Mann, & F. Katamba (Eds.), Methods
and problems in applied linguistic research (pp.72-85). Department
of Linguistics and Modern English Language, University of
Lancaster.
Cavalcanti, M.C. (1987). Investigating foreign language performance
through pause protocols. In C. Faerch & G. Kasper (Eds.),
Introspection in second language research (pp.230-250). Clevedon,
England: Multilingual Matters.
Christison, M.A., & Krahnke, K.J. (1986). Student perceptions of
academic language study. TESOL Quarterly, 20, 61-81.
Cohen, A. (1984). Studying second-language learning strategies: How
do we get the information? Applied Linguistics, 3, 221-236.
Cohen, A. (1987a). Studying learner strategies: How we get the
information? In A.L. Wenden and J. Rubin (Eds.), Learner strategies
in language learning (pp. 31-42). Englewood Cliffs, NJ: Prentice
Hall.
Cohen, A.D. (1987b). Using verbal reports in research on language
learning in C. Faerch & G. Kasper (Eds.). Introspection in second
language research (pp.82-95). Clevedon, England: Multilingual
Matters.
Cohen, A.D. (1990). Language learning: Insights for learners, teachers
and researchers. New York: Newbury House.
Cohen, A.D., & Aphek, E. (1981). Easifying second language learning.
Studies in Second Language Acquisition, 3, 221-236.
Cohen, A.D., & Cavalcanti, M.C. (1990). Feedback on compositions:
Teacher and student verbal reports. In B. Kroll (Ed.), Second
language writing: Research insights for the classroom (pp. 155-
177). Cambridge University Press.
Cohen, A.D., Glasman, H., Sosenbaum-Cohen, P.R., Ferrara, J., & Fine,
J. (1979). Reading English for specialized purposes: Discourse
analysis and the use of student informants. TESOL Quarterly,
13,551-564.
Cohen, A.D., & Hosenfeld, C. (1981). Some uses of mentalistic data in
second language research. Language Learning, 31, 285-313.

55
 Cohen, A.D., & Robbins, M. (1976). Toward assessing interlanguage
performance: The relationship between selected errors, learners
characteristics, and learners explanations. Language Learning,
26, 45-66.
Cumming, A. (1989). Writing expertise and second-language
proficiency. Language Learning, 39, 81-141.
Devine, J. (1988). A case study of two readers: models of reading and
reading performance. In P. Carrell, J. Devine, & D. Eskey (Eds.),
Interactive approaches to second language reading (pp. 127-
139). Cambridge University Press.
Ellis, R. (1989). Classroom learning styles and their effect on second
language acquisition: A study of two learners. System, 17, 249-
262.
Ericsson, K.A., & Simon, H.A. (1980). Verbal reports as data.
Psychological Review, 87, 215-251.
Ericsson, K.A., & Simon, H.A. (1984). Protocol analysis: Verbal reports
as data. Cambridge, MA: MIT Press.
Ericsson, K.A., & Simon, H.A. (1987). Verbal reports on thinking. In
C. F&aelig;rch & G. Kasper (Eds.), Introspection in second language
research (pp.24-53). Clevedon, England: Multilingual Matters.
F&aelig;rch, C., & Kasper, G. (1987). From product to process -
Introspective methods in second language research (pp.5-23).
Clevedon, England: Multilingual Matters.
Feldman, U.. & Stemmer, B. (1987). Thin_aloud a_retrospective da_in
C-te_ taking: diffe_languages_diff_learners_sa_approaches? In
C. F&aelig;rch & G. Kasper (Eds.), Introspection in Second Language
Research (pp. 251-266). Clevedon, England: Multilingual Matters.
Flavell, J. (1979). Metacognition and cognitive monitoring: a new
area of cognitive developmental inquiry. American Psychologist,

34, 906-911.
Fowler, F.J. (1988). Survey research methods. Newbury Park, CA:
Sage Publications.
Gaies, S.J. (1983). The investigation of language classroom processes.
TESOL Quarterly, 17, 205-217).
Garner, R. (1988). Verbal-report data on cognitive and metacognitive
strategies. In C.E. Weinstein, E.T. Goetz, & P.A. Alexander (Eds.),
Learning and study strategies: Issues in assessment, instruction,
and evaluation (pp. 63-76). New York: Academic Press.
Gerloff, P. (1987). Identifying the unit of analysis in translation: Some
uses of think-aloud protocol data. In C. F&aelig;rch & G. Kasper
(Eds.), Introspection in second language research (pp.135-158).
Clevedon, England: Multilingual Matters.

56
Gillette, B. (1987). Two successful language learners: An introspective
approach. In C. F&aelig;rch & G. Kasper (Eds.), Introspection in second
language research (pp. 268-279). Clevedon, England: Multilingual
Matters.
Grandcolas, B., & Soul&eacute;-Susbielles,
N. (1986). The analysis of the second
language classroom. Studies in Second Language Acquisition, 8,
293-308.
Grotjahn, R. (1987). On the methodological basis of introspective
methods. In C. F&aelig;rch & G. Kasper (Eds.), Introspection in second
language research (pp. 54-81). Clevedon, England: Multilingual
Matters.
Haastrup, K. (1976). Using thinking aloud and retrospection to uncover
learners lexical inferencing procedures. In C. F&aelig;rch & G. Kasper
(Eds.), Introspection in second language research (pp. 197-212).
Clevedon, England: Multilingual Matters.
Holec, H. (1987). The learner as manager: Managing learning or managing
to learn? In A. L. Wenden and J. Rubin (Eds.), Learner strategies
in language learning (pp. 145-157). Englewood Cliffs, NJ: Prentice
Hall.
Holscher, A., & Mohle, D. (1987). Cognitive plans in translation. In C.
F&aelig;rch & G. Kasper (Eds.), Introspection in second language
research (pp. 113-134). Clevedon, England: Multilingual Matters.
Hosenfeld, C. (1976). Learning about learning: Discovering our students
strategies. Foreign Language Annals, 9, 117-129.
Hosenfeld, C. (1977). A preliminary investigation of the reading strategies
of successful and nonsuccessful second language learners. System.
5, 110-123.
Hosenfeld, C. (1979). Cindy: A learner in todays foreign language
classroom. In W. Borne (Ed.), The foreign language learner in
todays classroom environment (pp.53-75). Northwest Conference
on the Teaching of Foreign Languages.

Hosenfeld, C. (1984). Case studies of ninth grade readers. In J.C. Alderson

& A.H. Urquhart (Eds.), Reading in a foreign language (pp. 231-
249). London: Longman.
Horwitz, E.K. (1987). Surveying student beliefs about language learning.
In A. Wenden & J. Rubin (Eds.), Learner strategies in language
learning (pp. 119-129). Englewood Cliffs, NJ: Prentice Hall.
Huang, X-H. (1985). Chinese EFL students learning strategies for oral
communication. TESOL Quarterly, 19, 167-168.
Jamieson, J., & Chapelle, C. (1987). Working styles on computers as
evidence of second language learning strategies. Language Learning,
37, 523-544.

57
Jones, R.A. (1977). Social and psychological factors in second language
acquisition: A study of an individual. In C.A. Henning (Ed.),
Proceedings of the Los Angeles Second Language Research Forum
(pp. 331-341). University of California, Los Angeles.
Jones, S., & J. Tetroe. (1987). Composing in a second language. In A.
Matsuhashi (Ed.), Writing in real time: Modelling production
processes (pp. 34-57). Norwood, NJ: Ablex.
Knight, S.L., Padron. Y.N., & Waxman, H.C. (1985). The cognitive
reading strategies of ESL students. TESOL Quarterly, 19789-792.
Kraemer, R., & Zisenwine, D. (1989). Changes in attitude toward learning
Hebrew in a South African setting. Language Learning, 39, 1-14.
Krings, H.P. (1987). The use of introspective data in translation. In C.
F&aelig;rch & G. Kasper (Eds.): Introspection in second language
research (pp.159-176). Clevedon, England: Multilingual Matters.
Lay, N.D.S. (1982). Composing processes of adult ESL learners: A case
study. TESOL Quarterly, 16, 406.
Lennon, P. (1989). Introspection and intentionality in advanced second-
language acquisition. Language Learning, 39, 375-396.
Lieberman, D.A. (1979). Behaviorism and the mind: A (limited) call for
a return to introspection. American Psychologist, 34, 319-333.
Lowe, T. (1987). An experiment in role reversal: Teachers as language
learners. ELT Journal, 41, 89-96.
Mann, S. (1982). Verbal reports as data: A focus on retrospection. In
S.S. Dingwall, S.Mann, & F. Katamba (Eds.), Methods and
problems in doing applied linguistic research (pp. 87-104).
Department of Linguistics and Modern English Language,
University of Lancaster.
Matsumoto, K. (1987). Diary studies of second language acquisition: A
critical overview. JALT Journal, 9, 17-34.
Matsumoto, K. (1989). An analysis of a Japanese ESL learners diary:
Factors involved in the L2 learning process. JALT Journal, 11,
167-192.
Matsumoto, K. (1992a). Research-paper writing strategies of professional
Japanese EFL writers. Manuscript submitted for publication.
Matsumoto, K. (1992b). Helping L2 learners reflect on classroom learning:
Student perceptions of retrospective self-reporting activities.
Manuscript submitted for publication.
Neu, J., & Scarcella, R. (1991). Word processing in the ESL writing
classroom: A Survey of student attitudes. In P. Dunkel (Ed.),
Computer-assisted language learning and testing: Research issues
and practice (pp. 169-187). New York: Newbury House.
Nisbett, R.E., & Wilson, T.D. (1977). Telling more than we can know:

58
 Verbal reports on mental processes. Psychological Review, 84, 231-
259.
Nunan, D. (1991). Language teaching methodology. New York: Prentice
Hall.
OMalley, J.M., & Chamot, A.U. (1990). Learning strategies in second
language acquisition. Cambridge: Cambridge University Press.
OMalley, J.M., Chamot, A.U., Stewner-Manzanares, G, Kupper, L., &
Russo, R.P. (1985). Learning strategies used by beginning and
intermediate ESL students. Language Learning, 35, 21-46.
Ostler, S. (1980). A survey of academic needs for advanced ESL. TESOL
Quarterly, 14, 4489-502.
Oxford, R.L. (1990). Language learning strategies: What every teacher
should know, Englewood Cliffs, NJ: Newbury House.
Pearson, E. (1988). Learner strategies and learner interviews. ELT Journal,
42, 173-178.
Phinney, M. (1991). Computer-assisted writing and writing apprehension
in ESL students. In P. Dunkel (Ed.), Computer-assisted Language
Learning and Testing: Research issues and practice (pp. 189-204).
New York: Newbury House.
Politzer, R.L., & McGroarty, M. (1985). An exploratory study of learning
behaviors and their relationship to gains in linguistic and com-
municative competence. TESOL Quarterly, 19, 103-123.
Poulisse, N., Bongaerts, T., & Kellerman, E. (1987). The use of
retrospective verbal reports in the analysis of compensatory
strategies. In C. F&aelig;rch & G. Kasper (Eds.), I
ntrospection in second
language research (pp. 213-229). Clevedon, England: Multilingual
Matters.
Radford, J., & Burton, A. (1974). Thinking: Its nature and development.
London: John Wiley & Sons.
Raimes, A. (1985). What unskilled ESL students do as they write: A
Classroom study of composing. TESOL Quarterly, 19, 229-258.
Raimes, A. (1989). Language proficiency, writing ability and composing
strategies: A study of ESL college student writers. Language
Learning, 37, 439-468.
Reid, J. (1987). The learning style preferences of ESL students. TESOL
Quarterly, 21, 87-111.
Robertson, D. (1984). English language use, needs, and proficiency among
foreign students at the University of Illinois at Urbana-Champaign.
TESOL Quarterly, 18, 144-145.
Rubin, J. (1981). The study of cognitive processes in second language
learning. Applied Linguistics, 2, 117-131.
Rubin, J., & Henze, R. (1981). The foreign language requirement: A

59
 suggestion to enhance its educational role in teacher training. TESOL
Newsletter, 15, 17, 19, 24.
Schallert, D.L., Alexander, P.A., & Goetz, E.T. (1988). Implicit instruction
of strategies for learning from text. In C.E. Weinstein, E.T. Goetz,
& P.A. Alexander (Eds.), Learning and study strategies: Issues in
assessment, instruction, and evaluation (pp. 193-214). New York:
Academic Press.
Schmidt, R.W., & Frota, S.N. (1986). Developing basic conversational
abilityin asecond language: A case study of an adult learner of
Portuguese. In R.R. Day (Ed.), Talking to learn: Conversation in
second language acquisition (pp.237-326). Rowley, MA: Newbury
House.
Schumann, F.M. (1980). Diary of a language learner: A further analysis.
In R.C. Scarcella & S.D. Krashen (Eds.), Research in second
language acquisition: Selected papers of the Los Angeles Second
Language Research Forum (pp. 51-57). Rowley, MA: Newbury
House.
Schumann, F.M., & Schumann, J.H. (1977). Diary of a language learner:
An introspective study of second language learning. In H. D, Brown,
C.A. Yorio, & R.H. Crymes (Eds.): On TESOL 77: Teaching
and learning English as a second language: Trends in research
and practice (pp. 241-249). Washington, D.C.: TESOL.
Seliger, W.S. (1983). The language learner as linguist: Of metaphors and
realities. Applied Linguistics, 4, 179-191.
Wenden, A.L. (1987). How to be a successful language learner: Insights
and prescriptions from L2 learners. In A.L. Wenden & J. Rubin
(Eds.), Learner strategies in language learning (pp. 103-117).
Englewood Cliffs, NJ: Prentice Hall.
Wenden, A.L. (1991). Learner strategies for learner autonomy. London:
Prentice Hall.
Wenden, A.L., & Rubin, J. (Eds.) (1987). Learner strategies in language
learning. Englewood Cliffs, NJ: Prentice Hall.
White, P. (1980). Limitations on verbal reports of internal events: A
refutation of Nisbett and Wilson and of Bem. Psychological Review,
87, 105-112.
Zamel, V. (1983). The composing processes of ESL students: Six case
studies. TESOL Quarterly, 17, 165-187.

60