IEEE TRANSACTIONS ON REHABILITATION ENGINEERING, VOL. 6, NO.

1, MARCH 1998

EEG-Based Communication and Control:

Short-Term Role of Feedback
Dennis J. McFarland, Lynn M. McCane, and Jonathan R. Wolpaw
Abstract When people learn to control the amplitudes of
certain electroencephalogram (EEG) components (e.g., the 812
Hz -rhythm over sensorimotor cortex) and use them to move a
cursor to a target on a video screen, feedback about performance
is normally provided by cursor movement and by trial outcome
(i.e., success or failure). We assessed the short-term effects of this
feedback on EEG control. After subjects received initial training
with feedback present, feedback was removed intermittently for
periods of several minutes. Subjects still displayed EEG control
when feedback was removed. Removal of cursor movement alone
appeared to have effects comparable to removal of both cursor
movement and trial outcome. These results show that, in the
short-term at least, -rhythm control is not dependent on the
sensory input provided by cursor movement. They also suggest
that feedback can have inhibitory as well as facilitory effects on
EEG control, and that these effects vary across subjects. This
finding has implications for the design of training procedures.
Index Terms Assistive communication, electroencephalography, feedback, learning, -rhythm, operant conditioning, prosthesis, rehabilitation, sensorimotor cortex.

I. INTRODUCTION

ANY people with severe motor disabilities require

alternative methods for communication and control.
Over the past decade, a number of studies have evaluated the
possibility that scalp-recorded electroencephalogram (EEG)
activity might be the basis for a new augmentative communication interface [11], [12], [14], [16], [20][23].
EEG-based communication systems measure specific features of EEG activity and use the results as a control signal.
In some systems, these features are potentials evoked by
stereotyped stimuli [3], [12], [20]. Other systems, such as our
own, use EEG components that are spontaneous in the sense
that they are not dependent on specific sensory events [6],
[11], [14], [16], [21][24].
With our current EEG-based communication system, subjects learn over a series of training sessions to use EEG to
move a cursor on a video screen. Each session consists of eight
3-min runs, and each run consists of about 30 trials. During
each trial, the subject is presented with a target somewhere on
the periphery of the screen and a cursor in the center. His or
her task is to move the cursor to the target. The cursor moves
as a linear function of EEG amplitude in specific frequency
bands at specific scalp locations until it either reaches the target
(i.e., a hit) or reaches some other place on the periphery (i.e.,
Manuscript received February 6, 1997; revised December 11, 1997. This
work was supported in part by the National Center for Medical Rehabilitation
Research of the National Institutes of Health under Grant HD30146.
The authors are with the Wadsworth Center, New York State Department
of Health and State University of New York, Albany, NY 12201 USA.
Publisher Item Identifier S 1063-6528(98)01986-7.

a miss). At present, cursor movement is usually controlled

by the amplitude of the -rhythm, which is 812 Hz activity
focused over sensorimotor cortex, or by the amplitude of
higher frequency (e.g., 2024 Hz) beta rhythm activity, also
focused over sensorimotor cortex.
During system operation, the subject receives two forms
of feedback concerning his or her performance. The first is
cursor movement (i.e., toward or away from the target), which
occurs during the trial. The second is the event at the end
of the trial: either flashing of the target to indicate a hit or
disappearance of the target to indicate a miss. Over the course
of subject training, one or both of these two forms of feedback
provide the information that allows the subject to learn to
control - or -rhythm amplitude and thereby control cursor
movement. However, it is not clear whether one or both forms
of feedback remain essential once training has occurred. Nor
is it clear whether they can have other effects that detract
from the subjects EEG control. For example, the display of
cursor movement, while certainly informative, could also be
distracting.
The present study examined the short-term effects of feedback in subjects who had completed an initial series of training
sessions. It assessed performance when one or both forms
of feedback were removed for periods of several minutes.
The results have implications for the further development of
EEG-based communication systems.
II. METHODS
Subjects were ten adults (two women and eight men, ages
3158). Three had spinal cord injuries 618 years prior to
participation (at levels C45, C67, and T4, respectively) and
were confined to wheelchairs. All gave informed consent for
the study, which had been reviewed and approved by the New
York State Department of Health Institutional Review Board,
and had no prior experience with EEG-based communication.
After an initial evaluation defined the frequencies and scalp
locations of -rhythm activity and related beta rhythm activity,
each subject participated in 12 training sessions at a rate of
13 sessions per week. As described below, the first ten were
standard training sessions in which the subject was given the
opportunity to acquire EEG control. The final two were special
sessions in which the role of feedback on this control was
evaluated. A comprehensive description of our EEG-based
communication system is available elsewhere [10].
A. Standard Session Format and System Operation
The subject sat in a reclining chair facing a 51-cm video
screen that was 3 m away, and was asked to remain motionless

10636528/98$10.00 1998 IEEE

IEEE TRANSACTIONS ON REHABILITATION ENGINEERING, VOL. 6, NO. 1, MARCH 1998

during performance. Scalp electrodes recorded 64 channels

of EEG [18], each referenced to an electrode on the right
ear (amplification 20 000; bandpass 160 Hz). A subset of
channels were digitized at 196 Hz and used to control cursor
movement online as described below. In addition, all 64
channels were digitized at 128 Hz and stored for later analysis.
In this study, the subjects were trained to control onedimensional (1-D) [i.e., vertical] cursor movement. This movement was controlled as follows. After digitization of the
right-ear referenced EEG signals, two EEG channels, one over
the sensorimotor cortex of each hemisphere, were derived
according to either a bipolar derivation (three subjects), a
common average reference derivation (three subjects), or a
Laplacian derivation (four subjects). The channels of the
bipolar derivation were FC3 versus CP3 and FC4 versus CP4
[18]. The channels of the common average reference (CAR)
derivation were C3 and C4 each versus a CAR composed of 19
electrodes distributed over the entire scalp (i.e., the electrodes
of the 1020 system [5]). This 19-channel CAR, used to
increase the speed of online analysis, provided EEG signals
equivalent to those from a full 64-channel CAR (i.e., the
correlation between the 19-electrode and 64-electrode common
[10]). The channels of
average reference results is
the Laplacian derivation were C3 versus the average of F3,
T7, P3, and CZ (which were all equidistant from C3 and at
right angles to each other) and C4 versus the average of F4,
CZ, T8, and P4. The use of these three different derivations
provided data for a simultaneous study assessing their relative
effectiveness in allowing subjects to acquire EEG control.
Cursor movement was a linear function of -rhythm amplitude. Every 100 ms, the most recent 200-ms segment from
each channel was analyzed by an autoregressive algorithm
[9], and the amplitude (i.e., square root of power) in a 3Hz wide frequency band was calculated. This frequency band
corresponded to the arch-shaped -rhythm (six subjects) or
the central beta rhythm (four subjects), both of which are
generated in sensorimotor cortex [4], [7], [8], [13]. The sum
of the amplitudes from the two channels was the independent
variable in a linear equation that defined a vertical cursor
movement. The movement was in units of cursor steps. A
positive value caused upward movement and a negative value
caused downward movement. Thus, every 100 ms, the cursor
moved the defined number of steps up or down the screen. The
intercept of the equation was set so that, if future performance
was similar to previous performance, net cursor movement
over all trials would be zero [10]. As a result, the intercept
reduced bias in one direction or another and maximized the
influence that the subjects EEG control had on the direction
(i.e., upward or downward) of cursor movement. The slope
was set on the basis of previous performance so that the
average duration of cursor movement would be about two
seconds.
In each session, one dimensional cursor control was practiced in eight runs of three minutes each, separated by one
minute breaks. A run consisted of a series of trials. Each trial
began with a one second period during which the screen was
blank. Next, as shown at the top of Fig. 1, a target consisting
of a horizontal bar appeared at the top or bottom edge of the

(a)

(b)

(c)
Fig. 1. The three trial formats: standard (STD), no cursor movement (NCM),
and no feedback (NFB). The sequence of events during an STD trial is 1) a
target appears on the screen, 2) the cursor appears and begins to move, 3)
the cursor moves toward or away from the target, depending on the subjects
EEG, 4) if the cursor hits the target, the target flashes, but if the cursor misses
the target the target disappears and the cursor remains on the screen, 5) there
is a brief interval between trials, and 6) another trial begins as in 1). Thus, in
STD trials, feedback is present as both cursor movement and end result (i.e.,
a target flash for a hit or the cursor frozen on the screen for a miss). In NCM
runs, cursor movement feedback is absent. In NFB runs, there is no feedback
at all, neither the cursor movement nor the end result is shown.

screen. One second later, the cursor appeared in the center of

the screen and began to move vertically ten times per second
according to the linear equation described above. The distance
from the top to the bottom of the screen was 188 cursor steps
(i.e., the cursors initial position was 94 steps from the top and
94 steps from the bottom). The subjects task was to move the
cursor to the target. The trial ended when the cursor touched
the top or bottom edge. When it touched the correct edge,
the target flashed for one sec as a reward and the computer
registered a hit. When it touched the other edge, the target
disappeared, the cursor remained fixed on the screen for one
second, and the computer registered a miss. In either case,
the next trial then began with one second of blank screen.
Equal numbers of top and bottom targets appeared in an order
randomized in blocks of eight, and a miss did not cause the
target to be repeated. Thus, accuracy expected in the absence
of any EEG control was 50%.
B. Evaluation of the Effects of Feedback
In sessions 11 and 12, the first two runs were standard
(STD). That is, they were identical to those of the preceding
10 training sessions as described above. The next 6 runs were
a random mix of three types of runs. Two were STD runs. In
two NCM (no cursor movement) runs, the cursor did not
appear on the screen (i.e., Fig. 1, middle). The subject saw

MCFARLAND et al.: EEG-BASED COMMUNICATION AND CONTROL

TABLE I
EACH SUBJECTS ELECTRODE DERIVATION, CENTER FREQUENCY OF EEG CONTROL SIGNAL, ACCURACIES FOR SESSIONS 9 AND 10 AND FOR STANDARD (STD) AND
NO-CURSOR-MOVEMENT (NCM) RUNS OF SESSIONS 11 AND 12, AND r 2 VALUES FOR STD, NCM, AND NO-FEEDBACK (NFB) RUNS OF SESSIONS 11 AND 12

only the target and the terminal event signaling a hit or miss.
In two NFB (no feedback) runs, neither the cursor nor the
terminal event appeared on the screen (i.e., Fig. 1, bottom).
The subject saw only the target, which simply remained on
the screen for 5 s and then disappeared. No time limit was
imposed in the other conditions.
We compared all STD and NCM runs in accuracy (i.e.,
percent of targets hit) and the STD, NCM, and NFB runs in
value, the coefficient of determination [25], which indicates
the proportion of the total variance of the EEG amplitudes for
top and bottom targets that is accounted for by target position
(i.e., by the difference between the mean amplitudes for top
and bottom targets).
III. RESULTS
A. Initial Acquisition of EEG Control
As Table I shows, the ten subjects varied widely in the level
of EEG control they acquired over the first 10 sessions. Eight
of the 10 achieved some measure of control: their accuracy
for sessions 9 and 10 was significantly (
,
in all cases) greater than 50%. These results are
consistent with our previous experience, which indicates that
high accuracy (i.e., 90%) usually takes several months to
develop and that subjects vary greatly in their learning rates.
Table I also indicates that, for this relatively small number of
subjects, neither the EEG channel derivation used (i.e., bipolar,
CAR, or Laplacian) nor the presence of spinal cord injury
clearly affected success.

did not end in success or failure, they simply lasted 5 s). An

analysis of variance with condition as a within-subjects factor
revealed that feedback did not have a significant effect on the
overall level of performance (
).
Thus, in the short-term at least, the removal of feedback did
not appear to degrade overall performance.
Analysis of for each subject and each condition, shown in
Table I, confirmed the results presented above and in Fig. 2(a).
The
data are summarized in Fig. 2(b). For
values, as for
accuracies, analysis of variance with condition (STD, NCM, or
NFB) as a within-subjects factor revealed no significant effect
of feedback (
).
Although removal of feedback did not have a significant
effect on the performance of the population, Table I suggests
that some subjects were affected. For example, accuracy for
subject C was 71.5% in the STD condition and only 48.5%
in the NCM condition. In contrast, accuracy for subject G
was 54.3% in the STD condition and 72.3% in the NCM
condition. These disparate effects are reflected in the fact that,
while accuracy for the STD runs of sessions 11 and 12 was
significantly correlated with accuracy for sessions 9 and 10
(
,
), accuracy for the NCM runs of sessions
11 and 12 did not show a similar correlation (
,
). Thus, performance for sessions 9 and 10, which
were composed only of STD runs, predicted performance on
the STD runs, but not the NCM runs, of sessions 11 and 12.
IV. DISCUSSION
A. The Effects of Feedback

B. Short-Term Effects of Feedback on EEG Control

Table I also displays for sessions 11 and 12 of each subject
the accuracies for the STD and NCM runs and the
values
for the STD, NCM, and NFB runs. Fig. 2(a) shows the mean
( SE) accuracies for the STD runs and for the NCM runs.
(NFB data are not included in this analysis since NFB trials

There is little question that feedback about performance

plays an essential role in skill development. Without feedback,
the subject could not determine which internal states (e.g.,
which imagery) moved the cursor in one direction and which
moved it in the other. In the present case, feedback in the form
of cursor movement and end result provides the information

10

IEEE TRANSACTIONS ON REHABILITATION ENGINEERING, VOL. 6, NO. 1, MARCH 1998

which feedback was present. This suggests that, depending on

the individual subject, feedback of cursor movement can have
beneficial or deleterious short-term effects on performance.
B. Possible Origins of Beneficial and
Deleterious Short-Term Effects

(a)

(b)

Fig. 2. (a) Mean ( SE) accuracies (in %) for the STD and NCM runs of
sessions 11 and 12, and (b) R2 values ( SE) for the STD, NCM, and NFB
runs of sessions 11 and 12.

that allows the subject to acquire control over the EEG signal,
and thus over cursor movement.
At the same time, however, feedback can have other effects,
good or bad, on performance. The multiple possible effects
of feedback have been addressed in a variety of studies [1],
[17]. Some effects are lasting, which is to say that they affect
learning. Others are transient, in that they only affect performance. It is not clear whether feedback is needed or desirable
in all phases of performance. Salmoni et al. [17] suggests that
excessive guidance can sometimes actually degrade learning.
It does seem clear that feedback should be as rapid as possible.
Delayed feedback degrades performance. Smith and Smith
[19] suggest that ideal humanmachine systems should provide
instantaneous feedback. This theoretical ideal is not possible
for EEG control, since frequency analysis requires some finite
sample of EEG data (e.g., 200 ms) and an additional 20 ms
is needed for computation and screen display. Our experience
to date suggests that this delay is not a major impediment to
performance. It is comparable to the delay inherent in most
humanmachine interfaces [2]. We have evaluated the effects
of signal length on alternative signal processing algorithms
with this issue in mind [10].
In the present study, short-term removal of feedback did not
have a significant effect on the overall performance of the subject population. Nevertheless, some subjects were affected by
removal of feedback, since performance was no longer significantly correlated with performance during previous sessions in

The feedback provided by cursor movement might furnish

continual motivation as well as guidance [17]. It might serve to
maintain the subjects interest and thereby ensure attention to
the task. It might also improve performance by allowing rapid
reaction to cursor movements that are in the wrong direction.
On the other hand, some subjects might be distracted by the
cursor so that they pay insufficient attention to essential aspects
of the task. Their attention to the moving cursor might prevent
concentration on internal states such as relaxation or motor
imagery; close attention to cursor movement might impede
maintenance of the relaxed state that facilitates -rhythm
activity.
Alternatively or in addition, the feedback of cursor movement might produce EEG responses that interfere with the
EEG control essential for accurate cursor movement. For
example, cursor movement in the wrong direction can elicit
frustration, a response likely to be associated with generalized
EEG desynchronization. Conversely, cursor movement in the
correct direction might lead the subject to anticipate hitting
the target, and this anticipation might be associated with
EEG synchronization or desynchronization that is not targetspecific. Furthermore, cursor movement, as a visual stimulus,
might affect the visual alpha rhythm, which is similar in
frequency to the -rhythm and is sometimes detectable at the
central scalp locations where the -rhythm is recorded. Such
a response would introduce noise into the EEG signal used to
control cursor movement.
Finally, it is possible that the removal of cursor movement
could have a nonspecific facilitory affect on subjects who had
not been performing well, an effect that could be produced by
any substantial change in the stimuli occurring during performance. It might be that these subjects develop a conditioned
response (e.g., frustration) that inhibits performance, and that
the change in the stimulus configuration represented by the
removal of cursor movement disrupts this response and thereby
improves performance.
C. Practical Implications
Whatever the mechanisms of the beneficial and/or deleterious effects of cursor-movement feedback, the results suggest
some modifications that might facilitate performance. Specific
changes could be targeted to specific populations of subjects
(e.g., to those who do worse with cursor movement feedback),
or, ideally, changes might be found that increase the beneficial
effects of feedback and decrease the deleterious effects for all
subjects.
Most simply, the potentially distracting aspect of cursor
movement might be reduced by decreasing the brightness
and/or size of the cursor. Such change would probably not
impair the guidance provided by the movement, but could
reduce the distraction. Alternatively or in addition, cursor

MCFARLAND et al.: EEG-BASED COMMUNICATION AND CONTROL

movement feedback might be shown on only some fraction

of the trials. This change would be analogous to motor control
tasks in which feedback is provided on only a proportion of
trials [17].
Feedback in some form is clearly necessary for learning EEG control. However, it is possible that the feedback
provided by the end result (i.e., target hit or miss) will
prove sufficient if it is the only feedback provided throughout
training. Thus, it is possible that cursor movement feedback
might be completely eliminated. On the other hand, it is also
possible that more refined cursor movement, combined perhaps
with a decrease in brightness, size, or frequency, will prove
helpful, particularly early in training.
Finally, the finding that subjects could perform well without
seeing cursor movement indicates that, for the short-term at
least, the prominent visual stimulation of the moving cursor
was not essential for the maintenance of performance. This
further differentiates our EEG-based control system from those
that depend on EEG responses evoked by stereotyped sensory
stimuli [3], [12], [20].

REFERENCES
[1] A. H. Black, The operant conditioning of the electrical activity of the
brain as a method for controlling neural and mental processes, in The
Psychophysiology of Thinking, F. J. McGuigan and R. A. Schoonover,
Eds. New York: Academic, 1973, pp. 3568.
[2] D. R. Bradley and S. B. Abelson, Desktop flight simulators: Simulation
fidelity and pilot performance, Behavioral Res. Methods, Instrum.
Comput., vol. 27, pp. 152159, 1995.
[3] L. A. Farwell and E. Donchin, Talking off the top of your head:
Toward a mental prosthesis utilizing event-related brain potentials,
Electroencephal. Clin. Neurophysiol., vol. 70, pp. 510523, 1988.
[4] H. Gastaut, Etude electrocorticographique de la reactivite des rythmes
rolandiques, Revenue Neurol., vol. 87, pp. 176182, 1952.
[5] H. H. Jasper, The tentwenty electrode system of the International Federation, Electroencephal. Clinical Neurophysiol., vol. 10, pp. 371375,
1958.
[6] J. Kalcher, D. Flotzinger, C. Neuper, S. Golly, and G. Pfurtscheller,
Graz braincomputer interface IIToward communication between
humans and computers based on online classification of three different
EEG patterns, Med. Biol. Eng. Comput., vol. 34, pp. 382388, 1996.
[7] J. W. Kozelka and T. A. Pedley, Beta and mu rhythms, J. Clin.
Neurophysiol., vol. 7, pp. 191207, 1990.
[8] W. N. Kuhlman, EEG feedback training: Enhancement of somatosensory cortical activity, Electroencephal. Clin. Neurophysiol., vol. 45,
pp. 290294, 1978.
[9] S. L. Marple, Digital Spectral Analysis With Applications. Englewood
Cliffs, NJ: Prentice-Hall, 1987.
[10] D. J. McFarland, T. Lefkowicz, and J. Wolpaw, Design and operation
of an EEG-based braincomputer interface (BCI) with digital signal
processing technology, Behavioral Res. Methods, Instrum. Comput.,
vol. 29, pp. 337345, 1997.
[11] D. J. McFarland, G. W. Neat, R. F. Read, and J. R. Wolpaw, An
EEG-based method for graded cursor control, Psychobiol., vol. 21, pp.
7781, 1993.
[12] G. R. McMillan, G. L. Calhoun, M. S. Middendorf, V. H. Schnurer,
D. F. Ingle, and V. T. Nasman Direct brain interface utilizing selfregulation of steady-state visual evoked response (SSVER), in Proc.
RESNA95 Annu. Conf., 1995, vol. 15, pp. 693695.
[13] G. Pfurtscheller and A. Berghold, Patterns of cortical activation during
planning of voluntary movement, Electroencephal. Clin. Neurophysiol.,
vol. 72, pp. 250258, 1989.

11

[14] G. Pfurtscheller, D. Flotzinger, and J. Kalcher, Brain computer interface

a new communication device for handicapped persons, J. Microcomput.
Applications, vol. 16, pp. 293299, 1993.
[15] G. Pfurtscheller, A. Stancak, and G. Edinger, On the existence of
different types of central beta rhythms below 30 Hz, Electroencephal.
Clin. Neurophysiol., vol. 102, pp. 316325, 1997.
[16] B. Rockstroh, T. Elbert, A. Canavan, W. Lutzenberger, and N. Birbaumer, Slow Cortical Potentials and Behavior, 2nd ed. Baltimore,
MD: Urban & Schwarzenberg, 1989.
[17] A. W. Salmoni, R. A. Schmidt, and C. B. Walter, Knowledge of results
and motor learning: A review and critical reappraisal, Psychologic.
Bull., vol. 95, pp. 355386, 1984.
[18] F. Sharbrough, C. E. Chatrian, R. P. Lesser, H. Luders, M. Nuwer, and
T. W. Picton, American Electroencephalographic Society quidelines
for standard electrode position nomenclature, J. Clin. Neurophysiol.,
vol. 8, pp. 200202, 1991.
[19] T. J. Smith and K. U. Smith, Feedback-control mechanisms of human
behavior, in Handbook of Human Factors, G. Salvendy, Ed. New
York: Wiley, 1987.
[20] E. E. Sutter, The brain response interface: Communication through visually guided electrical brain responses, J. Microcomput. Applications,
vol. 15, pp. 3145, 1992.
[21] J. R. Wolpaw, D. J. McFarland, and A. T. Cacace, Preliminary studies
for a direct brain-to-computer parallel interface, in Projects for Persons
with Disabilities, IBM Tech. Symp., 1986, pp. 1120.
[22] J. R. Wolpaw, D. J. McFarland, G. W. Neat, and C. A. Forneris, An
EEG-based braincomputer interface for cursor control, Electroencephalography and Clinical Neurophysiology, vol. 78, pp. 252259,
1991.
[23] J. R. Wolpaw and D. J. McFarland, Multichannel EEG-based
braincomputer communication, Electroencephal. Clin. Neurophysiol.,
vol. 90, pp. 444449, 1994.
[24]
, Development of an EEG based braincomputer interface, in
Proc. RESNA95 Annu. Conf., 1995, vol. 15, pp. 645649.
[25] T. H. Wonnacott and R. Wonnacott, Introductory Statistics. New York:
Wiley, 1977.

Dennis J. McFarland received the B.S. and Ph.D.degrees from the University
of Kentucky, Lexington, in 1971 and 1978, respectively.
He is currently a Research Scientist at the Wadsworth Center for Laboratories and Research at the New York State Department of Health, Albany, NY.
His major research interests are in development of EEG-based communication
and in auditory psychophysics.

Lynn M. McCane is currently working towards the Master of Science degree

in biology education as a graduate student at the University of Albany, Albany,
NY.
From 1993 to 1996, she had been a Research Technician for the BrainComputer Interface Project.

Jonathan R. Wolpaw received the A.B. degree from Amherest College,

Amherst, MA, in 1966 and the M.D. degree from Case Western Reserve
University, Cleveland, OH, in 1970.
He completed a residency in neurology at the University of Vermont,
Burlington, and a fellowship training in neurophysiological research at the
National Institute of Health. He is currently a Research Physician and a
Professor at the Wadsworth Center for Laboratories and Research of the
New York State Department of Health and the State University of New
York, Albany. His major research interest is in developing and using operant
conditioning of spinal reflexes as a new model for defining the plasticity
underlying a simple form of learning in vertebrates. More recently, he has
also become involved in developing an EEG-based brain-computer interface
as a new communication channel for those with severe motor disabilities.