Anda di halaman 1dari 11

This article has been accepted for publication in a future issue of this journal, but has not been

fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TBME.2017.2732479, IEEE
Transactions on Biomedical Engineering
IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING 1

An EOG-Based Human-Machine Interface for


Wheelchair Control
Qiyun Huang, Shenghong He, Qihong Wang, Zhenghui Gu, Nengneng Peng, Kai Li, Yuandong Zhang, Ming
Shao, and Yuanqing Li*, Fellow, IEEE

Abstract— Objective: Non-manual human-machine interfaces or muscular dystrophy among other conditions. With modern
(HMIs) have been studied for wheelchair control with the aim of life-support technology, people severely paralyzed can lead
helping severely paralyzed individuals regain some mobility. The lives that are enjoyable and productive if they can be provided
challenge is to rapidly, accurately and sufficiently produce control
commands, such as left and right turns, forward and backward with novel non-manual means of communication and control
motions, acceleration, deceleration, and stopping. In this paper, [1].
a novel electrooculogram (EOG)-based HMI is proposed for A major task of designing a non-manual HMI for wheelchair
wheelchair control. Methods: Thirteen flashing buttons, each control is to rapidly, accurately, and sufficiently produce suf-
of which corresponds to a command, are presented in the ficient commands, including left and right turns, forward and
graphical user interface (GUI). These buttons flash on a one-
by-one manner in a pre-defined sequence. The user can select backward motions, acceleration, deceleration, and stopping.
a button by blinking in sync with its flashes. The algorithm Since many severely paralyzed patients maintain normal func-
detects the eye blinks from a channel of vertical EOG data and tions of their head and eyes, electroencephalogram (EEG) and
determines the user’s target button based on the synchronization electrooculogram (EOG), resulting from brain activities and
between the detected blinks and the button’s flashes. Results: eye movements, respectively, can be used in HMIs and have
For healthy subjects/patients with spinal cord injuries (SCIs), the
proposed HMI achieved an average accuracy of 96.7%/91.7% and attracted a great deal of attention over the past few decades.
a response time of 3.53 s/3.67 s with 0 false positive rates (FPRs). Both EEG and EOG bear merits because they are noninvasive,
Conclusion: Using one channel of vertical EOG signals associated technically less demanding, and widely available at relatively
with eye blinks, the proposed HMI can accurately provide suffi- low cost [2].
cient commands with a satisfactory response time. Significance: EEG-based HMI is a type of brain computer interface (BCI),
The proposed HMI provides a novel non-manual approach for
severely paralyzed individuals to control a wheelchair. Compared which conveys the user’s intent by brain signals (such as
with a newly established EOG-based HMI, the proposed HMI EEG) which do not depend on neuromuscular activity for
can generate more commands with higher accuracy, lower FPR their generation [3]. Common EEG patterns used in BCIs
and fewer electrodes. include event-related potentials (ERPs, e.g. P300 potential) [4],
Index Terms—human-machine interface (HMI), electrooculo- [5], mu/beta rhythm related to motor imagery (MI) [6]–[8],
gram (EOG), eye blink, wheelchair control. and steady-state visual evoked potentials (SSVEPs) [9], [10].
Several BCIs, aimed at controlling the movement of a robotic
device, have been reported [11]–[17]. He et al. achieved real-
I. I NTRODUCTION
time control of a robotic quadcopter in a 3-D space based on
A wheelchair can help paralyzed individuals regain mobility, MI [17]. The subjects turned the quadcopter left or right by
and human-machine interfaces (HMIs) can be used to translate imagining clenching left or right hand respectively. Imaging
human intentions into wheelchair control commands. Most both hands together caused the quadcopter to rise, while
conventional HMIs for wheelchair control, such as joysticks intentionally imaging nothing caused it to fall. In [13], Long
and keyboards, are manual. However, these manual interfaces and Li proposed a hybrid BCI incorporating MI and P300
become useless for severely paralyzed persons suffering from in order to enrich control commands, in which MI was used
amyotrophic lateral sclerosis (ALS), spinal cord injury (SCI), to implement left and right turns and deceleration, while the
acceleration was triggered by the user’s attention on a flashing
This work was supported by the National Natural Science Foundation button on screen which elicited P300 potential. Furthermore,
of China under Grant 61633010 and Grant 91420302, in part by the Na-
tional Key Basic Research Program of China (973 Program) under Grant Li designed a multimodal brain switch by combining P300 and
2015CB351703, and in part by Guangdong Natural Science Foundation under SSVEP for go/stop control of the wheelchair [14]. Combining
Grant 2014A030312005. the works in [13] and [14], Li et al. presented a hybrid
*Y. Li (e-mail: auyqli@scut.edu.cn), Q. Huang, S. He, Z. Gu, N. Peng,
and K. Li are with the School of Automation Science and Engineering, South wheelchair control system based on three widely used EEG
China University of Technology, Guangzhou, 510640, China, and also with patterns: MI, P300 and SSVEP. To our knowledge, BCIs for
Guangzhou Key Laboratory of Brain Computer Interface and Applications, robotic device control typically implement the MI paradigm to
Guangzhou, 510640, China.
Q. Wang, Y. Zhang, and M. Shao are with the Affiliated Sichuan Provincial steer the device due to its fast-response performance. However,
Rehabilitation Hospital of Chongqing Medical University, Chengdu, 610036, the disadvantage of the MI paradigm is that the number of
China. commands is limited by the number of imaginative motions
Copyright (c) 2016 IEEE. Personal use of this material is permitted.
However, permission to use this material for any other purposes must be that can be classified (typically 2 or 3). Incorporating MI with
obtained from the IEEE by sending an email to pubs-permissions@ieee.org. other EEG patterns helps increase the number of commands,

0018-9294 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TBME.2017.2732479, IEEE
Transactions on Biomedical Engineering
IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING 2

but it also leads to other problems, such as a relatively longer (Asynchronous Experiments). The results demonstrated that
response time and a higher user workload. Additionally, for the proposed HMI based on only one type of eye movement
MI-based BCIs, a long training period is generally required (blink) and one EOG channel could rapidly, accurately and
for most users. sufficiently produce commands for wheelchair control.
EOG has also been used to design non-manual HMIs. The remainder of this paper is organized as follows: Section
Typical eye movements include gaze, blink, wink, and frown II presents the methodologies, including data acquisition, the
[18]–[21]. All of these movements can generate prominent GUI mechanism, and the algorithm used; in Section III, the
EOG with high signal-to-noise ratios (SNRs). Thus, studies results of two experiments are presented; further discussion is
have focused on realizing controls or communications through included in Section IV; and Section V concludes the paper.
EOG-based HMIs [18]–[23]. However, to the best of our
knowledge, only a few of these studies [21], [22] have II. M ETHODS
addressed the development of motion control of a robotic
device in real time, such as a robot [21] and a wheelchair A. Data Acquisition System
[22]. Barea et al. proposed an EOG-based HMI for wheelchair A customized EOG acquisition device was designed to
control by detecting the user’s gaze direction (up, down, record and preprocess the EOG signals. The major modules
right and left) [22]. The four gaze directions were directly included a pre-amplifier, a driven right leg (DRL) circuit, a
interpreted into four control commands (forward/accelerate, bandpass filter (1-30 Hz), and a post-amplifier. The overall
backward/decelerate, left turn, and right turn). In [21], Ma gain of the amplifiers was approximately 2,000, and the
and Cichocki et al. proposed a hybrid HMI for robot control sampling rate was 250 Hz. The DRL was used to reduce the
that operated in two separate modes: the EEG mode and the common-mode interference. Fig. 1 illustrates the placements
EOG mode. The EEG mode was used to select a pre-defined of three wet electrodes (“CH1”, “COM”, and “COMLEG”),
complex movement, such as sitting down or dancing. In the each of which corresponds to one channel of the device.
EOG mode, a multi-threshold algorithm was used to classify “CH1” was placed above the left eyebrow and used to record
seven types of eye movements (double blinks, triple blinks, vertical EOG signals. The reference electrode, “COM”, was
left wink, right wink, left gaze, right gaze and frown) as seven attached to the left mastoid. “COMLEG” was located on the
commands (stop, forward motion, turn 90◦ left, turn 90◦ right, right mastoid and was used as the input of the DRL. The
head left, head right, and mode switch, respectively). All single impedances of all the electrodes were maintained below 5 kΩ.
blinks were regarded as unintended blinks and were ignored.
The subjects were asked to use the proposed HMI to control
mobile robots to complete a series of pre-defined movements.
The system achieved an average EOG accuracy (number of
correctly detected commands/number of detected commands)
of 79.5%. Two main challenges still exist for EOG-based
HMIs: (i) it is challenging to rapidly and accurately discrim-
inate intended (voluntary) and unintended (involuntary) eye
movements, and (ii) the accuracy of discriminating different
types of eye movements, such as blinks and winks, needs to
be further improved.
To overcome the limitations of existing systems, this paper
proposes a novel EOG-based HMI for wheelchair control.
Fig. 1. Electrode placement. CH1: EOG electrode; COM: reference; COM-
In our system, a graphical user interface (GUI) with several LEG: electrode to record the input signals of the DRL circuit.
flashing buttons is presented to the user. The buttons are
highlighted one by one in a pre-defined sequence. Each button
corresponds to a specific command, such as a left or right
turn; a total of 13 commands are provided. The user can B. Graphical User Interface and Control Mechanism
issue a command by selecting the corresponding button, and The GUI used in this study is illustrated in Fig. 2. It consists
this is accomplished through the user’s synchronized intended of two levels: a switch GUI (Fig. 2 (a)) and a control panel
blinks with the flashes. The algorithm detects the blinks from a (Fig. 2 (b)). Initially, the switch GUI is presented to the user,
channel of vertical EOG signals and distinguishes the potential which includes a single button (“On”) flashing at 1 Hz. The
target button from non-target ones. The final decision is made user can select this button by blinking synchronously with
by a decision making process. its flashes to switch on the control panel (see the details in
Experiments for both healthy subjects and patients with the Detection Algorithm section). Once the control panel is
SCIs were conducted. First, all of the subjects participated in switched on, the GUI in Fig. 2 (b) is presented, in which
a Synchronous Experiment in which they tested the proposed 13 buttons are included. The buttons flash one by one in a
HMI without a wheelchair. Indicators, such as the accuracy, pre-defined sequence. The interval between the onsets of two
the response time, the false positive rate, and the information sequential flashes is 90 ms. Thus, the period of a flashing
transfer rate were calculated. Next, the subjects drove a round (a complete cycle in which each button flashes once)
wheelchair using the proposed HMI to complete different tasks is 1.17 s (13 × 90 ms). Among the 13 buttons on the control

0018-9294 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TBME.2017.2732479, IEEE
Transactions on Biomedical Engineering
IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING 3

C. Algorithm
In this study, the algorithm can be divided into two parts:
(i) the blink detection and (ii) the decision-making process.
The blink detection is performed after each button flash. First,
a sub-segment (100-500 ms after the onset of the flash) is de-
rived from the recorded EOG signals, and several features are
extracted, such as the peak value of the sub-segment and the
duration of the blink. Then, the algorithm determines whether
these features satisfy threshold conditions for amplitude, speed
and duration. If all of the threshold conditions are satisfied,
(a) a blink is detected for the corresponding button. At the end
of each flashing round, the algorithm selects a button that
contains a blink in the sub-segment as the potential target of
this round according to the delay between the onset of the flash
and the peak of the blink waveform. If no blink is detected, the
potential target of this round is empty. A command is issued
only if the corresponding button is selected as the potential
target at least twice in the latest three rounds, i.e., if the
detected blinks are synchronized with the button’s flashes.
Then, the GUI is paused for a pre-defined time (e.g., 2 s)
to allow the associated wheelchair movement to be executed,
and a new flashing round starts when the pause time is over.
(b) If there is no repeated selection in the latest three rounds, the
GUI continues to flash. This algorithm is used to select most
Fig. 2. The GUI consists of two levels: a switch GUI (a) and a control buttons, and some differences exist for the “On”, “Off”, and
panel (b). In the switch GUI, the “On” button is used to switch on the “Stop” buttons. Further details are described in the following
control panel. In the control panel, thirteen buttons are displayed, including
two speed buttons (“Forward1”, “Forward2”), 8 direction buttons (“L1”-“L4”, paragraphs.
“R1”-“R4”), a “Stop” button, a “Backward” button, and an “Off” button to 1) Blink detection: Blink detection is performed after each
switch off the control panel. button flash. The recorded EOG is first filtered by a digital
bandpass filter (1-10 Hz), and then a sub-segment (100-500
ms after the onset of a flash) is derived from the filtered
EOG signal for each button flash. Several features, including
the peak value of the filtered signal amax , the time delay of
panel, the “Forward1” and “Forward2” buttons imply forward the peak tp , the maximal dfferentiated value smax , and the
motions with low speed of 0.1 m/s and high speed of 0.3 m/s, duration dpn (as shown in Fig. 3), are extracted from each sub-
respectively. When the wheelchair is static, the user selects segment. tp is the time delay between the onset of the flash
the “Forward1” button to move forward at the low speed. and the peak of the waveform. The duration dpn is defined as
The “Forward2” button is valid only when the wheelchair is the interval from the peak to the valley of the differentiated
moving at the low speed, and it corresponds to accelerating to shape. The differential of the original signal is approximated
the high speed. If the wheelchair is moving at the high speed by the first-order difference:
and the user selects the ”Forward1” button, then the wheelchair ′

decelerates to the low speed. For the backward motion, the F (n) = F (n) − F (n − 1), (1)
speed is set at 0.1 m/s for safety. Direction control is achieved where F (n) is the value of the nth sampling point in a sub-

with eight direction buttons (“L1”-“L4”, “R1”-“R4”). The four segment, and F (n) is the corresponding differential value.
buttons on the left from “L1” to “L4” correspond to turning Similar to the multi-threshold algorithm proposed in [21],
left at 5◦ , 30◦ , 45◦ , and 90◦ , respectively. Similarly, “R1” we set thresholds for amplitude, duration and speed (differenti-
to “R4” correspond to turning right at 5◦ , 30◦ , 45◦ , and 90◦ , ated value) to indicate whether there is a blink waveform in the
respectively. The “Off” button is used to switch off the control sub-segment of a button’s flash. Thus, a successful detection
panel and reset the GUI to the switch GUI. of a blink should satisfy the following inequalities:

In our system, the control state indicates that the user is amax ≥ Ath
currently performing a control task by blinking according to smax ≥ Sth (2)
a button’s flashes, whereas the idle state indicates that the D1 ≤ dpn ≤ D2
user does not wish to perform any control task. If the user where Ath and Sth are the amplitude and speed thresholds, re-
wishes to stay in the idle state for a long time (in this case, spectively, and D1 and D2 represent the minimal and maximal
the wheelchair is static), he/she needs to switch off the control thresholds of the time duration. The values of all thresholds
panel to reduce the incidence of false commands issued in the are determined in a calibration process (see the Calibration
idle state. section).

0018-9294 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TBME.2017.2732479, IEEE
Transactions on Biomedical Engineering
IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING 4

4000 and the GUI starts a new flashing round. If there is no repeated
blink waveform
potential target in the latest three rounds, the GUI continues
a max
to flash.
When the switch GUI is presented, there is only one button
Ath
3500 (“On”) flashing at 1 Hz. Thus, the decision-making process is
Amplitude (mV)

simplified: The “On” button is selected if a blink is detected in


its EOG sub-segment at least twice in the latest three flashes.
Moreover, in this study, the user needs to select the “On”/“Off”
3000
button twice within 10 s to switch the control panel on or off
to avoid false startup or shutdown of the control panel.
With the exception of the synchronous decision strategy,
a fast decision strategy can be achieved by directly using
tp the potential target of the latest round as the final decision.
2500
100 200 300 400 500
As a result, the time consumed to generate a command is
Time (ms)
reduced, but the accuracy might be affected, and an unintended
(a) blink is more likely to result in a false command. Further
150
comparison between the synchronous decision strategy and the
fast decision strategy is discussed in the Discussion section.
100
In this study, we apply only the fast decision strategy on the
stop command. Specifically, if the “Stop” button is selected as
s max
S th the potential target, then a stop command is issued directly to
Amplitude (V/s)

50
stop the wheelchair.
0
D. Calibration
−50 differentiated shape The thresholds Ath , Sth , D1 , D2 and the average delay
Tp vary among individuals. Thus, a calibration process is
−100 performed for each user before he/she starts to control the
wheelchair. Specifically, after the EOG acquisition device
d pn
−150 t1 t2 starts to record signals, a single button (“Test”) is presented
100 200 300 400 500 on the center of the screen, flashing 10 times at 1 Hz.
Time (ms) The user is asked to blink once according to each flash.
(b)
Similar to the blink detection process, the recorded EOG is
Fig. 3. Typical waveforms of (a) an eye blink and (b) its differentiated shape. filtered, and a sub-segment is derived for each flash. Waveform
amax : the peak value of the filtered signal; dpn : the duration of a blink; tp : features, including amax , smax , tp , and dpn , are extracted
the delay between the onset of the flash and the peak of the waveform; Ath : from these 10 sub-segments. If the user misses one flash,
the amplitude threshold; smax : the maximal differentiated value; Sth : the
speed threshold. The x-axis indicates the timing after the onset of a flash. the corresponding sub-segment is removed, and the features
are averaged across the rest of the sub-segments. The average
values of amax , smax , and tp are selected as the amplitude
2) Decision making: When the control panel of the GUI is threshold, Ath , the speed threshold, Sth , and the average
switched on, at the end of each flashing round, the algorithm delay, Tp , respectively. The duration thresholds D1 and D2
selects one button from those containing a blink in its EOG are calculated by multiplying the average value of dpn by
sub-segment as the potential target of this round by evaluating empirical factors (e.g., D1 = 0.8 × dpn and D2 = 1.2 × dpn ).
the delay tp . Specifically, the evaluation process is described
by the following equation: III. E XPERIMENTS AND R ESULTS
e = |tp − Tp | (3) In this study, experiments under different scenarios were
conducted to verify the effectiveness of the proposed EOG-
where e is the evaluation value and Tp represents the average based HMI. First, four healthy male subjects with normal eye
delay, which varies among individuals and is determined in the movements participated in three experiments to evaluate the
calibration process. The button with the minimal evaluation overall performance of the proposed HMI. Next, four patients
value is selected as the potential target of this round. To with spinal cord injuries were involved in two experiments to
improve the accuracy of the system, a command is issued show the potential application of the proposed HMI in daily
if the corresponding button is selected at least twice in the life. The wheelchair used was a UL8W [PIHSIANG MA-
latest three rounds, which indicates that the detected blinks are CHINERY CO. LTD.], 0.67m × 0.6m in size; a regular laptop
synchronized with the target button’s flashes (for simplicity, was placed in the front of the wheelchair to present the GUI.
this strategy is referred to as the “synchronous decision strat- Two ultrasonic ranging modules (HC-SR04) were installed at
egy” in the rest of this paper). Once the associated wheelchair the front and back of the wheelchair. The wheelchair would
movement is executed, all previous selections are discarded be stopped immediately if one of the feedback distances was

0018-9294 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TBME.2017.2732479, IEEE
Transactions on Biomedical Engineering
IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING 5

TABLE I
RESULTS FROM HEALTHY SUBJECTS BASED ON THE PROPOSED HMI

Subjects Accuracy (%) RT (s) FPR (events/min) ITR (bits/min) No. of Commands No. of Electrodes
S1 95 3.83 0 / 13 3
S2 93.3 3.56 0 / 13 3
S3 100 3.34 0 / 13 3
S4 98.3 3.42 0 / 13 3
Mean 96.7 3.53 0 57.3 13 3

TABLE II
RESULTS FROM HEALTHY SUBJECTS BASED ON AN ESTABLISHED HMI

Subjects Accuracy (%) RT (s) FPR (events/min) ITR (bits/min) No. of Commands No. of Electrodes
S1 85 3.13 4.3 / 7 5
S2 87.1 3.19 3.5 / 7 5
S3 83.6 3.13 4.1 / 7 5
S4 89 3.16 2.8 / 7 5
Mean 86.2 3.15 3.67 35.8 7 5

reduced below 0.5 m. The following performance indicators In the first session, the subjects completed 60 trials
were used to evaluate the proposed HMI: (3 blocks × 20 trials/block) using the proposed HMI without
1) Time, the time required to complete the experiment; the wheelchair. Specifically, at the beginning of a trial, a
2) Violations, the number of times that the wheelchair ran random target cue was presented on the control panel for 4
out of bounds; s. The 13 buttons on the control panel then started flashing
3) Response Time (RT), the time required to generate a one by one in a pre-defined sequence. The subject selected
command; the target by blinking according to its flashes. Once a button
4) Operations, the number of commands generated; was determined, the control panel stopped flashing, and the
5) Accuracy, the probability of correct selection; selected button was highlighted for 300 ms as the feedback
6) False Positive Rate (FPR), the false commands generated to the subject. Then, the system started a new trial. After
per minute during the idle time (when there were no the three blocks, a 10-min idle task was implemented in
intended blinks); which the subjects just relaxed and browsed the Web with the
7) Information Transfer Rate (ITR), bits of information proposed HMI running in the background. Any false output
transferred per minute. would be recorded to calculate the false positive rate (FPR).
Other indicators, such as the accuracy and the response time
A. For Healthy Subjects (RT), were calculated according to the experimental results. In
addition, the information transfer rate (ITR, bits/min), which
Four healthy male subjects (numbered S1-S4, aged between
has been widely used to evaluate HMIs [24], was calculated
24 and 26) with normal eye movements were involved in one
according to the following formula [4]:
synchronous experiment (with cues) and two asynchronous
experiments (indoor and outdoor). Each subject performed 1−P
IT R = 60(log2 N +P log2 P +(1−P ) log2 [ ])/T (4)
the three experiments on three different days. Among those N −1
subjects, S3 and S4 had prior experience of the proposed where N is the number of commands, P is the average
HMI, whereas S1 and S2 were using the system for the first accuracy and T is the average response time.
time. Before the experiments, each subject received a brief To compare the proposed HMI with state-of-the-art technol-
introduction to the system and completed an informed consent ogy, we reproduced an established EOG-based HMI proposed
form. Next, each subject completed the Calibration process by Ma et al. in [21]. In Ma’s system, seven types of eye
to determine the thresholds, as described in Section II-D: movements (double blinks, triple blinks, left wink, right wink,
Calibration. Further descriptions of the three experiments are left gaze, right gaze and frown) were identified based on the
given below. recorded EOG signals and corresponded to seven commands
1) Synchronous Experiment for Healthy Subjects: This to control a mobile robot. All single blinks were regarded as
experiment aimed to compare the proposed HMI with an unintended blinks and were abandoned. In the second session
established EOG-based HMI. It consisted of two sessions, of this experiment, the subjects performed three blocks, each
which were based on the proposed HMI and an established containing 20 trials, using Ma’s HMI. For one trial, a random
EOG-based HMI. Each session contained three blocks. In each target cue was presented on screen for 4 s. The subject then
block, the subjects were tasked with selecting 20 randomly performed the corresponding eye movement. As feedback to
generated target buttons (i.e. 20 trials). In addition, there was the subjects, the identification result was displayed on the
a 1-min break between each two consecutive blocks. screen for 1 s, and the system then started a new trial. Similar

0018-9294 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TBME.2017.2732479, IEEE
Transactions on Biomedical Engineering
IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING 6

TABLE III
RESULTS OF THE INDOOR EXPERIMENT

Subjects Time of Operations Selection errors Stop RT (s) A/D RT (s) Violations
Part 1 (s) in Part 1 in Part 1
S1 379 47 1 2.20 3.57 0
S2 420 50 2 2.03 3.66 0
S3 350 35 1 1.91 3.46 0
S4 344 41 1 1.55 3.54 0
Mean 373 43.3 1.3 1.92 3.56 0

to the first session, a 10-min idle task was implemented for hands slightly), and the supervisor recorded it as a selection
each subject, and the corresponding FPR was calculated. error. The response time in asynchronous experiments was
The experimental results of the two sessions are shown defined as the interval between the moment when the subjects
in Tables I and II. The proposed HMI used three electrodes heard a voice prompt, such as “Forward” and “Stop”, and
to generate thirteen commands. On average, it achieved an the moment that the corresponding command was issued.
accuracy of 96.7%, an RT of 3.53 s, and an ITR of 57.3 Moreover, if there were no intended blinks but a command
bits/min. No false output was recorded (FPR = 0). For Ma’s was issued, it was recorded as a false output.
HMI, five electrodes were needed to generate seven command- The results of the Indoor Experiment are shown in Table III.
s. The average accuracy, RT, and ITR of Ma’s HMI were All indicators were averaged across the two trials for each sub-
86.2%, 3.15 s, and 35.8 bits/min, respectively. Due to the ject. For all of the subjects, the A/D (acceleration/deceleration)
unintended eye movements, the average FPR of MI’s HMI response time was 3.56 s. By employing the fast decision
was 3.67 events/min. Note that the accuracy of Ma’s HMI strategy, the stop response time was 1.92 s. Note that the
(86.2%) in our experiment was higher than that presented response time of the direction control commands were not
in [21] (79.5%). One possible reason for the difference was recorded because the direction control should be completely
that in our experiment, the subjects manipulated the HMI based on the subjects’ own observations for safety. No vio-
without controlling any peripheral equipment, which might lations or emergency stops were recorded. The experimental
help them concentrate on the tasks. In summary, compared results demonstrated that the proposed EOG-based HMI could
with Ma’s HMI, the proposed HMI could produce more be used to effectively control the wheelchair.
control commands with higher accuracy, higher ITR, lower
FPR and fewer electrodes. The RT for the proposed HMI
was slightly longer than that for Ma’s HMI, which was an
acceptable trade-off.
2) Asynchronous Experiment I: The Indoor Experiment:
In the Indoor Experiment, we built a path (with width of 1.3
m) in a quiet and well-illuminated laboratory, as shown in
Fig. 4. The path could be divided into two parts. Part 1 was
designed as a simple maze inside a room, and Part 2 was
localized in a lab corridor. These parts were connected by a
door with a width of 1.1 m. The obstacles surrounding the
path were removed for safety. The subjects were asked to
use the proposed HMI to drive the wheelchair through the
path without crossing the boundaries. Each subject performed
two trials. In each trial, the subject started from a beginning Fig. 4. Top-down view of the pre-defined path in the Indoor Experiment. Part
point (B1 or B2), went through Parts 1 and 2 by following 1 is a simple maze inside a room, and Part 2 is a lab corridor. The parts are
connected by a door with width of 1.1 m. B1 and B2 are the starting points;
the arrows on the ground, and finally reached the end. In End is the end point.
Part 1, the subjects should decide each movement based on
their own observations and favorites without any guidance, 3) Asynchronous Experiment II: The Outdoor Experiment:
whereas in Part 2, the supervisor randomly ordered the subjects The Outdoor Experiment was designed to verify that the pro-
to perform movements, such as stop, moving forward and posed HMI could be used to effectively control a wheelchair
backward, acceleration and deceleration. The total Time and in an outdoor environment. We selected a rectangular space
Operations were recorded for Part 1 only, because in Part 2 (9 m × 5 m) in a campus parking slot as the experimental
they were randomly determined or affected by the supervisor. field, as shown in Fig. 5 (a). Five fixed destinations and various
Specifically, the Time in Part 1 was the interval from beginning obstacles were positioned in the field (Fig. 5 (b)). The subjects
at B1 or B2 to reaching the door. Once a command was issued, were tasked with driving the wheelchair to these destinations
the system added 1 to the Operations. If the issued command sequentially (i.e., starting at 1 and ending in 5) without breaks.
was not the target one, the subjects gave a sign (e.g., raising To compare the proposed HMI with a common HMI (the

0018-9294 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TBME.2017.2732479, IEEE
Transactions on Biomedical Engineering
IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING 7

(a) (b)

(c) (d)

Fig. 5. (a) Actual scene of the Outdoor Experiment in a campus parking slot. (b) General view of the experimental field (9 m × 5 m) with destinations (black
circles with numbers) and obstacles (red circles). (c) Trajectory that S4 maneuvered using a joystick. (d) Trajectory that S4 maneuvered using the proposed
HMI.

joystick), each subject performed two trials using either the B. For Patients with SCIs
proposed HMI or a joystick. The experienced subjects S3 and To verify the effectiveness of the proposed HMI for the
S4 were involved. During the experiment, the subjects were end-users, we further applied it to patients with SCIs. Four
supposed to control the wheelchair independently, while extra patients (numbered P1-P4; one female and three males; aged
measures were implemented to enable the supervisor to stop between 22 and 32; see Table VI) from Sichuan Provin-
the wheelchair in case of emergency. cial Rehabilitation Hospital were involved in a Synchronous
The results of the outdoor experiment are shown in Table Experiment and an Asynchronous Experiment. Each patient
IV. For both subjects, the total time spent using the proposed completed the two experiments in one day. All of the patients
HMI was approximately 1.22-fold longer than that using a were able to perform normal eye movements and had no
joystick. No violations or emergency stops were recorded prior experience in using EOG-based HMIs. More individual
during the experiments. We also drew the trajectories that information of the patients is presented in Table V. The ex-
S4 drove through in the experiment using a joystick and the periments were approved by the Ethics Committee of Sichuan
proposed HMI respectively, as shown in Figs. 5 (c) and 5 (d). Provincial Rehabilitation Hospital and complied with the Code
When using the joystick, the track was smoother compared of Ethics of the World Medical Association. Written informed
with using the proposed HMI because the proposed HMI only consents for the experiment and the publication of individual
provided discrete commands. information were obtained from the patients and their legal
guardians. Before the experiments, each patient received a
brief introduction to the system and completed the Calibration
TABLE IV
RESULTS OF THE OUTDOOR EXPERIMENT process as described in the Calibration section. More details
of the two experiments are described below.
Subjects Time when using the Time when using the Violations 1) Synchronous Experiment for Patients with SCIs: The
proposed HMI (s) joystick (s) Synchronous Experiment for patients was similar to the
S3 290 238 0 Synchronous Experiment for healthy subjects. Each patient
S4 263 215 0 completed three blocks (60 trials) using the proposed HMI
Mean 277 227 0 without the wheelchair. For one trial, a random target cue was
presented on the control panel for 4 s. The 13 buttons on
the control panel started flashing one by one in a pre-defined

0018-9294 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TBME.2017.2732479, IEEE
Transactions on Biomedical Engineering
IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING 8

TABLE V Physical Demand, Temporal Demand, Overall Performance,


INDIVIDUAL INFORMATION OF PATIENTS Effort, and Frustration Level [26]. The subscales are rated
within a 100-point range with 5-point steps. All patients
Patients Gender Age Pathogenesis Time of onset ASIA
(month) completed the questionnaire independently. As shown in Table
P1 Female 23 Myelitis 4 C4-C
VII, the average rating of the temporal demand was 40, and
P2 Male 25 Falling injury 48 C5-B the ratings for other subscales were below 25, which indicated
P3 Male 32 Falling injury 8 C7-B that the overall workload of using the proposed HMI to control
P4 Male 22 Myelitis 36 C5-C
a wheelchair was acceptable.
AISA: American Spinal Injury Association. Details
of AISA impairment scales can be found in [25]
IV. D ISCUSSION

TABLE VI In this paper, we propose a novel EOG-based HMI that


RESULTS OF THE SYNCHRONOUS EXPERIMENT FOR PATIENTS provides 13 control commands including left and right turns
with different degrees, forward and backward motions, accel-
Patients Accuracy (%) RT (s) FPR ITR eration, deceleration, and stopping. This HMI was realized by
(events/min) (bits/min)
setting 13 flashing buttons, each of which corresponds to a
P1 88.3 4.3 0 /
P2 90 3.2 0 /
control command, on the GUI. The users can select a target
P3 95 3.9 0 / button to issue a corresponding control command by blinking
P4 93.3 3.3 0 / synchronously with the flashes of the button.
Mean 91.7 3.67 0 48.8 In our system, the first challenge is to distinguish the target
and non-target buttons. Specifically, when the user performs
a blink according to a flash of the target button, the blink
waveform appears in the sub-segment of the EOG after the
sequence. The subject selected the target by blinking according onset of the flash with a time delay. The delay, which is mainly
to its flashes. The selected button was highlighted for 300 ms due to the human response time for reacting to the flash, is
as feedback to the subject. The system then started a new trial. defined as the interval between the onset of the flash and
The 10-min idle task was implemented after the three blocks the peak of the blink waveform. According to our paradigm
to record the FPR. design, the time difference between two adjacent onsets of
Indicators, such as the accuracy, RT, ITR, and FPR, were button flashes is 90 ms and the length of the EOG sub-segment
calculated and are shown in Table VI. On average, the pro- of each button’s flash is 400 ms (100-500 ms); therefore
posed HMI achieved an accuracy of 91.7% and an RT of a specific blink may be associated with several temporally
3.67 s. No false output was recorded, and the ITR was 48.8 adjacent buttons. For instance, as shown in Fig. 6 (a), the same
bits/min. blink appears not only in the target button’s sub-segment but
2) Asynchronous Experiment for Patients with SCIs: The also in those of several temporally adjacent buttons, which
Asynchronous Experiment for patients was designed to verify might result in having more than one button that satisfies the
whether the patients could effectively control a wheelchair waveform detection criterion. As shown in Fig. 6 (b), for a
using the proposed HMI. This experiment was similar to the user, the peaks of several blink waveforms are located within
Outdoor Experiment for healthy subjects except that there was a time interval (typically from 280 to 320 ms after the onset
no joystick as the control condition. The experimental field of the target flash in our experiment). One possible reason
was built in an open space in the hospital for the patients’ is that although the response time for a flash varies among
convenience, and the settings were exactly the same as those individuals, it remains relatively stable for a specific user.
for the healthy subjects. Each patient completed two trials, and According to the offline analysis, the length of the interval
in each of these trials, the patient drove the wheelchair through was within 40 ms for all of our subjects. If the blink waveform
the five destinations sequentially (starting at 1 and ending at is located far away from the interval, it will be assigned to a
5, as shown in Fig. 5 (b)) using the proposed HMI. non-target button. Thus, to distinguish the target button, we
According to the experimental results, all patients com- use the average delay Tp (e.g., 300 ms after the onset of the
pleted the experiment without any violations or collisions, target flash) as a criterion, calculate the delay for each button
demonstrating that patients with normal eye movements could flash, and then select the button with the corresponding delay
effectively control the wheelchair using the proposed HMI. closest to Tp as the potential target of the flashing round. For
The time cost to complete both the Synchronous Experi- the temporally adjacent non-target buttons’ flashes, the delays
ment and the Asynchronous Experiment was approximately were generally far from Tp due to the interval of 90 ms. For
40 minutes in total (including the Calibration process). To instance, for the green waveform in Fig. 6(a) corresponding to
measure the workload involved in using the proposed HMI a non-target button’s flash, the delay was approximately 200
over a long period of time, each patient completed a work- ms. While for the black waveform corresponding to the target
load questionnaire based on the Hart and Staveland’s NASA button’s flash, the delay was approximately 290 ms. Thus, we
Task Load Index (TLX) method after the two experiments can distinguish the target and non-target buttons.
were completed. The NASA TLX method measures the total The second challenge of the EOG-based HMIs is to rapid-
workload by dividing it into six subscales: Mental Demand, ly and accurately distinguish intended eye movements from

0018-9294 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TBME.2017.2732479, IEEE
Transactions on Biomedical Engineering
IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING 9

TABLE VII
WORKLOAD RATINGS OF PATIENTS

Patients Mental Physical Temporal Performance Effort Frustration


demand demand demand level
P1 20 10 40 20 30 0
P2 20 5 40 20 5 0
P3 20 5 30 20 10 0
P4 20 10 40 25 30 0
Mean 24 14 40 22 23 0
Note: a lower rating means a lower demand level or better performance

unintended ones. In the proposed HMI, the discrimination


between intended blinks and unintended blinks was based on
the synchrony between the detected blinks and the button’s
3800 flashes. Specifically, if a button was selected as the potential
90 ms target at least twice in the latest three rounds, the detected
blinks were thought to be in synchrony with the button’s
3500
flashes, and the button was determined as a target in our
algorithm. This synchronous decision strategy improves the
Amplitude (mV)

accuracy of the system, with a tradeoff of increasing the


3200
time required to generate a command. According to the
results of the Synchronous Experiment for healthy subjects,
the overall accuracy is 96.7%, and the average response times
2900
for all commands and for ”STOP” were 3.53 s and 1.92 s,
respectively, which are satisfactory.
Another challenge for EOG-based HMIs is to discriminate
different types of eye movements such as blink and wink based
2600
100 200 300 400 500 on EOG signals. In the proposed HMI, only one type of eye
Time (ms) movement (blink) was employed to ensure that the problem
(a) was avoided.
As previously mentioned, the stop command was issued
4000
based on the fast decision strategy to achieve a fast-response
performance. The discount is that an unintended blink might
result in a false stop with a theoretical probability of 7.7%
(i.e., based on a probability of 1/13 that the stop button is
3500
Amplitude (mV)

selected from the 13 buttons by an unintended blink). This


rate is considered acceptable for two reasons: (i) during the
experiments, the users seldom perform unintended blinks since
they need to voluntarily blink to generate commands; and (ii) a
3000 false stop does not lead to any risky situation. According to the
experimental results, the average Stop RT was 1.92 s, and no
false stops were recorded (FPR = 0). The results demonstrate
Tp
the effectiveness of our stop strategy.
2500
Safety is a major concern for wheelchair control and was
100 200 300 400 500
Time (ms) guaranteed from the following three folds in this study. (i)
(b) For the proposed system, the glide distance for stopping is
approximately 0.6 m (wheelchair velocity: 0.3 m/s; Stop RT:
Fig. 6. (a) Temporally adjacent buttons’ EOG sub-segments. Each of the approximately 2 s). In the experiments, because the positions
sub-segments includes the same blink waveform (the black waveform is for of all obstacles and boundaries were fixed, the users could
a target button, whereas the others are for temporally adjacent buttons). (b) blink to stop ahead of time in order to avoid any violation
Waveforms of six blinks according to six target button flashes. The peaks are
located within a time interval typically from 280 to 320 ms after the onset or emergency stop. (ii) According to the experimental results,
of the flash. Tp represents the average delay between the onset of the flashes the false positive rate was 0 for all users, which guaranties
and the peaks of the blink waveforms. The x-axis indicates the timing after that the system outputs commands only when the users are
the onset of a flash, and the y-axis indicates the amplitude of EOG.
intended to do so. (iii) In case of emergency, two ultrasonic
ranging modules (HC-SR04), installed at the front and back
of the wheelchair, will stop the wheelchair immediately if one
of the feedback distances is reduced below 0.5 m.

0018-9294 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TBME.2017.2732479, IEEE
Transactions on Biomedical Engineering
IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING 10

As for the paradigm design, the proposed HMI implemented R EFERENCES


a pre-defined flashing sequence instead of random sequences. [1] B. A. B. Z. Simmons, R. A. Robbins, S. M. Walsh, and S. Fischer,
For an oddball paradigm such as the P300-based BCI speller, “Quality of life in ALS depends on factors other than strength and
random sequences were necessary for generating the oddball physical function,” Neurology, vol. 55, no. 3, pp. 388-392, Aug. 2000.
[2] T. M. Vaughan, W. J. Heetderks, L. J. Trejo, W. Z. Rymer, M. Weinrich,
effects and for reducing interferences from adjacent buttons. M. M. Moore, A. Kübler, B. H. Dobkin, N. Birbaumer, E. Donchin,
However, in this study, we used EOG signals associated with E. W. Wolpaw, and J. R. Wolpaw, “Brain-computer interface technology:
eye blinks instead of P300 extracted from EEG. As long a review of the Second International Meeting,” IEEE Trans. Neural Syst.
Rehabil. Eng., vol. 11, no. 2, pp. 94-109, Jun. 2003.
as the time interval between two temporal adjacent button [3] J. R. Wolpaw, N. Birbaumer, D. J. McFarland, G. Pfurtscheller, and T. M.
flashes was sufficiently long and the subject was blinking Vaughan, “Brain-computer interfaces for communication and control,”
synchronously with the flashes of the target button, the EOG Electroenceph. Clin. Neurophysiol., vol. 113, no. 6, pp. 767-791, Jun.
2002.
signals associated with the blinks would be correctly assigned [4] H. Zhang, C. Guan, and C. Wang, “Asynchronous P300-based brain-
to the target button. Thus, a pre-defined sequence with a 90- computer interfaces: A computational approach with statistical models,”
ms interval between two temporal adjacent button flashes was IEEE Trans. Biomed. Eng., vol. 55, no. 6, pp. 1754-1763, Jun. 2008.
[5] J. Jin, B. Z. Allison, E. W. Sellers, C. Brunner, P. Horki, X. Wang,
used in our paradigm. According to the experimental results and C. Neuper, “Adaptive P300 based control system,” J. Neural Eng.,
(e.g., an average accuracy of 96.7% for healthy subjects), the vol. 8, no. 3, pp. 36006-36020, Apr. 2011.
90-ms interval was sufficient to determine the target button [6] G. Pfurtscheller, C. Brunner, A. Schlögl, and F. H. Lopes da Silva,
“Mu rhythm (de)synchronization and EEG single-trial classification of
almost without the interferences from the temporal adjacent different motor imagery tasks,” NeuroImage, vol. 31, no. 1, pp. 153-159,
buttons. Jan. 2006.
Furthermore, the proposed HMI provides discrete com- [7] J. Yue, Z. Zhou, J. Jiang, Y. Liu, and D. Hu, “Balancing a simulated
inverted pendulum through motor imagery: An EEG-based real-time
mands for direction control. Note that the actual turning angles control paradigm,” Neuroscience Letters, vol. 524, no. 2, pp. 95-100,
cannot be absolutely accurate because many factors could Jul. 2012.
affect the accuracy of direction control, such as the topography, [8] T. Zhang, T. Liu, F. Li, M. Li, D. Liu, R. Zhang, H. He, P. Li, J. Gong,
C. Luo, D. Yao, and P. Xu, “Structural and functional correlates of motor
friction, and the working state of the motor. However, because imagery BCI performance: Insights from the patterns of fronto-parietal
the proposed HMI generated control commands with high attention network,” NeuroImage, vol. 134, pp. 475-485, Jul. 2016.
accuracy and a relatively short RT, all subjects completed [9] M. Middendorf, G. McMillan, G. Calhoun, and K. S. Jones, “Brain-
computer interfaces based on the steady-state visual-evoked response,”
the experiments successfully, indicating that the accuracy of IEEE Trans. Rehab. Eng., vol. 8, no. 2, pp. 211-214, Jun. 2000.
direction control was sufficient for this application. To further [10] X. Cheng, Y. Wang, M. Nakanishi, X. Gao, T. P. Jung, and S. Gao,
demonstrate this accuracy, we compared the proposed HMI “High-speed spelling with a noninvasive brain-computer interface,” P.
Nati. Acad. Sci. USA, vol. 112, no. 44, pp. 6058-6067, Apr. 2015.
with a joystick. The time consumed using the proposed HMI [11] I. Iturrate, J. M. Antelis, A. Kubler, and J. Minguez, “A Noninvasive
was 1.22-fold longer than that using the joystick, which is Brain-Actuated Wheelchair Based on a P300 Neurophysiological Proto-
thought to be an acceptable result because the joystick is a col and Automated Navigation,” IEEE Trans. Robotics, vol. 25, no. 3,
pp. 614-627, Mar. 2009.
manual and mature HMI for controlling a wheelchair. [12] B. Rebsamen, C. Guan, H. Zhang, C. Wang, C. Teo, M. H. Ang,
and E. Burdet, “A brain controlled wheelchair to navigate in familiar
environments,” IEEE Trans. Neural Syst. Rehabil. Eng., vol. 18, no. 6,
pp. 590-598, Jun. 2010.
V. C ONCLUSION [13] J. Long, Y. Li, H. Wang, T. Yu, J. Pan, and F. Li, “A hybrid brain
computer interface to control the direction and speed of a simulated or
real wheelchair,” IEEE Trans. Neural Syst. Rehabil. Eng., vol. 20, no. 5,
A major challenge in the design of a non-manual HMI for pp. 720-729, Sep. 2012.
[14] Y. Li, J. Pan, F. Wang, and Z. Yu, “A hybrid BCI system combining
wheelchair control is to rapidly, accurately, and sufficiently P300 and SSVEP and its application to wheelchair control,” IEEE Trans.
produce control commands. In this paper, a novel HMI based Biomed. Eng., vol. 60, no. 11, pp. 3156-3166, Nov. 2013.
on only one type of eye movement (blink) and one EOG [15] K. Tanaka, K. Matsunaga, and H. Wang, “Electroencephalogram-based
control of an electric wheelchair,” IEEE Trans. Robotics, vol. 21, no. 4,
channel is presented for wheelchair control. The GUI includes pp. 762-766, Aug. 2005.
13 flashing buttons, which correspond to 13 control commands [16] T. Carlson and J. d. R. Millan, “Brain-controlled wheelchairs: A robotic
associated with direction control with different degrees, for- architecture,” IEEE Robot. Automat. Mag., vol. 20, no. 1, pp. 65-73,
Mar. 2013.
ward and backward motions, acceleration, deceleration, and [17] K. Lafleur, K. Cassady, A. Doud, K. Shades, E. Rogin, and B. He,
stopping. The user selects a target button by blinking in sync “Quadcopter control in three-dimensional space using a noninvasive
with its flashes. The algorithm discriminates the intended and motor imagery-based brain-computer interface,” J. Neural Eng., vol. 10,
no. 4, pp. 46003-46017, Aug. 2013.
unintended blinks mainly based on the synchrony between the [18] A. Bulling, J. A. Ward, H. Gellersen, and G. Tröster, “Eye movement
eye blinks and the flashes of a button. According to the ex- analysis for activity recognition using electrooculography,” IEEE Trans.
perimental results, the average accuracies and RTs for healthy Pattern Anal. Mach. Intell., vol. 33, no. 4, pp. 741-753, Apr. 2011.
[19] K. M. Dalton, B. M. Nacewicz, T. Johnstone, H. S. Schaefer, M. A.
subjects/patients with SCIs were 96.7%/91.7% and 3.53 s/3.67 Gernsbacher, H. H. Goldsmith, A. L. Alexander, and R. J. Davidson,
s, respectively. Compared with a newly published EOG-based “Gaze fixation and the neural circuitry of face processing in autism,”
HMI, the proposed HMI provides more commands with higher Nat. Neurosci., vol. 8, no. 4, pp. 519-526, Apr. 2005.
[20] D. W. Hansen and Q. Ji, “In the Eye of the Beholder: A Survey of
accuracy and ITR, lower FPR and fewer electrodes. The trade- Models for Eyes and Gaze,” IEEE Trans. Pattern Analysis and Machine
off is that the response time is slightly longer. Thus, the Intelligence, vol. 32, no. 3, pp. 478-500, Mar. 2010.
effectiveness of the system is demonstrated. In the future work, [21] J. Ma, Y. Zhang, A. Cichocki, and F. Matsuno, “A novel EOG/EEG
hybrid human-machine interface adopting eye movements and ERPs:
we will continue to reduce the cost of the system and will Application to robot control,” IEEE Trans. Biomed. Eng., vol. 62, no. 3,
expand the application range. pp. 876-888, Mar. 2015.

0018-9294 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TBME.2017.2732479, IEEE
Transactions on Biomedical Engineering
IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING 11

[22] R. Barea, L. Boquete, M. Mazo, and E. Lopez, “System for Assisted


Mobility Using Eye Movements Based on Electrooculography,” IEEE
Trans. Neural Syst. Rehabil. Eng., vol. 10, no. 4, pp. 209-218, Dec.
2002
[23] H. Manabe, M. Fukumoto, and T. Yagi, “Direct gaze estimation based
on nonlinearity of EOG,” IEEE Trans. Biomed. Eng., vol. 62, no. 6, pp.
1553-1562, Jun. 2015.
[24] B. Obermaier, C. Neuper, C. Guger, and G. Pfurtscheller, “Information
transfer rate in a five-classes brain-computer interface,” IEEE Trans.
Neural Syst. Rehabil. Eng., vol. 9, no. 3, pp. 283-288, Oct. 2001.
[25] J. F. Ditunno, W. Young, W. H. Donovan, and G. Creasey, “The Interna-
tional Standards Booklet for Neurological and Functional Classification
of Spinal Cord Injury,” Paraplegia, vol. 32, no. 2, pp. 70-80, Feb. 1994.
[26] S. G. Hart and L. E. Staveland, “Development of NASA-TLX (Task
Load Index): Results of Empirical and Theoretical Research,” Advances
in Psychology, vol. 52, no. 6, pp. 139-183, Jan. 1988.

0018-9294 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.