A literature survey on the contemporary

methodologies used in Brain computer interface for

spelling application
Subhadeep Karan
Department of Intelligent systems
Indian Institute of Information Technology
Allahabad,U.P.,India -211012
Email: subhadeepkaran@gmail.com

AbstractBrain computer interface (BCI) is a methodology

of utilizing the electrophysiological measures or the electroencephalographic (EEG) activity of the brain function in order to
communicate with the exterior world through a non-muscular
channel. It has become one of the primarily focussed fields
of research that includes challenges for the various fields like
Artificial intelligence, Cognitive neuroscience, user interface and
signal processing. Spelling can be considered as the first actual
BCI application and still remain a big challenge to the community. Generally, for communication application in BCI, this
application acts as benchmark. In this survey paper, the focus is
on the contemporary tactics used for the spelling words which
comprises of methods based on P300, motor imagery and steady
state visual evoked potential (SSVEP).This paper also highlights
some hurdles in the current BCI spellers and virtual keyboards.

1. I NTRODUCTION
A brain computer interface is radical approach in order to
communicate with machines. In BCI, the electroencephalographic activity is sensed and measured which is in turn
interpreted as a computer commands without the need of any
physical movements [1], [2], [3].
It is possible to register the modulation of the brain activity
(signals) from within the cortex for the neurons [4] or also on
the head scalp using electroencephalography. For several millions of people having neurotic disorders or other disabilities
so severe that they are unable to communicate with the others
in such cases BCI can be considered as a boon. For example,
people suffering from the Lou Gehrigs disease or any spinal
cord injuries, even amputation are appropriate for the Brain
computer interface [2], [5]. The EEG recording methodology
for BCI is preferred over more Electrocorticography (Ecog)
which is more efficient to register brain signals because in
case of EEG the recording can be done with inexpensive
instruments and it also gives more refined temporal resolution
in brain signals. Generally, BCI is divided into 4 major parts:
1) acquisition of signal; 2) aritfact removal (conditioning);
3) classification of the signal; and 4) action based on the output
of the classifier. EEG classification methods are based on what
types of response is to be detected: 1) slow cortical potential;
2) motor imagery; 3) steady state visual evoked potential; and

Fig. 1.

GUI for Bremen-BCI[9], [10]

item event related potential. And these responses are produced

using external stimulus such as auditory or visual and in some
cases even without them. Here, in this work we focus on the
spelling application of non-invasive brain computer interface
system. It still remains one of the most active research field in
BCI community, although a large number of application are
developed for instance Video games[6], robotic arm controller
etc. [7], [8].
The rest of the paper is devised as follows: Spellers based
on steady state visual evoked potential in section 2.Use of
detection P300 for the speller application presented in section
3. In section 4 the speller developed using the motor imagery.
Finally, these spellers are discussed in section 5.
2. BCI SPELLERS BASED ON STEADY- STATE VISUAL
EVOKED POTENTIAL (SSVEP)
The steady-state visual evoked potential based BCI system
mirrors the users attention to the oscillating visual stimulus
[11], [12]. In such systems, flickering of the lights is utilized
as stimuli but with different frequencies. And corresponding
to the SSVEP, the response tends to appear in the cortex
region with same frequency but with a higher harmonics [13].
Amplitude and phase which define a SSVEP response depends
on various factors such as: intensity, structure of the iterative
visual pattern (in stimulus) and frequency [14]. Controlling
neuroprosthetic devices, for the restoration of grasping ability
in injured persons[15] and in gaming industry [6] are one of

Fig. 2.

GUI for Bremen-BCI[22]

the few applications where BCI based on SSVEP are used and
these BCIs are reported to be performing and reliable as per
the expectation [16], [17], [18], [19]. The fewer number of
commands of an SSVEP BCI involves an intelligent tactics
to come up with a GUI. Bremen-BCI speller is a SSVEP
based and it was evaluated in the CeBIT fair 2008 and also
in RehaCare 2008 both took place in Germany [20], [21], [9],
[10]. The GUI which is presented in Fig. 1 comprises of a
virtual keyboard with 32 characters and its placed at the midlocation of the screen. The 5 boxes which are placed near
the outer edges and upper left corner on the screen are made
to flicker with various different frequencies and these boxes
stand for instructions such as Select, Up, Down, Right and
Left setup. This setup is also more comfortable for the user
as they dont have to shift their gaze, as the stimuli is part of
the GUI.
It was found that the detection of the commands was done
with an accuracy of 93% and with 23 bits per minute average
transfer rate for information which is roughly around 3.6
characters per minute (cpm).
Cecotti [22] (CBCI) recently developed a speller which
skips the calibration step and its ready once the user is
prepared. On the contrary, to the SSVEP based speller where
the visual stimuli instructions are merged, in this case the
stimuli (visual) are completely integrated in GUI. The speller
has the ability to write all the Latin alphabets andSS (its a
symbol refer the paper) was used to separate the words. Fig.
2 depicts the screen layout of BCI. Here, the GUI corresponds
to a 3 possible choices in the menu when anyone of the choices
is selected; the content of this selection is splits into three new
options. There are three instructions which are dedicated for
navigation and refer to the three boxes which comprises of
all the possible letters. Healthy users/subjects were selected
to test and on average accuracy of 93% and transfer rate for
information was 38.62 bits per minute (5.5 cpm). And the
highest writing speed (information transfer) was found to be
7.34 cpm for a particular subject.
3. U SE OF P300 FOR THE SPELLER APPLICATION
Event related potentials (ERP) are one of the widely used
methodologies in the BCI systems. It is defined as any conventional electro-physical response to the stimulus. A typical
P300 speller constructed on the basis of typical ERP is

Fig. 3.

GUI for Classical P300 based BCI-speller

Fig. 4.

GUI for intendiX [26]

an application which allows spelling character within their

thought process only. Farewell et al. [23] introduced the very
first BCI which was a P300 based speller in 1988. P300
wave is an apt example of ERP [24]. The generation of
P300 was possible due to the oddball paradigm in which
the subject is provided with a random visual stimulus which
results a surprise effect on the subject. The screen layout for
the classical speller (P300) which is illustrated to the user on
the computer screen is depicted in Fig. 3, which comprises of
6X6 matrix which contains alphanumeric values [23], [25]. In
the experiment, the user was required to focus on the character
which he/she wants to spell. A positive deflection at a response
time (latency) of about 300ms from the onset of the stimulus
in the EEG can be detected (i.e. P300 detection to the time
locked to the onset for the illumination/intensification of the
cell). Here, the ERPs are generated by their intensification
randomly with equal number of events for the row and column
intensification and the process of intensification is repeated
Nepoch times for all the characters.
Initially, P300 speller was developed only for the Latin characters, but recently a P300 speller for Chinese characters has
been proposed[27].Over the years, in the field of BCI speller,
advancement was done in the signal processing, detection and
classification techniques like Artificial neural network(ANN)
[28], [29] , Bayesian based Linear discriminant analysis(LDA)
[30], [31], or Support vector machines[32]. However, the
development in the graphical user interface (GUI) for the P300
speller has been dormant. There were propositions like in [33],
where green/blue was to be preferred over other colours to

Fig. 5.

GUI for AIRLab-BCI [35]

be used in the flickering matrix. An alternative paradigm has

been proposed for the traditional flashes of the column/row
paradigm (RCP). A checkerboard paradigm (CBP) which is
an amalgamation of the 2 checker boards in order to avert the
well-known issue of ambiguity in the target neighbourhood.
A matrix of dimension 8x9 filled with alphanumeric characters and keyboard commands. 8 subjects/participants used
the checkerboard paradigm (CBP) and row/column paradigm
(RCP). The CBP outperformed RCP both in terms of accuracy
and bit rate. As the accuracy for CBP was 92% with bit rate of
23 bits per minute and 78% accuracy with 17 but per minute
for RCP.
In order to improve P300 speller, research work is carried
in mainly two sectors: 1) better methodologies resulting in
improved reliability across subjects and time and 2) possible
improvements in the P300 detection.
The present P300 lacks the reliability and the required
robustness across time and subjects which is keeping P300
speller far from being a commercial/clinical application. But
there are few exceptions like in [34], where a speller based on
P300 can be used for last stage amyotrophic lateral sclerosis
(ALS) patient at home. The French National Research Agency
(FANR) is aiming at imitating and improvising such stories
by providing a user dedicated BCI which can be used by a
non-technical person through Robust BCI keyboard project. In
2009, intendiX solution was presented by G.tec Company and
this system used visually evoked EEG potentials which are
VEP or P300. Here, the subjects are asked to concentrate on
the target character from the matrix on the screen. The screen
layout for the intendiX solution is depicted in Fig. 4. This
system was able to identify the idly state, so that means the
BCI selects the character only when the user intends to use it.
4. U SE OF MOTOR IMAGERY FOR SPELLER -BCI
Similar to the SSVEP-BCI, the interface is limited by the
number of available instructions as it is impossible to associate
an imagery movement to each and every character. In order to
overcome, this issue it is necessary to come up with a strategy
combined with few basic BCI instructions e.g. laughing to
shaking left/right hand. Recently, a motor imagery based
speller was proposed at AIRLab at the Technical University

Fig. 6.

GUI for Hex-o-Spell [36], [37]

of Milan, Italy [35]. Fig. 5 illustrates the GUI for this speller.
This speller utilizes a predictive methodology where the words
are suggested and at the same time disables the improbable
symbols. The performance in such BCI did not outperform
the other types of BCI like specified in section 2 and 3. But
with few subjects they reached a speller speed of around 3
cpm. In such cases, the subjects with lower accuracy initially,
demonstrated an improvement after more training session
reaching a speed of 2.7 cpm [35].
Hex-o-Spell is a speller proposed by a research group named
Fraunhofer FIRST (IDA), Germany[36], [37]. This speller had
the capability to write 29 characters along with a backspace
command and its asynchronous in nature. It used 2 mental
states Imagined right hand and right foot movements respectively. As presented in Fig. 6, the GUI of the Hex-0-Spell 6
hexagonal fields where constructed surrounding a circle. In
that circle, the arrow rotates in a clockwise direction when
right hand movement is imagined and stops when the right foot
movement is imagined. Once the arrow pointing hexagonal
field is selected the contents of that field are arranged in the
six hexagonal field with one character in each hexagonal field.
This BCI was tested with two healthy volunteers in CeBIT fair
2006 in Germany with once user capable of writing at a speed
of between 2.5 and 5.1 cpm and the other between 4.5 and
7.4 cpm.
5. D ISCUSSION
Even with numerous different results reported in literature,
an objective based comparison between various available
spellers is pretty difficult due to inter subject variability and
the experimental conditions. But each and every BCI paradigm
has its own pros and cons. External visual stimuli is required
in BCIs which are based on P300 or SSVEP. The visual
stimuli cannot be considered as a real disadvantage in speller
application. Indeed, motor imagery based BCI also considers
GUI. The layout for the P300 speller is similar to a virtual

keyboard where each symbol is generally available on the

screen. On the contrary to P300 speller, the SSVEP based
speller must consider the numerous constraints based on the
visual stimuli. A large number of neural stimuli can be
generated with LEDs when operated with different frequencies
[18], [25]. As the application and the stimuli (visual) do not
share the same location, in such situations an external device
is necessary. If LCD screen is utilized for the visual stimuli,
then factors such as screen size, refresh rate, pixel resolution
restricts the number of simultaneous stimuli. For such reasons,
it is not easy to present to the subject a virtual keyboard
and having direct access to the letter. The efficiency of BCI
should be reliable over a long time and among the users, for
a BCI to be efficient. Spellers based on motor imagery can be
termed as efficient as Hex-o-Spell. However, calibration of the
system and rigorous training sessions are required for motor
imagery or P300 based BCIs. BCIs which are based on the
motor imagery suffer due to 2 major reasons: 1) BCI illiteracy
2) Functioning and executional capability is highly dependent
subject whereas BCIs based on SSVEP have a high transfer
rate and training sessions are not necessary[16], [22].
6. C ONCLUSION
With the development in other fields like signal processing,
cognitive neuroscience, BCI community has used these advancements in order to leverage its development and currently
a large number of BCI literatures are available. However, composing an email, writing a simple text message still remains
quite a tough task for the people suffering from disabilities.
With the recent competitions in the field of BCI, it has now
become possible to assess machine learning algorithms against
each other; however such are restricted to only one aspect
of BCI. Indeed, its now necessary that the GUI in this field
should receive as much attention as the signal processing
receives [38]. For BCIs based on SSVEP the number of flashes
available on the display unit is highly constrained. All the
spellers discussed and presented in this work are based only
on one type of brain activity but forthcoming BCIs need to
collectively evaluate the various brain responses, as hybrid
BCIs [39]. Such solutions are expected to overcome the issue
of BCI illiteracy to some extent and be proving more zippy
and robust spellers. In order to understand this problem, some
work is recently being done for P300 [40], [41], systems
of sensorimotor [42] and steady-state visual evoked potential
(SSVEP)[20]. In future, work should be undertaken in order to
assess, understand and compare different but well-known and
proven system with keeping the same experimental setup. With
recent reliable and established frameworks like OpenVIBE
and BCI2000, one could be hopeful to get more acceptable
comparison of all the spellers [43], [44].
R EFERENCES
[1] B. Z. Allison, E. W. Wolpaw, and J. R. Wolpaw, Brain?computer
interface systems: progress and prospects, Expert Review of Medical
Devices, vol. 4, pp. 463474, 2007.
[2] N. Birbaumer and L. G. Cohen, Brain-computer interfaces: communication and restoration of movement in paralysis, Journal of Physiologylondon, vol. 579, pp. 621636, 2007.

[3] A. Kostov and M. Polak, Parallel man-machine training in development

of EEG-based cursor control, IEEE Transactions on Rehabilitation
Engineering, vol. 8, pp. 203205, 2000.
[4] D. J. McFarland and J. R. Wolpaw, Brain-computer
interfaces for communication and control, Commun. ACM,
vol. 54, no. 5, pp. 6066, May 2011. [Online]. Available:
http://doi.acm.org/10.1145/1941487.1941506
[5] N. Birbaumer, N. Ghanayim, T. Hinterberger, I. Iversen, B. Kotchoubey,
A. Kbler, J. Perelmouter, E. Taub, and H. Flor, A spelling device for
the paralysed, Nature, vol. 398, pp. 297298, 1999.
[6] E. C. Lalor, S. P. Kelly, C. Finucane, R. Burke, J. R. Smith, R. B. Reilly,
and G. McDarby, Steady-State VEP-Based Brain-Computer Interface
Control in an Immersive 3D Gaming Environment, Eurasip Journal on
Advances in Signal Processing, vol. 2005, pp. 31563164, 2005.
[7] F. Cincotti, D. Mattia, F. Aloise, S. Bufalari, G. Schalk, G. Oriolo,
A. Cherubini, M. G. Marciani, and F. Babiloni, Non-invasive braincomputer interface system: Towards its application as assistive technology,
Brain Research Bulletin, vol. 75, pp. 796803, 2008.
[8] T. Luth, D. Ojdanic, O. Friman, O. Prenzel, and A. Graser, Low
level control in a semi-autonomous rehabilitation robotic system via
a Brain-Computer Interface, in IEEE International Conference on
Rehabilitation Robotics, 2007.
[9] I. S. D. Valbuena and A. Graser, Spelling with the bremen braincomputer interface and the integrated ssvep stimulator, in roceedings of
the 4th International Brain-Computer Interface Work-shop and Training
Course, 2008, pp. 291296.
[10] I. Volosyak, H. Cecotti, D. Valbuena, and A. Graser, Evaluation of
the bremen ssvep based bci in real world conditions, in Rehabilitation
Robotics, 2009. ICORR 2009. IEEE International Conference on, 2009,
pp. 322331.
[11] F.-B. Vialatte, M. Maurice, J. Dauwels, and A. Cichocki, Steady-state
visually evoked potentials: Focus on essential paradigms and future
perspectives, Progress in Neurobiology, vol. 90, pp. 418438, 2010.
[12] B. Blankertz, M. Krauledat, G. Dornhege, and J. Williamson, A note
on brain actuated spelling with the berlin brain-computer interface, in
Universal Access in HCI, Part II, HCII 2007, 2007, pp. 759768.
[13] G. R. Mller-Putz, R. Scherer, C. Brauneis, and G. Pfurtscheller,
Steady-state visual evoked potential (ssvep)-based communication:
impact of harmonic frequency components, Journal of Neural
Engineering, vol. 2, no. 4, p. 123, 2005. [Online]. Available:
http://stacks.iop.org/1741-2552/2/i=4/a=008
[14] Z. Wu, Y. Lai, Y. Xia, D. Wu, and D. Yao, Stimulator
selection in ssvep-based {BCI}, Medical Engineering and Physics,
vol. 30, no. 8, pp. 1079 1088, 2008, ce:titleSpecial Issue
(part): Bioengineering in Taiwan/ce:title. [Online]. Available:
http://www.sciencedirect.com/science/article/pii/S1350453308000234
[15] G. R. Muller-Putz and G. Pfurtscheller, Control of an Electrical Prosthesis With an SSVEP-Based BCI, IEEE Transactions on Biomedical
Engineering, vol. 55, pp. 361364, 2008.
[16] G. Bin, X. Gao, Z. Yan, B. Hong, and S. Gao, An online multi-channel
SSVEP-based brain-computer interface using a canonical correlation
analysis method, Journal of Neural Engineering, vol. 6, 2009.
[17] H. Cecotti and A. Graeser, Convolutional Neural Network with embedded Fourier Transform for EEG classification, in International
Conference on Pattern Recognition, 2008, pp. 14.
[18] X. Gao, D. Xu, M. Cheng, and S. Gao, A BCI-based environmental
controller for the motion-disabled, IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 11, pp. 137140, 2003.
[19] O. Friman, I. Volosyak, and A. Graser, Multiple Channel Detection of
Steady-State Visual Evoked Potentials for Brain-Computer Interfaces,
IEEE Transactions on Biomedical Engineering, vol. 54, pp. 742750,
2007.
[20] B. Allison, T. Lth, D. Valbuena, A. Teymourian, I. Volosyak, and
A. Graeser, BCI Demographics: How Many (and What Kinds of)
People Can Use an SSVEP BCI? IEEE Transactions on Neural Systems
and Rehabilitation Engineering, vol. 18, pp. 107116, 2010.
[21] H. Cecotti, I. Volosyak, and A. Graser, Evaluation of an SSVEP based
Brain-Computer Interface on the command and application levels, 2009.
[22] H. Cecotti, A Self-Paced and Calibration-Less SSVEP-Based BrainComputer Interface Speller, IEEE Transactions on Neural Systems and
Rehabilitation Engineering, vol. 18, pp. 127133, 2010.
[23] L. A. Farwell and E. Donchin, Talking off the top of your head:
toward a mental prosthesis utilizing event-related brain potentials,

[24]
[25]
[26]
[27]
[28]

[29]
[30]
[31]
[32]
[33]

[34]

[35]
[36]

[37]

[38]
[39]
[40]

[41]

[42]
[43]

[44]

Electroencephalography and Clinical Neurophysiology, vol. 70, pp. 510

523, 1988.
S. Sutton, M. Braren, J. Zubin, and E. R. John, Evoked-Potential
Correlates of Stimulus Uncertainty, Science, vol. 150, pp. 11871188,
1965.
E. Donchin, K. M. Spencer, and R. Wijesinghe, The mental prosthesis:
assessing the speed of a P300-based brain-computer interface, IEEE
Transactions on Rehabilitation Engineering, vol. 8, pp. 174179, 2000.
G. Technologies. [Online]. Available: http: // www. gtec. at/
J. Jin, B. Z. Allison, C. Brunner, B. Wang, X. Wang, J. Zhang, C. Neuper,
and G. Pfurtscheller, P300 Chinese input system based on Bayesian
LDA, Biomedizinische Technik, vol. 55, pp. 518, 2010.
H. Cecotti and A. Grser, Convolutional Neural Networks for P300
Detection with Application to Brain-Computer Interfaces, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 33, pp. 433
445, 2011.
N. Liang and L. Bougrain, Averaging techniques for single-trial analysis
of oddball event-related potentials, 2008.
B. Rivet, H. Cecotti, E. Maby, and J. Mattout, Impact of Spatial Filters
During Sensor Selection in a Visual P300 Brain-Computer Interface,
Brain Topography, pp. 19.
P.-J. Kindermans, D. Verstraeten, and B. Schrauwen, A Bayesian Model
for Exploiting Application Constraints to Enable Unsupervised Training
of a P300-based BCI, PLOS One, vol. 7, 2012.
A. Rakotomamonjy and V. Guigue, BCI Competition III: Dataset IIEnsemble of SVMs for BCI P300 Speller, IEEE Transactions on
Biomedical Engineering, vol. 55, pp. 11471154, 2008.
K. Takano, T. Komatsu, N. Hata, Y. Nakajima, and K. Kansaku,
Visual stimuli for the P300 braincomputer interface: A comparison of
white/gray and green/blue flicker matrices, Clinical Neurophysiology,
vol. 120, pp. 15621566, 2009.
T. M. Vaughan, D. J. McFarland, G. Schalk, W. A. Sarnacki, D. J.
Krusienski, E. W. Sellers, and J. R. Wolpaw, The wadsworth BCI research and development program: at home with BCI, IEEE Transactions
on Neural Systems and Rehabilitation Engineering, vol. 14, pp. 229233,
2006.
T. DAlbis, A predictive speller for a brain-computer interface based on
motor-imagery. Masters thesis.
B. Blankertz, G. Dornhege, M. Krauledat, M. Schroder, J. Williamson,
R. Murray-Smith, and K.-R. Muller, THE BERLIN BRAINCOMPUTER INTERFACE PRESENTS THE NOVEL MENTAL
TYPEWRITER HEX-O-SPELL, 2006.
K.-R. Mller, M. Tangermann, G. Dornhege, M. Krauledat, G. Curio, and
B. Blankertz, Machine learning for real-time single-trial EEG-analysis:
From braincomputer interfacing to mental state monitoring, Journal of
Neuroscience Methods, vol. 167, pp. 8290, 2008.
B. Allison, O. Hahn, and A. NW, The I of BCIs: Next Generation Interfaces for Brain-Computer Interface Systems That Adapt to Individual
Users, 2009.
G. B. C. B. T. S. E. R. S. T. O. G. M.-P. C. G. Pfurtscheller, A. Brendan
and N. Bir-baumer., The hybrid bci, in Frontiers in Neuroprosthetics,
2010.
C. Guger, G. Edlinger, W. Harkam, I. Niedermayer, and G. Pfurtscheller,
How many people are able to operate an EEG-based brain-computer
interface (BCI)? IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 11, pp. 145147, 2003.
C. Guger, S. Daban, E. Sellers, C. Holzner, G. Krausz, R. Carabalona,
F. Gramatica, and G. Edlinger, How many people are able to control a
P300-based braincomputer interface (BCI)? Neuroscience Letters, vol.
462, pp. 9498, 2009.
C. Vidaurre and B. Blankertz, Towards a Cure for BCI Illiteracy, Brain
Topography, vol. 23, pp. 194198, 2010.
G. Schalk, D. J. McFarland, T. Hinterberger, N. Birbaumer, and J. R.
Wolpaw, BCI2000: a general-purpose brain-computer interface (BCI)
system, IEEE Transactions on Biomedical Engineering, vol. 51, pp.
10341043, 2004.
Y. Renard, F. Lotte, G. Gibert, M. Congedo, E. Maby, V. Delannoy,
O. Bertrand, and A. Lcuyer, OpenViBE: An Open-Source Software
Platform to Design, Test, and Use Brain-Computer Interfaces in Real and
Virtual Environments, Teleoperators and Virtual Environments, vol. 19,
pp. 3553, 2010.