Chapter1
INTRODUCTION
1.1 General Introduction
The patterns of interaction between these neurons are represented as thoughts and
emotional states. According to the human thoughts, this pattern will be changing which in
turn produce different electrical waves. A muscle contraction will also generate a unique
electrical signal. All these electrical waves will be sensed by the brain wave sensor and it
will convert the data into packets and transmit through Bluetooth medium [1]. LAU will
receive the brain wave raw data and it will extract and process the signal using Mat lab
platform. Then the control commands will be transmitted to the robotic module to process
With this entire system, we can move a robot according to the human thoughts and it can
be turned by blink muscle contraction.
The basic idea of BCI is to translate user produced patterns of brain activity into
corresponding commands. A typical BCI is composed of signal acquisition and signal
processing (including pre-processing, feature extraction and classification) [2]. Although
some BCI systems do not include all components and others group two or three
components into one algorithm, most systems can be conceptually divided into signal
acquisition, pre-processing, feature extraction, and classification The brain signals that
are widely used to develop EEG-based BCIs include P300 potentials, which are a positive
potential deflection on the on-going brain activity at a latency of roughly 300ms after the
random occurrence of a desired target stimulus from non-target stimuli the stimuli can be
in visual, auditory, [3] or tactile modality SSVEP, which are visually evoked by a
stimulus modulated at a fixed frequency and occur as an increase in EEG activity at the
stimulus frequency and the ERD and ERS, which are induced by performing mental
tasks, such as motor imagery, mental arithmetic, or mental rotation [4].
The main reason for this is that the BCI is not stable due to the non-stationary
nature of the EEG signals. Thus, to make these mobile robots usable in real-world
situations, BCI systems need to be explored.
1.2 Aim
The aim of this project is to read the electrical signals from the brain and to extract
the raw data from these signals. It also aims in providing corresponding movement of the
robotic arm based on the raw data.
1.3 Objectives
To extract the electrical signals from brain using an EEG headset and transmit
them to PC/Laptop using Bluetooth.
To provide corresponding movements of the arm in correspondence to the data
obtained from processing the electrical signals in a PC/Laptop.
To enable the robotic arm to pick and place an object and also to study the
efficiency of the same.
The proposed robotic arm is being developed to in view of the following problems:
Trying to develop a prototype that can be used to allow bedridden and semi
paralysed patients to pick n place certain objects over certain distances without
having to move by himself
The proposed prototype can also be effective in providing employment to
Physically Challenged people.
1.5 Methodology
The idea is to develop a prototype robotic arm that can be controlled using the
brain signals.
The electrical signals are read by an EEG NeuroSky Headset and are then
transmit ted to a PC/Laptop via Bluetooth.
The signals are then processed in MATLAB using a Level Analysis Platform
to ex tract the raw data from these signals.
This raw data is then sent via XBee transceiver to the ARM microcontroller in
the Robotic arm which results in the corresponding movements of the robotic
arm.
The basic idea of BCI is to translate user produced patterns of brain activity into
corresponding commands. A typical BCI is composed of signal acquisition and signal
processing (including pre-processing, feature extraction and classification). One of the
most important features in a BCI system is represented by acquisition. The most spread
acquisition technique is EEG, and it represents a cheap and portable solution for
acquisition. The EEG technique assumes brainwaves recording by electrodes attached to
the subjects scalp. EEG signals present low level amplitudes in the order of microvolts
and frequency range from 1 Hz up to 100 Hz. Specific features are extracted and
associated with different states of patient brain activity, and further with commands for
developed applications.
Using EEG one more drawback can be eliminated (i.e. dangerous surgery can be
avoided for invasive method where electrodes are placed inside of brain called implants).
The EEG Headset or the Brainwave Sensor detects the electrical signals from the
brain and sends them in the form of data packets to a PC/Laptop via Bluetooth.
This received data is processed in MATLAB and the control commands are then
transmitted to the Arm via RF.
Based on the data received by the Microcontroller from the PC/Laptop it performs
certain predetermined actions depending upon the levels of concentration.
1.6 Applications
Can be used as a Smart Prosthetic for physically challenged persons that can
help them to move objects by interpreting the Brainwaves of the user.
Can also be used in Industries and companies to provide employment to
physically challenged persons.
1.7.1 Hardware
1.7.2 Software
MATLAB
KEIL
FLASHMAGIC
1.8 Advantages
The Robotic Arm can be used to fetch objects located at a certain distance
without requiring the user to move.
It is a Smart Prosthetic for physically challenged persons that can help them to
move objects.
1.9 Disadvantages
This paper describes about a brain controlled robot based on BCI. BCIs are
systems that can bypass conventional channels of communication (i.e., muscles and
thoughts) to provide direct communication and control between the human brain and
physical devices by translating different patterns of brain activity into commands in real
time. With these commands a mobile robot can be controlled. Here the robot is self-
controlled with the ultrasonic sensor. The intention of the project work is to develop a
robot that can assist the disabled people in their daily life to do some work independent
on others. Human brain consists of millions of interconnected neurons.
This paper proposes and implements a brain signal (mind) controlled prosthetic
arm to yield different movements in prosthetic arm. The key lies in the mapping of the
EEG signal to the prosthetic arm to achieve the objective. In this project, we are
developing a cost effective BCI prosthetic arm that will help the physically challenged
people to lead an independent life with the help of their brain signals using non-invasive
techniques.
This paper deals with the development of a practical BCI for disabled people
which shall allow them to use all their remaining functionalities and to use the currently
best ones. Two different fusion techniques are explored showing graceful performance
degradation in the case of signal attenuation. Such a system allows a very reliable control
and a smooth handover if the subjects get exhausted or fatigued during the day.
This paper presents a BMI system based on using the brain EEG signals
associated with 3 arm movements (close, open arm and close hand) for controlling a
robotic arm. Signals recorded from one subject using Emotive Epoch device. Four
channels only were used, in our experiment, AF3, which located at the prefrontal cortex
and F7, F3, FC5 which located at the supplementary motor cortex of the brain. Three
different techniques were used for features extraction which are: WT, FFT and PCA.
Multi-layer Perceptron Neural Network trained by a standard back propagation algorithm
was used for classifying the three considere -d tasks. Classification rates of 91.1%, 86.7%
and 85.6% were achieved with the three used features extraction techniques respectively.
Experimental results show that the proposed system achieved high classification rates
than other systems in the same application.
controlled. The intention of the project work is to develop a robot that can assist the
disabled people in their daily life to do some work independent on others.
REFERENCES:
[1] Brainwave Controlled Robot Using Bluetooth, Siliveru Ramesh1, K. Harikrishna2, J. Krishna
Chaithanya3 International Journal of Advanced Research in Electrical, Electronics and Instrumentation
Engineering [an ISO 3297:2007 CERTIFIED ORGANISATION] Vol 3. Issue 8, August 2014.
[2] Interaction of Human with Robotic Arm, Arul Prasath, Harihara Ganesh. S, Hariharan B, Hariharan
International Journal of Innovative research in Computer and Communication Engineering [an ISO
3297:2007 CERTIFIED ORGANISATION] Vol.4, Issue 3, March 2016.
[3] Multimodal Fusion of Muscle and Brain Signals for a Hybrid-BCI Robert Leeb, Hesam Sagha,
Ricardo Chavarriga and Jose Del R Milan. 32nd Annual International Conference of the IEEE EMBS
Buenos Aires, Argentina, August 31- September 4, 2010.
[4] Brain EEG signal processing for controlling a robotic arm, International Conference on Computer
Engineering and Systems(ICCES) November 2013 DOI:10.1109/ICCES.2013.6707191.
[5] The Brain Signal Detection for Controlling the Robot, India. International Journal of Scientific
Engineering and Technology (ISSN: 2277-1581) Volume No.3 Issue No.10, 1st October, 2015 pp. 1280-
1283
Chapter 2
HARDWARE DESCRIPTION
The Hardware requirement for the project along with their key features are as given
below:
Key Features
Long range data integrity
Low power
ADC and I/O line support
Advanced Networking & Security
Easy-to-Use
Technical Specifications
Uses the TGMA1 module, dry Electrodes and Ear clip electrodes.
Automatic wireless pairing.
6-hours battery runs pairing.
Bluetooth v2.1 class 2 (10 meters range).
Android supports.
2.3 LPC2148
Features of the LPC2148
16bit/32bit ARM7TDMI-S microcontroller in a tiny LQFP64 PACKAGE.
8KB to 40 KB of on-chip static RAM and 32kb to 512 KB of on-chip flash
memory. 128-bit wide interface/accelerator enable high-speed 60MHz operation.
One or two 10-bit ADCs provide a total of 6/14 analog input, with conversion
times as low as 2.44 us per channel.
Single 10-bit DAC provides variable analog output.
Two 32 bit timer/external event counter, PWM unit and watchdog.
Low power RTC with independent power and 32 KHz clock input.
On chip integrated oscillator operates with an external crystal from 1 MHz to
25MHz.
Power saving modes include Idle and Power-down.
Individual enable/disable of peripheral functions as well as peripheral clock
scaling for
Additional power optimization.
Processor wake-up from Power-down mode via external interrupt or BOD.
Single power supply chip with POR and BOD circuits:
CPU operating voltage range of 3.0 V to 3.6 V (3.3 V 10 %) with 5 V tolerant
I/O
Multiple serial interfaces including two UARTs (16C550), two Fast I2C-bus (400
kbit/s),
SPI and SSP with buffering and variable data length capabilities.
Vectored Interrupt Controller (VIC) with configurable priorities and vector
addresses.
Up to 45 of 5 V tolerant fast general purpose I/O pins in a tiny LQFP64 package.
Up to 21 external interrupt pins available.
60 MHz maximum CPU clock available from programmable on-chip PLL with
settling Time of 100 s.
2.5 Bluetooth
A) Bluetooth-based application
The Bluetooth lighting control and monitor application provides four main
functions to the phone user, namely:
1) Device registration
2) Lighting status monitoring
3) Lighting control
4) Diagnostics utility
B) Application Configuration
Prior to implementation of Bluetooth-based application on the phone, several
software packages are required which include Java Development Kit (JDK), the Eclipse
software environment, Android Development Tools (ADT) and Android SDK (Software
Development Kit).These open-source software packages can be downloaded from:
1) www.eclipse.org(The Eclipse )
2) Developer.android.com(ADT and SDK)
3) www.oracle.com (JDK)
The Bluetooth server application is developed in Java using the Eclipse integrated
development environment (IDE) which ensures the easy and quick development of the
application.
Chapter 3
The EEG Headset is the main part of the project. It detects the brain signals
whenever there is an eye blink and sends them in the form of data packets to a PC/Laptop
via Bluetooth. This received data is processed in MATLAB and the control commands
are then transmitted to ARM via ZIGBEE module.
The user of the project is supposed to wear the Headset with the electrodes placed
at the appropriate areas. The main electrode is to be placed on the forehead and the
second electrode, also called the reference electrode which is usually an ear clip is to be
placed at the bottom of ear.
Once the user wears the headset and then the headset is turned on by sliding the
ON/OFF button provided in the headset towards ON position. Once it is ON it is to be
paired with the PC using the Bluetooth Module. Once pairing is done the headset can be
used to read RAW Brain signals.
The main source of the EEG signals is the Neurons. The brain's electrical charge
is maintained by billions of neurons. Neurons are electrically charged by membrane
transport proteins that pump ions across their membranes. Neurons are constantly
exchanging ions with the extracellular milieu, for example to maintain resting
potential and to propagate action potentials. Ions of similar charge repel each other, and
when many ions are pushed out of many neurons at the same time, they can push their
neighbours, who push their neighbours, and so on, in a wave. This process is known as
volume conduction.
When the wave of ions reaches the electrodes on the scalp, they can push or pull
electrons on the metal on the electrodes. Since metal conducts the push and pull of
electrons easily, the difference in push or pull voltages between any two electrodes can be
measured by a voltmeter. Recording these voltages over time gives us the EEG.
In our project we make use of a NeuroSkys Brainsense EEG Headset that makes
use of the Thinkgear technology inside all the products of NeuroSky and enables devices
to interface with the human brain signals. It includes the sensor at the forehead, contact
and the reference points located the ear clip along with the on-board chip that processes
all the data and makes it suitable for transmission over Bluetooth.
The proprietary algorithm used in the NeuroSky headset is known as eSense and
is used for characterising mental states. This algorithm amplifies the brainwave signals
and then removes the ambient noise and muscle movement from the brainwaves. The
eSense algorithm is applied to the remaining signal that results in interpreted meter values
that describes how effectively the user is involving in Attention, similar to concentrating
and Meditation, similar to relaxing.
For each of the Attention and Meditation the meter generates a value on an eSense
scale ranging from 0 to 100. A value that lies between 40 to 60 is said to be neutral,
whereas values from 60 to 80 are considered to be slightly elevated and the values above
80 are considered to be highly elevated, which means they indicate heightened levels of
that particular eSense.
In the same way the values between 20 to 40 are said to be slightly lowered and
that below 20 is said to be strongly lowered levels of eSense and these levels indicate
distractions, wandering thoughts, agitation or abnormality.
These values are then sent over Bluetooth for further processing on MATLAB
which determines the further actions to be performed. Our project involves 2 modes of
controlling the Robot. The first mode is to control using the concentration levels and the
other is using blink detection. This data is interpreted by MATLAB and is sent to the
ARM controller in the robot module via ZIGBEE.
The ARM controller makes use of H-Bridge for controlling the DC Motors for the
movement of the Robot as well as the ARM that is mounted over the robot. The entire
robot module is powered by a DC source or say battery.
Chapter 4
IMPLEMENTATION
The basic idea of BCI is to translate user produced patterns of brain activity into
corresponding commands. A typical BCI is composed of signal acquisition and signal
processing including pre-processing, feature extraction and classification. One of the most
important features in a BCI system is represented by acquisition. The most spread
acquisition technique is EEG, and it represents a cheap and portable solution for
acquisition. The EEG technique assumes brainwaves recording by electrodes attached to
the subjects scalp. EEG signals present low level amplitudes in the order of microvolts
and frequency range from 1 Hz up to 100 Hz. Specific features are extracted and
associated with different states of patient brain activity, and further with commands for
developed applications. Using EEG one more drawback can be eliminated i.e. dangerous
surgery can be avoided for invasive method where electrodes are placed inside of brain
called implants.
The EEG Headset or the Brainwave Sensor detects the electrical signals from the
brain and sends them in the form of data packets to a PC/Laptop via Bluetooth. This
received data is processed in MATLAB and the control commands are then transmitted to
the Arm via ZIGBee.
Once the data is received by the PC via Bluetooth the Matlab processing begins and gives
out the discrete values of the eSense and is displayed on the screen along the the specified
actions.
Design Flow
The project flow diagram of Mind Controlled Robotic arm is shown in figure 5. It shows
various stages of the working of mind controlled robotic arm. After switching on the
Mindwave headset and the Robotic arm, the headset will initialize and will starts sensing
the neuron signals. At each sensing of the signals, it will transfer them to laptop using the
Bluetooth. The sensed raw data will be read and displayed by the Matlab [16]. It will also
help in sending these raw data to microcontroller of robotic arm using the RF Transmitter.
Two types of data can be measured by the Neurosky mindwave headset i.e. Attention
level and Meditation level. These levels will be received by microcontroller which will
help in controlling the robotic arm actions. The microcontroller should continuously
analyze the incoming brainwaves and map them into the appropriate actions.
Fig 4.1 and Fig 4.2 shows the Attention values of eSense as interpreted by the EEG
Headset and post processing in Matlab. This data is now sent to ARM controller over
ZIGBee. in the robot module there will be an XBee receiver will receive the data which is
transmitted by the XBee transmitter. According to the data received by the XBee the
ARM processor will give the directions to the motors and the robot is self-controlled
robot with ultrasonic sensor and connected with a relay and a driver circuit.