This article has been accepted for publication in a future issue of this journal, but has not been

fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/JBHI.2014.2305403, IEEE Journal of Biomedical and Health Informatics

> Ref. No. TITB-00489-2013.R3

An Innovative Non-intrusive Driver Assistance

System for Vital Signal Monitoring
Ye Sun, Student Member, IEEE, Xiong Yu, Member, IEEE

AbstractThis paper describes an in-vehicle non-intrusive

bio-potential measurement system for driver health monitoring
and fatigue detection. Previous research has found that the
physiological signals including eye features, Electrocardiography
(ECG), Electroencephalography (EEG) and their secondary
parameters such as HR and HRV are good indicators of health
state as well as driver fatigue. A conventional bio-potential
measurement system requires the electrodes to be in contact with
human body. This not only interferes with the driver operation,
but also is not feasible for long-term monitoring purpose. The
driver assistance system in this paper can remotely detect the
bio-potential signals with no physical contact with human skin.
With delicate sensor and electronic design, ECG, EEG, and eye
blinking can be measured. Experiments were conducted on a high
fidelity driving simulator to validate the system performance. The
system was found to be able to detect the ECG/EEG signals
through cloth or hair with no contact with skin. Eye blinking
activities can also be detected at a distance of 10 cm. Digital signal
processing algorithms were developed to decimate the signal noise
and extract the physiological features. The extracted features
from the vital signals were further analyzed to assess the potential
criterion for alertness and drowsiness determination.
Index TermsHealth monitoring; driver fatigue; non-intrusive;
eye blinking; ECG; EEG; road safety; intelligent vehicle.

I. INTRODUCTION
rowing aging population is a global phenomenon in recent
decades. The increasing number of elderly car drivers and
the prevalence of chronic diseases call for driver
assistance systems to monitor the health state of drivers. For
medical-assistance systems, the reliable measurement of vital
signals such as EEG and ECG is one of the most important
features [1]. EEG, the recording of electrical activity along the
scalp, reflects the brain activities and is widely used in the
diagnosis of coma and encephalopathy. ECG and the secondary
parameters including heart rate (HR) and heart rate variability
(HRV) are key indicators of the cardiac health state. The
stressful condition of driving and the possible sudden scenarios
on the road, e.g. fatal traffic accidents, may cause severe effects
especially on the drivers with chronic diseases [2]. Therefore, a
driver assistance system that can monitor the multiple vital
signals during driving is highly desirable for elderly drivers or
drivers with chronic diseases.
For drivers at all ages, drowsiness is one of the most
prevalent root causes of accidents. It leads to nearly 17% of all
fatal crashes in recent years based on the data published by the
National Highway Traffic Safety Administration (NHTSA) [3].
In particular, truck driver fatigue is a factor in 3-6% of fatal
crashes involving large trucks. The term driver fatigue is

defined as decreased mental alertness that impairs performance

during some cognitive tasks such as driving [4]. The sustained
mental or physical fatigue can eventually result in sleepiness.
Some studies considered sleepiness and fatigue as similar
mental conditions [5], [6]. In this paper we also used the general
concept of sleepiness, drowsiness, and fatigue.
A number of physiological parameters have been found to
indicate the state of fatigue. Eye activity [7], [8], EEG [9], [10],
ECG and HRV [11], [12] are the major physiological indicators
used for assessment of sleepiness. Feature extraction was
developed based on different types of physiological signals to
classify drowsy versus awaken drivers. Therefore, the vital
signals (such as ECG, EEG, and eye blinking) are not only
indicators of the health state of drivers but also can be used to
detect the driver fatigue.
The conventional ECG and EEG systems use Ag/AgCl
electrodes with a wet electrolyte. Skin preparation is required
before placing the electrodes. For eye activity detection in
driving application, video cameras are commonly used to detect
the eye-related parameters such as PERCLOS [13] and
reopening time [7]. Image processing algorithms and related
computation processors are necessary to extract the features
from video images. However, few other methods were explored
to detect eye activities.
In contrast to the conventional wet electrodes, capacitive
electrode (CE) provides an alternative way of surface potential
measurement without direct contact with skin. This
non-intrusive electrode can sense the bio-potentials outside the
hair or cloth [14]. It enables long-term monitoring without skin
irritations. A number of devices using CE have been reported
for health care with sensors placed on bed [15], chairs [16] and
wheelchair [17], exercise assistance [18], and automobiles [1].
ECG or EEG signals can be detected from these systems.

Fig. 1. Driver assistance system

2168-2194 (c) 2013 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See
http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/JBHI.2014.2305403, IEEE Journal of Biomedical and Health Informatics

> Ref. No. TITB-00489-2013.R3

In the automobile application, high interference environment
(e.g. when engine is on) and severe motion artifacts are
unavoidable. Wet electrodes were commonly used to guarantee
good contact with skin and prevent relative motion. Capacitive
electrodes were also attempted by several groups for
unobtrusive ECG measurement [1], [19]-[22]. Large area metal
plates were installed on the steer or car seat as grounding or
driven-right-leg (DRL) electrodes to ensure stable signals.
In this study, we developed an innovative in-vehicle
non-intrusive driver assistance system that can measure ECG,
EEG and eye-blinking activities in one device (Fig.1). Unipolar
electrode was adopted to minimize the placement and enhance
the simplicity of installation and comfort of drivers. It also
provides an easy way to detect the eye activities compared with
a video camera. The proposed non-contact vital signal
monitoring system can not only monitor the health state of
drivers, but also detect driver fatigue. In this paper, details of
technology concept and system design were described.
Experiments were conducted on a high fidelity driving
simulator. It was found that EEG, ECG, and eye-activity
signals can be reliably measured by the system. From these,
basic medical care functions such as HR monitoring can be
accomplished in real time. Moreover, eye features, EEG
spectrum, and HRV features from the subjects in alertness and
sleepiness states were analyzed to explore the potential
algorithm for driver fatigue detection.
II. BACKGROUND
A. Skin-Electrode Interface
The bio-potentials on the skin are generated by the
bioelectric activity of organs such as heart and brain, and
conducted by neurons. Between the surface bio-potentials and
the electrodes, there are different layers including stratum
corneum, cloth, insulation layer, and air gap. Each of these
layers is equivalent to a RC parallel component. The
skin-electrode impedance can be calculated by the equivalent
model shown in Fig. 2 (a). The general total impedance of the
skin-electrode interface is [14]
n
1
(1)
Z s= R + Ri ||
jCi
i =1
where R is the total resistance of the conductive layer in the
path which is negligible in the calculation; Ri and Ci represent
the resistance and capacitance of a certain layer i ; n is the
number of layers.

(a)
(b)
Fig. 2. Equivalent models. (a) Equivalent skin-electrode interface of
capacitive electrodes; (b) Equivalent circuit of the following analog front end.

2
B. Specification of In-Vehicle Application
During driving process, drivers keep adjusting the operation
of vehicles according to the changing environment. The actual
interfaces between the skin and the electrode change with
motion. The impedance of skin-electrode interface is not stable,
which will affect the reliability of signal acquisition. When
using a small-area electrode, the coupling capacitance during
motion can vary from 0.1 to 100 pF. This calls for ultrahigh
input impedance of the amplifier to ensure the high sensitivity.
The typical input impedance of commercial amplifiers can be
up to 1015 /0.2 pF (i.e, INA116, Texas Instrument, Inc.). It is
sufficiently high to acquire the signal but cannot eliminate the
influence of the changes in the capacitance of interfaces. One
solution is bootstrapping to enhance the input impedance which
has been adopted by several groups for capacitive ECG
detection [14], [23].
Other than motion artifacts, high interference is also a
challenge for application of non-contact physiological
monitoring system for automobile applications. During driving,
the unscreened environment with high EM interference from
vehicle and cellphone can easily immerse the physiological
signals. The displacement current induced by the EM field
couples to the body and generates both common-mode and
differential-mode voltages. DRL circuit and good grounding of
body are usually used to reduce the common-mode voltage [1],
[22]. However, the DRL circuit requires additional electrodes
and wire connection. High CMRR amplifiers and good
shielding are also key issues to eliminate the common-mode
effects. For differential-mode voltage, highly matched
electrodes are required. High input impedance is also desirable
to lessen the differential noise [24]. In summary, delicate
system design is necessary to implement non-contact vital
signal sensing for automobile application.
III. SYSTEM DESIGN
A. System Architecture
Our proposed system adopts unipolar design to simplify the
system installation and increase the comfort of drivers. The
unipolar pattern was also adopted by [25]. The advantages and
feasibility were approved by [26]. In our design, a small
conductive plate is used as the electrode with the area of
4.5 2 cm2. Another plate of the same material and size is
attached to the steer as the ground connection. The entire
system is mainly composed of five components: electrodes,
bias current path, amplifier, multistage filter, and shielding. Fig.
3 shows the block diagram and the photo illustrations. The
circuit board is as small as a US quarter.
1) Electrode
In the proposed system, metal plates including stainless steel,
soft copper tap and solder mask, are tested as the electrode
material. Besides the common metallic electrodes, an appealing
type is the conductive polymers which are organic polymers
with high electrical conductivity. It features the advantages of
polymers to allow for flexible design of electrodes. The
electrons in this structure have high mobility when the material
is doped by oxidation which removes some of these delocalized
electrons. During doping, small fraction silicon atoms are
replaced by electron-rich or electron-poor atoms to create

2168-2194 (c) 2013 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See
http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/JBHI.2014.2305403, IEEE Journal of Biomedical and Health Informatics

> Ref. No. TITB-00489-2013.R3

Fig. 3. Block diagram and photos of the non-contact physiological detection sensor.

n-type and p-type semiconductors, respectively. In this article,

we evaluate four types of conductive polymers according to the
dopants. The conductive-polymer electrodes are flexible and
biocompatible, and can be conveniently integrated with
clothing.
2) Instrumentation Amplifier
To obtain high input impedance and low noise, an
instrumentation amplifier INA116 (Texas Instrument, Inc.) was
applied for amplification. The input impedance is 1015 /0.2
pF. The CMRR is around 90 dB at 0-1 kHz with the gain of 10
V/V.
3) Bias current path
The instrumentation amplifier has the input bias current with
a typical value of 3 fA. The bias current path provides an
equivalent resistance for the bias current to ground to avoid
saturation. However, the equivalent parallel resistance can
lower the input impedance if not larger than 100 T . Some
studies used the reverse current of signal diodes to provide the
bias current [35] [36]. In our design, we calculated the input
resistance and capacitance to the ground according to the actual
condition to form a bias path with impedance as 1014 . Then
we made the layout dimension based on the calculation strictly.
4) Filter
The frequency components of normal ECG signal ranges
from 0.01 to 100 Hz with energy concentrating in 0.5-45 Hz.
The major bands of EEG signal range in 0-30 Hz. Therefore, a
lowpass filter with cutoff frequency at 45 Hz is designed to
filter out the high frequency noise including the power line
interference. A fourth order Butterworth filer with the multiple
feedback topologies is chosen.
5) Shielding and grounding
Comparing with the DRL three-electrode circuit, the
single-ended circuit has higher noise level and less stability;
however, it requires less attachment and causes less distraction
to drivers. One compensative method is to reduce the common
mode voltage by good shielding and lessen the coupling
capacitance between body and ground by use of grounding
electrodes. In the design, we found that a thick copper shielding
box could reduce the interference significantly. The typical
detected signals are shown in Section V.
B. Equivalent Circuit Model
The equivalent circuit model considering the skin-electrode
interface is shown in Fig. 2 (b). The capacitance of each layer

can be computed as C = r 0 S / d , where S is the effective

area which is 4.5 2 cm2, and d is the thickness of 0.2-1 mm
for the cotton layer (dominant layer). Therefore, the
capacitance of the skin-electrode interface is ranging from 8 to
40 pF. The capacitance and the bias resistance form a high pass
filer with the cutoff frequency at around 0.08 Hz. The transfer
function for the circuit model can be expressed
Vo
AZ in
.
(2)
=
Vs
Z sb , sc + Z in + (1 A) jC f Z in Z sb , sc
The EM interference induced voltage can be calculated as
vn = id Z e [27], where id is the interference induced current
through circuit ground (0.1-1 nA [27]), and Z e is the electrode
impedance ranging from 1-10 k . Therefore the interference
induced voltage is around 0.1-10 V which is within tolerant
range.
IV. EXPERIMENT
A. Equipment
The experiments were conducted on a high fidelity driving
simulator. The simulator consists of a vehicle body with dual
and rear view mirrors, six angled projection surfaces, audio
systems and master control systems, providing 360o vision to
the drivers. The scenarios and driving routes can be designed
and loaded into the simulator. Prior to the experiment, normal
urban and highway scenarios were developed for driving
testing. The driver assistance system was installed as Fig. 1.
Fig. 4 shows the driving simulator and a typical scenario.
The driving experience on the simulator was evaluated by a
survey. Thirty subjects (8 female and 22 male, age 20-30,
average driving experience 4.11 years) participated in the
survey by driving the simulator for 15 minutes and answering a
questionnaire. Nine questions were designed to estimate the
similarity between driving in the simulator and real
environment in terms of the entire driving experience and eight
specific aspects such as the feeling of accelerating, distance,
collision, etc. The results showed that the driving simulator
reasonably well simulated the realistic driving environment
based on the feedbacks from the survey.
B. Subjects and Experiment protocol

2168-2194 (c) 2013 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See
http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/JBHI.2014.2305403, IEEE Journal of Biomedical and Health Informatics

> Ref. No. TITB-00489-2013.R3

eyes, scratching, yawning and nodding, were counted as

drowsiness countermeasures. The number of the occurrence of
the countermeasures determined the level of drowsiness. The
results were allied to each other. In the test-retest examination,
the results were similar. One segment considered as Class 5 was
considered as Class 4. Another two segments evaluated as
Class 2 were then considered to be Class 3. These Class 3
segments were removed for further analysis. Typical scoring
results of morning and afternoon sessions are shown in Fig. 5.
C. Signal Preprocessing
The measured bio-potential signal is contaminated with
several sources of noise, such as Electromyography (EMG),
power line interference, electronic noise, and baseline drifts.
The EMG signal is caused by human motion and muscle
contraction, which typically ranges between 6-500 Hz [32]; the
60-Hz interference is due to the power line; and the drift is
caused by the movement of electrode and breathing. The noise
is filtered partially by the hardware filtering. A digital bandpass
filer was introduced to further reduce the noise. The four major
bands of EEG signal ranges in 0-4 Hz, 4-8 Hz, 8-13 Hz and
13-30 Hz respectively. Therefore, the cutoff frequency of the
digital filter was designed to be changeable. The preprocessed
signals were then used in the subsequent analyses.
1

1
1
2
1
1
2

(a) Scale results in the morning section

1
1
2
4
2
4
3
5

(b) Scale results in the afternoon section

Fig. 5. Typical scale results using continuous Wierwille and Ellsworth
criteria. 1-Class 1, 2-Class 2, 3-Class 3, 4-Class 4, 5-Class5.
Detector hanging
from the roof
handle

(a)
(b)
Fig. 6. Eye blinking detection. (a) Experiment protocol; (b) a typical signal
detected from the system. t1, t2 are the eye closing time and the reopening time;
t1+t2 is the overall blinking time; t3 is the interval between blinking activities.
(a) wave-Typical signal

Voltage (mV)

4
2
0
4

wave-Detected signal

2
0
0.0

0.2
0.4
0.6
Time (s)
(b) wave-Typical Signal

0.8

1.0

Voltage (mV)

Fig. 4. High fidelity driving simulator

Twelve volunteer drivers (nine male and three female) aged

24-30 (26 2.12) years participated in the experiment on the
driving simulator. All the participants were recruited from the
university and selected as self-evaluated healthy and not
addicted to drug or alcohol. Each subject was instructed to drive
with his/her normal driving behaviors. Since the driver
assistance system can keep monitoring the health state by
measuring the vital signals, its performance was verified
simultaneously with the experiment of driver fatigue detection.
In the driving simulator, a copper strip detector was attached
to the safety handle from the roof, which hung to the side of the
drivers head at a distance of around 10 cm. The detector
remotely monitored the bio-potentials associated with eye
blinking activities (Fig. 6 (a)). For subjects wearing glasses, the
electrode was also attached to the glasses frame. For EEG
recording, the electrodes were placed on top of the hair
following the 10-20 international standard of EEG electrode
placement [28]. The Fz and Oz channels were adopted as
suggested by [29], [30]. Neither hair was removed, nor
coupling gel applied. The electrodes were fixed by a hairband.
For ECG recording, the electrode was attached on the chest of
drivers through cloth.
The study involved two experimental sessions in the same
day. Each session took 15-20 minutes. One was conducted in
the morning when the subjects were self-evaluated as alert, and
the other was in the afternoon after lunch when the subjects
were more possibly drowsy. The ECG, EEG signal, and eye
blinking activities were detected in the experiments.
In the experiment, the face of the driver was also recorded by
a video camera for calibration. The level of drowsiness was
estimated based on the Wierwille and Ellsworth criteria [31]
which were also adopted for driver drowsiness evaluation by
[29]. The continuous scale included five descriptors: Not
Drowsy (Class 1), Slightly Drowsy (Class 2), Moderately
Drowsy (Class 3), Very Drowsy (Class 4), and Extremely
Drowsy (Class 5) [31]. One-minute segment was evaluated by
the scale as one of the five descriptors. In the evaluation
process, two observers scored the drowsiness levels. The
observers strictly followed the description of drowsiness as
suggested by Wierwille and Ellsworth [31]. The full scale of the
five descriptors was adopted, which means that one very alert
segment was defined to be Class 1 and one very drowsy was
defined to be Class 5. Most segments could reach an agreement.
The disagreement occurred between adjacent classes in few
cases. For example, one observer evaluated to be Class 1 and
the other considered it as Class 2. This case did not affect the
results as Class 1 and 2 were both considered as alertness. Only
in two cases, the observers could not reach an agreement for
Class 3 and Class 4. These two segments were not used in
further analysis. To further avoid biasing judgment, we also
used the quantity to verify the observation results as [37]. In
the observation, the slow eyelid closures, rubbing the face or

2
0
4

wave-Detected Signal

2
0
0.0

0.2

0.4

0.6

0.8

Time (s)

2168-2194 (c) 2013 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See
http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

1.0

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/JBHI.2014.2305403, IEEE Journal of Biomedical and Health Informatics

> Ref. No. TITB-00489-2013.R3

5
HRV parameters including both time domain and frequency
domain measures can be computed for further analysis.

(c) wave-Typical signal

Voltage (mV)

4
2
0
4

wave-Detected signal

2
0
0.0

0.2

0.4
0.6
Time (s)

0.8

1.0

Fig.7. EEG bands compared with clinic signals. (a) alpha wave; (b) theta wave;
(c) delta wave. In each figure, the upper trace is the typical EEG band signal and
the lower one is that detected and computed by our system.

V. RESULT AND ANALYSIS

A. Vital Signal Measurement
1) Eye blinking detection
A typical eye blinking signal is shown in Fig. 6 (b). The
eye-blinking activities cause pulses in the bio-potential signals.
Using peak identification algorithm developed in this work, the
occurrence of eye blinking can be determined in real time.
Comparison study was conducted between the recorded signals
and the video record. When the eyes started to close, a positive
edge occurred immediately; whereas during reopening, a
negative edge followed. These formed a sharp pulse for one
blinking cycle. Several key parameters can be analyzed from
the measured eye blinking signal. In Fig. 6 (b), t1, t2 represent
the eye closing time and the reopening time; t1+t2 is the overall
blinking time; whereas t3 is the interval between blinking
activities. Drowsy drivers typically have problems to control
their eyes [13]. Physiologically this behaves as rapid blinking at
the on-set of drowsiness and slow blinking as the drivers are
deeply affected.
2) EEG detection
Based on the raw EEG data detected from the system, the
Fast Fourier Transform (FFT) algorithm was applied to
decompose the signal into four frequency bands, i.e., , , ,
and bands. The detected wave components were compared
with the standard clinic signals in order to evaluate the
performance of the device. Fig. 7 (a)-(c) compared , , and
waves, respectively. In each figure, the lower trace is the signal
obtained from the non-intrusive system whereas the upper one
is the typical reference signals detected by a clinic EEG device.
The patterns of the two signals are allied to each other, which
verify that the brain waves associated with EEG can be detected
by the system.
3) ECG detection
The non-intrusive ECG measurement was verified by
comparing with a clinic 12-lead ECG system. The clinic system
used Ag/AgCl wet electrodes and electrolyte attached directly
to the skin. Our electrode was placed right above the clinic
electrode outside the cloth to compare the performance. A
typical ECG signal detected from chest by our device is
displayed as the lower trace in Fig. 8; whereas that detected
from the clinic chest lead is shown as the upper trace. The two
signals were detected simultaneously. In the capacitive ECG
signal, the R-R intervals and the QRS complexes are
comparable to the clinic one. To detect HR and HRV
automatically, an algorithm was developed to pick the QRS
peak and calculate the HR and HRV in real time. The HR and

B. System Stability in Driving Scenarios

During normal driving, drivers stay still in most of time;
however, at some time, the movements and friction between the
electrode and human body can cause severe motion artifacts for
capacitive electrodes. Therefore, it is needed to analyze the
feasibility of applying the non-intrusive system in automobile
applications. From implementation perspective, however, it is
not necessary for a perfect signal at every second during driving
[1]. For example, for each one-minute segment, if the detected
signal can be stable for more than 15 seconds, this segment can
be considered as a successful one. The stability of the system is
defined as the number of successful segments over total number
of segments. The results from a typical subject are shown in
Table 1 (total 20 segments were analyzed).
From Table 1, it can be seen that the performance for HR and
HRV detection is reasonably stable, which qualifies the
function for long-term application. The performance for eye
activity detection is unsatisfactory when the electrode is
mounted on the roof. The performance, however, is
significantly improved when placed on the glasses frame. In the
data collection process, the electrodes made of soft metal taps
were placed/wrapped on the frame of the eyeglasses. They
could be bended and attached. It is also found that in highway
scenarios where drivers have less motion, the system
performance demonstrates high stability. Under a traffic
condition, where more motions are involved in vehicle
operation, the results are still acceptable in most cases. It is also
noticed during the experiments that when the drivers were
asked to reduce their motion, the recorded signals became more
stable.
C. Fatigue Detection Background
A few physiological parameters of drivers have been
observed to be well connected to drowsiness. In the
eye-activity-based assessment, one of the most widely used
parameters is the PERCLOS which refers to the duration of eye
closure over 80% of time [13]. Other parameters such as
reopening time [7], the intervals between eye blinks, and the
speed of blinking [8] were also studied by some researchers.
Clinic ECG

Q
S

ECG detected by our system

R

Q
S

2
Time (s)

Fig. 8. ECG signal detection comparing with clinic devices. Upper trace-ECG
signal detected by clinic devices; lower trace-ECG signal detected by the
proposed system.
TABLE I
STABILITY OF THE PERFORMANCE DURING DIFFERENT SCENARIOS
Scenarios
QRS
Eye blinking pulse
EEG
Roof
Glasses
Highway
100%
15%
90%
80%
Urban
95%
35%
75%
35%
Turning
95%
5%
85%
15%
Urban (Attention)
100%
75%
90%
85%

2168-2194 (c) 2013 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See
http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/JBHI.2014.2305403, IEEE Journal of Biomedical and Health Informatics

> Ref. No. TITB-00489-2013.R3

HRV

Time
domain
Frequency
domain

Variables
SDNN
RMSSD
LF
HF
LF/HF

TABLE II
SELECTED MEASURES OF HRV
Units
Description
ms
Standard deviation of all NN intervals.
ms
The square root of the mean of the sum of the squares of
differences between adjacent NN intervals.
ms2
Total spectral power or power spectral density (PSD) of all
NN intervals between 0.04 and 0.15 Hz.
ms2
Total spectral power or power spectral density (PSD) of all
NN intervals between 0.15 and 0.4 Hz.
Ratio of low to high frequency power.
TABLE III
RESULTS OF THE SELECTED MEASURES

Variables
Eye
activities

Blinking frequency
Blinking duration

Alertness Condition
7.0 1.6 times per minute
11.3 1.1ms

EEG

Power density of alpha wave

Power density of beta wave
SDNN
RMSSD
LF
HF
LF/HF

1306.51 799.48 ms2

1189.80 422.28 ms2
106.72 30.38 ms
28.67 9.40 ms
511.15 115.47 ms2
244.26 101.69 ms2
2.55 1.37

HRV

In EEG-based assessment, Lal et al. [9] developed an algorithm

to assess driver drowsiness based on the changes in EEG bands
(, , , and bands). Lin et al. [10] estimated the drowsiness
level by independent component analysis (ICA) of EEG and
found the optimal locations to place EEG electrodes. Jap et al.
[33] compared four algorithms based on the four EEG bands.
The result showed that a slight decrease of alpha activity and a
significant decrease of beta activity were associated with
fatigue. The ECG features are also extracted to analyze
sleepiness, including both time domain and frequency domain.
Yang et al. [34] classified sleep into wake, rapid eye movement
(REM) and non-REM stages using very low frequency (VLF),
low frequency (LF), high frequency (HF) and the LF/HF ratio
in HRV analysis. Roman et al. [8] used Detrended Fluctuation
Analysis and observed some scaling differences between sleep
deprived and non-deprived groups.
D. Feature Extraction
In this study, we also extracted and analyzed the features
from the three types of vital signals. For eye activities, the
blinking frequency (blinking times per min), blink duration
(t1+t2 in Fig. 6 (b)) were adopted as the measures of eye
blinking activities. For EEG signals, first, two digital bandpass
filters with the pass band ranging in 8-13 Hz and 13-30 Hz were
applied to the data to extract the alpha wave and beta wave from
the original signals. Then power density of the two waves was
calculated to compare the alertness and sleepiness conditions of
each subject. For ECG signal, HRV and its selected measures in
time domain and frequency domain were selected and
calculated as described in Table 2.
To differentiate driver states, Class 1,2 were marked as
alertness and Class 4,5 were marked as drowsiness in the data
analyses. One-way ANOVA was applied to test the
significance of each feature.

Results
Sleepiness Condition
8.2 3.0 times per minute
12.7 2.7 ms (Class 4)
16.0 2.8 ms (Class 5)
908.77 502.44 ms2
1106.33 425.95 ms2
97.04 45.45 ms
31.10 22.07 ms
606.67 162.70 ms2
568.33 312.05 ms2
1.01 1.55

p-value of one way ANOVA

0.234
0.110
0.000
0.159
0.635
0.546
0.729
0.111
0.002
0.017

E. Result and Discussion for Fatigue Detection Application

1) Eye blinking results
In the alertness condition, the average blinking frequency of
the twelve subjects was 7.0 1.6 times per minute; the average
blinking duration was around 11.3 ms. In the Class 4
drowsiness condition, two of the subjects blinked faster than in
the alertness condition; the rest were similar. The average
blinking frequency of the subjects was 8.2 3.0 times per
minute. In the Class 5 drowsiness condition, the eyelid began to
close slowly acting as the duration of each blinking activities
becoming much longer. The blinking duration shows the
significance between alertness and extremely drowsiness
(p<0.05). All the results are illustrated in Table 3.
2) EEG results
Two of the EEG results were not feasible for data analysis as
the electrodes slipped away from the hair in the experiment.
The rest ten groups of data were used for feature extraction.
Comparing the EEG signals in alertness and fatigue conditions
of each subject, the alpha wave decreased slightly while the
beta wave remained relatively stable. This result is analogous to
that in [33]. However, the ANOVA analysis did not show the
significant linkage of either alpha or beta waves with extent of
drowsiness.
3) HRV results
In the time domain, the SDNN and RMSSD (described in
Table 2) are two main statistical measures of HRV. From the
results, little change can be found from the SDNN and RMSSD,
which means that the HR did not show significant difference
between the alertness and sleepiness conditions. In the
frequency domain, the spectral power density of LF, HF and the
ratio of LF/HF were extracted from the HRV data. The power
density of LF component remained similar in the two
conditions, whereas the HF increased apparently from alertness
to drowsiness. The power density of HF and the ratio of LF/HF
are significant measures to distinguish alertness and
drowsiness. This result is similar to those observed in [12].

2168-2194 (c) 2013 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See
http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/JBHI.2014.2305403, IEEE Journal of Biomedical and Health Informatics

> Ref. No. TITB-00489-2013.R3

VI. DISCUSSION AND CONCLUSION
In this paper, we developed an innovative non-invasive
driver assistance system. The system features high sensitivity in
the measurement of bio-potentials on human body and requires
no physical contact with skin. This method causes less mental
or physical loads to the drivers and is advantageous for long
term driver monitoring purpose. The system can measure
physiological signals such as eye blinking activity, EEG and
ECG signals in the real time, which are widely accepted vital
signals for health monitoring and drowsiness measures. The
performance of the system was verified on a high-fidelity
driving simulator. Experiments were conducted on subjects
with different alertness and sleepiness conditions. The eye
activity, EEG and ECG features were recorded on the alert and
drowsy drivers using the non-intrusive system. Results showed
that the blinking duration, and the LF, HF components from
HRV are significant physiological measures between alertness
and drowsiness, which are consistent with other studies. Our
long term goal is to develop this technology into a robust
in-vehicle driver diagnosis and medical assistance system to
improve the health and safety of drivers.
REFERENCES
[1]

[2]

[3]
[4]

[5]

[6]
[7]

[8]

[9]

[10]

[11]

[12]

[13]

[14]

T. Wartzek, B. Eilebrecht, J. Lem, H.-J. Lindner, S. Leonhardt, and M.

Walter, ECG on the road: Robust and unobtrusive estimation of heart
rate, IEEE Trans. Biomed. Eng., vol. 58, pp. 31123120, 2011.
A. H. Taylor and L. Dorn, Stress, fatigue, health and risk of road traffic
accidents among professional drivers: the contribution of physical
inactivity, Annu. Rev. Public Health, vol. 27, pp. 371391, 2006.
L. Copeland, Study: Sleepiness a factor in 17% of road deaths, USA
Today, Nov.8, 2010.
A. M. Williamson, A. Feyer, and R. Friswell, The impact of work
practices on fatigue in long distance truck drivers, Accident Anal. Prev.,
vol. 28, pp.709719, 1996.
Y. Dong, Z. Hu, K. Uchimura, and N. Murayama, Driver inattention
monitoring system for intelligent vehicles: a review, IEEE Trans. Intell.
Transp., vol. 12, no. 2, pp. 596614, 2011.
S. K. L. Lal, and A. Craig, A critical review of the psychophysiology of
driver fatigue, Biol. Psychol., vol. 55, pp. 173194, 2001.
P. P. Caffier, U. Erdmann, and P. Ullsperger, Experimental evaluation of
eye-blink parameters as a drowsiness measure, Eur. J. Appl. Physiol.,
vol. 89, pp. 319325, 2003.
B. Roman, S. Pavel, P. Miroslav, V. Petr, and P. Lubomir, Fatigue
Indicators of drowsy drivers based on analysis of physiological signals,
in J. Crespo, V. Maojo, and F. Martin (Eds.): ISMDA 2001, LNCS 2199,
pp. 6268, 2001.
S. K. L. Lal, A. Craiga, P. Boorda, L. Kirkupb, and H. Nguyenc,
Development of an algorithm for an EEG-based driver fatigue
countermeasure. J. Saf. Res., vol. 34, pp. 321328, 2003.
C. T. Lin, R. C. Wu, S. F. Liang, W. H. Chao, Y. J. Chen, and T. P. Jung,
EEG-based drowsiness estimation for safety driving using independent
component analysis, IEEE Trans. Circuits Syst. I, Reg. Papers., vol. 52,
no. 12, pp. 27262738, 2005.
S. Redmond, and C. Heneghan, Electrocardiogram-based automatic
sleep staging in sleep disordered breathing, in Proc. IEEE Int. Computer
Cardiogram Conf., pp. 609612, 2003.
E. Michail, A. Kokonozi, and I. Chouvarda, EEG and HRV markers of
sleepiness and loss of control during car driving, in Proc. 30th Annual Int.
IEEE EMBS Conf., August 20-24, 2008, Vancouver, British Columbia,
Canada.
R. Grace, and S. Steward, Drowsy driver monitor and warning system,
in Proc. the 1st Int. Driving Symp. Human Factors Driver Assessment,
Training and Vehicle Design, Aspen, CO, Aug. 2001, pp. 6469.
Y. M. Chi, T.-P. Jung, and G. Cauwenberghs, Dry-contact and
noncontact biopotential electrodes: methodological review, Biomed.
Eng., IEEE Reviews, vol. 3, pp.106119, 2010.

7
[15] Y. G. Lim, K. K. Kim, and K. S. Park, ECG recording on a bed during
sleep without direct skin-contact, IEEE Trans Biomed Eng., vol. 54, pp.
718725, 2007.
[16] A. Aleksandrowicz, and S. Leonhardt, Wireless and non-contact ECG
measurement system-The Aachen SmartChair, Acta Polytechnica, vol. 2,
pp. 6871, 2007.
[17] O. Postolache, P. Giro, J. Joaquim, and G. Postolache, Unobstrusive
heart rate and respiration rate monitor embedded on a wheelchair, in
Proc. 4th IEEE Int. Work. Medical Meas. and Applica. Cetraro, pp. 83
88, 2009.
[18] Y. D. Lee, and W. Y. Chung, Wireless sensor network based wearable
smart shirt for ubiquitous health and activity monitoring, Sensor
Actuator B, vol. 140, pp. 390395, 2009.
[19] S. Leonhardt, and A. Aleksandrowicz, Non-contact ECG monitoring for
automotive application, in Proc 5th Int Summer School Symp Med.
Devices Biosensor (ISSS-MDBS), pp. 183185, 2008.
[20] T. Matsuda, and M. Makikawa, ECG monitoring of a car driver using
capacitively-coupled electrodes, in Proc. IEEE 30th Ann. Int. Conf. Eng.
Med. Biol. Soc. pp. 13151318, 2008.
[21] T. Maruyama, M. Makikawa, N. Shiozawa, and Y. Fujiwara, ECG
measurement using capacitive coupling electrodes for man-machine
emotional communication, IEEE/ICME Int. Conf. Complex Med. Eng.,
pp. 378383, 2007.
[22] M. Walter, B. Eilebrecht, T. Wartzek, and S. Leonhardt, The smart car
seat: personalized monitoring of vital signals in automotive applications,
Pers. Ubiquit. Comput., vol. 15, pp. 707715, 2011.
[23] C. J. Harland, T. D. Clark, and R. J. Prance, Electric potential probes new directions in the remote sensing of the human body, Meas. Sci.
Technol., vol. 13, pp. 163169, 2002.
[24] P. S. Luna-Lozano and R. Pallas-Areny, Power-line interference in the
ECG obtained with capacitive electrodes, Proc. 5th Eur. IFMBE Conf.,
pp. 910913, 2011.
[25] G. D. Gargiulo, A. L. McEwan, P. Bifulco, M. Cesarelli, C. Jin, J. Tapson,
A. Thiagalingam, and A. Van Schaik, Towards true unipolar
bio-potential recording: a preliminary result for ECG, Physiol. Meas.,
vol. 34, pp. N1N7, 2013.
[26] N. Meziane, J. G. Webster, M. Attari and A. J. Nimunkar, Dry electrodes
for electrocardiography, Physiol. Meas., vol. 34, pp. R47R69, 2013.
[27] T. V. Thakor and J. G. Webster, Ground-free ECG recording with two
electrodes, IEEE Trans. Biomed. Eng., vol. BME-27, no. 12, pp. 699
704, 1980.
[28] H. H. Jasper, The ten-twenty electrode system of the International
Federation, Electroencephalogr. Clin. Neurophysiol., vol. 10, pp. 371
375, 1958.
[29] R. N. Khushaba, S. Kodagoda, S. Lal, and G. Dissanayake, Driver
drowsiness
classification
using
fuzzy
wavelet-packet-based
feature-extraction algorithm, IEEE Trans. Biomed. Eng., vol. 58, pp.
121131, 2011.
[30] J. Santamaria and K. H. Chiappa, The EEG of Drowsiness, New York:
Demos Publication, 1978.
[31] W. W. Wierwille and L. A. Ellsworth, Evaluation of driver drowsiness
by trained raters, Accident Anal. Prevent., vol. 26, no. 5, pp. 571581,
1994.
[32] A. B. Barreto, S. D. Scargle, and M. Adjouadi, A practical EMG-based
human-computer interface for users with motor disabilities, J. Rehabil.
Res. Dev., vol. 37, pp. 5364, 2000.
[33] B. T. Jap, S. Lal, P. Fischer, and E. Bekiaris, Using EEG spectral
components to assess algorithms for detecting fatigue, Expert Syst.
Appl., vol. 36, pp. 23522359, 2009.
[34] G. Yang, Y. Lin, and P. Bhattacharya, A driver fatigue recognition
model based on information fusion and dynamic Bayesian network,
Inform. Sciences, vol. 180, pp.19421954, 2010.
[35] R. J. Prance, T. D. Clark, H. Prance, and A. Clippingdale, Non-contact
VLSI imaging using a scanning electric potential microscope, Meas. Sci.
Technol., vol. 9, pp. 1229-1235, 1998.
[36] Y. M. Chi and G. Cauwenberghs, Micropower non-contact EEG
electrode with active common-mode noise suppression and input
capacitance cancellation, 31st IEEE-EMBS, Minneapolis, MN, Sep.,
2009.
[37] J. A. Healey and R. W. Picard, Detecting stress during real-word driving
tasks using physiological sensors, IEEE Trans. Intell. Transport. Sys.,
vol. 6, no. 2, pp. 156166, 2005.

2168-2194 (c) 2013 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See
http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/JBHI.2014.2305403, IEEE Journal of Biomedical and Health Informatics

> Ref. No. TITB-00489-2013.R3

Ye Sun received the B.S. degree in Precise

Instrumentation of Measurement and Control
from Tianjin University, China, in 2009. She is
currently pursuing the Ph.D. degree in
Electrical Engineering at Case Western
Reserve University, Cleveland, OH.
She works as a research assistant at Case
Western Reserve University since 2009. Her
research interests include biomedical signal
measurement, body sensor network, personal
state monitoring in transportation safety.

Xiong (Bill) Yu (M03) received his B.S.

degrees in Hydraulic Engineering and Computer
Science from Tsinghua University, China in
1997, M.S. degree in civil engineering from
Tsinghua University in 2000, M.S. degree in
electrical engineering and computer science
from Purdue University in 2002, Ph.D. in Civil
Engineering from Purdue University in 2003.
He is currently an associate professor in the
Department of Civil Engineering with secondary
appointments in the Department of Electrical Engineering and
Computer Science, Case Western Reserve University, Cleveland, OH.
His recent research activities include sensors and materials for
sustainability, human health and environment. Prof. Yu is a member of
IEEE,
ASCE,
ASME,
and
SPIE.

2168-2194 (c) 2013 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See
http://www.ieee.org/publications_standards/publications/rights/index.html for more information.