Design and Implementation of an Intelligent

Vehicle Driving Controller

Jau-Woei Perng, Yi-Hsuan Wen, Wei-Jen Chang, Min-Yan Ciou
Dept. Mechanical and Electro-Mechanical Engineering
National Sun Yat-sen University, NSYSU
Kaohsiung, Taiwan
jwperng@faculty.nsysu.edu.tw
AbstractThe goal of this paper is to implement the control
and design of an intelligent vehicle based on an embedded DSP
platform (eZdsp
TM
F2812). The overall system including steering
wheel AC serve motor, brake actuator, throttle driving circuit and
the sensors are equipped in a golf-car as a platform. Otherwise,
digital image processing technology is used to realize the
autonomous driving system which can achieve multi-mode of
lane-keeping, lane-change and obstacle-avoidance, the overall
system has been examined on campus roads.
Index TermsLane-keeping, Lane-change, DSP
I. INTRODUCTION
With the progress of science and technology, everyone is not
only focus on the convenience but also conscious of the vehicle
safety. Advanced Safety Vehicle (ASV) technology is currently
becoming a major area of research in the automotive industry
[1-2]. Therefore there are many relative research units and
automotive manufacturers for this autonomous driving research,
and Intelligent Transportation System (ITS) [3] is main thrust,
ITS can be broadly defined as the application of advanced
technologies, such as communications, sensors, and computing,
to the transportation system, supply the information of
real-time used to improve the safety, efficiency, and comforts
of transportation systems, and diminish the impact to
environment. In 1986, set up Prometheus project by 19
countries in Europe Prometheus, program for European traffic
with highest efficiency an unprecedented safety project, let the
vehicle safety technology becomes significant gradually and
higher priority.
According to statistics, a total of 13,939 fatal crashes
occurred during traffic in the United States from 1994 to 2005.
As a direct consequence of these accidents, 24,565 people died
(Fatality Analysis Reporting System, National Center for
Statistics and Analysis, U.S. Department of Transportation)[4].
Therefore, a set of services to provide automatic driving and
vehicular auxiliary functions of the system is one of important
research, that can effectively reduce the number of traffic
fatalities, and improve driver`s traffic safety. According to
reference [5] is safety support system for reducing the
efficiency of traffic accidents. The traffic safety system can be
divided into active and passive, and active driving safety
system for the ultimate goal is automatic driving.
In many of autonomous driving systems, that part of
lane-keeping use vision system to detect lane, and compute
some road information, such as offset and displacement of
angle, and sent the information to the controller operating the
steering wheel to ensure that the vehicles driving in the inside
lane. The lane detection methods mainly from the image
processing technology, reference [6-7] are all lane detection
based on processing technology.
The lane-change of main purpose is the pre-estimate of the
track switch, the command into the steering wheel and the
steering wheel motor control to reach lane-change function by
trajectory tracking control. In most parts of the road trajectory
prediction use fuzzy logic control [8-9] and yaw rate control
[10-11]. Currently, the world still lane-keeping and
lane-change mainly due to overtaking strategy for advanced
and complicated control and need to integrate the lane-keeping
and lane-change, for this strategy should consider the front of
the vehicle, that the relative distance and speed, and then
estimate the safe distance and proper speed for
overtaking[12-13]. When the successful change to another lane,
that still need to consider is successful beyond the front of the
vehicle or not, and deciding to change back to the original lane.
If at this time there are other vehicles or other factors influence,
will have to decide whether to allow overtaking strategy by
oneself, if that strategy can`t be executed then the
Car-following maneuver will instead of it, therefore overtaking
algorithm is more complex and difficult than lane-keeping and
lane-change[14-15].
The contribution of this paper is to develop an autonomous
driving system in digital signal processing (DSP) platform. The
overall system including steering wheel AC serve motor, brake
actuator, throttle driving circuit and the sensors is equipped in a
golf-car as a experimental car, and combine In-Vehicle sensors,
actuators and Vision system to reach autonomous driving
system, such as using a steering wheel AC motor in order to
keep the experimental car in the center of lane, Global Position
System (GPS) and laser range finder are used to detect velocity
and obstacle distance, and so on. In this paper, the major
functions of autonomous driving system include Lane-keeping,
Lane-change and obstacle-avoidance, and verify on the campus
roads.
Figure1: Overall structure of our developed system
Proceedings of 2011 8th Asian Control Conference (ASCC)

TuB5.5
Kaohsiung, Taiwan, May 15-18, 2011
- 1111 -
vel
and
be
sof
allo
ste
hu
cha
DS
F
sys
har
aut
A
veh
vel
the
lan
pri
com
CC
pro
and
the
wi
B
I
the
and
ins
ste
ang
dis
me
thr
Co
lin
fee
sen
(A
sig
tra
aut
The regulation
locity from the
d brake. Since
used for cont
ft-computing t
ows a human
eering wheel co
ge computatio
allenge that im
SP platform.
Figure1 show
stem in experi
rdware device
tonomous driv
A. In-vehicle S
The GPS and
hicle state sen
locity, accelera
en headway dis
ne is obtained b
imary subsyst
mposed of an
CD camera. Th
ovides the look
d displacement
e road, yaw an
dth.
B. Vehicle Con
In the autonom
e experimental
d so on. For s
stalled by the s
eering. The ang
gle sensor whic
splacement of
eter which is se
rottle pedal is in
onverter (DAC
near actuator w
edback informa
nsor are measu
ADC) encoding
gnal, voltage
ansducer, respe
tonomous driv
Figure2:
II. OVERALL
n layer is respo
e supervisory
the real golf-c
troller design,
technique, nam
reasoning me
ontroller into t
on load and rea
mplementing the
ws the structur
imental car, w
s and processi
ing in steering
Sensors and Su
d inertia meas
nsors, which a
ation, angular r
stances between
by the laser ran
tems include
n image proces
he vision system
k-ahead road in
t of angle relat
ngle with resp
ntrol Actuator
mous driving sy
platform, owi
steering wheel
side of the stee
gle of steering w
ch is set around
brake pedal i
et under the dri
nstead of interf
) and the brake
which is conn
ation of the ste
ured in terms o
g an analog vo
variation ou
ectively. Figure
ing system.
: Structure of the a
L STRUCTURE
onsible for exec
layer by the c
car model is to
it motivates a
mely fuzzy log
ethod to design
the regulation
al-time proces
e fuzzy logic c
re of the auto
which are supp
ing software i
, throttle, and b
ubsystem
surements unit
are used to sen
rate and angle
n the subject ve
nge finder. Be
the vision sy
ssor PC and a
m detects the la
nformation, suc
tes to maintain
pect to road ta
Installation
ystem, the golf
ng to it is chea
l control, an A
ering wheel to
wheel is measu
d the axis of ste
is measured by
iving seat. As t
face circuit usin
e pedal is auto
nected to the b
eering angle an
of an Analog D
oltage into a n
utput from a
e2 is shown th
autonomous drivin
cuting the desi
control of thro
oo complicated
an application
ic control, wh
n the human-l
layer. Due to
ssing, it will b
ontrol (FLC) o
onomous driv
ported by add-
in order to ful
brake control.
t (IMU) are t
nse the vehicl
respectively, a
ehicle in the sa
sides, the utiliz
ystem, which
a monochroma
ane-markings, a
ch as lateral off
n the centerline
angent, and ro
f-car is selected
aper, convenien
AC servomotor
enable automa
ured by a steer
eering wheel. T
y a displacem
o speed contro
ng Digital Ana
omated by usin
brake pedal. T
nd brake posit
Digital Conver
normalized dig
linear posit
he structure of
ng system
red
ttle
d to
n of
hich
like
the
e a
on a
ving
-on
lfill
two
les
and
ame
zed
is
atic
and
fset
e of
oad
d as
nce
r is
atic
ring
The
ment
ol, a
alog
ng a
The
tion
rter
ital
tion
the
C. V
The r
develop
platform
embedd
several
throttle-
commu
finder a
sensor a
commu
intellige
and spe
lane-kee
constan
records
about f
laser ran
D. A
In th
appropr
and ach
driving
signal o
of lane
comman
obstacle
the obst
is less t
lane-cha
There
Lane-ch
the syst
choice s
the lane
is main
obstacle
whether
the syst
obstacle
determi
the expe
continu
than 9 m
success
Vehicle Embedd
real-time softw
ped by C prog
m (eZdsp
TM
F2
ded system whi
sensors, and c
-, brake-, an
unicate with PC
and inertia mea
are through An
unicate with F
ence and beha
eed control fo
eping, lane-c
nt speed. The
the real-time
forward obstac
nge finder and
utonomous dri
is paper, fuzzy
riate steering a
hieve the lane-
in the center
of yaw rate and
e-change and
nd to achi
e-avoidance m
tacle in front of
than a safe di
ange and reach
Figure3: Flowc
e are three mai
hange, Lane-ke
tem is in the st
speed setting.
e keeping mode
ntained at th
e-avoidance m
r the obstructio
tem will choic
es in front of
ine whether a s
erimental car fo
ue to maintain th
meters and the
fully dodge ob
ded System
ware for auton
gramming lang
2812) which is
ich processes th
compute the co
nd SW- act
C and obtain the
surements unit
nalog Digital C
F2812. Beside
avior is also pr
or various dri
change, and
driver-vehicl
status of veh
cle and lane-d
the vision syst
iving system
y logic algorith
angle comman
-keep function
of line safely.
yaw angle is u
converted into
eve the lan
mode, laser rang
f the experimen
istance, the im
hed a obstacle-
chart of the autono
n modes of the
eeping, and Ob
tandby state, an
The autonomo
e. In lane-keepi
he driver pre
mode, at first, t
on is exist or no
ce lane-keepin
f experimental
safe distance o
for more than 9
he lane-keepin
system will pe
bstacle, as show
nomous driving
guage under th
s the core contr
he signals trans
ontrolling comm
tuators, throu
e information o
t. Brake sensor
Converter (ADC
es, the driver
rovided for ste
iving conditio
obstacle-avoi
le interface p
hicle, and the
eparture comin
tem, respective
hm is used to
nd, control ste
n, let experime
In lane-chang
used to predict t
o a steering w
ne-change fu
ge finder is us
ntal care, when
mplementation
avoidance func
omous driving syst
e system operat
bstacle-avoidan
nd waiting for
ous system def
ing mode, the v
e-set one. W
the system wi
ot, if no obstruc
ng mode, if th
l car and the
or not. If this o
meters and the
ng mode, if the
erform twice lan
wn in Figure3.
g function is
he embedded
roller of this
smitted from
mands to the
ugh RS-232
of laser range
r and steering
C) module to
r-compatible
eering wheel
ons, such as
dance with
provides and
information
ng from the
ely.
compute the
eering wheel
ental car can
ge mode, the
the trajectory
wheel angle
unction. In
sed to detect
n the distance
of the twice
ction.
tem
tion, namely,
nce. Initially
the driver to
fault mode is
vehicle speed
While choice
ill determine
ction in front,
here are any
system will
obstacle with
e system will
distance less
ne-change to
- 1112 -
vis
vis
lat
tha
pur
app
lan
A.
I
obt
ang
and
CC
int
the
is i
sho
Th
of
me
In
env
str
dis
on
red
in
III. AU
In this paper
sion system an
sion system is
eral offset, dis
at supply for t
rpose of fuzz
propriate steer
ne-keeping safe
Vision syste
In this paper,
tain road infor
gle as well as
d CCD camera
CD camera is
terface commun
e front of expe
including binar
own in Figure4
Figure
he original ima
image is 320*2
eters, as shown
the binary filte
vironment con
ength compar
stinguish lane a
e-third of the
duce the compu
Figure 7.
UTONOMOUS DR
of autonomou
nd fuzzy logic c
s to obtain the
splacement of
the fuzzy logic
zy logic contr
ring angle com
ely.
em[16]
the main purp
rmation, such a
road width. T
a, PC is used
s capture the
nicate with PC
rimental car. T
ry filter, ROI, e
4.
4: The main flowc
ges are capture
240, the effecti
n in Figure5.
Figure5: The orig
er, we choose t
ndition, becaus
red to the roa
and road and t
original image
uting time of b
RIVING SYSTEM
us driving sys
controller. The
e information
angle as well
c controller as
roller is used
mmand that im
ose of vision s
as lateral offset
The vision syst
to settle imag
image date t
C, the CCD cam
The image proc
edge detection,
chart of the road d
ed by CCD cam
ive region of ca
ginal image of road
threshold value
se of the lane
ad, so, after B
through ROI, c
e as the ROI re
behind image p
MDESIGN
stem is includ
e main purpose
of road, such
as width of ro
input. The m
to compute
mplementation
system is used
t, displacement
tem including
ge processing a
through the P
mera is installed
cess of flow ch
lane detection
etection
mera and the s
amera is about
d
e depend on lo
is more obvio
Binary filter c
choice the mid
egion, in order
process, as sho
ding
e of
as
oad,
main
the
of
d to
t of
PC
and
PCI
d in
hart
n, as
size
t 10
ocal
ous
can
ddle
r to
own
In the
lane edg
equation
If
Where
respecti
Figure7
In the
and sear
these co
of road,
B. Fuz
1. Lan
A fu
experts
the diffe
One o
the cho
vision s
lateral o
output
inferenc
implica
Where A
The fu
defuzzif
(CAD).
Figure6: I
e edge detectio
ge. Assume f
n as follow:
, / 2
0, 0
i H j W
i j
If
= =
= =
_
then
else
H and W
ively. The lan
7.
Figure7: Image
e lane detectio
rch the left and
onditions, the v
, as showed in
F
zzy controller
ne-keeping
zzy controller
procedural kn
ferent linguistic
of the most imp
ice of inputs a
system is used
offset and disp
is steering ang
ce method (mi
ation. The equa

M A B
R

=
A and B are x
uzzy logic
fication that m
The equation
Image segmentatio
on, the main of
( , ) f i j is the im
2
( , ) ( f i j f +
( , f i W
( , ) f i j
is the Heigh
nd of edge is
e segmentation ba
n, first of all, s
d right from mi
vision system
Figure 9.
Figure8: The Fina
r[17-18]
consists of a
nowledge and a
c values that th
portant issues i
and outputs of
to get the fuzz
placement of a
gle command.
inminmax)
ation as follow:
( )
( )
, min
=
B
A
x y
x

and y belong t
control outpu
method is cent
as follow:
on based on ROI
f purpose is to d
mage intensity
, / 2 ) i W j =
/ 2 ) 25 W j =
0 =
ht and width
s detected, as
sed on edge detect
search the botto
iddle of every
can obtain the
l image
rule bases co
a variable base
hey consider.
in fuzzy contro
f the system. In
zy controller in
angle, the fuzz
In this paper,
is used to solv
:
( ) ( )
) ( )
,
A B
B
x y
y

.
o the fuzzy set
ut is obtain
ter average de
detect the
then the
255 =
55
(1)
h of image,
s showed in
tion
om of image
pixel. Under
e information
ontaining the
es containing
ller design is
n this paper,
nput that are
zy controller
, Mamdanis
ve the fuzzy
(

(2)
t.
ned through
fuzzification
- 1113 -
( )
( )
1
1
m
l l
l
m
l
l
p h B
y
h B
- =
=
'
=
'
_
_
(3)
Where represents the membership function resulting
from the inference of the rules, and
( )
l
h B'
i
p is the membership
function for the different values of the output variable of the
rules. This method is very commonly used in control
applications.
Triangular and trapezoid membership functions are chosen
for inputs and outputs, as shown in Figure 10. These are simpler
easier to computation and tune.
(a)
(b)
(c)
Figure9: (a)(b)(c)Membership functions of fuzzy logic controller
Fuzzy if-then rules are based on human-like and experience
of experts to establish. For example, If lateral offset to NB
(vehicle in the left of center line), displacement of angle to N
(the angle between the centerline of the vehicle is left),
according to the human intuition and expertise that steering
wheel command should be NB (steering wheel to rotate
clockwise substantially), etc..According to the above described,
the fuzzy table is built, as shown in Figure 11.
Figure10: Rule base of fuzzy logic controller
Under these conditions, fuzzy logic controller can infer the
appropriate steering wheel angel command and make the
experimental car driving in the center of lane.
2. Lane-change and Obstacle-avoidance
Figure11: The trajectory of lane-change.
At least lane-change and obstacle-avoidance section, inertial
measurement unit is used to search the information of the driver,
such as yaw rate, yaw angle, acceleration and so on. According
to above information, we can simulate the lane-change
trajectory of driver and convert to the corresponding trajectory
of the steering wheel rotation angle. The information of driver
and implementation of lane-change is similar like sine and
cosine waves, as shown in Figure11. The figure is the
trajectories of twice lane-change where red block is the
experimental vehicle implementation of left lane-change results
and green block is the experimental vehicle implementation of
right lane-change results. According to above, we use actuator
to control the steering wheel to reach purpose of lane-change
and obstacle-avoidance. When implementation of lane-change
is an open-loop system, in order to increase the stability and
safety of the autonomous driving system, once the experimental
car switched to the adjacent lane, the system will immediately
redetect the land mark and add the road information to enhance
the stability.
In the detection of obstacles, the laser range finder is used to
scan the obstruction, in front of the center of vehicle 85 to 95
degrees as the obstruction area, when the distance is less than a
safe distance, the system will implement twice lane-change to
automatically dodge obstacles to ensure the smooth flow of
traffic.
IV. EXPERIMENTAL RESULTS
In this section, we present a set of experiments showing the
behavior of the autonomous driving system. The system is
implemented based on an embedded DSP platform and the
overall system has been equipped in our golf-car platform. The
experimental car is equipped with several sensors, such as laser
range finder, steering sensor, brake sensor, inertia
measurements unit (IMU) and so on, and throttle/brake
actuators, as shown in Figure12. The system has been examined
successfully on the NSYSU campus road. The image
processing interface including lateral offset, displacement of
angel, width of road, and so on.
- 1114 -
Figure12: Hardware diagram
A. Lane-keeping at Constant Speed
The image is data of lane-keeping, the data sequence of
displacement of angle, lateral offset, obstacle distance, vehicle
speed, steering angle, as shown in Figure13. The experimental
result of the vision-based lane-keeping at 5 km/h on the
NSYSU campus road. The lateral offset and displacement of
angle vibrating about zero presents that the experimental car is
automatically driving in the centerline of the road and operated
the throttle circuit to keep constant speed.
Figure13: The information of lane-keeping.
B. Lane-change at Constant Speed
Figure14: The information of left lane-keeping
Figure14 shows the information of left lane-change, before
and after the implementation of left lane-change that the offset
and angle vibrating about zero presents that the experimental
car is automatically driving in the centerline of the road. The
whole chart is lane-keeping mode to left lane-change mode and
back to lane-keeping mode. While implement of left
lane-change, red frame grabbed, and the steering angle is
formation of cosine wave, the amplitude and cycle is about 90
degree and 8 seconds, respectively. When the experimental car
had moved to the adjacent lane, the vision system re-caught the
lane mark and thus the system returned to the lane-keeping
mode. Experiment of lane-change mode at the constant speed 5
km/h.
Figure15 shows information of left lane-change, the
experimental result of the vision-based right lane-change at
constant speed about 5 km/h. Before and after implementation
of right lane-change, the lateral offset and displacement of
angle vibrating about zero presents that the experimental car is
automatically driving in the centerline of the road. While
implement of left lane-change, red frame grabbed, and the
steering angle is formation of sine wave, the amplitude and
cycle is about 90 degree and 8 seconds, respectively.
- 1115 -
F
V. CONCLUSIONS
The main contribution of this paper is to develop DSP
platform automatic driving system, a combination of security,
automation, el nd the entire
sy
ectronic, humanization features, a
stem is installed in the golf-car. The embedded system not
only provides lane keeping, lane change, and
obstacle-avoidance in front of automatic driving function, in
addition, also consider the issue of increasing importance
attached to comfort, therefore fuzzy logic algorithm is used to
simulate the human-like behavior of the driver, in order to
increase the stability of vehicle systems, when switch to the
adjacent lane, once the vision system detects lane that is
immediately into the lane keeping mode in order to ensure the
safety. At least the overall system has been examined on
campus roads.
REFERENCE
[1] G. Leen, an
Computers
Figure15: The information of right lane-change
C. Obstacle-avoidance at Constant Speed
Figure16 shows information of obstacle-avoidance. Before
and after the implementation of obstacle-avoidance that the
lateral offset and displacement o angle vibrating about zero
presents that the experimental car is driving in the centerline of
the road. While implementation of obstacle-avoidance, red
frame grabbed, the forward there is obstacle and the distance is
less than 9 meters, the system will implementation
obstacle-avoidance mode. In this time, the information of the
road, such as lateral offset, displacement of angle, will be lost
because of the fact that the vision system can`t detect lane mark
when implementation of the lane-to-lane transition.
igure16: The information of obstacle-avoidance
d D. Heffernan, Expanding automotive electronic systems,
, pp. 88-93, 2002.
zuka, Advanced Vehicle Control Systems AVCSResearch
mated Highway Systems in California Path, IEEE Vehicle
Systems for Intelligent Transportation
ing Maneuver,
[6]
[7]
[8]
[2] M. Tomi
for Auto
Navigation and Information Systems, pp. 41-45, 1994.
[3] I. Masaki, Machine-Vision
Systems, IEEE Intelligent systems, vol. 13, no. 6, pp. 24-31, 1998.
[4] J. E. Naranjo, C. Gonzalez, R. Garcia, and T. dePedro, Lane-Change
Fuzzy Control in Autonomous Vehicles for the Overtak
IEEE Transactions on Intelligent Transportation Systems, vol. 9, no. 3,
pp. 438-450, 2008.
[5] A. Eidehall, J. Pohl, F. Gustafsson, and J. Ekmark, Toward
Autonomous Collision Avoidance by Steering, IEEE Transactions
on Intelligent Transportation Systems, vol. 8, no. 1, pp. 84-94, 2007
D. Pomerleau, Visibility estimation from a moving vehicle using the
RALPH vision system, IEEE conference on Intelligent Transportation
Systems, pp. 906-911, 1997.
D. Pomerleau, RALPH: Rapidly adapting lateral position handling,
IEEE Proceedings of Intelligent Vehicle Symposium, pp. 506-511,
1995.
J. Mar, and L. Hung-Ta, The car-following and lane-changing collision
prevention system based on the cascaded fuzzy inference system,
IEEE Transactions on Vehicular Technology, vol. 54, no. 3, pp. 910-924,
2005.
[9] K. R. S. Kodagoda, W. S. Wijesoma, and E. K. Teoh, Fuzzy Speed and
Steering Control of an AGV, IEEE Transactions on Control Systems
Technology, vol. 10, no. 1, 2002.
[10] M. Canale, L. Fagiano, A. Ferrara, and C. Vecchio, Vehicle Yaw
Control via Second-Order Sliding-Mode Technique, IEEE
Transactions on Industrial Electronics, vol. 55, no. 11, pp. 3908-3916,
2008.
[11] K. Jeongmin, and K. Hyunsoo, Electric Vehicle Yaw Rate Control
using Independent In-Wheel Motor, IEEE Power Conversion
Conference, vol. 2, no. 5, pp. 705-710, 2007.
[12] G. Usman, and F. Kunwar, Autonomous vehicle overtaking- an online
solution, Automation and Logistics, IEEE International Conference on
vol. 5, no. 7, pp. 596-601, 2009.
[13]
ystem, International Conference on
[15]
[16]
lgorithm (TFALDA) for Autonomous Driving, Intelligent
sis with a Fuzzy Logic Controller, International Journal of
H. Jin-ying, P. Hong-xia, Y. Xi-wang, and L. Jing-da, Fuzzy Controller
Design of Autonomy Overtaking S
Intelligent Engineering Systems, vol. 25, no. 29, pp. 281-285, 2008.
[14] W. Y. He, DSP-based Cooperative Urban Cruise Control and
Stop-and-go with Intersection Safety Maneuver, MS Thesis, National
Chiao Tung University, Hsinchu, Taiwan, Republic of China, July, 2007
Y. T. Wu, An Embedded DSP Lane-keeping, Lane-change, and
Overtaking Control System in Autonomous Vehicles, MS Thesis,
National Chiao Tung University, Hsinchu, Taiwan, Republic of China,
July, 2007.
Y. U. Young, and O. S. Young, Three-Feature Based Automatic Lane
Detection A
Transportation Systems, vol. 4, no. 4, 2003.
[17] L. A. Zadeh, Fuzzy Sets, Information and Control, vol. 8, pp. 338-353,
1965.
[18] E. H. Mamdani, and S. Assitian, An Experiment in Linguistic
Synthe
Man-Machine Studies, vol. 7, no. 1, pp. 1-13, 1975.
- 1116 -