I J J projects vector
h
q in the null space of the Jacobian J(q). Since vector
h
q is
projected in the null space of J it does not violate the primary
objective (5). It means that it is possible to generate internal
motions which reconfigure the manipulator structure but do
not produce any velocity of the end effector [15]. This enables
us to exploit system redundancy to solve the additional con-
straint, which is in our case avoiding joint mechanical limits.
Vector
h
q can be chosen in the following way:
( )
0
T
h
k
ce | |
=
|
c
\ .
q
q
q
, (8)
where k
0
>0 and ( ) e q is the secondary objective function.
Objective function which maximizes the distance from joint
mechanical limits is defined with the following expression:
( )
1
1
2
n
i i
i iMAX iMIN
n
=
| |
e =
|
\ .
q q
q
q q
, (9)
where
iMAX
q ,
iMIN
q and
i
q are maximum, minimum joint
limit and middle value of the joint range, respectively. De-
sired angular positions and accelerations are obtained by inte-
grating and differentiating desired velocity, respectively. Once
the desired motion is defined it is necessary to calculate driv-
ing torques capable of realizing that motion.
IV. COMPUTED TORQUE CONTROL
Main idea of computed torque control is to cancel nonline-
arities in system by introducing a feedforward nonlinear con-
trol term calculated on the base of a known model of robot
dynamics. It is assumed that the system model is exactly
known, which is never the case in practice, due to complex
friction models, unmodeled nonlinearities and the assumption
of rigid body model. This is the main drawback of this control
approach. Feedforward nonlinear control is used in conjunc-
tion with a PD controller, which should achieve trajectory
tracking for linearized system and improve robustness of the
controller [4]. Block diagram of the computed torque control
is shown in Fig. 3.
Fig. 3 Computed torque control scheme
Control input u can be chosen in the following way:
( ) ( )
d d d d P V
+ +
0
u = H q q +h q ,q K e K e , (10)
where
d
q ,
d
q and
d
q are vectors of desired joint angular
positions, velocities and accelerations, K
P
and K
V
are matrices
of feedback gains and
d
= e q q is the vector of joint position
errors. Combining the expression for control input (10) with
the robot dynamic model (3) an error differential equation for
closed-loop system is obtained:
1 1
0
P V
+ + = e H K e H K e . (11)
Matrices K
P
and K
V
are diagonal Hurwitz gain matrices
whose elements are chosen to provide desired dynamic behav-
ior of the system. In this paper -SVR was used to estimate
the nonlinear feedforward control term in order to avoid the
dependency on knowing the accurate dynamic model. Inputs
for the regression are
d
q ,
d
q ,
d
q and the outputs are driving
torques at the joints. Equation (10) can now be written as:
( ) ,
d d d P V
f + + u = q , q q K e K e , (12)
where ( ) ,
d d d
f q , q q is approximated with support vector
regression method.
V. BASICS OF SUPPORT VECTOR REGRESSION
SVR is one of the numerous supervised learning algorithms
used for determining unknown dependencies between input
and output data. In the general regression learning problem,
learning machine is given l training data pairs (x
i
, y
i
), i=1...l,
where inputs x are n-dimensional vectors and outputs y are
real-valued functions. The result of the learning process is an
approximating function f
a
(x,w) which describes the input-
output relationship. Vector w is the matrix of weights which
are the subject of learning.
In every regression task it is necessary to define some
measure, or error of approximation, in order to quantify learn-
ing performance. Learning process is actually optimization
problem where it is necessary to minimize the expected error
R(w) given as:
( ) ( ) ( ) ( )
a
R L y, f , dP , y =
}
w x w x , (13)
where loss function L can be L
1
, L
2
, or any other norm. In our
work Vapniks -insensitivity loss function has been used.
This is the linear loss function with -insensitivity zone,
which defines the -tube around the output data, defined as:
( )
( )
( ) ( )
0
a
a
a
, y f ,
y f ,
y f , , y f ,
c
s c
=
c > c
x w
x w
x w x w
. (14)
Many function approximation algorithms minimize only the
empirical error given with:
( ) ( )
1
1
l
emp i a i
i
R L y , f ,
l
=
=
x w , (15)
that may lead to the problem of bad generalization. The prob-
lem is especially emphasized when training data set is small
compared to the number of different possible data at the input.
This problem may be overcome by using SRM (Structural
Risk Minimization), technique from statistical learning theory,
which minimizes not only the empirical error but also the
generalization error (elements of the weight matrix w). In case
of the linear regression problem input-output relationship can
be established as:
( )
T
f , b = + x w w x , (16)
where b is the bias. Structural error is minimized by minimiz-
ing the function of the following form:
( )
2
1
1
2
l
i a i
i
R C y f ,
=
= +
w x w . (17)
The parameter C is the penalty parameter which determines
the extent to which the empirical error is penalized relatively
to the penalization of the large values in the weighting matrix.
Robot arm is coupled and highly nonlinear system so it is
necessary to determine a nonlinear approximating function.
Nonlinear SVR is performed by mapping the data from the
original l-dimensional input space
l
R e x to transformed
higher dimensional feature space
f
R e z with a nonlinear
mapping function:
( )
l f
z x , R R = u , (18)
Now the learning problem can be solved as a linear regres-
sion problem in higher dimensional feature space. It is actual-
ly quadratic optimization problem with inequality constraints
in z space. Function is called kernel and it is chosen in ad-
vance and fixed for a given problem. In our paper, as a kernel
function, Radial Basis Function (RBF) has been used. The
solution for the hyperplane ( )
T
f b = + w z x , which is linear in
feature space, will create nonlinear regressing hypersurface in
original input space [8].
One of the main issues is how to form the training data set
properly. Training set should be small, but on the other hand
must be sufficiently dense. If the data set is sparse with a
small number of assigned target positions, underfitting may
arise. However, if the data set is too dense with a large num-
ber of assigned points overfitting may arise, which leads to
bad generalization error.
VI. SIMULATION RESULTS
A. The SVR training
In order to create training data set a set of 100 points, nor-
mally distributed in the robot arm workspace, was generated.
These points were generated by using direct kinematics, in
order to be sure that robot arm can reach them. The procedure
for creating the training set is the following:
1. Initial state of the system is defined (
0
q ,
0
q ,
0
q )
2. An arbitrary point (one of 100 generated points) in the
workspace is selected as the target position for robot
hand
3. Desired cruising speed for robot hand is defined
4. Systems motion is simulated. Inverse kinematics
problem is solved in each iteration to provide the de-
sired motion (
d
q ,
d
q ,
d
q ) of the hand along the
straight line between the initial and target position
and driving torques are calculated in each iteration
5. If the distance between the current and target position
is sufficiently small (predefined and constant value)
simulation is stopped and returned to step 2 while
current robot state becomes initial state for the
subsequent simulation
6. After robot hand goes through all 100 predefined
points in workspace simulation is finished
On every linear segment of trajectory, between two target
points, one hundred points are randomly chosen. Hence, the
complete training data set has 10.000 data pairs. In each of
those points vector of input and output data was stored in
memory to form the training set. Input vector
(
T
d
q ,
T
d
q ,
T
d
q )
T
has the dimension 12, while output vector
has the dimension 4.
Library for Support Vector Machines LIBSVM, with its
functions svmtrain and svmpredict, has been used both in
training and validation phase [16]. Obtained input and output
data were first normalized to a range [0, 1] in order to provide
equal contribution of all inputs which consist of different
physical units, namely angular positions, velocities and accel-
erations that have different ranges of possiblle values.
LIBSVM does not support multiregression. Hence, four
support vector machines had to be trained independently for
four joints of the arm. The inputs to the svmtrain function are
10.000 normalized input-output data pairs and the output is
the learned model which includes number of support vectors
and learned weight coeficients w. In order to provide good
generalization ability of SVRs it is necessary to test the
learned models on unseen input data. Therefore, another data
set with 10.000 data pairs, called test set, was generated the
same way as the training data set. These points are different
from the points in training set. There are three parameters that
have a significant influence on success of SVR training: the
insensitivity zone , penalty parameter C and width of the
RBF kernel . Experimentaly chosen values of these
parameters for 4 trained SVRs as well as MSE (Mean Square
Error) on a training set and test set are given in Table1.
TABLE I
PARAMETER VALUES USED FOR TRAINING SVR
Joint C
MSE
Training set
MSE
Test set
1. 10 0.01 0.4 4.3e-5 9.03e-5
2. 10 0.01 0.4 4.03e-5 8.8e-5
3. 100 0.005 0.8 1.07e-5 8.66e-5
4. 50 0.005 0.8 1.15e-5 9.28e-5
B. The SVR validation
Validation of the learned models was performed by simu-
lating robot arm movements. Driving torques were no more
calculated from robot dynamic model, but predicted using the
svmpredict function from LIBSVM library. This function
takes vector (
T
d
q ,
T
d
q ,
T
d
q )
T
as an input and returns predict-
ed driving torque of a corresponding joint and MSE of re-
gression as an output. Values of all inputs were saturated on
their boundary values, used for normalization in training
phase, to provide correct results of prediction. Function for
predicting driving torques is executed in every iteration of
simulation after the input vector is determined by the inverse
kinematics algorithm. The simulation in the validation phase
was performed in the same way as in the training phase with
the only difference in step 4, where driving torques are no
more calculated but, predicted using SVR.
Two different situations have been considered. First, robot
reaching task without external disturbance has been simulated.
In this case only feedforward control input (torque), obtained
using SVR, was applied on a system to run the simulation.
Stick diagram of simulated motion for reaching task without
disturbance is shown in Fig. 4.
Fig. 4 Stick diagram of robot model and illustration of robot reaching task for
two consecutive target positions; Green line shows hands real trajectory and
black dashed line shows hands desired trajectory
Two consecutive movements of robot arm are shown in Fig
4. Robot hand starts from its initial position and moves linear-
ly towards the first assigned target. When the hand comes
sufficiently close to the desired point, new target is assigned
and hand continues towards the new goal. It can be seen that
estimated driving torques are capable of achieving trajectory
tracking with satisfying accuracy and providing rectilinear
motion of robot hand from initial to target position, when
there are no external disturbances. In the second case robot
arm motion in the presence of external disturbance has been
considered. Disturbance was introduced to the system in the
simulation as the constant external force F
1
=10 N (equivalent
to the weight of 1 kg) acting on the left hand in negative di-
rection of Z-axis. It was generated as smooth Gaussian func-
tion with a constant value in a specific time range. Disturb-
ance acts on robot hand during the whole movement. Simula-
tion represents the situation when the robot is carrying a 1 kg
heavy object. Although the disturbance is constant force in
global coordinates, it generates time-varying moments in ro-
bot joints. Stick diagram of simulated motion for reaching
task in the presence of disturbance is shown in Fig. 5.
Fig. 5 Stick diagram of robot model and illustration of robot reaching task in
the presence of external disturbance; Green line shows hands real trajectory
and black dashed line shows hands desired trajectory
In the second simulation estimated feedforward control was
used in conjunction with a PD feedback control. The results
have shown that computed torque control with feedforward
part estimated with SVR can achieve positioning task in pres-
ence of disturbance, but not the accurate trajectory tracking. It
can be seen that simulated reaching task has no overshoots,
which is very important for manipulation tasks.
VII. CONCLUSION
Application of SVR for learning robot arm dynamic model
has been presented in this paper. Robot arm with 4 DOFs was
used for the reaching task, where only position of the robot
hand was considered. Therefore, the system has one redundant
DOF for the considered task. Redundancy resolution on ve-
locity level with avoiding of joint mechanical limits has been
included in problem solution. It was confirmed through nu-
merical simulations that -SVR is capable for learning nonlin-
ear feedforward part of computed torque control, i.e. for learn-
ing robot arm dynamic model. When there are no external
disturbances driving torques, which represent the output of
learning machine, are capable of achieving trajectory tracking.
In the presence of disturbances additional PD feedback con-
trol is needed and it is able to achieve only positioning task,
not an accurate trajectory tracking. The goal for future work is
to train SVR to learn driving torques for robot arm with 7
DOFs, when both desired position and orientation are defined.
Also it will be considered to include the model of flexible
robot spine with 6 DOFs and to design fuzzy controller for
decision making in motion planning phase. This controller
should choose the strategy for object reaching and decide
whether a robot should move only arm to reach the object or
synchronous motion of spine and arm is needed.
ACKNOWLEDGMENT
This work was funded by the Ministry of education and sci-
ence of the Republic of Serbia under contract III44008 and by
Provincial secretariat for science and technological develop-
ment under contract 114-451-2116/2011.
REFERENCES
[1] F. Abdessemed, SVM-Based Control System for a Robot Manipula-
tor, Int J Adv Robotic Sy, vol. 9, pp. 1-19, June, 2012.
[2] D. Nguyen-Tuong, J. Peters, M. Seeger, B. Schlkopf, Learning In-
verse Dynamics: a Comparison, Proc. European Symposium on Artifi-
cial Neural Networks-Advances in Computational Inteligence and
Learning (ESANN), Bruges, Belgium, pp. 13-18, 23-25 April, 2008.
[3] J. E. Slotine, W. Li, Applied nonlinear control, 1
st
ed., Englewood
Cliffs, New Jersey, Prentice Hall, 1991.
[4] D. Nguyen-Tuong, M. Seeger, J. Peters, Computed Torque Control
with Nonparametric Regression Models, Proc. American Control Con-
ference, Seattle, USA, pp. 212-217, 11-13 June, 2008.
[5] D. Nguyen-Tuong, J. Peters, Model Learning for Robot Control: A
Survey, Cognitive Processing, vol. 12, no. 4, pp. 319-340, Novemer
2011.
[6] M. Rakovi, B. Borovac, M. Nikoli, Support Vector Regression for
Approximation and Generation of Motion in Humanoid Robots, Inter-
national Journal of Industrial Engineering and Management, vol. 1, no.
1, pp. 1-8, 2010.
[7] M. Rakovi, M. NIkoli, B. Borovac, Humanoid Robot Reaching Task
Using Support Vector Machine, Proc. Research and Education in Ro-
botics-EUROBOT 2011, Prague, Czech Republic, vol. 161, pp. 263-
276, June 15-17, 2011.
[8] V. Kecman, Learning and Soft Computing Support Vector Machines,
Neural Networks and Fuzzy Logic Models, 1st ed. Cambridge, Massa-
chusetts: The MIT Press, 2001.
[9] B. Schlkopf, A. J. Smola, Learning with Kernels-Support Vector Ma-
chines, Regularization, Optimization and Beyond, 1st ed. Cambridge,
Massachusetts: The MIT Press, 2002.
[10] A. J. Smola, B. Schlkopf, A tutorial on support vector regression,
Statistics and Computing, vol. 14, no. 3, pp. 199-222, August, 2004.
[11] V. Potkonjak, M. Vukobratovi, K. Babkovi, B. Borovac, General
model of dynamics of human and humanoid motion: feasibility, poten-
tials and verification, Int. Jour. of Humanoid Robotics, vol. 3, no. 2,
pp. 21-48, 2006.
[12] B. Borovac, M. Rakovi, S. Savi, M. Nikoli, Dual Arm and Multi-
segment Spine Motion Control for Assistive Humanoid Robots, Proc.
14. International Workshop on Research and Education in Mechatronics
- REM, Vienna, Austria, pp. 235-241, 6-7 Jun, 2013, ISBN 978 390275
907-8
[13] J. Konczak, J. Dichgans, The development toward stereotypic arm
kinematics during reaching in the first 3 years of life, Exp Brain Res,
vol. 117, no. 2, pp. 346354, 1997.
[14] J. Konczak, M. Borutta, J. Dichgans, The development of goal-directed
reaching in infants, II. Learning to produce task-adequate patterns of
joint torque, Exp Brain Res, vol. 113, no. 3, pp. 465474, 1997.
[15] L. Sciavicco, B. Siciliano, Modelling and Control of Robot Manipula-
tors, 2
nd
ed. London, Great Britain, Springer-Verlag, 2005.
[16] C. C. Chang, C. J. Lin, LIBSVM: A Library for Support Vector Ma-
chines, ACM Trans. Intell. Syst. Technol., vol. 2, no. 3, May, 2011.