SICE Annual Conference 2010

August 18-21, 2010, The Grand Hotel, Taipei, Taiwan

Optimal Control Designs for Systems

with Input Saturations and Rate Limiters
Yuto Yuasa Noboru Sakamoto Yoshio Umemura
Department of Aerospace Engineering Department of Aerospace Engineering Scientific Analysis Engineering Department
Nagoya University Nagoya University AISIN AW CO., LTD
Nagoya, Japan Nagoya, Japan Anjo, Japan
Email: yuasa@suzu.nuae.nagoya-u.ac.jp Email: sakamoto@nuae.nagoya-u.ac.jp Email: i26409_umemura@aisin-aw.co.jp

Abstract—Most systems in practice have some input satura- II. O PTIMAL C ONTROL FOR S YSTEMS WITH I NPUT
tions due to physical restrictions. But taking input saturations S ATURATIONS
into account directly is not easy in ordinary control design
methods. In this report, we show that the method recently A. Problem Definition
developed by one of the authors is useful to approximate the exact
solutions of Hamilton-Jacobi equations for systems including We consider nonlinear optimal regulation problems to mini-
input saturations. First, we propose a method to design nonlinear mize quadratic form cost functions for nonlinear time-invariant
optimal controllers for systems with input saturations by solving systems with state nonlinearities and input saturations. The
Hamilton-Jacobi equations. Next, it will be extend to a design
method of nonlinear optimal controllers for systems with rate state equation Σ and the cost function J are given as (1) with
limited actuators. Two numerical examples are illustrated for R being defined as a diagonal matrix.
the input saturation and input rate limited problems. ⎧
Keywords—Nonlinear Optimal Control; Saturation; Rate Lim- ⎨ Σ : ẋ= f (x) + g(x) · sat(u) , x(0) = x0
∞  , (1)
iter; Hamilton-Jacobi Equation
⎩ J= xT Qx + uT Ru dt
0

where
I. I NTRODUCTION
Q ≥ 0 , R > 0 , x ∈ Rn , u ∈ Rm ,
Most systems in practice have some input saturations due f (·) : Rn → Rn , g(·) : Rn → Rn×m ,
T
to physical restrictions. But taking input saturations into ac- sat(u) = [sat1 (u1 ) , · · · , satm (um )]
count directly is not easy in ordinary control design methods.
The effect of input saturations sometimes causes undesired and
phenomenon such as windup etc. Rate limiter is one sort of ⎧
⎨ ui (ui ≤ ui )
input saturation known as a main reason of oscillation phe-
sati (ui ) = ui (ui < ui < ui ) ,
nomenon such as aircraft PIO(Pilot-Induced Oscillation)[5]. ⎩
ui (ui ≤ ui )
Anti-windup controller and MPC (Model Predictive Control)
are well-known as general way to solve these problems. But ui ≤ 0 , u i ≥ 0 , (i = 1, 2, · · · , m).
they are not practical because of the difficulty of calculation
or the online calculation load[2][3]. An alternative approach B. Calculation Algorithm
is to solve Hamilton-Jacobi equations derived from an optimal In this subsection, we derive a Hamilton-Jacobi equations
control problem including input saturations. A method using and solve it by the method in [1]. First the dynamic program-
neural network is proposed in reference[4]. But it is less accu- ming is applied with the Hamiltonian
rate and little theoretical rigor. Recently a solution method to
Hamilton-Jacobi equations for systems with state nonlinearity H(x, p, u) = pT (f (x) + g(x) · sat(u))
is proposed[1]. In this report, we show that this method is + xT Qx + uT Ru,
also useful to approximate the exact solutions of Hamilton-
Jacobi equations for systems including input saturations. First, where p is the co-state. To minimize H(x, p, u) with u, we
we propose a method to design nonlinear optimal controllers have to minimize the terms of u,
for systems with input saturations by solving Hamilton-Jacobi
equations. Next, it will be extend to a design method of uT Ru + pT g(x) · sat(u) =
nonlinear optimal controllers for systems with rate limited m

actuators. Two numerical examples are illustrated for the input ri ui 2 + pT gi (x) · sati (ui ) ,
saturation and input rate limited problems. i=1

- 2042 - PR0001/10/0000-2042 ¥400 © 2010 SICE

  T
where g(x) = g1 (x) g2 (x) . . . gn (x) . For each i (i = C. A Numerical Example
1, 2, · · · , n),
The proposed method is applied to a numerical example.
ri ui 2 + pT gi (x) · sati (ui )
⎧ The state equation is given as
⎨ ri ui 2 + pT gi (x)ui (ui ≤ ui )
= ri ui 2 + pT gi (x)ui (ui < ui < ui ) . (2)        
⎩ x˙1 −1 0 x1 1 1 sat1 (u1 )
ri ui 2 + pT gi (x)ui (ui ≤ ui ) x˙2
=
0 1 x2
+
1 1 sat2 (u2 )
.
Defining u∗i (x, p) as ui minimizing (2), u∗i (x, p) is calculated
as The maximum and minimum values of saturations are u1 =
⎧
⎨ ui (ui ≤ ûi (x, p)) −1, u1 = 1, u2 = −100, u2 = 100 and the weighting matrices
u∗i (x, p) = ûi (x, p) (ui < ûi (x, p) < ui ) are Q = I2×2 , R = I2×2 . After applying the stable manfold
⎩
ui (ûi (x, p) ≤ ui ) algorithm 25 times, the co-state p(x) is approximated by 3-
∗
⇔ ui (x, p) = sati (ûi (x, p)) , order polynomials. The simulation is calculated, where the
initial state is x(0) = [−64 −341]T . The system and the input
where ûi (x, p) = − 12 R−1 gi (x)T p. Thus the optimal input responses are shown in Fig. 1. As a comparison, the system
u∗ (x, p) is obtained as and the input responses by the linear optimal controller are
shown in Fig. 2. In Fig. 1, it is observed that when actuator
u∗ (x, p) = sat (û(x, p)) , (3)
where û(x, p) = − 12 R−1 g(x)T p.
Then the Hamilto-Jacobi equation to be solved is 100
T   50
∂V ∂V
f (x) + g(x) · sat û x, 0
∂x ∂x
T −50
∂V
x1 , x2 , u1 , u2

+ xT Qx + sat û x, −100
∂x
 −150
∂V
· R · sat û x, = 0.
∂x −200

−250 x1
If a solution V (x) of this equation can be obtained, the optimal x2
feedback u∗ (x) can be constructed by substituting p(x) = −300 u1
u2
(∂V /∂x)(x) in (3). −350
0 1 2 3 4 5 6 7 8
Since the soltions of Hamilton-Jacobi equations and the t
solutions of Hamilton’s canonical equations are equivalent, we
Fig. 1. Response by the nonlinear controller
consider the Hamilton’s canonical equations. Defining a new

function sat(u) = sat(u) − u to bring out the linear terms of 100
the canonical equation, the new Hamiltonian H  is 50
 T T
H (x, p) = p f (x) + x Qx 0

1 −50
− (g(x)T p)T R−1 g(x)T p
x1 , x2 , u1 , u2

4 −100
 (û(x, p))T R · sat
+ sat  (û(x, p)) . −150

Then the canonical equation is derived by −200

⎧ x1
⎪ ∂H  (x, p) −250
⎨ ẋ = −300
x2
u1
∂p . (4)
 u2
⎪
⎩ ṗ = − ∂H (x, p) −350
0 1 2 3 4 5 6 7 8
∂x t

The solutions x(t), p(t) of this equation can be calculated by

the iterative calculation of the stable manifold theorem[1]. The
co-state p(x) in the optimal feedback (3) can be calculated
by deleting the parameter t from the solutions x(t), p(t)
of the canonical equations. The saturation functions are ap-
proximated by C 2 class functions so that the right sides
of the canonical equations are C 1 class functions. In this
approximation, the smoothed parts are small enough not to
affect the whole calculations.
Fig. 2. Response by the linear controller
u1 is saturated, actuator u2 in the nonlinear controller takes
larger value than that of the linear controller to compensate
the saturation of u1 . Comparing the system responses, one
observes that the states of Fig. 1 converge more rapidly. The
cost function value J of the nonlinear controller is J = 5.202,
while the value of the linear controller is J = 5.412.

- 2043 -
 III. O PTIMAL C ONTROL FOR S YSTEMS WITH R ATE Now, the same caltulation procedure as in Sec. II, except
L IMITED ACTUATOR for the two variable function sat(x, u), can be applied. The
A. Problem Definition and Approximation of Rate Limiter optimal input u∗ (x) is given as
We consider nonlinear optimal regulation problems to mini- u∗ (x ) = V −1 sat(V us , û(x , p(x ))), (9)
mize quadratic form cost functions for nonlinear time-invariant
systems with state nonlinearities and rate limiters. The state where û(x , p) = − 12 V −1 R−1 B T p , B = [0 1]T and p is the
equation Σ and the cost function J are given as (5) with R co-state. The feedback thus constructed uses the augmented
being a diagonal matrix. The parameters and the functions are state us . In actual systems with rate limited actuators, such
defined as well as (1). The rate limiter functions are described as aircraft longitudinal dynamics, the measurement of the
as RL(u). saturated input (the elavator angle in the aircraft model) is
⎧ available.
⎨ Σ : ẋ= f (x) + g(x) · RL(u) , x(0) = x0
∞ C. A Numerical Example
, (5)
⎩ J= L(x(t), u(t)) dt
0 The proposed method is applied to a numerical example.
The state equation is given as
where       
x˙1 0 1 x1 0
L(x, u) = xT Qx + uT Ru , Q > 0 , R > 0. = + RL(u).
x˙2 −16 0 x2 1
In general, rate limiter is difficult to handle analytically The maximum and minimum values of the rate limiter are
and various approximation methods are proposed[5][6]. The u = −2, u = 2. Constants V, Q, R are V = 20, R =
most common way of approximating rate limiters is to use I2×2 , Q = 0.01. After applying the iterative stable manifold
the integrater-feedback block in Fig. 3 (see, eg., [2]). This algorithm 30 times, the co-state p(x) is approximated by 3-
order polynomials. The simulation is calculated, where the
sat
u us initial state is x(0) = [0.03 − 0.257]T , us (0) = 0. The system
+ 1
V s system and the input responses by the nonlinear controller are shown
-
in Fig. 4. As a comparison, the system and input responses by
linear optimal controllers for the augmented system (8) and the
original system (5) are shown in Figs. 5 and 6, respectively.
Fig. 3. Rate Limiter Approximation Block As a comparative controller, the response and the input by
the linear optimal controller for (8) are shown in Fig. 5, the
block is written as response and the input by the linear optimal controller for (5)
u̇s = sat(V (u − us )). (6) are shown in Fig. 6.
In all figures, one sees that rates of inputs are restricted
By going through the block, the rate of inputs u by a by 2. Particularly, in Fig. 6, the gap between controller
controller is limited, yielding actual inputs us to the system. input and actual input to the system is more evident. In
The maximum and minimum values of the rate limiter are general, controllers based on the augmented system have better
u, u. V = diag[v1 · · · vm ] is a constant matrix and the bigger performance (see, Figs. 4 and 5). This difference may be the
these values are, the better the approximation accuracy is. effect of the additional us feedbacks. If the measurement of us
is available, it is expected that the controllers designed for the
B. Calculation Algorithm
augmented systems improve the responses. Comparing Fig. 4
We first apply the dynamic programming for the augmented and Fig. 5, the nonlinear controller achieves faster stabilization
system (5) and (6) by rewriting (6) as with less oscillation and smaller input. Since rate limiters
induce phase lag as are well-known, the nonlinear controller,
u̇s = −V us + sat(V us , V u). (7)
considering the saturation in the rate limiter, switches the input
The function sat(x, u) means a saturation function which toward the negative direction before the linear controller does.
maximum and minimum limits are u+x, u+x. The augmented This lag between the nonlinear and linear controller inputs may
system is written as yield the difference in performance. In these simulations, cost
      function values are obtained as J = 2.4977 for the nonlinear
x˙1 f (x1 ) + g(x1 )us 0
= + sat(V us , V u). (8) controller, J = 3.8277 for the linear controller with augmented
u˙s −V us 1
state and J = 13.677 for the linear controller for the original
For this state equation, the cost function J is redefined as system.
 ∞
J= (x (t)T Qa x (t) + u(t)T Ru(t)) dt, IV. C ONCLUSIONS
0   In this report, first, we proposed an optimal control design
 T Q 0 method for systems with input saturations and then, extended
x = [x1 us ] , Qa = .
0 0 it to systems with rate limited actuator. The designs are

- 2044 -
 0.6 [2] P. Hippe: Windup in control: Its effects and their prevention, Verlag
x1 Springer, London, (2006)
0.4
x2 [3] D. Q. Mayne, J. B. Rawlings, C. V. Rao and P. O. M.Scakaert: Con-
us strained model predictive control: Stability and optimality: Automatica,
u Vol.36, pp.789-814, (2000)
0.2
[4] M. Abu-Khalaf, F. L. Lewis: Nearly Optimal control laws for nonlinear
systems with saturating actuators using a neural network HJB approach:
x1,x2,u1,u2

0 Automatica, Vol.41, pp.779-791, (2005)

[5] D. H. klyde, D. G. Mitchell: Investigating the role of rate limiting in
−0.2 pilot-induced oscillations: Jounal of Guidance, Control, and Dynamics,
Vol. 27, No. 5, pp.804-813, (2004)
−0.4 [6] K. Yamada, I. Jikuya, N. Sakamoto, F. Goto: Pilot-induced oscillation
analysis with rate limiter: Transactions of the Japan Society of Mechan-
−0.6 ical Engineers, Vol. 73, No. 728, pp.1059-1066, (2007)

−0.8
0 1 2 3 4 5
t

Fig. 4. Response by the nonlinear controller

0.8
x1
0.6 x2
us
0.4 u

0.2
x1,x2,u1,u2

−0.2

−0.4

−0.6

−0.8
0 1 2 3 4 5
t

Fig. 5. Response by the linear controller for the augmented system

3
x1
x2
2 us
u

1
x1,x2,u1,u2

−1

−2

−3
0 1 2 3 4 5
t

Fig. 6. Response by the linear controller for the original system

based on the stable manifold approach to approximate the

solution of Hamilton-Jacobi equations by iterative calculations.
The effectiveness of the proposed methods is shown by two
numerical examples.
R EFERENCES
[1] N. Sakamoto and A. J. van der Schaft: Analytical approximation
methods for the stabilizing solution of the Hamilton-Jacobi equation:
IEEE Transactions on Automatic Control, Vol.53, No.10, pp.2335-2350
(2008)

- 2045 -