Proceedings in Applied Mathematics and Mechanics, 30 October 2007

Numerical Solution of Differential Riccati Equations Arising in Optimal

Control for Parabolic PDEs

Hermann Mena1, and Peter Benner1,
1
Mathematik in Industrie und Technik
Fakultat fur Mathematik
TU Chemnitz
D-09107 Chemnitz

The numerical treatment of linear-quadratic regulator problems on finite time horizons for parabolic partial differential equa-
tions requires the solution of large-scale differential Riccati equations (DREs). Typically the coefficient matrices of the
resulting DRE have a given structure (e.g. sparse, symmetric or low rank). Here we discuss numerical methods for solving
DREs capable of exploiting this structure. These methods are based on a matrix-valued implementation of the BDF methods.
The crucial question of suitable stepsize and order selection strategies is also addressed.
Copyright line will be provided by the publisher

1 Introduction
Differential Riccati equations (DREs) arise in several applications, especially in control theory. Here, we are mainly interested
in constrained optimization problems for parabolic partial differential equations (PDEs) that can be formulated as abstract
Cauchy problems. Imposing a quadratic cost functional, we obtain a linear quadratic regulator (LQR) problem for an infinite-
dimensional system. The optimal control is then given in feedback form in terms of the solution of an operator DRE on finite
time horizons (see, e.g., [1]). In [2] we derive an approximation result enabling us to solve the problem numerically. Hence,
large-scale DREs resulting from the semi-discretization of the corresponding PDEs in space have to be solved. Moreover,
nonlinear control problems solved by model predictive control techniques require the solution of DREs with time-varying
coefficients [2]. Hence, we consider time-varying symmetric DREs of the form

X(t) = Q(t) + X(t)A(t) + AT (t)X(t) X(t)S(t)X(t), X(t0 ) = X0 , (1)

where t [t0 , tf ] and Q(t), A(t), S(t) are piecewise continuous, locally bounded n n matrix-valued functions. Moreover,
in most control problems, fast and slow modes are present. This implies that the associated DRE will be fairly stiff which in
turn demands for implicit methods to solve such DREs numerically. Therefore, we will focus here on the stiff case. In the
next section, we summarize a numerical method capable of exploiting the given structure of the resulting DRE. Then, we test
our method on an numerical example in Section 3. Finally, we state some conclusions and remarks in Section 4.

2 Solving large-scale DREs

In the literature there is a large variety of approaches to compute the solution of the DRE (1) (see, e.g., [35]). However,
none of these methods is suitable for large-scale control problems, since the computational effort grows at best like n3 , where
n is the dimension of the state of the control system. We developed an efficient matrix valued versions of the backward
differentiation formulae (BDF) methods suitable for large-scale DREs. Low rank approximations and perturbations are the
key ingredients to establish this task. We briefly summarize this method here. Details can be found in [2, 6].
Solving the DRE using BDF methods requires the solution of an algebraic Riccati equation (ARE) in every step. Since the
ARE is a nonlinear matrix equation, it is natural to apply Newtons method or variants of it (see, e.g., [7]). The application
of Newtons method yields a Lyapunov equation, F T X + XF = W W T , to be solved in every step. Here F is assumed
to be stable (i.e., its eigenvalues lie in the open left complex half plane). The Lyapunov equation is solved by the low rank
version of the alternating direction implicit (ADI) algorithm, see [8, 9]. We note that F can be represented as the sum of a
sparse matrix and a low rank perturbation. This implies the usage of the Sherman-Morrison-Woodbury formula for solving
linear systems to increase the efficiency.
The convergence of Newtons method is locally quadratic whereas, for the ADI algorithm it is superlinear if the set of
shift parameters is chosen appropriately. The selection of the shift parameters leads to a rational min-max-problem. Since the
knowledge about the shape of the complex spectrum is crucial for computing the optimal solution, this is infeasible for the

E-mail: hermann.mena@mathematik.tu-chemnitz.de, Phone: +49 371 531 36726, Fax: +49 371 531 22509
E-mail: benner@mathematik.tu-chemnitz.de Phone: +49 371 531 22540, Fax: +49 371 531 22509

Copyright line will be provided by the publisher

2 PAMM header will be provided by the publisher

8 8
x 10 x 10
0.35 1
BDF123 BDF123
0.9
0.3
5
0.8
0.25 0.7
4
0.6

step size
0.2

error
X11

3 0.5
0.15
0.4
2
0.1 0.3

0.2
1 BDF3 0.05
BDF123 0.1

0 0 0
0 5 10 15 0 5 10 15 2 4 6 8 10
time time step size 4
x 10

Fig. 1 left: solution component x11 center: step size vs. time, right: error vs step size.

large-scale problems. In fact, it could be more expensive than the iteration itself. Therefore, we proposed a new procedure
which has a good performance in practice [10].
Variable step-size BDF methods for DREs as well as order selection strategies are derived in [2]. The corresponding
computations are all performed directly in terms of the low rank factors of the solution. At certain stages of the corresponding
procedure, square-roots of negative numbers may enter the computations which leads to complex solution factors. Using a
split representation of these factors, it is possible to do all computations in real arithmetics to keep the memory requirements
and computational effort low.

3 Numerical example
We consider a linear-quadratic control problem of a one-dimensional heat flow proposed in [11]. In order to deal with a
time-varying DRE we modify the example to use piecewise constant coefficients in the PDE, see [2]. In Figure 1 (left) we can
visualize the behavior of the fixed step size BDF solver of order three (BDF3) as well as for the variable step size and order
BDF solver up to order three (BDF123). In the center we see that the step size is drastically reduced every time that it passes
through a discontinuity point. On the other hand, the error (right) tends to be constant.

4 Conclusions and remarks

We conclude that the BDF schemes allow an efficient implementation for the large-scale DREs considered here. Moreover,
step size control and order selection strategies can be implemented in terms of the low rank factors avoiding complex arith-
metics. In [2] we discuss the numerical solution of optimal control problems for instationary heat, convection-diffusion and
diffusion-reaction equations formulating them as abstract LQR problems. We also study optimal control problems with non-
linear PDEs. Tracking and stabilization type control problems are treated with model predictive control techniques where we
need to solve linearized problems on small time frames.
Among the one step methods for solving ordinary differential equations (ODEs) that can deal with stiffness efficiently,
Rosenbrock methods (Runge-Kutta type methods) are commonly used. Therefore, analogous to the BDF methods we also
develop efficient matrix valued versions of these ODE methods suitable for large-scale DREs in [2].

References
[1] A. Bensoussan, G. Da Prato, M. Delfour, and S. Mitter, Representation and Control of Infinite Dimensional Systems, 2nd edition
(Birkhauser, Boston, MA, 2007).
[2] H. Mena, Numerical Solution of Differential Riccati Equations Arising in Optimal Control Problems for Parabolic Partial Differential
Equations, PhD thesis, Escuela Politecnica Nacional, Quito, Ecuador, 2007.
[3] C. Choi and A. Laub, IEEE Trans. Automat. Control 35, 770776 (1990).
[4] E. Davison and M. Maki, IEEE Trans. Automat. Control 18, 7173 (1973).
[5] L. Dieci, SIAM J. Numer. Anal. 29(3), 781815 (1992).
[6] P. Benner and H. Mena, BDF methods for large-scale differential Riccati equations, in: Proc. of Mathematical Theory of Network
and Systems, MTNS 2004, edited by B. D. Moor, B. Motmans, J. Willems, P. V. Dooren, and V. Blondel (2004).
[7] P. Benner, Supplemento ai Rendiconti del Circolo Matematico di Palermo, Serie II No. 58, 2156 (1999).
[8] T. Penzl, SIAM J. Sci. Comput. 21(4), 14011418 (2000).
[9] J. Li and J. White, SIAM J. Matrix Anal. Appl. 24(1), 260280 (2002).
[10] P. Benner, H. Mena, and J. Saak, On the parameter selection problem in the Newton-ADI iteration for large-
scale Riccati equations, Chemnitz Scientific Computing Preprints 06-03, TU Chemnitz, October 2006, Available at
http://www.tu-chemnitz.de/mathematik/csc/2006/csc06-03.pdf.
[11] J. Abels and P. Benner, CAREX a collection of benchmark examples for continuous-time algebraic Riccati equations (version 2.0),
SLICOT Working Note 1999-14, November 1999, Available from http://www.slicot.org.

Copyright line will be provided by the publisher