Lecture 3

Closed loop properties of MPC

3-1

Closed loop properties of MPC

Review: infinite horizon cost

Infinite horizon predictive control with constraints
Closed loop stability
Constraint-checking horizon
Connection with constrained optimal control

3-2

Review: infinite horizon cost

Short prediction horizons cause poor performance and instability, so

? use an infinite horizon cost: J(xk , uk ) =

1
X
i=0

kxi|k k2Q + kui|k k2R

? keep optimization finite-dimensional by using dual mode predictions:

ui|k =

optimization variables

i = 0, . . . , N

1,

mode 1

Kxi|k

i = N, N + 1, . . .
3

mode 2

u0|k
6
7
uk = 4 ... 5
uN 1|k

mode 1:
mode 2:

ui|k = Kxi|k

uk optimized online
K chosen oine

3-3

Review: infinite horizon cost

B Cost for mode 2:

1
X

i=N

kxi|k k2Q + kui|k k2R = kxN |k k2P

P is the solution of the Lyapunov equation

P

(A + BK)T P (A + BK) = Q + K T RK

B Infinite horizon cost:

J(xk , uk ) =
=

3-4

N
X1

kxi|k k2Q + kui|k k2R + kxN |k k2P

i=0
uTk Huk

+ 2xTk F T uk + xTk Gxk

Review: MPC online optimization

B Unconstrained optimization: ru J(x, u ) = 2Hu + 2F x = 0, so
u (x) =

Fx

=) linear controller: uk = KMPC xk

KMPC = LQ-optimal if K = LQ-optimal (in mode 2)
B Constrained optimization:
u (x) = arg min
u

uT Hu + 2xT F T u

subject to Ac u bc + Bc x
=) nonlinear controller: uk = KMPC (xk )

3-5

Constrained MPC example

B Plant model:

xk+1 = Axk + Buk ,

1.1
A=
0

yk = Cxk

2
0
1
, B=
, C=
0.95
0.0787

1 uk 1

Constraints:

B MPC optimization (constraints applied only to mode 1 predictions):

minimize
u

subject to

N
X1
i=0

kxi|k k2Q + kui|k k2R + kxN |k k2P

1 ui|k 1, i = 0, . . . , N

Q = C T C, R = 0.01, N = 2

. . . performance? stability?
3-6

Constrained MPC example

Closed loop response for x0 = (0.8, 0.8)
1
0.5
u

0
0.5
1
0

20

40

20

40

60

80

100

60

80

100

60

80

100

60

80

100

50
0
y

50
100
150
200

sample, k

unstable
3-7

Constrained MPC example

Closed loop response for x0 = (0.5, 0.5)
1
0.5
u

0
0.5
1
0

20

40

20

40

4
2
0
2

sample, k

stable, but . . .
3-8

Constrained MPC example

Optimal predicted cost x0 = (0.5, 0.5)
90
80

Predicted cost J(k)

70
60
50
40
30
20
10
0

20

40

60

80

100

sample, k

. . . increasing Jk =) closed loop response does not follow predicted trajectory

3-9

Stability analysis
How can we guarantee the closed loop stability of MPC?
(a). Show that a Lyapunov function exists demonstrating stability
(b). Ensure that optimization feasible is at each time k = 0, 1, . . .
B For Lyapunov stability analysis:
? consider first the unconstrained problem
? use predicted cost as a trial Lyapunov function
B Guarantee feasibility of the MPC optimization recursively
by ensuring that feasibility at time k =) feasibility at k + 1

3 - 10

Discrete time Lyapunov stability

Consider the system xk+1 = f (xk ), with f (0) = 0
B Definition: x = 0 is a stable equilibrium point if
for all R > 0 there exists r such that
kx0 k < r =) kxk k < R for all k

B If continuously dierentiable V (x) exists with

(i). V (x) is positive definite and
(ii). V (xk+1 ) V (xk ) 0
then x = 0 is a stable equilibrium point

3 - 11

Discrete time Lyapunov stability

Consider the system xk+1 = f (xk ), with f (0) = 0
B Definition: x = 0 is an asymptotically stable equilibrium point if
(i). x = 0 is stable and
(ii). r exists such that kx0 k < r =) lim xk = 0
k!1

B If continuously dierentiable V (x) exists with

(i). V (x) is positive definite and
(ii). V (xk+1 ) V (xk ) < 0 whenever xk 6= 0
then x = 0 is an asymptotically stable equilibrium point

3 - 12

Lyapunov stability
Trial Lyapunov function:
J (xk ) = J(xk , uk )
where J(xk , uk ) =

1
X
i=0

kxi|k k2Q + kui|k k2R

? J (x) is positive definite if:

(a). R 0 and Q 0, or
(b). R 0 and Q 0 and (A, Q) is observable

since then J (xk )

0 and J (xk ) = 0 if and only if xk = 0

? J (x) is continuously dierentiable

. . . from analysis of MPC optimization as a multiparametric QP

3 - 13

Lyapunov stability
Construct a bound on J (xk+1 )

0 1

tail at k + 1

optimal at k

J (xk ) using the tail of the

optimal prediction at time k
Optimal predicted sequences at time k:
2 3
2 3
u0|k
x1|k
6 u 7
6 x 7
6 1|k 7
6 2|k 7
6 . 7
6 . 7
6 . 7
6 . 7
6 . 7
6 . 7

uk = 6
xk = 6 7
7
6uN 1|k 7
6 xN |k 7
6
7
6 7

6 KxN |k 7
6 xN |k 7
i
4
5
4
5
..
..
.
.
(

3 - 14

= A + BK)

Lyapunov stability
Construct a bound on J (xk+1 )

Tail sequences at time k + 1:

2
3
2
3
u1|k
x2|k
6
7
6 . 7
..
6
7
6 .. 7
.
6
7
6
7
6 u
7
6 x
7
6 N 1|k 7
6 N |k 7
k+1 = 6
k+1 = 6
u
7x
7
6 KxN |k 7
6 xN |k 7
6
7
6 2 7
6K xN |k 7
6 xN |k 7
i
4
5
4
5
..
..
.
.

tail at k + 1

0 1

J (xk ) using the tail of the

optimal prediction at time k

optimal at k

optimal at k :
tail at k + 1 :

J (xk ) =

J(xk , uk )

k+1 ) = J(xk+1 , u
k+1 )
J(x

1
X
i=0
1
X
i=1

= A + BK)

kxi|k k2Q + kui|k k2R

kxi|k k2Q + kui|k k2R

3 - 14

Lyapunov stability
Construct a bound on J (xk+1 )

J (xk ) using the tail of the

optimal prediction at time k

Predicted cost for the tail:

k+1 ) = J (xk )
J(x

kxk k2Q

kuk k2R

k+1 is suboptimal at time k + 1, so

but u
k+1 )
J (xk+1 ) J(x
Therefore
J (xk+1 ) J (xk )

3 - 15

kxk k2Q

kuk k2R

Lyapunov stability
The bound J (xk+1 )
1

J (xk )

kuk k2R implies:

the closed loop cost cannot exceed the initial predicted cost,
since summing both sides over all k 0 gives
1
X

k=0

kxk k2Q

kxk k2Q + kuk k2R J (x0 )

x = 0 is asymptotically stable
? if R 0 and Q
? if R

0, this follows from Lyapunovs direct method

0, Q 0 and (A, Q) observable, this follows from:

(a). stability of x = 0
(b).

lim

k!1

(kxk k2Q

kuk k2R )

( Lyapunovs direct method

P
2
2
=0 ( 1
k=0 kxk kQ + kuk kR < 1

3 - 16

Stability analysis
How can we guarantee the closed loop stability of MPC?

(b). Ensure that optimization feasible is at each time k = 0, 1, . . .

B Guarantee feasibility of the MPC optimization recursively

by ensuring that feasibility at time k =) feasibility at k + 1

3 - 17

Terminal constraint
The basic idea

more sophisticated (MPC-) controller required to satisfy constraints

@
@
?

@
R
@

terminal set
origin
OCC

C
stabilizing linear controller satisfies constraints
3 - 18

Terminal constraint
Terminal constraint: xN |k 2 , where = terminal set
xk

x1|k

safe region
for mode 2
control law

xN |k
xN

Choose so that:
(a). x 2

=)

(b). x 2

=)

1|k

xN +1|k

u Kx u
xxx

(A + BK)x 2

then is invariant for the mode 2 dynamics and constraints, so

u ui|k u
xN |k 2 =)
for i = N, N + 1, . . .
x xi|k x
i.e. constraints are satisfied over
the infinite mode 2 prediction horizon
3 - 19

Stability of constrained MPC

Prototype MPC algorithm
At each time k = 0, 1, . . .
(i). solve

uk = arg min J(xk , uk )

uk

s.t. u ui|k u, i = 0, . . . , N

x xi|k x, i = 1, . . . , N
xN |k 2

(ii). apply uk = u0|k to the system

Asymptotically stabilizes x = 0 with region of attraction FN ,
8
u ui u, i = 0, . . . , N
>
<
FN = x0 : 9 {u0 , . . . , uN 1 } such that x xi x, i = 1, . . . , N
>
:
xN 2

9
1>
=
>
;

= the set of all feasible initial conditions for N -step horizon

and terminal set

3 - 20

Terminal constraints
Make as large as possible so that the feasible set FN is maximized, i.e.
= X1 = lim Xj
j!1

where
? Xj = initial conditions for which constraints are satisfied for j steps
with u = Kx

i
u K(A + BK) x u
= x:
i = 0, . . . , j
x (A + BK)i x x
? X1 = X for some finite if |eig(A + BK)| < 1
+
x 2 X1 if constraints are satisfied on a finite constraint checking horizon
3 - 21

Terminal constraints Example

Plant model:

xk+1 = Axk + Buk ,

yk = Cxk

1.1
2
0
A=
B=
C= 1
0 0.95
0.0787

input constraints:
mode 2 feedback law:

1 uk 1

K = 1.19

7.88

= KLQ for Q = C T C, R = 1

3 - 22

Terminal constraints example

1u1

Constraints:

0.8
0.6

Kx = 1
0.4

K x=1

x2

0.2

x=1

x=

X2

0.2

K x=

0.4

Kx =
0.6
0.8
5

3 - 23

0
x1

Terminal constraints example

Constraints:

1u1
0.8
0.6
0.4

x2

0.2
0

X1

0.2
0.4
0.6
0.8
5

0
x1

X4 = X5 = = Xj for all j > 4 so X4 = X1

3 - 23

Terminal constraints example

In this example X1 is determined in a finite number of steps because
1
2

(A + BK) is strictly stable, and

(A + BK), K is observable

shortest distance of hyperplane

K(A + BK)i x 1 from origin

u
kK(A + BK)i k2

! 1 as i ! 1

) X1 is bounded because x0 2
/ X1 if x0 is sufficiently large

Here {x :

1 K(A + BK)i x 1} contains X4 for i > 4

+
X 1 = X4
constraint checking horizon: = 4

3 - 24

Terminal constraints
General case
Xj = {x : F

Let
then:

x 1, i = 0, . . . j} with

(i). X1 = X for finite

strictly stable
( , F ) observable

(ii). X = X1 i x 2 X+1 whenever x 2 X

Proof of (ii)
(a). for any j, Xj+1 = Xj \ x : F

j+1

x1

so Xj Xj+1 limj!1 Xj = X1
(b). if x 2 X+1 whenever x 2 X , then

x 2 X whenever x 2 X

but X x : F x 1 and it follows that X X1

(a) & (b) ) X = X1
3 - 25

C4A Model constraints

Predictive Control
Terminal
constraint checking horizon

3 - 28

Terminal constraint set

Algorithm for computing constraint checking horizon Nc
Algorithm
(computation
:
for
input constraints
u ofu ) u:
N := 0

umax := max K ( A + BK ) N +1 x s.t. u ! K ( A + BK )i x ! u , i = 0, N

x

umin := min K ( A + BK ) N +1 x s.t. u ! K ( A + BK )i x ! u , i = 0, N

x

N := N + 1

no

umax ! u ?
and
umin " u ?
yes

N c := N
3 - 26

2 linear programs
solved at each step

Constrained MPC
Define the terminal set as XNc
MPC algorithm
At each time k = 0, 1, . . .
(i). solve

uk = arg min J(xk , uk )

uk

s.t. u ui|k u, i = 0, . . . , N + Nc
x xi|k x, i = 1, . . . , N + Nc

(ii). apply uk = u0|k to the system

Note
? predictions for i = N, . . . N + Nc :

xi|k = (A + BK)i

ui|k = K(A + BK)i

xN |k
N

xN |k

? xN |k 2 XNc implies linear constraints so online optimization is a QP

3 - 27

Closed loop performance

Longer horizon N ensures improved predicted cost J (x0 )
and is likely (but not certain) to give better closed-loop performance
Example: Cost vs N for x0 = ( 7.5, 0.5)
N
J (x0 )
Jcl (x0 )

6
7
8
11 > 11
364.2 357.0 356.3 356.0 356.0
356.0 356.0 356.0 356.0 356.0
P
2
2
Closed loop cost: Jcl (x0 ) := 1
k=0 kxk kQ + kuk kR

For this initial condition:

MPC with N = 11 is identical to constrained LQ optimal control (N = 1)!

3 - 28

Closed loop performance example

Predicted and closed loop inputs for N = 6

predicted
closedloop

1
0.8
0.6
0.4

input, u

0.2
0
0.2
0.4
0.6
0.8
1
0

10

15
time step, k

20

25

30

3 - 29

Closed loop performance example

Predicted and closed loop states for N = 6
0.8
predicted
closedloop

0.7
0.6

x6|0 -

0.5
0.4
x2

0.3

F6

0.2
0.1
0
0.1

3 - 29

x1

Closed loop performance example

Predicted and closed loop states for N = 11
0.8
predicted
closed-loop

0.7
0.6
0.5
0.4
x2

0.3

x11|0

F6

0.2
0.1
0
-0.1

-8

-7

-6

-5

-4

x1

-3

-2

-1

3 - 29

Choice of mode 1 horizon performance

B For this x0 : N = 11 ) xN |0 lies in the interior of
m
terminal constraint is inactive
+
no reduction in cost for N > 11
B Constrained LQ optimal performance is always obtained with N
for some finite N1 dependent on x0

N1

B N1 may be large, implying high computational load

but closed loop performance is often close to optimal for N < N1
(due to receding horizon)
in this example Jcl (x0 ) optimal for N

3 - 30

Choice of mode 1 horizon region of attraction

Increasing N increases the feasible set FN
1.5
N = 26
N = 24
N = 20
N = 16
N = 12
N=8
N=4
N=0

x2

0.5

F12

0.5

F8
F4

1.5
10

0
x1

10

3 - 31

Summary

B Linear MPC ingredients:

? Infinite cost horizon

(via terminal cost)

? Terminal constraints

(via constraint-checking horizon)

B Constraints are satisfied over an infinite prediction horizon

B Closed-loop system is asymptotically stable
with region of attraction equal to the set of feasible initial conditions
B Ideal optimal performance if mode 1 horizon N is large enough
(but finite)

3 - 32