Anda di halaman 1dari 45

# ECE 5700

## Di i l C l S Digital Control Systems

Chapter 10 p
Dr Bradley J Bazuin Dr. Bradley J. Bazuin
Western Michigan University
College of Engineering and Applied Sciences
Department of Electrical and Computer Engineering
1903 W. Michigan Ave.
Kalamazoo MI, 49008-5329
Linear Quadratic Optimal Control Q p
Quadratic Cost Function Q
The Principle of Optimality
Linear Quadratic Optimal Control
The Minimum Principle
Least-Squares Curve Fitting Least Squares Curve Fitting
Least-Squares System Identification
Recursive-Least-Squares System Identification
Optimal State Estimation Kalman Filters
ECE 5700
Notes and figures are based on or taken from materials in the course textbook: C.L. Phillips and H.T Nagle,
Digital Control System Analysis and Design, Prentice Hall, 1995, ISBN: 0-13-309832.
2
Root Locus and Frequency Response design q y p g
methodologies are considered classical or traditional
approaches.
Both are effective Both are effective
Both have a great deal of trial and error, where experience and
practice are invaluable
Pole Assignment is a modern method, where poles are
selected based on desired or required performance.
Are any of the resulting systems optimal? Are any of the resulting systems optimal?
If so, in what regard can they be considered optimal.
Prove it .
ECE 5700 Notes and figures are based on or taken from materials in the course textbook: C.L. Phillips and H.T
Nagle, Digital Control System Analysis and Design, Prentice Hall, 1995, ISBN: 0-13-309832.
3
Cost Functions
Assume that we can write a mathematical function to
capture the system cost in a cost function.
( ) ( ) ( ) [ ]

=
N
N
k u k r k y L J , , , L
A function determined be actual, estimated, or guessed factors.
Often described in terms of inputs and outputs
= k 0
Once a function is defined function minimization can be
( ) ( ) ( ) ( )

=
+ =
N
k
T T
N
k u R k u k y k y J
0
Once a function is defined, function minimization can be
performed and an optimal solution can be achieved.
ECE 5700 Notes and figures are based on or taken from materials in the course textbook: C.L. Phillips and H.T
Nagle, Digital Control System Analysis and Design, Prentice Hall, 1995, ISBN: 0-13-309832.
4
Input Constraints p
System cost is often dependent upon the required inputs y p p q p
needed to drive the plant.
Finite power, finite voltages, finite control energy
( ) ( )
i i
U k u
( )
i i
M k u
2
( ) ( ) ( )

=

N
k
T
N
k u k R k u E
0
The final equation is a quadratic form, where R(k) may be a
sample-time varying weight function.
Any control sequence that satisfies the condition may be called an
= k 0
ECE 5700 Notes and figures are based on or taken from materials in the course textbook: C.L. Phillips and H.T
Nagle, Digital Control System Analysis and Design, Prentice Hall, 1995, ISBN: 0-13-309832.
5
Chapter Design Techniques p g q
Optimal linear regulator control systems with quadratic p g y q
cost functions
Using:
( ) ( ) k x K k u = ( ) ( ) k q K k u =
or
System Identification least squares system identification
yielding a transfer function that best fits the available
data.
K l fil i i l h i f Kalman filtering an optimal technique for state
estimation.
ECE 5700 Notes and figures are based on or taken from materials in the course textbook: C.L. Phillips and H.T
Nagle, Digital Control System Analysis and Design, Prentice Hall, 1995, ISBN: 0-13-309832.
6
Quadratic Cost Function Q
Based on state-space, but we allow time-varying plants p , y g p
( ) ( ) ( ) ( ) ( ) k u k B k x k A k x + = +1
( ) ( ) ( ) k x k C k y =
The quadratic Cost Function is described as
( ) ( ) ( ) ( ) ( ) ( )

+ =
N
T T
k u k R k u k x k Q k x J
Q(k) and R(k) are symmetric, positive definite matrices
( ) ( ) ( ) ( ) ( ) ( )

=
+ =
k
N
k u k R k u k x k Q k x J
0
ECE 5700 Notes and figures are based on or taken from materials in the course textbook: C.L. Phillips and H.T
Nagle, Digital Control System Analysis and Design, Prentice Hall, 1995, ISBN: 0-13-309832.
7
Output/State Term (1) p ( )
The first term is related to the system output or states. y p
For example
( ) ( ) ( ) k x k C k y =
( ) ( ) ( ) ( ) ( ) ( ) k x Q k x k x C C k x k y k y
T
T
T T
= =
A quadratic polynomial in x(k)
C C Q
T
=
( ) ( ) ( ) ( ) [ ]
( )
( )

= =
k x
k x
q q
q q
k x k x k x Q k x F
T
2
1
22 21
12 11
2 1
( ) ( ) ( ) ( ) ( )
2 2
k x q k x k x q q k x q F + + + ( ) ( ) ( ) ( ) ( )
2 22 2 1 21 12 1 11
k x q k x k x q q k x q F + + + =
ECE 5700 Notes and figures are based on or taken from materials in the course textbook: C.L. Phillips and H.T
Nagle, Digital Control System Analysis and Design, Prentice Hall, 1995, ISBN: 0-13-309832.
8
Output/State Term (2) p ( )
With no loss of generality (and for convenience) Q g y ( ) Q
symmetric.
( ) ( ) ( ) ( ) [ ]
( )
( )

= =
k x q q
k x k x k x Q k x F
T 1 12 11
2 1
For F positive definite
( ) ( ) ( ) ( ) [ ]
( )

k x q q
Q
2 22 12
2 1
( ) ( ) ( ) ( )
2
2 22 2 1 12
2
1 11
2 k x q k x k x q k x q F + + =
For F positive definite,
0 , 0
0 , 0
= =

x F
x F
Minimizing F will minimize the desired combination of the
states.
ECE 5700 Notes and figures are based on or taken from materials in the course textbook: C.L. Phillips and H.T
Nagle, Digital Control System Analysis and Design, Prentice Hall, 1995, ISBN: 0-13-309832.
9
Input Related Quadratic Cost Function p Q
A similar argument to the input/states exists for g p
u R u G
T
=
Where G is positive semi-definite
0 0
0 , 0
= =

u G
u G
0 , 0 = = u G
ECE 5700 Notes and figures are based on or taken from materials in the course textbook: C.L. Phillips and H.T
Nagle, Digital Control System Analysis and Design, Prentice Hall, 1995, ISBN: 0-13-309832.
10
Cost Function
By selected Q(k) and R(k) minimization will effect inputs y Q( ) ( ) p
and states in differing ways
( ) ( ) ( ) ( ) ( ) ( )

+ =
N
T T
N
k u k R k u k x k Q k x J
R(k) = 0, only state dependent
Q(k) diagonal, only state power a concern
( ) ( ) ( ) ( ) ( ) ( )

= k
N
Q
0
Q(k) diagonal, only state power a concern
Q11 large all other small, state x1 minimization focus
ECE 5700 Notes and figures are based on or taken from materials in the course textbook: C.L. Phillips and H.T
Nagle, Digital Control System Analysis and Design, Prentice Hall, 1995, ISBN: 0-13-309832.
11
Principle of Optimality (1) p p y ( )
Developed by R. Bellman: p y
If a closed-loop control u
0
(k)=f[x(k)] is optimal over the interval
0<=k<=N, it is also optimal over any subinterval m<=k<=N, where
0<=m<=N.
Applying the principle:
( ) ( ) ( ) ( ) ( ) ( ) k u k R k u k x k Q k x F
T T
+ =
Mi i i b l b i N
( ) ( ) ( ) ( ) ( ) ( ) k u k R k u k x k Q k x F
k
+ =
N N
F F F J + + + = L
1 0
Minimize by element or subset starting at N

+ =
+ +
= + + + = =
N
m N i
i N m N m N m N N m
F F F F J J S
1
2 1
L
ECE 5700 Notes and figures are based on or taken from materials in the course textbook: C.L. Phillips and H.T
Nagle, Digital Control System Analysis and Design, Prentice Hall, 1995, ISBN: 0-13-309832.
12
Principle of Optimality (2) p p y ( )
Optimize starting with S1 p g

+ =
+ +
= + + + = =
N
m N i
i N m N m N m N N m
F F F F J J S
1
2 1
L
0
i i
N

## And work backwards

0
1 1
S F F S
optimize
N
N i
i
= =

=
Each successive S uses the previously optimized S
0
i-1
0
1 1 1
1
2
S F F F F S
N N N
N
N i
i
+ = + = =

=

p y p
i 1
Based on dynamic programming
ECE 5700 Notes and figures are based on or taken from materials in the course textbook: C.L. Phillips and H.T
Nagle, Digital Control System Analysis and Design, Prentice Hall, 1995, ISBN: 0-13-309832.
13
What is optimization? p
Take partial derivatives with respect to the specified p p p
variable; and set the result equal to zero.
Example: N=2
( ) ( ) ( ) k u k x k x + = + 2 1 ( ) ( ) ( )
( ) ( ) ( ) ( )

= =
+ = =
2
0
2
0
2
1 1
k
T T
k
i
k u k u k x k x F J
( ) ( ) ( ) ( ) ( ) ( )
2 2 2 2 2 2
2 1 0 2
2 2 1 1 0 0 u x u x u x F F F J + + + + + = + + =
( ) ( )
2 2
1 2 1
2 2 u x J J S + = =
( )
[ ] ( ) 0 2 2
2
1
= =

u S
u
( )
2 0
1
2 x S =
( ) ( ) ( ) ( ) ( )
( ) ( ) [ ] ( ) ( ) ( ) ( ) ( ) ( )
2 2 2 2 2
2 2 2 2 2 0
1 0 2 2
1 2 1 1 4 1 5 1 1 1 1 2
1 1 2 1 1
u x u x u x u x
u x x u x S J J S
+ + = + + + =
+ + = + + = =
ECE 5700 Notes and figures are based on or taken from materials in the course textbook: C.L. Phillips and H.T
Nagle, Digital Control System Analysis and Design, Prentice Hall, 1995, ISBN: 0-13-309832.
14
Example Continued (2) p ( )
Take partial derivatives with respect to the specified p p p
variable; and set the result equal to zero.
( ) 0 2 = u

( ) ( ) ( ) ( )
2 2
2
1 2 1 1 4 1 5 u x u x S + + =
( ) ( )
( )
2 0
1
2 x S =
( )
( ) ( ) 0 1 4 1 4
1
2
= + =

u x S
u
( ) ( ) 1 1 x u =
( ) ( ) ( ) ( ) ( )
2 2 2 0
2
1 3 1 2 1 1 4 1 5 x x x x x S = + =
( ) ( ) 1 1 x u =
( )
2 0
( )
2 0
2
1 3 x S =
( ) ( ) ( ) ( ) ( )
( ) ( ) [ ] ( ) ( )
2 2 2
2 2 2 2 2 0
2 2 3
0 0 0 0 2 3
0 0 1 3 0 0
u x u x
u x x u x S J S
+ + + =
+ + = + + = =
( )
( ) ( ) [ ] ( ) 0 0 2 0 0 2 6
0
3
= + + =

u u x S
u
( ) ( ) 0
8
12
0 x u =
( ) ( ) [ ] ( ) ( ) [ ]
2 2 2 0
3
0 5 . 1 0 0 5 . 1 0 2 3 x x x x S + + =
ECE 5700 Notes and figures are based on or taken from materials in the course textbook: C.L. Phillips and H.T
Nagle, Digital Control System Analysis and Design, Prentice Hall, 1995, ISBN: 0-13-309832.
15
( ) ( ) ( ) ( )
2 2 2 2
0 4 0
4
9
0 0
4
3
x x x x = + + =
Example Continued (3) p ( )
Take partial derivatives with respect to the specified p p p
variable; and set the result equal to zero.
( ) 0 2 = u ( ) ( ) 1 1 x u =
( ) ( ) 0 5 . 1 0 x u =
( )
( )
2 0
1
2 x S =
( ) ( )
( )
2 0
2
1 3 x S =
( ) ( )
( )
2 0
3
0 4 x S =
To achieve this minimum
( )
2 0
2
0
3
0 4 x J S = =
To achieve this minimum,
K is time variable for a time invariant system
( ) ( ) ( ) k x k K k u = ( ) { } 2 , 1 , 0 0 , 1 , 5 . 1 = = k for k K
ECE 5700 Notes and figures are based on or taken from materials in the course textbook: C.L. Phillips and H.T
Nagle, Digital Control System Analysis and Design, Prentice Hall, 1995, ISBN: 0-13-309832.
16
The feedback gain for this simple equation was time g p q
varying for a time invariant system.
No other set of inputs will yield a smaller Cost Function.
The optimal control design is solved for in reverse time,
we start at N and work back to 0.
The optimal gain at K(i) cannot be calculated until the remaining The optimal gain at K(i) cannot be calculated until the remaining
K(j) for i<j<=N are known.
A construct to efficiently perform this computation is A construct to efficiently perform this computation is
needed
Maybe as N becomes large, there is a constant that K(0) y g , ( )
approaches that we could use
ECE 5700 Notes and figures are based on or taken from materials in the course textbook: C.L. Phillips and H.T
Nagle, Digital Control System Analysis and Design, Prentice Hall, 1995, ISBN: 0-13-309832.
17
Linear Quadratic Optimal Control Q p
Solving the LQ Linear quadratic optimal design problem g Q q p g p
For the state space system we which to determine
The optimal control inputs as a function of the states
( ) ( ) [ ] k x f k u =
0
That minimizes the cost function
( ) ( ) ( ) ( ) ( ) ( )

+ =
N
T T
N
k u k R k u k x k Q k x J ( ) ( ) ( ) ( ) ( ) ( )

= k
N
Q
0
ECE 5700 Notes and figures are based on or taken from materials in the course textbook: C.L. Phillips and H.T
Nagle, Digital Control System Analysis and Design, Prentice Hall, 1995, ISBN: 0-13-309832.
18
Matrices Review
Proofs not provided for quadratic function differentiation p q
[ ] x Q y y Q x
T T
T
T
=
[ ] y Q y Q x
x
T
=

[ ] x Q y Q x
y
T T
=

[ ] x Q x Q x
x
T
=

2
y
ECE 5700 Notes and figures are based on or taken from materials in the course textbook: C.L. Phillips and H.T
Nagle, Digital Control System Analysis and Design, Prentice Hall, 1995, ISBN: 0-13-309832.
19
Matrix Definition of S
From optimality p y

= + + + = =
N
F F F F J J S

=
= + + + =
N
i
i N N
F F F F J
0
1 0
L
U i h d fi i i f F

+ =
+ +
= + + + = =
m N i
i N m N m N m N N m
F F F F J J S
1
2 1
L
m N m m N m N m N N m
F S J J J J S
+
+ = + =
1 1
Using the definition of F
( ) ( ) ( ) ( ) ( ) ( ) m N u m N R m N u m N x m N Q m N x S S
T T
m m
+ + =
+1
S
( )
0
1
=

+
m N u
S
m
ECE 5700 Notes and figures are based on or taken from materials in the course textbook: C.L. Phillips and H.T
Nagle, Digital Control System Analysis and Design, Prentice Hall, 1995, ISBN: 0-13-309832.
20
LQ Derivation (2) Q ( )
Beginning at N g g
( ) ( ) ( ) ( ) ( ) ( ) N u N R N u N x N Q N x F S
T T
N
+ = =
1
( )
0
1
=

N u
S
( ) N u
( ) ( ) ( ) N x N Q N x S
T
=
0
1
( ) ( ) ( ) ( ) ( ) ( )
0
1 2
+ = F S S
T T
N
( ) ( ) ( ) ( ) ( ) ( )
( ) ( ) ( ) ( ) ( ) ( ) 1 1 1 1 1 1 + +
+ =
N u N R N u N x N Q N x
N u N R N u N x N Q N x
T T
T T
( ) ( ) [ ] ( ) ( ) ( ) [ ] 1 1 1 1 N B N A N Q N B N A S
T
( ) ( ) [ ] ( ) ( ) ( ) [ ]
( ) ( ) ( ) ( ) ( ) ( ) 1 1 1 1 1 1
1 1 1 1
2
+ +
+ + =
N u N R N u N x N Q N x
N u B N x A N Q N u B N x A S
T T
T
ECE 5700 Notes and figures are based on or taken from materials in the course textbook: C.L. Phillips and H.T
Nagle, Digital Control System Analysis and Design, Prentice Hall, 1995, ISBN: 0-13-309832.
21
LQ Derivation (3) Q ( )
Beginning at N g g
( ) ( ) [ ] ( ) ( ) ( ) [ ]
( ) ( ) ( ) ( ) ( ) ( ) 1 1 1 1 1 1
1 1 1 1
2
+ +
+ + =
N u N R N u N x N Q N x
N u B N x A N Q N u B N x A S
T T
T
( )
0
1
2
=

N u
S
( )
( ) ( ) ( ) [ ] 1 1
2
+ =

N u B N x A N Q B
S
T
( )
( ) ( ) ( ) [ ]
( ) ( ) [ ] ( ) { } ( ) ( ) 1 1 2 1 1
1 1
1
+ + +
+

N u N R B N Q N u B N x A
N u B N x A N Q B
N u
T
T
{ } ( ) ( ) ( ) ( ) { } ( ) 1 1 2 2 1 2 + + N u N R B N Q B N x A N Q B
T T
( ) ( ) ( ) { } ( ) ( )
( ) ( ) 1 1
1 1 1
1
+ =

N x N K
N x A N Q B N R B N Q B N u
T T o
ECE 5700 Notes and figures are based on or taken from materials in the course textbook: C.L. Phillips and H.T
Nagle, Digital Control System Analysis and Design, Prentice Hall, 1995, ISBN: 0-13-309832.
22
( ) ( ) 1 1 = N x N K
LQ Derivation (4) Q ( )
Now that we solved for the optimal u p
( ) ( ) [ ] ( ) ( ) ( ) [ ]
( ) ( ) ( ) ( ) ( ) ( ) 1 1 1 1 1 1
1 1 1 1
0 0
0 0
2
0
+ +
+ + =
N u N R N u N x N Q N x
N u B N x A N Q N u B N x A S
T T
T
Where P is
( ) ( ) ( ) 1 1 1 2
0
= N x N P N x S
T
( ) ( ) N Q N P =
( ) ( ) ( ) ( ) [ ] ( ) ( ) ( ) ( ) [ ]
( ) ( ) ( ) ( ) 1 1 1 1
1 1 1 1 1 1 1
+ +
=
N K N R N K N Q
N K N B N A N P N K N B N A N P
T
T
ECE 5700 Notes and figures are based on or taken from materials in the course textbook: C.L. Phillips and H.T
Nagle, Digital Control System Analysis and Design, Prentice Hall, 1995, ISBN: 0-13-309832.
23
Generalizing the LQ Derivation (1) g Q ( )
For K and u
o
the derivation follows
Subscripts are N-m unless explicitly shown
[ ] [ ] Q A A
u R u x Q x S S
T T
T
T T
m
o
m
+ + =
+1
[ ] [ ] u R u x Q x u B x A P u B x A
T T
m N
T
+ + =
+ 1
[ ] 0 2 2
1
1
= + =

+
+
u R u B x A P B
S
m N
T
m
[ ]
1

+
u
m N
[ ]
x K
x A P B R B P B u
m N
T
m N
T o
=
+ =
+

+ 1
1
1
ECE 5700 Notes and figures are based on or taken from materials in the course textbook: C.L. Phillips and H.T
Nagle, Digital Control System Analysis and Design, Prentice Hall, 1995, ISBN: 0-13-309832.
24
Generalizing the LQ Derivation (2) g Q ( )
For S
( ) ( ) ( ) 1 1 1
0
+ + + = m N x m N P m N x S
T
m
o
T
o T
m
o
m
u R u x Q x S S + + =
+1
( ) [ ] ( ) [ ] [ ] [ ] x K R x K x Q x x BK A P x BK A S
T
T
m N
T
m + + =
+
+
1
1
0
[ ] [ ] { } x K R K Q BK A P BK A x S
T
m N
T
T
m + + =
+
+
1
1
0
For P
( ) ( ) N Q N P =
x P x S
m N
T
m =

+1
0
( ) ( ) ( ) ( ) [ ] ( ) ( ) ( ) ( ) [ ]
( ) ( ) ( ) ( ) m N K m N R m N K m N Q
m N K m N B m N A m N P m N K m N B m N A m N P
T
T
+ +
+ = 1
[ ] [ ] K R K Q K B A P K B A P
T
T
+ +
ECE 5700 Notes and figures are based on or taken from materials in the course textbook: C.L. Phillips and H.T
Nagle, Digital Control System Analysis and Design, Prentice Hall, 1995, ISBN: 0-13-309832.
25
[ ] [ ] K R K Q K B A P K B A P
T
m N m N
+ + =
+ 1
Generalized LQQ
The derived equation set for k=N-m q
( ) [ ] x A P B R B P B k u
k
T
k
T o
+ =
+

+ 1
1
1
( ) 0 = N u
o
( ) ( ) N Q N P =
( ) [ ] A P B R B P B k K
k
T
k
T
+ =
+

+ 1
1
1
[ ] [ ] K R K Q K B A P K B A P
T
k
T
k
+ + =
+1
( ) x P x k S
T
=
0
[ ] [ ] K R K Q K B A P K B A P
k k
+ +
+1
( ) ( ) ( ) ( ) ( ) 0 0 0 1
0 0
x P x N J N S
T
= = +
ECE 5700 Notes and figures are based on or taken from materials in the course textbook: C.L. Phillips and H.T
Nagle, Digital Control System Analysis and Design, Prentice Hall, 1995, ISBN: 0-13-309832.
26
Alternate Equation for P q
Recognizing an element, P can be simplified g g , p
[ ] [ ]
[ ] [ ]
{ }
K R K K B A P B K Q K B A P A
K R K Q K B A P K B A P
T
k
T T
k
T
T
k
T
k
+ + =
+ + =
+ +
+
1 1
1
K i i i i h i f h d b k
[ ] { }
[ ] { } [ ] K R B P B K A P B K Q K B A P A
K R K K B P B K A P B K Q K B A P A
k
T T
k
T T
k
T
T
k
T T
k
T T
k
T
+ + + =
+ + + =
+ + +
+ + +
1 1 1
1 1 1
Ks iterative computation contains the inverse of the end bracket
[ ] { }
[ ] [ ]
A P B K Q K B A P A P
k
T T
k
T
k
+ =
+ +
1
1 1
[ ] [ ]
[ ] { }
[ ] Q K B A P A
A P B K A P B K Q K B A P A
A P B R B P B R B P B K
k
T
k
T T
k
T T
k
T
k
T
k
T
k
T T
+ =
+ + =
+ + +
+
+ + +
+

+ +
1
1 1 1
1
1
1 1
ECE 5700 Notes and figures are based on or taken from materials in the course textbook: C.L. Phillips and H.T
Nagle, Digital Control System Analysis and Design, Prentice Hall, 1995, ISBN: 0-13-309832.
27
[ ] Q
k+1
Are K or P dependent upon x or u? p p
( ) [ ] A P B R B P B k K
k
T
k
T
+ =
+

+ 1
1
1
[ ] Q K B A P A P
k
T
k
+ =
+1
Only based on the system state space definition and cost function
elements, A, B, R, and Q.
[ ] Q
k k +1
Can the equation be readily computed in MATLAB
Just backward iterations are required.
h i f i b d f i ? How much information must be stored for execution?
K is required for each time instance.
J
N
can be computed based on the initial conditions
N
p
ECE 5700 Notes and figures are based on or taken from materials in the course textbook: C.L. Phillips and H.T
Nagle, Digital Control System Analysis and Design, Prentice Hall, 1995, ISBN: 0-13-309832.
28
Example 10.2 p
N=2 optimization

=
0 0
0 1
Q
1 = R
p
( ) ( ) ( ) k u k x k x

= +
0952 . 0
00484 . 0
905 . 0 0
0952 . 0 1
1

0 0
Q
( )

= =
0 0
0 1
2 Q P
ECE 5700 Notes and figures are based on or taken from materials in the course textbook: C.L. Phillips and H.T
Nagle, Digital Control System Analysis and Design, Prentice Hall, 1995, ISBN: 0-13-309832.
29
MATLAB Computations p
a =
x1 x2
1 1 0 0952
Initial Conditions:
Q =
x1 1 0.0952
x2 0 0.9048
b =
u1
x1 0.00484
2 0 0952
1 0
0 0
R =
1
P =
1 0
0 0
It ti l f N 1 x2 0.0952
c =
x1 x2
y1 1 0
d
Iterative loop for N= 1
D =
1.0000
K =
0 0
0.0048 0.0005
P =
2 0000 0 0952
d =
u1
y1 0
Sampling time: unspecified
Discrete-time model.
2.0000 0.0952
0.0952 0.0091
Iterative loop for N= 0
D =
1.0002
K =
0.0187 0.0030
0 0048 0 0005
numd =
0 0.0048 0.0047
dend =
1.0000 -1.9048 0.9048
0.0048 0.0005
P =
2.9996 0.2765
0.2765 0.0419
ECE 5700 Notes and figures are based on or taken from materials in the course textbook: C.L. Phillips and H.T
Nagle, Digital Control System Analysis and Design, Prentice Hall, 1995, ISBN: 0-13-309832.
30
Example 10.3 p
Settling the previous system in 5 time constants, or 5 sec.
The sample rate was T=0.1.
( ) ( ) ( ) ( )

=
+

=
N
k
T T
N
k u k u k x k x J
0
1
0 0
0 1
( ) ( ) ( ) k x k K k u =
0.8
0.9
1
K coefficients in time
= k 0
0 0
1
X states, Cost J N=17.8148
0 4
0.5
0.6
0.7
T
i
m
e

(
s
e
c
)
0.5
T
i
m
e

(
s
e
c
)
0.1
0.2
0.3
0.4
T
0
T
ECE 5700 Notes and figures are based on or taken from materials in the course textbook: C.L. Phillips and H.T
Nagle, Digital Control System Analysis and Design, Prentice Hall, 1995, ISBN: 0-13-309832.
31
0 1 2 3 4 5 6
0
K
0 1 2 3 4 5 6
-0.5
x values
Minimum Principle p
The minimum principle that follows results in the same
If the input u
o
(k) and corresponding trajectory x
o
(k) are
solution as previously stated. The principle states:
p ( ) p g j y ( )
optimal, there exists a nontrivial vector sequence {p
o
(k)} such
that u
o
(k) is the value of u(k) that minimizes the Hamiltonian
and the costate vector p
o
(k) satisfies the following for kN
( ) ( ) ( ) ( ) ( ) ( ) [ ] ( ) ( ) ( ) ( ) ( ) [ ] k u k B k x k A k p k u k R k u k x k Q k x H
o
T
o
T
o
T
o
+ + + + = 1
2
1
and the costate vector p (k) satisfies the following for kN
( )
( ) k x
H
k p
o
o

=
( ) ( ) ( ) N x N Q N p
o o
=
ECE 5700 Notes and figures are based on or taken from materials in the course textbook: C.L. Phillips and H.T
Nagle, Digital Control System Analysis and Design, Prentice Hall, 1995, ISBN: 0-13-309832.
32
( )
Minimum Principle Iterations p
The consequences of this principle are q p p
( ) ( ) ( ) ( ) ( )
T
o
T
o o
k p k A k x k Q k p 1 + + =
( ) ( ) ( ) ( )
T
o
T
o
k p k B k R k u 1
1
+ =

( )
( ) k x
H
k p
o
o

=
( ) ( ) ( ) ( ) p
Finding the solution requires the solution of a two-point
boundary value problem. (for x optimal moving forward in
time and p optimal backward). p p )
This section is stated for relevance, not derivation!
ECE 5700 Notes and figures are based on or taken from materials in the course textbook: C.L. Phillips and H.T
Nagle, Digital Control System Analysis and Design, Prentice Hall, 1995, ISBN: 0-13-309832.
33
Steady State Optimal Control y p
For time-invariant systems with time-invariant cost y
functions, the optimal gain will approach a steady state
value with decreasing time.
As seen in the previous example As seen in the previous example.
We can use these values as steady state coefficients. We ca use t ese va ues as steady state coe c e ts.
ECE 5700 Notes and figures are based on or taken from materials in the course textbook: C.L. Phillips and H.T
Nagle, Digital Control System Analysis and Design, Prentice Hall, 1995, ISBN: 0-13-309832.
34
Deriving the steady-state. g y
Using the iterative equations with time-invariant elements. g q
( ) [ ] A P B R B P B k K
k
T
k
T
+ =
+

+ 1
1
1
[ ] Q K B A P A P
k
T
k
+ =
+1
Creating a difference equation for P
[ ] Q
k k +1
[ ] [ ] Q A P B R B P B B A P A P
k
T
k
T
k
T
k
+ + =
+

+ + 1
1
1 1
This equation is in the form of a discrete Riccati equation
[ ] A P B R B P B B P A Q A P A P
k
T
k
T
k
T
k
T
k
+ + =
+

+ + + 1
1
1 1 1
This equation is in the form of a discrete Riccati equation
The inverse in this equation always exists, as R is positive definite
and P is positive semidefinite. Therefore, a solution is possible.
ECE 5700 Notes and figures are based on or taken from materials in the course textbook: C.L. Phillips and H.T
Nagle, Digital Control System Analysis and Design, Prentice Hall, 1995, ISBN: 0-13-309832.
35
Steady State Optimum y p
Assuming constants g
[ ] A P B R B P B K
T T
+ =

1
[ ] A P B R B P B B P A Q A P A P
T T T T
+ + =

1
Solving the Discrete Riccatti Equation
Use MATLAB its easy.
The textbook approach is perfectly correct but tedious except for The textbook approach is perfectly correct, but tedious except for
those how love the math.
ECE 5700 Notes and figures are based on or taken from materials in the course textbook: C.L. Phillips and H.T
Nagle, Digital Control System Analysis and Design, Prentice Hall, 1995, ISBN: 0-13-309832.
36
Example 10.4 p
Use MATLAB to compute the p
dlqr: Linear-quadratic regulator design for discrete-time systems
dare: Solve discrete-time algebraic Riccati equations.
ECE 5700 Notes and figures are based on or taken from materials in the course textbook: C.L. Phillips and H.T
Nagle, Digital Control System Analysis and Design, Prentice Hall, 1995, ISBN: 0-13-309832.
37
Optimal State Estimation
K l Filt Kalman Filters
The Kalman filter has the form of a current observer.
The gain matrix will be computed differently.
The plant for Kalman Filters is defined as
( ) ( ) ( ) ( ) ( ) ( ) ( )
w(k) are random plant disturbances
( ) ( ) ( ) ( ) ( ) ( ) ( ) k w k B k u k B k x k A k x + + = +
1
1
( ) ( ) ( ) ( ) k v k x k C k y + =
w(k) are random plant disturbances
v(k) are random inaccuracies in y(k) measurements.
w(k) and v(k) are uncorrelated and have zero mean, Gaussian
di t ib ti distributions
Independent wide-sense stationary random variables.
( ) ( ) [ ] ( ) j i R j i E
T
( ) ( ) [ ] ( ) j i R j i E
T
( ) ( ) [ ] 0
T
j i E
ECE 5700 Notes and figures are based on or taken from materials in the course textbook: C.L. Phillips and H.T
Nagle, Digital Control System Analysis and Design, Prentice Hall, 1995, ISBN: 0-13-309832.
38
( ) ( ) [ ] ( ) j i R j v i v E
v
= ( ) ( ) [ ] ( ) j i R j w i w E
w
= ( ) ( ) [ ] 0 = j w i v E
Kalman Filter Basis
The estimate of the states
with error covariance
( ) ( ) ( ) k q k x k e =

( ) ( ) ( ) [ ]

= =
N
N
T
e e e e e e
e e e e e e
E k e k e E k P
M O M M
L
L
2 2 2 2 1
1 1 2 1 1
The Cost Function is the trace of the P(k) matrix
th f th l t d i

N N N N
e e e e e e L
2 1
the sum of the error elements squared variane
( ) ( ) [ ] ( ) [ ] ( ) [ ] ( ) [ ]
2 2
2
2
1
k e E k e E k e E k P trace k J
N
+ + + = = L
( ) ( ) ( ) ( )
2 2
2
2
1
k k k k J
N
+ + + = L
ECE 5700 Notes and figures are based on or taken from materials in the course textbook: C.L. Phillips and H.T
Nagle, Digital Control System Analysis and Design, Prentice Hall, 1995, ISBN: 0-13-309832.
39
( ) ( ) ( ) ( )
2 1
k k k k J
eN e e
+ + +
Note that the expected values of the squared estimation p q
error are minimized, not the actual error or squared error.
If you dont know the statistics, the solution isnt valid.
The Kalman filter is optimal over the averaged squared estimation The Kalman filter is optimal over the averaged squared estimation
error.
Note also: for any positive definite Q and cost function
The Kalman filter will minimize the cost function
( ) ( ) ( ) [ ]
T
k e Q k e trace k J =
The Kalman filter will minimize the cost function.
The individual terms are minimized; therefore, all cost functions of
this form are simultaneously or inherently minimized.
ECE 5700 Notes and figures are based on or taken from materials in the course textbook: C.L. Phillips and H.T
Nagle, Digital Control System Analysis and Design, Prentice Hall, 1995, ISBN: 0-13-309832.
40
Kalman Filter Equations q
Predictive state estimation qbar with state estimate q q q
( ) ( ) ( ) ( ) ( ) [ ] k q C k y k G k q k q + =
( ) ( ) ( ) ( ) ( ) k u k B k q k A k q + = +1
Prediction and Estimated Error Covariance are
( ) ( ) ( ) ( ) ( ) ( ) ( ) [ ]
T
k q k x k q k x E k M =
The Kalman gain matrix
( ) ( ) ( ) [ ]
1
( ) ( ) ( ) ( ) ( ) ( ) ( ) [ ]
T
k q k x k q k x E k P =
( ) ( ) ( ) [ ]
1
+ =
v
T T
R C k M C C k M k G
ECE 5700 Notes and figures are based on or taken from materials in the course textbook: C.L. Phillips and H.T
Nagle, Digital Control System Analysis and Design, Prentice Hall, 1995, ISBN: 0-13-309832.
41
Kalman Filter Equations (2) q ( )
Iterative Update for the Estimated Error: p
or
( ) ( ) ( ) ( )
( ) ( ) ( ) ( ) [ ] ( ) k M C R C k M C C k M k M k P
v
T T
+ =
1
Iterative Update for the Prediction Error:
( ) ( ) ( ) ( ) k M C k G k M k P =
( ) ( )
T
T
B R B A k P A k M 1 + = +
or
( ) ( )
w
B R B A k P A k M
1 1
1 + = +
( ) ( ) [ ] ( )
T
w
T
B R B A k M C k G I A k M
1 1
1 + = +
( ) ( ) ( ) ( ) [ ] ( ) { }
T
w
T
v
T T
B R B A k M C R C k M C C k M k M A k M
1 1
1
1 + + = +

( ) ( ) ( ) ( ) [ ] ( )
T
T T T T
B R B A k M C R C k M C C k M A A k M A k M
1
1 + + = +

ECE 5700 Notes and figures are based on or taken from materials in the course textbook: C.L. Phillips and H.T
Nagle, Digital Control System Analysis and Design, Prentice Hall, 1995, ISBN: 0-13-309832.
42
( ) ( ) ( ) ( ) [ ] ( )
w v
B R B A k M C R C k M C C k M A A k M A k M
1 1
1 + + = +
Kalman State Equation q
To remove the state estimation
( ) ( ) ( ) ( ) ( ) ( ) ( ) [ ] [ ] 1 1 1 1 + + + = k u B k q A C k y k G k u B k q A k q
( ) ( ) ( ) 1 1 + = k u B k q A k q
( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) k y k G k u B C k G B k q A C k G A k q + + = 1 1
Iterations
( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) k y k G k u B C k G B k q A C k G A k q + + = 1 1
( ) ( ) [ ] ( )
T
T
B R B A k M C k G I A k M 1 + + ( ) ( ) [ ] ( )
w
B R B A k M C k G I A k M
1 1
1 + = +
( )
T
( ) ( ) ( ) [ ]
1
+ =
v
T T
R C k M C C k M k G
ECE 5700 Notes and figures are based on or taken from materials in the course textbook: C.L. Phillips and H.T
Nagle, Digital Control System Analysis and Design, Prentice Hall, 1995, ISBN: 0-13-309832.
43
( )
T
w
B R B M
1 1
0 =
Steady State Kalman y
If G(k) is assumed to be constant, the implementation ( ) , p
becomes that of the current estimator described in Chap. 9.
but remember, the Kalman filter performs state estimation.
Feedback is still needed Feedback is still needed.
Combining the optimal infinite-horizon feedback
previously defined with the steady-state Kalamn filter
estimator, an Infinite-Horizon Linear-Quadratic Gaussian
(IH-LQG) architecture can be defined. The controller- ( Q )
estimator transfer function is
( ) [ ] G K B C G K B A C G A I z K z z D
ce
+ + =
ECE 5700 Notes and figures are based on or taken from materials in the course textbook: C.L. Phillips and H.T
Nagle, Digital Control System Analysis and Design, Prentice Hall, 1995, ISBN: 0-13-309832.
44
IH-LQG Control System Q y
See Figure 10-7 g
update equations
( ) ( ) ( ) ( ) ( ) ( ) k q A C G k y G k u B C G B k q A k q + + + = + 1 1
z-domain equations
( ) ( ) k q K k u =
( ) ( ) ( ) ( ) ( ) ( ) z Y G z z U B C G B z Q A C G A z Q z + + =
( ) ( ) ( ) ( ) ( ) ( ) { } z Q A C G z Y G z z U B C G B A I z z Q +
1
( ) ( ) ( ) ( ) ( ) ( ) { } z Q A C G z Y G z z U B C G B A I z z Q + =
ECE 5700 Notes and figures are based on or taken from materials in the course textbook: C.L. Phillips and H.T
Nagle, Digital Control System Analysis and Design, Prentice Hall, 1995, ISBN: 0-13-309832.
45