Lecture1 – p.1/37
Example
Number of cancer cells at time
(exponential growth) State
Drug concentration Control
minimize
Lecture1 – p.3/37
Deterministic Optimal Control
state
Lecture1 – p.4/37
Basic Idea
Lecture1 – p.5/37
Design an appropriate objective functional
–balancing opposing factors in functional
–include (or not) terms at the final time
Lecture1 – p.6/37
Big Idea
Lecture1 – p.7/37
Deterministic Optimal Control- ODEs
associated state variable to maximize
!
$
#
#
"
subject to
%
'
&
#
and
(
)*
*
&
"
Lecture1 – p.8/37
Contd.
. -
/
,
+
Put into state DE and obtain
. -
. -
/
,
,
+
0
corresponding optimal state
. -
/
,
0
, optimal pair
. -
. -
/
,
,
+
Lecture1 – p.9/37
Necessary, Sufficient Conditions
Necessary Conditions
If , are optimal, then the following
4 3
4 3
5
2
2
1
conditions hold
..
.
Sufficient Conditions
If , and (adjoint) satisfy the conditions
4 3
4 3
7
2
2
1
..
.
then , are optimal.
4 3
4 3
5
2
2
1
Lecture1 – p.10/37
Adjoint
: 9
;
8
associated state variable to maximize
: 9
;
<
@
: 9
: 9
: 9
;
C;
:
?>
=
<
8
B
B
A
subject to
D
: 9
: 9
: 9
: 9
;
;
<
<
8
E
B
and
G9
;
<
<
<
HI
I
E
Lecture1 – p.12/37
Quick Derivation of Necessary Condition
K
J
L
corresponding state. variation function,
ON
P
M
.
R
Q
u*(t)+ ah(t)
u*(t)
another control.
O N
ON
P
M
K
J
state corresponding to ,
O N
M
K
S
Q
T
O N
P
U
S
Q
T
O N
O N
P
O N
P
P
P
K
M
W
Q
V
T
OU
Lecture1 – p.13/37
Contd.
At ,
_
Z
Z
X
a `
Y
Y
]
y(t,a)
x * (t)
x0
_
b
b
Z
Z
X
X
[
c
Y
Y
]
]
e
\
_
g_
b
f
d
X
^
^
Y
]
a
Z
^
Y
Lecture1 – p.14/37
ih
t
w k h
k j
l
Adding
t xh
h m
v w sr
q p
n n o n o n
jy
xh
h
Contd.
x j t
jy
z m
l
u
x j
to our
{ | x j
i v
z m
k
k j
{ | x j
m
m k m
m xh
m
xh u
y
gives
}j
y
z m
l
j
v
z m }j
{
j k
{
v
k ~ m
~ m
y
y j
v
}j z m
l
j
z m
l {
v
k
}j
{
m
k
v
Lecture1 – p.15/37
l
Contd.
|
l
|
|
here we used product rule and
¡
l
.
|
Lecture1 – p.16/37
¢ ¢
¢ ¢ ¤ £
¥
¥
¤ £
° ®
¾ µ § ¦
´ ·°
Contd.
´
¥ © ª ¨
Arguments of
Arguments of
» »
¨ ¹ ¨ « ¬ «
© ¸
§ ¦ ¤
¸ µ ¸
º° « ¬ «
¤ ®
°
´
© ª ¨ ¯ ª ¨
®
« ¬ «
¤ «
¯ ¸
» ¼ º° ® ±l°
µ « ²
± °
© ¸ «
terms are
terms are
¾
» ½ ´ » ¤ ®
²
·° ·°
® « ¤
¬ ¤ ® ³
µ ´
à ¢ ¶ ¢ »
² ´ ·° ¤
» ·° ³
´ ®
¬ »
´
¤ ¤
± ´ µ
ÂÁ
À o
¿ ¿ o ¿ o ¿ ¶
¢
± ·°
²
· ° ´
² ·
´ § « ¬ «
´ ® ® ¤
.
¹
° ¤
¯ ª ¨ µ
³
º°
·°
´
® ´
´
.
µ «
«
¯ ¸ ¤ ¬
´ » ¼ º°
³
¢
¤
· ½ ´
Lecture1 – p.17/37
ß
ËÊ
Í Û ÚÈ Ê È
Ë ÉÊ
Ì
á Í Ì Í Ì
Þ ª Ð
Ò Ë Ê
Ý Ü
Ê
Choose
Þ ª Ð Û Î
Ò Ñ ª ÏÐ
Ó Ö Ä
Contd.
Ô ÆÅ
Ò | ËÊ
Õ Ç
Ô Þ Ø È Ò
Ö Ì
Ó
Ô
Ìß
s.t.
Õ
ËÈ Ê ËÊ Ô
Optimality condition.
Ö×Ì
Þ Ø Ì
arbitrary variation Ë
Ìà
Ò ËÊ
ËÈÊ
Ò Ø Ì
Ñ l
Ó
Ô
Ò ËÊ
Õ
Ô
Ò
Í Ì
Ó
transversality condition
Õ
Û Ô
ÌÙ
for all
âÛ
ã
adjoint equation
Lecture1 – p.18/37
Using Hamiltonian
Generate these Necessary conditions from
Hamiltonian
å ä
å ä
å ä
ê
é
ì é
í
ç
è
ë
æ
æ
integrand (adjoint) (RHS of DE)
maximize w.r.t. at
î
è
è
ï
optimality eq.
ì é
ð
ð
ë
ë
ñ
ñ
ï
adjoint eq.
ò
ê
é
ì é
í
ë
ë
ó
ô
ï
transversality condition
ä
ê
é
ð
ë
Lecture1 – p.19/37
Converted problem of finding control to maximize
objective functional subject to DE, IC to using
Hamiltonian pointwise.
For maximization
ö
at as a function of
ü
ú
ù
÷
÷ õö
ø
For minimization
ö
at as a function of
û
ü
ú
ù
÷
÷ õö
ø
Lecture1 – p.20/37
Two unknowns and
þ
ý
ÿ
introduce adjoint (like a Lagrange multiplier)
Three unknowns , and
þ
ý
ÿ
nonlinear w.r.t.
ý
Eliminate by setting
þ
ý
and solve for in terms of and
þ
þ
ý
ÿ
Two unknowns and
þ
ÿ
Lecture1 – p.21/37
Pontryagin Maximum Principle
If and
are optimal for above problem, then there
at each time, where Hamiltonian is defined by
and
transversality condition
Lecture1 – p.22/37
Hamiltonian
$
&
%
"
"
#
!
!
maximizes w.r.t. , is linear w.r.t.
'
#
#
$
(
&
&
%
"
"
!
!
bounded controls, .
,
+
+
*
Example:
0
&
&
%
#
. "
/
"
1
3
%
#
1
5
'
3
#
#
4
C B
D
? >
@
A
8 7
9
?
6
=
:
<
subject to
F
? >
? >
@
? >
@
JI>
K
E
E
G
G
H
What optimal control is expected?
Lecture1 – p.24/37
m c
n [ c c Y
[ Z d d [\Z
d d [ b
S L
N M
O
` R R
[ b
S
[ b
^ c
Y
h
R
Q i Q P
e
d ]
m g j d e
k R R
Y
[ b ^ R
_ VUTS
`
]
`
l XW
^
[ c V
[
Y `T
integrand
Y h `
[ g
c
[ W
]
c
i cT c
Q i f
k
j a
a R
P
[ W [ g
`
f
c
Example 1 worked
h
[
Q e c
`
RHS of DE
[
at
R
g R
S
cT
R
W
Lecture1 – p.25/37
Example 2
| s
D
w v
@
w v
x
q p
9
w
o
{
z
r
y
subject to
s
}
w v
@
w v
w v
@
Jv
y
u
~
z
What optimal control is expected?
Lecture1 – p.26/37
Hamiltonian
The adjoint equation and transversality condition
give
@
@
Lecture1 – p.27/37
Example 2 continued
¤ £
¥
@
¡
¢
¦
¤ £
¥
@
¡
§
¢
¦
¨
Lecture1 – p.28/37
Graphs, Example 2
Example 2.2
3
State 2
−1
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
Time
8
6
Adjoint
0
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
Time
0
−2
Control
−4
−6
−8
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
Time
Lecture1 – p.29/37
® ®
» »
µ Á ´ µ ´ µ ´ ¬ ¼ µ \ ´
¼ µ º
® µ º ® ª ©
¬® «
¨
®
¯ ¯
®
® ®
¶ ¼ ¶
¼ · ¬
_
¯
¯ ¬
º
Example 3
µ ¯ ¬
¹ ¬ ¹ ® °
¹ ¶ ¶ °
®
» ® ¯
»
¯
¯
¶ ¸«
¬ «
¯ ± ² U±°
® µ » ° µ
¹
¯ ¶
® ³
¹ ®
½ » ¹
¶
«
»
¶
at
· ¬ ¶
»± ¬
±
¾
¿
» ¬
À«
± µ ¾
µ
¹ ¶
»
Lecture1 – p.30/37
Contd.
É
Æ
ÄÅÃ
È
Â
Â
Ç
Ç
Æ
Æ
ÄÅÃ
É
É
Ê Â
/
Ç
Ç
ÌË
ÐË
Í
É
Ï Ç
Ñ
Â
Ä
Î
Solve for and then get .
É
Ò
Ò
Â
Ó
Î
Ì
Â
ÙÚ Ø
ÙÚ Ø
Ö Ô
Ö Ô
Ü
È
×
×
Ä
Û
É
Æ
Ê
Ê
Lecture1 – p.31/37
ä
æ ó ó ò ê
ô ê ò
ã
ì æ
à ßÝÞ
ç ë ë
ë
ë ì
ë ð
í â á
Exercise
ä ã
å
æ
ï î
ä æ
èç
Jðã
é
ç
ä
ó
ë
ô
ë ï ì
æ
control
ë
Lecture1 – p.32/37
ü ü ü
þ
õ
û û
ø ÷ö
ü
þ
þ þ ú ù
ü
ü û
þ
þ ý
ÿ
ü
ü
û
û
þ ý
û
û ý
ý
þ þ
û
þ
Exercise completed
þ
control
û
ø
ø
û
Lecture1 – p.33/37
Contd.
!
"
#
There is not an “Optimal Control" in this case.
Want finite maximum.
Here unbounded optimal state
unbounded OC
Lecture1 – p.34/37
Opening Example
&% Number of cancer cells at time
'
&
$
)
$
&%
'
&%
'
+
$
(
*
,
&)
0/$
*
1
minimize
2
%
'
&%
'
&)
$
(
/
See need for bounds on the control. See salvage
term.
Lecture1 – p.35/37
Further topics to be covered
Lecture1 – p.36/37
more info
3
4
Optimal Control Theory in Application to Biology
short course lectures and lab notes
Book: Optimal Control applied to Biological Models
CRC Press, 2007, Lenhart and J. Workman
Lecture1 – p.37/37