Types of
Optimization Models
Stochastic
(probabilistic
information on data)
Discrete, Integer
(S = Zn)
Linear
(f and g are linear)
Deterministic
(data are certain)
Continuous
(S = Rn)
Nonlinear
(f and g are nonlinear)
Unidimensional Search
Chapter 5
(1) If have a search direction, want to minimize in that direction by numerical methods
(3) Types of search that are better or best is often problem dependent. Some of the types are:
Chapter 5
In mathematics and the arts, two quantities are in the golden ratio if the
ratio between the sum of those quantities and the larger one is the same as
the ratio between the larger one and the smaller.
The golden ratio is a mathematical constant approximately 1.6180339887.
The golden ratio is also known as the most aesthetic ratio between the two
sides of a rectangle.
The golden ratio is often denoted by the Greek letter (phi).
Firstly, divide a square such that it makes two precisely equal rectangles.
1+ 5
2
Leonardo Da Vinci
Many artists who lived after Phidias have
used this proportion. Leonardo Da Vinci
called it the "divine proportion" and featured
it in many of his paintings, for example in the
famous "Mona Lisa". Drawing a rectangle
around her face. You will realize that the
measurements are in a golden proportion.
You can further explore this by subdividing
the rectangle formed by using her eyes as a
horizontal divider.
The Parthenon
Phi was named for the
Greek sculptor Phidias.
The exterior dimensions of
the Parthenon in Athens,
built in about 440BC, form a
perfect golden rectangle.
The dividence of every long line to the short line equals the golden ratio.
Lenght of the face / Wideness of the face
Lenght between the lips and eyebrows / Lenght of the nose,
Lenght of the face / Lenght between the jaw and eyebrows
Lenght of the mouth / Wideness of the nose,
Wideness of the nose / Distance between the holes of the nose,
Length between the pupils / Length between teh eyebrows.
All contain the golden ratio.
The shape of the inner and outer surfaces of the sea shells, and their
curves fit the golden ratio..
Golden Section
Origin of golden section:
I1
I1 I 2 I 3
I1 fI1 f 2 I1
f f 1 0 f1, 2
2
I2 = fI1
1 5
5 1
f
0.618034
2
Final interval:
I N f N I1
I2 = fI1
I3 = fI2
1 12 23 3 5 5
8 8
1 2 3 5
8
xU
xL xL
xL
xU
Golden-Section Search
divides intervals by
K = 1.6180
f ( X ) X 2 X 2 X X X 2 X1 5
4
1
2
1
2
2
2
1
f
0 4 X 13 4 X 2 X 1 2 X 1 2
X 1
Chapter 4
f
0 2 X 12 2 X 2
X 2
min at (1,1)
12 X 12 4 X 2 2 4 X 1
H
4 X1
2
10 4
at (1,1), H
0
4 2
Chapter 5
1. Compute f ( x ( 0) ) and f ( x 0 x)
If f ( x ( 0) x) f ( x ( 0 ) ), let x NEW xOLD
If f ( x ( 0) x) f ( x ( 0) ), let xNEW 2xOLD
2. Compute f ( x (1) ) f ( x ( 0) xNEW )
Continue doubling x until f ( x ( 0 ) x ( k ) ) f ( x ( 0) x ( k 1) )
3. Compute f ( x ( k 1) x ( k 2 ) )
1
(2 )
2
Chapter 5
Newtons Method
Newtons method for an equation is
Chapter 5
f ( x) f ( x0 ) f ( x0 )( x x0 ) 0
f ( x0 )
( x x0 )
f ( x0 )
or
f ( x0 )
x x0
f ( x0 )
Application to Minimization
The necessary condition for f(x) to have a local minimum is f(x) = 0. Apply Newtons method.
x ( k 1)
(k )
f (x )
(k )
x
(k )
f (x )
1
f ( x) f ( x ) f ( x )( x x ) f ( x ( k ) )( x x ( k ) ) 2
2
df(x)
Let
0
dx
Then f ( x ( k ) ) f ( x ( k ) )( x x ( k ) ) 0
( A)
Chapter 5
(k )
(k )
(k )
Examples
Minimize
f ( x) a0 a1 x a2 x 2
f ( x) a1 2a2 x
f ( x) 2a2
Chapter 5
x (1)
(0)
a
2
a
x
a
a
2
x (0) 1
x (0) 1 x (0) 1
2a2
2 a2
2 a2
Minimize
f ( x) x 4 x 2 1
f ( x) 4 x 3 2 x
f ( x) 12 x 2 2
x (1)
4
3
x
2x
x (0)
12 x 2 2
3 2
0.100
12 2
Chapter 6
k 1
x s x x
k
Steepest Descent
Search Direction
s f ( x )
k
f ( x) 0
H (x )
Chapter 6
Step Length
How to pick
analytically
numerically
s f ( x )
0
s f ( x ) 1s
1
Chapter 6
s f ( x ) s
2
k 1
0 (conjugate directions)
f ( x
k 1
) f ( x ) 2f ( x )( x x )
f ( x
k 1
) f ( x ) H x H k s
k
1
k 1
k
s H f ( x ) f ( x ) k
T
k 1
k
1
(s ) f ( x ) f ( x ) H / k
Chapter 6
k T
k 1
k 1
=0,
1
k 1
k
) f ( x ) H H f ( x ) k s 0
k
k 1
T f ( x )f ( x )
k
k
k
T f ( x )f ( x )
s
k 1
f ( x
k 1
) k s
Chapter 6
Chapter 6
Chapter 6
Chapter 6
Chapter 6
2
2
Minimize f ( x1 3) 9( x2 5) using the method of conjugate gradients with x10 1 and x20 1 as an initial point.
1
In vector notation,x 0
1
f
4
72
x0
Chapter 6
x0
4
72
0 0.
5.011
3.554
0.197
(3.554) 2 (0.197) 2
0.00244.
(4) 2 (72) 2
0
and
Chapter 6
0.00244
0.197
72 0.022
1.223
1 3.564
x2
0.022
5.011
3.0000
5.0000
which is the optimum (in 2 steps, which agrees with the theory).