ECE 580

Basic Optimization Problem

Section 6.1

Motivation, Notation and Vocabulary

Stan Zak
January 09, 2013

ECE 580Basic Optimization Problem p. 1/17

Motivation

Optimizationan act, process, or

methodology of making something (as a
design, system, or decision) as fully perfect,
functional, or effective as possible. The
mathematical procedures (as finding the
maximum of a function) involved in this.

ECE 580Basic Optimization Problem p. 2/17

Motivation

Optimizationan act, process, or

methodology of making something (as a
design, system, or decision) as fully perfect,
functional, or effective as possible. The
mathematical procedures (as finding the
maximum of a function) involved in this.
Optimization making the best decision.

ECE 580Basic Optimization Problem p. 2/17

Motivation

Optimizationan act, process, or

methodology of making something (as a
design, system, or decision) as fully perfect,
functional, or effective as possible. The
mathematical procedures (as finding the
maximum of a function) involved in this.
Optimization making the best decision.
Engineering design, management, etc.

ECE 580Basic Optimization Problem p. 2/17

Motivation

Optimizationan act, process, or

methodology of making something (as a
design, system, or decision) as fully perfect,
functional, or effective as possible. The
mathematical procedures (as finding the
maximum of a function) involved in this.
Optimization making the best decision.
Engineering design, management, etc.
We focus on choices that involve real numbers
(examples later).

ECE 580Basic Optimization Problem p. 2/17

Motivation

Optimizationan act, process, or

methodology of making something (as a
design, system, or decision) as fully perfect,
functional, or effective as possible. The
mathematical procedures (as finding the
maximum of a function) involved in this.
Optimization making the best decision.
Engineering design, management, etc.
We focus on choices that involve real numbers
(examples later).
What does best mean?
ECE 580Basic Optimization Problem p. 2/17
Objective function

Measure goodness by a function f .

(Cost function or objective function)

ECE 580Basic Optimization Problem p. 3/17

Objective function

Measure goodness by a function f .

(Cost function or objective function)
Want to minimize f . (Smaller = better)

ECE 580Basic Optimization Problem p. 3/17

Objective function

Measure goodness by a function f .

(Cost function or objective function)
Want to minimize f . (Smaller = better)
= set of all possible choices.
(Feasible set)

ECE 580Basic Optimization Problem p. 3/17

Notation

Rn denotes a set of real n-tuples

ECE 580Basic Optimization Problem p. 4/17

Notation

Rn denotes a set of real n-tuples

x Rn means

x1
x2
x= , xi R

..
.
xn

ECE 580Basic Optimization Problem p. 4/17

Notation

Rn denotes a set of real n-tuples

x Rn means

x1
x2
x= , xi R

..
.
xn
f is a real-valued function of n variables,
n
f :R R

ECE 580Basic Optimization Problem p. 4/17

Example

f = f (x1 , x2 ) = x1 x2 + 7,
an example of a real-valued function of two
variables,
f : R2 R

ECE 580Basic Optimization Problem p. 5/17

Example

f = f (x1 , x2 ) = x1 x2 + 7,
an example of a real-valued function of two
variables,
f : R2 R
" #
x1
x= 7 a = f (x1 , x2 ) R
x2

ECE 580Basic Optimization Problem p. 5/17

Optimization problem

min f (x)
subject to x

ECE 580Basic Optimization Problem p. 6/17

Optimization problem

min f (x)
subject to x
What about maximization?
(Bigger = better)

ECE 580Basic Optimization Problem p. 6/17

Optimization problem

min f (x)
subject to x
What about maximization?
(Bigger = better)
How to solve optimization problem?

ECE 580Basic Optimization Problem p. 6/17

Optimization problem

min f (x)
subject to x
What about maximization?
(Bigger = better)
How to solve optimization problem?
Analytically

ECE 580Basic Optimization Problem p. 6/17

Optimization problem

min f (x)
subject to x
What about maximization?
(Bigger = better)
How to solve optimization problem?
Analytically
Numerically

ECE 580Basic Optimization Problem p. 6/17

Example: Linear regression

Given points on the plane:

(t0 , y0 ), . . . , (tn , yn )

ECE 580Basic Optimization Problem p. 7/17

Example: Linear regression

Given points on the plane:

(t0 , y0 ), . . . , (tn , yn )
Want to find the line of best fit through these
points.

ECE 580Basic Optimization Problem p. 7/17

Example: Linear regression

Given points on the plane:

(t0 , y0 ), . . . , (tn , yn )
Want to find the line of best fit through these
points.
Best = minimize the average squared error.

ECE 580Basic Optimization Problem p. 7/17

Minimizing the average squared error

16

14 mt2+c-y2

12

10

6 mt1+c-y1

2 mto+c-yo

0
0 1 2 3 4 5 t

ECE 580Basic Optimization Problem p. 8/17

Line of best fit

Equation of line: y = mt + c;

ECE 580Basic Optimization Problem p. 9/17

Line of best fit

Equation of line: y = mt + c;
Optimization problem: Find m and c to
n
1 X 2
min (mti + c yi )
n i=0

ECE 580Basic Optimization Problem p. 9/17

Line of best fit

Equation of line: y = mt + c;
Optimization problem: Find m and c to
n
1 X 2
min (mti + c yi )
n i=0

Solution: In this case we can find the solution

analytically (using least-squares theory).

ECE 580Basic Optimization Problem p. 9/17

Line of best fit

Equation of line: y = mt + c;
Optimization problem: Find m and c to
n
1 X 2
min (mti + c yi )
n i=0

Solution: In this case we can find the solution

analytically (using least-squares theory).
Related application: system identification.

ECE 580Basic Optimization Problem p. 9/17

Battery charger circuit

R1 R3 R5

I1 I3 I5

I2 R2 I4 R4
30 +
Volts

Battery Battery Battery

10 6 20
Volts Volts Volts

ECE 580Basic Optimization Problem p. 10/17

Charger circuit specifications

Current I1 I2 I3 I4 I5
Upper Limit (Amps) 4 3 3 2 2
Lower Limit (Amps) 0 0 0 0 0

ECE 580Basic Optimization Problem p. 11/17

Design objective

Find I1 , . . . , I5 to maximize power transferred to batteries, that is,

max 10I2 + 6I4 + 20I5

subject to I1 = I2 + I3
I3 = I4 + I5
I1 4
I2 3
I3 3
I4 2
I5 2,
I1 , I2 , I3 , I4 , I5 0.

ECE 580Basic Optimization Problem p. 12/17

Solving example problem

This is a linear programming problem.

ECE 580Basic Optimization Problem p. 13/17

Solving example problem

This is a linear programming problem.

Solution: Use Simplex algorithm, or interior
point algorithm.

ECE 580Basic Optimization Problem p. 13/17

Example: Savings in bank

Bank interest paid monthly at rate r

(compound).

ECE 580Basic Optimization Problem p. 14/17

Example: Savings in bank

Bank interest paid monthly at rate r

(compound).
We wish to deposit some money into the bank
every month for n months, such that the total
does not exceed D dollars.

ECE 580Basic Optimization Problem p. 14/17

Example: Savings in bank

Bank interest paid monthly at rate r

(compound).
We wish to deposit some money into the bank
every month for n months, such that the total
does not exceed D dollars.
Goal: maximize the total amount of money
accumulated at the end of n months.

ECE 580Basic Optimization Problem p. 14/17

Example: Savings in bank

Bank interest paid monthly at rate r

(compound).
We wish to deposit some money into the bank
every month for n months, such that the total
does not exceed D dollars.
Goal: maximize the total amount of money
accumulated at the end of n months.
Let xi be amount deposited in beginning of ith
month

ECE 580Basic Optimization Problem p. 14/17

Optimization problem

max (1 + r)n x1 + (1 + r)n1 x2 +

+ (1 + r)xn
subject to x1 + + xn D
x1 , . . . , xn 0

ECE 580Basic Optimization Problem p. 15/17

Inverted pendulum system

u
M

ECE 580Basic Optimization Problem p. 16/17

Optimal control of inverted pendulum system

Wish to move the cart to a given position in 1

sec., and balance the pendulum.

ECE 580Basic Optimization Problem p. 17/17

Optimal control of inverted pendulum system

Wish to move the cart to a given position in 1

sec., and balance the pendulum.
Many possible control actions.

ECE 580Basic Optimization Problem p. 17/17

Optimal control of inverted pendulum system

Wish to move the cart to a given position in 1

sec., and balance the pendulum.
Many possible control actions.
Choose one that minimizes control energy.

ECE 580Basic Optimization Problem p. 17/17

Optimal control of inverted pendulum system

Wish to move the cart to a given position in 1

sec., and balance the pendulum.
Many possible control actions.
Choose one that minimizes control energy.
What if control action is bounded?

ECE 580Basic Optimization Problem p. 17/17