EC400B Math for Micro: Tools for Optimization

EC400B Math for Micro: Lecture 1
Francesco Nava
September 2014
Nava (LSE)
EC400B Lecture 1
September 2014
1 / 27
Course Outline
Lecture 1: Tools for optimization (Quadratic forms).
Lecture 2: Tools for optimization (Taylors expansion) and
Unconstrained optimization.
Lecture 3: Concavity, convexity, and quasi-concavity.
Lecture 4: Constrained Optimization I: Equality Constraints,
Lagrange Theorem.
Lecture 5: Constrained Optimization II: Inequality Constraints,
Kuhn-Tucker Theorem.
Lecture 6: Constrained Optimization III: The Maximum Value
Function, Envelope Theorem, Implicit Function Theorem.
Nava (LSE)
EC400B Lecture 1
September 2014
2 / 27
Course Outline
Lagrange Theorem.
Nava (LSE)
EC400B Lecture 1
September 2014
2 / 27
Course Outline
Lagrange Theorem.
Nava (LSE)
EC400B Lecture 1
September 2014
2 / 27
Course Outline
Lagrange Theorem.
Nava (LSE)
EC400B Lecture 1
September 2014
2 / 27
Course Outline
Lagrange Theorem.
Nava (LSE)
EC400B Lecture 1
September 2014
2 / 27
Course Outline
Lagrange Theorem.
Nava (LSE)
EC400B Lecture 1
September 2014
2 / 27
Administration
Contact Info: LIF 3.20, Extension 6353, f.nava@econ.lse.ac.uk
Office Hours:
Thu 11 September 10:00-11:20 a.m.
Fri 12 September 10:00-11:20 a.m.
Mon 15 September 10:00-11:20 a.m.
Tue 16 September 10:00-11:20 a.m.
Wed 17 September 10:00-11:20 a.m.
Thu 18 September 10:00-11:20 a.m.
Wed 24 September 11:00-12:20 a.m.
or by appointment (e-mail f.nava@econ.lse.ac.uk).

Course Material: available on Moodle
Nava (LSE)
EC400B Lecture 1
September 2014
3 / 27
Suggested Textbooks
Knut Sydsaeter, Peter Hammond, Atle Seierstad and Arne Strom
Further Mathematics for Economic Analysis.
Alpha C. Chiang Fundamental Methods of Mathematical Economics.
Carl P. Simon and Lawrence E. Blume Mathematics for Economists.
Morton I. Kamien and Nancy L. Schwartz Dynamic Optimization:
The Calculus of Variations and Optimal Control in Economics and
Management.
Akira Takayama Mathematical Economics.
Nava (LSE)
EC400B Lecture 1
September 2014
4 / 27
What is a quadratic form?

Quadratic forms are useful because:
(i) The simplest functions after linear ones;

(ii) Conditions for optimization techniques are stated in terms of
quadratic forms;
(iii) Economic optimization problems have a quadratic objective function,
such as risk minimization problems in finance, where riskiness is
measured by the quadratic variance of the returns from investments.
Nava (LSE)
EC400B Lecture 1
September 2014
5 / 27
What is a Quadratic Form?

Among the functions of one variable, the simplest functions with a
unique global extremum are the quadratic forms:
y = x2
and
y = x 2
The level curve of a general quadratic form in R2 is

a11 x12 + a12 x1 x2 + a22 x22 = b
and can take the form of an ellipse, a hyperbola, a pair of lines, a
point, or possibly, the empty set.
Nava (LSE)
EC400B Lecture 1
September 2014
6 / 27
What is a Quadratic Form?

Among the functions of one variable, the simplest functions with a
unique global extremum are the quadratic forms:
y = x2
and
y = x 2
The level curve of a general quadratic form in R2 is

a11 x12 + a12 x1 x2 + a22 x22 = b
and can take the form of an ellipse, a hyperbola, a pair of lines, a
point, or possibly, the empty set.
Nava (LSE)
EC400B Lecture 1
September 2014
6 / 27
Definition of Quadratic Form

Definition
A quadratic form on Rn is a real valued function
X
Q(x1 , x2 , ..., xn ) =
aij xi xj
ij
The general quadratic form of

a11 x12 + a12 x1 x2 + a22 x22
can be written (non uniquely) as:
x1 x2
Nava (LSE)
a11 a12
0 a22
EC400B Lecture 1

x1
x2
September 2014
7 / 27

Definition
A quadratic form on Rn is a real valued function
X
Q(x1 , x2 , ..., xn ) =
aij xi xj
ij
The general quadratic form of

a11 x12 + a12 x1 x2 + a22 x22
can be written (non uniquely) as:
x1 x2
Nava (LSE)
a11 a12
0 a22
EC400B Lecture 1

x1
x2
September 2014
7 / 27

Each quadratic form can be represented as
Q(x) = xT A x
where A is a unique symmetric matrix:
a11 a12 /2 ... a1n /2

a12 /2 a22
... a2n /2
..
..
..
..
.
.
.
.
a1n /2 a2n /2 ...
ann
Conversely, if A is a symmetric matrix, then the real valued function
Q(x) = xT A x, is a quadratic form.
Nava (LSE)
EC400B Lecture 1
September 2014
8 / 27
Definiteness of Quadratic Forms

A quadratic form always takes the value 0 when x = 0.
We want to know whether x = 0 is a max, a min, or neither.
A Simple Example:
Consider the quadratic form:
y = ax 2
If a > 0, ax 2 is non negative and equals 0 only when x = 0.
If so, the quadratic form is positive definite and 0 is a global min.
If a < 0, the quadratic form is negative definite and 0 is a global max.
Nava (LSE)
EC400B Lecture 1
September 2014
9 / 27
Examples in Two Dimensions

In two dimensions, for instance:
a positive definite quadratic form is:
x12 + x22
a negative definite quadratic form is:
x12 x22
an indefinite quadratic form is:
x12 x22
since it can take both positive and negative values.
Nava (LSE)
EC400B Lecture 1
September 2014
10 / 27
Semidefiniteness of Quadratic Forms

There are two intermediate cases:
1
if a quadratic form is always non negative, but also equals zero for
some x 6= 0, it is called positive semidefinite.
if a quadratic form is never positive, but can be zero at points other

than the origin, it is called negative semidefinite.
An example of a positive semidefinite quadratic form is:

(x1 + x2 )2
which can be 0 for points such that x1 = x2 .
Nava (LSE)
EC400B Lecture 1
September 2014
11 / 27
Definiteness and Semidefiniteness of Symmetric Matrixes

We apply the same terminology to the symmetric matrix A, such that:
Q(x) = xT A x
Definition
Let A be an (n n) symmetric matrix. Then A is:
positive definite if xT A x > 0 for all x Rn \{0}
positive semidefinite if xT A x 0 for all x Rn \{0}
negative definite if xT A x < 0 for all x Rn \{0}
negative semidefinite if xT A x 0 for all x Rn \{0}
indefinite if xT A x > 0 and yT A y < 0 for some x, y Rn \{0}
Nava (LSE)
EC400B Lecture 1
September 2014
12 / 27
Application (More on This Later)
A function f (x) of one variable is concave on some interval if its

second derivative f 00 (x) 0 on that interval.
The generalization of this result to higher dimensions states that a

function is concave on some region if its matrix of second derivatives
(Hessian matrix) is negative semidefinite for all x in the region.
Nava (LSE)
EC400B Lecture 1
September 2014
13 / 27
The Determinant
Definition
The determinant of a matrix is a unique scalar associated with the matrix.
The determinant of a (2 2) matrix:

A=
a11 a12
a21 a22
is computed as follows:
det(A) = |A| = a11 a22 a12 a21
Nava (LSE)
EC400B Lecture 1
September 2014
14 / 27
The Determinant
The determinant of a (3 3) matrix:
a11 a12 a13

A = a21 a22 a23
a31 a32 a33
is computed as follows:

a
a
|A| = a11 22 23
a32 a33
Nava (LSE)

a12 a21 a23

a31 a33
EC400B Lecture 1

+ a13 a21 a22

a31 a32
September 2014
15 / 27
Principal Submatrices and Principal Minors
Definition
Let A be a (n n) matrix. The (k k) submatrix of A formed by deleting
n k columns, say columns i1 , i2 , ..., ink and the same n k rows from
A, i1 , i2 , ..., ink , is called a k th order principal submatrix of A.
Definition
The determinant of a (k k) principal submatrix is called a k th order
principal minor of A.
Nava (LSE)
EC400B Lecture 1
September 2014
16 / 27
Example Principal Minors

Consider a general (3 3) matrix A:
There is one third order principal minor: |A|.
There are three second order principal minors:

a11 a12
a11 a13

,

a21 a22
a31 a33 ,

a22 a23

a32 a33
There are three first order principal minors: a11 , a22 and a33 .
Nava (LSE)
EC400B Lecture 1
September 2014
17 / 27
Leading Submatrices and Leading Minors
Definition
Let A be a (n n) matrix. The k th order principal submatrix of A
obtained by deleting the last n k rows and columns from A is called the
k th order leading principal submatrix of A denoted Ak .
Definition
The determinant of the k th order leading principal submatrix is called the
k th order leading principal minor of A denoted |Ak |.
Nava (LSE)
EC400B Lecture 1
September 2014
18 / 27
Testing Definiteness
Let A be an (n n) symmetric matrix. If so:
A is positive definite if and only if all its n leading principal minors are
strictly positive:
|A1 | > 0, |A2 | > 0, |A3 | > 0 . . .
A is negative definite if and only if all its n leading principal minors

alternate in sign as follows:
|A1 | < 0, |A2 | > 0, |A3 | < 0 . . .
the k th order leading principal minor having the same sign of (1)k .
Nava (LSE)
EC400B Lecture 1
September 2014
19 / 27
Let A be an (n n) symmetric matrix. If so:
A is positive definite if and only if all its n leading principal minors are
strictly positive:
|A1 | > 0, |A2 | > 0, |A3 | > 0 . . .
A is negative definite if and only if all its n leading principal minors

alternate in sign as follows:
|A1 | < 0, |A2 | > 0, |A3 | < 0 . . .
the k th order leading principal minor having the same sign of (1)k .
Nava (LSE)
EC400B Lecture 1
September 2014
19 / 27
A is positive semidefinite if and only if every principal minor of A is

non-negative:
|A1 | 0, |A2 | 0, |A3 | 0 . . .
A is negative semidefinite if and only if every principal minor of odd

order is non-positive and every principal minor of even order is
non-negative:
|A1 | 0, |A2 | 0, |A3 | 0 . . .
Nava (LSE)
EC400B Lecture 1
September 2014
20 / 27
A is positive semidefinite if and only if every principal minor of A is

non-negative:
|A1 | 0, |A2 | 0, |A3 | 0 . . .
A is negative semidefinite if and only if every principal minor of odd

order is non-positive and every principal minor of even order is
non-negative:
|A1 | 0, |A2 | 0, |A3 | 0 . . .
Nava (LSE)
EC400B Lecture 1
September 2014
20 / 27
Special Case: Diagonal Matrixes
Consider the following (3 3) diagonal
a1 0
A = 0 a2
0 0
matrix A:
0
0
a3
A also corresponds to the simplest quadratic forms:

a1 x12 + a2 x22 + a3 x32 .
Nava (LSE)
EC400B Lecture 1
September 2014
21 / 27

Such quadratics forms are:
positive definite if and only if all the ai s are positive.
negative definite if and only if all the ai s are negative.
positive semidefinite if and only if all the ai s are non-negative.
negative semidefinite if and only if all the ai s are non-positive.
indefinite if there are two ai s of opposite sign.
Nava (LSE)
EC400B Lecture 1
September 2014
22 / 27

Nava (LSE)
EC400B Lecture 1
September 2014
22 / 27

Nava (LSE)
EC400B Lecture 1
September 2014
22 / 27
Special Case: (2 2) Matrixes

Consider the (2 2) symmetric matrix:

a b
A=
b c
The associated quadratic form is:

Q(x1 , x2 ) = xT A x =
x1 x2
a b
b c

x1
x2

=
ax12 + 2bx1 x2 + cx22
Nava (LSE)
EC400B Lecture 1
September 2014
23 / 27

Consider the (2 2) symmetric matrix:

a b
A=
b c
The associated quadratic form is:

Q(x1 , x2 ) = xT A x =
x1 x2
a b
b c

x1
x2

=
ax12 + 2bx1 x2 + cx22
Nava (LSE)
EC400B Lecture 1
September 2014
23 / 27
If a = 0, then Q cannot be definite since Q(1, 0) = 0.
If a 6= 0, add and subtract b 2 x22 /a to get:

b2
b2
Q(x1 , x2 ) = ax12 + 2bx1 x2 + cx22 + x22 x22
a
a

2
2
b
2bx1 x2 b 2
+ 2 x2 x22 + cx22
= a x12 +
a
a
a

2

ac b 2
b
= a x1 + x2 +
x22
a
a
Nava (LSE)
EC400B Lecture 1
September 2014
24 / 27
If a = 0, then Q cannot be definite since Q(1, 0) = 0.
If a 6= 0, add and subtract b 2 x22 /a to get:

b2
b2
Q(x1 , x2 ) = ax12 + 2bx1 x2 + cx22 + x22 x22
a
a

2
2
b
2bx1 x2 b 2
+ 2 x2 x22 + cx22
= a x12 +
a
a
a

2

ac b 2
b
= a x1 + x2 +
x22
a
a
Nava (LSE)
EC400B Lecture 1
September 2014
24 / 27

If coefficients a and (ac b 2 )/a are positive, Q cannot be negative.
It will equal 0 only when x1 + b x2 /a = 0 and x2 = 0 (ie x1 = x2 = 0).
Thus Q(x1 , x2 ) is positive definite since:

a b

>0
|a| > 0 and det(A) =
b c
Conversely, in order for Q to be positive definite, we need both:

a b

>0
b c
Nava (LSE)
EC400B Lecture 1
September 2014
25 / 27


a b

>0
b c

a b

>0
b c
Nava (LSE)
EC400B Lecture 1
September 2014
25 / 27


a b

>0
b c

a b

>0
b c
Nava (LSE)
EC400B Lecture 1
September 2014
25 / 27
Similarly, Q is negative definite if and only if both coefficients are

negative, which occurs if and only if:

a < 0 and ac b 2 > 0
That is, when the leading principal minors alternative in sign.

If ac b 2 < 0, the two coefficients will have opposite signs and
Q(x1 , x2 ) will be indefinite.
Nava (LSE)
EC400B Lecture 1
September 2014
26 / 27
Similarly, Q is negative definite if and only if both coefficients are

negative, which occurs if and only if:

a < 0 and ac b 2 > 0
That is, when the leading principal minors alternative in sign.

If ac b 2 < 0, the two coefficients will have opposite signs and
Q(x1 , x2 ) will be indefinite.
Nava (LSE)
EC400B Lecture 1
September 2014
26 / 27
Examples of (2 2) Matrices
Consider a symmetric matrix:

A=
2 3
3 7
Since |A1 | = 2 and |A2 | = 5, A is positive definite.


B=
2 4
4 7
Since |B1 | = 2 and |B2 | = 2, B is indefinite.
Nava (LSE)
EC400B Lecture 1
September 2014
27 / 27
Examples of (2 2) Matrices

A=
2 3
3 7
Since |A1 | = 2 and |A2 | = 5, A is positive definite.


B=
2 4
4 7
Since |B1 | = 2 and |B2 | = 2, B is indefinite.
Nava (LSE)
EC400B Lecture 1
September 2014
27 / 27

EC400B Math for Micro: Tools for Optimization

Diunggah oleh

Informasi Dokumen

Deskripsi Asli:

Judul Asli

Hak Cipta

Format Tersedia

Bagikan dokumen Ini

Bagikan atau Tanam Dokumen

Opsi Berbagi

Apakah menurut Anda dokumen ini bermanfaat?

Apakah konten ini tidak pantas?

Hak Cipta:

Format Tersedia

EC400B Math for Micro: Tools for Optimization

Diunggah oleh

Hak Cipta:

Format Tersedia

EC400B Math for Micro: Lecture 1

or by appointment (e-mail f.nava@econ.lse.ac.uk).

What is a quadratic form?

(i) The simplest functions after linear ones;

What is a Quadratic Form?

The level curve of a general quadratic form in R2 is

What is a Quadratic Form?

The level curve of a general quadratic form in R2 is

Definition of Quadratic Form

The general quadratic form of

Definition of Quadratic Form

The general quadratic form of

Definition of Quadratic Form

a11 a12 /2 ... a1n /2

Definiteness of Quadratic Forms

Examples in Two Dimensions

Semidefiniteness of Quadratic Forms

if a quadratic form is never positive, but can be zero at points other

An example of a positive semidefinite quadratic form is:

Definiteness and Semidefiniteness of Symmetric Matrixes

Application (More on This Later)

A function f (x) of one variable is concave on some interval if its

The generalization of this result to higher dimensions states that a

a11 a12 a13

Principal Submatrices and Principal Minors

Example Principal Minors

Leading Submatrices and Leading Minors

A is negative definite if and only if all its n leading principal minors

A is negative definite if and only if all its n leading principal minors

A is positive semidefinite if and only if every principal minor of A is

A is negative semidefinite if and only if every principal minor of odd

A is positive semidefinite if and only if every principal minor of A is

A is negative semidefinite if and only if every principal minor of odd

Special Case: Diagonal Matrixes

Consider the following (3 3) diagonal

A also corresponds to the simplest quadratic forms:

Special Case: Diagonal Matrixes

Special Case: Diagonal Matrixes

Special Case: Diagonal Matrixes

Special Case: (2 2) Matrixes

The associated quadratic form is:

ax12 + 2bx1 x2 + cx22

Special Case: (2 2) Matrixes

The associated quadratic form is:

ax12 + 2bx1 x2 + cx22

Special Case: (2 2) Matrixes

If a = 0, then Q cannot be definite since Q(1, 0) = 0.

If a 6= 0, add and subtract b 2 x22 /a to get:

Special Case: (2 2) Matrixes

If a = 0, then Q cannot be definite since Q(1, 0) = 0.

If a 6= 0, add and subtract b 2 x22 /a to get:

Special Case: (2 2) Matrixes

Conversely, in order for Q to be positive definite, we need both:

Special Case: (2 2) Matrixes

Conversely, in order for Q to be positive definite, we need both:

Special Case: (2 2) Matrixes

Conversely, in order for Q to be positive definite, we need both:

Special Case: (2 2) Matrixes

Similarly, Q is negative definite if and only if both coefficients are

Special Case: (2 2) Matrixes

Similarly, Q is negative definite if and only if both coefficients are

Consider a symmetric matrix: