Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms

and Matrix Norms Mean Value Theorem

Multivariable Calculus Review
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Multi-Variable Calculus
Point-Set Topology
Compactness
The Weierstrass Extreme Value Theorem
Operator and Matrix Norms
Mean Value Theorem
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Multi-Variable Calculus
Norms:
A function : R
n
R is a vector norm on R
n
if

(x) 0 x R
n
with equality i x = 0.

(x) = [[(x) x R
n
R

(x + y) (x) + (y) x, y R
n
We usually denote (x) by |x|. Norms are convex functions.
l
p
norms
|x|
p
:= (

n
i =1
[x
i
[
p
)
1
p
, 1 p <
|x|

= max
i =1,...,n
[x
i
[
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Multi-Variable Calculus
Norms:
A function : R
n
R is a vector norm on R
n
if

(x) 0 x R
n
with equality i x = 0.

(x) = [[(x) x R
n
R

(x + y) (x) + (y) x, y R
n
We usually denote (x) by |x|. Norms are convex functions.
l
p
norms
|x|
p
:= (

n
i =1
[x
i
[
p
)
1
p
, 1 p <
|x|

= max
i =1,...,n
[x
i
[
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Multi-Variable Calculus
Norms:
A function : R
n
R is a vector norm on R
n
if

(x) 0 x R
n
with equality i x = 0.

(x) = [[(x) x R
n
R

(x + y) (x) + (y) x, y R
n
We usually denote (x) by |x|. Norms are convex functions.
l
p
norms
|x|
p
:= (

n
i =1
[x
i
[
p
)
1
p
, 1 p <
|x|

= max
i =1,...,n
[x
i
[
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Multi-Variable Calculus
Norms:
A function : R
n
R is a vector norm on R
n
if

(x) 0 x R
n
with equality i x = 0.

(x) = [[(x) x R
n
R

(x + y) (x) + (y) x, y R
n
We usually denote (x) by |x|. Norms are convex functions.
l
p
norms
|x|
p
:= (

n
i =1
[x
i
[
p
)
1
p
, 1 p <
|x|

= max
i =1,...,n
[x
i
[
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Multi-Variable Calculus
Norms:
A function : R
n
R is a vector norm on R
n
if

(x) 0 x R
n
with equality i x = 0.

(x) = [[(x) x R
n
R

(x + y) (x) + (y) x, y R
n
We usually denote (x) by |x|. Norms are convex functions.
l
p
norms
|x|
p
:= (

n
i =1
[x
i
[
p
)
1
p
, 1 p <
|x|

= max
i =1,...,n
[x
i
[
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Elementary Topological Properties of Sets
Let D R
n
.

D is open if for every x D there exists > 0 such that

x + B D where
x + B = x + u : u B
and B is the unit ball of some given norm on R
n
.

D is closed if every point x satisfying

(x + B) D ,=
for all > 0, must be a point in D.

D is bounded if there exists > 0 such that

|x| for all x D.
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Elementary Topological Properties of Sets
Let D R
n
.

D is open if for every x D there exists > 0 such that

x + B D where
x + B = x + u : u B
and B is the unit ball of some given norm on R
n
.

D is closed if every point x satisfying

(x + B) D ,=
for all > 0, must be a point in D.

D is bounded if there exists > 0 such that

|x| for all x D.
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Elementary Topological Properties of Sets
Let D R
n
.

D is open if for every x D there exists > 0 such that

x + B D where
x + B = x + u : u B
and B is the unit ball of some given norm on R
n
.

D is closed if every point x satisfying

(x + B) D ,=
for all > 0, must be a point in D.

D is bounded if there exists > 0 such that

|x| for all x D.
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Elementary Topological Properties of Sets
Let D R
n
.

D is open if for every x D there exists > 0 such that

x + B D where
x + B = x + u : u B
and B is the unit ball of some given norm on R
n
.

D is closed if every point x satisfying

(x + B) D ,=
for all > 0, must be a point in D.

D is bounded if there exists > 0 such that

|x| for all x D.
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Elementary Topological Properties of Sets: Compactness
Let D R
n
.

x D is said to be a cluster point of D if

(x + B) D ,=
for every > 0.

D is compact if it is closed and bounded.

Theorem: [Weierstrass Compactness Theorem]
A set D R
n
is compact if and only if every innite subset of D
has a cluster point and all such cluster points lie in D.
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Elementary Topological Properties of Sets: Compactness
Let D R
n
.

x D is said to be a cluster point of D if

(x + B) D ,=
for every > 0.

D is compact if it is closed and bounded.

Theorem: [Weierstrass Compactness Theorem]
A set D R
n
is compact if and only if every innite subset of D
has a cluster point and all such cluster points lie in D.
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Elementary Topological Properties of Sets: Compactness
Let D R
n
.

x D is said to be a cluster point of D if

(x + B) D ,=
for every > 0.

D is compact if it is closed and bounded.

Theorem: [Weierstrass Compactness Theorem]
A set D R
n
is compact if and only if every innite subset of D
has a cluster point and all such cluster points lie in D.
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Elementary Topological Properties of Sets: Compactness
Let D R
n
.

x D is said to be a cluster point of D if

(x + B) D ,=
for every > 0.

D is compact if it is closed and bounded.

Theorem: [Weierstrass Compactness Theorem]
A set D R
n
is compact if and only if every innite subset of D
has a cluster point and all such cluster points lie in D.
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Continuity and The Weierstrass Extreme Value Theorem
The mapping F : R
n
R
m
is continuous at the point x if
lim
xx0
|F(x) F(x)| = 0.
F is continuous on a set D R
n
if F is continuous at every point
of D.
Theorem: [Weierstrass Extreme Value Theorem]
Every continuous function on a compact set attains its extreme
values on that set.
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Continuity and The Weierstrass Extreme Value Theorem
The mapping F : R
n
R
m
is continuous at the point x if
lim
xx0
|F(x) F(x)| = 0.
F is continuous on a set D R
n
if F is continuous at every point
of D.
Theorem: [Weierstrass Extreme Value Theorem]
Every continuous function on a compact set attains its extreme
values on that set.
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Continuity and The Weierstrass Extreme Value Theorem
The mapping F : R
n
R
m
is continuous at the point x if
lim
xx0
|F(x) F(x)| = 0.
F is continuous on a set D R
n
if F is continuous at every point
of D.
Theorem: [Weierstrass Extreme Value Theorem]
Every continuous function on a compact set attains its extreme
values on that set.
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Operator Norms
A R
mn
|A|
(p,q)
= max|Ax|
p
: |x|
q
1
Examples: |A|
2
= |A|
(2,2)
= max|Ax|
2
: |x|
2
1
|A|

= |A|
(,)
= max|Ax|

: |x|

1
= max
1i m

n
j =1
[a
ij
[ (max row sum)
|A|
1
= |A|
(1,1)
= max|Ax|
1
: |x|
1
1
= max
1j n

m
i =1
[a
ij
[ (max column sum)
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Operator Norms
A R
mn
|A|
(p,q)
= max|Ax|
p
: |x|
q
1
Examples:
|A|
2
= |A|
(2,2)
= max|Ax|
2
: |x|
2
1
|A|

= |A|
(,)
= max|Ax|

: |x|

1
= max
1i m

n
j =1
[a
ij
[ (max row sum)
|A|
1
= |A|
(1,1)
= max|Ax|
1
: |x|
1
1
= max
1j n

m
i =1
[a
ij
[ (max column sum)
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Operator Norms
A R
mn
|A|
(p,q)
= max|Ax|
p
: |x|
q
1
Examples: |A|
2
= |A|
(2,2)
= max|Ax|
2
: |x|
2
1
|A|

= |A|
(,)
= max|Ax|

: |x|

1
= max
1i m

n
j =1
[a
ij
[ (max row sum)
|A|
1
= |A|
(1,1)
= max|Ax|
1
: |x|
1
1
= max
1j n

m
i =1
[a
ij
[ (max column sum)
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Operator Norms
A R
mn
|A|
(p,q)
= max|Ax|
p
: |x|
q
1
Examples: |A|
2
= |A|
(2,2)
= max|Ax|
2
: |x|
2
1
|A|

= |A|
(,)
= max|Ax|

: |x|

1
= max
1i m

n
j =1
[a
ij
[ (max row sum)
|A|
1
= |A|
(1,1)
= max|Ax|
1
: |x|
1
1
= max
1j n

m
i =1
[a
ij
[ (max column sum)
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Operator Norms
A R
mn
|A|
(p,q)
= max|Ax|
p
: |x|
q
1
Examples: |A|
2
= |A|
(2,2)
= max|Ax|
2
: |x|
2
1
|A|

= |A|
(,)
= max|Ax|

: |x|

1
= max
1i m

n
j =1
[a
ij
[ (max row sum)
|A|
1
= |A|
(1,1)
= max|Ax|
1
: |x|
1
1
= max
1j n

m
i =1
[a
ij
[ (max column sum)
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Operator Norms
Properties:

|Ax|
p
|A|
(p,q)
|x|
q
.

|A| 0 with equality |Ax| = 0 x or A 0.

|A| = max|Ax| : |x| 1

= max[[ |Ax| : || 1 = [[ |A|

|A + B| = max|Ax + Bx| : |x| 1 max|Ax| +|Bx| A 1

= max|Ax
1
| +|Bx
2
| : x
1
= x
2
, |x
1
| 1, |x
2
| 1
max|Ax
1
| +|Bx
2
| : |x
1
| 1, |x
2
| 1
= |A| +|B|
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Operator Norms
Properties:

|Ax|
p
|A|
(p,q)
|x|
q
.

|A| 0 with equality |Ax| = 0 x or A 0.

|A| = max|Ax| : |x| 1

= max[[ |Ax| : || 1 = [[ |A|

|A + B| = max|Ax + Bx| : |x| 1 max|Ax| +|Bx| A 1

= max|Ax
1
| +|Bx
2
| : x
1
= x
2
, |x
1
| 1, |x
2
| 1
max|Ax
1
| +|Bx
2
| : |x
1
| 1, |x
2
| 1
= |A| +|B|
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Operator Norms
Properties:

|Ax|
p
|A|
(p,q)
|x|
q
.

|A| 0 with equality |Ax| = 0 x or A 0.

|A| = max|Ax| : |x| 1

= max[[ |Ax| : || 1 = [[ |A|

|A + B| = max|Ax + Bx| : |x| 1 max|Ax| +|Bx| A 1

= max|Ax
1
| +|Bx
2
| : x
1
= x
2
, |x
1
| 1, |x
2
| 1
max|Ax
1
| +|Bx
2
| : |x
1
| 1, |x
2
| 1
= |A| +|B|
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Operator Norms
Properties:

|Ax|
p
|A|
(p,q)
|x|
q
.

|A| 0 with equality |Ax| = 0 x or A 0.

|A| = max|Ax| : |x| 1

= max[[ |Ax| : || 1 = [[ |A|

|A + B| = max|Ax + Bx| : |x| 1 max|Ax| +|Bx| A 1

= max|Ax
1
| +|Bx
2
| : x
1
= x
2
, |x
1
| 1, |x
2
| 1
max|Ax
1
| +|Bx
2
| : |x
1
| 1, |x
2
| 1
= |A| +|B|
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Operator Norms
Properties:

|Ax|
p
|A|
(p,q)
|x|
q
.

|A| 0 with equality |Ax| = 0 x or A 0.

|A| = max|Ax| : |x| 1

= max[[ |Ax| : || 1 = [[ |A|

|A + B| = max|Ax + Bx| : |x| 1 max|Ax| +|Bx| A 1

= max|Ax
1
| +|Bx
2
| : x
1
= x
2
, |x
1
| 1, |x
2
| 1
max|Ax
1
| +|Bx
2
| : |x
1
| 1, |x
2
| 1
= |A| +|B|
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Frobenius Norm and Inner Product
A R
mn
|A|
F
=

trace(A
T
A)

1/2
Identify R
mn
with R
(mn)
by simply stacking the columns of a
matrix one on top of the other to create a very long vector in
R
(mn)
.
This mapping takes a matrix in R
mn
to a vector in R
(mn)
by
stacking columns. It is called vec (or sometimes cvec).
Using vec we can dene an inner product on R
mn
(called the
Frobenius inner prodiuct) by writting
A, B
F
= vec (A)
T
vec (B) = trace(A
T
B) .
Note |A|
2
F
= A, A
F
.
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Frobenius Norm and Inner Product
A R
mn
|A|
F
=

trace(A
T
A)

1/2
Identify R
mn
with R
(mn)
by simply stacking the columns of a
matrix one on top of the other to create a very long vector in
R
(mn)
.
This mapping takes a matrix in R
mn
to a vector in R
(mn)
by
stacking columns. It is called vec (or sometimes cvec).
Using vec we can dene an inner product on R
mn
(called the
Frobenius inner prodiuct) by writting
A, B
F
= vec (A)
T
vec (B) = trace(A
T
B) .
Note |A|
2
F
= A, A
F
.
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Frobenius Norm and Inner Product
A R
mn
|A|
F
=

trace(A
T
A)

1/2
Identify R
mn
with R
(mn)
by simply stacking the columns of a
matrix one on top of the other to create a very long vector in
R
(mn)
.
This mapping takes a matrix in R
mn
to a vector in R
(mn)
by
stacking columns. It is called vec (or sometimes cvec).
Using vec we can dene an inner product on R
mn
(called the
Frobenius inner prodiuct) by writting
A, B
F
= vec (A)
T
vec (B) = trace(A
T
B) .
Note |A|
2
F
= A, A
F
.
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Frobenius Norm and Inner Product
A R
mn
|A|
F
=

trace(A
T
A)

1/2
Identify R
mn
with R
(mn)
by simply stacking the columns of a
matrix one on top of the other to create a very long vector in
R
(mn)
.
This mapping takes a matrix in R
mn
to a vector in R
(mn)
by
stacking columns. It is called vec (or sometimes cvec).
Using vec we can dene an inner product on R
mn
(called the
Frobenius inner prodiuct) by writting
A, B
F
= vec (A)
T
vec (B) = trace(A
T
B) .
Note |A|
2
F
= A, A
F
.
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Frobenius Norm and Inner Product
A R
mn
|A|
F
=

trace(A
T
A)

1/2
Identify R
mn
with R
(mn)
by simply stacking the columns of a
matrix one on top of the other to create a very long vector in
R
(mn)
.
This mapping takes a matrix in R
mn
to a vector in R
(mn)
by
stacking columns. It is called vec (or sometimes cvec).
Using vec we can dene an inner product on R
mn
(called the
Frobenius inner prodiuct) by writting
A, B
F
= vec (A)
T
vec (B) = trace(A
T
B) .
Note |A|
2
F
= A, A
F
.
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Dierentiation
Let F : R
n
R
m
, and let F
i
denote the i th component functioon of F:
F(x) =

F
1
(x)
F
2
(x)
.
.
.
F
m
(x)

,
where each F
i
is a mapping from R
n
to R
m
.
If the limit
lim
t0
F(x + td) F(x)
t
=: F

(x; d)
exists, it is called the directional derivative of F at x in the direction h.
If this limit exists for all d R
n
and is linear in the d argument,
F

(x; d
1
+ d
2
) = F

(x; d
1
) + F

(x; d
2
),
then F is said to be dierentiable at x, and denote the associated linear
operator by F

(x).
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Dierentiation
Let F : R
n
R
m
, and let F
i
denote the i th component functioon of F:
F(x) =

F
1
(x)
F
2
(x)
.
.
.
F
m
(x)

,
where each F
i
is a mapping from R
n
to R
m
.
If the limit
lim
t0
F(x + td) F(x)
t
=: F

(x; d)
exists, it is called the directional derivative of F at x in the direction h.
If this limit exists for all d R
n
and is linear in the d argument,
F

(x; d
1
+ d
2
) = F

(x; d
1
) + F

(x; d
2
),
then F is said to be dierentiable at x, and denote the associated linear
operator by F

(x).
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Dierentiation
Let F : R
n
R
m
, and let F
i
denote the i th component functioon of F:
F(x) =

F
1
(x)
F
2
(x)
.
.
.
F
m
(x)

,
where each F
i
is a mapping from R
n
to R
m
.
If the limit
lim
t0
F(x + td) F(x)
t
=: F

(x; d)
exists, it is called the directional derivative of F at x in the direction h.
If this limit exists for all d R
n
and is linear in the d argument,
F

(x; d
1
+ d
2
) = F

(x; d
1
) + F

(x; d
2
),
then F is said to be dierentiable at x, and denote the associated linear
operator by F

(x).
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Dierentiation
One can show that if F

(x) exists, then

lim
yx0
|F(y) (F(x) + F

(x)(y x))|
|y x|
= 0,
or equivalently,
F(y) = F(x) + F

(x)(y x) + o(|y x|),

where o(t) is a function satisfying
lim
t0
o(t)
t
= 0 .
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Dierentiation
One can show that if F

(x) exists, then

lim
yx0
|F(y) (F(x) + F

(x)(y x))|
|y x|
= 0,
or equivalently,
F(y) = F(x) + F

(x)(y x) + o(|y x|),

where o(t) is a function satisfying
lim
t0
o(t)
t
= 0 .
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Dierentiation
One can show that if F

(x) exists, then

lim
yx0
|F(y) (F(x) + F

(x)(y x))|
|y x|
= 0,
or equivalently,
F(y) = F(x) + F

(x)(y x) + o(|y x|),

where o(t) is a function satisfying
lim
t0
o(t)
t
= 0 .
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Dierentiation Facts
(i) If F

(x) exists, it is unique.

(ii) If F

(x) exists, then F

(x; d) exists for all d and F

(x; d) = F

(x)d.
(iii) If F

(x) exists, then F is continuous at x.

(iv) (Matrix Representation) Suppose F

(x) is continuous at x, Then

F

(x) = F(x) =

F
1
x
1
F
1
x
2

F
1
x
n
F
2
x
1
F
2
x
2

F
2
x
n
.
.
.
F
n
x
1

F
m
x
n

F
1
( x)
T
F
2
( x)
T
.
.
.
F
m
( x)
T

,
where each partial derivative is evaluated at x = (x
1
, . . . , x
n
)
T
.
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Dierentiation Facts
(i) If F

(x) exists, it is unique.

(ii) If F

(x) exists, then F

(x; d) exists for all d and F

(x; d) = F

(x)d.
(iii) If F

(x) exists, then F is continuous at x.

(iv) (Matrix Representation) Suppose F

(x) is continuous at x, Then

F

(x) = F(x) =

F
1
x
1
F
1
x
2

F
1
x
n
F
2
x
1
F
2
x
2

F
2
x
n
.
.
.
F
n
x
1

F
m
x
n

F
1
( x)
T
F
2
( x)
T
.
.
.
F
m
( x)
T

,
where each partial derivative is evaluated at x = (x
1
, . . . , x
n
)
T
.
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Dierentiation Facts
(i) If F

(x) exists, it is unique.

(ii) If F

(x) exists, then F

(x; d) exists for all d and F

(x; d) = F

(x)d.
(iii) If F

(x) exists, then F is continuous at x.

(iv) (Matrix Representation) Suppose F

(x) is continuous at x, Then

F

(x) = F(x) =

F
1
x
1
F
1
x
2

F
1
x
n
F
2
x
1
F
2
x
2

F
2
x
n
.
.
.
F
n
x
1

F
m
x
n

F
1
( x)
T
F
2
( x)
T
.
.
.
F
m
( x)
T

,
where each partial derivative is evaluated at x = (x
1
, . . . , x
n
)
T
.
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Dierentiation Facts
(i) If F

(x) exists, it is unique.

(ii) If F

(x) exists, then F

(x; d) exists for all d and F

(x; d) = F

(x)d.
(iii) If F

(x) exists, then F is continuous at x.

(iv) (Matrix Representation) Suppose F

(x) is continuous at x, Then

F

(x) = F(x) =

F
1
x
1
F
1
x
2

F
1
x
n
F
2
x
1
F
2
x
2

F
2
x
n
.
.
.
F
n
x
1

F
m
x
n

F
1
( x)
T
F
2
( x)
T
.
.
.
F
m
( x)
T

,
where each partial derivative is evaluated at x = (x
1
, . . . , x
n
)
T
.
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Dierentiation Facts
(i) If F

(x) exists, it is unique.

(ii) If F

(x) exists, then F

(x; d) exists for all d and F

(x; d) = F

(x)d.
(iii) If F

(x) exists, then F is continuous at x.

(iv) (Matrix Representation) Suppose F

(x) is continuous at x, Then

F

(x) = F(x) =

F
1
x
1
F
1
x
2

F
1
x
n
F
2
x
1
F
2
x
2

F
2
x
n
.
.
.
F
n
x
1

F
m
x
n

F
1
( x)
T
F
2
( x)
T
.
.
.
F
m
( x)
T

,
where each partial derivative is evaluated at x = (x
1
, . . . , x
n
)
T
.
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Chain Rule
Let F : A R
m
R
k
be dierentiable on the open set /.
Let G : B R
k
R
n
be dierentiable on the open set B.
If F(A) B, then the composite function G F is dierentiable on
/ and
(G F)

(x) = G

(F(x)) F

(x).
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Chain Rule
Let F : A R
m
R
k
be dierentiable on the open set /.
Let G : B R
k
R
n
be dierentiable on the open set B.
If F(A) B, then the composite function G F is dierentiable on
/ and
(G F)

(x) = G

(F(x)) F

(x).
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Chain Rule
Let F : A R
m
R
k
be dierentiable on the open set /.
Let G : B R
k
R
n
be dierentiable on the open set B.
If F(A) B, then the composite function G F is dierentiable on
/ and
(G F)

(x) = G

(F(x)) F

(x).
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
The Mean Value Theorem
(a) If f : R R is dierentiable, then for every x, y R there
exists z between x and y such that
f (y) = f (x) + f

(z)(y x).
(b) If f : R
n
R is dierentiable, then for every x, y R there is
a z [x, y] such that
f (y) = f (x) +f (z)
T
(y x).
(c) If F : R
n
R
m
continuously dierentiable, then for every
x, y R
|F(y) F(x)|
q

sup
z[x,y]
|F

(z)|
(p,q)

|x y|
p
.
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
The Mean Value Theorem
(a) If f : R R is dierentiable, then for every x, y R there
exists z between x and y such that
f (y) = f (x) + f

(z)(y x).
(b) If f : R
n
R is dierentiable, then for every x, y R there is
a z [x, y] such that
f (y) = f (x) +f (z)
T
(y x).
(c) If F : R
n
R
m
continuously dierentiable, then for every
x, y R
|F(y) F(x)|
q

sup
z[x,y]
|F

(z)|
(p,q)

|x y|
p
.
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
The Mean Value Theorem
(a) If f : R R is dierentiable, then for every x, y R there
exists z between x and y such that
f (y) = f (x) + f

(z)(y x).
(b) If f : R
n
R is dierentiable, then for every x, y R there is
a z [x, y] such that
f (y) = f (x) +f (z)
T
(y x).
(c) If F : R
n
R
m
continuously dierentiable, then for every
x, y R
|F(y) F(x)|
q

sup
z[x,y]
|F

(z)|
(p,q)

|x y|
p
.
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
The Mean Value Theorem
(a) If f : R R is dierentiable, then for every x, y R there
exists z between x and y such that
f (y) = f (x) + f

(z)(y x).
(b) If f : R
n
R is dierentiable, then for every x, y R there is
a z [x, y] such that
f (y) = f (x) +f (z)
T
(y x).
(c) If F : R
n
R
m
continuously dierentiable, then for every
x, y R
|F(y) F(x)|
q

sup
z[x,y]
|F

(z)|
(p,q)

|x y|
p
.
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
The Mean Value Theorem: Proof
If f : R
n
R is dierentiable, then for every x, y R
n
there exists
z [x, y] such that
f (y) = f (x) +f (z)(y x).
Proof: Set (t) = f (x + t(y x)).
Chain rule

(t) = f (x + t(y x))

T
(y x).
In particular, is dierentiable.
By the 1-D MVT there exists t (0, 1) such that
(1) = (0) +

(t)(1 0),
or equivalently,
f (y) = f (x) +f (z)
T
(y x)
where z = x + t(y x).
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
The Mean Value Theorem: Proof
If f : R
n
R is dierentiable, then for every x, y R
n
there exists
z [x, y] such that
f (y) = f (x) +f (z)(y x).
Proof: Set (t) = f (x + t(y x)).
Chain rule

(t) = f (x + t(y x))

T
(y x).
In particular, is dierentiable.
By the 1-D MVT there exists t (0, 1) such that
(1) = (0) +

(t)(1 0),
or equivalently,
f (y) = f (x) +f (z)
T
(y x)
where z = x + t(y x).
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
The Mean Value Theorem: Proof
If f : R
n
R is dierentiable, then for every x, y R
n
there exists
z [x, y] such that
f (y) = f (x) +f (z)(y x).
Proof: Set (t) = f (x + t(y x)).
Chain rule

(t) = f (x + t(y x))

T
(y x).
In particular, is dierentiable.
By the 1-D MVT there exists t (0, 1) such that
(1) = (0) +

(t)(1 0),
or equivalently,
f (y) = f (x) +f (z)
T
(y x)
where z = x + t(y x).
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
The Mean Value Theorem: Proof
If f : R
n
R is dierentiable, then for every x, y R
n
there exists
z [x, y] such that
f (y) = f (x) +f (z)(y x).
Proof: Set (t) = f (x + t(y x)).
Chain rule

(t) = f (x + t(y x))

T
(y x).
In particular, is dierentiable.
By the 1-D MVT there exists t (0, 1) such that
(1) = (0) +

(t)(1 0),
or equivalently,
f (y) = f (x) +f (z)
T
(y x)
where z = x + t(y x).
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
The Mean Value Theorem: Proof
If f : R
n
R is dierentiable, then for every x, y R
n
there exists
z [x, y] such that
f (y) = f (x) +f (z)(y x).
Proof: Set (t) = f (x + t(y x)).
Chain rule

(t) = f (x + t(y x))

T
(y x).
In particular, is dierentiable.
By the 1-D MVT there exists t (0, 1) such that
(1) = (0) +

(t)(1 0),
or equivalently,
f (y) = f (x) +f (z)
T
(y x)
where z = x + t(y x).
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
The Mean Value Theorem: Proof
If f : R
n
R is dierentiable, then for every x, y R
n
there exists
z [x, y] such that
f (y) = f (x) +f (z)(y x).
Proof: Set (t) = f (x + t(y x)).
Chain rule

(t) = f (x + t(y x))

T
(y x).
In particular, is dierentiable.
By the 1-D MVT there exists t (0, 1) such that
(1) = (0) +

(t)(1 0),
or equivalently,
f (y) = f (x) +f (z)
T
(y x)
where z = x + t(y x).
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
The Second Derivative
Let f : R
n
R so that f : R
n
R
n
.
The second derivative of f is just the derivative of f as a
mapping from R
n
to R
n
:
[f (x)] =
2
f (x).
This is an n n matrix:
If f exists and is continuous at x, then it is given as the matrix
of second partials of f at x:

2
f (x) =

2
f
x
i
x
j
(x)

.
Moreover,
f
x
i
x
j
=
f
x
j
x
i
for all i , j = 1, . . . , n. The matrix

2
f (x) is called the Hessian of f at x. It is a symmetric matrix.
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
The Second Derivative
Let f : R
n
R so that f : R
n
R
n
.
The second derivative of f is just the derivative of f as a
mapping from R
n
to R
n
:
[f (x)] =
2
f (x).
This is an n n matrix:
If f exists and is continuous at x, then it is given as the matrix
of second partials of f at x:

2
f (x) =

2
f
x
i
x
j
(x)

.
Moreover,
f
x
i
x
j
=
f
x
j
x
i
for all i , j = 1, . . . , n. The matrix

2
f (x) is called the Hessian of f at x. It is a symmetric matrix.
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
The Second Derivative
Let f : R
n
R so that f : R
n
R
n
.
The second derivative of f is just the derivative of f as a
mapping from R
n
to R
n
:
[f (x)] =
2
f (x).
This is an n n matrix:
If f exists and is continuous at x, then it is given as the matrix
of second partials of f at x:

2
f (x) =

2
f
x
i
x
j
(x)

.
Moreover,
f
x
i
x
j
=
f
x
j
x
i
for all i , j = 1, . . . , n. The matrix

2
f (x) is called the Hessian of f at x. It is a symmetric matrix.
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
The Second Derivative
Let f : R
n
R so that f : R
n
R
n
.
The second derivative of f is just the derivative of f as a
mapping from R
n
to R
n
:
[f (x)] =
2
f (x).
This is an n n matrix:
If f exists and is continuous at x, then it is given as the matrix
of second partials of f at x:

2
f (x) =

2
f
x
i
x
j
(x)

.
Moreover,
f
x
i
x
j
=
f
x
j
x
i
for all i , j = 1, . . . , n.
The matrix

2
f (x) is called the Hessian of f at x. It is a symmetric matrix.
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
The Second Derivative
Let f : R
n
R so that f : R
n
R
n
.
The second derivative of f is just the derivative of f as a
mapping from R
n
to R
n
:
[f (x)] =
2
f (x).
This is an n n matrix:
If f exists and is continuous at x, then it is given as the matrix
of second partials of f at x:

2
f (x) =

2
f
x
i
x
j
(x)

.
Moreover,
f
x
i
x
j
=
f
x
j
x
i
for all i , j = 1, . . . , n. The matrix

2
f (x) is called the Hessian of f at x. It is a symmetric matrix.
Multivariable Calculus Review
Outline Multi-Variable Calculus Point-Set Topology Compactness The Weierstrass Extreme Value Theorem Operator and Matrix Norms Mean Value Theorem
Second-Order Taylor Theorem
If f : R
n
R is twice continuously dierentiable on an open set
containing [x, y], then there is a z [x, y] such that
f (y) = f (x) +f (x)
T
(y x) +
1
2
(y x)
T

2
f (z)(y x).
shown that
[f (y) (f (x) + f

(x)(y x))[
1
2
|x y|
2
p
sup
z[x,y]
|
2
f (z)|
(p,q)
,
for any choice of p and q from [1, ].
Multivariable Calculus Review