Nadine Yamout
February, 2017
DEFINITION
A set is any well-specified collection of elements.
EXAMPLE
• The set of students in a class.
• The set of cars in Australia.
Given two sets A and B , new sets can be formed through the following
set operations on A and B :
• Union: A ∪ B is the set of all elements that are either in A or in B
(or in both).
A ∪ B = {x : x ∈ A or x ∈ B }
• Intersection: A ∩ B is the set of all elements that are common to
both A and B .
A ∩ B = {x : x ∈ A and x ∈ B }
• Minus: A − B is the set of elements of A that are not in B .
A − B = {x : x ∈ A and x 6∈ B }
EXAMPLE
Consider the sets A = {1, 2, 3, 4} and B = {3, 4, 5, 6}. Compute A ∪ B ,
A ∩ B , and A − B .
DEFINITION
Euclidean n-space Rn is defined as the set of all n-tuples (x1 , ..., xn ) of
real numbers xi .
Rn := {(x1 , ..., xn ) : xi ∈ R, i = 1, ..., n}
An element of Rn is often called a point in Rn or a vector in Rn .
EXAMPLE
• R: The line 3: point in R
2
• R : The plane (2, 5): vector in R2
• R3 : The space (6, 9, 5): vector in R3
EXAMPLE
Consider the vectors x = (2, 3, 7) and y = (4, 8, 1). Compute x + y ,
x − y , and 2x .
DEFINITION
In Rn , the length of a vector x ∈ Rn , usually called the norm kx k of x ,
is defined by
q
kx k = x12 + x22 + · · · + xn2
EXAMPLE
Compute the norm of the vector x = (2, 3, 5).
THEOREM
For all x , y ∈ Rn and α ∈ R, the norm k.k satisfies the following
properties:
• kx k ≥ 0
• kx k = 0 if and only if x = 0
• kαx k = |α|. kx k
• kx + y k ≤ kx k + ky k (Triangle Inequality)
EXAMPLE
Consider the vectors x = (1, 0) and y = (0, 1). Show the Triangle
Inequality for these two vectors.
DEFINITION
Given x , y ∈ Rn , the dot product or inner product of the vectors x and
y , denoted by x · y , is defined as
x · y = (x1 , ..., xn ) · (y1 , ..., yn ) = x1 y1 + · · · + xn yn
Remark: The norm and the dot product are related through the
p
following identity: kx k = x · x .
THEOREM
For all x , y , z ∈ Rn and α ∈ R, the dot product satisfies the following
properties:
• x ·y =y ·x
• x · (y + z) = x · y + x · z
• (αx) · y = x · (αy ) = α(x · y )
• |x · y | ≤ kx k ky k (Cauchy-Shwarz Inequality)
EXAMPLE
Compute the dot product of the vectors x = (1, 0, 5) and y = (3, 6, 1).
THEOREM
Let x and y be two vectors in Rn and let θ ∈ [0, 2π] be the angle
between x and y . Then
x · y = kx k ky k cos θ
DEFINITION
Given x , y ∈ Rn , the distance between the vectors x and y , denoted by
d(x , y ), is defined as
p
d(x , y ) = (x1 − y1 )2 + · · · + (xn − yn )2
The distance function d is called the metric.
EXAMPLE
In R2 , choose two points x = (x1 , x2 ) and y = (y1 , y2 ) with x1 − y1 = a
and x2 − y2 = b. Then Pythagoras states that
p p
d(x , y ) = (x1 − y1 )2 + · · · + (xn − yn )2 = a2 + b 2
Remark: The metric and the norm are related through the following
identity: d(x , y ) = kx − y k .
DEFINITION
Let X and Y be two sets. Then, f is a function from the domain X to
the range Y if for each x ∈ X , there exists a unique y ∈ Y which is
assigned by f to x .
DEFINITION
Two functions f : X → Y and g : X → Y are equal if f (x) = g (x) for all
x ∈ X.
DEFINITION
Let f : X → Y and g : Y → Z be two functions. Then the composition
of f and g is a mapping h = g ◦ f : X → Z where h(x) = g [f (x)].
EXAMPLE
Consider f : R → R and g : R → R defined by
f (x) = x 3 + x 4 g (x) = x + 1
Compute g ◦ f and f ◦ g .
THEOREM
Suppose that k is an arbitrary constant and that f : R → R and
g : R → R are differentiable functions at x then,
d d d
• dx f (x) ± g (x) = dx f (x) ± dx g (x)
d d
• dx kf (x) = k dx f (x)
d d d
• dx f (x).g (x) = dx f (x) g (x) + f (x) dx g (x)
d d
d f (x) dx f (x) g (x) − f (x) dx g (x)
• dx =
g (x) g (x)2
THEOREM
Suppose that k is an arbitrary constant and that f : R → R and
g : R → R are differentiable functions at x then,
d d
• dx f (x)k = kf (x)k −1 dx f (x)
d
• dx x k = kx k −1
d
d dx f(x)
• dx ln(f (x)) =
f (x)
d d
• dx exp(f (x)) = dx f (x) exp f (x)
DEFINITION
A function from Rm to R (sometimes called a real-valued function of
m variables) is a rule which associates to each point in Rm some point
in R. We write
f : Rm → R
to indicate that f (x) ∈ R is defined for x ∈ Rm .
EXAMPLE
A function f : R2 → R given by
f (x , y ) = x 2 + 2y 3
where (x , y ) ∈ R2
DEFINITION
For any vector v , the direction d is given by normalizing v to a unit
length:
v
d=
kv k
EXAMPLE
Find the direction of the vector v = (1, 2).
DEFINITION
For a function f : Rm → R and a direction d , define
f (a + sd) − f (a)
f 0 (d , a) = lim
s →0 s
provided the limit exists. This limit is called the directional derivative
of the function f at the point a in the direction d .
EXAMPLE
Find the directional derivative of the function f (x1 , x2 ) = x1 x2 at the
point a = (a1 , a2 ) in the directions of the vectors
• u = (1, 0)
• v = (1, 2)
DEFINITION
Consider the function f : Rm → R. We define the j th partial derivative
of f at the point a to be the directional derivative of f at a with
respect to the vector ej , provided this derivative exists; and we denote
it by Dj f (a). That is,
f (a + sej ) − f (a)
Dj f (a) = lim
s →0 s
Remark: To calculate the partial derivative of a function f with
respect to any variable, pretend the other variables are constant and
differentiate f .
EXAMPLE
Using two different methods, find the partial derivatives at the point
a = (a1 , a2 ) of the function f (x1 , x2 ) = 2x12 x2 + 5x1
DEFINITION
Consider the function f : Rm → R. We define the gradient vector of f
at the point a as
EXAMPLE
Find the gradient vector at the point a = (a1 , a2 ) of the function
f (x1 , x2 ) = x12 x23 + x1 exp(x2 )
DEFINITION
Consider the function f : R2 → R. Since f (x1 , x2 ) is a function of two
variables, then the partial derivatives D1 f (x1 , x2 ) and D2 f (x1 , x2 ) are
also functions of two variables. Hence, they can be differentiated again
with respect to x1 and x2 at the point a. That is, we define
∂ 2 f (a) ∂ 2 f (a)
D11 f (a) = D12 f (a) =
∂ x12 ∂ x1 ∂ x2
∂ 2 f (a) ∂ 2 f (a)
D21 f (a) = D22 f (a) =
∂ x2 ∂ x1 ∂ x22
EXAMPLE
Find the second-order partial derivatives at the point a = (a1 , a2 ) of
the function f (x1 , x2 ) = 2x12 x2 + 5x1
DEFINITION
Consider the function f : Rm → R. The m × m matrix representing the
second-order partial derivatives of the function f at the point a is
called the Hessian matrix of f :
···
D11 f (a) D12 f (a) D1m f (a)
D21 f (a) D22 f (a) ··· D2m f (a)
Hf (a) = .. .. .. ..
. . . .
Dm1 f (a) Dm2 f (a) ··· Dmm f (a)
EXAMPLE
Find the Hessian matrix at the point a = (a1 , a2 ) of the function
f (x1 , x2 ) = x12 x23 + x1 exp(x2 )
DEFINITION
A function from Rm to Rn (sometimes called a vector-valued function
of m variables) is a rule which associates to each point in Rm some
point in Rn . We write
f : Rm → Rn
to indicate that f (x) ∈ Rn is defined for x ∈ Rm .
EXAMPLE
A function f : R3 → R2 given by
f (x , y , z) = (x 2 + 3yz , z 3 + xyz)
where (x , y , z) ∈ R3
DEFINITION
Consider the function f : Rm → Rn . If the partial derivative of the
component function fi of f exists at the point a, then one can form
the matrix that has Dj fi (a) as its entry in row i and column j . This
n × m matrix is called the Jacobian matrix of f :
D1 f1 (a) D2 f1 (a) · · ·
Dm f1 (a)
D1 f2 (a) D2 f2 (a) · · · Dm f2 (a)
Jf (a) = .. .. .. ..
. . . .
D1 fn (a) D2 fn (a) · · · Dm fn (a)
EXAMPLE
Find the Jacobian matrix at the point a = (a1 , a2 , a3 ) of the function
f (x1 , x2 , x3 ) = (x1 + 3x2 x3 , exp(x2 ) + 5x12 x2 x3 )