of
Matrix Modeling and Computing
with
MATLAB
Robert E. White
Department of Mathematics
North Carolina State University
white@math.ncsu.edu
Updated on 07-20-06.
Copyright: CRC/Chapman Press a Division of
Taylor and Francis Group
Contents
List of Tables xi
Preface xiii
Introduction xv
2 Vectors in Space 47
2.1 Vectors and Dot Product . . . . . . . . . . . . . . . . . . . . . . 47
2.2 Cross and Box Products . . . . . . . . . . . . . . . . . . . . . . . 56
2.3 Lines and Curves in R3 . . . . . . . . . . . . . . . . . . . . . . . 67
2.4 Planes in R3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
2.5 Extensions to Rq . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
3 Ax = d: Unique Solution 95
3.1 Matrix Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
3.2 Matrix Products . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
3.3 Special Cases of Ax = d . . . . . . . . . . . . . . . . . . . . . . . 117
3.4 Row Operations and Gauss Elimination . . . . . . . . . . . . . . 127
3.5 Inverse Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
3.6 OX Factorization . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
3.7 Determinants and Cramer’s Rule . . . . . . . . . . . . . . . . . . 159
v
vi CONTENTS
Bibliography 397
Index 399
List of Figures
vii
viii LIST OF FIGURES
xi
Preface
xiii
xiv PREFACE
developed from very simple models to more complex models. The reader can
locate sections pretaining to a particular application by using the index.
°R
MATLAB is used to do some of the more complicated computations. Al-
though the primary focus is to develop by-hand calculation skills, most sec-
tions at the end have some MATLAB calculations. The MATLAB m-files used
in the text are listed in the index and are included in the book’s Web site:
http://www4.ncsu.edu/~white. The approach to using computing tools in-
cludes: first, learn the math and by-hand calculations; second, use a computing
tool to confirm the by-hand calculations; third, use the computing tool to do
more complicated calculations and applications.
I hope this book will precipitate discussions concerning the core mathemat-
ical course work that scientists and engineers are required to study. Discrete
models and computing have become more common, and this has increased the
need for additional study of matrix computation, and numerical and linear al-
gebra. The precise topics, skills, theory and appropriate times to teach these
are certainly open for discussion. The matrix algebra topics in this book are
a small subset of most upper level linear algebra courses, which should be en-
hanced and taken by a number of students. This book attempts to make a
bridge from two- and three-variable problems to more realistic problems with
more variables, but it emphasizes skills more than theory.
I thank my colleagues who have contributed to many discussions about the
content of this text. And, many thanks go to my personal friends and Liz White
who have listened to me emote during the last year.
Bob White
MATLAB is a registered trademark of The MathWorks, Inc. For product
information, please contact:
One can view an p×q matrix as a table of objects with p rows and q columns.
The objects are usually real or complex numbers, but they could be characters
or records of information. A simple example is data for the last 12 months of
car sales where there are p = 12 rows and q = 2 columns. The first column
will have the month’s number and the second column will have the number of
cars sold in the corresponding month. By examining the data one would like
to make a prediction about futures sales. This is where the modeling enters.
If the graph of the sales versus months "looks" like a straight line, then the
data may be modeled by a linear function of time | = pw b + f. The slope p b
and intercept f must be chosen so that the computed sales are "close" to the
car sales data. This is done by appropriate manipulations of the two column
vectors and computing a solution of the resulting system of algebraic equations.
Once p b and f have been found, the predicted sales for w larger than 12 can
easily be calculated by evaluating the linear function. The modeling process is
complicated by incorrect sales data, changing prices and other models such as
a parabolic function of time.
This text examines a variety of applications, which have matrix models and
often have algebraic systems that must be solved either by-hand calculations
or using a computing tool. Applications to projectiles, circuits, mixing tanks,
trusses, heat conduction, motion of a mass, curve fitting and image enhancement
will be initially modeled in very simple ways and then revisited so as to make
the model more accurate. This is typical of the modeling process where there is
an application, a model, mathematical method, computations and assessment
of the results. Then this cycle is repeated so as to enhance the application’s
model.
The first two chapters deal with problems in two- and three-dimensional
space where the matrices have no more than three rows or columns. Here
geometric insight can be used to understand the models. In Section 2.5 the
extension to higher dimensions is indicated for vectors and matrices, solution
to larger algebraic systems, more complicated curve fitting, time dependent
problems with systems of dierential equations and image modeling. Chapters
three, four and five have the basic matrix methods that are required to solve
systems in higher dimensions. Chapters six and seven contain time dependent
models and introduce linear systems of dierential equations. The last two
xv
xvi INTRODUCTION
1 }3 }2 }
Chapter 1
1
2 CHAPTER 1. VECTORS IN THE PLANE
The set of rational numbers has a countable but infinite number of elements.
Also, the addition and product of two rational numbers are rational numbers.
{ = ±(={1 · · · {g · · · )10h
±({1 @10 + · · · + {g @10g + · · · )10h =
Real numbers contain the rational numbers, but not all real numbers are
rational. For example, consider { = 31@2 where 3 is a prime number. If { were
a rational number, then { = p@q giving 3 = p2 @q2 and 31 q2 = p2 . The
left side has an odd number of prime factors 3, and the right side has an even
number of prime factors 3. This contradicts the unique factorization property
and, hence, { cannot be a rational number.
This is a floating point number with base equal to 10 where {1 is not equal
to zero, {l are integers between 0 and 9, the exponent h is an integer between
given integers i and j and g is a positive integer called the precision of the
floating point system
Associated with each real number, {, and its floating point approximate
number, i o({), is the floating point error, i o({) {. This error decreases as
the precision, g, increases. Each computer calculation has some floating point
1.1. FLOATING POINT AND COMPLEX NUMBERS 3
error. Moreover, as additional floating point calculations are done, there may
be an accumulation of these floating point errors.
Example 1.1.1. Let { = 1=5378 and i o({) = 0=154 101 where g = 3. The
floating point error is
i o({) { = =0022=
The error will accumulate with any further operations containing i o({), for
example, i o({)2 = =237 101 and
If } 5 C> then real(}) = d is called the real part and imag(}) = e is called the
imaginary part. Two complex numbers are equal if and only if both the real
and imaginary parts are equal.
Complex numbers are often represented graphically where the real part is
located on the horizontal axis and the imaginary part is located on the vertical
axis. The complex number is then either viewed as point in the complex plane
or as directed line segment or arrow from the origin 0 + l0 to the complex point
} = d + el= Figure 1.1.1 illustrates this for } = 2 + 1l= One point of confusion
concerns the starting point of the arrow, which is the origin, but often the arrow
is moved parallel to its original position, as is illustrated by the top dashed arrow
in Figure 1.1.1 that is parallel to }= This is done to depict addition of two arrows
by placing the beginning of the arrow at the end of another arrow.
There are several important operations: conjugate of }, modulus of }, addi-
tion, subtraction, product and division. The conjugate of } is another complex
number formed by the reflection of } about the real axis. The modulus of } is
the length of the vector. The algebraic operations are defined in the natural
way by using l2 = 1=
Definitions 1.1.1. Attributes of complex numbers } = d + el and z = f + gl=
} + 7 + 2l = 3 + 4l=
Addition of complex numbers is defined by adding the real and imaginary parts,
} + 7 + 2l = ({ + l|) + 7 + 2l = ({ + 7) + (| + 2)l and, then, the above two real
1.1. FLOATING POINT AND COMPLEX NUMBERS 5
equations are
{ + 7 = 3 and
| + 2 = 4=
Thus, the complex solution of } + 7 + 2l = 3 + l4 is } = 4 + 2l=
Definitions 1.1.3. Operations with complex numbers } = d+el and z = f+gl=
}±z (d ± f) + (e ± g)l
}z (d + el)(f + gl) = (df eg) + (ef + dg)l and
d + el d + el f gl df + eg ef dg
}@z = = 2 + 2 l=
f + gl f + gl f gl f + g2 f + g2
The following eleven algebraic properties are easily established. The first five
properties are the additive group properties, the next five are the multiplicative
group properties for non-zero complex numbers and the last is the distributive
property.
Theorem 1.1.1 (Field Properties of Complex Numbers) Let }> z> z1 > z2 5 C=
Then
} + z 5 C, } + 0 = }, } + (}) = 0, } + z = z + },
} + (z1 + z2 ) = (} + z1 ) + z2 ;
}z 5 C, }1 = }, }(} 1 ) = 1 when } 6= 0, }z = z},
}(z1 z2 ) = (}z1 )z2 ;
}(z1 + z2 ) = }z1 + }z2 =
Example 1.1.6. This example illustrates how one can solve complex algebraic
equations where we must solve for both the real and imaginary parts of an
unknown complex number } = { + l|= Here we use the above field properties
1 + 4l
= 2+l
}+1
1 + 4l = (2 + l)(} + 1)
= (2 + l)} + (2 + l)=
1 + 4l 5 + 20l
=
(1@5 + (7@5)l) + 1 6 + 7l
5 + 20l 6 7l
=
6 + 7l 6 7l
170 + 85l
=
85
= 2 + l=
The complex numbers are extensions of the real numbers, and many of
the algebraic and absolute value properties extend to the complex numbers.
For example, if d and f are real numbers and |d| is the absolute value, then
|df| = |d| |f|. When d and f are extended to complex numbers and absolute
value is extended to the modulus of a complex number, then one can show
1.1. FLOATING POINT AND COMPLEX NUMBERS 7
|}z| = |}| |z| = In order to see this is true, let } = d + el and z = f + gl and
show |}z|2 = |}|2 |z|2 :
¯ ¯
If } = z, then we could write this as ¯} 2 ¯ = |}|
2
1.1.6 Exercises
1. Consider the real number 1@3.
(a). Find i o(1@3) with base 10 and three digits, g = 3.
2 2
(b). Compute (i o(1@3)) and the error = (i o(1@3)) (1@3)2 =
1@2
2. Let s be prime number. Show s is not a rational number.
3. Let } = 7 l and z = 2 + 3l=
(a). Graph }, z, z and 2z + 3}=
(b). Compute }z and }@z=
4. Let } = 2 + 3l and z = 2 3l=
(a). Graph }, z, z and 2z + 3}=
(b). Compute }z and }@z=
5. For } = cos(2@3) + l sin(2@3) compute 1 + } + } 2 =
6. For } = cos(2@4) + l sin(2@4) compute 1 + } + } 2 + } 3 =
7. Use the properties in Theorem 1.1.1 to solve for the complex number
} = { + |l
} + 7 + 2l = 3 + 4l=
8. Prove the remaining properties in Theorem 1.1.1.
9. Find the real and imaginary parts of
2+l
=
3 7l
10. Find the real and imaginary parts of
2+l
+ 5=
10 + 7l
11. Solve for the complex number } = { + |l
2 + 4l
= 7 + l=
}+l
12. Solve for the complex number } = { + |l
2 + 4l
= 1 + l=
2} l
13. Prove the remaining properties in Theorem 1.1.2.
14. Does the quadratic formula hold when the coe!cients d> e and f are
complex numbers? You may wish to consider some special cases such as d =
1> e = 0 and f = l= One needs to be able to define the square root of a complex
number!
15. Use MATLAB and enter } = 2 + 3l and z = 2 3l=
(a). Graph } and z=
(b). Compute }z and }@z=
(c). Use quiver() to plot the vectors for } and z=
16. Consider the calculations
s in Figure 1.1.2 given by the MATLAB code
complex_prod.m. Let } = 3 + 1l=
(a). Use by-hand calculations to find } 2 , } 3 , their norms and arguments.
(b). Modify complex_prod.m to confirm your by-hand calculations.
10 CHAPTER 1. VECTORS IN THE PLANE
i (}) = d} + e=
The constants d and e are given complex numbers, and } = {+l| is any complex
number where { and | are real numbers. This means
the functions x({> |) and y({> |) have real values. The solution of i (}) = g =
g1 + lg2 is equivalent to equating the real and imaginary parts and reduces to
solving two real equations for { and |
{ | + 3 = 0 and { + | 1 = 7=
The solution of i (}) = 2 + l can be found by equating the real and imaginary
parts x({> |) = 2, y({> |) = 1 and solving for { and |= Or, one can use the
properties in Theorem 1.1.1 to find } as follows:
}+l
= 2+l
}l
}+l = (2 + l)(} l) = (2 + l)} (2 + l)l
}(1 (2 + l)) = (2 + l)l l
}(1 l) = 3l + 1
1 + 3l 1 l 2 + 4l
} = = = 1 + 2l=
1+l 1l 2
A quadratic complex function is
i (}) = d} 2 + e} + f=
The coe!cients d, e and f are given complex numbers, and one would like to
solve i (}) = d} 2 + e} + f = 0= In order to generalize the quadratic formula
to the complex case, the square root of a complex number needs to be defined.
Since the square root function is the inverse of the square function, let us focus
on
i (}) = } 2 =
12 CHAPTER 1. VECTORS IN THE PLANE
In Section 1.1 we used the polar representation } = |}| (cos() + l sin()) and
Theorem 1.1.2 with } = z to obtain
Thus, one must choose x and y such that { = x2 y 2 and | = 2xy= The use of
trigonometric identities confirms
1@2 1@2
x = |}| cos(@2) and y = |}| sin(@2)
are the solution of these two equations and, hence, equation (1.2.2) is correct
for the definition of square root of }=
s
Example 1.2.3. Use equation (1.2.2) to find the square root of } = 1 + 3l.
The modulus of } is 2 and the angle is @3 or 60 degrees so that } = 2(cos(@3)+
l sin(@3))= Equation (1.2.2) gives
s 1@2
(1 + 3l) = 21@2 (cos(@6) + l sin(@6))
s s
= 2( 3@2 + l(1@2))
p s
= 3@2 + l(1@ 2)=
s
Note the solution of the complex quadratic equation } 2 = 1 + 3l is plus or
minus this square root.
Figure 1.2.1 illustrates the linear, square and square root functions of a
complex variable; this was generated by the MATLAB code complex_power.m.
The complex numbers in the domain correspond to the domain semicircle with
radius equal to three. The linear function has values in the complex plane where
the domain semicircle has been doubled in radius and shifted to the right. The
square function has values in the complex plane where the domain semicircle
has been expanded in both the radial direction and in an angular direction to
one complete circle of radius equal to three squared. The smallest curve depicts
the square root function. Now, the domain semicircle has been contracted in
both the radial and angular components to one half of the semicircle and with
radius equal to the square root of three.
1.2. COMPLEX VALUED FUNCTIONS 13
semester of calculus,
{2 {3 {4 {5
h{ = 1+{+ + + + + ··· >
2! 3! 4! 5!
{2 { 4
cos({) = 1 + · · · and
2! 4!
{3 {5
sin({) = { + ··· =
3! 5!
Extend the exponential function’s domain from the real numbers to the complex
numbers by replacing { by l|
(l|)2 (l|)3 (l|)4 (l|)5
hl| = 1 + l| + + + + + ···
2! 3! 4! 5!
|2 |3 |4 |5
= 1 + l| l + + l + ···
2! 3! 4! 5!
|2 |4 |3 |5
= [1 + · · · ] + l[| + ···]
2! 4! 3! 5!
= cos(|) + l sin(|)=
} = { + l|
= |}| (cos() + l sin ())
= |}| hl =
Second, this leads to the representation of the cosine and sine functions
Third, consider the product of hl{ and hl| and use the trigonometric identities
for the sum of angles
Fourth, another identity, which follows from the repeated use of hl{ hl| = hl({+|)
with | = {> is de Moivre’s formula
This leads to all q solutions } = (hl2@q )m = hl(2@q)m where m = 0> 1> ===> q 1=
They are solutions because
}q = (hl(2@q)m )q
= hl(2)m
= cos((2)m) + l sin((2)m)
= 1=
Definition 1.2.2. Let m = 0> 1> · · · > q1= Then hl(2@q)m are solutions to } q = 1
and are called the qwk roots of unity.
Example 1.2.5. Let q = 3 and find the three solutions of } 3 = 1= Here
2@q = 2@3 or 120 degrees. The three solutions are
hl(2@3)0 = cos((2@3)0) + l sin((2@3)0) = 1
hl(2@3)1 = cos((2@3)1) + l sin((2@3)1) = 0=5000 + 0=8660l
hl(2@3)2 = cos((2@3)2) + l sin((2@3)2) = 0=5000 0=8660l=
Another special case with q = 12 is illustrated in Figure 1.2.2 where the 12
vectors with radius equal to one are the solutions. The other 12 vectors have
increasing moduli and have staggered angles
(1=05h(l2(1+=05)@q) )m =
The figure was generated by the MATLAB code euler_form.m.
The solution of the more general problem } q = g is straightforward once
the complex number g is written in polar form
g = |g| hl = |g| hl hl2
1@q
} = g1@q = |g| (hl hl2 )1@q
= |g|1@q hl@q hl2@q = (1.2.10)
l2@q m q
Define z h and use the fact (z ) = 1 to obtain all q solutions } =
g1@q zm where m = 0> 1> ===> q1= A summary of the above is given in the following
theorem.
Theorem 1.2.1 (Solutions of } q = g)= Let z hl2@q and g = |g| hl = Then
z satisfies z = z1 > zq = 1 and 1+z+· · ·+zq1 = 0= Moreover, the solutions
1@q l@q m
of } q = g are given by } = |g| h z = g1@q zm where m = 0> 1> · · · > q 1=
Example 1.2.6. Let q = 3 and find the three solutions of } 3 = 1 + l= Here
2@q = 2@3 and g = 1 + l = 21@2 hl@4 = The three solutions are
21@6 hl@12 hl(2@3)0 = 21@6 (cos(@12 + (2@3)0) + l sin(@12 + (2@3)0))
= 1=0842 + 0=2905l
21@6 hl@12 hl(2@3)1 = 21@6 (cos(@12 + (2@3)1) + l sin(@12 + (2@3)1))
= 0=7937 + 0=7937l
21@6 hl@12 hl(2@3)2 = 21@6 (cos(@12 + (2@3)2) + l sin(@12 + (2@3)2))
= 0=2905 1=0842l=
1.2. COMPLEX VALUED FUNCTIONS 17
square and square root functions. An array operation on a row vector produces
another row vector whose components are given by the operation. For example,
if d = [2 3]> then d=ˆ2 = [22 32 ]=
The MATLAB code euler_form.m, see Figure 1.2.2, uses a for-loop to gen-
erate a sequence of vectors associated with the complex exponential function
raised to a variety of powers. The vectors are created by the MATLAB com-
mand quiver(x,y,dx,dy,1), which graphs a vector beginning at the point ({> |)
and ending at ({ + g{> | + g|).
1.2.6 Exercises
1. Let i (}) = (2 + l)} + 7 + l where } = { + l|
(a). Find x({> |) and y({> |) such that i (}) = x + ly=
(b). Find the solution of i (}) = 10 2l=
2. Let i (}) = (3 + l)} + 1 + 2l where } = { + l|
(a). Find x({> |) and y({> |) such that i (}) = x + ly=
(b). Find the solution of i (}) = 1 2l=
3. Let i (}) = (} + 2l)@(} l) where } = { + l|
(a). Find x({> |) and y({> |) such that i (}) = x + ly=
(b). Find the solution of i (}) = 1 2l=
4. Let i (}) = (2} + l)@(} + l) where } = { + l|
(a). Find x({> |) and y({> |) such that i (}) = x + ly=
(b). Find the solution of i (}) = 1 + 2l=
5. Use equation (1.2.2) to find the square root of 1 + l= s
6. Use equation (1.2.2) to find the square root of 1 + 3l=
7. Use the complex quadratic formula to solve d} 2 + e} + f = 0=
(a). Solve the quadratic equation } 2 + 2} l = 0=
(b). Check your answer.
8. Consider the completing square method of developing the quadratic for-
mula.
(a). Prove: if } and z are complex numbers and }z = 0> then either
} = 0 or z = 0=
(b). Use this fact to derive the quadratic formula for complex numbers.
9. Use de Moivre’s formula in equation (1.2.9) to compute (1 + l)10 =
10. Use de Moivre’s formula in equation (1.2.9) to compute (3 + 2l)7 =
11. Use Theorem 1.2.1 to find all three solutions to } 3 = 3 + 4l=
s
12. Use Theorem 1.2.1 to find all four solutions to } 4 = 1 + 3l=
13. Let z hl2@q = Show z satisfies z = z1 > zq = 1 and 1 + z + · · · +
q1
z = 0=
14. Use MATLAB to verify the third equation in Example 1.2.6.
15. Use MATLAB to experiment with dierent size semicircles in the code
complex_power.m.
16. Use MATLAB to experiment with powers q = 3> 6 and 12 in the code
euler_form.m.
1.3. VECTORS IN R2 19
1.3 Vectors in R2
Complex numbers are represented either by ordered pairs of real numbers or as
directed line segments. The first number in the pair is the real part and the
second number is the imaginary part. The directed line segment or arrow starts
at the origin and ends at the complex point. There are a number of important
similar examples such as displacement in the plane of a mass, force acting on
a mass as well as the velocity and acceleration of a mass. As in the complex
numbers, these quantities can be added, subtracted and scaled, and have length
and angles.
In order to move an object, a force must be exerted on the mass, and this
will also have horizontal and vertical components. By adjusting the size of the
components, the direction and magnitude of the force can be determined to
obtain the desired motion. Let i1 be the force in the horizontal direction, i2
be the vertical force and denote the force vector as
$
i = [i1 i2 ] = force vector.
The resulting motion will have speeds in the horizontal and vertical directions.
The velocity vector lists these speeds so that the velocity vector also has a
direction and magnitude. The speed in the horizontal direction is y1 and when
this is coupled with the vertical speed y2 we get a velocity vector
$
y = [y1 y2 ] = velocity vector.
Newton’s law of motion can be applied in both the horizontal and vertical
directions
gy1 gy2
i1 = p and i2 = p =
gw gw
1.3. VECTORS IN R2 21
Figure 1.3.2: f2 = d2 + e2
Another result pertains to the case when the angle is not 90 degrees as is
illustrated in the triangle in Figure 1.3.3 where the triangle with sides d> e and
f is a union of two right triangles. The left right triangle has base equal to
e cos() and height equal to e sin()= The other right triangle has the same
height, but the base is d e cos() (assuming d A e)= Therefore, applying the
Pythagorean theorem to the right triangle on the right side gives
f2 = (e sin())2 + (d e cos())2
= e2 sin2 () + d2 2de cos() + e2 cos2 ()
= e2 (sin2 () + cos2 ()) + d2 2de cos()
= e2 + d2 2de cos()=
f2 = e2 + d2 2de cos()=
$ $
$ $
Figure 1.3.4: $
d + e >
d e and v e
$
$
Definitions 1.3.2. Let $d = [d1 d2 ], e = [e1 e2 ] and 0 = [0 0] be the zero
$
vector. $d equals e means both d1 = e1 and d2 = e2 = The addition of two
vectors is another vector whose components are the sum of the components of
the two vectors
$
$
d + e [ d1 + e1 d2 + e2 ]=
Let v be a real number. The scalar product of a vector is another vector whose
components are the scalar times the components of the vector
v
$
d [ vd1 vd2 ]=
$
$
The proofs are straightforward. The proof of $d + e = e +
$
d uses the
commutative property for addition of real numbers
$ $
d + e = [ d1 + e1 d2 + e2 ]
= [ e1 + d1 e2 + d2 ]
$ $
= e + d=
$ $
Example 1.3.2. Let $
d = [2 4], e = [1 2] and v = 3= Then $
d + e = [1
$
$
6]>
$
d e = [3 2] and v e = [3 6]=
Example 1.3.3. One can use the above basic properties to solve vector equa-
$
tions for an unknown vector
${ = [{1 {2 ]= Suppose vectors
$
d and e are given
$
and $
{ must satisfy the vector equation
$
d + $
{ = e . Solve this by adding the
$
vector d to both sides and use the algebraic rules
$ $
(
$d +$
{)
$d = e d
$
$
$
$ $
({ + d) d = e d
$
$ $
{ + (
$
d
$
d) = e d
$
$
$
{ = e d=
= arctan(d2 @d1 )=
The cosine law may be rewritten in terms of the norms of the vectors
$ $
$
$
d > e >$
f =$
d e and the angle between the vectors
$
d and e
° $° ° ° ° °
° °2 2 °$°2 °$°
°$d e ° = k
$
d k + ° e ° 2 k
$
d k ° e ° cos()=
This allows one to compute the angle between two given vectors. In fact, this
can be simplified to easily compute cos()> sin() and the area of the paral-
lelogram formed by two vectors. In Section 1.4 this will be applied to the
computation of work and torque.
1.3. VECTORS IN R2 25
$
$
Example 1.3.4. If $
d = [2 4] and e = [1 2]> then $
d e = [3 2]>
° $°2° ° °
$°2
° 2 °
°$d e ° = 13> k$d k = 20> ° e ° = 5 so that
s s
13 = 20 + 5 2 20 5 cos()
= 20 + 5 20 cos()
cos() = 12@20 = 3@5
= cos1 (3@5) 53=13 degrees.
The unit vector associated with
$
d is
$
d [2 4] s s
$
= s = [1@ 5 2@ 5]=
kdk 2 5
1.3.5 Exercises
$
$
1. Consider the vector
$
d = [2 5]= Write it in l m and column forms.
$
$
2. Consider the vector
$
d = [1 3]= Write it in l m and column forms.
$
$
$ $ $ $
3. Let
$
d = [2 5] and e = [1 3]. Graph $d > e > e >d + e and
$
d e=
$
$
$
$
4. Let
$
d = [2 3] and e = [1 5]. Graph 2 $
d > e > e >
$
d + e and
$
$
d 2e =
1.4. DOT PRODUCT AND WORK 27
$
$
5. Let
$
d = [2 5] and e = [1 3]. Solve $
d + ${ = e=
$
$ $
6. Let
$
d = [1 7] and e = [1 3]. Solve 2 $
d = e + {=
$
$ $
7. Consider Theorem 1.3.3. Prove ( $d + e )+ $f =$
d +( e + f )=
$
$
$
$
8. Consider Theorem 1.3.3. Prove v( d + e ) = v d + v e =
9. d = [1 7]. Compute k
Let
$ $
d k, the angle and unit vector of
$
d=
10. Let d = [1 3]. Compute k d k, the angle and unit vector of
$
$
$d=
$
$
11. Let d = [2 5] and e = [1 3]. Graph these vectors and use the cosine
law to find the angle between them.
$
12. Let
$d = [1 2] and e = [1 3]. Graph these vectors and use the
cosine law to find the angle between them.
13. Consider the aircraft heading problem. If the desired velocity is to the
northeast with speed 500 and the wind velocity has speed 120 and from the
north, find the heading velocity.
14. Consider the aircraft heading problem. If the desired velocity is to the
northwest with speed 500 and the wind velocity has speed 120 and from the
east, find the heading velocity.
$
$ $ $ $
15. Use MATLAB to compute 2 $
d > e > e >
d + e and
$d 2 e where $
d = [2
$
3] and e = [1 5]=
$
$
16. Use MATLAB to graph $
d and e where $d = [2 3] and e = [1 5]=
Place this into the left side of equation (1.4.1) and cancel the norms
° ° ° ° ° °
2 °$°2 2 °$°2 °$°
k
$
d k + ° e ° 2(d1 e1 + d2 e2 ) = k
$
d k + ° e ° 2 k $
d k ° e ° cos()
° °
°$°
2(d1 e1 + d2 e2 ) = 2 k$d k ° e ° cos()
° °
°$°
d1 e1 + d2 e2 = k$
d k ° e ° cos()= (1.4.2)
$
Definition 1.4.1. The dot product of two vectors $d = [d1 d2 ] and e = [e1 e2 ]
is given by the real number on either side of equation (1.4.2). The following are
common notations for the dot product of row or column vectors, respectively,
° °
$
$
°$°
d • e d1 e1 + d2 e2 = k $
d k ° e ° cos() or
aW b d1 e1 + d2 e2 = kak kbk cos()=
$
Definitions 1.4.2. Let $
d and e be non-zero vectors. Two vectors are parallel
if and only if they are multiples of each other, that is, there is a real number
$
v such that $
d = v e = Two vectors are perpendicular (also called orthogonal) if
$
and only if the cos() is zero, that is,
$
d • e = aW b = 0=
$
2
Example 1.4.1. Let $
d = [2 3] and e = [1 2]. Then k $
d k = 22 + 32 = 13>
° °
°$°2 $
° e ° = (1)2 + 22 = 5> and $d • e = 2(1) + 3(2) = 4=
$ $
d • e 4 4
cos() = ° ° s s s
$
°$° = 13 5 = 65 and
k d k° e °
µ ¶
1 4
= cos s 60=26 degrees.
65
Example 1.4.2. Consider the following three vectors written as column vectors
¸ ¸ ¸
2 22 6
a= >b = and c = =
3 33 4
Vectors a and b are parallel because a =(1@11)b= Vector a is perpendicular to
c because the dot product aW c = 2(6) + 3(4) = 0= Also, b and c must be
perpendicular. Consider a vector d whose first component is 2, but the second
component is unknown. Choose the second component so that the vector d
is perpendicular to vector a= Let the vector d be represented by its transpose
dW = [2 {]= The vector d being perpendicular to a requires their dot product
to be zero
¸
£ ¤ 2
aW d = 2 3
{
= 2(2) + 3{ = 0=
Thus, { = 4@3 and dW = [2 4@3].
$
Definition 1.4.3. The projection
° of
° e onto $
d is a vector with direction the
$
°$°
same as d and length equal to ° e ° cos()
d °
$ $°
° °
$
$
d $
$
kdk
° e ° cos() = $
2 d • e=
kdk
The dot product has a number of interesting algebraic properties. One is
the order of the vectors does not change the value of the dot product.
$
$
d • e = d1 e1 + d2 e2
= e1 d1 + e2 d2
$ $
= e • d=
Additional dot product properties listed in the following theorem are also easy
to prove.
30 CHAPTER 1. VECTORS IN THE PLANE
$ $
Theorem 1.4.2 (Properties of Dot Product) Let $d > e >
f be given vectors
and let v be a real number. The dot product properties are
$ $
$ $
$
$
d • e = e • d > v(
$d • e ) = (v
$
d)• e >
$
$ $
$ $
d •( e + f) = $
d • e + d •$f and
¯ ¯ ° °
¯
$ $¯
$
°
$°
¯ d • e ¯ k d k ° e ° (Cauchy-Schwarz inequality).
Since the norm of a vector can be written in terms of the dot product
2
dk =
k
$ $
d •
$
d>
several important properties of a norm can be derived from the above dot
product properties. The following norm properties are similar to the absolute
value of a real number.
$
Theorem 1.4.3 (Properties of Norm) Let $d and e be given vectors and let
v be a real number. The norm properties are
$
k
$
d k 0; k$
d k 0 if and only if
$
d = 0>
kv
$
d k = |v| k
$
d k and
° ° ° °
°$
$
° $
°$°
° d + e ° k d k + ° e ° (triangle inequality).
Choose two vectors as illustrated in Figure 1.4.1 where both vectors have unit
length. Then the first and second components are the cosine and sine of their
$
angles. Let the vectors $d and e have angles equal to and !> respectively,
and let the vector
$f be the reflection vector $d about the horizontal axis so
that
$
d = [cos() sin()]
$
e = [cos(!) sin(!)] and
$
f = [cos(!) sin(!)]=
$
d • $
f
cos( + !) =
k d k k
$ $
fk
cos() cos(!) + sin()( sin(!))
= =
1(1)
Thus, the above are also equivalent to ua + vb = 0 implies both u and v must
be zero. Such vectors are called linearly independent, which will be studied in
1.4. DOT PRODUCT AND WORK 33
more detail in Section 5.1. In Figure 1.4.3 the area of the larger parallelogram,
formed by the scaled vectors ua and vb> is uv times the area of the smaller
parallelogram. This follows from determinants
¸
d1 u e1 v
det([ua vb]) = det( )
d2 u e2 v
= d1 u(e2 v) e1 v(d2 u)
= uv det([a b])=
° ° °$°
°$° ° °
The work along the direct path is ° g ° ° i ° cos()= The work along the second
path is g1 i1 + g2 i2 , and the work along the third path is g2 i2 + g1 i1 > which is
the same as path two because the addition of real numbers commutes. Thus,
the work done is independent of the three paths taken for constant displacement
and force vectors.
to column vectors one can compute the dot product as a row vector times a
column vector.
AA cola = a’
cola =
2
1
AA colb = b’
colb =
1
-1
AA colb’*cola
ans =
1
1.4.6 Exercises
1. Verify the cosine identity in equation (1.4.2) for
$ s
$ s
d = [ 3 1] and e = [1 3]=
2. Verify the cosine identity in equation (1.4.2) for the vectors in column
form s ¸ ¸
3 1
a= and b = =
1 1
3. Use the cosine identity in equation (1.4.2) to find the angle between the
vectors in column form
¸ ¸
1 3
a= and b = =
2 1
4. Use the cosine identity in equation (1.4.2) to find the angle between the
$
vectors
$
d = [2 1] and e = [1 1]=
$
5. Find { such that $
d = [3 1] and e = [{ 1] are perpendicular.
6. Find { such that the following column vectors are perpendicular
¸ ¸
1 3
a= and b = =
2 {
9. Use the formula in equation (1.4.3) to compute the area of the parallel-
ogram given by the column vectors
¸ ¸
1 3
a= and b = =
2 1
10. Use the formula in equation (1.4.3) to compute the area of the parallel-
$
ogram given by the vectors $
d = [2 1] and e = [1 1]=
$
11. Let $
d = [2 1] and e = [1 1]= Verify the area of the parallelogram
$
given by the vectors 6$d and 3 e is 18 times the area of the parallelogram given
$
by the vectors
$d and e =
12. Consider the work in moving an object up a ramp whose horizontal edge
$
is 25 and vertical edge is 5. If the force vector is i = [10 1]> find the work.
13. Consider the work in moving an object up a ramp that requires a force
vector with force of 3 in the horizontal direction and 1 in the vertical direction.
$
If the displacement vector is g = [30 6]> find the work.
$
14. Use MATLAB to compute the dot product of $d = [2 1] and e = [3 4]=
15. Use MATLAB to compute the area of the parallelogram between $
d =
$
[1 2] and e = [3 1]=
Example 1.5.1. Suppose a line contains the point (1> 2) and satisfies the
equation 3{ + 4| 11 = 0= Since { = 1 and | = 2 satisfies this equation, the
1.5. LINES AND CURVES IN R2 AND C 39
point is in fact on the line. Solve the given equation for | to get
3 11 |2 3
|= {+ or = = p=
4 4 {1 4
In order to find the parametric equation (1.5.3), use the second form for the
slope
|2 3 |2 {1
= or = = w=
{1 4 3 4
This means | = 3w + 2 and { = 4w + 1= Note there are other possible choices
for the parameter such as w = 2bw=
$
Three vector representations are one point and a direction vector g , two
$
distinct points, and one point and a normal vector q = The first representation
is given by the vector version of the parametric equations
[{ |] = w[g1 g2 ] + [{0 |0 ]
$
$ $
u = wg + u 0= (1.5.4)
This is illustrated in Figure 1.5.1 where the direction vector and the point are
given; the given point is ({0 > |0 ) and the vector $u 0 starts at the origin and
$
ends this given point. The vector w g in the figure depicts an extension of the
given direction vector. Then any point on the line, represented by the vector
$
u beginning at the origin and ending at the point on the line, is a vector sum
$
of w g and $u 0=
If two points are given on the line by ({0 > |0 ) and ({1 > |1 ), then consider the
two vectors starting at the origin and ending at these points, $u 0 = [{0 |0 ] and
$
$
$
u 1 = [{1 |1 ]. Then u 1 u 0 is a vector that is parallel to the line and can
40 CHAPTER 1. VECTORS IN THE PLANE
If a point and a normal vector are given, then the normal vector must be
perpendicular to any vector parallel to the line. If
$
u and
$
u 0 are two vectors
$
$
that are associated with two points on the line, then u u 0 and
$q must be
perpendicular so that
$
q • (
$u $u 0 ) = 0= (1.5.6)
This vector equation is related to the second version of the single equation
(1.5.2) representation
d({ {0 ) + e(| |0 ) = 0
[d e] • [{ {0 | |0 ] = 0
$
q • (
$u $
u 0 ) = 0=
Example 1.5.2. Find the vector equation (1.5.4) for the line with point (2> 1)
and parallel to the vector [1 4]=
$ $ $
u = wg + u0
[{ |] = w[1 4] + [2 1] or
{ = 1w + 2 and | = 4w + 1=
Although the equations look dierent, it is easy to verify that the parametric
equations do satisfy this last single equation representation.
gi g2 i
(w0 ) = 0 and 2 (w0 ) A 0=
gw gw
In order to compute the derivatives, use the basic properties of the dot product
i (w) = (
$u (w) $s ) • (
$
u (w)
$s)
$
$
$
= (w g + u 0 s ) • (w g +
$
$
u0 $
s)
$
$
$
= (w g + u 0 s ) • w g + (w g + u 0
$ $
$ $s ) • (
$u 0$s)
$
$
$
= (w g ) • (w g ) + 2(w g ) • ( u 0 s ) + ( u 0 s ) • ( u 0
$
$
$
$
$
$
s)
2
$ $ $
$ $
$
$
$
$
= w g • g + 2w g • ( u 0 s ) + ( u 0 s ) • ( u 0 s )=
gi $
$ $ $
g2 i $
$
u0
= 2w g • g + 2 g • ( $
s ) and 2 = 2 g • g A 0=
gw gw
Thus, the w0 where the first derivative is zero is
$ $
u 0
g • ( $
s)
w0 = $
$ =
g • g
42 CHAPTER 1. VECTORS IN THE PLANE
Since the second derivative is positive, the minimum distance is the square root
of i (w) evaluated at w0 =
Example 1.5.3. Consider the line given by { + 2| = 4= The line contains the
$
points (0> 2) and (4> 0) and is parallel to g = [4 2]= If the point not on this
line is (2> 5), then the vector associated with the point is
$
s = [2 5]=
$ $
g • ( u0$
s)
w0 = $
$
g • g
[4 2] • ([0 2] [2 5])
=
[4 2] • [4 2]
2 1
= = =
20 10
1
$
The desired point on the line is $
1
u ( 10 )= 10 g +
$
u0 = 1
10 [4 2] + [0 2] =
4 18
[0 2] = [ 10 10 ]=
For the geometric approach to this problem observe in Figure 1.5.2 that
vector from $
s to $
u (w0 ) must be perpendicular to the given line. This means
$
the direction vector of the line g must be perpendicular to $u (w0 )
$
s = Now
use the dot product to obtain
$
g • ($
u (w0 )
$
s) = 0
$
$
$ $
g • (w0 g + u 0 s ) = 0
$
$ $ $
w0 g • g + g • ( u 0 $
s) = 0
dW d w0 = dW p=
Here d is a column vector with two rows, and w0 is called a least squares solution
to d w = p, which has two equations and one unknown.
1.5. LINES AND CURVES IN R2 AND C 43
$ $
u = $
f + $
d + e
= [wd 0] + [0 d] + [d sin(w) d cos(w)]
= [d(w sin(w)) d(1 cos(w)]=
The angle w is the parameter, and it is given in radians so that the arclength
opposite the angle is dw= Figure 1.5.4 depicts this curve with d = 1 and was
generated by the MATLAB code cycloid.m.
Euler’s formula can be used to generate sine and cosine curves with dierent
amplitudes, d, frequencies, i> and phases, . By adding the associated complex
numbers one can easily generate very complicated signals such as a person’s
voice. Consider just one tone and one sine function of time w
d sin(2i w + )=
44 CHAPTER 1. VECTORS IN THE PLANE
At time w = 0> this is dhl , which is called a phasor where is the phase angle.
In order to have more than one tone, additional frequencies must be used.
Consider two possible tones
The signal associated with these two is simply the addition of the two sine
functions. In terms of complex variables this is the imaginary part of
By letting time vary the complex variable will move in the complex plane, and
one can plot its imaginary part as a function of time. This is illustrated in
Figure 1.5.5, which was generated by the MATLAB code phasor.m.
AA x = [1 2 3 4];
AA y = [1 4 9 16];
AA plot(x,y)
The command plot(x,y) will generate a graph connecting the four points (1> 1),
(2> 4), (3> 9) and (4> 16)=
If one needs to have more data points, then this can be done by for-loops
or by implied loops called vector operations. The for-loops is more flexible,
but it is slower. The following for-loop creates three row vectors that have 10
components.
AA for j = 1:10
t(j) = j*.5;
x(j) = t(j)+1;
y(j) = t(j)^2;
end
AA plot(x,y)
The following uses vector operations to generate the same graph. The MATLAB
command y = t.^2 is an array operation, which produces another row vector
with the same number of components as the row vector t and whose components
are the squares of those in t. For example, if t = [1 2 3], then t.^2 = [1 4 9].
AA t = .5:.5:5;
AA x = t + 1;
AA y = t.^2;
46 CHAPTER 1. VECTORS IN THE PLANE
AA plot(x,y)
The MATLAB code cycloid.m, see Figure 1.5.4, uses vector operations, and the
code phasor.m, see Figure 1.5.5, uses for-loops with a MATLAB command pause
so that one can step through the loop.
1.5.5 Exercises
1. Let { 3| = 9=
(a). Find the slope-intercept equation.
(b). Find the parametric equations.
2. Let { = 2w + 1 and | = 3w 2=
(a). Find the slope-intercept equation.
(b). Find the single algebraic equation.
3. Let the line contain the point (2> 7) and be parallel to [1 1]=
(a). Find the vector equation using a direction vector.
(b). Find the parametric equations.
4. Let the line contain the point (2> 7) and be perpendicular to [1 1]=
(a). Find the vector equation using a normal vector.
(b). Find the single algebraic equation.
5. Find the point on the line $
u = w[1 2] + [0 1] that is closest to the
point (5> 5)=
6. Find the point on the line given by { = 2w + 2 and | = w + 1 that is
closest to the point (5> 5)=
7. Use MATLAB to create a graph of the line given by { = w + 2 and | = 2w
with 0 w 5=
8. Use MATLAB to create a graph of the curve given by { = w + 2 and
| = 2w2 with 0 w 3=
9. Use the MATLAB code cycloid.m to experiment with dierent wheels,
d = 0=5> 1 and 2=
10. Use the MATLAB code phasor.m to experiment with a variety of the
inputs:
(a). Vary the frequencies.
(b). Vary the amplitudes.
(c). Vary the phase angles.
Chapter 2
Vectors in Space
Vectors in space are introduced, and the dot, cross and box products are stud-
ied. Lines and planes are carefully described as well as extensions to higher
dimensional space. Applications to work, torque, inventories and visualizations
are included.
47
48 CHAPTER 2. VECTORS IN SPACE
For example, if $
d = [2 3 4], then either by row vectors
$ $
$
$
d = 2 l + 3 m + 4 n or by column vectors
a = 2e1 + 3e2 + 4e3 =
law of motion in space can be derived by applying it to each of the three direc-
tions
gy1 gy2 gy3
i1 = p > i2 = p and i3 = p
gw gw gw
$
where p is the mass, i = [i1 i2 i3 ] is the force vector and $
y = [y1 y2 y3 ]
is the velocity vector. In vector notation this means
$
i = [i1 i2 i3 ]
gy1 gy2 gy3
= [p p p ]
gw gw gw
= p $d where
$
gy 1 gy2 gy3
d = [ ] is the acceleration vector.
gw gw gw
$
beginning point is the end point of vector
$
d . The addition
$d + e corresponds
$
$
to the diagonal of the parallelogram formed by vectors $d and e . If e is
$
added to $
d , then the subtraction
$d e is the other diagonal from the end of
$
$
e to the end of d =
$
$
Definitions 2.1.2. Let $d = [d1 d2 d3 ], e = [e1 e2 e3 ] and 0 = [0 0 0]
$
be the zero vector. $d equals e means d1 = e1 , d2 = e2 and d3 = e3 = The
addition of two vectors is another vector whose components are the sum of the
components of the two vectors
$ $
d + e [ d1 + e1 d2 + e2 d3 + e3 ]=
Let v be a real number. The scalar product of a vector is another vector whose
components are the scalar times the components of the vector
v
$
d [ vd1 vd2 vd3 ]=
The following theorem lists some basic algebraic properties of the above oper-
ations for vectors in space R3 > which are analogous to those in the plane R2 .
2.1. VECTORS AND DOT PRODUCT 51
$
Theorem 2.1.1 (Vector Space Properties) Let $
d > e and $
f be vectors in
3
R and let v and w be real numbers. Then the following rules hold
$
$
$ $
$
$ $
d + e = e + d > ($d + e )+ $f = $d +( e + f )>
$
$
$
$
$
$
d + 0 = d and d + ( d ) = 0 ;
(v + w)
$
d = v $
d + w$
d , v(w
$ d > 1
d ) = (vw)
$ $d =$d and
$
$
$
$
v( d + e ) = v d + v e =
$
$
The proofs are straightforward. The proof of v( $
d + e ) = v$d + v e uses
the distributive property of addition
$
v(
$d + e ) = [ v(d1 + e1 ) v(d2 + e2 ) v(d3 + e3 ) ]
= [ vd1 + ve1 vd2 + ve2 vd3 + ve3 ]
= [ vd1 vd2 vd3 ] + [ ve1 ve2 ve3 ]
$
= v $d +ve =
$
Example 2.1.2. Let $
d = [2 4 1], e = [1 2 4] and v = 3= Then
$
$
d + e = [1 6 5]>
$
$
d e = [3 2 3] and
$
v e = [3 6 12]=
Definition 2.1.3. Let $d = [d1 d2 d3 ]. The length or Euclidean norm of the
$
vector d is given by the Pythagorean theorem
q
k
$d k d21 + d22 + d23 =
As illustrated in Figure 2.1.3 two vectors in space form a plane and so the
$ $
cosine law may be rewritten in terms of the norms of the vectors $
d > e >
f =
$
$
$
$
d e and the angle between the vectors d and e
° $° ° ° ° °
° °2 2 °$°2 °$°
°$d e ° = k $d k + ° e ° 2 k$d k ° e ° cos()= (2.1.1)
This allows one to compute the angle between two given vectors.
Example 2.1.3. Consider the vectors in Figure 2.1.3 $d = [3 2 1] and
$
e = [1 4 2]= The vector starting
° at the point (1 4 2) and ending at (3 2 1) is
$
$
°$ $°
°
represented by d e > and ° d e ° is the distance between the end points of
° $°
$
$
°$ °2 2
the vectors
$
d and e . Then $d e = [2 2 1]> ° d e ° = 9> k$d k = 14>
° °
°$°2
° e ° = 21, and equation (2.1.1) becomes
s s
9 = 14 + 21 2 14 21 cos()
s
= 14 + 21 14 6 cos()=
52 CHAPTER 2. VECTORS IN SPACE
Thus,
s
cos() = 13@(7 6) and
s
= cos1 (13@(7 6)) 40=70 degrees.
Place this into the left side of equation (2.1.1) and cancel the norms to get
° °
°$°
d1 e1 + d2 e2 + d3 e3 = k
$
d k ° e ° cos()= (2.1.2)
As in the case for vectors with two components, equation (2.1.2) is of funda-
mental importance because the left side is an easy calculation, and the right
side makes a connection with the geometry.
$
Theorem 2.1.2 (Cosine Identity) Let $d = [d1 d2 d3 ] and e = [e1 e2 e3 ]
be non-zero vectors and let be the angle between the two vectors. Then cos()
can be computed by
d1 e1 + d2 e2 + d3 e3
cos() = ° ° =
°$°
k
$d k° e °
$
Definition 2.1.4. The dot product of two vectors
$d = [d1 d2 d3 ] and e = [e1
e2 e3 ] is given by the real number on either side of equation (2.1.2). The
following are common notations for the dot product of row or column vectors,
respectively,
° °
$
$
°$°
d • e d1 e1 + d2 e2 + d3 e3 = k
$
d k ° e ° cos() or
aW b d1 e1 + d2 e2 + d3 e3 = kak kbk cos()=
The cosine identity can be restated as either
$
$
d • e
cos() = ° $°
° ° or
k
$
d k° e °
aW b
cos() = .
kak kbk
2.1. VECTORS AND DOT PRODUCT 53
$
Definitions 2.1.5. Let $d and e be non-zero vectors. Two vectors are parallel
if and only if they are multiples of each other, that is, there is a real number v
$
such that
$d = v e = Two vectors are perpendicular or orthogonal if and only if
$
the cos() is zero, that is,
$
d • e = aW b = 0=
$
Example 2.1.4. Consider the vectors in Figure 2.1.3 $d = [3 2 1] and e = [1
° °
2 °$°2 $
4 2]= Then k $
d k = 14> ° e ° = 21 and $d • e = 3(1) + 2(4) + 1(2) = 13 so
that
$
$
d • e 13 13
cos() = ° ° = s s = s and
°$ °
k
$d k° e ° 14 21 7 6
µ ¶
13
= cos1 s 40=70 degrees.
7 6
The dot product has a number of interesting algebraic properties. One is it
is associative with respect to a scalar multiple
$
v(
$
d • e ) = v(d1 e1 + d2 e2 + d3 e3 )
= (vd1 )e1 + (vd2 )e2 + (vd3 )e3
$
= (v
$
d)• e =
Additional properties are listed in the following theorems, which are analogous
to Theorems 1.4.2 and 1.4.3, and are also easy to prove. The Cauchy-Schwarz
inequality in (2.1.3) follows directly from the cosine identity in (2.1.2). Real
valued functions that satisfy equations (2.1.4)-(2.1.6) are called norms, and they
are analogous to the absolute value of a single real number. An interpretation of
the triangle inequality is the minimum distance between two points is a straight
line. These concepts generalize to higher dimensional vectors, see Sections 2.5
and 3.1.
$ $
Theorem 2.1.3 (Properties of Dot Product) Let $
d > e >f be given vectors in
space R3 and let v be a real number. Then the following hold
$ $
$ $
$
$
d • e = e • d > v(
$d • e ) = (v
$
d)• e>
$
$ $
$ $
d •( e + f) = $
d • e + d •$f and
¯ ¯ ° °
¯$ $¯
$
°
$°
¯ d • e ¯ k d k ° e ° (Cauchy-Schwarz inequality); (2.1.3)
54 CHAPTER 2. VECTORS IN SPACE
$
Theorem 2.1.4 (Properties of Norm) Let $d > e be given vectors in space R3
and let v be a real number. Then the following hold
$
k
$d k 0; $
d = 0 if and only if k $
d k = 0> (2.1.4)
$
$
kv d k = |v| k d k and (2.1.5)
° $° ° °
°$ ° °$°
° d + e ° k
$d k + ° e ° (triangle inequality). (2.1.6)
For example, if the mass is moved from the point (1> 1> 1) to the point (2> 3> 4),
$
then g = [1 2 3]= Suppose the force vector’s components are 100 in the x-
$
direction, 10 in the y-direction and 5 in the z-direction so that i = [100 10
$
$
5]= Then the work is g • i = 1(100) + 2(10) + 3(5) = 135=
2.1.5 Exercises
1. Plot the points (3> 1> 2) and (1> 3> 1)= Indicate the corresponding vectors
starting at the origin.
2. Plot the points (3> 1> 1) and (1> 1> 3)= Indicate the corresponding vectors
starting at the origin.
3. Find the unit basis vector and column vector representations of
$
$
d = [3 1 2] and e = [1 3 1]=
$ $
5. Let d = [3 1 2] and e = [1 3 1]=
$
$
(a). Compute 2 e and 2 $d + e=
$
(b). Find the vector
$
{ = [{1 {2 {3 ] so that 2
$
d +
$
{ = e=
$
$
$
$ $
$
$
$
6. Let d = 2 l + 4 m n and e = l + 2 m + 3 n
$
$
(a). Compute 2 e and 2$d +3e =
56 CHAPTER 2. VECTORS IN SPACE
$
$
$
$
(b). Find the vector
${ = {1 l + {2 m + {3 n so that 3$
d + $
{ = e=
$
$
$
$
$
$
$ $
7. Let d = l +4m 2n ° and° e = l + m + 3 n
°$°
(a). Compute k $d k and ° e ° =
$
(b). Find
$d • e=
$
8. Let $d = [0 1 2] and e °= [1° 4 1]=
°$°
(a). Compute k $d k and ° e ° =
$
(b). Find
$d • e=
9. Use the cosine identity in equation (2.1.2) to find the angle between the
vectors in column form
5 6 5 6
1 1
a = 7 2 8 and b = 7 3 8 =
1 1
10. Use the cosine identity in equation (2.1.2) to find the angle between the
$
vectors $
d = [3 1 1] and e = [1 1 1]=
11. Consider Theorem 2.1.3.
$
$ $
$
(a). Prove v(
$d • e ) = (v$d)• e = d • (v e )=
$
$
(b). Verify
$
d •(e + $f)= d • e +
$ $d •$f for the vectors
$
d = [2 1
$
$
3], e = [3 4 1] and f = [1 1 4]=
(c). What is the dierence between "prove" and "verify"?
12. Consider Theorem 2.1.4. Prove the results of lines (2.1.4) and (2.1.5).
13. Consider the work in moving an object along an incline in space whose
x-direction is 25, y-direction is 20 and z-direction is 5. If the force vector is
$
i = [10 1 2]> find the work.
14. Consider the work in moving an object along an incline in space whose
$
$
$
$
$
displacement vector is g = 20 l + 15 m + 3 n . If the force vector is i =
$
$
$
10 l + 2 m + 1 n > find the work.
15. Use MATLAB to compute the dot product of $
d = [2 1 1] and
$
e = [3 4 2] and their norms.
16. Use MATLAB command quiver3(x,y,z,dx,dy,dz,1) to plot the vectors
$
$
d = [2 1 1] and e = [3 4 2]=
[d1 d2 0] × [e1 e2 0] [0 0 d1 e2 e1 d2 ]=
$
$
$
$
$
Note, if
$
d = [1 0 0] = l and e = [0 1 0] = m > then l × m = [0 0
$
1(1) 0(0)] = n = This is illustrated in Figure 2.2.1 where the three possible
planes are viewed from the back (negative) sides. In the case $
d = $
u and
$
$
$
$
e = i the third component of u × i is the torque for rotation about the
z-axis.
the third component in the cross product of the two vectors projected into the
xy-plane. One can also project into the yz-plane and the xz-plane with similar
area formulae. The above analysis can be applied to the other two coordinate
planes to yield
The cosine identity eventually leads to a formula for the DUHD of the
$
parallelogram formed by two vectors in space
$
d = [d1 d2 d3 ] and e = [e1 e2
2.2. CROSS AND BOX PRODUCTS 59
e3 ]
° °
°$°
(DUHD)2 = (k
$d k ° e ° sin())2
° °
°$°
= (k
$d k ° e °)2 (1 cos2 ())
° ° $
$
$
°$° 2 d • e 2
= (k d k ° e °) (1 ( ° °
°$° ) )
k
$d k° e °
°$°2
2 ° ° $
= k
$d k ° e ° ( $d • e )2 =
Next substitute the vector components into the above and endure a messy
algebraic computation to obtain
The formula in equation (2.2.1) means the area in space squared is the sum of
the three "projected areas squared."
$
Definition 2.2.1. The cross product of
$
d = [d1 d2 d3 ] and e = [e1 e2 e3 ]
$
is another vector
$
d × e whose components are given by the projected areas
$ $
d × e [d2 e3 e2 d3 d1 e3 + e1 d3 d1 e2 e1 d2 ] (2.2.2)
$
$
$
(d2 e3 e2 d2 ) l (d1 e3 e1 d3 ) m + (d1 e2 e1 d2 ) n =
$
$
As we show below $d × e is perpendicular to both
$d and e and, thus, it
$
is perpendicular to the plane formed by
$
d and e = Equation (2.2.1) implies the
norm of the cross product is equal to the area of the parallelogram formed by
the two vectors. Determinants are often used as an easy way to remember the
definition of a cross product of two vectors.
The components of the row vectors are usually real or complex numbers.
An abuse of notation is to let the top row be replaced by a row of unit basis
vectors so that
5 $ $ $ 6
$
l m n
$
d × e det(7 d1 d2 d3 8)=
e1 e2 e3
$
Example 2.2.1. Let $
d = [1 3 2] and e = [4 1 5]=
5 $ $ $ 6
$
l m n
$
d × e = det(7 1 3 2 8)
4 1 5
$
$
$
= (3(5) 1(2)) l (1(5) 4(2)) m + (1(1) 4(3)) n
$
$
$
= 17 l 13 m 11 n
= [17 13 11]=
$
The cross product is perpendicular to both the vectors $
d and e . In order
to prove this, it su!ces to show the dot product of one of the vectors with the
cross product is zero
$ $
(
$
d × e)•
d = [d2 e3 e2 d3 d1 e3 + e1 d3 d1 e2 e1 d2 ] • [d1 d2 d3 ]
= (d2 e3 e2 d3 )d1 + (d1 e3 + e1 d3 )d2 + (d1 e2 e1 d2 ) d3
= d2 e3 d1 e2 d3 d1 d1 e3 d2 + e1 d3 d2 + d1 e2 d3 e1 d2 d3
= 0=
2.2. CROSS AND BOX PRODUCTS 61
$
Definition 2.2.3. The box product of $
d > e and $
f is the real number
$ $
(
$
d × e )•
f=
$ $
$
(
$
d × e )•
f = $
f • (
$d × e)
= f1 (d2 e3 e2 d3 )
f2 (d1 e3 e1 d3 )
+f3 (d1 e2 e1 d2 )
= det(D).
$ $
An interpretation of equation (2.2.4) is that
$d •( e ×
f ) gives plus or minus the
$
volume when the base of the parallelepiped is generated by the vectors e and
$
f . Equation (2.2.5) is formed by interchanging the rows and columns of the
previous matrix; the new matrix (2.2.5) is called the transpose of the matrix in
(2.2.4). If the column vectors in (2.2.5) are represented by a> b and c> then the
matrix in (2.2.5) may be written as [a b c] and det([a b c]) is plus or minus
the volume.
5 6 5 6 5 6 5 6
d1 e1 f1 0
u 7 d2 8 + v 7 e2 8 + w 7 f2 8 = 7 0 8
d3 e3 f3 0
ua + vb + wc = 0=
Thus, the above are also equivalent to ua + vb + wc = 0 implies all u> v and w
must be zero. Such vectors are called linearly independent and will be further
studied in Section 5.1. The vector equation ua+vb+wc = 0 means the diagonal
of the parallelepiped formed by the vectors ua> vb and wc is the zero vector. See
Figure 2.2.4 where the volume of the larger parallelepiped, given by the scaled
vectors ua> vb and wc> is uvw times the volume of the smaller parallelepiped. This
2.2. CROSS AND BOX PRODUCTS 65
The first volume is 15= The second volume is 2(3)(4)15 = 360 because the new
vectors are scalar multiples of the previous three with u = 2> v = 3 and w = 4=
014
AA A = [a’ b’ c’] % convert rows to columns
A=
120
201
114
AA det(A)
ans =
-15
AA det(A’) % det(A) = det(A’)
ans =
-15
AA newA = [c ; a; b] % volume with base via a and b
newA =
014
121
201
AA det(newA)
ans =
-15
AA axb = cross(a,b)
axb =
2 1 -4
AA dot(c,axb) % box product
ans =
-15
2.2.5 Exercises
° $°
$
$
°$ °
1. Let $
d = [2 3 1] and e = [3 0 1]= Compute $
d × e and ° d × e °=
$
$
$ $
$ $
$ $
2.
° Let°
$
d = 2 l + 4 m + n and e = l + 3 m n = Compute $d × e and
° $°
°$ d × e °=
$
$
$ $
$
$
$
3. Let $
d = l + 3 m + n and e = 0 l + 3 m n = Find the area of the
parallelogram formed by these vectors.
$
4. Let $
d = [2 0 1] and e = [3 1 1]= Find the area of the parallelogram
formed by these vectors.
$
$ $
5. Show $
d × e =e × d=
$
$
6. Show d × ( e × f ) may not be equal to (
$
$ $d × e)× $f . Do both
$
computations for $d = [2 3 1]> e = [3 0 1] and $
f = [1 2 3].
7. Consider the wrench torque in Example 2.2.3. If the ends of the wrench
are given by $u = [1 3 1] and the force vector has components in the x,y and
$
z-directions equal to 2> 3 and 1> respectively, find the torque vector $u × i.
2.3. LINES AND CURVES IN R3 67
8. Consider the wrench torque in Example 2.2.3. If the ends of the wrench
$
$
$ $
are given by $u = l + 2 m + n and the force vector is i = [2 1 4]> compute
$
the torque vector $
u × i.
$
$
$ $
9. A parallelepiped is given by three vectors $
d = 4l +1m + n> e =
$
$
$ $
$
$
l + 5 m + n and $f =2 l + m +5n=
(a). Use the box product to compute the volume.
(b). Use a determinant to find the volume.
$
10. A parallelepiped is given by three vectors
$
d = [1 3 1]> e = [0 3 1]
$
and f = [1 2 4]=
(a). Use the box product to compute the volume.
(b). Use a determinant to find the volume.
11. Prove the determinant identities in equations (2.2.4) and (2.2.5).
12. Use the MATLAB command det(A) and cross(a,b) to do the computa-
tions in exercise nine.
13. Use the MATLAB command det(A) and cross(a,b) to do the computa-
tions in exercise ten.
This is called a vector equation for the line and is illustrated in Figure 2.3.1.
The direction vector may be found from two distinct points on the line by
$ $
subtracting the corresponding position vectors g = u 1 $
u 0 = The parametric
equations can be derived from the three scalar equations that correspond to the
vector equation with $u (w) = [{(w) |(w) }(w)]
Example 2.3.1. Find the vector and parametric representations of the line
u 0 = [1 2 4] and
containing the two points (1> 2> 4) and (2> 5> 3)= Let
$ $
u 1 = [2
68 CHAPTER 2. VECTORS IN SPACE
i (w0 ) = (
$u (w0 )
$s ) • (
$u (w0 )
$
s)
$
= (( u 0 s ) + w0 g ) • ( u (w0 )
$
$
$
$s)
$
$
$
$
= ( u 0 s ) • ( u (w0 ) s ) + 0
$
= (
$u 0 $s ) • ((
$u 0 $s ) + w0 g )=
The computation of the minimum distance can be done by using the cross
product and the Lagrange identity in Theorem 2.2.1
° $° °$°2
° °2 2 ° ° $
°$d × e ° = k $d k ° e ° (
$
d • e )2 = (2.3.1)
$ $
$
$
Next, insert w0 = g • (
u 0 $
s )@( g • g ) into i (w0 ) and use the cross product
$
$
identity with
$d =$u 0 $
s and e = g
$
i (w0 ) = (
$
u 0
$s ) • (
$
u 0 $s ) + ($
u0 $s ) • (w0 g )
$
$ $
(
$u0
$s ) • (
$
u 0 $ s ) g • g ((
$u 0 $s ) • g )2
= $
$
g • g
° ° 2
° $°
°($u0$s)× g°
= ° ° =
°$°2
°g°
$
Theorem 2.3.1 (Minimum Distance from Point to Line) Let $
u (w) =
$u 0 +w g
represent a line, and let a point (s1 > s2 > s3 ) not be on this line and have position
$
$ $
$
$
vector
$s = [s1 s2 s3 ]= $u (w0 ) =
$
u 0 + w0 g where w0 = g • ( u 0 $
s )@ g • g
gives the point on the line that is closest to the point (s1 > s2 > s3 )= Moreover, the
minimum distance is ° $°
° °
°($
u 0 $s)× g°
° ° =
°$°
°g°
$
Example 2.3.2. Let a line be given by
$
u 0 = [1 1 1] and g = [1 2 2], and
find the minimum distance from the point (4> 4> 1) to this line. According to
70 CHAPTER 2. VECTORS IN SPACE
$
the above theorem the point on the line that is closest is
$
u (1) =
$
u 0 + 1 g = [1
1 1] + 1[1 2 2] = [2 3 3] where
One can compute the minimum distance directly by evaluating $u (1) and com-
puting the distance, or by using the above cross product formula, which only
requires the two points and the direction vector. The direct calculation is
$
u (1)
$
s = [2 3 3] [4 4 1] = [2 1 2]
$
$
k u (1) s k = 3=
$ $
u (w) =
$
u 0 + w g and
$
$
( ) = $
0+ =
$
$
Since
$u (w)
$
( ) = (
$
u 0
$ 0 ) + w g > these two dot product equations
give two scalar equations for the two unknown parameters w and
$
$ $ $
$
(
$
u 0
$
0) • g + g • g w • g = 0 and
$
$ $ $
$
u
(
$
0
$
)• + g • w •
0 = 0=
By Cramer’s rule as discussed in Section 2.4 and in Theorem 2.4.1, the solution
of this system will exist if the determinant of the 2 × 2 coe!cient matrix is not
zero "
$ $ $
$ # ° ° °$°2
g • g • g °$°2 ° ° $
$
det( $ $ $
$ ) = ° g ° ° ° + ( g • )2 =
g • •
If the direction vectors are not parallel, then the cross product will not be zero.
$
$
$
But, by the Lagrange identity in equation (2.3.1) with $d = g and e =
° $°2°
°$
this determinant is ° g × ° 6= 0= Once equation (2.3.2) is solved for w = w0
and = 0 , the points on the two lines given by the position vectors
$u (w0 ) and
$
$
$
( ) can be computed as well as the minimum distance k u (w ) ( )k =
0 0 0
The cross product can be used to compute the minimum distance without
$
$
having to solve the algebraic system in (2.3.2). Observe the cross product g ×
$
$
must be perpendicular to both g and and, hence, it must also be parallel
72 CHAPTER 2. VECTORS IN SPACE
$
$
to
$u (w0 ) $ ( 0 )= This means the angle between g × and $u (w0 )
$ ( 0 )
must be either zero or 180 degrees so that by the cosine identity
° $°
$
$ °$ °
($u (w0 )
$ ( 0 )) • ( g × ) = k
$
u (w0 )
$
( 0 )k ° g × ° (±1)
° $°
$
$
$
$ °$ °
((
$u 0 $ 0 ) + g w0 0 ) • ( g × ) = k $
u (w0 )
$
( 0 )k ° g × ° (±1)
° $°
$
$ °$ °
(u 0
$ $
0 ) • ( g × ) + 0 + 0 = k $
u (w0 )
$
( 0 )k ° g × ° (±1)=
Theorem 2.3.2 (Minimum Distance Between Lines) Let two lines be given
$
$
by the vector equations $ u 0 + w g and
u (w) =
$ $
( ) =
$ 0 + = If the two
$
$
lines are not parallel and do not intersect, then g × is not the zero vector
and the solution of equation (2.3.2) gives the points on the lines with minimum
distance. Moreover, the minimum distance may be computed by (2.3.3).
the parameter. Consider a circle with radius u, and assume the } component
increases a height k for every revolution. One parametric equation for this is
{(w) = u cos(2w)>
|(w) = u sin(2w) and
}(w) = kw=
The MATLAB code helix.m was used to generate Figure 2.3.3.
Another interesting curve is a projectile’s path as it moves through the air
and is subjected to gravity, air resistance and wind forces. The projectile may
be an artillery shell directed at a target or a sport ball moving towards a goal
post. In order to model the motion, we apply Newton’s law of motion to each of
the three directions. Mass times acceleration is equal to the sum of the forces.
Suppose the wind velocity is given by speeds in the three directions $
z = [z1
z2 z3 ]= Assume the force acting on the mass from air resistance is proportional
to the velocity relative to the air, and let f be the proportionality constant. (A
more realistic model for air resistance or drag is a quadratic function of the
velocity.) The velocity vector of the mass is $
y = [{0 | 0 } 0 ] and is derived from
$
the position vector u = [{ | }]= The three equations are
p{00 = fz1 f{0
p| 00 = fz2 f| 0 and
p} 00 = fz3 f} 0 pj=
Let the initial position be at the origin so that {(0) = 0> |(0) = 0 and }(0) =
74 CHAPTER 2. VECTORS IN SPACE
0 and for ease of notation consider a unit mass p = 1= The initial velocity
components are given {0 (0)> | 0 (0) and } 0 (0)=
Each of these equations has the form
If g and f are constants, then this can be solved by making the change of
variables z g fy and, thus, z0 = 0 fy 0 and
z(w) = z(0)hfw
g fy(w) = (g fy(0))hfw
y(w) = g@f (g@f y(0))hfw = (2.3.4)
The dierential equation for y = {0 uses p = 1> g = fz1 , {0 (0) given and
{(0) = 0= Then equation (2.3.4) gives
The MATLAB code proj3d.m was used to create the graph for this curve in
Figure 2.3.4.
2.3.5 Exercises
1. Let a line be given by the two points (1> 0> 3) and (4> 1> 4)= Find the vector
equation for this line.
$
2. Let a line be given by the point (1> 2> 1) and the direction vector g = [1
1 2]= Find the parametric equations for this line.
3. Suppose a line has vector equation $u (w) = [1 1 2] + w[1 1 1]=
(a). Find the point on the line that is closest to the point (2> 4> 7)=
(b). Compute this distance.
(c). Use the cross product as in Theorem 2.3.1 to compute the distance.
4. Suppose a line has vector equation $u (w) = [1 2 2] + w[1 2 1]=
(a). Find the point on the line that is closest to the point whose position
vector is
$
s = [1 5 3]=
(b). Compute this distance.
(c). Use the cross product as in Theorem 2.3.1 to compute the distance.
76 CHAPTER 2. VECTORS IN SPACE
5. Consider two lines given by $u (w) = [1 1 2] + w[1 1 1] and $ (w) = [1
2 2] + [1 2 1]= Use the cross product as in Theorem 2.3.2 to compute the
minimum distance between the two lines.
6. Consider two lines given by $u (w) = [1 0 1] + w[1 1 3] and $ (w) = [1
-2 2] + [1 2 1]= Use the cross product as in Theorem 2.3.2 to compute the
minimum distance between the two lines.
7. Consider the two lines given in Example 2.3.3.
(a). Compute the components in the matrix equation (2.3.2).
(b). Solve this system for w = w0 and = 0 =
(c). Compute the minimum distance directly by k $u (w0 )
$
( 0 )k =
8. Use the MATLAB lines3d.m.
(a). Execute this code to view the two lines in Example 2.3.3. Use
"rotate" to obtain dierent perspectives.
(b). Modify this code to graph the two lines in either exercise 5 or 6.
9. Use the MATLAB helix.m.
(a). Execute this code to view the "spiral staircase" with radius 6 and
height of 10 per revolution.
(b). Modify the code to consider a "spiral ramp" with radius 60 and
height of 30 per revolution.
10. Use the MATLAB proj3d.m.
(a). Execute this code with the wind speeds $z = [5 10 0] and $z = [5
10 0]=
(b). Execute this code with dierent air resistance coe!cients f = 0=01
and f = 0=005 (drier air!).
2.4 Planes in R3
This section will give four descriptions of a plane in space. The minimum
distance from a point to a plane with an application to curve fitting to data will
be presented. Determinants will be used to solve resulting algebraic systems.
The normal vector can be derived from algebraic equation formula from
[d e f] • [{ {0 | |0 } }0 ] = 0
[6 2 3] • [{ 1@2 | 1@2 } 2@3] = 0=
The third description uses three distinct points that are not on the same
line. Let the position vectors
$
u 0 = [{0 |0 }0 ]>
$
u 1 = [{1 |1 }1 ] and
$
u 2 = [{2
78 CHAPTER 2. VECTORS IN SPACE
The fourth description requires a point and two non-parallel vectors in the
plane. The two non-parallel vectors could be calculated from three points as
above. Any point in the plane can be found by adding the position vector of
the given point, a scaled first vector and a scaled second vector
$
$
u = $
u + u
0
$
d +ve =
The scaling factors are real numbers u and v such that the linear combination
$
u
$d + v e is a vector beginning at the point ({0 > |0 > }0 ) and ending at a desired
point in the plane ({> |> })= See Figure 2.4.3 for an illustration of the linear
combination.
Example 2.4.2. Find the plane that contains the points (1> 1> 3)> (2> 2> 1) and
(0> 3> 2)= Choose the position vectors
$
u 0 = [1 1 3]> $u 1 = [2 2 1] and $
u 2 = [0
3 2] and compute
$
d = [2 2 1] [1 1 3] = [1 1 2] and
$
e = [0 3 2] [1 1 3] = [1 2 1]=
2.4. PLANES IN R3 79
d1 u + e1 v = i1 and d2 u + e2 v = i2 =
Multiply the second equation by d1 > multiply the first equation by d2 and sub-
tract the two to get
(d1 e2 d2 e1 )v = d1 i2 d2 i1 =
A similar calculation eliminates u and gives
(d1 e2 d2 e1 )u = i1 e2 i2 e1 =
The coe!cient on the left side is the definition of the determinant of the 2 × 2
matrix D= By moving the column vector f into the first or second column of D>
one can express the right sides using determinants. This is Cramer’s rule for
two unknowns.
Theorem 2.4.1 (Cramer’s Rule) Let D be the 2×2 matrix in equation (2.4.2).
If ghw(D) 6= 0> then the solution is
¸ ¸
i1 e1 d1 i1
u = det( )@ det(D) and v = det( )@ det(D)=
i2 e2 d2 i2
Example 2.4.3. Consider Example 2.4.2 where the plane has representation
in equation (2.4.1) or { + | + } = 5= The point ({> |> }) = (4> 3> 2) is in this
plane. The problem is to find u and v so that equation (2.4.1) holds. Since
(4> 3> 2) is in this plane, we only need to consider the first two of the scalar
equations { = 4 = 1 + 1u 1v> | = 3 = 1 + 1u + 2v
uv = 3
u + 2v = 2=
2.4. PLANES IN R3 81
Example 2.4.4. Return to the Example 2.4.2 where the plane is given by
$
$
u = $u 0 + u
$
d +ve
[{ | }] = [1 1 3] + u[1 1 2] + v[1 2 1]=
Let the point not on this plane be given by the position vector
$s = [10 11
$
$
12] and compute s u 0 = [10 11 12] [1 1 3] = [9 10 9]= Then equation
(2.4.3) is ¸ ¸ ¸
6 3 u0 1
= =
3 6 v0 2
Apply Cramer’s rule to obtain u0 = 0@27 and v0 = 9@27= Then $u min
$
u0+
$
$
$
$
s
u d + v e = [2@3 5@3 8@3] and k s u
0 0 k = 28@ 3= This is consistent
min
with equation (2.4.4)
$q • (
$u0 $
s)
k
$
s
$
u min k = $
kqk
[1 1 1] • [9 10 9] s
= s = 28@ 3=
3
| = p{ + f
(45=00)4 + 2043=33 = 1863=33=
2000
1950
1910
AA A_big\d_big
ans =
1.0e+003 *
-0.0450
2.0433
A more careful study of what is done to generate possible solutions will be given
in the next two chapters.
2.4.6 Exercises
1. Consider the plane given by { + 2| + 4} = 12=
(a). Verify the point (2> 1> 2) is in this plane.
(b). Find the normal to the plane.
(c). Find the normal equation representation of this plane.
2. Consider the plane given by 5{ + | + 4} = 20=
(a). Verify the point (0> 4> 4) is in this plane.
(b). Find the normal to the plane.
(c). Find the normal equation representation of this plane.
3. Let a plane be given by the three points (1> 1> 1)> (0> 3> 0) and (3> 0> 1)=
(a). Find a normal vector to this plane.
(b). Compute the algebraic equation that describes this plane.
4. Let a plane be given by the three points (1> 0> 1)> (1> 3> 0) and (0> 1> 1)=
(a). Find a normal vector to this plane.
(b). Compute the algebraic equation that describes this plane.
5. Consider a plane given by
Find the point in the plane that is closest to the point (5> 4> 6)=
86 CHAPTER 2. VECTORS IN SPACE
Find the point in the plane that is closest to the point (5> 4> 6)=
9. A consumer would like to predict the price of a laptop computer based
upon prices from the previous three months of $1000> $960 and $910 for the
first, second and third months of the year. Model this by assuming the price
decreases as a linear function of time similar to equation (2.4.5).
(a). Find the slope and the y-intercept of a straight line "closest" to this
data.
(b). Use these to predict the price after four months.
10. A consumer would like to predict the price of a laptop computer based
upon prices from the previous months one, two and four equal to $1000> $960
and $870, respectively. Model this by assuming the price decreases as a linear
function of time similar to equation (2.4.5).
(a). Find the slope and the y-intercept of a straight line "closest" to this
data.
(b). Use these to predict the price after five months.
2.5 Extensions to Rq
This section is an introduction to vectors with more than three components and
to the remaining chapters in this book. There are sections with applications
to inventory and price vectors, larger algebraic systems of equations, curve
fitting with more than three data points, higher dimensional spaces of vectors,
systems of ordinary dierential equations and matrices used in the context of
visualization. Even though we have mostly considered vectors with two or three
components, which may represent points in space or physical quantities such as
force and velocity vectors, we have used vectors with a number of components to
store information. Examples are the spiral and projectile curves in Subsection
2.3.3 and the lines as given by the MATLAB commands in Subsection 2.3.4.
In the line example, four vectors with 61 components were created to store
the values for the parameter and the {, | and } values of the points on the
line. Many of the operations on vectors have natural generalization to higher
dimensions, but some of the geometric aspects are lost as we move beyond
dimension two and three.
Example 2.5.1. 5 6 5 6
2 400
9 4 : 9 250 :
9 : 9 :
9 5 : 9 280 :
s =9
9
: and p = 9
: 9
:=
:
9 3 : 9 900 :
7 7 8 7 700 8
1 100
If the stock is doubled and the price is decreased by 10%, then the new stock
and price vectors are
5 6 5 6
4 360
9 8 : 9 225 :
9 : 9 :
9 10 : 9 252 :
9
2s = 9 : 9 :=
6 : and .9p = 9 810 :
9 : 9 :
7 14 8 7 630 8
2 90
The total value of the original stock and price is given by a "dot product" of
the stock and price vectors
The total value of the two stores can be computed in two ways
sW p + b
sW p = 10900 + 8460 or
W
(b
s + s) p = 19260=
88 CHAPTER 2. VECTORS IN SPACE
Large department stores have thousands of products on their shelves, and their
inventory numbers and prices can be stored in vectors with large numbers of
components.
ua + vb + wc = r0
5 65 6 5 6
d1 e1 f1 u {0
7 d2 e2 f2 8 7 v 8 = 7 |0 8 =
d3 e3 f3 w }0
Given the three column vectors and the point, we want to find u> v and w=
Determinants and a generalization of Cramer’s rule can be used to compute
these coe!cients. For three unknowns the rule is (see Section 3.7)
£ ¤ £ ¤
u = det( r0 b c )@ det( a b c )>
£ ¤ £ ¤
v = det( a r0 c )@ det( a b c ) and
£ ¤ £ ¤
w = det( a b r0 )@ det( a b c )=
W W
Example 2.5.2. Consider£ the Example
¤ 2.2.5 whereW a = [1 2 1]> b = [2 0
W
1]> c = [0 1 4] and det( a b c ) = 15= If r0 = [5 6 4], then solve
5 65 6 5 6
1 2 0 u 5
7 2 0 1 87 v 8 = 7 6 8=
1 1 4 w 4
For larger algebraic problems determinants are not used and usually row
operations are better suited for the solution. An example of a large system
would be from models of building whose support is from beams connected at a
number of nodes or joints. In order for the building to be in a static state, the
sum of the forces at each node must be zero. Each node has a small number of
equations, but the building usually will have a large number of nodes. Thus,
there are a large number of equations. This will be discussed in chapter three.
2.5. EXTENSIONS TO RQ 89
p1 + f = 2000
p2 + f = 1950
p3 + f = 1910 and
p4 + f = 1890=
$
the direction vector g to be
5 $ $ $ 6
l m n
$
q1 ×
$
q2 = det(7 1 1 1 8)
6 2 3
$
$
$
= l (1) m (3) + n (4)
= [1 3 4]=
Find a point on both planes by setting } = 0 and solve the equations {+|+0 = 1
and 6{ + 2| + 0 = 6 to obtain { = 1 and | = 0= Then the vector equation for
the line is
$
u = [1 0 0] + w[1 3 4]=
If there are p equations and q variables with p ? q, then one would like
to be able to describe the set of points satisfying the p equations. Problems
of this nature arise as parts of models for structures, circuits and fluids. These
problems will be discussed in chapter five.
16-bit images they range from 0 to 65535 = 216 1. The black image pixel is
associated with 0, and the white image pixel is associated with 255 (8-bit) or
65535 (16-bit). If these matrices are converted to floating point numbers, then
one can use matrix operations to change the image. For example, if the image
is too bright, then multiply the corresponding matrix by a constant less than
one so that the resulting pixels have smaller values.
In MATLAB one can “view” the image in several ways. First, just look at the
matrix components. Second, use the MATLAB command mesh() to generate a
surface of the image where the indices of the matrix are on the xy-plane and
the intensity of the image is on the z-axis. Third, one can map the matrix into
a standard image file such as a *.jpg file. This can be done by the MATLAB
command imwrite(). The inverse of the imwrite() is imread(), which generates
a matrix from a *.jpg file.
The following MATLAB function file creates a 50×40 matrix that represents
the letter N. The input parameter g in the function lettern(g) should be between
0 and 255 and will indicate how bright the letter will be. Other letter generating
functions are similar.
MATLAB function lettern()
function letn = lettern(g)
letn = zeros(50,40);
letn(10:40,4:8) = g;
letn(10:40,32:36) = g;
for j = 8:32
letn(48-j,j) = g;
letn(47-j,j) = g;
letn(46-j,j) = g;
letn(45-j,j) = g;
letn(44-j,j) = g;
letn(43-j,j) = g;
letn(42-j,j) = g;
end
The MATLAB code file imagncsu.m creates the letters NCSU and displays
them via mesh(), imwrite() as a *.jpg file and imwrite() for the negative image,
see Figures 2.5.1, 2.5.2 and 2.5.3.
MATLAB code file imagncsu.m
ncsu = [lettern(5) letterc(7) letters(9) letteru(11)];
mesh(ncsu);
newncsu = 20*ncsu;
newncsu = newncsu(50:-1:1,:);
negncsu = 255*ones(size(newncsu))-newncsu;
newncsu1 = uint8(newncsu);
imwrite(newncsu1, ’ncsu.jpg’);
negncsu1 = uint8(negncsu);
imwrite(negncsu1, ’negncsu.jpg’);
92 CHAPTER 2. VECTORS IN SPACE
Images and signals often have unwanted noise associated with the transmis-
sion and reception. Matrix manipulations can "filter" some of the unwanted
attributes. Chapters eight and nine will investigate space and frequency filters.
The frequency filters use the "fast Fourier transform." This is associated with
matrix products where the Fourier matrix has complex numbers as components.
2.5.7 Exercises
1. A computer store stocks five dierent laptops and has 5> 6> 2> 1 and 5 units
of the respective models. Suppose the prices per unit are 1000> 850> 1700> 2220
and 1150, respectively.
(a). Find the stock and price vectors.
(b). Compute this store’s total value of the laptop stock by using the
dot product of these vectors.
2. Consider the computer store in exercise one. The same owner wishes to
open a second store with the same five laptop models and 4> 5> 3> 3 and 2 units
of the respective models.
(a). Find the stock and price vectors for the second store.
(b). Compute the total value of the stock in both stores by using the
dot product.
3. Let ua + vb + wc = r0 where aW = [1 0 1]> bW = [2 1 1]> cW = [4 1 4]
and rW0 = [10 1 4]=
(a). Write this as a matrix equation.
(b). Use determinants and Cramer’s rule to solve the matrix equation.
4. Let ua + vb + wc = r0 where aW = [1 0 1]> bW = [3 1 2]> cW = [2 1
4] and rW0 = [10 1 4]=
(a). Write this as a matrix equation.
(b). Use determinants and Cramer’s rule to solve the matrix equation.
5. Two planes are given by 2{ + | + 3} = 12 and { + 3| + } = 6=
(a). Find the normal vectors to each plane.
(b). Find the vector equation for the line given by the intersection of the
two planes.
6. Two planes are given by { + | + } = 3 and { + 3| + } = 12=
(a). Find the normal vectors to each plane.
(b). Find the vector equation for the line given by the intersection of the
two planes.
7. Use the MATLAB command A\d to solve the matrix equation in exercise
three.
8. Use the MATLAB command A\d to solve the matrix equation in exercise
four.
9. Use the MATLAB function file lettern(g) to experiment with dierent
brightness levels. Use mesh(letn) to view the letter n, for example,
AA lettern(50)
AA mesh(letn)
10. Use the MATLAB function file lettern(g) as a template to create letter
matrices for your first name.
11. Use the MATLAB code file imagncsu.m as a template to create a *.jpg
file for your first name.
Chapter 3
Ax = d: Unique Solution
This and the next two chapters contain the basic methods for solving matrix
equations of the form Dx = d where D is a matrix and x and d are column vec-
tors. A much more complete description of solution methods and linear algebra
can be found in [5]. The possibilities of a unique solution, no solution, mul-
tiple solutions and least square solutions are discussed. In this chapter most
of the algebraic systems are assumed to be square so that the number rows
(equations) and columns (variables) are equal. Row operations and elementary
matrices are used to do by-hand computations of the Gauss elimination, LU fac-
torization and inverse matrix methods. These methods are also implemented in
MATLAB. Applications to steady state circuits, mixing tanks, heat conduction
and support trusses are given.
95
96 CHAPTER 3. AX = D: UNIQUE SOLUTION
The set of all q × 1 column vectors with real components is denoted by Rq , and
Cq is the set of all q × 1 column vectors with complex components.
Four basic operations with vectors are scalar product, addition, augmenta-
tion and transpose.
Definitions 3.1.2. Let a and b be q × 1 column vectors and let v be a real or
complex number.
Scalar product va is another q × 1 column vector whose components are vdl
5 6
vd1
9 :
va 7 ... 8 = [vdl ] where l = 1> · · · > q=
vdq
aW = [d1 · · · dq ]=
One can combine scalar multiplication and vector addition to form a linear
combination of these vectors
5 6 5 6 5 6
1 3 0
9 6 : 9 : 9 :
2a+3b c = 2 9 : + 39 2 : 9 1 :
7 3 8 7 1 8 7 4 8
5 3 2
5 6 5 6
2(1) + 3(3) 0 11
9 2(6) + 3(2) 1 : 9 17 :
= 9 : 9 :
7 2(3) + 3(1) 4 8 = 7 1 8 =
2(5) + 3(3) (2) 21
This is in contrast to the augmentation of the three column vectors, which is a
4 × 3 matrix or array 5 6
1 3 0
9 6 2 1 :
D=9 7 3 1 4 8 =
:
5 3 2
The set of q × 1 column vectors Rq is called a vector space because it has
the properties listed in the next theorem.
Theorem 3.1.1 (Vector Space Properties of Rq ) Let a, b and c be in Rq and
let v and w be real numbers. Let a + b and va denote vector addition and scalar
product. Then the following hold:
a + b 5 Rq , a + 0 = a where 0 5 Rq has zeros as components, a + (a) = 0,
a + b = b + a, a + (b + c) = (a + b) + c;
va 5 Rq , 1a = a, v(a + b) = va + vb, (v + w)a = va + wa and v(wa) = (vw)a=
The definitions of dot product and norms for vectors in R2 and R3 , see Sec-
tions 1.4 and 2.1, have the following generalizations and analogous properties.
Definitions 3.1.3. Let a and b be given vectors in Rq = The dot product of a
and b is a real number defined by
a • b aW b = d1 e1 + d2 e2 + · · · + dq eq =
The vectors a and b are called orthogonal if and only if their dot product in
zero
a • b = aW b = 0=
The Euclidean norm of a is a real number defined by
kak (a • a)1@2 = (aW a)1@2 = (d21 + d22 + · · · + d2q )1@2 =
In order to illustrate the dot product and norm, consider Example 3.1.1
where q = 4
a • b = 1(3) + 6(2) + 3(1) + 5(3) = 27 and
s
kak = (12 + 62 + 32 + 52 )1@2 = 71=
98 CHAPTER 3. AX = D: UNIQUE SOLUTION
Many properties of the dot product and norm, that hold for vectors in the plane
and space, also hold for vectors in Rq = The proofs of these are similar except
for the Cauchy-Schwarz inequality.
Theorem 3.1.2 (Properties of Dot Product) Let a> b and c be given vectors
in Rq and let v be a real number. Then the following hold
a • b = b • a> v(a • b) = (va) • b>
a • (b + c) = a • b + a • c and
|a • b| kak kbk (Cauchy-Schwarz inequality); (3.1.1)
Theorem 3.1.3 (Properties of Norm) Let a and b be given vectors in Rq and
let v be a real number. Then the following hold
kak 0; a = 0 if and only if kak = 0>
kvak = |v| kak and
ka + bk kak + kbk (triangle inequality).
The Cauchy-Schwarz inequality in (3.1.1) for vectors in R2 and R3 follows
from the cosine identity in (2.1.2). The proof for vectors in Rq makes use of
the dot product properties and a clever observation. Let w be a real number
and define the following function with b 6= 0
i (w) (a + wb) • (a + wb) 0=
Note, it can be written as a quadratic function of w
i (w) a • a + 2wa • b + w2 b • b=
The first and second derivatives are
gi
= 0 + 2a • b + 2wb • b and
gw
g2 i
= 0 + 2b • b=
gw2
Choose w0 so that i (w0 ) is a minimum, that is,
gi
= 0 + 2a • b + 2w0 b • b = 0=
gw
Thus,
a • b
w0 = and
b•b
i (w0 ) = a • a + 2w0 a • b + w20 b • b
2
(a • b)
= a•a 0=
b•b
The inequality is equivalent to the Cauchy-Schwarz inequality.
3.1. MATRIX MODELS 99
3.1.2 Matrices
In Section 2.5 matrices were used to store the intensity at each point or pixel
in an image. Matrices can be used to store a variety of information, which
often requires visualizing or modifications. There are four element-wise or array
operations that will be very useful.
Definitions 3.1.4. An augmentation of q p×1 column vectors with 1 m q
of the form 5 6
d1m
9 .. :
7 . 8
dpm
is an p × q matrix D where the first index 1 l p is the row number, the
second index 1 m q is the column number and
5 6
d11 d12 · · · d1q
9 .. : = [d ] =
D = 7 ... ..
. ··· . 8 lm
dp1 dq2 ··· dpq
Example 3.1.3. This example illustrates how array operations and matrices
can be used to create a graph in MATLAB of a surface given by } = i ({> |) =
100 {2 4| 2 = In order to graph the surface, a collection of points in the
xy-plane must be selected and then the values of i ({> |) at these points must
be computed and stored. Suppose three points in the x-direction are { = 0> 4
and 8, and four points in the y-direction are | = 0> 2> 4 and 6. There are
a total of 12 points in the xy-plane; the 12 values of { can be stored in a
4 × 3 matrix [, and the 12 values of | can be stored in a 4 × 3 matrix \= The
corresponding 12 values of i ({> |) can be computed and stored in a 4×3 matrix
100 CHAPTER 3. AX = D: UNIQUE SOLUTION
I= The following MATLAB commands do this. Note, the row 4 and column 3
component of I is 100 82 4(62 ) = 108= By using smaller increments in
the MATLAB command meshgrid(), one can generate much larger matrices and
a more accurate depiction of the surface, which is graphed by the MATLAB
command mesh(X,Y,F).
AA [X Y] = meshgrid(0:4:8,0:2:6)
X=
048
048
048
048
Y=
000
222
444
666
AA F = 100 - X.^2 - 4*Y.^2
F=
100 84 36
084 68 20
036 20 -28
-44 -60 -108
AA mesh(X,Y,F)
The following MATLAB commands create the surface and contour plots of
this cost function as given in Figure 3.1.2. The three matrices [, \ and F are
20 × 20 and note how array operations are used to compute the possible costs
in the matrix F. An estimate of the minimum cost is given by { = 9 and | = 9
3.1. MATRIX MODELS 101
The solution is easy to compute. For example, if z = 100 and = @6> then
¸ ¸
i1 200s
= =
i2 100 3
If decreases, then the tension and compressive force magnitudes will increase.
If there are more than two bars, then the number of joints will increase as well
as the number of force vectors and force vector equations. This eventually leads
to much larger algebraic systems as demonstrated in Subsections 3.4.4.
3.1. MATRIX MODELS 103
l1 l2 + l3 = 0=
Ohm’s law states that the voltage drop across each resistor is the resistance
times the current. Kirchho’s voltage law requires the sum of the voltage drops
in each loop be equal to zero
H1 U1 l1 + U3 l3 = 0
H2 U2 l2 U3 l3 = 0=
The matrix version for the three unknown currents and the above three scalar
equations is
5 65 6 5 6
1 1 1 l1 0
7 U1 0 U3 8 7 l2 8 = 7 H1 8 =
0 U2 U3 l3 H2
Most circuits have many more loops so that the resulting algebraic systems are
very large. In these cases it is not practical to solve them using Cramer’s rule.
In the following sections viable alternative methods will be developed.
3.1.6 Exercises
1. Let aW = [2 5 7 1 1], bW = [0 1 2 7 4] and cW = [1 4 2 1 3]=
(a). Find 3a 4b + 2c=
(b). Find the augmentations of a, b and c=
(c). Find a • b=
(d). Find kak =
2. Let aW = [0 8 3 1 2], bW = [1 1 2 4 4] and cW = [2 1 5 1
3]=
(a). Find 2a + 4b 3c=
(b). Find the augmentations of a, b and c=
(c). Find b • c=
(d). Find kck =
3. Consider Example 3.1.2.
(a). Find D=ˆ3.
(b). Find a=@b where a and b are the first and second columns of D.
(c). Find vlq(D).
4. Consider Example 3.1.2.
(a). Find D=ˆ(1@2).
(b). Find b= c where b and c are the second and third columns of D.
(c). Find b • c=
5. Consider the surface } = i ({> |) = 400 2{2 | 2 =
(a). By hand compute the 2 × 3 matrix
¸
i (1> 1) i (1> 2) i (1> 3)
I = =
i (2> 1) i (2> 2) i (2> 3)
aW x d1 {1 + · · · + dq {q =
vD [vdlm ]=
3.2. MATRIX PRODUCTS 107
D + E [dlm + elm ]=
The matrix vector product could also have been computed by a linear combi-
nation of the two column vectors of D
5 6 5 6 5 6 5 6 5 6
1 2 3 8 11
9 1 : 9 0 : 9 3 : 9 0 : 9 3 :
Dx = 3 9 : 9 : 9 : 9 : 9
7 7 8 + 4 7 4 8 = 7 21 8 + 7 16 8 = 7 37 8 =
:
2 3 6 12 18
If ¸
2 4
F= >
1 2
then D + F is not defined because their row and column numbers are not equal.
If v = 7, then vD is defined to be the 4 × 2 matrix
5 6 5 6
7(1) 7(2) 7 14
9 7(1) 7(0) : 9 7 0 :
vD = 9 : 9
7 7(7) 7(4) 8 = 7 49 28 8 and
:
7(2) 7(3) 14 21
5 6 5 6
11 2+3 0 5
9 1 + 2 0 + 1 : 9 1 1 :
D+E = 9 : 9
7 7 + 0 4 + 5 8 = 7 7 9 8=
:
22 3+4 0 7
1. D + (E + F) = (D + E) + F, D + E = E + F and D + ] = D=
2. D(vx) = (vD)x where v is a real number, D(x + y) = Dx + Dy.
3. The matrix-vector product can be computed as a linear combination
of the columns. Let column m of D be denoted by a:m
Dx = {1 a:1 + · · · + {q a:q =
The proofs of the above six properties are routine. The last property is the
column version of a matrix-vector product whose proof is
5 6
d11 {1 + · · · + d1p {p
9 .. :
Dx = 7 . 8
dq1 {1 + · · · + dqp {p
5 6 5 6
d11 {1 d1p {q
9 .. : 9 .. :
= 7 . 8 + ··· + 7 . 8
dq1 {1 dqp {p
5 6 5 6
d11 d1q
9 : 9 :
= {1 7 ... 8 + · · · + {q 7 ... 8 =
dp1 dpq
Example 3.2.2. Verify the matrix addition properties in the above theorem
for
5 6 5 6 5 6
1 2 1 3 1 5
D = 7 2 0 8 > E = 7 2 1 8 and F = 7 2 3 8 =
4 6 2 5 4 3
5 6 5 6 5 6
1 2 1 3 1 5
D + (E + F) = 7 2 0 8 + (7 2 1 8 + 7 2 3 8)
4 6 2 5 4 3
5 6
1 + (1 + 1) 2 + (3 5)
= 7 2 + (2 + 2) 0 + (1 + 3) 8
4 + (2 + 4) 6 + (5 + 3)
5 6
(1 1) + 1 (2 + 3) 5
= 7 (2 + 2) + 2 (0 + 1) + 3 8 = (D + E) + F=
(4 + 2) + 4 (6 + 5) + 3
3.2. MATRIX PRODUCTS 109
5 6 5 6
1 2 1 3
D+E = 7 2 0 8+7 2 1 8
4 6 2 5
5 6
11 2+3
= 7 2+2 0+1 8
4+2 6+5
5 6
11 3+2
= 7 2 + 2 1 + 0 8 = E + D=
2+4 5+6
5 6 5 6
1 2 0 0
D+] = 7 2 0 8+7 0 0 8
4 6 0 0
5 6
1+0 2+0
= 7 2 + 0 0 + 0 8 = D=
4+0 6+0
Example 3.2.4. Verify the associative rule D(EF) = (DE)F for the following
3 × 3 matrices
5 6 5 6 5 6
1 0 0 1 0 0 2 1 0
D=7 0 1 0 8 > E = 7 1@2 1 0 8 and F = 7 1 2 1 8 =
0 2@3 1 0 0 1 0 1 2
5 6 5 65 6
1 0 0 1 0 0 2 1 0
D(EF) = 7 0 1 0 8 (7 1@2 1 0 8 7 1 2 1 8)
0 2@3 1 0 0 1 0 1 2
5 6 5 6 5 6
1 0 0 2 1 0 2 1 0
= 7 0 1 0 8 (7 0 3@2 1 8) = 7 0 3@2 1 8 =
0 2@3 1 0 1 2 0 0 4@3
5 65 6 5 6
1 0 0 1 0 0 2 1 0
(DE)F = (7 0 1 0 8 7 1@2 1 0 8) 7 1 2 1 8
0 2@3 1 0 0 1 0 1 2
5 6 5 6 5 6
1 0 0 2 1 0 2 1 0
= (7 1@2 1 0 8) 7 1 2 1 8 = 7 0 3@2 1 8 =
1@3 2@3 1 0 1 2 0 0 4@3
fxl (D{)
where is the density, D{ is the volume and f is the specific heat of the wire.
112 CHAPTER 3. AX = D: UNIQUE SOLUTION
The Fourier heat law states that the heat flows from hot to cold, and the
amount of heat moving normal to a cross section is directly proportional to
the product of the time interval, cross section area and the derivative of the
temperature normal to the cross section area. Any change in the heat energy
of the segment is from diusion from left or right sides. The proportionality
constant is called the thermal conductivity N so that the heat entering from
the right is
xl+1 xl
w D N
{
and the heat entering from the left is
xl xl1
w D N =
{
xl xl1 xl+1 xl
f(xqhz
l )(D{) = fxl (D{) w D N + w D N =
{ {
Divide by f(D{) and let (w@{2 )(N@f) to get for l = 1> 2 and 3
xqhz
l = xl (xl1 + 2xl xl+1 ).
xqhz
1 = x1 (x0 + 2x1 x2 )>
xqhz
2 = x2 (x1 + 2x2 x3 ) and
xqhz
3 = x3 (x2 + 2x2 x4 ).
3.2. MATRIX PRODUCTS 113
The initial temperature is given and one can compute the temperature at the
next time step by using the above.
In order for the discrete model to give accurate approximations of the con-
tinuous heat conduction process, the step sizes in both time and space must
be suitably small, and for the above model they must satisfy the stability con-
straint 1 2 A 0= Since (w@{2 )(N@f), changing either the step sizes
or the physical properties will alter 1 2= If diusion of heat occurs in all
three directions, then the discrete model can have a large number of unknown
temperatures. For example, if each direction has 100 unknown temperatures,
then there will be 1003 unknown temperatures in a three dimensional box!
The approximation for the temperature at time equal to 1000 is u1 where the
superscript denotes the first time step
u1 = u Du + d
5 6 5 65 6 5 6
0 2 1 0 0 (0=16)200
= 7 0 8 (0=16) 7 1 2 1 8 7 0 8 + 7 0 8
0 0 1 2 0 (0=16)70
5 6
32=0
= 7 0 8=
11=2
The approximate temperature at the second time step equal to 2000 is u2 where
the superscript denotes the second time step
u2 = u1 D u1 + d
5 6 5 65 6 5 6
32=0 2 1 0 32=0 (0=16)200
= 7 0 8 (0=16) 7 1 2 1 8 7 0 8 + 7 0 8
11=2 0 1 2 11=2 (0=16)70
5 6
53=7600
= 7 6=9120 8 =
18=8160
114 CHAPTER 3. AX = D: UNIQUE SOLUTION
u2 = (u Du + d) D (u Du + d) + d
= u 2Du + D2 u+2d Dd=
One can continue solving for the temperature at the next time step. This is illus-
trated in Figure 3.2.2, which was generated by the MATLAB code wire_time.m.
As the time steps increase, the temperature vector reaches a steady state, which
is a straight line. The means for larger time steps
u u Du + d=
3.2.5 Exercises
1. Compute Dx (both row and column versions), D + E, D + F and vD
where
¸ ¸
1 2 3 4 1 3 3 4
D = > F= >
4 1 2 5 2 1 12 5
5 6
¸ 2
0 22 2 4 9 1 :
E = > x=9 :
7 5 8 and v = 9=
1 1 3 7
4
8. b = DE + DE
Verify the (D + D)E b where
5 6
¸ ¸ 1 2
1 5 3 b= 1 0 1
D= > D and E = 7 3 0 8 =
8 1 2 3 11 0
1 3
3.3. SPECIAL CASES OF AX = D 117
Plot {1 on the horizontal axis and {2 on the vertical axis. The first scalar
equation may be written in slope-intercept form of a line
Do the same for the second scalar equation. The two lines are either not parallel,
parallel and do not overlap, parallel and overlap, or they are "nearly" parallel.
If they are not parallel, then there is exactly one solution. If they are parallel
and do not overlap, then there is no solution. If they are parallel and overlap,
then there are multiple solutions given by the points on the single line. If the
two lines are "nearly" parallel, then small changes in the data can cause large
changes in the intersection, that is, the solution. In the last case the small
changes in the data can be caused by measurement, roundo or human errors.
Case One. If the lines are not parallel, then
¸
d11 d12
det(D) = det( ) 6= 0=
d21 d22
Case Two. The lines are parallel and do not overlap. For example,
{1 + {2 = 1 and
2{1 + 2{2 = 4=
{1 + {2 = 1 and
2{1 + 2{2 = 2=
{1 + {2 = 1 + and
(1 106 ){1 + {2 = 1=
Problems whose solutions are very sensitive to small changes in the data are
called ill-conditioned.
The challenge is to detect these possibilities for large systems. This chapter
is restricted to the unique solution case, and chapter five studies the multiple
solution case. Another possibility is having more equations than unknowns,
which will be investigated in the fourth chapter.
If each component on the diagonal is not zero (dll 6= 0 for all l), then the
solution is {l = gl @dll =
More general q×q matrices are lower triangular D = O and upper triangular
D=X
5 65 6 5 6
d11 0 ··· 0 {1 g1
9 .. . :
.. : 9
9 d21 d22 . {2 : 9 g2 :
9 :99 .
:
: =
9
9 .. :
:
9 . . . :7 . 8 7 . 8
7 .. .. .. 0 8 .
dq1 · · · dq>q1 dqq { q gq
Ox = d and
5 65 6 5 6
d11 d12 ··· d1q {1 g1
9 .. .. :9 : 9 :
9 0 d22 . . :9 {2 : 9 g2 :
9 :9 .. : = 9 .. :
9 . .. .. :7 8 7 8
7 .. . . dq1>q 8 . .
0 ··· 0 dqq {q gq
X x = d=
Both of these are relatively easy to solve provided all the diagonal components
dll are not zero. The lower triangular system is solved by first finding {1 and
then {2 and so on until {q is found; this is called forward substitution= The
upper triangular system is solved in reverse order by first finding {q and then
{q1 and so on until {1 is determined; this is called backward substitution.
120 CHAPTER 3. AX = D: UNIQUE SOLUTION
Example 3.3.1. Let q = 3 and solve the following lower triangular system
5 65 6 5 6
1 0 0 {1 11
7 2 3 0 8 7 {2 8 = 7 28 8 =
4 5 6 {3 60
The system is equivalent to the following three scalar equations
1{1 = 11>
2{1 + 3{2 = 28 and
4{1 + 5{2 + 6{3 = 60=
The first equation gives {1 = 11= Put this into the second equation 2(11)+3{2 =
28 to compute {2 = (28 2(11))@3 = 2= Finally, put {1 = 11 and {2 = 2 into
the third equation and find {3 = (60 4(11) 5(2))@6 = 1=
Another way to solve this is by using the column version of the matrix-vector
product 5 6 5 6 5 6 5 6
1 0 0 11
{1 7 2 8 + {2 7 3 8 + {3 7 0 8 = 7 28 8 =
4 5 6 60
The first equation gives {1 = 11= Now, put this into the left column on the left
side and move it to the right side
5 6 5 6 5 6 5 6 5 6
0 0 11 1 0
{2 7 3 8 + {3 7 0 8 = 7 28 8 11 7 2 8 = 7 6 8 =
5 6 60 4 16
The second equation gives {2 = 2= The last step is to put {2 = 2 into the left
column and move it to the right side
5 6 5 6 5 6 5 6
0 0 0 0
{3 7 0 8 = 7 6 8 2 7 3 8 = 7 0 8=
6 16 5 6
The third equation yields {3 = 1=
Example 3.3.2. Let q = 4 and solve the following upper triangular system
5 65 6 5 6
2 3 0 1 {1 25
9 0 1 2 3 : 9 {2 : 9 :
9 :9 : = 9 1 :=
7 0 0 3 1 7 8 {3 8 7 9 8
0 0 0 2 {4 6
The four equivalent scalar equations are
2{1 + 3{2 + 0{3 + 1{4 = 25>
1{2 2{3 + 3{4 = 1>
3{3 1{4 = 9 and
2{4 = 6=
3.3. SPECIAL CASES OF AX = D 121
The solution of the last equation is {4 = 6@2 = 3= Put this into the third
equation and solve for {3 = (9 + 1(3))@3 = 4= The second equation becomes
1{2 2(4) + 3(3) = 1 so that {2 = 0= The first equation gives 2{1 + 3(0) + 0(4) +
1(3) = 25 so that {1 = 11=
This can also be solved by using the column version of the matrix-vector
product
5 6 5 6 5 6 5 6 5 6
2 3 0 1 25
9 0 : 9 1 : 9 2 : 9 3 : 9 1 :
{1 9 : 9 : 9 : 9 : 9
7 0 8 + {2 7 0 8 + {3 7 3 8 + {4 7 1 8 = 7 9 8 =
:
0 0 0 2 6
The last equation gives {4 = 3= Put this into the right column on the left side
and move it to the right side
5 6 5 6 5 6 5 6 5 6 5 6
2 3 0 25 1 22
9 0 : 9 1 : 9 2 : 9 1 : 9 3 : 9 8 :
{1 9 : 9 : 9 : 9 : 9 : 9
7 0 8 + {2 7 0 8 + {3 7 3 8 = 7 9 8 3 7 1 8 = 7 12 8 =
:
0 0 0 6 2 0
The third equation gives {3 = 4= Put {3 = 4 into the right column on the left
side and move it to the right side
5 6 5 6 5 6 5 6 5 6
2 3 22 0 22
9 0 : 9 1 : 9 8 : 9 2 : 9 0 :
{1 9 : 9 : 9 :
7 0 8 + {2 7 0 8 = 7 12 8 4 7
9 :=9 :=
3 8 7 0 8
0 0 0 0 0
The second and first equations give {2 = 0 and then {1 = 11=
The solution process in all of the above cases requires that the diagonal
components not be zero. In this case, there is one and only one solution, which
can be found by using either the row or column versions of the matrix-vector
product. For large systems there may be some advantages to using one of the
two matrix-vector products. This is often dependent on the computing tools
used to do the solves.
Theorem 3.3.1 (Triangular Solves) Consider a lower or an upper triangular
matrix D = O or X= If each diagonal component of the matrix is not zero, then
the triangular systems Ox = d or X x = d have one and only one solution.
Later we will learn two methods for finding these factors. First, solve Oy = d
5 65 6 5 6
1 0 0 |1 200
7 1@2 1 0 8 7 |2 8 = 7 0 8 =
0 2@3 1 |3 70
The equation associated with the first row gives |1 = 200= The equation associ-
ated with the second row is (1@2)200+1|2 = 0 implies |2 = 100= The equation
associated with the last row is (2@3)100 + 1|3 = 70 implies |3 = 410@3= Sec-
ond, solve X x = y
5 65 6 5 6
2 1 0 {1 200
7 0 3@2 1 8 7 {2 8 = 7 100 8 =
0 0 4@3 {3 410@3
The last row implies {3 = (410@3)@(4@3) = 102=5. The second row gives
(3@2){2 102=5 = 100 or {2 = 202=5@(3@2) = 135=0= The top row gives
2{1 135 = 200 or {1 = 335@2 = 167=5=
If the matrix is tridiagonal, a matrix whose only non-zero components are
in the diagonal, sub-diagonal and super-diagonal, then one may be able to find
the OX factors by a process similar to the following. Let D be 3 × 3 tridiagonal
matrix and assume the O and X factors have the special forms
5 6 5 65 6
d11 d12 0 1 0 0 1 d12 0
7 d21 d22 d23 8 = 7 1 1 0 8 7 0 2 d23 8
0 d32 d33 0 2 1 0 0 3
5 6
1 d12 0
= 7 1 1 1 d12 + 2 d23 8=
0 2 2 2 d23 + 3
The matrices are equal if and only if they have the same number of rows and
columns and their ij-components are all equal. This gives us five non-trivial
3.3. SPECIAL CASES OF AX = D 123
The time dependent solution can be generated by the MATLAB code wire _time
_cur.m, and the graphical solution is illustrated in Figure 3.3.1. The steady
state solution is given by u = u Du + d so that one must solve Du = d
5 65 6 5 6 5 6
2 1 0 0 x1 x0 i w@(f)
9 1 2 1 0 : 9 x2 : 9 0 : 9 i w@(f) :
9 :9 : 9 : 9
7 0 1 2 1 8 7 x3 8 = 7 0 8 + 7 i w@(f) 8 =
:
0 0 1 2 x4 x5 i w@(f)
1.0e+003 *
0.8700
1.2700
1.2700
0.8700
AA [L U] = lu(A) % finds the LU factorization of A
L=
1.0000 0.0000 0.0000 0.0000
-0.5000 1.0000 0.0000 0.0000
0.0000 -0.6667 1.0000 0.0000
0.0000 0.0000 -0.7500 1.0000
U=
2.0000 -1.0000 0.0000 0.0000
0.0000 1.5000 -1.0000 0.0000
0.0000 0.0000 1.3333 -1.0000
0.0000 0.0000 0.0000 1.2500
AA y = L\d % first step using LU
y=
1.0e+003 *
0.4700
0.6350
0.8233
1.0875
AA u = U\y % second step using LU
126 CHAPTER 3. AX = D: UNIQUE SOLUTION
u=
1.0e+003 *
0.8700
1.2700
1.2700
0.8700
AA A3 = [A(:,1) A(:,2) d A(:,4)] % use Cramer’s Rule to find u3
A3 =
2 -1 470 0
-1 2 400 0
0 -1 400 -1
0 0 470 2
AA u3 = det(A3)/det(A)
u3 =
1270
3.3.6 Exercises
1. Find the solution of
5 65 6 5 6
2 0 0 {1 4
7 1 3 0 8 7 {2 8 = 7 14 8 =
3 4 1 {3 19
2. Find the solution of
5 65 6 5 6
2 0 0 {1 h
7 1 0 8 7 {2 8 = 7 2 8 =
2 4 3 {3 10
3. Find the solution of
5 65 6 5 6
1 7 8 6 {1 10
9 0 2 3 4 : 9 {2 : 9 23 :
9 :9 :=9 :
7 0 0 1 2 8 7 {3 8 7 7 8=
0 0 0 3 {4 3
4. Find the solution of
5 65 6 5 6
2 3 7 6 {1 s
9 0 5 0 1 : 9 {2 : 9 2 :
9 :9 :=9 :
7 0 0 7 2 8 7 {3 8 7 15 8 =
0 0 0 4 {4 16
5. Use the LU factors in Example 3.3.3 to solve
5 65 6 5 6
2 1 0 {1 210
7 1 2 1 8 7 {2 8 = 7 10 8 =
0 1 2 {3 80
3.4. ROW OPERATIONS AND GAUSS ELIMINATION 127
6. Solve
5 65 65 6 5 6
1 0 0 2 7 8 {1 1
7 1 2 0 8 7 0 9 10 8 7 {2 8 = 7 2 8 =
2 0 3 0 0 1 {3 3
7. Consider the heat conduction with current in Example 3.3 4. Verify the
solutions of Oy = d and X u = y=
8. Consider the heat conduction with current in Example 3.3 4. Replace
i = 0=1 by i = 0=05, the left end temperature from 70 to 80 and the right end
temperature from 70 to 75. Find the steady state solution.
9. Use MATLAB to do the computations in exercise 7.
10. Use MATLAB to do the computations in exercise 8.
11. Use the MATLAB code wire_time_cur.m to do the time dependent
computations in exercise 8.
If one adds equation one (row one) to equation two (row two), then the new
algebraic system is
;
? {1 {2 + {3 = 5
2{3 = 6
=
4{2 + {3 = 1=
128 CHAPTER 3. AX = D: UNIQUE SOLUTION
Note this is in upper triangular form X x = db where X = S23 H21 (1) D and
b = S23 H21 (1) d= Since adding and interchanging equations can be reversed,
d
the solution of the upper triangular system must also be a solution of the
original system. The solution is {3 = 6@2 = 3> {2 = (1 3)@4 = 1 and
{1 = (5 3 + (1)) = 1=
There are three row operations, which are used to transform matrices to
upper triangular matrices. They can be used to solve algebraic systems, find
the OX factors, evaluate determinants, find multiple solutions and eigenvectors.
The row operations can be represented by elementary matrices Hlm (d), Slm and
Hl (f)=
Add d times (urz_m) to urz_l. This can be represented by an q × q
elementary matrix Hlm (d) with the only non-zero components being ones on the
diagonal and d equal to the ij-component. For example, for q = 3> d = 2>
l = 3 and m = 2 5 6
1 0 0
H32 (2) = 7 0 1 0 8 =
0 2 1
Note, Hlm (d) is the inverse operation to Hlm (d), that is, Hlm (d) Hlm (d) = L=
For example,
5 65 6 5 6
1 0 0 1 0 0 1 0 0
H32 (2) H32 (2) = 7 0 1 0 8 7 0 1 0 8 = 7 0 1 0 8=
0 2 1 0 2 1 0 0 1
0 0 0 1
0 0 0 1
long as the diagonal components of the upper triangular matrix are not zero.
The following 2 × 2 example illustrates this may not always be possible
{1 + {2 = 1 and
2{1 + 2{2 = 3=
The augmented matrix is
¸
1 1 1
[D d] = =
2 2 3
In order to obtain a zero in the 21-component add (2) times urz_1 to urz_2
¸ ¸ ¸
1 0 1 1 1 1 1 1 b
H21 (2) [D d] = = = [X d]=
2 1 2 2 3 0 0 1
The 22-component of X is zero! Moreover, the second row is a shorthand way
of writing 0{1 + 0{2 = 1> which is impossible. Such algebraic systems are called
inconsistent and have no solution.
Stage 2. The last row corresponds to the equation 5{3 = 5 and so {3 = 1= Put
this into the equations corresponding to the rows 1 and 2 to get
Then {2 = 0 and {1 = 1=
Example 3.4.2. Consider another 3 × 3 algebraic system
;
? 2{1 + 6{2 + 8{3 = 16
4{1 + 15{2 + 19{3 = 38
=
2{1 + 3{3 = 8=
Stage 1. Get zeros in the first column by adding (2) times urz_1 to urz_2,
and then adding (1) times urz_1 to urz_3
5 6
2 6 8 16
H31 (1) H21 (2) [D d] = 7 0 3 3 6 8=
0 6 5 8
Stage 2. The last row corresponds to the equation 1{3 = 4 and so {3 = 4= Put
this into the equations corresponding to the rows 1 and 2 to get
l1 l2 + 40@(11) = 0 and
l2 4(40@(11)) = 10=
Example 3.4.4. Consider the steady state heat wire in Example 3.3.4 where
there were five segments and electrical current generating heat within the wire.
The algebraic problem for the temperature in each interior segment is
5 65 6 5 6
2 1 0 0 x1 470
9 1 2 1 0 : 9 x2 : 9 400 :
9 :9 : 9 :
7 0 1 2 1 8 7 x3 8 = 7 400 8
0 0 1 2 x3 470
and the augmented matrix is
5 6
2 1 0 0 470
9 1 2 1 0 400 :
[D d] = 9 :
7 0 1 2 1 400 8 =
0 0 1 2 470
Stage 1. Obtain zeros in the subdiagonal by using
5 6
2 1 0 0 470=00
9 0 3@2 1 0 635=00 :
H43 (3@4) H32 (2@3) H21 (1@2) [D d] = 9
7 0 0 4@3 1 823=33
:=
8
0 0 0 5@4 1087=50
Stage 2. The last row corresponds to the equation (5@4)x4 = 1087=5 and so
x4 = 1087=50@1=25 = 870= Put this into the equations corresponding the rows
1, 2 and 3 to get x3 = x2 = 1270 and x1 = 870= The reader should note the
similarities of these calculations with the LU factorization method as given in
Subsection 3.3.4.
Now equate the first and second components in each vector equation to obtain
a system with six scalar equations
;
A
A i1 i2 f = 0
A
A
A
A i2 v = 10
?
i2 f i4 = 0
A
A i2 v i3 = 0
A
A
A
A i1 i5 f i6 = 0
=
i3 + i5 v = 0=
In order to do the first stage of Gauss elimination, one must use row oper-
ation to transform the lower portion of the augmented matrix to zero compo-
nents. Fortunately, this matrix has many zero components. The required row
3.4. ROW OPERATIONS AND GAUSS ELIMINATION 135
The negative signs indicate bars 1, 3 and 6 are under compression, and the other
bars 2, 4 and 5 are under tension. As the angle decreases, the magnitudes of
the forces increase, except for bar 3.
136 CHAPTER 3. AX = D: UNIQUE SOLUTION
3.4.6 Exercises
1. Find the augmented matrix for
;
? 2{1 2{2 + {3 = 7
4{1 + {2 + {3 = 0
=
3{2 + {3 = 2=
2. Find the augmented matrix for
;
A
A {1 2{2 + {3 5{4 = 1
?
{1 2{2 + 5{4 = 1
A
A 3{1 2{ 2 + {3 + 2{4 = 0
=
4{1 + {3 {4 = 5=
3. Let q = 4 and find the elementary matrices H43 (7)> S23 and H4 (10)=
4. Let q = 3 and find the elementary matrices H21 (2)> S13 and H3 ()=
5. Consider the algebraic system
;
? {1 + 7{2 + {3 = 1
2{1 + 0{2 + 7{3 = 2
=
14{2 + {3 = 6=
3.4. ROW OPERATIONS AND GAUSS ELIMINATION 137
16. Use MATLAB to find the solutions in the six-bar truss problem for =
@6> @4 and 2@6= You may use the MATLAB code support.m.
17. Use MATLAB to find the solutions in the six-bar truss with two weights
as specified in exercise 14. You may modify the MATLAB code support.m, and
experiment with dierent weights and angles.
Dx =D(D1 d) = (DD1 )d = Ld = d=
Unfortunately, one can not always find such matrices! We shall see in Section
3.7 D with a non-zero determinant implies that an inverse matrix exists.
Basic Examples.
1. Elementary Matrices.
If D = Hlm (d)> then D1 = Hlm (d)= If D = Slm > then D1 = Slm = If D = Hl (f)>
then D1 = Hl (1@f)=
2. Diagonal Matrices.
If D is a diagonal matrix with non-zero diagonal components dll 6= 0> then D1
is a diagonal matrix with diagonal components equal to 1@dll = For example, if
q = 3 and 5 6
4 0 0
D = 7 0 2 0 8,
0 0 3
then because DD1 = L
5 6
1@4 0 0
D1 = 7 0 1@2 0 8 =
0 0 1@3
3. Block Diagonal Matrices.
If D is a block diagonal matrix with diagonal blocks that have inverses, then
D has an inverse matrix which is also a block diagonal matrix. For example,
consider a 5 × 5 matrix with two diagonal blocks
5 6
1 0 0 0 0
¸ 9 0 1 0 0 0 :
D11 9 :
D = =99 0 4 1 0 0 : : where
D22 7 0 0 0 2 0 8
0 0 0 0 3
5 6
1 0 0 ¸
2 0
D11 = H32 (4) = 7 0 1 0 8 and D22 = =
0 3
0 4 1
Both D11 and D22 have inverses and, hence, the inverse of D is
5 6
1 0 0 0 0
1 ¸ 9 0 1 0 0 0 :
D11 9 :
D1 = 1 =99 0 4 1 0 0 :.
:
D22 7 0 0 8
0 1@2 0
0 0 0 0 1@3
4. D is a 2 × 2 Matrix with ghw(D) 6= 0=
The inverse matrix E must satisfy the matrix equation
DE = L
£ ¤ £ ¤
D b1 b2 = e1 e2 =
This is equivalent to two vector equations
Db1 = e1 and Db2 = e2 =
140 CHAPTER 3. AX = D: UNIQUE SOLUTION
One can solve these by either Cramer’s rule or Gauss elimination. For example,
use Cramer’s rule to find the inverse of
¸
d11 d12
D= =
d21 d22
Db1 = e1
¸ ¸ ¸
d11 d12 e11 1
= =
d21 d22 e21 0
Then
¸
1 d12
e11 = det( )@ det(D) = d22 @ det(D) and
0 d22
¸
d11 1
e21 = det( )@ det(D) = d21 @ det(D).
d21 0
Db2 = e2
¸ ¸ ¸
d11 d12 e12 0
= =
d21 d22 e22 1
Then
¸
0 d12
e12 = det( )@ det(D) = d12 @ det(D) and
1 d22
¸
d11 0
e22 = det( )@ det(D) = d11 @ det(D).
d21 1
The alternative is to use row operations and Gauss elimination. The Gauss-
Jordan method is a variation of Gauss elimination, which can be adapted to
larger matrices and to computers.
First, use row operations and the augmented matrix to solve Db1 = e1
¸
2 1 1
[D e1 ] =
1 2 0
¸ ¸ ¸
1 0 2 1 1 2 1 1
H21 (1@2) [D e1 ] = = =
1@2 1 1 2 0 0 3@2 1@2
At this point either use backward substitution or more row operation to obtain
an identity matrix in the left side of the augmented matrix. The row operations
include two row multiples and then adding (1@2) times urz_2 to urz_1 :
¸
1 1@2 1@2
H2 (2@3) H1 (1@2) H21 (1@2) [D e1 ] =
0 1 1@3
¸
1 0 2@3
H12 (1@2) H2 (2@3) H1 (1@2) H21 (1@2) [D e1 ] = =
0 1 1@3
The right column in the transform augmented matrix is the first column of the
inverse matrix. Let H be the product of the four elementary matrices and write
the above as
H [D e1 ] = [L b1 ]=
In order to find the second column of the inverse matrix, the same row opera-
tions are needed to transform the augmented matrix from [D e2 ] to [L b2 ]
H [D e2 ] = [L b2 ]=
An e!cient way to do both these solves is to combine the two columns of the
inverse matrix into the right side of a larger augmented matrix. That is, use
the row operations to transform [D e1 e2 ] = [D L] to [L b1 b2 ] = [L E]
H [D e1 e2 ] = [L b2 b2 ]=
In order to find the inverse of an q × q matrix D> one must solve q vector
equations
DE = L
£ ¤ £ ¤
D b1 b2 · · · bq = e1 e2 · · · eq
£ ¤ £ ¤
Db1 Db2 · · · Dbq = e1 e2 · · · eq or
Dbm = em for m = 1> = = = > q=
Use row operations and elementary matrices to transform the augmented matrix
[D L] to [L D1 ]=
Definition 3.5.2. Consider the q × q matrix D and find the inverse matrix, if
it exists. The Gauss-Jordan method for solving this system has two stages:
Stage 1. Transform the augmented matrix [D L] to upper triangular form
[X O]=
142 CHAPTER 3. AX = D: UNIQUE SOLUTION
(a). start with the left column, column m = 1, and use row operations
to transform column m = 1 to zeros below row l = 1>
(b). move to the next column, column m = 2, and use row operations
to transform column m = 2 to zeros below row l = 2 and
(c). repeat this until column m = q 1 has been done.
Stage 2. Transform the upper triangular matrix [X O] to [L D1 ]
(a). use elementary matrices so that the diagonal of U is transformed
to have ones on all of the diagonal components,
(b). start with the right column of X , column m = q, and use row operations
to transform column m = q to zeros above row l = q>
(c). move to the next left column, column m = q 1, and use row operations
to transform column m = q 1 to zeros above row l = q 1 and
(d). repeat this until column m = 2 has been done.
Example 3.5.1. Use the Gauss-Jordan method to find the inverse of
5 6
1 2 1
D = 7 1 3 2 8=
1 0 1
Stage 2. Divide urz_3 by 2 and then subtract it from urz_2 and urz_1
H13 (1) H23 (1) H3 (1@2) H32 (2) H31 (1) H21 (1) [D L] =
5 6
1 2 0 5@2 1 1@2
7 0 1 0 1@2 0 1@2 8 =
0 0 1 3@2 1 1@2
Finally, subtract (2) times urz_2 from urz_1
H12 (2) H13 (1) H23 (1) H3 (1@2) H32 (2) H31 (1) H21 (1) [D L] =
3.5. INVERSE MATRICES 143
5 6
1 0 0 3@2 1 1@2
7 0 1 0 1@2 0 1@2 8 =
0 0 1 3@2 1 1@2
One can easily verify the inverse is given by the right side of the transformed
augmented matrix 5 6
3@2 1 1@2
D1 = 7 1@2 0 1@2 8 =
3@2 1 1@2
The solution of Dx = d = [1 2 3]W is easily computed by using the inverse
matrix 5 65 6 5 6
3@2 1 1@2 1 1
x = D1 d = 7 1@2 0 1@2 8 7 2 8 = 7 1 8 =
3@2 1 1@2 3 2
Example 3.5.2. Use the Gauss-Jordan method to find the inverse of
5 6
0 5 6
D = 7 1 3 1 8=
2 1 1
Stage 1. Because the 11-component is zero, urz_1 and urz_2 (or urz_3)
must be interchanged
5 6
1 3 1 0 1 0
S12 [D L] = 7 0 5 6 1 0 0 8
2 1 1 0 0 1
H12 (3) H13 (1) H23 (6@5) H3 (1@2) H2 (1@2) H32 (1) H31 (2) S12 [D L] =
5 6
1 0 0 2@25 1@25 13@25
7 0 1 0 1@25 12@25 6@25 8 =
0 0 1 5@25 10@25 5@25
The inverse is the right side of the transformed augmented matrix
5 6
2 1 13
D1 = (1@25) 7 1 12 6 8 =
5 10 5
Example 3.5.3. Consider the steady state heat wire problem in Sections 3.3
and 3.4 where there were five segments and current generating heat within the
wire. The algebraic problem for the temperature in each interior segment is
5 65 6 5 6
2 1 0 0 x1 470
9 1 2 1 0 : 9 x2 : 9 400 :
9 :9 : 9 :
7 0 1 2 1 8 7 x3 8 = 7 400 8 =
0 0 1 2 x3 470
Stage 2. Multiply rows 1-4 by 1@2> 2@3> 3@4 and 4@5, respectively, to get for
G H4 (4@5) H3 (3@4) H2 (2@3) H1 (1@2)
H12 (1@2) H23 (2@3) H34 (3@4) G H43 (3@4) H32 (2@3) H21 (1@2) [D L] =
5 6
1 0 0 0 8@10 6@10 4@10 2@10
9 0 1 0 0 6@10 12@10 8@10 4@10 :
9 :=
7 0 0 1 0 4@10 8@10 12@10 6@10 8
0 0 0 1 2@10 4@10 6@10 8@10
So, the inverse matrix is
5 6
8 6 4 2
9 6 12 8 4 :
D1 = (1@10) 9 :
7 4 8 12 6 8 >
2 4 6 8
Db1 = e1
¸ ¸ ¸
1 1 e11 1
=
2 2 e21 0
e11 + e21 = 1 and 2e11 + 2e21 = 0!
146 CHAPTER 3. AX = D: UNIQUE SOLUTION
In the last section of this chapter determinants of q×q matrices will be defined,
and any matrix with a non-zero determinant will be shown to have an inverse
matrix.
One very important property of inverse matrices is the solution of Dx = d
is D1 d. Other properties are summarized in the following theorem.
The inverse of A is
5 65 6 5 6
1 0 0 1@4 0 0 1@4 0 0
D1 = D1 1
2 D1 = 7 2 1 0 8 7 0 1@2 0 8 = 7 2@4 1@2 0 8 =
0 0 1 0 0 1 0 0 1
3.5. INVERSE MATRICES 147
If all the diagonal components of X are not zero, the solutions of Dx = d and
X x = HDx = Hd = d b are the same.
0
1
In the previous section an application to a six-bar truss generated a 6 × 6
matrix. The inverse of the matrix can be found using the MATLAB code sup-
port.m, and the interested reader should examine this application and exercises
10 and 11.
3.5.5 Exercises
¸
3 1
1. Let D = =
1 3
(a). Use Cramer’s rule to find the inverse of D=
(b). Use Gauss-Jordan method to find the inverse.
(c). Find the inverse of
5 6
3 1 0 0
9 1 3 0 0 :
9 :
7 0 0 3 0 8=
0 0 0 4
5 6
1 1 1
2. Let D = 7 1 0 3 8 =
0 2 3
(a). Use the Gauss-Jordan method to find the inverse matrix.
W
(b). 6 Dx = d = [0 10 20] =
Use5the inverse to solve
3 1 0
3. Let D = 7 1 3 1 8 =
0 1 3
(a). Use the Gauss-Jordan method to find the inverse matrix.
(b). Use5the inverse 6to solve Dx = d = [1 2 3]W =
0 2 1
4. Let D = 7 1 3 5 8 =
2 2 0
(a). Use the Gauss-Jordan method to find the inverse matrix.
(b). Use the inverse to solve Dx = d = [4 3 5]W .
5. Verify the5 row operations
6 used in Example 3.5.3.
1 2 0 1
9 2 1 0 0 :
6. Let D = 97 0 0 2 0 8=
:
1 2 0 2
(a). Use the Gauss-Jordan method to find the inverse matrix.
W
(b). Use the 5 = [2 1 4 61] =
5 inverse to 6solve Dx = d
1 0 0 1 2 0
7. Let D1 = 7 0 1 0 8 and D2 = 7 0 1 0 8 =
4 0 1 0 0 1
3.6. OX FACTORIZATION 149
3.6 OX Factorization
If the given matrix can be written as a product of a lower and upper triangular
matrices, then one may be able to solve Dx = d= Suppose D = OX and both O
and X have non-zero diagonal components. By the associative property
Dx = (OX )x = O(X x) = d=
By using the tridiagonal structure of the matrix we were able to find factors
5 6 5 65 6
2 1 0 1 0 0 2 1 0
7 1 2 1 8 = 7 1@2 1 0 8 7 0 3@2 1 8
0 1 2 0 2@3 1 0 0 4@3
D = OX=
The objective of this section is to show how one can use row operations and
elementary matrices to possibly find the OX factors for more general matrices.
Block versions of the OX factors will be described and applied to a three-loop
circuit problem.
Second, one can use row operations and elementary matrices to find OX factors
¸
2 1
D =
1 2
¸
2 1
H21 (1@2) D = = X=
0 3@2
Now let O be the inverse of the elementary matrix H21 (1@2) giving
¸
1 1 2 1
H21 (1@2) (H21 (1@2) D) = H21 (1@2) = OX
0 3@2
¸ ¸
1 0 2 1
D = = OX=
1@2 1 0 3@2
Third, the lower and upper matrices may not have an inverse
¸
1 1
D =
1 1
¸
1 1
H21 (1) D = =X
0 0
¸ ¸
1 0 1 1
D = = OX=
1 1 0 0
3.6. OX FACTORIZATION 151
Example 3.6.2. This example does require the interchange of rows to find the
factorization of the 3 × 3 matrix
5 6
0 5 6
D = 7 1 3 1 8=
2 1 1
O(X x) = S12 d
5 6 5 65 6 5 6
1 0 0 1 3 1 {1 2
7 0 1 0 8 (7 0 5 6 8 7 {2 8) = 7 6 8 =
2 1 1 0 0 5 {3 3
¸ Dx¸ = d
¸
D11 D12 x1 d1
= = (3.6.1)
D21 D22 x2 d2
The block elementary matrix has an inverse, which is very easy to find
¸1 ¸
L1 ] L1 ]
= =
D21 D1
11 L2 D21 D1
11 L2
154 CHAPTER 3. AX = D: UNIQUE SOLUTION
Both the block lower and block upper triangular matrices will have inverses
provided the diagonal blocks have inverses.
Example 3.6.3. Consider the above 5 × 5 matrix with q1 = 2, q1 = 3>
¸ ¸
2 1 1 0 1
D11 = , D12 =
1 2 0 1 0
5 6 5 6
1 0 2 0 0
D21 = 7 0 1 8 and D22 = 7 0 2 0 8 =
1 0 0 0 2
b22 D22 D21 D1 D12
D is easy to
compute and has an inverse
11
5 6 5 6
2 0 0 1 0 ¸ ¸
b22 7 0 2 2@3 1@3 1 0 1
D 0 87
0 1 8
1@3 2@3 0 1 0
0 0 2 1 0
5 6
4 1 2
= 1@3 7 1 4 1 8 and
2 1 4
5 6
5 2 3
b1
D = 1@4 7 2 4 2 8 =
22
3 2 5
If D and D11 have inverses, then the left side is a product of three matrices with
inverses. Consequently, the product on the left side must have an inverse, and
the 2 × 2 block diagonal matrix on the right side must have an inverse. This
means the 22-block Db22 has an inverse. The following theorem summarizes the
above results.
The battery increases the potential from zero to E. Within the battery there is
a small resistance r so that by Ohm’s law there is a drop in potential of lu from
H to H lu, which is depicted by the segment from points d to e= The segment
from e to f has very small resistance and is neglected. The larger resistance
U between points f and g has a potential drop equal to lU from H lu to
H lu lU = 0=
The derivation of the five voltage equations and two current equations at
nodes one and two assumes the potentials at nodes one and two are given by {1
and {2 and the potential at the bottoms nodes is set to zero. The voltage drops
across resistors Un for n = 1> · · · > 5 are plus or minus ln Un , which depends on
the direction of the current.
Voltage from node one to ground:
{1 + H1 l1 U1 = 0 and
{1 l3 U3 = 0=
Voltage from node one to node two:
{1 + H2 (l2 )U2 = {2 .
Voltage from node two to ground:
{2 + H3 (l5 )U5 = 0 and
{2 l4 U4 = 0=
At each node the sum of the currents must be zero:
l1 + l2 l3 = 0 and
l2 l4 + l5 = 0=
3.6. OX FACTORIZATION 157
3.6.5 Exercises
1. Let D be the 4 × 4 matrix
5 6
2 1 0 0
9 1 2 1 0 :
D=9 :
7 0 1 2 1 8 =
0 0 1 2
(a). Use row operations and elementary matrices to find the OX factors.
(b). Use these to solve Dx = d = [470 400 400 470]W =
2. Let D be the 3 × 3 matrix
5 6
1 2 1
D = 7 1 3 6 8=
1 0 1
(a). Use row operations and elementary matrices to find the OX factors.
(b). Use these to solve Dx = d = [1 2 3]W =
3. Let D be the 3 × 3 matrix in Example 3.6.2. Interchange row one and
three, that is, use S13 and not S12 .
(a). Use row operations and elementary matrices to find the OX factors.
(b). Use these to solve Dx = d = [6 3 2]W =
3.7. DETERMINANTS AND CRAMER’S RULE 159
(a). Use row operations and elementary matrices to find the OX factors.
(b). Use these to solve Dx = d = [6 3 2]W =
5. Let D be the 4 × 4 matrix in exercise 1.
(a). View this as a block 2 × 2 matrix where q = 4 = 2 + 2 = q1 + q2 =
Find the four blocks and the Schur complement matrix D b22 =
(b). Use equations (3.6.2) and (3.6.3) to solve Dx = d = [470 400 400
470]W =
6. Let D be the 5 × 5 matrix
5 6
2 0 1 0 1
9 0 3 0 2 0 :
9 :
9 1 0 3 1 0 : =
9 :
7 0 1 1 3 1 8
1 1 0 1 3
Here we have interchanged rows and columns and the determinant does not
change. That is, det(D) = det(DW ) where DW is called the transpose of A and
is defined as 5 6 5 W 6
d11 d21 d31 a
DW 7 d12 d22 d32 8 = 7 bW 8 =
d13 d23 d33 cW
Cramer’s rule, for det(D) 6= 0> allows one to easily solve the 3 × 3 algebraic
systems Dx = d
The justification of the formula for the first unknown follows from the properties
of the cross product and the column version of a matrix-vector product
Dx = d
{1 a + {2 b + {3 c = d
{1 a = d {2 b {3 c=
Recall the cross product b × c is perpendicular to both b and c= Use this fact
and take the dot product of both sides with b × c
({1 a) • (b × c) = (d {2 b {3 c) • (b × c)
(a • (b × c)){1 = d • (b × c) b • (b × c){2 c • (b × c){3 )
det(D){1 = det([d b c]) 0{2 0{3 =
The derivations of the equations for the other two unknowns are similar. Fur-
thermore, these concepts generalize to algebraic systems with q unknowns.
det(D) d11 (1)1+1 P11 + d12 (1)1+2 P12 + · · · + d1q (1)1+q P1q =
The coe!cients of dlm > (1)l+m Plm > are called cofactors.
For q = 3 it is easy to expand the 2 × 2 determinants to reveal the following
seven rules:
162 CHAPTER 3. AX = D: UNIQUE SOLUTION
Example 3.7.1. Consider the matrix in Sections 3.1 and 3.4 (Example 3.4.3)
from the two-loop circuit. First, the determinant is evaluated by expanding
down column one, which has only two non-zero components,
5 6
1 1 1
det(D) = det(7 1 0 3 8)
0 2 3
¸ ¸
0 3 1 1
= (1)2 (1) det( ) + (1)2+1 (1) det( )
2 3 2 3
= 6 5 = 11=
One can also use the following elementary row operations to transform the
matrix to upper triangular form
5 6
1 1 1
H32 (2) H11 (1) D = X = 7 0 1 4 8 =
0 0 11
Apply rule three twice to get det(H32 (2)(H11 (1) D)) = det(H11 (1) D) = det(
D) = det(X )= The determinant of an upper triangular matrix can be expanded
by the left column so that the determinant is the product of the diagonal com-
ponents. Therefore, det( D) = det(X ) = (1)(1)(11) = 11=
Example 3.7.2. Consider the 4 × 4 matrix for the heat conduction problem in
Sections 3.3 and 3.4 (Example 3.4.4) and expand it by column one using rule
3.7. DETERMINANTS AND CRAMER’S RULE 163
one
5 6
2 1 0 0
9 1 2 1 0 :
det(D) = det(9
7 0 1
:)
2 1 8
0 0 1 2
5 6 5 6
2 1 0 1 0 0
= 2 det(7 1 2 1 8) (1) det(7 1 2 1 8)
0 1 2 0 1 2
= 2(4) 3 = 5=
Use the third rule three times and expand the upper triangular matrix by the
left column to get
Dx = d
{1 a:1 + · · · + {q a:q = d=
In the case q = 3, we computed the dot product of both sides with the cross
product of the second and third columns. Since the cross product of two vectors
is perpendicular to both the vectors, the dot product on the right side simplified
to the det([d a:2 a:3 ]) and, consequently, {1 = det([d a:2 a:3 ])@ det(D)=
164 CHAPTER 3. AX = D: UNIQUE SOLUTION
To see how this is established, consider the reduced notational burden of the
case q = 4= We must find {1 in
The other three determinants are also expanded by column one and are equal
to zero because they have two identical columns.
By using other columns of cofactors one can solve for the other unknowns.
3.7. DETERMINANTS AND CRAMER’S RULE 165
The other two columns in the inverse matrix may be computed in a similar
fashion to yield 5 6
3 2 1
D1 = (1@2) 7 1 0 1 8 =
3 2 1
If any column of an q×q matrix is a linear combination of the other columns,
then the determinant of the matrix must be zero. Equivalently, if the deter-
minant of the matrix is not zero, then no column of the matrix D is a linear
combination of the other columns (the columns are linearly independent, see
Section 5.1 for a more general discussion)
tank, and the dierential equations for the center and right tanks are similar,
generating a system of three dierential equations
; 0 g{
? { = gw = (1 + 4(|@24)) 10({@24)
| 0 = g|
gw = (2 + 10({@24) + 2(}@24)) (4(|@24) + 8(|@24))
= 0 g}
} = gw = 8(|@24) (2(}@24) + 6(}@24))=
The solution of the system of dierential equations is more complicated than
the system for the projectile problem in Subsection 2.3.3. The time dependent
solution will be discussed in more detail in chapters six and seven.
Presently, consider the steady state solution, which requires all amounts
to be independent of time so that their time derivatives are zero. The above
system of dierential equations becomes an algebraic system
;
? 0 = (1 + 4(|@24)) 10({@24)
0 = (2 + 10({@24) + 2(}@24)) (4(|@24) + 8(|@24))
=
0 = 8(|@24) (2(}@24) + 6(}@24))=
The equivalent vector equation is
5 65 6 5 6
10@24 4@24 0 { 1
7 10@24 12@24 2@24 8 7 | 8 = 7 2 8 =
0 8@24 8@24 } 0
This has a solution because the determinant of the matrix is not zero
5 6
10@24 4@24 0 ¸
12@24 2@24
det(7 10@24 12@24 2@24 8) = (10@24)
8@24 8@24
0 8@24 8@24
¸
10@24 2@24
(2@24)
0 8@24
= 5@144=
Cramer’s rule can easily be used to find the steady state solutions { = 7=2>
| = 12=0 and } = 12=0. In chapter seven we will show that the time dependent
solutions will approach the steady state solutions as time goes to infinity.
168 CHAPTER 3. AX = D: UNIQUE SOLUTION
Multiplying both sides by 24 and use Cramer’s rule to find the solution.
AA A = [-10 4 0;
3.7. DETERMINANTS AND CRAMER’S RULE 169
10 -12 2;
0 8 -8];
AA det(A)
ans =
-480
AA d = [-24 -48 -72]’;
AA A\d % uses Gauss elimination
ans =
8.4000
15.0000
24.0000
AA A1 = [d A(:,2) A(:,3)]; % uses Cramer’s rule
AA A2 = [A(:,1) d A(:,3)];
AA A3 = [A(:,1) A(:,2) d];
AA x = det(A1)/-480
x=
8.4000
AA y = det(A2)/-480
y=
15
AA z = det(A3)/-480
z=
24.
3.7.6 Exercises
1. Consider Example 3.7.1 and verify
(a). det(DW ) = det(D) and
(b). det(S23 D) = det(D), that is, interchange rows 2 and 3.
2. Consider Example 3.7.1 and verify
(a). det(H32 (10) D) = det(D)> that is, multiply row 2 by 10 and add to
row 3 and
(b). det(H2 (4)D) = 4 det(D), that is, multiply row 2 by 4.
3. Consider Example 3.7.1 and compute the determinant
(a). by expanding row l = 3 and
(b). by expanding column m = 2.
4. Let D be the 4 × 4 matrix
5 6
3 1 0 0
9 1 3 1 0 :
D=9 7 0 1 3 1 8 =
:
0 0 1 3
Ax = d: Least Squares
Solution
This chapter contains methods for "solving" matrix equations of the form
Dx = d where D is a matrix with more rows (equations) than columns (vari-
ables). The normal equations will be developed and used to solve these over
determined algebraic systems. Applications include data fitting such as sales
predictions, radioactive decay, world population models, flow rate data and
parameter identification. By-hand calculations will be done as well as imple-
mentations in MATLAB for larger and nonlinear problems.
171
172 CHAPTER 4. AX = D: LEAST SQUARES SOLUTION
Dx
¸ = d
p
[a b] = d
f
pa + fb = d
is the p and f that forces the residual vector r d (pa + fb) = [u1 u2 · · · uq ]
to be as "close" as possible to the zero vector. If the sum of the squares of the
residual vector’s components is a minimum, then this is called the least squares
solution. The objective is to find p and f that minimize
This problem will be solved three ways: graphically, calculus and normal equa-
tions (in Section 4.2).
For the above high definition television data a = [1 2 3]W , b = [1 1 1]W
and d = [2000 1950 1910]W and gives
This is a special case of the normal equations, and as we shall see they give a
solution to the least squares problem even if the number of data points is larger
than q = 3.
4.1. CURVE FITTING TO DATA 173
Example 4.1.1. Suppose a new computer company has had increased sales
and would like to make a prediction of future sales based on past sales. During
the past six months sales have been 78, 85, 90, 95, 104 and 113 computers, see
Table 4.1.1.
A first look at this data suggests an increase of about 8 computers each
month. A more precise way to view this is to determine the slope and intercept
so that | = pw + f is "closest" to the data points as is depicted in the left graph
in Figure 4.1.1, which will be discussed in detail in the next section. Each of
the data points can be inserted into the linear equation and gives a system with
six equations
;
A
A p1 + f = 78
A
A
A
A p2 + f = 85
?
p3 + f = 90
A
A p4 + f = 96
A
A
A
A p5 + f = 104
=
p6 + f = 113=
of time where the three coe!cients are chosen so that the quadratic function
is increasing and concave up for times near the data. Let | = dw2 + ew + f
where time starts in 1900, w = 0, and ends in 2000, w = 100= There are q = 11
equations giving the algebraic system
;
A
A d02 + e0 + 1f = 1=65
A
A 2
? d102 + e10 + 1f = 1=75
A
d20 + e20 + 1f = 1=86
A
A ..
A
A .
A
=
d1002 + e100 + f = 6=06=
The equivalent matrix equation uses an 11 × 3 matrix D and has the form
5 6 5 6
0 0 1 1=65
9 102 10 :
1 5 6 9 1=75 :
9 : d 9 :
9
9 202 20 :7 9 :
: e 8 = 9 1=86 : =
1
9 .. .. :
.. 9 .. :
7 . . 8 f
. 7 . 8
1002 100 1 6=06
In this case the least square function I (d> e> f) depends on the data and the
three unknown coe!cients d> e and f. The normal equations as described in
the Section 4.2 will be used to solve this system.
Compute the first derivative with respect to p and set it equal to zero
gi
= 0 = 0 2dW a 0 + 2aW bf0 + aW a2p + 0=
gp
This derivative is also called the partial derivative of F(m,c) with respect to the
CI
m variable and is written as Cp =
Fix p = p0 and consider I (p> f) as a function of f. Define the j(f)
j(f) I (p0 > f) = dW d2dW ap0 2dW bf + 2aW bp0 f + aW ap20 + bW bf2 =
4.1. CURVE FITTING TO DATA 177
Compute the first derivative with respect to f and set it equal to zero
gj
= 0 = 0 02dW b + 2aW bp0 + bW b2f + 0=
gf
This derivative is also called the partial derivative of F(m,c) with respect to the
c variable and is written as CICf =
These two equations can be written in vector form
W ¸ ¸ W ¸
a a aW b p0 a d
= =
bW a bW b f0 bW d
Note they are the same as in equations (4.1.2), but now the length of the vectors
is not restricted to q = 3! The matrix equation can also be written as
DW Dx = DW d where ¸ ¸
W aW p0
D = [a b]> D = and x = =
bW f0
This is called the normal equations corresponding to the least squares problem
Dx = d=
Return to the sales prediction problem in Example 4.1.1 where
a = [1 2 3 4 5 6]W
b = [1 1 1 1 1 1]W and
d = [78 85 90 96 104 113]W >
giving aW a = 91> aW b = 21 = bW a, bW b = 6, aW d = 2100 and bW d = 566= The
normal equation is
¸ ¸ ¸
91 21 p0 2100
= >
21 6 f0 566
and the solution is easily computed to be p0 = 34@5 = 6=8000 and f0 =
1058@15 70=5333 with I (p0 > f0 ) 8=1333= These numbers are consistent
with the estimates given by the graphical approach in Figure 4.1.2. Now use
the equation | = p0 w + f0 to predict any future sales. For example, at the
month w = 15 the predicted sales is
|(15) = p0 15 + f0 = 6=8000(15) + 70=5333 172=
The data, straight line model | = p0 w + f0 and prediction are illustrated in
Figure 4.1.1, which was generated by the MATLAB code market.m.
The normal equations are solved, see the MATLAB code decay.m, to give p =
0=0439 and f = 5=0372= This implies u = p = 0=0439 and x(0) = hln(x(0) =
hf = h5=0372 = 154=0352 giving
AA b’*d
ans =
566
AA A’*A % matrix product for normal equations
ans =
91 21
21 6
AA A’*d
ans =
2100
566
AA lssol=(A’*A)\A’*d % solution via Gauss elimination
lssol =
6.8000
70.5333
AA lssol2 = A\d % better method...uses QR factors
lssol2 =
6.8000
70.5333
The reader should examine the MATLAB codes market.m and decay.m for
additional least squares computations.
4.1.6 Exercises
1. Consider the least squares solution of the system
;
? 1p + f = 1
2p + f = 3
=
3p + f = 7=
3. Consider the computer sales problem in Example 4.1.1 after two addi-
tional months of sales: month 7 there were 120 computers sold and month 8
there were 131 computers sold. Consider all 8 data points, see Table 4.1.1.
(a). Find the least squares function in equation (4.1.1).
(b). Solve the system in equation (4.1.2).
(c). Compute | = pw + f at time w = 15 months.
4. Consider tuition, fees and books (lumped as costs) at University Expen-
sive. These have risen over the past four years and one would like to predict
what they will be four years from now. Suppose
Years Costs
1 11,300
2 11,900
3 12,700
4 13,400
9. Use the MATLAB codes decay.m to examine sensitively of the least squares
solution when there is a 5% error or uncertainty in the data for the radioactive
decay problem.
previous section we shall define "small" in terms of the least squares function
W
I (x) r(x) r(x), which is a function from n-dimensional space into the real
numbers, that is, I : Rq $ R=
Definition 4.2.2. Consider Dx = d where D is p × q and p A q. The
least squares solution is an q × 1 column vectors x such that the least squares
function evaluated at this x is less than or equal to all other evaluations of the
least squares function
In the previous section with two variables it was not too complicated to
solve the minimization problem. For larger number of columns q A 2, it is
useful to take advantage of the properties of the transpose matrix to derive the
normal equations. Let y = x + (y x) where x is the least squares solution.
Then
r(y) = d Dy
= dD(x + (y x))
= dDxD(y x)
= r(x)D(y x)=
Put this into the least squares function and use (D(y x))W (D(y x)) 0
If DW r(x) = 0> then for all y I (y) I (x)= The vector equation DW r(x) = 0
can be written as
DW r(x) = 0
W
D (d Dx) = 0
DW Dx = DW d=
If the q × q matrix DW D has an inverse, then the solution of this equation will
also be the solution of the least squares problem.
Definition 4.2.3. Consider Dx = d where D is p × q and p A q. The normal
equation uses the q × q matrix DW D and is
DW Dx = DW d=
and the solution is {1 = 45=00 and {2 1863=33= The column vectors in the
3 × 2 matrix are linearly independent because Dz = 0 implies z = 0 as follows
from
5 6 Dz = 5
0 6
1 1 ¸ 0
7 2 1 8 }1 = 7 0 8 or equivalently
}2
3 1 0
}1 + }2 = 0
2}1 + }2 = 0 and
3}1 + }2 = 0=
The only way the three equations can hold is if }1 = }2 = 0, that is, z = 0=
In order to show DW D has an inverse, suppose DW Dz = 0 and show z = 0=
Use the associative and the transpose properties
zW (DW Dz) = zW 0
(zW DW )Dz = 0
(Dz)W (Dz) = 0=
One di!culty in solving the normal equations is that they are often "ill-
conditioned" problems. This means small variations in the data or numerical
approximations can result in large deviations from the exact solution. For such
problems, other methods that use the QR factorization of the matrix should be
used, see [5]. The MATLAB command A\d is an implementation of these more
robust methods.
which means the normal equations must hold because all rows of the transpose
matrix times the residual vector are zero.
In Subsection 2.4.4 a geometric interpretation was given for the special case
p = 3 and q = 2. The least squares solution is the point on the plane, given by
the linear combination of the two column vectors, that is closest to the point
associated with the vector d= The column vectors must be perpendicular to the
residual vector, and these two resulting equations are just the normal equations.
4.2. NORMAL EQUATIONS 187
When p A 3 or q A 2> one can generalize this approach. The plane will
now be replaced by the range of the matrix, which is the linear combination of
q p × 1 column vectors and can be written as Dy. The concept of distance
between two vectors d and Dy in m-dimensional space will be
and, hence, the distance squared between d and Dy is r(y)W r(y)= The least
squares solution is the point in the range of D that is closest to d. Define
perpendicular or orthogonal vectors a and r in m-dimensional space to mean
aW r = 0= The normal equations now simply require the columns of D to be
orthogonal to the residual vector.
5 65DW Dx W
6 = D d5 6
25333=00 302=50 3=85 d 17=347
104 7 302=50 3=85 =0550 8 7 e 8 = 104 7 =221 8
3=85 =0550 =0011 f =003
whose solution is d = 0=0005, e = 0=0075 and f = 1=7393=
Figure 4.2.1 illustrates the data and a linear and quadratic least squares
approximation, which was done using the MATLAB code pop.m. The left graph
in Figure 4.2.1 clearly indicates the data are more quadratic than linear. Once
the two least squares problems have been solved, the two least squares functions
are evaluated and are uhvolq = 2=2911 and uhvtxdg = =0583, which confirms
our visual observations. The right graph in Figure 4.2.1 extends the time axis
from 100 (year 2000) to 150 (year 2050) so that any predictions can be made.
For example, the predicted population at w = 130 (year 2030) based on the
quadratic least squares function is
| = dw2 + ew + f
= (0=0005)1302 + (0=0075) 130 + 1=7393 9=38=
4.2.6 Exercises
1. Consider the following matrices
5 6
¸ ¸ 1 3 4
3 2 0 1
x= > D= and E = 7 1 0 5 8 =
4 7 1 2
3 1 2
;
A
A p1 1 + p2 1 + f = 100
?
p1 2 + p2 1 + f = 98
A
A p1 1 + p2 2 + f = 95
=
p1 2 + p2 2 + f = 94=
The vector version of this, where x = [p1 p2 f]W , d = [100 98 95 94]W and
D is 4 × 3> is
5 6 5 6
1 1 1 5 6 100
9 2 p1
9 1 1 : 9 98
: 7 p2 8 = 9
:
:=
7 1 2 1 8 7 95 8
f
2 2 1 94
192 CHAPTER 4. AX = D: LEAST SQUARES SOLUTION
5 6 DW Dx = DW d 5 6
5 6 1 1 1 5 6 5 6 100
1 2 1 2 9 : p 1 1 2 1 2 9 :
7 1 1 2 2 89 2 1 1 : 7 p2 8 = 7 1 1 2 2 8 9 98 :
7 1 2 1 8 7 95 8
1 1 1 1 f 1 1 1 1
2 2 1 94
5 65 6 5 6
6 9 6 p1 579
7 9 10 6 8 7 p2 8 = 7 576 8 =
6 6 4 f 387
Generally, there should be a much larger set of data points than four for the
three unknowns.
The normal equations reduce to a 4 × 4 matrix solve, which can easily be done
using MATLAB command A\d.
AA [A d]
2 3 1 1 20000
3 4 2 1 19900
5 4 3 1 19300
8 5 3 1 19100
12 6 4 1 18900
15 8 6 1 18400
19 10 7 1 17800
23 11 7 1 17200
27 13 9 1 16500
AA A’*A
2090 1064 722 114
1064 556 374 64
722 374 254 42
114 64 42 9
AA A’*d
2031100
1154600
754500
167100
AA A\d % (A’*A)\(A’*d) is a less robust solver
194 CHAPTER 4. AX = D: LEAST SQUARES SOLUTION
1.0e+004 *
-0.00934343434343
-0.00636363636364
-0.00474747474747
2.04242424242424
This means p1 93=43> p2 63=63> p3 47=47 and f 20> 424=24=
The predicted price in country A at a sales level ({> |> }) = (40> 16> 12) is
p1 40 + p2 16 + p3 12 + f 15> 098=98=
The accuracy of the model depends on the data and the choice of the linear
approximation to the data. In this case the price data is fairly large and the least
squares function rW r should be compared with dW d such as in rW r@dW d = 6=03
105 =
y p1 {1 + p2 {2 + p3 {3 + p4 {4 + p5 {5 + f=
The equations for the seven comparable homes yield the system
;
A
A p1 10 + p2 2=5 + p3 3 + p4 2000 + p5 9 + f = 260
A
? p1 12 + p2 2=0 + p3 5 + p4 2500 + p5 8 + f = 270
..
A
A .
A
=
p1 10 + p2 2=0 + p3 4 + p4 2400 + p5 9 + f = 280=
The corresponding vector equation has a 7 × 6 matrix, five unknown slopes and
an unknown intercept
5 6
5 6 p1 5 6
10 2=5 3 2000 9 1 9 p2 : 260
9 12 2=0 5 2500 8 1 : 9 : 9 :
9 :9 p3 : 9 270 :
9 .. .. .. .. 9
.. .. : 9 p : : = 9 . :=
7 . . . . . . 8 9 4 : 7 .. 8
7 p5 8
10 2=0 4 2400 9 1 280
f
4.3. MULTILINEAR DATA FITTING 195
As in the price data application, the normal equation is easy to solve using
MATLAB.
AA [A d]
1.0e+003 *
0.0100 0.0025 0.0030 2.0000 0.0090 0.0010 0.2600
0.0120 0.0020 0.0050 2.5000 0.0080 0.0010 0.2700
0.0150 0.0020 0.0040 1.9000 0.0060 0.0010 0.2300
0.0080 0.0030 0.0030 1.6000 0.0090 0.0010 0.2500
0.0110 0.0025 0.0040 2.3000 0.0070 0.0010 0.2600
0.0090 0.0020 0.0030 2.0000 0.0080 0.0010 0.2500
0.0100 0.0020 0.0040 2.4000 0.0090 0.0010 0.2800
AA lssol = A\d
lssol =
-0.5489 % equals p1
6.7198 % equals p2
0.8193 % equals p3
0.0425 % equals p4
7.7542 % equals p5
93.3308 % equals f
AA r = d - A*lssol
AA r’*r/(d’*d)
5.7814e-005
AA lssol(1)*9 + lssol(2)*2.5 + lssol(3)*4 +
lssol(4)*1900 + lssol(5)*9 + lssol(6)
259.0114 % equals the appraised value of the home
Assume the home being appraised has the following attributes: 9 years old, 2=5
bathrooms, 4 bedrooms, 1900 heated square feet and general condition equal
to 9. Once the five slopes and intercept have been computed, the home can be
appraised by evaluating
appraised value = p1 {1 + p2 {2 + p3 {3 + p4 {4 + p5 {5 + f
= p1 9 + p2 2=5 + p3 4 + p4 1900 + p5 9 + f
259=0=
196 CHAPTER 4. AX = D: LEAST SQUARES SOLUTION
Notice the first slope, p1 , is a negative number, which indicates the value of
the home decreases as the age increases. The other four attributes add to the
value of the home and so their slopes should be positive.
AA [A d]
3.0000 9.0000 12.0000 1.5000
5.0000 8.0000 13.0000 2.0000
7.0000 7.0000 16.0000 2.0000
9.0000 5.0000 18.0000 2.0000
11.0000 4.0000 21.0000 1.5000
12.0000 3.0000 23.0000 1.5000
14.0000 2.0000 26.0000 1.5000
15.0000 1.0000 29.0000 1.5000
AA lssol = A\d
lssol =
0.2849 % equals d
0.2361 % equals e
-0.1099 % equals f
AA r = d - A*lssol;
AA r’*r
0.2313
The coe!cients in the other two dierential equations
g|
d{ + be| + b
= b f} and
gw
g}
d{ + ee| + e
= e f}
gw
may be computed in a similar way by replacing gl with gbl and gel > respectively,
g| |l+1 |l1
(wl ) gbl and
gw wl+1 wl1
g} }l+1 }l1
(wl ) gel =
gw wl+1 wl1
198 CHAPTER 4. AX = D: LEAST SQUARES SOLUTION
4.3.5 Exercises
1. Consider the multilinear two variable data in Table 4.3.1.
(a). By-hand calculations confirm the normal equations.
(b). Suppose the data for { = 2 and | = 2 is 105 and not 94. Is it
appropriate to model the data by p1 { + p2 | + f?
(c). Do the calculations with 105 and compare the new residual vector
with the old residual vector.
2. Consider the multilinear two variable data
i xl yl }l (dependent)
1 1 1 100
2 2 1 98
3 1 2 95
4 2 2 94
=
5 3 1 95
6 1 3 89
7 3 2 91
8 2 3 86
9 3 3 84
(a). b
Find 8 × 3 matrix D and the 8 × 1 column vector d.
(b). Use MATLAB to solve the least squares problem.
4.4. PARAMETER IDENTIFICATION 199
I (s1 > s2 > · · · > sq ) = (g1 x(w1 > s1 > s2 > · · · > sq ))2 + · · ·
+(gp x(wp > s1 > s2 > · · · > sq ))2
where x(w> s1 > s2 > · · · > sq ) is a solution of a dierential equation and q is the
number of parameters and p is the number of data points. In the radioac-
tive decay problem there are two parameters s1 = x(0) and s2 = u> and the
dierential equation is
gx
= ux with solution x(w> s1 > s2 ) = s1 hs2 w = x(0)huw =
gw
The Nelder-Mead simplex method searches for the minimum of a real-valued
function I (p) of p variables. In the case of three variables a sketchy outline
of the method is as follows.
Step 1. Pick four points p0 > p1 > p2 and p3 to form a tetrahedron in three
dimensional space.
Step 2. Reorder the points so that I (p0 ) I (p1 ) I (p2 ) I (p3 )=
Step 3. Choose a new fourth point, qhz_p3 , by locating it on the line
containing p3 and (p0 + p1 + p2 )@3=
Step 4. Repeat steps two and three until values of I (p) in step two are
"nearly" the same.
In step two (p0 + p1 + p2 )@3 is the center of the face (triangle) in the
tetrahedron with vertices p0 > p1 and p2 and
choose so that the new set of four points is "closer" to the minimum of I (p).
We shall rely on the implementation given by MATLAB.
The MATLAB implementation of the Nelder-Mead method has the form
fminsearch(’Function’, guess_ vector). ’Function’ is a function file and guess_
vector is an initial guess for the parameters of the function I (p) such that it is
a minimum. For example, use fminsearch() to find the minimum of
I (s1 > s2 > s3 ) = 100 + (s1 2)2 + (s2 3)2 + (s3 4)2 =
The following MATLAB implementation has two function files and a code file.
The function file rad_sol.m has the exponential function and the file ls_rad.m
has the above least squares function. The code file rad_fmin.m has the call to
the MATLAB command fminsearch(), and it also lists the two function files.
MATLAB Code rad_fmin.m
% function r = rad_sol(x1,x2,t)
% r = x1*exp(-x2*t);
%
%function lsrad = ls_rad(xx)
% t = 10:16;
% a = [101 95 90 86.5 82 79 78.2 ];
% lsrad = 0;
% for i = 1:7
% lsrad = lsrad + (a(i)-rad_sol(xx(1),xx(2),t(i)))^2;
% end
%
4.4. PARAMETER IDENTIFICATION 201
format long
[xx lsval] = fminsearch(’ls_rad’,[120 .03])
rad_sol(xx(1),xx(2),60)
for i = 0:100
amount(i+1) = rad_sol(xx(1),xx(2),i);
end
plot(amount)
The two function files must be in your directory as well as the above code file
rad_fmin.m. Execution of this code file gives
AA rad_fmin
xx =
1.0e+002 *
1.55946611533598 0.00044865208356
lsval =
9.73565029005973
rad_sol(xx(1),xx(2),60)
10.56557646847834.
This means the initial amount is 155.95 and the decay rate is 0.04486. The
predicted amount at time equal to 60 is 10.5655. These values are very close
to the output from using the linear fit to ln(x(w)) = ln(x(0)) + (u)w> where
the initial amount is 154.0352, decay rate is 0.0439 and amount at time 60 is
11.5172.
The least square problems for the first three can be solved by the normal equa-
tions. The dierential equations for the population x(w) are based on the birth
202 CHAPTER 4. AX = D: LEAST SQUARES SOLUTION
and death rates, e and g. For small time intervals they may be approximated
by constants and in this case we get the exponential function. In the logistic
model the birth and death rates are allowed to vary linearly with the population
so that e g = f(P x)= One can view P as the maximum population, and f
will determine how fast the maximum population will be attained.
The parameters P and f can be determined from a least squares approxi-
mation of the data derived by using centered finite dierences to approximate
the derivative
gx 1
(wl ) = f(P x(wl ))
gw x(wl )
x(wl+1 ) x(wl1 ) 1
= fP fx(wl )=
wl+1 wl1 x(wl )
1.051582719560504e+002
% predicted population at 2050 using 273.13 at 2000
4.011070115341667e+002
% predicted population at 2050 using 282.12 to restart at 2000.
4.063488610657162e+002
The computed values for the parameters are P = 590=37, F = 0=15092 and
f = 0=00002983= The least squares function evaluated at these parameters is
equal to 105=04. Figure 4.4.1 is a graph of this logistic function and the data
used to find above parameters.
The following table indicates dierent interpretations of the data depending
on the data and the method used.
Method 2000 Data P F f 2050 Pred.
pop_us.m 275.13 586 .1613 .00002982 404
pop_us.m 282.12 676 .1328 .00002489 422
pop_us_fmin.m 275.13 590 .1509 .00002983 406
pop_us_fmin.m 282.12 744 .1175 .00002208 433
United States? additional * * * 419
F1 = T1 @F 1@2 and
T2 T1 cosh(F 1@2 O)
F2 = =
F 1@2 sinh(F 1@2 O)
This means the temperature x({> F> T1 > T2 ) is a function of the three parame-
ters as well as {=
The temperature data was taken in rod with length O = 70 and at points
starting at { = 10 in increments of 4 until { = 66 is recorded in Table 4.4.2,
which was taken at the Math Instructional and Research Laboratory [3]. The
surrounding temperature was xvxu = 21=47. The nonlinear least squares func-
tion has p = 15 data points and q = 3 parameters and is
I (F> T1 > T2 ) = (x1 x({1 > F> T1 > T2 ))2 + · · ·
+(x15 x({15 > F> T1 > T2 ))2 =
206 CHAPTER 4. AX = D: LEAST SQUARES SOLUTION
Figure 4.4.2 was generated by the MATLAB code heat_fmin.m and illustrates
the temperature data and the temperature function with the parameters
4.4.5 Exercises
1. Use the MATLAB command fminsearch() to find the minimum of
I (s1 > s2 > s3 ) = 100 + 2(s1 2)2 + 4(s2 3)2 + 10(s3 4)2 =
2. Use the MATLAB command fminsearch() to find some of the many min-
imum points of
Ax = d: Multiple Solutions
5.1.1 Subspaces in Rq
Let a, b and c be vectors in Rq and let v and w be real numbers. Let a + b and
va denote vector addition and scalar multiplication. Theorem 3.1.1 listed the
following vector space properties for the set of all vectors in Rq :
a + b 5 Rq , a + 0 = a where 0 5 Rq has zeros as components, a + (a) = 0>
a + b = b + a, a + (b + c) = (a + b) + c;
va 5 Rq , 1a = a, v(a + b) = va + vb, (v + w)a = va + wa and v(wa) = (vw)a=
Definition 5.1.1. A subset V of Rq is called a vector subspace of Rq if and
only if for all a, b and c in V
a + b 5 V, a + 0 = a where 0 5 V has zeros as components, a + (a) = 0
209
210 CHAPTER 5. AX = D: MULTIPLE SOLUTIONS
where a 5 S, a + b = b + a, a + (b + c) = (a + b) + c;
va 5 V, 1a = a, v(a + b) = va + vb, (v + w)a = va + wa and v(wa) = (vw)a=
The lines and planes that contain the origin are examples of subspaces in
R3 = The following characterization of a subspace in Rq easily follows from the
vector space properties of Rq =
x = x0 + wd
where x0 represents the given point in R3 > w is a free real number and d is the
given direction vector in R3 = If the given point is the origin, then the line is
called a one-dimensional subspace of R3 and
The set of all points in a line, x = 0 + wd = wd> has the two properties in
Theorem 5.1.1. Any scalar multiple of a point is in the line, which follows from
vx = v(wd) = (vw)d where v is a real number. The sum of any two points in
the line is also in the line, which follows from x = w1 d and y = w2 d giving
x + y = w1 d + w2 d = (w1 + w2 )d. If a line in R3 does not contain the origin, then
it is called an a!ne subset of R3 =
Example 5.1.2. Planes in R3 were described in Section 2.4 by a given point
$
and two vectors that are not parallel as
$
u = $
u 0 + u
$
d + v e where the two
parameters u and v are free real numbers. The column vector notation for a
plane in R3 is
x = x0 + ua + vb
where the two non-parallel vectors are a and b with a 6= bn for any real number
n. If the given point is the origin, then the plane is called a two-dimensional
subspace of R3 and
If the given point is not the origin, then the plane is also called an a!ne subset
of R3 =
Example 5.1.3. Any linear combination of vectors is also a subspace such as
the linear combination of the two vector in R4 = Let
The two vectors a and b will be derived in Example 5.2.3 from the set of vectors
in R4 such that
5 6
5 6 0
2 1 1 3 9 0 :
Dx = 7 2 4 2 11 8 x = 9 7 0 8=
:
0 3 1 8
0
In other words, the range of D is the set of all linear combinations of its column
vectors.
Example 5.1.4. Let 5 6
2 1 1 3
D = 7 2 4 2 11 8 =
0 3 1 8
The range space of D is the linear combination of the four column vectors in
D= By subtracting row one from row two and then row two from row three we
have 5 6
2 1 1 3
HD = H32 (1) H21 (1) D = 7 0 3 1 8 8 =
0 0 0 0
212 CHAPTER 5. AX = D: MULTIPLE SOLUTIONS
Because the third and fourth columns can be written as a linear combination
of the first two columns, the range space of HD may be described by the linear
combination of the first two column vectors in HD.
Definition 5.1.3. A set of vectors y1 > · · · > yn in Rp is called linearly indepen-
dent if and only if
f1 y1 + · · · + fn yn = 0 implies f1 = f2 = · · · = fn = 0=
imply f1 = 0 and f2 = 0=
Example 5.1.4 (continued). The first two column vectors in the new matrix
HD are linearly independent because
5 6 5 6 5 6
2 1 0
f1 7 0 8 + f2 7 3 8 = 7 0 8
0 0 0
Thus, if one can find all homogeneous solutions and at least one particular
solution, then the general form of all solutions is known. The homogeneous
solutions are elements of the nullspace. If the nullspace has a basis given by
y1 ,· · · ,yn , then the general homogeneous solution is a linear combination of
these vectors
xk = f1 y1 + · · · + fn yn =
If d is in the range space of D, then there is a particular solution Dxs = d=
x = f1 y1 + · · · + fn yn + xs =
Two other vectors that satisfy this equation and are not parallel are ba = [1
W b W
1 0] and b = [0 1 1] . They also describe the original plane because
their vector equation is equivalent to the above algebraic equation. That is,
b • (x 0) = 0 where n
n b =a b×b b = [1 1 1]W and so n b = (1@2)n and
{1 {2 + {3 = 0. One can explicitly show ab is a linear combination of a and
b
ua + vb = b
a
5 6 5 6 5 6
1 1 1
u 7 2 8 + v 7 0 8 = 7 1 8 =
3 1 0
The solution of the first two equations easily gives u = 1@2 and v = 3@2= Note,
they also satisfy the third equation.
The following examples illustrate how one can find the general solution for
a single equation
This suggests y1 = a = [2 1 0]W and y2 = b = [3 0 1]W form a basis for
the nullspace of D = [1 2 3]. Clearly, both vectors satisfy the homogeneous
equation and are not parallel and, so, in R3 they are linearly independent. By
Theorem 5.1.2 the general solution is
5
6 5 6 5 6
2 3 10
x = xk + xs = u 7 1 8 + v 7 0 8 + 7 0 8 =
0 1 0
Both column vectors on the right side are homogeneous solutions and are not
parallel and, hence, they form a basis for the nullspace of D = [0 2 1]. By
Theorem 5.1.2 the general solution is
65 5 6 5 6
1 0 0
x = xk + xs = u 7 0 8 + v 7 1@2 8 + 7 5 8 =
0 1 0
By subtracting two times equation one from equation two we have a contradic-
tion 0 = 3 2! In terms of elementary matrices and the augmented matrix this
is
¸ ¸
1 0 1 2 3 1
H21 (2) [D d] =
2 1 2 4 6 3
¸
1 2 3 1
= = (5.1.1)
0 0 0 1
The last row is equivalent to 0{1 + 0{2 + 0{3 = 1! This means d = [1 3]W is
not in the range space of the 2 × 3 matrix D=
In the next example the two planes are not parallel and, so, the solution
set will be a line in R3 = Example 2.5.3 illustrates that the direction vector of
the line given by the intersection of the two planes could be found by taking
the cross product of the two normal vectors. The following example gives an
alternative way, which will be useful for higher dimensional systems.
Example 5.1.8. Find all the solutions of
½
{1 + 2{2 + 3{3 = 4
4{1 + 5{2 + 6{3 = 7=
Note, the direction vector [1 2 1]W is a multiple of the cross product of the
two normal vectors n = [1 2 3]W and n b = [4 5 6]W , which is n × n b = [3 6
W
3] =
In this example let {3 = w be the "free variable" so that H21 (1) [D g] represents
an equivalent system
¸ ¸ ¸ ¸
1 2 {1 4 3
= w =
0 1 {2 1 3
Solve for {2 = 3w and {1 = 3w 2(3w) = 9w so that xk = [9w 3w w]W = Since
H = H21 (1) has an inverse matrix and by Theorem 5.1.2, the general solution
is
5 6 5 6
9 2
x = xk + xs = w 7 3 8 + 7 1 8 =
1 0
218 CHAPTER 5. AX = D: MULTIPLE SOLUTIONS
l1 + l2 l3 = 0=
Systems of this form, called the equilibrium equations, also appear in structure,
fluid flow and optimizations models, and they will be discussed in more detail
in Section 5.3.
The representation of the current solution to the fourth equation leads to a
reduction in size of the larger algebraic problem. In order to find the homoge-
neous solution, let l2 = u and l3 = v be the "free variables" so that l1 = u v=
The vector form of this is
5 6 5 6 5 6 5 6
l1 uv 1 1
ik = 7 l2 8 = 7 u 8 = u 7 1 8 + v 7 0 8 =
l3 v 0 1
The two column vectors on the right side are homogeneous solutions.
The two column vectors are nullspace basis vectors, which can be recorded
in a 3 × 2 matrix 5 6
1 1
Q = 7 1 0 8=
0 1
This means [1 1 1]Q = [0 0] or equivalently
£ ¤ £ ¤W
( 1 1 1 Q )W = 0 0
5 6
1 ¸
0
QW 7 1 8 = =
0
1
5.1. SUBSPACES AND SOLUTIONS IN R3 219
In order to reduce the size of the algebraic problem from four to two un-
knowns, put ik into the top three equations to obtain
5 6 5 6 5 6 5 6 5 6
U1 0 0 1 1 1 H1
7 0 U2 0 8 (u 7 1 8 + v 7 0 8) + 7 1 8 {1 = 7 H2 8
0 0 U3 0 1 1 0
5 65 6 5 6 5 6
U1 0 0 1 1 ¸ 1 H1
7 0 U2 0 8 7 1 0 8 u + 7 1 8 {1 = 7 H2 8 =
v
0 0 U3 0 1 1 0
Multiply both sides by the transpose of the basis vectors Q and use the fact
that they are homogeneous solutions
5 6 5 6 5 6
U1 0 0 ¸ 1 H1
u
Q W (7 0 U2 0 8 Q + 7 1 8 {1 ) = Q W 7 H2 8
v
0 0 U3 1 0
5 6 5 6 5 6
U1 0 0 ¸ 1 H1
u
(Q W 7 0 U2 0 8 Q ) + Q W 7 1 8 {1 = Q W 7 H2 8
v
0 0 U3 1 0
¸ ¸ ¸ ¸
U1 + U2 U1 u 0 H1 H2
+ {1 =
U1 U1 + U3 v 0 H1
¸ ¸ ¸
U1 + U2 U1 u H1 H2
= =
U1 U1 + U3 v H1
5.1.5 Exercises
1. Consider Example 5.1.1.
(a). Use the definition of cross product to compute the two normal vec-
b.
tors n and n
(b). b
Find u and v such that ua + vb = b.
220 CHAPTER 5. AX = D: MULTIPLE SOLUTIONS
5.2.1 Solutions in R4
The solution in R4 of {1 + 2{2 + 3{3 = 10 is dierent from the solution of the
same equation in R3 . In Example 5.1.5 we derived the solution in R3 to be
5 6 5 6 5 6
2 3 10
x = u7 1 8+ v7 0 8+ 7 0 8=
0 1 0
In R4 the equation is {1 + 2{2 + 3{3 + 0{4 = 10 and there is a third free variable
{4 = w. Hence, the homogeneous solution in R4 must satisfy {1 + 2u + 3v = 0>
{2 = u> {3 = v and {4 = w= The vector form of the homogeneous solution is
5 6 5 6 5 6
2 3 0
9 1 : 9 0 : 9 0 :
xk = u 9 : 9 :
7 0 8 + v7 1 8 + w7 0 8=
9 :
0 0 1
0 0 1 0
The next example illustrates that increasing the number of variables can
convert an inconsistent smaller variable problem into a problem with multiple
solutions! The following system with two equations in R3 is inconsistent (has
no solution)
½
{1 + 2{2 + 3{3 = 1
2{1 + 4{2 + 6{3 = 3=
Use Theorem 5.1.2 with H = H21 (2) to get the general solution in R4
5 6 5 6 5 6
2 3 3
9 1 : 9 0 : 9 0 :
x = u9 : 9 : 9
7 0 8 + v7 1 8 + 7 0 8.
:
0 0 1
Example 5.2.1. In the above system in (5.2.1) with two equations and four
variables the augmented matrix is
¸
1 2 3 4 1
[D d] = .
2 4 6 9 3
The row echelon form matrix is
¸
1 2 3 4 1
H21 (2) [D d] = where
0 0 0 1 1
the two pivots are e1q1 = 1 with q1 = 1 and e2q2 = 1 with q2 = 4= The fixed
variables are {1 and {4 > and the free variables are {2 and {3 = The rank of the
2 × 4 matrix D is two.
Example 5.2.2. Consider the following system with three equations and four
unknowns
;
? {1 + 2{2 + {3 + 3{4 = 1
2{1 + 4{2 + {3 + 7{4 = 3
=
{3 + {4 = 7=
The augmented matrix is
5 6
1 2 1 3 1
[D d] = 7 2 4 1 7 3 8 =
0 0 1 1 7
Subtract 2 times row 1 from row 2 and then add row 2 to row 3 to get
5 6
1 2 1 3 1
H32 (1) H21 (2) [D d] = 7 0 0 1 1 1 8 =
0 0 0 2 8
The three pivots are e1q1 = 1 with q1 = 1, e2q2 = 1 with q2 = 3 and e3q3 = 2
with q3 = 4= The fixed variables are {1 > {3 and {4 > and the free variable is
{2 = u= The rank of the 3 × 4 matrix D is three.
The row echelon form matrix is a short way of writing the equivalent system
5 65 6 5 6 5 6
1 2 3 {1 1 2
7 0 1 1 8 7 {3 8 = 7 1 8 u 7 0 8 =
0 0 2 {4 8 0
A particular solution is found by setting the free variable u = 0 and solving the
upper triangular system for the fixed variables. Since the diagonal components
are the pivots, the upper triangular solve can be done to yield a particular
solution xs = [14 0 3 4]W = In order to find the homogeneous solution,
replace the right side d by the zero vector and solve
5 65 6 5 6
1 2 3 {1 2
7 0 1 1 8 7 {3 8 = u 7 0 8 =
0 0 2 {4 0
224 CHAPTER 5. AX = D: MULTIPLE SOLUTIONS
0 4
Example 5.2.3. Consider the following system with three equations and four
unknowns
;
? 2{1 + {2 + {3 + 3{4 = 4
2{1 + 4{2 + 2{3 + 11{4 = 10
=
3{2 + {3 + 8{4 = 6=
Subtract row 1 from row 2 and then subtract row 2 from row 3 to get
5 6
2 1 1 3 4
H32 (1) H21 (1) [D d] = 7 0 3 1 8 6 8=
0 0 0 0 0
The two pivots are e1q1 = 2 with q1 = 1 and e2q2 = 3 with q2 = 2. The fixed
variables are {1 and {2 > and the free variables are {3 = u and {4 = v= The rank
of the 3 × 4 matrix D is two.
The row echelon form matrix is a short way of writing the equivalent system
¸ ¸ ¸ ¸ ¸
2 1 {1 4 1 3
= u v =
0 3 {2 6 1 8
0 1
Use Theorem 5.1.2 with H = H32 (1) H21 (1) to obtain the general solution
in R4 5 6 5 6 5 6
2@6 1@6 1
9 1@3 : 9 8@3 : 9 2 :
x = u9 : 9
7 1 8 + v7 0 8 + 7 0 8=
: 9 :
0 1 0
H [D d] = [HD Hd] = E
is in row echelon form. Since H has an inverse, the solution sets of the algebraic
problems Dx = d and HDx = Hd are equal. However, there may not be a
solution. Reconsider Example 5.2.3 where the third equation has been changed
to 3{2 + {3 + 8{4 = 7= Then H [D d] = [HD Hd] = E becomes
5 6
2 1 1 3 4
H32 (1) H21 (1) [D d] = E = 7 0 3 1 8 6 8 =
0 0 0 0 1
The rank of the 3 × 4 matrix D is two, and the rank of the 3 × 5 augmented
matrix [D d] is three.
of column vectors obtained from the free variables given by the row echelon
matrix. The multiple solutions are all homogeneous solutions plus a particular
solution.
Outline for Solving Dx = d=
1. Find the augmented matrix [D d]=
2. Use elementary matrices to find the row echelon matrix H[D d]=
3. There is no solution if the row echelon form matrix is inconsistent.
4. Use the pivots to find the fixed and free variables.
5. Find a particular solution of HDxs = Hd=
Set the free variables equal to zero and
solve for the fixed variables.
6. Find the homogeneous solutions of HDxk = H0 = 0=
Replace Hd by the zero vector and
solve for the fixed variables in terms of the free variables.
7. The general solution is the linear combination of the homogeneous
solution plus a particular solution.
Example 5.2.4. This example is more complicated since it has four equations
and six unknowns
;
A
A {1 + {2 + {3 + {4 + {5 + {6 = 1
?
{1 + 2{2 + {4 + {6 = 1
A
A {2 + {3 + 2{4 + {5 + {6 = 2
=
2{5 + {6 = 4=
The following elementary row operations and the augmented matrix generate
the row echelon form matrix
5 6
1 1 1 1 1 1 1
9 1 2 0 1 0 1 1 :
[D d] = 9 7 0 1
:
1 2 1 1 2 8
0 0 0 0 2 1 4
5 6
1 1 1 1 1 1 1
9 0 1 1 0 1 0 0 :
H32 (1) H21 (1) [D d] = 9 7 0 0
:=
2 2 2 1 2 8
0 0 0 0 2 1 4
The free variables are {4 = u and {6 = v, and the row echelon matrix gives the
following system
5 65 6 5 6 5 6 56
1 1 1 1 {1 1 1 1
9 0 1 1 1 : 9 {2 : 9 0 : 9 0 : 9 :
9 :9 : = 9 :u9 : v9 0 :=
7 0 0 2 2 8 7 {3 8 7 2 8 7 2 8 7 1 8
0 0 0 2 {5 4 0 1
By setting the two free variables to zero and solving the upper triangular system,
we find a particular solution xs = [1 1 1 0 2 0]W = Set the vector d equal
5.2. ROW ECHELON FORM 227
to the zero vector and solve the following for the homogeneous solutions
5 65 6 5 6 5 6
1 1 1 1 {1 1 1
9 0 1 1 1 : 9 {2 : 9 0 : 9 0 :
9 :9 : = u 9 : v 9 : =
7 0 0 2 2 8 7 {3 8 7 2 8 7 1 8
0 0 0 2 {5 0 1
Use Theorem 5.1.2 with H = H32 (1) H21 (1) to find the general solution in
R6 5 6 5 6 5 6
1 0 1
9 1 : 9 1@2 : 9 1 :
9 : 9 : 9 :
9 1 : 9 0 : 9 1 :
9
x = xk + xs = u 9 : 9 :+9 :
1 : + v9 0 : 9 0 :=
9 : 9 : 9 :
7 0 8 7 1@2 8 7 2 8
0 1 0
The nullspace of D has dimension equal to two, and the dimension of the range
space is four.
1 0
-1 -.5
-1 0
1 0
0 -.5
0 1
AA A*N
0 0
0 0
0 0
0 0
5.2.5 Exercises
1. Consider the solution of 3{1 + 2{2 + {3 = 12 in R4 =
(a). Find the particular solution in R4 =
(b). Find the homogeneous solutions in R4 =
(c). Find the general solution in R4 =
2. Consider the row echelon form for some [D d]
5 6
1 2 3 6 1 1
9 0 0 1 2 1 2 :
9 :
7 0 0 0 0 2 3 8=
0 0 0 0 0 1
(a). Find the pivots, fixed and free variables, and the ranks of D and [D
d].
(b). Does the algebraic system Dx = d have a solution?
3. Consider the solution in R4 of
½
3{1 + 2{2 + {3 + {4 = 6
9{1 + 6{2 + 3{3 + 4{4 = 19=
where the first three unknowns are the currents and the fourth unknown is the
potential at the top node relative to the ground, see Figure 3.1.4. This system
is a special case of the equilibrium equations. In this section the general equi-
librium equations will be defined and solved by both block Gauss elimination
and nullspace methods. Applications to more complicated circuits, structures
and fluid flows models will be outlined. Additional material on equilibrium
equations can be found in chapter two of [8].
Definition 5.3.1. Let D be an q × q matrix and H be an p × q matrix with
p ? q= The equilibrium equations can be written as
Dx + H W y = f and
Hx = g
5.3. NULLSPACES AND EQUILIBRIUM EQUATIONS 231
Hx = g
HD1 (f H W y) = g=
HD1 H W y = HD1 f g=
x = D1 (f H W y)=
The block Gauss elimination method may also be described by a block row
operation, see Section 3.6,
¸ ¸ ¸ ¸ ¸
Lq 0 D HW x Lq 0 f
=
HD1 Lp H 0 y HD1 Lp g
W
¸ ¸ ¸
D H x f
= =
0 HD1 H W y HD1 f + g
x = z ]y
5 6 5 6 5 6
9@2 1@2 27@26
= 7 9@3 8 7 1@3 8 (90@13) = 7 18@26 8 =
0@4 1@4 45@26
Example 5.3.2. If there are more rows in H, then the solves become more
complicated as in
5 65 6 5 6
2 1 0 1 0 {1 1
9 1 2 1 1 1 : 9 {2 : 9 2 :
9 :9 : 9 :
9 0 1 2 0 1 : 9 : 9 :
9 : 9 {3 : = 9 3 : =
7 1 1 0 0 0 8 7 |1 8 7 4 8
0 1 1 0 0 |2 5
The inverse of D is 5 6
3 2 1
D1 = (1@4) 7 2 4 2 8
1 2 3
and, hence, } = D1 f and ] = D1 H W are
5 6 5 6
10 1 3
} = (1@4) 7 16 8 and ] = (1@4) 7 2 6 8 =
14 1 5
5.3. NULLSPACES AND EQUILIBRIUM EQUATIONS 233
whose solution is |1 = 8=8333 and |2 = 1=500= The final step requires the
computation of x = z ]y
5 6 5 6 5 6 5 6
{1 2=5 =25 =75 ¸ 5=8333
7 {2 8 = 7 4=0 8 + 7 =5 1=5 8 8=3333 = 7 1=8333 8 =
1=5000
{3 3=5 =25 1=25 3=1667
D (Q x0 + xs ) + H W y = f =
Q W (D (Q x0 + xs ) + H W y) = Q W f
Q W DQ x0 + Q W Dxs + Q W H W y = Q W f
Q W DQ x0 + Q W Dxs + 0 = Q W f =
Q W DQ x0 = Q W f Q W Dxs =
H(D (Q x0 + xs ) + H W y) = Hf =
234 CHAPTER 5. AX = D: MULTIPLE SOLUTIONS
HH W y = Hf H(D (Q x0 + xs ))=
The solution of
Z Q x0 = z
5 6
¸1 1 0 ¸ ¸
2 3 0 7 {1 0
1 0 8 =
2 0 4 {02 9
0 1
¸ 0 ¸ ¸
5 2 {1 0
= =
2 6 {02 9
x = Q x0 + xs
5 6 5 6 5 6
1 1 ¸ 0 27@26
18@26
= 7 1 0 8 + 7 0 8 = 7 18@26 8 =
45@26
0 1 0 45@26
5.3. NULLSPACES AND EQUILIBRIUM EQUATIONS 235
Set u = 0 and solve for the particular solution xs = [9 5 0]W . Find the
homogeneous solution by solving
¸ ¸ ¸ ¸
1 1 {1 0 0
= u =
0 1 {2 0 1
w = Q W (f Dxs )
5 6W 5 6 5 65 6
1 1 2 1 0 9
= 7 1 8 (7 2 8 7 1 2 1 8 7 5 8)
1 3 0 1 2 0
= 1(12) 1(1) + 1(8) = 19 and
Z = QW D
5 6W 5 6
1 2 1 0 £ ¤
= 7 1 8 7 1 2 1 8 = 1 2 3 =
1 0 1 2
Solve Z Q x0 = z
5 6
£ ¤ 1
1 2 3 7 1 8 x0 = 19 so that x0 = 19@6=
1
Then x =Q x0 + xs is as follows
5 6 5 6 5 6
1 9 35@6
x = 7 1 8 (19@6) + 7 5 8 = 7 11@6 8 =
1 0 19@6
236 CHAPTER 5. AX = D: MULTIPLE SOLUTIONS
Find y by solving HH W y = H W (f D (Q x0 + xs ))
5 6 5 6
¸ 1 0 ¸ ¸ 53@6
1 1 0 7 |1 1 1 0 7
1 1 8 = 44@6 8
0 1 1 |2 0 1 1
0 1 9@6
¸ ¸ ¸
2 1 |1 97@6
=
1 2 |2 35@6
There are three free variables {3 = u> {4 = v and {5 = w> and this implies
¸ ¸ ¸ ¸ ¸ ¸
1 0 {1 0 1 1 1
= u v w =
0 1 {2 0 0 1 1
The matrix for the solve in step five is only 3 × 3> which is smaller than the
7 × 7 matrix in the equilibrium equation,
5 6
U1 + U3 U1 U1
Q W DQ = 7 U1 U1 + U2 + U4 U1 U2 8 =
U1 U1 U2 U1 + U2 + U5
the vertical forces by the even subscripts so that for each bar there is a four-
dimensional force vector
£ ¤W
f h = i1h i2h i3h i4h =
Using the same labels as in Figure 3.4.1, the sum of the forces at each node
gives
Now there are still three two-dimensional vector equations, but there are 18
local force components. This can be written as Hx = g where H is an 6 × 18
matrix and g is a external force vector with six components. In Hx = g the two
equations from the two-dimensional vector equation at node one are in rows one
and two, and the remaining four rows are obtained from the vector equations
at nodes two and three where
5 6
0 0 1 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0
9 0 0 0 1 0 0 0 1 0 0 0 0 0 0 0 0 0 0 :
9 :
9 0 0 0 0 1 0 0 0 1 0 0 0 1 0 0 0 0 0 :
H = 9 9 :>
:
9 0 0 0 0 0 1 0 0 0 1 0 0 0 1 0 0 0 0 :
7 1 0 0 0 0 0 0 0 0 0 1 0 0 0 1 0 1 0 8
0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 1 0 1
g = [0 z 0 0 0 0]W and
x = [i11 i21 i31 i41 i12 i22 i32 i42 i13 i23 i33 i43 i34 i44 i35 i45 i36 i46 ]W =
that are proportional to the velocity components in the fluid. Let cell l have
horizontal and vertical components given by xl and yl where l = 1> 2> 3 and 4.
For each cell the mass in minus the mass out yields four equations for the eight
unknown components. For the four cells depicted in Figure 5.3.2 with incoming
mass for cells 1-3, we have the following four equations
Cell 1: v1 x1 + y2 y1 = 0>
Cell 2: v2 x2 + v2 y2 = 0>
Cell 3: x2 x3 + v3 y3 = 0 and
Cell 4: x1 x4 + y3 y4 = 0=
5.3.7 Exercises
1. Find the solution of the following equilibrium equation using the block
Gauss elimination method
5 65 6 5 6
1 0 0 2 {1 1
9 0 2 0 1 : 9 : 9 :
9 : 9 {2 : 9 6 :
7 0 0 3 1 8 7 {3 8 = 7 12 8 =
2 1 1 0 |1 0
2. Find the solution of the following equilibrium equation using the block
Gauss elimination method
5 65 6 5 6
3 1 0 1 0 {1 1
9 1 3 1 1 2 :9 {2 : 9 2 :
9 :9 : 9 :
9 0 1 3 0 1 :9 {3 :=9 3 :=
9 :9 : 9 :
7 1 1 0 0 0 87 |1 8 7 4 8
0 2 1 0 0 |2 5
First and second order linear initial value problems are studied. Initial value
problems require finding a function of time that satisfies an equation with deriv-
atives and an initial condition. Examples include the motion of projectiles in
Subsection 2.3.3, mixing tanks in Subsection 3.7.4 and population models in
Subsection 4.4.3. Additional applications include time dependent models of
tuned circuits and resonant mass-spring systems. Particular solutions are found
by both undetermined coe!cients and variation of parameters. The system for-
mulations are developed for the second order problems, and numerical solutions
are given by the MATLAB dierential equation solvers. Chapters six and seven
are an introduction to dierential equations, and this topic is covered in more
detail in [1] and [7]
243
244 CHAPTER 6. LINEAR INITIAL VALUE PROBLEMS
Example 6.1.1. Consider the single mixing tank model with U = 6, F = =1,
Y = 1000 and zero initial amount. The initial value problem is
D0 = =6 =006D and D(0) = 0=
We wish to verify D(w) = 100h=006w + 100 is the solution. The left and right
sides of the dierential equation must be equal and the solution evaluated at
time w = 0 must also be zero:
left side = D0 = (100h=006w +100)0 = 100(h=006w )0 = 100h=006w (=006)>
right side = =6 =006(100h=006w + 100) = =006(100h=006w ) and
D(0) = 100h=006(0) + 100 = 100(1) + 100 = 0=
It is always a good idea to check or verify a proposed solution is truly a solution.
(f1 w + f0 )0 = 3 (f1 w + f0 ) + 2w + 1
(0)w + (f1 1 + 0)1 = (3f1 + 2)w + (3f0 + 1)1=
Since this must hold for all time w, the coe!cients of w and 1 must be equal
0 = 3f1 + 2 and
f1 = 3f0 + 1=
{(0) = 10
3(0)
Fh (2@3)0 5@9 = 10
F 5@9 = 10=
{s = f2 w2 + f1 w + f0 =
{s = fh2w =
Since this must hold for all time w> f2 = df + 4= Provided d 6= 2, one can solve
for f to get f = 4@(2 d)= The general solution is
{(0) = 10
Fh d(0)
+ (4@(2 d))h2(0) = 10
F + 4@(2 d) = 10=
{s = f sin(w) + g cos(w)=
f = dg and
g = df + 10=
{(0) = 20
Fh2(0) 4 sin(0) 2 cos(0) = 20
F 0 2 = 20=
where the function y(w) (the variation of parameter) will be chosen so that the
dierential equation holds. Use the product rule to compute the derivative
¡ dw ¢0 ¡ ¢
h y(w) = d hdw y(w) + i (w)
¡ dw ¢0 ¡ ¢
h y(w) + hdw (y(w))0 = d hdw y(w) + i (w)
¡ ¢
hdw dy(w) + hdw y0 (w) = d hdw y(w) + i (w)=
This gives a compact formula for the particular solution. The integral exists if
the i (w) is continuous. In many cases the integral may exist, but it may not be
easy to compute.
Example 6.1.5. Reconsider Example 6.1.2 where d = 3 and i (w) = 2w + 1 and
use integration by parts to compute the integral
Z w
{s (w) = hdw hd i ( )g
Z w
= hdw hd (2 + 1)g
Z w Z w
= hdw hd 2 g + hdw hd g
This agrees with the method of undetermined coe!cients that was used in
Example 6.1.2. Note, the lower limit in the integral can be any constant, because
the dierence in the integral is part of the homogeneous solution.
The results of the above discussion are summarized in the following theorem.
Theorem 6.1.1 (First Order Linear Initial Value Problem) Consider the ini-
tial value problem O({) = {0 d{ = i (w) and {(0) = {0 . If i (w) is continuous
on the interval [0 W ], then for w in this interval the solution of the dierential
equation is
Z w
dw dw
{(w) = Fh + h hd i ( )g
Z w
dw
= h (F + hd i ( )g )=
6.1. FIRST ORDER LINEAR 249
6.1.6 Exercises
1. As in Example 6.1.1 verify { = (95@9)h3w (2@3)w 5@9 is a solution to
Example 6.1.2.
2. As in Example 6.1.1 verify { = 14h3w + (4)h2w is a solution to Example
6.1.3.
3. Consider the initial value problem {0 = 5{ + 3w + 2 and {(0) = 5=
(a). Find the homogeneous solution.
(b). Use undetermined coe!cients to find the particular solution.
(c). Find the solution to the initial value problem.
4. Consider the initial value problem {0 = 3{ + 3w2 + 2w + 1 and {(0) = 10=
(a). Find the homogeneous solution.
(b). Use undetermined coe!cients to find the particular solution
(see the remark at the end of Example 6.1.2).
(c). Find the solution to the initial value problem.
5. Consider the initial value problem {0 = 2{ + 10h3w and {(0) = 5=
(a). Find the homogeneous solution.
(b). Use undetermined coe!cients to find the particular solution.
(c). Find the solution to the initial value problem.
6. Consider the initial value problem {0 = 2{ + 4h2w and {(0) = 12=
(a). Find the homogeneous solution.
250 CHAPTER 6. LINEAR INITIAL VALUE PROBLEMS
g p p gp p p
{01 (w) = sin( n@pw) = cos( n@pw) n@pw = cos( n@pw) n@p
gw gw
g p p p p p
{001 (w) = cos( n@pw) n@p = sin( n@pw) n@p n@p=
gw
This means p{001 = n{1 = The verification for {2 (w) is similar. Because the
derivative operation is linear, the most general solution of p{00 + n{ = 0 is a
252 CHAPTER 6. LINEAR INITIAL VALUE PROBLEMS
Note, {0 (0) means first compute the derivative and then evaluate the derivative
at time equal to zero. This algebraic system in matrix form is
¸ ¸ ¸
0 1 f1 2
=
2 0 f2 1
and is easy to solve. So, the solution of the second order initial value problem
is
{(w) = (1@2) sin(2w) + 2 cos(2w)=
When the coe!cients are constants, we assume the solution has the form of an
exponential function
In order to find u> we put this exponential function into the dierential equation
¡ ¢00 ¡ ¢0
d huw + e huw + fhuw = 0
dhuw u2 + ehuw u + fhuw = 0
(du2 + eu + f)huw = 0=
6.2. SECOND ORDER LINEAR 253
is defined and not zero, see Section 1.2. Thus, the quadratic factor must be
zero, which is often called the auxiliary or characteristic equation,
du2 + eu + f = 0=
By the quadratic formula there are three cases to consider: two distinct real
roots, repeated real roots and complex roots. As we shall see, in all three cases
this leads to two homogeneous solutions {1 (w) and {2 (w) that are not multiples
of each other.
Definition 6.2.1. The dierential operator O({) is linear if and only if
f{) = b
O({ + |) = O({) + O(|) and O(b fO({)=
The second order dierential operator O({) d{00 +e{0 +f{ is linear because
Because of the linear property, any linear combination of the two homogeneous
solutions will also be a homogeneous solution
Example 6.2.1 illustrates this where the characteristic equation is 1u2 +0u+4 = 0
and has conjugate imaginary roots u = ±l2> which leads to the sine and cosine
functions as outlined in the Section 6.3. The next two subsections discuss the
distinct real and repeated real root cases.
Example 6.2.3. The second order dierential equation {00 + 4{0 + 4{ = 0 has
the characteristic equation
u2 + 4u + 4 = 0
whose solution is
p s
u = (4 ± 42 4(1)(4))@2 = (4 ± 0)@2 = 2 and 2=
The solution of the dierential equation is
{(w) = f1 h2w + f2 wh2w =
If the initial conditions are {(0) = 1 and {0 (0) = 3, then
{(0) = f1 h2(0) + f2 (0)h2(0) = f1 + f2 0 = 1 and
{0 (0) = f1 h2(0) (2) + f2 (1h2(0) + (0)h2(0) (2))
= f1 (2) + f2 = 3=
The matrix version of the algebraic system is
¸ ¸ ¸
1 0 f1 1
=
2 1 f2 3
whose solution is f1 = 1 and f2 = 5= The solution of the initial value problem is
{(w) = h2w + 5wh2w =
Thus, one can solve the initial value problem for the both the distinct and
repeated real solutions of the characteristic equation. These results are sum-
marized in the following theorem
Theorem 6.2.1 (Homogeneous Linear Initial Value Problem) Let {1 (w) and
{2 (w) be homogeneous solutions of the dierential equation O({) = d{00 + e{0 +
f{ = 0. Consider the initial value problem given by the dierential equation
and the initial conditions {(0) = {0 and {0 (0) = y0 = L is a linear operator
and, so, f1 {1 (w) + f2 {2 (w) is also a homogeneous solution. If det(Z (0)) =
{1 (0){02 (0) {2 (0){01 (0) 6= 0> then one can find the coe!cients such that the
initial conditions hold, and the solution of the initial value problem is unique.
6.2.7 Exercises
1. Consider Example 6.2.1.
(a). Show {2 (w) is a homogeneous solution.
(b). Verify {(w) = (1@2) sin(2w) + 2 cos(2w) is the solution of the initial
value problem.
2. Verify {(w) = (8@7)hw (1@7)h6w is the solution of the initial value prob-
lem in Example 6.2.2.
6.3. HOMOGENEOUS AND COMPLEX SOLUTION 257
3. Verify {(w) = h2w + 5wh2w is the solution of the initial value problem in
Example 6.2.3.
4. Consider the initial value problem {00 3{0 + 2{ = 0 with {(0) = 1 and
0
{ (0) = 2=
(a). Find the homogeneous solutions.
(b). Find the solution to the initial value problem.
5. Consider the initial value problem 2{00 + 2{0 12{ = 0 with {(0) = 1
and {0 (0) = 1=
(a). Find the homogeneous solutions.
(b). Find the solution to the initial value problem.
6. Consider the initial value problem {00 + 2{0 + { = 0 with {(0) = 2 and
{0 (0) = 1=
(a). Find the homogeneous solutions.
(b). Find the solution to the initial value problem.
7. Consider the initial value problem 3{00 24{0 + 48{ = 0 with {(0) = 2
and {0 (0) = 3=
(a). Find the homogeneous solutions.
(b). Find the solution to the initial value problem.
8. Use the MATLAB command dsolve() to confirm any by-hand calculations
in exercises 4-7.
9. Let {(w) and |(w) be solutions of the initial value problem
(b). Let Z (0) be the Wronskian associated with the dierential equa-
tion. Assume det(Z (0)) 6= 0 and show }(w) must be zero and, hence, the
solution of the initial value problem is unique.
Suppose the mass is initially at rest and is one unit to the left of the equilibrium
position, that is, {(0) = 1 and {0 (0) = 0= Find f1 and f2 so that f1 {1 (w) +
f2 {2 (w) satisfies these initial conditions
p p
f1 cos( n@p0) + f2 sin( n@p0) = f1 1 + f2 0 = 1 and
p p p p p
f1 sin( n@p0)( n@p) + f2 cos( p@n0) n@p = f1 0 + f2 n@p = 0=
Example 6.3.2. Consider the initial value problem {00 + 6{0 + 25{ = 0 with
{(0) = 2 and {0 (0) = 3= First, the characteristic equation is u2 + 6u + 25 = 0
and the solution is
p
6 ± 62 4(1)25
u= = 3 ± 4l=
2(1)
Second, note this is the complex root case where = 3 and = 4 and, thus,
the homogeneous solutions are
Third, find f1 and f2 so that {(w) = f1 {1 (w) + f2 {2 (w) satisfies the given initial
conditions. Here we must compute the derivative of {(w) by using the product
rule
The algebraic solution is f1 = 2 and f2 = 9@4> and the solution to the initial
value problem is
Note, the mass-spring oscillates, but amplitudes of the oscillations rapidly de-
crease to zero. This is a result of the friction forces, which remove energy from
the mass-spring system.
e2 4pn ? 0 (underdamped) :
p p
h(e@2p)w cos( 4pn e2 w) and h(e@2p)w sin( 4pn e2 w)
e2 4pn = 0 (critically damped ) :
h(e@2p)w and wh(e@2p)w
e2 4pn A 0 (overdamped) :
s s
e2 4pn@2p)w e2 4pn@2p)w
h(e@2p and h(e@2p+ =
Because p> e and n are all positive, the coe!cients of time w in the above
exponential function are all negative. This implies the homogeneous solution
will approach zero as time increases, which is not the case if e is zero.
Example 6.3.3. Let p = 1, n = 1 and vary e = 1> 2 and 4= Consider the initial
value problem
As in Example 6.3.2 the initial conditions are used to find the coe!cients
Let e = 2 and in this case the characteristic equation has repeated real roots
u = 1 and 1 and the homogeneous solution is
f1 1 + f2 0 = 2 and
f1 (1) + f2 (1 + 0) = 0
The algebraic solution is f1 = 2 and f2 = 2> and the solution to the initial value
problem is
{(w) = 2hw + 2whw =
s
Let e = 4> which illustrates the distinct real roots case with u = 2 ± 3>
and the general homogeneous solution is
s s
{(w) = f1 h(2+ 3)w
+ f2 h(2 3)w
=
f1 1 + f2 1 = 2 and
s s
f1 (2 + 3) + f2 (2 3) = 0=
6.3.4 Exercises
1. Consider the homogeneous solutions {1 (w) = hw cos(w) and {2 (w) =
hw sin(w)=
(a). Compute their derivatives.
(b). Find the Wronskian matrix Z (0)=
(c). Show det(Z (0)) p= =
2. Verify {(w) = cos( n@pw) is the solution for Example 6.3.1.
3. Consider the initial value problem 3{00 + 27{ = 0 with {(0) = 2 and
0
{ (0) = 1=
(a). Find the homogeneous solutions of the dierential equation.
(b). Find the solution to the initial value problem.
4. Consider the initial value problem {00 + 2{0 + 2{ = 0 with {(0) = 1 and
{0 (0) = 1=
(a). Find the homogeneous solutions of the dierential equation.
(b). Find the solution to the initial value problem.
5. Consider the initial value problem 2{00 + 4{0 + 6{ = 0 with {(0) = 1 and
{0 (0) = 2=
(a). Find the homogeneous solutions of the dierential equation.
6.4. NONHOMOGENEOUS DIFFERENTIAL EQUATIONS 263
{s = d2 w2 + d1 w + d0 =
Match the coe!cients of w2 , w and 1 to get three algebraic equations, which may
be written in matrix form as
5 65 6 5 6
2 2 2 d0 2
7 0 2 4 8 7 d1 8 = 7 3 8 =
0 0 2 d2 2
{s = dh3w >
Match the coe!cient of the exponential function to find d = 10@20 = 1@2= The
particular solution of the dierential equation is {s = (1@2)h3w , and the general
solution is the homogeneous plus a particular solution
At this point observe Examples 6.4.1 and 6.4.3 can be combined to generate
more particular solutions by making use of the linear property of the dierential
operator O({) = d{00 + e{0 + f{= If O({) = i and O(|) = j, then O({ + |) =
O({) + O(|) = i + j so that a particular solution of O(}) = i + j is } = { + |=
In Example 6.4.1 i = 3w + 1 with { = (3@2)w + (7@4) and in Example 6.4.3
j = 10h3w with | = (1@2)h3w = Therefore, a particular solution of
{s = dwhw =
Example 6.4.5. Consider sine and cosine functions in the right side {00 +
3{0 + 2{ = 2 cos(w) + 3 sin(w) = i (w)= Assume the particular solution is a linear
combination of cos(w) and sin(w)
{s = d cos(w) + e sin(w)=
Put this into the dierential equation and match the coe!cients of cos(w) and
sin(w)
0
(d cos(w) + e sin(w))00 + 3 (d cos(w) + e sin(w)) + 2 (d cos(w) + e sin(w)) = i (w)
d cos(w) e sin(w) 3d sin(w) + 3e cos(w) + 2 (d cos(w) + e sin(w)) = i (w)
(d + 3e + 2d) cos(w) + (e 3d + 2e) sin(w) = i (w)=
6.4. NONHOMOGENEOUS DIFFERENTIAL EQUATIONS 267
Cramer’s rule can be used to solve for d = 7@10 and e = 9@10= A particular
solution is {s = (7@10) cos(w) + (9@10) sin(w)> and the general solution is
The solution of the initial value problem (IVP) O({) = i (w) with {(0) = {0 and
{0 (0) = y0 is found by choosing f1 and f2 in the general solution so that the
initial conditions are satisfied. This reduces to two algebraic equations
and in order to solve for the coe!cients, it must have non-zero determinant.
Using Theorem 3.7.3 and the discussion in Section 5.1, this is equivalent to the
column vectors in Z (0) being linearly independent. These results are summa-
rized in the following theorem.
Theorem 6.4.2 (Second Order Linear Initial Value Problem) Consider the
nonhomogeneous initial value problem O({) = d{00 + e{0 + f{ = i (w) with {(0) =
{0 and {0 (0) = y0 = Let {1 (w) and {2 (w) be homogeneous solutions and {s (w) be
a particular solution. If det(Z (0)) 6= 0> then the nonhomogeneous initial value
problem has a solution given by (6.4.1) and the algebraic solution of (6.4.2).
268 CHAPTER 6. LINEAR INITIAL VALUE PROBLEMS
The particular solution is {s (w) = w2 + (1@2)w + (1@2) so that {s (0) = 1@2 and
{0s (0) = 1@2= The algebraic system in (6.4.2) is
¸ ¸ ¸
1 0 f1 2 1@2
=
1 1 f2 1 (1@2)
whose solution is f1 = 3@2 and f2 = 3= Thus, the solution to the initial value
problem is
Example 6.4.4 with initial conditions {(0) = 1 and {0 (0) = 2= Steps 1-3
have been done and yield
The particular solution is {s (w) = 5whw giving {s (0) = 0 and {0s (0) = 5= The
algebraic system in (6.4.2) is
¸ ¸ ¸
1 1 f1 10
= =
1 2 f2 25
sin(w) + cos(zw) w
= lim
z$1 2z
1 w cos(w)
= sin(w) + =
2 2
270 CHAPTER 6. LINEAR INITIAL VALUE PROBLEMS
Since the amplitude of cos(w) is w@(2)> the spring eventually will be deformed
so much that the model is no longer valid. This is known as pure resonance,
and it can be controlled by inserting a suitable amount of resistive or damping
force, as is illustrated in the next subsection.
The solution of the above limiting case can also be derived directly by letting
z = 1 and finding the solution to {00 + { = sin(w)= Since i (w) = sin(w) is also a
particular solution, we speculate the particular solution has the form
By placing this into the dierential equation and collecting like terms we have
This gives the same particular solution as well as the same solution to the initial
value problem.
The MATLAB command ezplot is then used to graph the four curves, see Figure
6.4.1. Note, for large values of time and positive e the homogeneous solutions
have essentially decayed to zero so that the curves in this figure reflect the
particular solutions (1@2)w cos(w) for e = 0, frv(w) for e = 1, (1@2)frv(w) for
e = 2 and (1@4)frv(w) for e = 4
AA % let b = 0
AA solb0 = dsolve(’D2x+0*Dx+x = sin(t)’, ’x(0) = 0’, Dx(0) = 0’,’t’)
AA simple(solb0)
-1/2*cos(t)*t + 1/2*sin(t)
AA % let b = 1
AA solb1 = dsolve(’D2x+1*Dx+x = sin(t)’, ’x(0) = 0’, Dx(0) = 0’,’t’)
AA simple(solb1)
exp(-1/2*t)*cos(1/2*3^(1/2)*t) - cos(t) +
1/3*exp(-1/2*t)*cos(1/2*3^(1/2)*t)*3^(1/2
AA % let b = 2
AA solb2 = dsolve(’D2x+2*Dx+x = sin(t)’, ’x(0) = 0’, Dx(0) = 0’,’t’)
AA simple(solb2)
-1/2*cos(t) + 1/2*exp(-t) + 1/2*exp(-t)*t
AA % let b = 4
AA solb4 = dsolve(’D2x+4*Dx+x = sin(t)’, ’x(0) = 0’, Dx(0) = 0’,’t’)
AA simple(solb4)
6.4. NONHOMOGENEOUS DIFFERENTIAL EQUATIONS 271
6.4.5 Exercises
1. Find the homogeneous and particular solutions for {00 + 5{0 + 4{ = 3w + 4=
2. Find the homogeneous and particular solutions for {00 + 2{0 + 2{ = w2 +
2w + 3=
3. Find the homogeneous and particular solutions for {00 + 5{0 + 4{ = 3h2w =
4. Find the homogeneous and particular solutions for {00 + 5{0 + 4{ = 2h4w =
5. Find the homogeneous and particular solutions for {00 + 4{ = sin(w) +
3 cos(w)=
6. Find the homogeneous and particular solutions for {00 + 4{ = sin(2w)=
7. Solve the initial value problem for the dierential equation in Example
6.4.1 using the initial conditions {(0) = 1 and {0 (0) = 2=
8. Solve the initial value problem for the dierential equation in Example
6.4.3 using the initial conditions {(0) = 0 and {0 (0) = 1=
272 CHAPTER 6. LINEAR INITIAL VALUE PROBLEMS
9. Find the solution to the initial value problem {00 +5{0 +4{ = 6h2w +3w +4
with {(0) = 0 and {0 (0) = 0=
10. Find the solution to the initial value problem {00 + 4{ = 2h2w + w + 4
with {(0) = 0 and {0 (0) = 1=
11. Consider the force mass-spring problem {00 + { = cos(zw) with {(0) = 0
0
and { (0) = 0=
(a). Find the particular solution when z 6= 1=
(b). Solve the initial value problem when z 6= 1=
(c). Solve the initial value problem when z = 1=
12. Confirm any by-hand calculations in exercises 1-11 using MATLAB’s
command dsolve().
13. Use MATLAB’s commands dsolve() and ezplot() to solve and graph the
{00 + e{0 + { = cos(w) with {(0) = 0 and {0 (0) = 0 for e = 0> 1> 2 and 4=
drop across the capacitor is given by T = FYF where T is the charge and
T0 = L is the current. The model is derived from Kirchho voltage law, which
requires the sum of the voltage drops be equal to the imposed voltage
YO + YU + YF = H(w)
OL 0 + UL + T@F = H(w)=
Compute the derivative of both sides with respect to time to obtain a model
for the current
OL 00 + UL 0 + L@F = H 0 (w)=
The objective is to fix the resistance U, and to vary the inductance O or the
capacitance F so that the current L will have a maximum amplitude. In tuning
a radio the amplitudes will vary with the frequency of the various stations. The
desired station will have a large amplitude relative to other stations.
Example 6.5.1. Let H(w) = H0 sin(w) where H0 is a constant (unmodulated
carrier wave) and = iv 2 with iv equals the frequency of the carrier wave.
The model for the current is
where the coe!cients are positive constants. The homogeneous solutions L1 (w)
and L2 (w) are derived as in the previous sections from the characteristic alge-
braic equation. Because the O, U and F are positive, regardless of which of the
three cases (distinct, repeated or complex roots), the homogeneous solutions
will decay to zero and are called the transient solutions. The signal that re-
mains significant is the particular solution Ls (w)= The method of undetermined
274 CHAPTER 6. LINEAR INITIAL VALUE PROBLEMS
By placing this into the dierential equation, computing the derivatives and
collecting like terms we have for D Od 2 + Ue + d@F and E Oe 2
Ud + e@F
D cos(w) + E sin(w) = H0 cos(w) + 0 sin(w)=
Match the coe!cients of cos(w) and sin(w) to obtain two algebraic equations
whose matrix form is
¸ ¸ ¸
O 2 + 1@F U d H0
= =
U O 2 + 1@F e 0
Cramer’s rule easily gives the solution
O 2 + 1@F
d = H0 and
(O 2
+ 1@F)2 + (U)2
U
e = H0 =
(O + 1@F)2 + (U)2
2
Let be such that tan() = (O 2 + 1@F)@ (U) and write the particular
solution as
O 2 + 1@F
Ls (w) = H0 [ cos(w) +
(O 2 + 1@F)2 + (U)2
U
sin(w)]
(O 2 + 1@F)2 + (U)2
H0
= p [sin() cos(w) + cos() sin(w)]
(O + 1@F)2 + (U)2
2
H0
= p sin( + w)=
(O 2 + 1@F)2 + (U)2
The amplitude is the coe!cient of sin( + w) and will be a maximum if the
denominator is a minimum. Since the resistance is fixed, the denominator will
be a minimum when O 2 + 1@F = 0= Thus, the current will have a maximum
amplitude when
O 2 + 1@F = 0 or
1
= s =
OF
If = iv 2 and iv is the
s desired frequency, then O and F should be adjusted
so that = iv 2 = 1@ OF= As will be illustrated by the MATLAB numeri-
cal calculations at the end of this section, these results also approximate the
maximum current for the modulated signal.
6.5. SYSTEM FORM OF LINEAR SECOND ORDER 275
{0 = | and
|0 = {00 = (i (w) e{0 f{)@d
= i (w)@d e|@d f{@d=
This can be written in vector form as a first order system of dierential equa-
tions
¸0 0 ¸ ¸ ¸ ¸
{ { 0 1 { 0
= + = (6.5.2)
| |0 f@d e@d | i (w)@d
Let {1 (w) and {2 (w) be homogeneous solutions (i (w) = 0) of the second order
problem in (6.5.1). In terms of vector equation in (6.5.2) this means
¸0 ¸ ¸
{1 0 1 {1
= and
{01 f@d e@d {01
¸0 ¸ ¸
{2 0 1 {2
= =
{02 f@d e@d {02
Use the vector notation
¸ ¸ ¸ ¸
{1 {2 0 0 1
x1 , x2 , f and D =
{01 {02 i (w)@d f@d e@d
The two vectors x1 and x2 satisfy
x01 = Dx1 and x02 = Dx2 = (6.5.3)
{s = {1 y1 (w) + {2 y2 (w)
= h2w (h2w (w@8 3@16)) + h2w (h2w (w@8 1@16))
= (w + 1)@(4)=
In the first part of this section we were able to find the exact particular solution
for the unmodulated signal
and in this case the maximum amplitude of the current was determined to be
100
p
(O + 1@F)2 + (U)2
2
278 CHAPTER 6. LINEAR INITIAL VALUE PROBLEMS
where = iv 2= If = 710, U = 400> and L and C are chosen so that O 2 +
1@F 0> then the maximum amplitude is approximately
100 100
p = = 0=25=
2
(0) + (400) 2 400
p s
If O = 2 and F = 106 > then the tuned station is = 1@ 2(106 ) = 103 @ 2
707=1= A significant variation from O = 2 and F = 106 causes the amplitude
to decrease as long as is at this tuned signal and the signal is unmodulated.
The numerical calculations will illustrate the tuning formula O 2 + 1@F = 0
giving an approximation for tuning even if the signal is modulated.
The MATLAB command ode45() is a variation of the Runge-Kutta numerical
method and uses variable time steps. It requires three input parameters: the
character representation of the function file containing the right side of the
system, the row vector with the initial and final times and the row vector with
the initial charge and current. The call to ode45() is contained in the MATLAB
code file lrc.m. The MATLAB function file yplrc.m contains the right side of
the vector equation (6.5.2). In the file yplrc.m the charge is depicted by the
four character symbol y(1), and the current is depicted by the four character
symbol y(2). In yplrc.m the first and second components of the right side of
the vector equation (6.5.2) are denoted by the eight character symbols yplrc(1)
and yplrc(2), respectively.
MATLAB Code lrc.m
% This file has a call to ode45().
% It requires a second file called yplrc.m,
% which models a series LRC tuning circuit.
clear;
t0 = 0;
tf = .3; % final scaled time
y0 = [0 0]; % initial charge and current
[t y] = ode45(’yplrc’, [t0 tf], y0);
plot(t, y(:,2))
title( ’Series LRC Circuit’)
xlabel(’time’)
ylabel(’current’)
MATLAB Code yplrc.m
function yplrc = yplrc(t,y)
L = 20; R = 400; C = 10^(-6);
fa = 63/(2*pi); % scaled audio frequency
fs = 710/(2*pi); % scaled station frequency
E0 = 100*sin(2*pi*fa.*t); % modulated signal
% E0 = 100; % unmodulated signal
yplrc(1) = y(2);
yplrc(2) = (E0*sin(2*pi*fs.*t) - R*y(2) - (1/C)*y(1))/L;
6.5. SYSTEM FORM OF LINEAR SECOND ORDER 279
6.5.4 Exercises
1. In Example 6.5.1 derive the algebraic system and solution for the coe!-
cients d and e=
2. Use Example 6.5.1 as a template to solve equation (6.5.2) with H(w) =
H0 cos(w)=
3. Use variation of parameters to find the particular solution of {00 9{ =
2w + 3=
4. Use variation of parameters to find the particular solution of {00 + { =
sin(w)=
5. Use MATLAB codes lrc.m and yplrc.m to experiment with dierent F =
(0=1)106 , (1=0)106 and (10=0)106 and keep O = 2=
6. s H(w) = H0 sin(63w)
Consider equation (6.5.2) with
6
cos(1500w)=
(a). Use the formula = 1@ OF and F = 10 to estimate the O that
will give the maximum amplitude.
(b). Confirm this by executing the MATLAB code file lrc.m with a mod-
ified yplrc.m file.
280 CHAPTER 6. LINEAR INITIAL VALUE PROBLEMS
7. Use a variation of the MATLAB codes lrc.m and yplrc.m to solve and
graph the solutions of {00 + e{0 + { = cos(w) with {(0) = 0 and {0 (0) = 0 for
e = 0> 1> 2 and 4=
Chapter 7
This chapter contains the eigenvalue and eigenvector approach to solving linear
systems of dierential equations. This is a generalization of the simple single
dierential equation {0 = d{ + i to a system x0 = Dx + f where D is a matrix
and x a column vector of unknown functions of time. Applications will include
time dependent mixing tanks and heat conduction.
Furthermore, there can be more than two unknowns and two dierential equa-
tions. The objective of this section is to present some examples that will indicate
how one can approach these problems so that larger systems can be solved in a
systematic way.
281
282 CHAPTER 7. EIGENVALUES AND DIFFERENTIAL EQUATIONS
The two column vectors are examples of eigenvectors associated with the system
matrix in (7.1.1), and u1 and u2 are the corresponding eigenvalues, which will
be studied in more detail in Section 7.2. If initial conditions are required, then
one must choose the coe!cients f1 and f2 so that {(0) = {0 and |(0) = |0 =
Example 7.1.1. Reconsider overdamped mass-spring in Example 6.3.3, where
{00 + 4{0 + { = 0=
whose solution is
¸ ¸ ¸
{ 1 1
= f1 hu1 w + f2 hu2 w
| u1 u2
s s
where u1 = 2 + 3 and u2 = 2 3= Find f1 and f2 so that {(0) = 1 and
|(0) = 2. This requires the solution of the algebraic system
¸ ¸ ¸
{(0) 1 u1 0 1
= f1 h + f2 hu2 0
|(0) u1 u2
¸ ¸ ¸
1 1 1 f1
= =
2 u1 u2 f2
Since the roots are distinct, this has unique solution given by Cramer’s rule
s s
u2 2 4 3 2 u1 4 3
f1 = = s and f2 = = s =
u2 u1 2 3 u2 u1 2 3
7.1. SOLUTION OF X0 = DX BY ELIMINATION 283
In order for the second tank to have fixed volume, the flow rate out and not
into the first tank must be 6 (L/min) with concentration |(w)@100 (L/kg). This
gives the dierential equation for the second tank
Example 7.1.2. Suppose the initial amounts in the tanks are {(0) = 2 and
|(0) = 3= Find the coe!cients so that
¸ ¸ ¸
{(0) 1 1
= f1 h(1@25)0 + f2 h(3@25)0
|(0) 2 2
¸ ¸ ¸
2 1 1 f1
= =
3 2 2 f2
| = ({0 d{)@e=
Put this into the second dierential equation to yield a single second order
dierential equation
As an illustration reconsider Example 7.1.2 and derive the solution using the
elimination method.
| = 50{0 + 4{=
The solution has the exponential form huw where u is the solution of the char-
acteristic equation
50u2 + 8u + 12@50 = 0
whose solution is u = 1@25 and u = 3@25= This means the solution of the
second order dierential equation is
{ = f1 h(1@25)w + f2 h(3@25)w =
7.1. SOLUTION OF X0 = DX BY ELIMINATION 285
The first order system requires the solution of both { and |= Find | by using
| = 50{0 + 4{
| = 50(f1 h(1@25)w + f2 h(3@25)w )0 + 4(f1 h(1@25)w + f2 h(3@25)w )
= 50(f1 h(1@25)w (1@25) + f2 h(3@25)w (3@25)) +
4(f1 h(1@25)w + f2 h(3@25)w )
= f1 h(1@25)w (50@25 + 4) + f2 h(3@25)w (150@25 + 4)
= f1 h(1@25)w 2 + f2 h(3@25)w (2)=
The elimination method still works, but one can clearly see the additional com-
plications. Solve the first dierential equation for x2 in terms of x1
Insert x3 and x2 = x01 + 2x1 into the third equation and collect like terms to
obtain a single third order dierential equation
x000 00 0
1 + 6x1 + 10x1 + 4x1 = 0= (7.1.5)
Assume the solution of (7.1.5) is an exponential huw to get a third order char-
acteristic equation
u3 + 6u2 + 10u + 4 = 0
(u + 2)(u2 + 4u + 2) = 0=
s s
The algebraic solutions are u = 2, 2 + 2 and 2 2= Therefore, the
general solution to the third order dierential equation is
s s
x1 (w) = f1 h2w + f2 h(2+ 2)w
+ f3 h(22)w
=
7.1. SOLUTION OF X0 = DX BY ELIMINATION 287
Use equations (7.1.3) and (7.1.4) to find x2 (w) and x3 (w), and write the general
solution to the homogeneous system in vector form
5 6 5 6 5 6
x1 1 s1 s
7 x2 8 = f1 7 0 8 h2w + f2 7 2 8 h(2+ 2)w +
x3 1 1
5 6
1
s s
f3 7 2 8 h(2 2)w =
1
Find the three coe!cients so that the initial temperatures are x1 (0) = 20,
x2 (0) = 30 and x3 (0) = 20= Insert w = 0 into the above general solution to
obtain the algebraic system
5 6 5 65 6
20 1 s1 1
s f1
7 30 8 = 7 0 2 2 8 7 f2 8 =
20 1 1 1 f3
7.1.6 Exercises
1. Consider the overdamped mass-spring model {00 + 8{0 + { = 0.
(a). Define | = {0 and write this as a first order system for { and |=
(b). Find the vector form of the system solution.
(c). Find the coe!cients so that {(0) = 2 and |(0) = 0=
2. Consider dierential equation {00 3{0 + 2{ = 0.
(a). Define | = {0 and write this as a first order system for { and |=
(b). Find the vector form of the system solution.
(c). Find the coe!cients so that {(0) = 1 and |(0) = 2=
3. Consider the two-tank mixing model where both tanks have fixed volume
equal to 50 (L).
(a). Derive the model to show it has the form
¸0 ¸ ¸
{ 4@25 1@25 {
= =
| 4@25 4@25 |
(b). Use elimination to find the vector form of the general solution.
(c). Find the solution of the initial value problem {(0) = 2 and |(0) = 3=
4. Use elimination to solve the initial value problem
¸0 ¸ ¸ ¸ ¸
{ 1 1 { {(0) 1
= with = =
| 2 4 | |(0) 4
7.2. REAL EIGENVALUES AND EIGENVECTORS 289
Because u is not a zero vector, D uL must not have an inverse and, thus, the
determinant of D uL must be zero. This gives a single polynomial equation
that u must satisfy. Once u is found, solve algebraic system (D uL)u = 0 for
the unknown constant vector u that is associated with this u=
Definition 7.2.1. Let D be an q × q matrix. An eigenvalue is a number (real
or complex) u such that there is a nonzero vector u and Du = uu= The vector
u is called the eigenvector of D corresponding to the eigenvalue u=
Note, any constant times u is also an eigenvector, because D(fu) = f(Du) =
f(ux) = u(fu)= Examples 7.1.1-7.1.4 in the previous section give the following
examples of eigenvectors and eigenvectors .
¸ ¸
0 1 1
Example 7.2.1. Let D = and consider u = and u = u1 =
1 4 u1
s
2 + 3=
¸ ¸
0 1 1s
Du =
1 4 2 + 3
s ¸
0(1) + 1(2 + s3)
=
1(1) 4(2 + 3)
¸
s 1s
= (2 + 3) =
2 + 3
¸ ¸
4@50 1@50 1
Example 7.2.2. Let D = and consider u = and
4@50 4@50 2
u = u1 = 1@25=
¸ ¸
4@50 1@50 1
Du =
4@50 4@50 2
¸
(4@50)1 + (1@50)2
=
(4@50)1 + (4@50)2
¸
1
= (1@25) =
2
¸ ¸
2 3 1
Example 7.2.3. Let D = and consider u = and u = u2 =
1 2 1
1=
¸ ¸
2 3 1
Du =
1 2 1
¸
2(1) 3(1)
=
1(1) 2(1)
¸
1
= (1) =
1
7.2. REAL EIGENVALUES AND EIGENVECTORS 291
5 6 5 6
2 1 0 1
s
Example 7.2.4. Let D = 7 1 2 1 8 and consider u = 7 2 8 and
0 1 2 1
s
u = u3 = 2 2=
5 65 6
2 1 0 1
s
Du = 7 1 2 1 8 7 2 8
0 1 2 1
5 s 6
2(1) +s1( 2)
= 7 1(1) 2(s 2) + 1(1)
8
1( 2) 2(1)
5 6
s 1
s
= (2 2) 7 2 8 =
1
det(D uL) = 0=
In general this is a polynomial equation of degree q and, so, there are q possible
roots or eigenvalues. For each eigenvalue there is an eigenvector u that satisfies
the algebraic system
(D uL)u = 0=
A simple example is the system associated with the second order dierential
equation d{00 + e{0 + f{ = 0 where
¸
0 1
D= =
f@d e@d
In this case the determinant is easy to evaluate and gives the characteristic
algebraic equation
¸ ¸
0 1 1 0
det(D uL) = det( u )
f@d e@d 0 1
¸
0u 1
= det( )
f@d e@d u
= u(e@d u) 1(f@d)
= u2 + (e@d)u + f@d = 0=
The solutions may be complex numbers, but we will focus on systems where
the roots of det(D uL) = 0 are real and distinct.
292 CHAPTER 7. EIGENVALUES AND DIFFERENTIAL EQUATIONS
The two eigenvalues are u1 = 1 and u2 = 1= Find the eigenvector associated
with u2 = 1= Solve the algebraic system
(D u1 L)u = 0
¸ ¸ ¸
2 (1) 3 x1 0
= =
1 2 (1) x2 0
The two algebraic equations are
3x1 3x2 = 0 and x1 x2 = 0=
This gives one degree of freedom to choose, say, x1 = 1 and then x2 = 1= In
order to find the other homogeneous solution, let u1 = 1 and repeat the above
calculations.
Example 7.2.4 (continued). Find the eigenvalues using determinants by
solving
5 6
2 u 1 0
det(D uL) = det(7 1 2 u 1 8)
0 1 2 u
= (2 u)((2 u)(2 u) 1) 1(1(2 u) 0)
= (2 u)[(2 u)(2 u) 2] = 0=
s s
The three eigenvalues are u1 = 2, u2 = 2 + 2 and u3 = 2 2= Find the
eigenvector associated with u1 = 2= Solve the algebraic system
(D u1 L)u = 0
5 65 6 5 6
2 (2) 1 0 x1 0
7 1 2 (2) 1 8 7 x2 8 = 7 0 8 =
0 1 2 (2) x3 0
The three algebraic equations are
x2 = 0, x1 + x3 = 0 and x2 = 0=
This gives one degree of freedom to choose, say, x1 = 1 and so x3 = 1= The
other two eigenvectors can be found by similar calculations.
The solution of det(D uL) = 0 done by-hand calculations is a little messy and,
so, the MATLAB commands det() and solve() are used.
AA syms r
AA poly = det(A - r*eye(3)) % compute the determinant
-5/144 - 5/12*r - 5/4*r^2 - r^3
AA r = solve(poly) % solve for the roots of this polynomial
AA r1 = eval(r(1)) % converts the symbolic expression to numbers
-.1264
AA r2 = eval(r(2))
-.7643
AA r3 = eval(r(3)
-.3593
From here one can solve the algebraic system (D uL)u = 0 to find the eigen-
vectors, but an easier and more robust approach is to use another MATLAB
command eig(), which computes all the eigenvalues and their associated eigen-
vectors.
Since the eigenvalues are all negative, the time dependent solution converges to
the steady state solution as time increases.
7.2.5 Exercises
1. Consider Example 7.2.1. s s
(a). Verify u = u2 = 2 3 and u = [1 (2 3)]W are the other
eigenvalue and eigenvector.
(b). Solve the algebraic system (D u2 L)u = 0 to find the above eigen-
vector.
2. Consider Example 7.2.2.
(a). Verify u = u2 = 3@25 and u = [1 2]W are the other eigenvalue
and eigenvector.
(b). Solve the algebraic system (D u2 L)u = 0 to find the above eigen-
vector.
3. Consider Example 7.2.3.
(a). Verify u = u1 = 1 and u = [1 1@3]W are the other eigenvalue and
eigenvector.
(b). Solve the algebraic system (D u1 L)u = 0 to find the above eigen-
vector.
296 CHAPTER 7. EIGENVALUES AND DIFFERENTIAL EQUATIONS
x0 = Dx + f (w)= (7.3.1)
In this case, one can show, see Subsection 7.3.3, there are q linearly independent
homogeneous solutions formed by the eigenvectors um and eigenvalues um
Any homogeneous solution xk (w) satisfies x0k = Dxk and is a linear combination
of these
xk (w) = f1 u1 hu1 w + f2 u2 hu2 w + · · · + fq uq huq w =
A particular solution is any xs (w) that satisfies x0s = Dxs + f (w), and they
can be found for simple cases by the method of undetermined coe!cients, see
7.3. SOLUTION OF X0 = DX + F(W ) 297
2x1 + x2 = 0 and
10x1 + (5)x2 = 0=
xs = D1 f
¸1 ¸
4 1 6
=
10 1 12
¸ ¸ ¸
1@(6) 1@(6) 6 1
= = =
10@(6) 4@(6) 12 2
Since the first eigenvalue u1 = 6 is positive, the general solution may not con-
verge to the steady state solution!
x0 = Dx + gw + h
¸0 ¸ ¸ ¸ ¸
{1 4 1 {1 2 6
= + w+ =
{2 10 1 {2 6 12
Assume the particular solution has the form
xs (w) = aw + b
0 = Da + g and a = Db + h=
a = D1 g
¸ ¸ ¸
1@(6) 1@(6) 2 2@3
= = =
10@(6) 4@(6) 6 2@3
Use a in the second equation to solve for b
b = D1 (a h)
¸ ¸ ¸ ¸
1@(6) 1@(6) 2@3 6 7@9
= ( )= =
10@(6) 4@(6) 2@3 12 32@9
Thus, a particular solution is
xs (w) = aw + b
¸ ¸
2@3 7@9
= w+ =
2@3 32@9
Use the homogeneous solution from Example 7.3.1 to find the general solution
x0 = Dx + gh2w
¸0 ¸ ¸ ¸
{1 4 1 {1 4
= + h2w =
{2 10 1 {2 2
300 CHAPTER 7. EIGENVALUES AND DIFFERENTIAL EQUATIONS
2a = Da + g
2La = Da + g
(D 2L)a = g=
a = (D 2L)1 g
¸1 ¸
42 1 4
=
10 12 2
¸ ¸ ¸
1@(12) 1@(12) 4 1@2
= = =
10@(12) 2@(12) 2 3
x0 = Dx + g cos(3w) + h sin(3w)
¸0 ¸ ¸ ¸ ¸
{1 4 1 {1 1 3
= + cos(3w) + sin(3w)=
{2 10 1 {2 2 4
xs = a cos(3w) + b sin(3w)>
7.3. SOLUTION OF X0 = DX + F(W ) 301
and choose the vector coe!cients so that the dierential equation holds
Match the vector coe!cients of the sine and cosine function to obtain
a(3) = Db + h and b3 = Da + g=
The first equation can be solved for a = (Db + h)@(3) and put into the second
equation
b3 = D((Db + h)@(3)) + g
b3L = D2 b@(3) + Dh@(3) + g
(3L D2 @(3))b = Dh@(3) + g=
Solve this for b and then find a = (Db + h)@(3)= Another way to solve this
system is to write it as a block 2 × 2 algebraic system
¸ ¸ ¸
D 3L a g
= =
3L D b h
The method of undetermined coe!cients does not always work or is too com-
plicated to implement. An alternative is the variation of parameters method,
which can also be generalized to systems of dierential equations. This method
gives a closed formula for the particular solution, but it may involve some
di!cult integrations. A third way to solve systems is to use numerical approx-
imations such as implemented in the MATLAB command ode45().
302 CHAPTER 7. EIGENVALUES AND DIFFERENTIAL EQUATIONS
Then the homogeneous solution can be written as the matrix [(w) times the
£ ¤W
coe!cient vector c = f1 f2 · · · fq
xs (w) = [(w)v(w)=
£ ¤W
Here the column vector v(w) = y1 (w) y2 (w) · · · yq (w) is to be found so
that xs (w) = [(w)v(w) satisfies x0s = Dxs + f (w)=
In order to find ym (w), expand the matrix-vector product by columns
xs (w) = [(w)v(w)
= y1 (w)x1 (w) + y2 (w)x2 (w) + · · · + yq (w)xq (w)=
The derivative with respect to time may be computed by columns using the
product rule
Because each xm (w) um hum w is a homogeneous solution, x0m = Dxm , the above
can be rewritten using the column version of matrix-vector products
The general solution is the sum of the homogeneous solution in (7.3.3) and the
particular solution in (7.3.4)
which is a valid formula provided [(w) has an inverse and the components of
[ 1 f can be integrated.
A matrix [(w) in (7.3.2) formed by the columns of q linearly independent
homogeneous solutions is called a fundamental matrix of the system (7.3.1).
The fundamental matrix and its properties for dierent classes of eigenvalues
are more completely discussed in Chapter 9 in [7]. Here we will assume the
eigenvalues of D are distinct and real and then, with some theoretical eort,
one can show the eigenvectors are linearly independent. By using Theorem
3.7.3 and the properties of determinants
which agrees with the particular solution in Example 7.3.2 that was found by
undetermined coe!cients.
The by-hand calculations in this and similar examples using the variation of
parameters formula (7.3.4) can be complicated, and one should consider using
a symbolic computing tool such as Maple or the symbolic toolbox in MATLAB.
The following MATLAB code will execute the formula in (7.3.4) for Example
7.3.5.
MATLAB code varpar.m
% This is a symbolic calculation to find
% a particular solution using variation
% of parameters for a 2D system of DEs.
syms t
f = [2*t + 6;6*t + 12];
x1 = [exp(6*t); 2*exp(6*t)];
x2 = [exp(-t); -5*exp(-t)];
X = [x1 x2];
invX = inv(X)
vprime = invX*f
v = int(vprime,’t’)
xp = X*v
newxp = simple(xp)
Everyone should do at least one variation of parameter by-hand calculation,
and then confirm the calculation using a symbolic computation tool.
x0 = Dx + f (w)and x(0) = x0 =
[(0)c + xs (0) = x0
¸ ¸ ¸ ¸
1 1 f1 0 7@9 2
+ = =
2 5 f2 0 32@9 1
¸ ¸1 ¸ ¸
f1 1 1 2 0 7@9
= ( )
f2 2 5 1 0 32@9
¸ ¸ ¸
5@7 1@7 25@9 166@63
= = =
2@7 1@7 41@9 9@63
The left and right temperatures are x0 and x5 , and the initial temperatures for
the interior cells will be given.
Let the thermal properties be = 1, f = 1 and N = 0=001 with O = 10 so
that { = 2= Assume the left, right and initial temperature are always zero.
Model the increasing heat source in the wire by i = 0=1 0=1h0=1w = Then the
7.3. SOLUTION OF X0 = DX + F(W ) 307
0 = Da + g and b(0=1) = Db + h=
The first equation is easily solved by a = D1 g= The second equation can
be written as (=1)Lb = Db + h or as (D + (0=1)L)b = h= If 0=1 is not an
eigenvector, then (D(0=1)L) has an inverse and the second equation is solved
by b = (D + (0=1)L)1 h=
The initial conditions are satisfied by solving [(0)c = u0 (a + bh0=1(0) )
= 0 (a + b)= Since [(0) = X> the solution is c = X 1 (a + b)= Once these
computations have been done, the solution to the initial value problem is
u = [(w)c + a + bh0=1w
= f1 X (:> 1)hU(1>1)w + f2 X (:> 2)hU(2>2)w + f3 X (:> 3)hU(3>3)w + f4 X (:> 4)hU(4>4)w
+a + bh0=1w =
308 CHAPTER 7. EIGENVALUES AND DIFFERENTIAL EQUATIONS
The calculations have been implemented in the MATLAB code file heat4.m,
and the output is given in the Figure 7.3.1. Since the eigenvalues (=0009045,
=0006545, =0003455 and =0000955) are all negative, the homogeneous solu-
tion decays to zero. The particular solution decays to a and, so, the solution to
the initial problem converges to a = [800 1200 1200 800]W = This is suggested
by the right side of the graph in Figure 7.3.1.
7.3.6 Exercises
1. Find the eigenvalues
of¸ the following matrices:
2 1
(a). D= has repeated eigenvalues,
1@4 3¸
2 1
(b). D= has complex eigenvalues and
1 2 ¸
1 2
(c). D= has a zero eigenvalue and no inverse.
1@2 1
2. Use undetermined coe!cients
¸to find the particular solution of
2 3
x0 = Dx + f where D = with
¸ ¸3 2
1 2
(a). f= + w and
2 ¸ 3
1
(b). f= h2w =
1
7.3. SOLUTION OF X0 = DX + F(W ) 309
Chapters eight and nine use matrices to represent images. Basic matrix opera-
tions can be used to alter the image contrast, light-dark and blurred-sharpened
qualities of the image. In this chapter the matrix operations will be on the
original matrix, which is often called the space domain. An interesting variety
of images will be enhanced. A very nice general reference to image processing is
[2], where the indicated www site links to many interesting images. The pollen,
aerial, moon and micro chip images were used in [2], and the Mars Rover image
was taken from NASA [11].
311
312 CHAPTER 8. IMAGE PROCESSING IN SPACE DOMAIN
Images in Figures 8.1.3 and 8.1.4 are an aerail photo, which initially is too
bright in some areas. The aerail image in Figure 8.1.3 is provided courtesy of
NASA. The enhanced version was obtained by a power array transformation
and gives a clearer image of the water channel.
The Mars Rover image in Figure 8.1.6 has been enhanced to see the darker
regions in Figure 8.1.5. The Mars image in Figure 8.1.5 is provided courtesy of
NASA.
The second image of the moon in Figure 8.1.8 has sharper craters than
the original moon image in Figure 8.1.7. The moon image in Figure 8.1.7 is
provided courtesy of NASA.
Example 8.1.2. One can augment the matrices of letters to form words or
other objects. In this example four letters are created by functions similar
to the letter function letterc(g) to form the matrix qfvx and then the image
NCSU. The mesh plot of the matrix is given in Figure 8.1.12, the image NCSU
with black background is in Figure 8.1.13 and the "negative" of the image
NCSU with white background is in Figure 8.1.14. The "negative" is created
by replacing the components in the matrix ncsu by 255 qfvx(l> m), that is,
qhjqfvx(l> m) = 255 qfvx(l> m)= The following MATLAB code imagncsu.m was
used to generate these figures.
imwrite(newncsu1, ’ncsu.jpg’);
negncsu1 = uint8(negncsu);
imwrite(negncsu1, ’negncsu.jpg’);
8.1.5 Exercises
1. Consider the letter function letterc(g).
(a). Experiment with dierent input values in the letter function let-
terc(g).
(b). Modify the shape of the letter "C" generated by letterc(g).
(c). Modify the background shade of the letter.
2. Consider the image of NCSU.
(a). Define additional letter functions for the letters in your first name.
(b). Modify the MATLAB code imagncsu.m to generate your first name
image.
(c). Modify the background shade for your first name image.
3. Experiment with dierent colors in the image NCSU as generated in
color_imagncsu.m.
8.2. CONTRAST AND HISTOGRAMS 321
If the image is dark (light), the graph of the histogram will be shifted to
the left (right). If the image lacks contrast, the graph of the histogram will be
clustered. In order to find the histogram of an image matrix, one must total the
number of components that have a particular shade. This can be done by using
three nested loops where the two outside loops range over the pixels (indices of
the image matrix) and the inner most loop ranges over all possible shades. The
first example illustrates this for the pollen image in Figure 8.1.1.
Example 8.2.1. Find the histogram for the pollen image in pollen.jpg. First,
we find the 8-bit image matrix sroohqpdw by using the MATLAB command
imread(), and second, the three nested loops are used to construct the his-
togram function. The following MATLAB code pollenhist.m does this and then
generates the graph of the histogram in Figure 8.2.1.
MATLAB code pollenhist.m
% This code creates a histogram for pollen.jpg and
% plots its graph.
322 CHAPTER 8. IMAGE PROCESSING IN SPACE DOMAIN
clear;
% Creates an image matrix for pollen.jpg.
pollenmat = imread(’pollen.jpg’);
u = double(pollenmat);
[nx ny] = size(u)
nshades = 256;
hist = zeros(nshades,1);
for i = 1:nx % creates the histogram, hist
for j = 1:ny
for k = 0:nshades-1
if u(i,j) == k
hist(k+1) = hist(k+1)+1;
end
end
end
end
plot(hist);
The histogram in Figure 8.2.1 indicates low contrast and a darker image
because the pixels are clumped together and have lower shades. In subsequent
subsections we will transform the image matrix so that the histogram is more
evenly distributed.
8.2. CONTRAST AND HISTOGRAMS 323
The truncation below 0 or above 255 can seriously distort the associated
image. The MATLAB command uint8() does this truncation as illustrated in
the following trivial calculations:
AA u = [-100.3 45.0 257; 23.7 150.0 301.1]
-100.3000 045.0000 257.0000
023.7000 150.0000 301.1000
AA shift_u = u + 100
-00.3000 145.0000 357.0000
123.7000 250.0000 401.1000
AA new_u = uint8(shift_u)
000 145 255
124 250 255
In terms of the MATLAB command uint8(), the shift transformation can be
written as
nshades = 256;
shift = 100; % Transform by shifting to light.
% shift = -50; % Transform by shifting to dark.
ulight = u + ones(nx,ny)*shift;
hist = zeros(nshades,1);
for i=1:nx
for j=1:ny
for k=0:nshades-1
if ulight(i,j)==k
hist(k+1)=hist(k+1)+1;
end
end
end
end
plot(hist);
pollen_light = uint8(ulight);
imwrite(pollen_light, ’pollenlight.jpg’);
will be spread over a larger range of shades. In order to have the transformed
image matrix have components between 0 and 255, we apply a piecewise linear
transformation to each component of the image matrix. An illustration of this
is in Figure 8.2.4, where one can choose dierent values for the points (u1 > v1 )
and (u2 > v2 )= The MATLAB function file for a piecewise linear function is in
spread.m.
MATLAB function spread.m
function uspread = spread(r1,s1,r2,s2,u)
if (u? r1)
uspread = ((s1-0)/(r1-0))*u;
end
if ((uA=r1) & (u?= r2))
uspread = ((s2 - s1)/(r2 - r1))*(u - r1)+ s1;
end
if (uAr2)
uspread = ((255 - s2)/(255 - r2))*(u - r2) + s2;
end
Since the values of the piecewise linear function may not be 8-bit integers, they
must be rounded or truncated to be integers between 0 and 255.
Definition 8.2.3. The piecewise linear transformation of an image matrix x is
a 8-bit image matrix Wsz x whose (l> m) components are given by the composition
326 CHAPTER 8. IMAGE PROCESSING IN SPACE DOMAIN
of the piecewise linear function W (v) and the 8-bit valued MATLAB function
uint8()
(Wsz x)(l> m) uint8(W (x(l> m))=
8.2.5 Exercises
1. Consider the MATLAB code file pollenhist.m. Alter it to find the his-
togram of the Mars Rover image in mars3.jpg.
2. Consider the MATLAB code file pollenhist.m. Alter it to find the his-
togram of the aerial image in aerail.jpg.
3. Apply the shift transformation with vkli w = 40 and
¸
275 240=7 210
x= =
195=2 57 37=1
330 CHAPTER 8. IMAGE PROCESSING IN SPACE DOMAIN
10. Consider the MATLAB code file aerialpower.m and experiment with
additional values of s = 0=3> 0=5> 0=7 and 0=9= Note the histograms and the
corresponding images.
The graphical output is given in Figure 8.3.1. The left graph is the unblurred
image and the center graph is the blurred image after n = 10 "time" steps
with w = 0=00001 and { = 0=01= These values have been chosen so that
the blurring and deblurring process gives reasonable results. The graph in the
right side of Figure 8.3.1 is the deblurred image after n = 10 steps. Note
the additional "peaks" in the deblurred image. A more accurate deblurred
image is found at n = 8 steps, and the reader should run the code to see this.
So, the deblurring process can be too aggressive and insert nonexistent image
components.
xn+1
lm = xnlm + { (xnl1>m 2xnlm + xnl+1>m )
+| (xnl>m1 2xnlm + xnl>m+1 )
N w N w
with { and | =
f {2 f | 2
The reverse of heat diusion is to use negative time steps and to start with the
temperature distribution in the future. As in the 1D case deblurring an image
may be modeled by starting with a blurred image as a future "temperature"
and apply the explicit finite dierence model with negative time steps. In the
blurring of the image, one must choose the time step small enough so that 1
2{ 2| is positive. Like the 1D example backward iterations eventually start
to oscillate and give badly distorted images. The second example illustrates the
blurring and deblurring of the image NCSU.
Example 8.3.2. Consider the image NCSU that was created in Subsection
8.1.2. The following MATLAB code, blur2d_ncsu.m, illustrates deblurring in
2D using negative time steps. The reader will find it informative to experiment
with the time step sizes and the length of the outer time loops. The mesh()
plots are also interesting to observe, but are not printed out in this text. Figures
8.3.2, 8.3.3 and 8.3.4 are original, blurred and deblurred image files, respectively.
The deblurring is still a little fuzzy relative to the original image.
8.3. BLURRING AND SHARPENING 335
imwrite(blurncsu1, ’mac833.jpg’);
subplot(1,3,3)
%
% Backward in "time" or "deblurring"
%
for k = 1:16
for j = 2:ny-1
for i = 2:nx-1
newu(i,j) = u(i,j) - ...
(dt/(dx*dx))*(u(i-1,j) -2*u(i,j) + u(i+1,j))-...
(dt/(dy*dy))*(u(i,j-1) -2*u(i,j) + u(i,j+1));
end
end
mesh(newu);
axis([0 ny 0 nx 0 15]);
title(’Deblurred NCSU’)
k
pause;
u = newu;
end
deblurncsu = 20*u;
deblurncsu = deblurncsu(nx:-1:1,:);
deblurncsu1 = uint8(deblurncsu);
imwrite(deblurncsu1, ’mac834.jpg’);
steps and the 2D heat diusion method as implemented in the Example 8.3.2.
The MATLAB code pollensharpen.m does both the contrast enhancement in
Figure 8.3.5 and subsequent sharpening in Figure 8.3.6. The reader will find it
interesting to compare the last image with the original image in Figure 8.1.1
and to experiment with dierent values of lqwhqvlw| = 1=05 and frhi i = 0=05
and the length of the k-loop.
MATLAB code pollensharpen.m
clear;
% This code is a composition of the piecewise linear
% transformation and the negative time step diusion
% operation. The pollen image will have an increased contrast,
% be brighter and be sharper.
pollenmat = imread(’pollen.jpg’);
u = double(pollenmat);
[nx ny] = size(u)
nshades = 256;
%
% Transformation by piecewise linear function.
%
r1 = 80; s1 = 10;
r2 = 140; s2 = 245;
for i = 1:nx
for j = 1:ny
if (u(i,j)? r1)
uspread(i,j) = ((s1-0)/(r1-0))*u(i,j)
end
if ((u(i,j)A=r1) & (u(i,j)?= r2))
uspread(i,j) = ((s2 - s1)/(r2 - r1))*(u(i,j) - r1)+ s1;
end
if (u(i,j)Ar2)
uspread(i,j) = ((255 - s2)/(255 - r2))*(u(i,j) - r2) + s2;
end
end
end
pollenspread = uint8(uspread);
imwrite(pollenspread, ’mac835.jpg’);
%
338 CHAPTER 8. IMAGE PROCESSING IN SPACE DOMAIN
intensity = 1.2;
for k = 1:1
for j = 2:ny-1
for i = 2:nx-1
newu(i,j) = intensity*u(i,j) - ...
coe*(u(i-1,j) -2*u(i,j) + u(i+1,j))-...
coe*(u(i,j-1) -2*u(i,j) + u(i,j+1));
end
end
u = newu;
end
moon_sharpen = uint8(u);
imwrite(moon_sharpen, ’mac838.jpg’);
8.3.5 Exercises
1. Experiment with blur1d.m and vary the parameters for the time step and
the k-loop.
(a). Vary gw = 0=00005> 0=00010 and 0=00020=
(b). Vary n = 1 : 5> 1 : 10 and 1 : 20=
2. Experiment with blur2d_ncsu.m and vary the parameters for the time
step and the k-loop.
8.3. BLURRING AND SHARPENING 341
Image Processing in
Frequency Domain
343
344 CHAPTER 9. IMAGE PROCESSING IN FREQUENCY DOMAIN
A special case is d = 0 so that O(1) = 1@v= There are a number of other rules
for computing Laplace transforms, but here we list only four important rules.
Basic Rules for Laplace Transform. Assume all functions are such that
the improper integrals in the definition exist.
1. O(fi ) = fO(i ),
2. O(i + j) = O(i ) + O(j),
3. O( gi
gw ) = vO(i ) i (0) and
4. O(i j) = O(i )O(j) where i j is called the convolution of i and j
Z w
(i j)(w) i ( )j(w )g =
0
The third rule is important because it converts the derivative operation into
multiplication by the parameter v= The third property is established by using
the integration by parts formula
Z Z
ygx = xy xgy
gi
with gx = gw gw and y = hvw = Assume i (w) and the parameter v are such that
9.1. LAPLACE AND FOURIER TRANSFORMS 345
The proof of the convolution formula is more complicated, but it will be used
in the next subsection.
gx
= dx + i (w) with x(0) given.
gw
Compute the Laplace transform of both sides of the dierential equation and
use the first three basic rules
gx
O(
) = O(dx + i (w))
gw
vO(x) x(0) = dO(x) + O(i )=
Evidently,
x(w) = (i j)(w) + x(0)hdw
Z w
= i ( )j(w )g + x(0)hdw
0
Z w
= i ( )hd(w ) g + x(0)hdw
0
Z w
= hdw i ( )hd g + x(0)hdw =
0
This agrees with the solution in Section 6.1 where the homogeneous solution is
hdw and the particular solution is from the variation of parameters formula.
The solution of a dierential equation can be found using the Fourier trans-
formation. For example, find a solution of
g2 x
+ fx = i (w)=
gw2
As in the use of the Laplace transform, compute the Fourier transform of both
sides of the dierential equation and use the above rules
g2 x
I ( + fx) = I (i (w))
gw2
gx
(lzI ( ) + fI (x) = I (i (w))
gw
((lz)2 I (x) + fI (x) = I (i (w))=
9.1. LAPLACE AND FOURIER TRANSFORMS 347
1
L (w w0 ) (K(w (w0 )) K(w (w0 + )))
2
½ 1
= 2 > for w0 w w0 +
=
0> w ? w0 or w0 + ? w
(w w0 ) = lim L (w w0 )=
%$0
X
q1
= 2 (hl2@q )nm i (m@q)=
m=0
Definition 9.1.4. Let the complex number } hl2@q and let im be complex
numbers where m are integers from 0 to q 1. The discrete Fourier transform,
DFT, is a mapping from Cq into Cq whose nwk component is
X
q1
} nm im =
m=0
9.1. LAPLACE AND FOURIER TRANSFORMS 349
GI W (f ) = Ff
where the kj-components of the q×q matrix F are } nm = The matrix F is called the
Fourier matrix. Another useful perspective is that the nwk component equals
the evaluation of a polynomial at a { = } n
X
q1
[Ff ]n = sf (} n ) = im (} n )m where
m=0
X
q1
sf ({) im {m =
m=0
l2@3
s 9.1.1. Let q = 3 so that } = h
Example = cos(2@3) l sin(2@3) =
1@2l 3@2= The Fourier matrix is 3×3 with complex numbers as components
5 0 6 5 6
} }0 }0 1 1 1
F = 7 }0 }1 }2 8 = 7 1 }1 }2 8 =
}0 }2 }4 1 }2 }1
}0 }3 }6 }9 1 }3 }2 }1
Regroup the four components of f into even and odd subscripts by using a
permutation matrix S and note
5 65 6
1 1 1 1 i0
9 1 }2 } }3 : 9 i2 :
F4 f = (F4 S )(S f ) = 9 :9 :
7 1 1 }2 } 2 8 7 i1 8
1 }2 }3 } i3
¸ ¸
F2 GF2 fhyhq
= where
F2 GF2 frgg
¸ ¸
1 1 1 1
F2 = = with }b = hl2@4
1 }2 1 }b
¸ ¸ ¸
1 0 i0 i1
G = , fhyhq = and frgg = =
0 } i2 i3
Thus, the 4{4 matrix product reduces to two 2{2 matrix-vector products. A
similar reduction occurs when q = 8 = 23 and in general for q = 2g or g =
orj2 (q)= This generates a very fast implementation of the Fourier transform
called the fast Fourier transform. It requires (q@2)orj2 (q) operations, which is
much less than the 2q2 operations for the general matrix-vector product.
In applications q is typically very large and an e!cient method to do these
computations is required. In MATLAB the command t() is an implementation
of the DFT called the fast Fourier transform. The computation in Example
9.1.1 is easily done
AA t([1 7 2]) %n=3
10.0 -3.5000 - 4.3301i -3.5000 + 4.3301i.
We will make frequent use of this command.
9.1.6 Exercises
1. Use the Laplace transform to solve the initial value problem
gx
= f(80 x) with x(0) = 210=
gw
2. Use Laplace transform rule three twice to find the Laplace transform of
g2 x gx
with x(0) = 2 and (0) = 3=
gw2 gw
3. Let q = 5 and consider the DFT.
(a). Find the Fourier matrix F.
(b). Use the Fourier matrix and by-hand computations find the DFT of
f = [2 8 9 1 3]W =
(c). Use the MATLAB command t() to confirm the above calculation.
9.2. PROPERTIES OF DFT 351
GI W (f ) = Ff
where the kj-components of the q × q Fourier matrix F are } nmP = Or, the nwk
component of GI W (f ) is the evaluation of a polynomial sf ({) q1 m
m=0 im { at
a { = }n
X
q1
[Ff ]n = sf (} n ) = im (} n )m .
m=0
AA t([1 7 2 -2 3])’
ans =
11.0000
4.0902 + 6.1554i
-7.0902 - 1.4531i
-7.0902 + 1.4531i
4.0902 - 6.1554i
The first two properties follow from Euler’s formula. The third property follows
from
Move the terms on the right side to the left side and use the distribution prop-
erty to get
(} 1)(1 + } + } 2 + · · · + } q1 ) = 0=
Since } 1 is not zero, the second factor must be zero. Example 9.2.2 illustrates
the discrete Fourier transform of hl2nt for q = 3 and t = [0@3 1@3 2@3]W .
9.2. PROPERTIES OF DFT 353
DFT of Sine and Cosine Functions. Let [Ff ]n denote the nwk component
of the vector Ff . Then for f = hl2i t
½
l2i t 0> for n 6= i
[Fh ]n = and
q> for n = i
½
0> for n 6= q i
[Fhl2i t ]n = =
q> for n = q i
½
0> for n 6= q i or i
[F cos(2i t)]n = and
q@2> for n = q i or i
;
? 0> for n 6= q i or i
[F sin(2i t)]n = (q@2)l> for n = i =
=
(+q@2)l> for n = q i
The MATLAB code ttrig.m illustrates these rules, which are analogous to the
rules for the Fourier transform of the sine and cosine functions giving Dirac
delta functionals.
354 CHAPTER 9. IMAGE PROCESSING IN FREQUENCY DOMAIN
Figure 9.2.1 has the two graphs of the transformed functions. The reader should
note the locations and magnitudes of the transformed vectors, which confirms
the above important rules.
1 X nm
q1
} jm =
q m=0
Example 9.2.3. Use the Fourier matrix and show FF1 is the 3 × 3 identity
matrix
5 6 5 6
1 1 1 1 1 1
1
FF1 = 7 1 } 1 } 2 8 7 1 } 1 } 2 8
3
1 }2 }1 1 } 2 } 1
5 6
1 + 1 + 1 1 + }2 + } 1 + } + }2
17
= 1 + } + }2 1 + 1 + 1 1 + }2 + } 8
3
1 + }2 + } 1 + } + }2 1+1+1
5 6
3 0 0
17
= 0 3 0 8 = L=
3
0 0 3
AA f = [1 7 2]
AA g = t(f)
g = 10.0000 -3.5000 - 4.3301i -3.5000 + 4.3301i
AA it(g)
1.0000 7.0000 2.0000.
X
q1 X
q1
[Fa]n = sa (} n ) dm (} n )m and [Fb]n = sb (} n ) em (} n )m =
m=0 m=0
b = [e0 e1 e2 ]W
Definition 9.2.1. Let a and b have q components and let sa ({) and sb ({)
be polynomials associated with these vectors. The convolution of two vectors,
frqy(a> b)> is a 2q 1 vector such that
The analogue of the convolution property for the discrete Fourier transforms
requires the use of “padded” discrete Fourier transforms. This is a technical
di!culty caused by the fact that the dimension of vectors increases during the
convolution of two vectors. Let a be an q vector. A “padded” vector of a is a
2q 1 vector, A, with a in the first q components and zeros in the last q 1
components. For example, let q = 3 and a = [d0 d1 d2 ]W so that
A = [d0 d1 d2 0 0]W =
5 6 5 6
1 1
9 1 : 9 } :
9 : 9 :
= 9
9 1
: d0 e0 + 9
: 9 }2 : (d0 e1 + d1 e0 )
:
7 1 8 7 }3 8
1 }4
5 6
1
9
9 }2 :
:
+9
9 }4 : (d0 e2 + d1 e1 + d2 e0 )
:
7 } 8
}3
5 6 5 6
1 1
9
9 }3 :
:
9
9 }4 :
:
+9
9 } : (d1 e2 + d2 e1 ) + 9
: 9 }3 : d2 e2
:
7 }4 8 7 }2 8
}2 }
= F frqy(a> b)=
This is true for general vectors and gives the desired analogue of the convolution
of Fourier transform for functions.
Convolution Property of DFT. Let A and B be the “padded” vectors of
two q vectors a and b. The 2q 1 dimensional DFT of the convolution of a
and b is the component-wise or array product of the DFT of A and B
Example 9.2.4. The reader should confirm by-hand calculations the following
MATLAB computations, which verify the convolution property. Let q = 3 and
a = [2 3 6] and b = [8 6 3].
AA a = [2 3 6]
AA b = [8 6 3]
AA conv(a,b)
16 36 72 45 18
AA A = [a 0 0]
AA B = [b 0 0]
AA tA = t(A)
tA =
11.0000 -1.9271 - 6.3799i 1.4271 + 3.9430i
1.4271 - 3.9430i -1.9271 + 6.3799i
AA tB = t(B)
tB =
17.0000 7.4271 - 7.4697i 4.0729 - 0.6735i
4.0729 + 0.6735i 7.4271 + 7.4697i
358 CHAPTER 9. IMAGE PROCESSING IN FREQUENCY DOMAIN
AA lhs = t(conv(a,b))
lhs =
1.0e+002 *
1.8700 -0.6197 - 0.3299i 0.0847 + 0.1510i
0.0847 - 0.1510i -0.6197 + 0.3299i
AA rhs = tA.*tB
rhs =
1.0e+002 *
1.8700 -0.6197 - 0.3299i 0.0847 + 0.1510i
0.0847 - 0.1510i -0.6197 + 0.3299i.
If B is a signal or image, then its DFT will reflect the frequencies of the pe-
riodic parts of the signal or image. By masking or filtering out the unwanted
frequencies one can obtain a new signal or image by applying the inverse Fourier
transformation. A filter is a (2q 1) × 1 matrix, a (2q 1) × 1 column vector
for a 1D signal or image, whose components vary from 0 to 1 and with the same
dimension as the DFT of the padded vector. If the component is 1, then the
frequency is allowed to pass; if the component is 0, then the frequency is tossed
out.
Definition 9.2.2. Let I lowhu represent a (2q1)×1 matrix whose components
are in the unit interval. The filtered signal or image is
% are used to transform the filtered image back to the time domain.
clear;
t = 0:.001:1; % Define 10 sin(2 pi t) with high frequency variations.
x = 10*sin(2*pi*t) + 2*sin(2*pi*200*t) + 1*cos(2*pi*210*t);
%
tx = t(x,2002); % Padded t.
tx = tshift(tx);
%
figure(1)
subplot(2,1,1);
plot(t,x);
subplot(2,1,2);
plot(abs(tx));
%
freqfilter = zeros(1,2002); % Low pass filter.
freqfilter(1,700:1300) = 1;
newtx = freqfilter.*tx;
newx = itshift(newtx);
%
newx = it(newx,2002); % Padded inverse t.
%
figure(2)
subplot(2,1,1);
plot(abs(newtx));
360 CHAPTER 9. IMAGE PROCESSING IN FREQUENCY DOMAIN
subplot(2,1,2);
plot(t,real(newx(1,1:1001)));
9.2.6 Exercises
1. Let q = 3> t = [0 1@3 2@3]W and i = 0> 1 and 2.
(a). By-hand calculations find the DFT of cos(2i t).
(b). By-hand calculations find the DFT of sin(2i t).
(c). Use the MATLAB command t() to confirm the above calculations.
2. Consider the MATLAB code ttrig.m.
(a). Use a variation of this code to find the DFT of 3 cos(230t)=
(b). Use it to find the DFT of 3 cos(2100t)+4 sin(2120t)+20 cos(2t)=
3. Let q = 5 and consider the inverse DFT.
(a). Find the inverse Fourier matrix F1 =
(b). Verify it is the inverse of F, that is, show FF1 = L=
4. Consider Example 9.2.4 of the convolution property.
(a). By-hand calculations verify the MATLAB computations.
(b). Let q = 4 and use MATLAB to verify the convolution property for
a = [1 3 4 1]W and b = [8 9 4 1]W =
5. Consider the MATLAB code tsine.m. Adjust the filter matrix, freqfilter,
so that the following is filtered
9.3 DFT in Rq × Rq
In the previous section the DFT of a single vector was defined to be another
vector with nwk component
X
q1
(hl2@q )nm im = [Ff ]n = sf (} n ) where } hl2@q =
m=0
Three important properties were the DFT of sine and cosine functions, the
inverse DFT and the convolution identity. These were used to illustrate a low-
pass filter for a noisy 1D image or signal. The objective of this section is to
extend the DFT to 2D images, which are represented by matrices and not a
single vector. This will allow us to filter unwanted attributes of 2D images.
X X
q| 1 q{ 1
(hl2@q| )n| m| (hl2@q{ )n{ m{ im{ >m| = si (}{n{ > }|n| ) where
m| =0 m{ =0
X X
q| 1 q{ 1
si ({> |) im{ >m| {m{ | m| , }{ hl2@q{ and }| hl2@q| =
m| =0 m{ =0
n
Let GI W (i ) = j = si (}{n{ > }| | ). The inverse 2D DFT can be shown to have
m{ > m| -component equal to
1 X X
q| 1 q{ 1
im{ >m| = }{n| m| }|n{ m{ jn{ >n| =
q{ q|
n| =0 n{ =0
Both the 2D DFT and its inverse can easily be computed by using the
MATLAB commands t2() and it2(). For example, for q{ = 2 and q| = 3 :
362 CHAPTER 9. IMAGE PROCESSING IN FREQUENCY DOMAIN
AA a = [1 2 5; 6 7 11]
a= 1 2 5
6 7 11
AA ta = t2(a)
ta = 32.0000 -5.5000 + 6.0622i
-5.5000 - 6.0622i
-16.0000 0.5000 - 0.8660i
0.5000 + 0.8660i
AA it2(ta)
= 1 2 5
6 7 11.
As in the 1D DFT of the sine and cosine functions, the 2D DFT transforms
will identify the frequencies as “spikes” in the matrix of the Fourier transforms.
Let { = 0 : 1@q{ : 1 1@q{ and | = 0 : 1@q| : 1 1@q| .
The following MATLAB code ttrig2d.m also illustrates these rules. Figure
9.3.1 is the graph of the 2D DFT of the "padded" image of the indicated sine
and cosine functions of two variables. The frequency domain has been shifted
so that the center frequencies are the smaller frequencies. The "spikes" are
located near a circle of radius 400 from the center. Since the "spikes" are very
large, the vertical axis in the mesh plot is the orj(1 + dev(i i wx)).
X X
q| 1 q{ 1
sd ({> |) d(m{ > m| ){m{ | m| =
m| =0 m{ =0
The following MATLAB computation verifies this for the above 2×3 matrices
with 2 q{ 1 = 3 and 2 q| 1 = 5:
AA t2(conv2(a,b))
1.0e+002 *
Columns 1 through 3
8.6400 -2.7193 + 2.0698i -1.0757 + 0.8192i
-2.4000 - 1.8013i 1.0034 + 0.1752i 0.2677 - 0.1158i
-2.4000 + 1.8013i 0.5290 - 0.9709i 0.2249 - 0.4680i
Columns 4 and 5
-1.0757 - 0.8192i -2.7193 - 2.0698i
0.2249 + 0.4680i 0.5290 + 0.9709i
0.2677 + 0.1158i 1.0034 - 0.1752i
AA norm(t2(conv2(a,b)) - t2(a,3,5).*t2(b,3,5))
4.8969e-014.
If e is an image, then the DFT of e will reflect the frequencies of the periodic
parts of the image. By masking or filtering out the unwanted frequencies one
can obtain a new image by applying the inverse DFT. A filter is a matrix with
the same dimension as the DFT of the padded image. The components of the
filter usually vary from 0 to 1. If the component is 1, then the frequency is
allowed to pass; if the component is 0, then the frequency is deleted.
366 CHAPTER 9. IMAGE PROCESSING IN FREQUENCY DOMAIN
The following MATLAB code tsine2d.m illustrates this for a low frequency sine
wave with higher frequency sine “noise.”
MATLAB code tsine2d.m
% This code creates a noisy 2D sine wave.
% The 2D DFT is used to filter the unwanted
% higher frequencies.
clear;
nx = 200;
ny = 600;
for i = 1:nx % Define the big wave with periodic noise.
for j = 1:ny
u(i,j) = 0+100*(1+sin(2*pi*((j-1)/ny)*5)) + ...
15.*(1+sin(2*pi*((i-1)/nx)*80))+...
15.*(1+sin(2*pi*((j-1)/ny)*80));
end
end
sine = uint8(u);
imwrite(sine, ’sine.jpg’);
tu = t2(u,2*nx-1,2*ny-1); % Padded t.
tu = tshift(tu);
figure(1);
mesh(u’);
figure(2)
mesh(log(1+(abs(tu))));
filter = ones(2*nx-1,2*ny-1);
d0 = 150; % Use ideal low-pass filter.
for i = 1:2*nx-1
for j =1:2*ny-1
dist = ((i-nx)^2 + (j-ny)^2)^.5;
if dist A d0
filter(i,j) = 0;
end
end
end
figure(3)
mesh(filter);
fil_sine = filter.*tu;
figure(4)
mesh(log(1+abs(fil_sine)));
9.3. DFT IN RQ × RQ 367
fil_sine = itshift(fil_sine);
fil_sine = it2(fil_sine);
fil_sine = real(fil_sine(1:nx,1:ny));
fil_sine = uint8(fil_sine);
imwrite(fil_sine, ’sine_fil.jpg’);
Figure 9.3.2 is the noisy sine wave with unwanted higher frequencies. Figure
9.3.3 is the mesh plot of the real matrix associated with the noisy image. Figure
9.3.4 is the log plot of the magnitude of the shifted DFT for the noisy sine wave;
note the spikes indicating the higher frequencies away from the center. Figure
9.3.5 is the low-pass filter used to eliminate the spikes in the previous figure.
Figure 9.3.6 is the filtered DFT, and Figure 9.3.7 is the filtered image of the
noisy sine wave.
9.3.5 Exercises
1. Let d = [1 3 5 6; 2 4 7 19] and e = [2 3 4 7; 1 5 2 10]=
(a). Find the t2 of d and e and
(b). Verify it2(t2(d)) = d.
2. Use a variation of ttrig2da.m to find
(a). t2(sin(2({ + 3|)) and
(b). t2(cos(2({ + 3|))=
3. Consider a variation of ttrig2d.m to find the DFT of
x = 1 + cos(2{200) + 4 cos(2|200) + 16 sin(2{200) + 64 cos(2{200)=
4. Consider the vectors in exercise 1.
(a). Find frqy2(d> e) and
(b). Verify the convolution property for the 2D DFT.
5. Use a variation of tsine2d.m to find the 2D DFT of
x = 100(1 + cos(2|10)) + 30(1 + cos(2{120) + sin(2|120))=
6. Use a variation of tsine2d.m to find the 2D DFT of
x = 100(1 + cos(2|10)) + 30(1 + cos(2{120 + 2|120))=
368 CHAPTER 9. IMAGE PROCESSING IN FREQUENCY DOMAIN
The parameters z and g0 control the width and location of the band. In
the Butterworth band-reject filter the exponent q controls the steepest of the
boundaries in the band of frequencies to be rejected. There are similar versions
of high and low-pass filters.
if dist A d0
filter(i,j) = 0;
end
end
end
subplot(2,2,3);
mesh(filter);
fil_ncsu = filter.*tu;
subplot(2,2,4);
mesh(log(1+abs(fil_ncsu)));
fil_ncsu = itshift(fil_ncsu);
fil_ncsu = it2(fil_ncsu,2*nx-1,2*ny-1);
fil_ncsu = real(fil_ncsu(1:nx,1:ny));
fil_ncsu = uint8(fil_ncsu);
imwrite(fil_ncsu, ’sinencsuid_fil.jpg’);
for i = 1:2*nx-1
for j =1:2*ny-1
dist = ((i-nx)^2 + (j-ny)^2)^.5;
if dist ~= d0
filter(i,j)= 1/(1 + (dist*w/(dist^2 - d0^2))^(2*n));
else
filter(i,j) = 0;
end
end
end
9.4. FREQUENCY FILTERS IN RQ × RQ 375
added to the DFT of the padded image. The sharpened image is in Figure
9.4.11. The filter portion of the code is indicated below:
% Use Butterworth or Gaussian high-pass filter.
filter = ones(2*nx-1,2*ny-1);
d0 = 100;
n = 4;
for i = 1:2*nx-1
for j =1:2*ny-1
dist = ((i-nx)^2 + (j-ny)^2)^.5;
% Use Butterworth high-pass filter.
% filter(i,j)= 1/(1 + (dist/d0)^(2*n));
% filter(i,j)= 1.0 - filter(i,j);
% Use Gaussian high-pass filter.
filter(i,j) = exp(-dist^2/(2*d0^2));
filter(i,j) = 1.0 - filter(i,j);
end
end
..
.
% Update image with high frequencies.
fil_micro = tu + filter.*tu;
378 CHAPTER 9. IMAGE PROCESSING IN FREQUENCY DOMAIN
9.4.6 Exercises
1. Experiment with the parameter g0 in the ideal low-pass filter in fil-
ter_ncsuid.m.
2. Modify the filter in filter_ncsuid.m to become an ideal band-reject fil-
ter. Experiment with the location and width of the band of frequencies to be
rejected.
3. Experiment with the parameters q> z and g0 for the Butterworth band-
reject filter in filter_ncsubu.m.
4. Modify the filter in filter_ncsubu.m to become a Gaussian band-reject
filter and experiment with its parameters.
5. Experiment with the parameters q> z and g0 for the Butterworth band-
reject filter in filter_aerail.m.
6. Modify the filter in filter_aerial.m to become a Gaussian band-reject
filter and experiment with its parameters.
7. Experiment with the parameters in the high-pass Gaussian filter used
filter_micro.m. Also, modify the sharpening step to include other linear com-
binations:
% Update image with high frequencies.
fil_micro = a*tu + b*filter.*tu;
8. Repeat exercise 7 using the high-pass Butterworth filter.
Appendix A
Chapter One
Section 1.1
1. (a). 0.333 (b). 0.111; huuru = 0=000111
3. (a). 2z + 3} = 25 + 3l
(b). }z = 17s+ 19l; }@z = (11 23l)@13
5. } 2 = 1@2 + ( 3@2)l and 1 + } + } 2 = 0
7. } + 0 = (3 + 4l) (7 + 2l) = (3 7) + (4 2)l = 4 + 2l
9. 1@58 + (17@58)l
11. 9@25 + (12@25)l
13. Prove |} + z|2 (|}| + |z|)2 : Use the dot product as in Section 1.4.
Chapter One
Section 1.2
1. (a). x = 2{ | + 7 and y = { + 2| + 1
(b). } = 3@5 + (9@5)l
3. (a).
{2 + | 2 + | 2 3{
x= and y = 2
{2 + (| 1)2 { + (| 1)2
(b).
s } = 3@2 + l
5. 1 + l = 1=0987 + 0=4551l
7. (a). s ± (1=0987 + 0=4551l)
} = 1
9. (1 + l)10 = ( 2)10 hl(2@8)10 = 32l
11. 3 + 4l = 5hl where = arctan(4@3)
Solutions of } 3 = 3 + 4l are
51@3 hl@3 (hl2@3 )0 = 1=6289 + 0=5202l
51@3 hl@3 (hl2@3 )1 = 1=2650 + 1=1506l and
51@3 hl@3 (hl2@3 )2 = 0=3640 1=6708l
381
382 APPENDIX A. SOLUTIONS TO ODD EXERCISES
z = hl2@q
= cos(2@q) + l sin(2@q)
= cos(2@q) l sin(2@q)
1
z1 =
cos(2@q) + l sin(2@q)
1 cos(2@q) l sin(2@q)
=
cos(2@q) + l sin(2@q) cos(2@q) l sin(2@q)
cos(2@q) l sin(2@q)
=
1
Chapter One
Section 1.3 ¸
$ $
$
2
1. d = 2 l + 5 m and a =
5
$
$
3. d e = [3 2]
5. [{1 {2 ] = [1 3] [2 5] = [3 2]
$
7. Hint: $d = [d1 d2 ]> e = [e1 e2 ] and
$
f = [f1 f2 ] and
compute
s both sides.
9. k
$d k = 1 + 49> arctan(7@1)
s s 81=87 degrees and
$
d @ k
$d k = [1@ s50 s7 50] s
11. 13 = 29 + 10 2 29 10 cos(). Then cos() = 13@ 290 and
40=24 degrees.
$
13. k [353=55 473=55]
Chapter One
Section 1.4
1. The angle between the vectors is 30 degrees, which is also given by
s s
1 3 + 3(1) = 2(2) cos()=
s s
3. 1(3) + 2(1) = 5 10 cos() giving 98=13 degrees.
5. [3 1] • [{ 1] = 3{ 1 = 0 gives { = 1@3=
$
7. (a). Hint:
$d = [d1 d2 ]> e = [e1 e2 ] and compute both sides.
(b). Both sides are equal to 11.
(c). Proofrequires equality
¸ for all possible vectors.
2 1 3 2
s
9. duhd = (det( )) = 7 so that the duhd is 7=
¸2 1 ¸
2 1 12 3
11. det( ) = 3 and det( ) = 54=
1 1 6 3
$
$
13. zrun = i • g = 3(30) + 1(6) = 96=
APPENDIX A. SOLUTIONS TO ODD EXERCISES 383
Chapter One
Section 1.5
1. (a). | = (1@3){ 3 (b). { = 3w and | = w 3
3. (a). [{ |] = [2 7] + w[1 1] (b). { = 2 + w and | = 7 w
5. [0 1] + (3@5)[1 2] = [3@5 11@5]
Chapter Two
Section 2.1 5 6
3
$
$ $
$
3. d = 3 l + m + 2 n or a = 7 1 8
2
$
5. (a). 2
$d + e = [7 5 °5] ° (b). $
{ = [5 1 3]
$
s ° °
$
s $
$
7. (a). k d k = 21 and ° e ° = 11 (b). d • e = 3
s s
9. 1(1) + 2(3) 1(1) = 6 11 cos() giving 10=02 degrees
$
11. (a). Hint: $
d = [d1 d2 d3 ] and e = [e1 e2 e3 ] and
compute both sides.
(b). Both sides are equal to 26.
(c). Proof requires equality for all possible vectors.
$
$
13. zrun = i • g = 10(25) + 1(20) + 2(5) = 280
Chapter Two
Section 2.2
$
$
$ $
°
°$ $°° s
1.
$d × e = 3 l + m 9 n and ° d × e ° = 91
s
3. 46
$
5. Hint: $
d = [d1 d2 d3 ] and e = [e1 e2 e3 ] and
compute both cross products.
$
$
7. u × i = [0 1 3]
$ $
9. (a). (
$
d × e)• f = [4 3 19] • [2 1 5] = 84
(b). 84
11. Hint: expand the determinants.
Chapter Two
Section 2.3
1. [1 2 1] + w[3 1 1]
3. (a). [1
s 1 2] + (7@3)
p [1 1 1]
(b). 168@3 = 3(56)@3
p
(c). 56@3
s
5. 2
7. (a). ¸ ¸ ¸
5 3 w 4
=
3 3 3
(b). w = 1@2 and = 1@2
384 APPENDIX A. SOLUTIONS TO ODD EXERCISES
p
(c). 3@2
Chapter Two
Section 2.4
1. (a). 1(2) + 2(1) + 4(2) = 12
(b). [1 2 4]
(c). [1 2 4] • [{ 2 | 1 } 2] = 0
3. (a). [1 2 3]
(b). [1 2 3] • [{ 1 | 1 } 1] = 0
{ 2| 3| = 6
5. (a).
2 = 1 + 2u + v
3 = 1 + 3u + 0v
1 = 1 + 0u + 0v
Chapter Two
Section 2.5
1. (a). sW = [5 6 2 1 5] and pW = [1000 850 1700 2220 1150]
(b). sW p = 21470
3. (a).
5 65 6 5 6
1 2 4 u 10
7 0 1 1 87 v 8 = 7 1 8
1 1 4 w 4
Chapter Three
Section 3.1
1. (a). [8 19 17 23 7]
(b). 5 6
2 0 1
9 5 1 4 :
9 :
9 7 2 2 :
9 :
7 1 7 1 8
1 4 3
(c). 30
s
(d). 80
3. (a). 5 6
1 27 0
9 216 8 1 :
9 :
7 27 1 64 8
125 27 8
(b). [1@3 6@2 3@(1) 5@3]
(c). 5 6
sin(1) sin(3) sin(0)
9 sin(6) sin(2) sin(1) :
9 :
7 sin(3) sin(1) sin(4) 8
sin(5) sin(3) sin(2)
5. (a). ¸
397 394 389
I =
391 388 383
(b). [X Y] = meshgrid(1:1:3,1:1:3)
F = 400 - 20*X.^2 - Y.^2
7. (a). F({> |) = 4{| + (2{ + 2|)200@({|)
(b). About F(8=1> 8=1)=
9.
i1 i2
@12 386. -373.
2@12 200. -173.
3@12 141. -100.
4@12 115 -57.
11. l1 = 0> l2 = 5 and l3 = 5
Chapter Three
Section 3.2 ¸
2 1 6 8
1. Dx = [35 37]W and D + F =
6 2 14 10
¸
2 21 2 12
3. Both sides should equal =
7 3 17 17
386 APPENDIX A. SOLUTIONS TO ODD EXERCISES
¸
23 26 37 7
5.
1 0 9 7
¸
33 55
7. Both sides should equal =
37 135
9. = 32@100>
u1 = [64=00 00=00 22=40]W >
u2 = [87=04 27=64 30=46]W
u = [167=50 135=00 102=50]W
Chapter Three
Section 3.3
1. x = [2 4 3]W
3. x = [50 2 5 1]W
5. y = [210 115 156=67]W and x = u = [182=5 155=0 117=5]W
7. Use forward and backward substitution.
Chapter Three
Section
5 3.4 6
2 1 1 7
1. 7 4 1 1 0 8
0 3 1 2
5 6 5 6
1 0 0 0 1 0 0 0
9 0 1 0 0 : 9 0 0 1 0 :
3. H43 (3) = 9 :> S = 9 : > and
7 0 0 1 0 8 23 7 0 1 0 0 8
0 0 7 1 0 0 0 1
5 6
1 0 0 0
9 0 1 0 0 :
H4 (10) = 9
7 0
:=
0 1 0 8
0 0 0 10
5 6
1 7 1 1
5. (a). [D d] = 7 2 0 7 2 8
0 14 1 6
5 6
1 7 1 1
(b). H32 (2) H21 (1) [D d] = 7 0 14 b
5 0 8 = [X d]
0 0 6 6
(c). b is
The solution of the equivalent system X x = d
W
x = [5@2 5@14 1] =
5 6
1 1 0 1 6
9 1 0 1 1 4 :
7. (a). [D d] = 9
7 0 2
:
7 0 4 8
2 0 0 1 8
(b). H43 (2@9) H42 (2) H32 (2) H41 (2) H21 (1) [D d] =
APPENDIX A. SOLUTIONS TO ODD EXERCISES 387
5 6
1 1 0 1 6
9 0 1 1 2 1 :
9 b
: = [X d]
7 0 0 9 4 2 8
0 0 0 1@9 58@9
(c). The solution of the equivalent system X x = db is
W
x = [25 89 26 58] =
5 6
1 1 1 0
9. (a). [D d] = 7 1 0 4 20 8
0 2 4 10
5 6
1 1 1 0
(b). H32 (2) H21 (1) [D d] = 7 0 1 5 20 8 = [X d] b
0 0 14 50
(c). The solution of the equivalent system X x = db is
x = i = [40@7 15@7 25@7]W =
11. Hint: Do the elementary matrix products.
Chapter Three
Section 3.5 ¸
3@10 1@10
1. (a). D1 =
1@@10 3@10
(b). H12 (1@3) H2 (3@10) H1 (1@3) H21 (1@3) [D L] = [L D1 ]
5 61 5 6
3 1 0 0 3@10 1@10 0 0
9 1 3 0 0 : 9 0 :
(c). 9 : = 9 1@@10 3@10 0 :
7 0 0 3 0 8 7 0 0 1@3 0 8
0 0 0 4 0 0 0 1@4
5 61 5 6
3 1 0 8 3 1
3. (a). 7 1 3 1 8 = (1@21) 7 3 9 3 8
0 1 3 1 3 8
5 65 6 5 6
8 3 1 1 17@21
(b). (1@121) 7 3 9 3 8 7 2 8 = 7 30@21 8
1 3 8 3 31@21
5. Hint: Do the row5 operations.
6 5 6
1 0 0 1 2 0
7. (a). D1
1 =
7 0 1 0 8 and D1 2 =
7 0 1 0 8
4 0 1 0 0 1
5 6
1 2 0
(b). (D1 D2 )1 = D1 1
2 D1 =
7 0 1 0 8
4 0 1
5 6
1 2 0
(D2 D1 )1 = D1 1
1 D2 =
7 0 1 0 8
4 8 1
9. See text form MATLAB commands.
388 APPENDIX A. SOLUTIONS TO ODD EXERCISES
Chapter Three
Section 3.65 65 6
1 0 0 0 2 1 0 0
9 1@2 0 : 9 0 :
1. (a). 9 1 0 : 9 0 3@2 1 :
7 0 2@3 1 0 8 7 0 0 4@3 1 8
0 0 3@4 1 0 0 0 5@4
1 1 W
(b). y = [470 625 823 3 1087 2 ] and x = [870 1270 1270 870]W
5 65 6
1 0 0 2 1 1
3. (a). S13 D = 7 1@2 1 0 8 7 0 5@2 1@2 8 and
0 2 1 0 0 5
S13 d = [3 2 6]W
(b). y = [3 1@2 5]W and¸ x = [1 9 1]
W
Chapter Four
Section 4.1
1. (a). I (p> f) = 14p2 + 12pf + 3f2 56p 22f + 59
(b). p = 18@6 and f = 14@7
(c). | = 38@3
3. (a). I (p> f) = 204p2 + 60pf + 8f2 7976p 1634f + 85771
(b). p = 7=4167 and f = 68=75
(c). | = 180=0
5. (a). x(0) = 154=2983 and u = 0=04419
(b). x(60) = 10=8893
Chapter Four
Section 4.2 5 6 5 6
2 7 5 12
1. (a). DW = 7 0 1 8 and (b). E W DW = 7 5 19 8
1 2 10 27
3. Hint: Compute the transpose of DD1 = L.
5. (a).
¸ ¸ ¸
30 10 p 19130
=
10 4 f 7735
5 6 5 6
1 1 ¸ 0
9 2 1 : }1 9 0 :
(b). Dz = 9 : 9 :
7 3 1 8 }2 = 7 0 8 implies four equations must
4 1 0
hold:
Chapter Four
Section 4.3
1. (a). OK
(b). No. The data are not "near" a plane.
(c). p1 = 4=0, p2 = 1=0, f = 92 and rW r = 36 AA 0=25=
390 APPENDIX A. SOLUTIONS TO ODD EXERCISES
3. (a).
5 6 5 6
2 3 3 1 25000
9 3 4 4 1 : 9 24200 :
9 : 9 :
9 5 4 4 1 : 9 23900 :
9 : 9 :
9 8 5 5 1 : 9 23200 :
9 : 9 :
D=9
9 12 6 6 1 : and d = 9
: 9 22600 :
:
9 15 8 8 1 : 9 22100 :
9 : 9 :
9 19 10 10 1 : 9 21500 :
9 : 9 :
7 23 11 11 1 8 7 20500 8
27 13 13 1 20100
Chapter Four
Section 4.4
1. function Fnm = Fnm(p)
Fnm = 100 + 2*(p(1)-2)^2 + 4*(p(2)-3)^2 + 10*(p(3)-4)^2;
AA [p Fval] = fminsearch(’Fnm’, [10 10 10])
3. Use rad_fmin.m to get
x(w) = 155=9841h0=044993w and x(60) = 10=4834
5. Use variation of pop_us_fmin.m
Chapter Five
Section 5.1
1. (a). n = [2 2 2] and n b = [1 1 1]
(b). 5 u = 1@26 and 5v = 1@26 5 6
2@3 1@3 2
3. x = u7 1 8 + v7 0 8+ 7 0 8
0 1 0
5 6 5 6 5 6
1@2 0 5
5. x = u7 1 8 + v7 0 8 + 7 0 8
0 1 0
APPENDIX A. SOLUTIONS TO ODD EXERCISES 391
Chapter Five
Section 5.2 5 6
4
9 0 :
1. (a). xs = 9
7 0 8
:
50 6 5 6 5 6
2@3 1@3 0
9 1 : 9 0 : 9 0 :
(b). xk = u 9 : 9
7 0 8 + v7 1 8 + w7 0 8
: 9 :
5 6 0 5 6 0 5 1
6 5 6
4 2@3 1@3 0
9 0 : 9 1 : 9 0 : 9 0 :
(c). x=9 : 9 : 9
7 0 8 + u7 0 8 + v7 1 8 + w7
: 9 :
0 8
0 ¸0 0 1
3 2 1 1 6
3. (a).
0 50 0 16 1
5@3
9 0 :
(b). xs = 9
7 0 8
:
5 1 6 5 6
2@3 1@3
9 1 : 9 0 :
(c). xk = u 9 :
7 0 8 + v7 1 8
9 :
5 06 5 06 5 6
5@3 2@3 1@3
9 0 : 9 1 : 9 0 :
(d). x=9 : 9
7 0 8 + u7 0 8+ v7 1 8
: 9 :
5 1 6 0 0
2 1 1 3 4
5. (a). 7 0 3 1 8 6 8
0 0 0 0 0
392 APPENDIX A. SOLUTIONS TO ODD EXERCISES
5 6
1 0 2@6 1@6 6@6
(b). 7 0 1 1@3 8@3 6@3 8
0 0 0 0 0
5 6 5 6 5 6
1 2@6 1@6
9 2 : 9 1@3 : 9 8@3 :
(c). x=9 : 9 : 9
7 0 8 + u7 1 8 + v7 0 8
:
0 0 1
5 6
1 0 0 1 0 0 1
9 0 1 0 1 0 1@2 1 :
7. 9 :
7 0 0 1 1 0 0 1 8
0 0 0 0 1 1@2 2
Chapter Five
Section 5.3
1. HD15H W = [25@6]> y =
6 30@29 and x = [89@29 72@29 106@29]
W
1@2 1@2
3. Q =7 1 0 8 > z = [2=4828 3=6552]W and
0 1
x = [3=0690 2=4828 3=6552]W
Chapter Six
Section 6.1
1. Left side = (95@3)h3w 2@3
Right side = 3[(95@9)h3w 2w@3 5@9] + 2w + 1
{(0) = 95@9 5@9
3. (a). {k = Fh5w
(b). {s = (3@5)w 13@25
(c). { = (138@25)h5w + (3@5)w 13@25
5. (a). {k = Fh2w
(b). {s = 10h3w
(c). { = 5h2w + 10h3w
7. (a). {k = Fh5w
(b). {s = (25@13) cos(w) + (5@13) sin(w)
(c). {R = (38@13)h5w + (25@13) cos(w) + (5@13) sin(w)
9. {s = h5w h5 (3 + 2)g = w(3@5) 13@25
Chapter Six
Section 6.2
1. (a). Compute the second derivative and show p{002 + n{2 = 0=
(b). {(0) = (1@2)0 + 2 = 2
{0 (w) = (1@2) cos(2w)2 + 2( sin(2w))2
{0 (0) = (1@2) cos(0)2 + 2( sin(0))2 = 1
3. Compute {0 and {00 and show {00 + 4{0 + 4{ = 0=
{(0) = 1 + 0
{0 (w) = h2w (2) + 5(1h2w + wh2w (2))
APPENDIX A. SOLUTIONS TO ODD EXERCISES 393
{0 (0) = 2 + 5(1 + 0) = 3
5. (a). {k = f1 h3w + f2 h2w
(b). { = (3@5)h3w + (2@5)h2w
7. (a). {k = f1 h4w + f2 wh4w
(b). { = 2h4w + 11wh4w
9. (a). Hint: Subtract the two equations.
(b). Hint: Show } must be equal to zero.
Chapter Six
Section 6.3
1. (a). {01 = hw cos(w) + hw ( sin(w))
{02 = hw sin(w) + w
¸ h (cos(w))
1 0
(b). Z (0) =
(c). det (Z (0)) = 6= 0
3. (a). {k = f1 cos(3w) + f2 sin(3w)
(b). { = 2 cos(3w) +s (1@3) sin(3w) s
5. (a). {k = f1 hw cos(s 2w) + f2shw sin( 2w)
s
(b). { = 1hw cos( 2w) +s(3@ 2)hw sin( 2w)
s s s s
7. n=1: ( 3 + 3@2)h(2+ 3)w + ( 3@2)(2 + 3)h(2 3)w
n=4: 3hs2w + 6wh2w
s s
n = 10 : 6h2w sin( 6w) + 3h2w cos( 6w)
Chapter Six
Section 6.4
1. { = f1 hw + f2 h4w + (3@4)w + 1@16
3. { = f1 hw + f2 h4w + (1@6)h2w
5. { = f1 cos(2w) + f2 sin(2w) + 1 cos(w) + (1@3) sin(w)
7. { = 6hw + (13@4)h2w + (3@2)w 7@4
9. { = hw + (29@48)h4w + (1@3)h2w + (3@4)w + 1@16
1
11. (a). {s = 1$ 2 cos($w)
1 1
(b). limz$1 (( 1$ 2 ) cos(w) + 1$ 2 cos($w)) = (1@2)w sin(w)
Chapter Six
Section 6.5
3. {s = h3w y1 + h3w y2 = (2@9)w 1@3 where
y1 = (1@6)[(2@3)h3w w + (11@9)h3w ] and
y2 = (1@6)[(2@3)h3w w + (7@9)h3w ]
Chapter Seven
Section 7.1 ¸0 ¸ ¸
{ 0 1 {
1. (a). =
| 1 8 |
394 APPENDIX A. SOLUTIONS TO ODD EXERCISES
¸ ¸ s
{ 1s
(b). = f1 h(4+ 15)w +
| 4 + 15
¸ s
1s
f2 h(4 15)w
4 5
s s s s
(c). f1 = (4 15)@( 15) and f2 = (4 + 15)@( 15)
3. (a). Replace
¸ 100(L) ¸by 50 (L). ¸
{ 1 1
(b). = f1 h(2@25)w + f2 h(6@25)w
| 2 2
(c). ¸ f1 = 7@4
and¸f2 = 1@4 ¸
{ 1 1
5. = (1) h4w + 3 h2w
| 1 2
Chapter Seven
Section 7.2 ¸ ¸ ¸
0 1 1s s 1s
1. (a). Show = (2 3)
1 4 2 3 2 3
s
(b). Let u = 2 3¸and ¸ ¸
0u 1 x1 0
= implies
1 4 u x2 0
s s
(2 + 3)x1 + 1x2 = 0 and x1 + (2 + 3)x s 2 = 0=
Let x1 = 1 and then¸ compute
¸ x2 = (2 ¸ 3)=
+
2 3 1 1
3. (a). Show = (1)
1 2 1@3 1@3
¸ ¸ ¸
2 (1) 3 x1 0
(b). = implies
1 2 (1) x2 0
(1)x1 + (3)x2 = 0 and (1)x1 + (3)x2 = 0=
Let x1 = 1 and then compute
¸ x2 = 1@3=
4@25 u 1@25
5. det( ) = (4@25 u)2 4@252 = 0 implies
4@25 4@25 u
u = 6@25 and ¸ 2@25=
4u 1
7. det( ) = (4 u)(1 u) + 2 = 0 implies
2 1 u
u = 2 and 3=
Chapter Seven
Section 7.3
1. (a). u = 5@2 and 5@2
(b). u = 2 + l and 2 l
(c). u = 0 and 2= ¸ ¸
1 1
3. (a). u1 = 1 and u1 = ; u1 = 5 and u1 =
1 1
w ¸ w ¸
h h5w h @2 hw @2
(b). [(w) = and [(w)1 =
h w
h5w h5w @2 h5w @2
w 5w
¸ w
¸
h h h (w@2 + 0)
(c). [(w)y(w) =
hw h5w h5w (w@2 4@10)
APPENDIX A. SOLUTIONS TO ODD EXERCISES 395
¸
w 2@5
=
2@5 ¸ 3w ¸ 2w ¸
hw h5w h @3 h @3
(d). [(w)y(w) = =
hw h5w 0 h2w @3
¸ ¸ ¸ ¸
1 2w 1 3w 185@36 7@6
5. (37@4) h + (28@9) h + + w
1 ¸ 2 ¸ 109@36 ¸ 1@6
1 1 17@26
7. (7@2) h2w + (67@13) h3w + cos(2w)+
2 1
¸ 11@13
7@26
sin(2w)
10@13
Chapter Eight
Section 8.1
1. (c). Replace letc = zeros(50,40) with
shade = 100 % for example
letc = shade*ones(50,40)
3. Vary red, green and blue in color_imagncsu.m
Chapter Eight
Section 8.2
1. In the code file pollenhist.m
replace¸ pollen.jpg by mars3.jpg.
235 201 170
3. xqlw8(x 40) =
155 17¸ 0
255 245 222
5. qhzx =
211 115 102
9. Using s = 1=5
¸
255 234 191
qhzx =
171 27 14
Chapter Eight
Section 8.3
Have fun comparing the new images.
Chapter Nine
Section 9.1
1. vO(x) 210 = f80(1@v) fO(x)
O(x) = 210@(v + f) + f80(1@(fv) 1@(f(v + f)))
= O(210hfw + 80 80hfw )
x(w) = 130hfw + 80
3. (a). Let } = hl2@5 and
5 6
1 1 1 1 1
9 1 } }2 }3 }4 :
9 2 4
:
F5 = 99 1 }3 } } }3 : :
7 1 } } }4 }2 8
1 }4 }3 }2 }
396 APPENDIX A. SOLUTIONS TO ODD EXERCISES
5 65 6 5 6
1 1 1 1 1 2 23=0000
9
9 1 } }2 }3 }4 :9
:9 8 : 9
: 9 2=6910 + 9=4576l :
:
(b). F5 f = 9
9 1 }2 }4 } }3 :9
:9 9 :=9
: 9 3=8090 4=6695l :
:
7 1 }3 } }4 }2 87 1 8 7 3=8090 + 4=6695l 8
1 }4 }3 }2 } 3 2=6910 9=4576l
Chapter Nine
Section 9.2
1. (a). Let } = hl2@35 and i = 1= 65 6 5 6
1 1 1 sin(21(0)) 0=0
F3 sin(21t) = 7 1 } } 2 8 7 sin(21(1@3)) 8 = 7 1=5l 8
1 }2 } sin(21(2@3)) 1=5l
l2@5
3. (a). Let } = h and
5 6
1 1 1 1 1
9 1 }4 }3 }2 } :
19
9 1 }3 }
:
F1
5 = }4 }2 :
59
7 1 }2 }4 } }3
:
8
1 } }2 }3 }4
Chapter Nine
Section 9.3
Enjoy the image filters.
Bibliography
397
Index
Applications two-tank, 283
angles and area, 30 motion of a mass, 49
circuit, 244 mass-spring, 251
LRC tuning, 272 projectile, 244
modulated LRC, 277 projectile in space, 72
three-loop, 155, 236 phasors, 43
two-loop, 103, 218 torque, 34, 57, 61
data modeling, 83, 171 truss
flow rate identification, 196 five-bar, 149
multiple markets, 192 six-bar, 133, 237
price, 89 two-bar, 101
radioactive decay, 177, 200 visualization, 100
real estate appraisal, 194 work
sales prediction, 173 in space, 54
United States population, 201 independent of path, 34
world population, 173, 187 on a ramp, 33
fluid flow, 238 array operations, 99
heading velocity, 25 auxiliary equation, 253
heat conduction, 286
basis vectors, 213
in a wire, 111, 123
block Gauss elimination, 231
parameter identification, 204
box product, 62
time dependent, 306
image, 90 Cauchy-Schwarz inequality, 28, 53,
contrast, 324 98
filter in 1D, 358 characteristic equation, 253
filter in 2D, 365 complex algebraic equation, 6
filters, 370 n-th root, 15
sharpen, 336, 339 complex function
inventories, 86 exponential, 14
mass-spring, 251 linear, 10, 11
damped forced, 270 quadratic, 11
dampened, 260 square root, 12
forced, 268 complex number, 3
mixing exponential form, 14
one-tank, 244 field properties, 6
three-tank, 166, 293 polar form, 4
399
400 INDEX