LinearAlgebraStudyGuide PDF

LINEAR ALGEBRA AND ITS APPLICATIONS – CHAPTER ONE - SUMMARY
Chapter 1 −4• + 5•" + 9•# = −9 , becomes:
1 −2 1 0
%0 2 −8 8&
1.1 System of Linear Equations
−4 5 9 −9
The basic form of a linear equation is:
• ! + •" !" + ⋯ + •$ !$ = %.
Where b is a real or complex constant and the
coefficients •$ are real and complex. The subscript If we write it as an augmented matrix. The
n is any positive integer. Any equation that may be notation for describing this matrix is 3 x 4, where
rearranged to the above form is a linear equation. the first number indicates the number of rows and
A set of these equations involving the same the second the number of columns.
variables is referred to as a system of linear
equations which may have a solution in the form SOLVING A LINEAR SYSTEM
of a list (& , &" , … , &$ ) that can be substituted for The procedure of solving linear systems follows
! , … , !$ respectively. For example, the linear very straight forward steps. The ultimate goal is to
system: replace the linear system with an equivalent
2! − !" + 1.5!* = 8 system (same solution set) that is easier to solve.
! − 4!* = −7 By using three basic operations: (1) Replacing one
equation of the system by the sum of itself and a
Has the solution list (5, 6.5, 3), which when multiple of another, (2) interchange two equations
substituted back into the linear system for and (3) multiply all terms in an equation by a
! , !" , !* results in a simplification of the two nonzero constant. None of these operations
equations to 8 = 8 and -7 = -7. The solution set is change the solution set of the system. These steps
the set of all possible solutions of a linear system are applicable to both the system of the linear
and two linear systems are equivalent if they have equations and the corresponding augmented
the same solution set. The solution set of a system matrix.
of linear equations may have one of the three
following properties: 1.2 Row Reductions and Echelon Forms
A leading entry of a row: Leftmost nonzero entry
1. There is no solution. -> There is a (in a nonzero row)
contradiction in the system. A rectangular matrix is in echelon form if it
2. There is exactly one solution. -> The satisfies these three properties: (1) All nonzero
equations only have one possible rows are above any rows of all zeros. (2) Each
solution. leading entry of a row is in a column to the right of
3. There are infinitely many solutions. -> The the leading entry of the row above it. (3) All entries
linear system is solved by defining one of in a column below a leading entry are zeros.
the variables as a parameter and then For a reduced echelon form, the following
going through the system of solving the properties are also satisfied: (4) The leading entry
equations. The solution will contain that (the leading entry of a row is the leftmost nonzero
variable. entry in a nonzero row) in each nonzero row is 1.
(5) Each leading 1 is the only nonzero entry in its
MATRIX NOTATION column.
A linear system may be written as a rectangular These forms of a matrix are attained by using the
matrix where each column corresponds to the methods of solving linear systems that are
value of the coefficient of one variable (this would described above. Here is an example of a matrix in
be the coefficient matrix). If another column at reduced echelon form:
1 0 0 29
the right of the matrix is added that contains the
%0 1 0 16&
values of the solutions of the equations it is called
0 0 1 3
the augmented matrix. For example:
• − 2•" + •# = 0
2•" − 8•# = 8
THEREOM: Each matrix is row equivalent to one
and only one reduced echelon matrix.
1|P a g e
•,
%=4 ⋮ 6
•3
A pivot position in a matrix is a location in that
matrix, which corresponds to a leading 1 in the
reduced echelon form of the matrix. A pivot A zero vector is a vector with all its entries as 0.
column contains a pivot position. Vectors can be used to describe a linear system as
such:
7, 89 + ⋯ + 73 8< = >
The systematic procedure to get this matrix (using
Which corresponds to the matrix:
[89 … 8< >]
the above matrix as an example) is to first reduce
the first entry of row 2 and 3 to zero, then reduce
the second entry of row 3 to zero (at this point we
SPAN OF VECTORS
The vector y is a linear combination of ?, … ?@
have the echelon form), then use the third row to
with the scalar factors 1, … 1@ when:
get the third entry in row 2 to zero. After this is
A = 1, B9 + ⋯ + 1@ BC
done, rows 2 and 3 can be used to get the second
The set of all linear combinations of ?, … ?@ ,

and third entries in the first row to zero. Make
where these vectors are in ℝ3 , is denoted by

sure to take into considering the right most
Span{?, … ?@ }. This is called the subset of ℝ3

column while doing this. At this point the reduced
spanned by ?, … ?@ . Span{?, … ?@ } is the

echelon form has been reached.
You can see the ease of use of the echelon form
and the educed echelon form when applied to an collection of all vectors that can be written in the
form of:
1, B9 + ⋯ + 1@ BC
augmented matrix of a linear system: The last row
of the echelon form pertains to the solution of one
variable, while the one above pertains to the To test whether a vector is in a span, one simply
solution of that one variable and another, etc. This has to equate the above sum with that vector and
way one can solve the whole system. Each row in see if there exists a solution.
the reduced echelon form pertains to one variable
of the system. 1.4 The Matrix Equation Ax = b
The non-reduced echelon form of an augmented A linear combination of vectors can be viewed as a
matrix can be used to determine the uniqueness of product of a matrix and a vector. A is an m x n
the solution of a linear system. Using the Existence matrix with columns 8, , … , 83 and if x is in ℝ3 ,
and Uniqueness Theorem: then the product of A and x, denoted by Ax, is the
linear combination of the columns of A using the
A linear system is consistent (it either has one corresponding entries in x as weights; that is,
unique solution or infinitely many solutions) if and
only if the rightmost column of the augmented 7,
matrix is not a pivot column – that is, if and only if EF = [G, G- … G3 ] 4 ⋮ 6 = 7, 8, + 7- G- +
an echelon form of the augmented matrix has no 73
row of the form ⋯ + 73 83 = b
[0 … 0 •] •••ℎ • •!•"#$!
If a linear system is consistent, then the solution THEOREM: If A is an m x n matrix, with columns
G, … G3 and if b is in ℝ3 , the matrix equation Ax =
set contains either (i) a unique solution, when there
are no free variables, or (ii) infinitely many
b has the same solution as the vector equation
7, 8, + ⋯ + 73 8< = H which, in turn, has the
solutions, when there is at least one free variable.
same solution set as the system of linear equations

whose augmented matrix is [89 8I … 83 H].
1.3 Vector Equations
A vector is a matrix with only one column (column
vector):
% = &(
(
)
+, where •, and •- are any real numbers. From these 3 ways of representing a linear system
*
This is a vector in ℝ- .
(matrix equation, vector equation, or system of
linear equations), the following properties of a
A vector may be multiplied by a scalar (scalar
multiplication): system can be deduced, for which if one is true all
1% = 1 &()
+ = &2()
+ , where c is the scalar.
must be true, and if one is false, all must be false.
( * 2( *
Vectors are added by adding their corresponding
THEOREM: Let A be an m x n matrix (This is a
rows.
Vectors may also be generalised to ℝ3 :
coefficient matrix of the system!). Then the
following statements are logically equivalent:
2|P a g e
a) For each b in ℝJ , the equation Ax = b has THEOREMS: An indexed set S = {K, … K@ } of

a solution. two or more vectors is linearly dependent if
b) Each b in ℝJ is a linear combination of and only if at least one of the vectors in S is a
the columns of A. linear combination of the others.
c) The columns of A span ℝJ .
d) A has a pivot position in every row. If a set contains more vectors than there are
entries in each vector, then the set is linearly
PROPERTIES OF THE MATRIX-VECTOR dependent. That is, any set {K, … K@ } in ℝ3 is
PRODUCT Ax linearly dependent if p > n.
If A is an m x n matrix, u and v are vectors in
ℝ3 , and c is a scalar, then: If a set S = {K, … K@ } in ℝ3 contains the zero
a. A(u + v) = Au + Av; vector, then the set is linearly dependent.
b. A(cu) = c(Au).
1.8 Introduction to Linear Transformations
1.5 Solution Sets of Linear Systems The notation of a linear system in terms of Ax
A system of linear equations is homogeneous = b as opposed to the vector equation form
if it can be written in the form of Ax = 0 where has the advantage that Ax = b can be viewed
A is an m x n matrix and 0 is the zero vector in as a transformation (transforming x into b).
ℝJ . This system always has one solution (x =
0), this is called the trivial solution. A A transformation T from ℝ3 to ℝJ is a rule
nontrivial solution only exists if Ax = 0 has at that assigns to each vector x in ℝ3 a Vector
least one free variable (when the solution of T(x) in ℝJ . The set ℝ3 is called the domain of
the system require a parameter to be solved). T and ℝJ is called the codomain of T. The
notation T: ℝ3 -> ℝJ indicates that the
THEOREM: Suppose the equation Ax = b is domain of T is ℝ3 and the codomain is ℝJ .
consistent for some given b, and let p be a For x in ℝ3 , the vector T(x) in ℝJ is called the
solution. Then the solution set of Ax = b is the image of x (under the action of T). The set of
set of all vectors of the form w = p + v, where all images T(x) is called the range of T.
v is any solution of the homogeneous
equation Ax = 0. A transformation T is linear if:
1.7 Linear Independence a. T(u + v) = T(u) + T(v) for all u, v in the

An indexed set of vectors {K, … K@ } in ℝ3 is domain of T;
said to be linearly independent if the vector b. T(cu) = cT(u) for all scalars c and all u in
equation the domain of T.
x, K, + ⋯ + x@ K@ = 0
has only the trivial solution. Otherwise, if From this it results that if a transformation is
there exists scalars corresponding to the x linear, then
variables such that not all scalars are zero, T(0) = 0
then the set is said to be linearly dependent. and
T(cu + dv) = cT(u) + dT(v)
This principle can be applied to matrix for all b vectors u, v in the domain of T and all
columns. The columns of a matrix are linearly scalars c, d.
independent if and only if the main equation
Ax = 0 has only the trivial solution. 1.9 The Matrix of a Linear Transformation
Every linear transformation from ℝ3 to ℝJ is
The linear dependence of a set of two vectors actually a matrix transformation x -> Ax.
can be easily tested be multiplying one vector
with a scalar and equating it to the second. THEOREM: For a transformation T from ℝ3 to
For two or more vectors: ℝJ there exists a unique matrix such that
T(x) = Ax for all x in ℝ3 .
3|P a g e
Terminology for mappings:
A mapping T: ℝ3 -> ℝJ is said to be onto ℝJ

if each b in ℝJ is the image of at least one x in
ℝ3 .
A mapping T: ℝ3 -> ℝJ is said to be one-to-

one if each b in ℝJ is the image of at most
one x in ℝ3 . This is the case if and only if the
equation T(x) = 0 has only the trivial solution.
Applying this to a matrix:

T: ℝ3 -> ℝJ is a linear transformation and A is
the standard matrix for T. Then:
1) T maps ℝ3 onto ℝJ if and only if the
columns of A span ℝJ ;
2) T is one-to-one if and only if the columns
of A are linearly independent.
4|P a g e
1. Find the reduced row echelon form of
Solution:
Solution:
Solution:
Solution:
5. Compute the rank of
Solution:
Solution:
Solution:

Solution:
9. Find a matrix with the following property, or say why you cannot have one.
Solution:
10. Find a matrix with the following property, or say why you cannot have one.
Solution:
Chapter 2 - Matrix Algebra
Two main topics that help us solve equation systems:

· Partitioned matrices
· Matrix factorization
2.1 Matrix Operations:

· Matrix ! defined by " rows and # columns has " × # dimension. The entry on the
$ %& row and ' () column is represented by *+, and is called the -$. '/ - entry of !.
Column <
* … &'7
* 3, … *35
2 33 9
54 45
:1 &54 … * 8
! = :1 * +3
8' &87+, … &
*89 +5 8 Row ;
:1 54 45 54 8
0*63 *67
… & 6, … *65 7
&' %%%%%%%%%%%%%%%&7 %%%%%%%%%%%%%%%%%&9
· If we name the columns of the matrix by : , !" , … , !# and write the matrix in
column form, we obtain:
$ = [! !" … !# ]
· The diagonal entries of a matrix are%&'' , &(( , &)) , …. They form the main diagonal of
the matrix.
· A diagonal matrix is a square matrix (in * × * dimension) whose nondiagonal
entries are zero. An example is the identity matrix whose main diagonal entries
are all “1” and the other entries are all “0”.
· If all entries of a matrix are zero, this matrix called a zero matrix.
Sums and Scalar Multiples:

$, + and%- are matrices having the same size;%. and%/ are scalars:
· $0+ =+0$
· 1$ 0 +2 0 - = $ 0 1+ 0 -2
· $03=$
· .1$ 0 +2 = .$ 0 .+
· 1. 0 /2$ = .$ 0 /$
· .1/$2 = 1./2$
o Two matrices are called equal when they have the same size and have
the same entries on the same location; they have the same columns and
same rows.
o We can sum two matrices up if only if they have the same size.
· $1+42 = 1$+24
1
o If matrix%$ is in size ? × * and matrix%+ in size%* × @,%$+ multiplication

is in size%? × @. This means that the row number of%$+ is the row
number of%$, the column number of%$+ is the columns number of%+.
$+ = [A A" … AB ] = [$A $A" … $AB ]
Each column in%$+ is the linear combination of columns in%$ using weights from the
corresponding column in%+.
o The 1;, <2 - entry of %$+ multiplication is defined as: 1$+287 = &8' C'7 0
&8( C(7 0 D 0 &89 C97
o .EF8 1$+2 = .EF8 1$2 G +
Properties of Matrix Multiplication:

· (!") = ( !)"
· (! + ") = ! + "
· (! + ") = ! + "
· #( !) = (# )! = (#!)
· $% = = $&
Here' , !'and'" are matrices with appropriate sizes and # a scalar.
Notes:
· We cannot say that ! = ! . This may be correct for some special cases but in
general ! * ! .
· If ! = ", we cannot cancel' from both sides and say ! = ".
· If ! = 0, we cannot say that either = 0 or ! = 0. This may be correct for some
special cases, but in general we cannot have this conclusion.
Power of a Matrix:
-
· = . . ….
/
1
· If / = 0, 2=2
The Transpose of a Matrix:

· The transpose of matrix' is shown by' 3 .
· If matrix' is in size'4 × 5, ' 3 is in size'5 × 4.
· The transpose of a matrix is obtained by switching the rows with columns and vice
versa.
· ( 3 )3 =
· ( + !)3 = 3 + !3
· (# )3 = # 3
· ( !)3 = !3 3 …………… (When taking the transpose of their multiplication, and
! are written in reverse order!)
2
2.2 The Inverse of a Matrix:

· To be able to take the inverse of a matrix, it should be a square matrix.
· If there is a'" matrix that provides " = $ and' " = $, we say that' is invertible
and'" is the inverse of' . The inverse of matrix' is shown as' 67 .
67 67
= $ and =$
9 :
· If we say that = 8 ?,
; <
67 7 < D:
= @A6BC 8 ?
D; 9
· If'9< D :; * 0, is invertible.
· If'9< D :; = 0,' is not invertible.
· Here, we notice that 9< D :; is the determinant of matrix' . So, we conclude that'
is invertible as long as det' is nonzero.
· If' is invertible, for each'E vector in'F& , the equation 2 = : has the solution''
2 = 67 E.
· If' is invertible, 67 'is invertible: ( 67 )67 =
· If' and'! are invertible matrices with size 5 × 5,' ! is also invertible with size
5 × 5 : ( !)67 = !67 67 …………… (When taking the inverse of their
multiplication, and ! are written in reverse order!)
· If' is invertible, 3 'is invertible: ( 3 )67 = ( 67 )3
Elementary Matrices:
· An elementary matrix is a matrix that is obtained by performing a single row
operation on an identity matrix.
· If we do an elementary row operation on a 4 × 5 matrix' , the new matrix can be
written as'G . Here, 4 × 4 matrix'G is created by doing the same row operation
on'$% .
· Every elementary matrix'G is invertible and the inverse of'G is the elementary
matrix of the same type that transforms'G back into'$.
· Matrix' is invertible if only if' is row equivalent to'$& . Also, the row operations
that reduce' to'$& also transform'$& into' 67.
An Algorithm for Finding H6I J

· Write identity matrix with compatible size next to' : [ $]
· Perform row operations to obtain the identity matrix on the left. The matrix
forming on the right side will be the inverse of' J [$ 67 ]
· This means that [ $ ] is row equivalent to [$ 67 ]. If not, we say that' does not
have an inverse.
67
· We can check if = $ to see if we have done the row operations correctly.
3
Another View of Matrix Inversion:

· The columns of $& are represented by KI , KL , … , KM . We can break the [ $ ] to
[$ 67 ] row reduction operations into'5 systems as follows:
2 = KI ,''''' 2 = KL ,''''… ,''''' 2 = KM
such that [ KI KL … KM ] = [ $]
2.3 Characterization of Inverse Matrices:

is a square matrix with dimension 5 × 5. Then, the following statements are all true or
all false N The Invertible Matrix Theorem
· is invertible.
o There is a 5 × 5'" matrix such that " = $.
o There is a 5 × 5'O matrix such that O = $.
· is row equivalent to $& .
· has 5 pivot positions.
· 2 = 0 has only the trivial solution.
· The columns of are linearly independent.
· The linear transformation 2 P 2 is one-to-one.
· 2 = E has at least one solution for each'E vector in F& .
· The columns of span F& .
· The linear transformation 2 P 2 maps F& onto F& .
3
· is invertible.
67
If ! = $, then ! = and = !67 .
Invertible Linear Transformations:

QJ F& R F& is a linear transformation and 'is the standard matrix for'Q. Q'is invertible if
is invertible. This means that'Q is invertible if there is the'S function such that:
STQ(2)U = 2
QTS(2)U = 2
With ! a linear transformation, !(") = #$% " is the function which satisfies the above
equations.
Partitioned Matrices:
We can divide a matric into divisions and name each division (block / submatrice) as #&' .
Addition and Scalar Multiplication:
within the same blocks (the blocks in the same location - (*, +) . Also, the scalar
If two matrices are the same size and partitioned in the same way, the addition is done
multiplication is done block by block.
4
Multiplication of Partitioned Matrices:

The partitions of two matrices should be conformable (should match) for block
multiplication. This means that since multiplication within two matrices is done by
multiplying the rows of the 1st matrix with the corresponding columns of the 2nd matrix,
set of - and . columns, the 2nd matrix should be divided into a set of - and . rows.
the partition of the matrices should be done in consistency: If the 1st matrix is divided into
· Partitioned matrices are multiplied in the same way row-column rule applies. The
If matrix # is in size / × 0 and matrix 1 in size 0 × 2:

block entries are taken as one scalar.
94;% (1)
·
94;6 (1)
#1 = [345% (#) 3456 (#) … 3457 (#)] 8 ?
<
94;7 (1)
= 345% (#)94;% (1) @ 3456 (#)94;6 (1) … @ 3457 (#)94;7 (1)
Inverses of Partitioned Matrices:

#%% #%6
Additional information: # = A C formed matrix is called a block upper
B #66
triangular matrix.
In order to find the inverse of a partitioned matrix, we assume that there is a matrix 1
which is the inverse of matrix # and is also partitioned with suitable block sizes. Then we
write the block equations obtained from: #1 = D.
· A block diagonal matrix is a partitioned matrix with zero block off the main
diagonal of blocks. This matrix is invertible provided that each block on the
diagonal is invertible.
2.5 Matrix Factorizations:

If # is the product of two or more matrices, we say that we can do factorization of #.
The LU Factorization:
# is a / × 0 matrix and can be row reduced to echelon form without row interchanges. We
can write # = EF where E is a / × / lower triangular matrix with 1’s as the main
diagonal entries and U is a / × 0 echelon form of #. An example is shown below:
H B B B K I I I I
# = G II H B BJ G B K I I I
I H B B B B K IJ
I I I H B B B B B
E F
5
# = EF L #" = M N E(F") = M
O = F" N EO = M P F" = O
Solve EO = M and find O.

Then, solve O = F" and find ".
·
o Both steps are easy to solve because both E and F are triangular matrices.
·
An LU Factorization Algorithm:
If possible, by doing row replacement operations, reduce # to an echelon form F.
Place entries in E such that the same sequence of row operations reduces E to D.
·
·
2.6 The Leontief Input - Output Model:

" = Q" @ R
Amount Intermediate Final
produced demand demand
Here, Q is the consumption matrix for an economy.
D" = Q" @ R
(D S Q)" = R!
" = (D S Q) R (This can be written provided that Q and R have nonnegative entries and
$%
each column sum of Q is less than 1) " is the production vector here and has nonnegative
entries.
A Formula For (T S U)$V :

(D S Q)$% W D @ Q @ Q 6 @ Q X @ Y @ Q Z where each column sum of Q is less than 1; / is the
round number of the intermediate demand. When / is sufficiently large; the formula
approaches closer to (D S Q)$% .
The Economic Importance of Entries in (T S U)$V :

They are useful in predicting how production (") changes depending on
demand (R).
·
Generally, the equation #" = M can always be written as (D S Q)" = M where

Q = D S #. In a sufficiently large system with mostly zero entries, the column sums
·
of Q can be less than 1. Then, Q Z N B. So, solving #" = M and finding #$% will be
easier.
2.7 Applications to Computer Graphics:

· Examining mathematics behind graphical images and their manipulations.
· Images consist of points connected to each other by lines or curves and the
information about how to fill in closed regions formed by the lines and curves.
6
Homogeneous Coordinates:
We can identify a point with coordinates (\, ^) in _6 with the point (\, ^, H) in _X . The
difference is that the second point is one unit above the \^-plane. So, it is said that (\, ^)
has homogeneous coordinates (\, ^, H).
Composite Transformations:
In order to move a figure on a computer screen, we need two or more basic
transformations. The composition of these transformations are related with matrix
multiplication with the use of homogeneous coordinates.
Homogeneous 3D Coordinates:
(`, a, b, c) are homogeneous coordinates for (\, ^, d) and c e B:
a b
·
\= , ^= , d=
c̀ c c
Perspective Projections:
distance from the screen. A perspective projection maps each point (\, ^, d) onto an image
A 3D object is seen as 2D on a computer screen. A person views the screen at a certain
point (\ I , ^ I , B) and so the two points and the eye position (the center of projection) are on
a line. The perspective projection can be represented by a matrix and the data on this
matrix is found by some calculations with the coordinates given.
2.8 Subspaces of _f :
A set called c in _f is a subspace for _f and:
The zero vector is in c.

For each g and h vectors in c, the sum g @ h is also in c.
·
For each g vector in c and each scalar 3, the vector 3g is in c.

·
·
Column Space and Null Space of a Matrix:

The column space of matrix # is the set Col # of all linear combinations of the
columns of #.
·
# = [iV … if ] j Col # = Span{iV , … , if }
The null space of matrix # is the set Nul # of all solutions of the equation #" = B.
The null space of a / × 0 matrix is a subspace of _f .
·
The set of all solutions of #" = B of / homogeneous linear equations in 0

·
unknowns is a subspace of _f .
·
7
Basis for a Subspace:

A linearly independent set in c that spans c is a basis for subspace c of _f . When
dealing with problems in a subspace, it is better to work with small sets that are the basis
for subspaces.
· The pivot columns of matrix # are a basis for the column space of #.
2.9 Dimension and Rank:
Coordinate Systems:
k = lMV , … , Mm n is a basis for subspace c. For every " in c:
3%
["]k = o < q is the coordinate vector of " (relative to base k ), where 3% , … , 3p are the
3p
weights.
The Dimension of a Subspace:

Dimension of a subspace c is shown by dim c; that is the number of vectors in any
basis for c.
·
The rank of a matrix # is shown by rank #; that is the dimension of the column
o The dimension of zero subspace is zero.
space of #.
·
Rank Theorem: If # has 0 columns: rank # @ dim Nul # = 0

Basis Theorem: If c is a 2-dimensional subspace; any linearly independent set of
·
2 elements c is a basis for c. Any set of p elements of c that spans c is also a basis
·
for c.
For a square matrix # in size 0 × 0 , below statements are equivalent to the
statement that # is invertible:
·
The columns of # form a basis of _f .

Col # = _ .
f
o
dim Col # = 0
o
rank # = 0
o
Nul # = {B}
o
dim Nul # = B
o
o
8
1. Find all solutions to
Solution:
We create the augmented matrix and row reduce:
Thus, the solutions are x1 = 1, x2 = 2, x3 = 4
2. Simplify, or write “undefined” if the multiplication is not valid.
Solution:
Solution:
Solution:
Solution:
Solution:
Solution:
Undefined
Solution:
Solution:
Solution:
Chapter 3 - Determinants
3.1 Introduction to Determinants:

· All matrices mentioned need to be square matrices.
· If the determinant of a matrix is non zero, then it is invertible.
!"# = $%% !"#%% & $%' !"#%' + $%( !"#%( & ) + *&1,%-. $%. !"#%.
· The determinant of a × matrix is calculated by the formula below:
= /*&1,%-0 $%0 !"#%0

02%
Here, #%0 named matrices are the submatrices obtained by closing the 1st row and
column 3 in the matrix 4# . The closed row (1st row) and column elements are
eliminated and the remaining entries form #%0 submatrices.
· 560 = *&1,6-0 !"#60 is called the cofactor of matrix 4# . So, rewriting the above
expression:
!"# = $%% 5%% + $%' 5%' + ) + $%. 5%. : This is the cofactor expansion across the 1st
row of matrix4#.
· If we want to write the cofactor expansion across the 7 89 row or 3 89 4column, here
!"# = $6% 56% + $6' 56' + ) + $6. 56.

are the general formulas, respectively:
!"# = $%0 5%0 + $'0 5'0 + ) + $.0 5.0

· Considering the factor *&1,6-0 that is in the formula of the cofactor 560 , we need to
add the +4or & sign according to the location of the entry $60 as shown below:
+ & + …
:& + & 4
+ & + 4>
; 4 4 <
· The determinant of a triangular matrix is the multiplication of the entries on the
main diagonal.
· Hint: The easiest way to calculate a determinant is by choosing the way that
necessitates the least operations. This is obtained by choosing the row or the
column with the most number of zeros. We can reach the least zero containing row
or column by row operations. The zero entries will have no contribution to the
determinant since for those entries 4$60 560 will be equal to zero; so we will need to
calculate 4$60 560 only for nonzero entries.
3.2 Properties of Determinants:

Row operations can affect the value of the determinant: (let # be the original form
of the matrix and ? the operated form of the matrix)
·
o If a multiple of a row (or column) is added to/subtracted from another row

(or column), the determinant remains the same ( !"? = !"#).
o If two rows (or columns) are interchanged, the sign of the determinant
changes ( !"? = & !"#).
1
If a row (or column) is multiplied by a factor, the determinant is also

multiplied by that factor4* !"? = A B !"#,.
o
*&1,D B EFGHIJKLM4HN S T4444when4#4is4invertible

!"# = C FOPHLM4OQ4R
UT4444444444444444444444444444444when4#4is4not4invertible
·
Here, V is the number of row interchanges, W any echelon form of # obtained by

row replacements and interchanges without scaling.
· When4 !"# = U, this means that the columns of the matrix are linearly dependent
and also the rows of the matrix are linearly dependent as well.
Rows of matrix # are the columns of matrix4#X . When the columns of matrix4#X are
linearly dependent, this means that 4#X is singular. Because of the Invertible
·
Matrix Theorem, # is singular as well.

· If two rows or two columns in a matrix are the same, or a row or a column is zero,
we can say that they are the same or zero rows/columns are linearly dependent.
So, if we obtain such cases by row operations, without any further calculations, we
can conclude that4 !"# = U.
Column Operations:
· !"#X = !"#
Determinants and Matrix Products:

· If # and ? are square matrices with the same dimensions,4 !"#? = * !"#,* !"?,
A Linearity Property of the Determinant Function:

Write matrix A in the form of column vectors with one of the vectors fixed (let us
say 3 89 vector is fixed):
·
# = [YZ … Y\]Z ^ Y\-Z … Y_ `

a is the transformation from c. to c:
a*^, = !"[YZ … Y\]Z ^ Y\-Z … Y_ `
·
· a*d^, = da*^,4444444444and444444444444a*f + g, = a*f, + a*g,
3.3 Cramer’s Rule, Volume, and Linear Transformations:
Cramer’s Rule:
When we search for the solution of4#^ = j, the entries of the solution vector ^ are found
by:
!"#6 *j,
^6 =
!"#
Here, #6 *j, means that 7 89 column of matrix4# is replaced by the vector4j. Notice that the
fraction is undefined when !"# = U is zero, which means that matrix4# should always be
invertible to avoid undefinable cases.
2
A Formula For4k]Z :
yz8|~ *•\ ,
{*7T 3, & !m"Vp4qu4#]% } = ^x = 4
yz8|
·
Where •\ is the 3 89 4column of the identity matrix
!"#6 €•\ • = *&1,6-0 !"#06 = 506
%
#]% = $ 3#
yz8|
·
Where $ 3# is the adjugate of # which is the matrix containing the cofactors in the
ascending 7T 3 order.
Determinants as Area or Volume:

For a ‚ × ‚ matrix, ƒ !"#ƒ gives the area of the parallelogram.
For a „ × „ matrix, ƒ !"#ƒ gives the volume of the parallelepiped.
·
For vectors YZ 4 and Y† which are not multiples of each other, the area of the
·
parallelogram obtained from these YZ 4 and Y† is the same as the area of the
·
parallelogram obtained from YZ 4and Y† + dYZ where d4is a scalar.

For vectors YZ T Y† and Y‡ which are not multiples of each other, the volume of the
parallelepiped obtained from these YZ T Y† and Y‡ 4is the same as the volume of the
·
parallelepiped obtained from YZ 4and Y† + dYZ and Y‡ where d is a scalar.
Linear Transformations:
a is a transformation from c' ˆ c', ‰ is a parallelogram in c' :
{$V!$4qu4a*‰,} = ƒ !"#ƒ B {$V!$4qu4‰}
·
a is a transformation from c( ˆ c(, ‰ is a parallelepiped in c( :

{Šq‹Œ•!4qu4a*‰,} = ƒ !"#ƒ B {Šq‹Œ•!4qu4‰}
·
3
1. For what value of c is there a nonzero solution to the following equation? For
that value of c, find all solutions to the equation.
Solution:
To find the value of c, recall that if det(A)≠0 then Ax=0 has exactly one solution,
x=0. If det(A)=0 then Ax=0 has infinitely many solutions.
Proceed by computing the determinant:
If c = 2, the determinant is zero, and thus there are infinitely many solutions to
the equation. All but one of them are nonzero.
Finding all solutions for c=2 is the same as finding all solutions to the matrix
equation
The first row of this equation reads x+y=0. The second row reads 2x+2y=0.
These equations are redundant. Any (x,y) such that y=−x is a solution.
That is, any point of the form (x,−x) is a solution.
2. For what values of λ are there nontrivial solutions to
For each value, find a nontrivial solution.
Solution:
To analyze a system of linear equations, it is convenient to put it into the
standard form Ax= b, where x is a vector of unknowns.
The unknowns in the linear equations are x, y, and z.
Because the right-hand side involves these unknowns, we subtract it from both
sides. The equation becomes
This matrix equation is now of the form Ax=b. Because the right-hand side, b, is
zero, the equation is homogeneous.
Recall that if (det)A≠0 then the homogeneous equation Ax=0 has exactly one
solution, x=0. If (det)A=0 then Ax=0 has infinitely many solutions.
In order for there to be a nonzero solution, det A must be zero. Hence, we seek
the values of λ for which
Evaluating the determinant, we see there are nontrivial solutions when

(1−λ)(2−λ)(3−λ)=0
That is to say, there are nontrivial solutions when λ=1 or λ=2 or λ=3.
To find a nontrivial solution for λ=1, find a solution for
The second and third rows say that y=0, z=0. The first row provides no constraint
on x. Hence, any multiple of (100)’ is a nontrivial solution for λ=1.
Similarly, a nontrivial solution to
is given by any multiple of (010)’.

Similarly, a nontrivial solution to
is given by any multiple of (001)’.

3. Are there any real values of c for which there is a nontrivial (nonzero) solution
to
Solution:
We observe that this matrix equation is homogeneous because it is of the form
Ax=0.
If (det)A≠0 then the homogenous system Ax=0 has exactly one solution, x=0. If
(det)A=0 then Ax=0 has infinitely many solutions.
Because we are seeking values of c for which there are nontrivial solutions, we
require that (det)A=0.
We identify
Computing the determinant, we get

(det)A=2+c2.
2
For any real value of c, 2+c is positive, meaning the determinant is nonzero.
Hence for any value of c, there is only one solution to the matrix equation, the
trivial solution (00)’.
4. How many solutions are there to
Solution:
We observe that this equation is a linear system of the form Ax=b with nonzero b.
Hence, it is an inhomogeneous linear system.
Recall that if (det)A≠0 then the inhomogeneous linear system Ax=b has exactly
one solution, x=A−1b. If (det)A=0 then Ax=b has either no solutions or infinitely
many solutions.
Identifying A as the matrix in the problem and computing its determinant, we
have
Hence, there are either no solutions or there are infinitely many of them.
We try to find a single solution to
by writing out each of the three equations that it corresponds to:

x+y+z=1
x+y=2
z=3
We plug the value of z into the first equation to see
x+y=-2
x+y=2
These equations are a contradiction. Thus, there are no solutions to the linear
system.
5. How many solutions are there to
If there are any, find all of them.
Solution:
Observing that the linear system is of the form Ax=b for a nonzero b, we note that
this equation is inhomogeneous. Recall that for an inhomogeneous linear system
If (det)A≠0 then Ax=b has exactly one solution, x=A−1b. If (det)A=0 then Ax=b has
either no solutions or infinitely many solutions.
We begin by computing the determinant
Hence, there are either no solutions or there are infinitely many.

We try to find a single solution to
by writing out each of the three equations it corresponds to:
x+y+z=3
x+y=2
z=1
We plug the value of z into the first equation to see
x+y=2
x+y=2
Hence a solution is any (x,y,z) that satisfies z=1 and x+y=2. These are given by
6. Write this matrix equation as a system of 3 equations. Solve for x,y,z:
Solution:
The matrix product of an n×m matrix with an m×ℓ matrix is an n×ℓ matrix.
The (i,j) entry of the matrix product AB is the dot product of the ith row of A with
the jth column of B.
We identify
The left hand side of the matrix equaion is the matrix product AB, which is a 3×1
matrix.
The first entry of AB is the product of the first row of A with the first (and only)
column of B:
(AB)1=1 x+1 y+1 z.
Similarly, the second entry of AB is the product of the second row of A with the
first column of B:
(AB)2=0 x+1 y+1 z.
Finally, the third entry of AB is the product of the third row of A with the first
column of B:
(AB)3=0 x+0 y+1 z.
Hence, we rewrite the matrix equation as the system
x+y+z=4
y+z=3
z=1
To solve for x,y,z we observe that the third equation gives us z explicitly. From
that, we could plug into the second equation to get y. Then we could plug into the
first equation to get x.
Using z=1, the second equation gives y=2.
Plugging into the first equation, we get x+2+1=4. Hence, x=1.
The solution to the matrix equation is thus
7. Find the second degree polynomial going through (−1,1),(1,3), and (2,2).
Hint: To find the coefficients of y=a+bx+cx2, set up a 3×3 matrix satisfied

by a,b,c.
Solution:
We are trying to find the function y(x)=a+bx+cx2 such that y(−1)=1, y(1)=3, and
y(2)=2.
Let's plug the three values x into y and see what that says about a, b, and c.
Plugging the x values into the function we get
Simplifying, we get
We can write this linear system as the matrix equation

This equation is of the form Ax=b where
If A is a square, invertible matrix, the solution to Ax=b is given by x=A−1b.

We can compute that
Hence,
Hence, the polynomial through the points (−1,1),(1,3), and (2,2) is
8. Write the following system as a matrix equation for x,y,z:
Solution:
Recall that a matrix equation is of the form Ax=b, where A is a matrix of
constants, x is a column vector of unknowns, and b is a column vector of
constants.
In this problem, our unknowns are x,y,z, which we put into the vector x:
In order to put the system of equations into the form of a matrix equation, we
move all unknowns to the left hand side:
y+z=4
2x−y−z=0
Recall that the ith entry of the matrix-vector product Ax is the dot product of the
ith row of A with the only column of x.
We identify that y+z is the dot product of (0,1,1) with (x,y,z).
Similarly, 2x−y−z is the dot product of (2,−1,−1) with (x,y,z).
Putting the two left hand sides into a column vector, we can write
.
Hence, the system of linear equations can be written as
9. Find A−1 and use it to solve Ax=b, where
Solution:
To find the inverse of a 3×3 matrix,
1. Compute the minors of each element
2. Negate every other element, according to a checkerboard pattern
3. Take the transpose
4. Divide by the determinant of the original matrix
The minor of the (i,j)th entry of a matrix A is the determinant of the submatrix
obtained by removing the ith row and the jth column of A.
For example, the minor of the 1,1 entry is .
The minor of the 1,2 entry is .

Repeating this process, the minors of all entries are
Recall that the determinant of a 2×2 matrix is = ad−bc.

Computing all the determinants, the matrix of minors is
.
Next we negate every other entry, according to the pattern
.
The matrix of minors becomes
.
Transposing, we get
.
In order to divide by the determinant of A, we must first compute it. We can
compute that det(A)=6.
Hence,
To solve the linear system, recall that if A is square and invertible, then the
solution to Ax=b is x=A−1b.
Computing, we have
.
My matrix multiplication we get
.
10. Solve by matrix inversion:
Solution:
We observe that this matrix equation is in the standard form Ax=b, where
.
−1
Recall that if A is square and invertible, the solution to Ax=b is x=A b.
Hence, we need to find the inverse of A.
The inverse of a 2×2 matrix is given by swapping the diagonal entries, negating
the off-diagonal entries, and dividing by the determinant:
The determinant of A is 2 16−3 10=2.
Hence
The solution to the linear system is given by x=A−1b:
Computing the matrix multiplication, we get

Chapter 4 - Vector Spaces
4.1 Vector Spaces and Subspaces:

A vector space is the set ! composed of vectors. The operations between vectors are as below:
· Sum of two vectors " and # is shown as " + #.

· "+# = #+"
· (" + #) + $ = " + (# + $)
· "+% = "
· " + (&") = %
· The multiplication of " by the scalar ' is shown as '" that is also in the set !.
· '(" + #) = '" + '#
· (' + *)" = '" + *"
· '(*") = ('*)"
· 1" = "
Where ", #, $ vectors and ', * are scalars.
Subspaces:
If - is a subset of !, then it should satisfy the properties:
· The zero vector of ! is in - subset.

· - is closed under vector addition: For all " and # vectors in -, " + # is also in -.
· - is closed under vector multiplication by scalars: For all " vectors in - and scalars ',
'" is also in -.
A Subset Spanned by a Set:

Linear combination: sum of any scalar multiples of vectors
Span.#/ , … , #0 2: set of all vectors that can be written as linear combinations of #/ , … , #0 .
· Span.#/ , … , #0 2 is a subspace of ! provided that #/ , … , #0 are in the vector space !.

· Span.#/ , … , #0 2 is called the subspace spanned/generated by #/ , … , #0 .
· If - is a subspace of !, the spanning/generating set for - is the set #/ , … , #0 and so
- = Span.#/ , … , #0 2.
1
4.2 Null Spaces, Column Spaces, and Linear Transformations:
The Null Space of a Matrix:
· Nul 4 is the null space of the 5 × 6 matrix 4; it is the set of all solutions of the
equation 47 = %:
Nul 4 = {7: 7 where 7 8 9; and 47 = %}
· The set of all solutions (null space) of the system 47 = % (5 linear equations with 6
unknowns) is a subspace of 9;.
An Explicit Description of Nul <:
· Do row reduction operations on the augmented matrix [4 % ] in order to write the

variables in terms of free variables.
· Write the solution vector 7 in the form of the linear combination of vectors; in this
expression, the weights are the free variables. The vectors in this expression form a
spanning set for Nul 4.
· The spanning set produced here is linearly independent since the free variables are the
weights on the spanning vectors.
· If Nul 4 does not contain zero vectors, the number of vectors in the spanning set is the
same as the number of free variables in the equation 47 = %.
The Column Space of a Matrix:

· Col 4 is the column space of the 5 × 6 matrix 4; is the set of all linear combinations of
the columns of 4 which is shown as {>/ … >; }:
Col 4 = Span{>/ … >; }
· Span{>/ … >; } is a subspace of 9? when the matrix has the dimensions 5 × 6.
· Col 4 = {@: @ = 47 for some 7 in 9; }
· If the equation 47 = @ has a solution for each @ in 9? , then the column space of the
5 × 6 matrix 4 is all of 9? .
The Contrast between Nul < and Col <:

· If the matrix is not square, vectors in Nul 4 and Col 4 are found in different vector
spaces.
· If the matrix is square, Nul 4 and Col 4 have the zero vector in common. Sometimes
some nonzero vectors may belong to both Nul 4 and Col 4.
2
Kernel and Range of a Linear Transformation:

For a 5 × 6 matrix 4, the comparison between Nul 4 and Col 4 is shown in the below table:
Nul < Col <

· Nul 4 is a subspace of 9 .;
· Col 4 is a subspace of 9? .
· We are given that 47 = % which means that · Since Col 4 is the column space of the
the vectors in Nul 4 should satisfy the given matrix, the data of Col 4 is
equation, we need to solve for them. directly given to us.
· In order to find Nul 4, we need to perform · Col 4 is the set of all linear
row operations on [4 %]. combinations of the columns of 4, so it is
easy to find the vectors in it since we
have the columns of the matrix.
· No relationship is found between the entries · Each column of matrix 4 is in Col 4, so
in the matrix 4 and Nul 4. there is a relationship between Col 4
and matrix 4.
· A # vector in Nul 4 satisfies the · A # vector in Col 4 is such a vector that
equation 4# = %. 47 = # is a consistent equation.
· We can check if a # vector is in Nul 4 by · We can check if a # vector is in Col 4 by
checking if it satisfies 4# = %. row operations on [4 #], it takes some
time to check.
· Nul 4 = {%} if only if 47 = % has only the · Col 4 = 9? if only if 47 = @ has a
trivial solution that means we can find the solution for every @ in 9? .
solution for the equation.
· Nul 4 = {%} if only if 7 A 47 linear · Col 4 = 9? if only if 7 A 47 linear
transformation is one-to-one. transformation maps 9; onto 9? .
B is a linear transformation defined from vector space ! to vector space C:
· B(" + #) = B(") + B(#)

· B('") = 'B(")
· The kernel (null space) of B is the set of all " vectors which satisfy B(") = %.
4.3 Linearly Independent Sets; Bases:

· We say that the set .#/ , … , #0 2 is linearly independent if the equation
'D #/ + 'E #F + G + 'H #0 = % has only the trivial solution; 'D = I, … , 'H = I.
· We say that the set .#/ , … , #0 2 is linearly dependent if the equation
'D #/ + 'E #F + G + 'H #0 = % has a nontrivial solution; some of the weights 'D , … , 'H are
not equal to zero.
· 'D #/ + 'E #F + G + 'H #0 = % is called a linear dependence relation among the vectors
#/ , … , #0 .
· A set .#/ , … , #0 2 with two or more vectors (#/ J %) is linearly dependent if only if #K
(L M 1) is a linear combination of the vectors #/ , … , #KN/ .
· If - is a subspace of a vector space !, O = .@/ , … , @0 2 in ! is a basis for - provided that:
3
o Set O is linearly independent

o - = Span.@/ , … , @0 2
The Spanning Set Theorem:

By P = .#/ , … , #0 2, a set in ! and - = Span.@/ , … @0 2:
o If #Q vector in P is a linear combination of the other vectors in P , the set

excluding #Q formed from P still spans -.
o If - J {%}, some subset in P is a basis for -.
Bases for Nul < and Col <:

· A basis for Nul 4 is obtained from the linearly independent set solution, with Nul 4
containing nonzero vectors, found after doing row reduction operations on [4 % ].
· A basis for Col 4 is obtained from the pivot columns of the matrix 4.
Two Views of a Basis:

· If we do not stop deleting vectors from a spanning set when the set becomes linearly
independent and continue deleting other vectors, then the set will no longer be a linear
combination of the remaining vectors and so the smaller set will no longer span !. A
basis is the possible smallest spanning set.
· Also, a basis is the possible largest linearly independent set. If P is a basis for ! and if
P is enlarged by a vector $ from ! ; the new set cannot be linearly independent.
Because, P spans ! and so $ is a linear combination of the elements in P.
4.4 Coordinate Systems:

· O = .@/ , … , @0 2 is a basis for vector space !. For every 7 in !, there is a set of scalars
'D , … , 'R satisfying:
7 = 'D #/ + 'E #F + G + 'R #;
· O = .@/ , … , @0 2 is a basis for ! and 7 is in !. The coordinates of 7 depending on the
basis are the weights 'D , … , 'R shown as:
'D
7 = 'D @/ + 'E @F + G + 'R @; [7]O = T U V
'R
A Graphical Interpretation of Coordinates:

There is one-to-one mapping from the points in the set into 9; in a coordinate system on a set.
4
Coordinates in 9; :
If there is a basis O for 9; given, the O-coordinate vector of 7 is found by
'D
7 = 'D @/ + 'E @F + G + 'R @; [7]O = T U V. For a basis O = {@/ , … @; }, let
'R
where
WO = [@/ @F … @; ]. Then, 7 = 'D @/ + 'E @F + G + 'R @; = WO [7]O. Here, WO is the change-of-

coordinates matrix from O to 9;. Also, by Invertible Matrix Theorem:
WO ND 7 = [7]O
The Coordinate Mapping:

If O = {@/ , … , @; } is a basis for the vector space !, the coordinate mapping 7 X [7]O is one-to-
one linear transformation from ! into 9;. This one-to-one linear transformation is called an
isomorphism.
4.5 The Dimension of a Vector Space:

· If O = {@/ , … , @; } is a basis for the vector space !, any set in ! having more than 6
vectors must be linearly dependent.
· If a basis of vector space ! contains 6 vectors, every basis of ! must have exactly 6
vectors.
· If ! is spanned by a finite set, ! is finite-dimensional.
· dim ! represents the dimension of !; it is the number of vectors in a basis for !.
· The dimension of zero vector space is zero.
· If ! is not spanned by a finite set, ! is infinite-dimensional.
Subspaces of Finite-Dimensional Space:

· ! is a finite-dimensional vector space and - is a subspace of ! . Then, - is finite
dimensional and dim - Y dim !. An independent set in - can be expanded to a basis
for -.
· With Z \ 1 , ! is a Z -dimensional vector space. Every independent set containing
Z elements in ! is a basis for !. Every set containing Z elements spanning ! is also a
basis for !.
The Dimensions of Nul < and Col <:

· The dimension of Col 4 is the number of pivot columns in matrix 4.
· The dimension of Nul 4 is the number of free variables of the equation 47 = %.
5
4.6 Rank:
This section provides us the relationships between the rows and the columns of a matrix. The
maximum number of linearly independent columns of a 5 × 6 matrix 4 and the maximum
number of linearly independent columns in 4^ (which are the rows in 4) are equal.
The Row Space:

In a 5 × 6 matrix, each row has 6 entries and can be identified with a vector in 9;, which
means that is a subspace of 9;. The set of all linear combinations of the row vectors is the row
space of matrix 4, shown by Row 4. We also know that rows of A are the columns of 4^ , so we
can say that Row 4 = Col 4 (pivot positions in 4).
· The row spaces of matrices 4 and _ are the same provided that 4 and _ are row
equivalent. If _ is in echelon form, the rows of _ that are nonzero form a basis for the
row space of 4 as well as for that of _.
The Rank Theorem:

· rank 4: the dimension of column space of 4
· rank 4 = dimension of the column space of 5 × 6 matrix 4 = dimension of the row
space of 5 × 6 matrix 4
· rank 4 + dim Nul 4 = 6
Application to Systems of Equations:

The Rank Theorem is strongly used to obtain information about the linear equation systems.
Rank and The Invertible Matrix Theorem:

A is a 6 × 6 matrix. If 4 is an invertible matrix, then:
· The columns of 4 form a basis for 9;

· Col 4 = 9;
· dim Col 4 = 6
· rank 4 = 6
· Nul 4 = {%}
· dim Nul 4 = I
4.7 Change of Basis:

In some problems, we need to change a given basis O to a new basis ` in order to obtain the
solution. Each vector in basis O is changed to a new `-coordinate vector.
· O = {@/ , … , @; } and ` = {b/ , … , b; } are the bases of vector space !. There is a unique 6 ×
6 matrix W that provides:
W
[7]` = [7]
`cO O
6
W
are the `-coordinate vectors in the basis O:
`cO
The columns of
W
= [[@/ ]` [@F ]` … [@; ]` ]
`cO
Change of Basis in 9; :
If O = {@/ , … , @; } is a basis and g = {h/ , … , h; } is the standard basis in 9;, [@/ ]g = @/, [@F ]g =
@F ,..., [@; ]g = @; . So,
W [@F ]g
= [[@/ ]g … [@; ]g ]
gcO
And so;
WO = [@/ @F … @; ]
4.8 Applications to Difference Equations:
Discrete-Time Signals:
A signal in vector space j of discrete-time signals is a function defined only on the integers
and written as a sequence of numbers {kl }. When a process begins at a specific time, we can
write a signal as a sequence of the form (kq , kD , kE , … ) . The kl terms for u v I are omitted.
Linear Independence in the Space j of Signals:
Let us consider a set of only three signals in j ; {wl }, {yl } and {zl } . They are linearly
independent provided that 'D "Q + 'E #Q + '| $Q = I (for all u) is satisfied when 'D = 'E = '| = I.
'D "Q~/ + 'E #Q~/ + '| $Q~/ = I for all u
'D "Q~F + 'E #Q~F + '| $Q~F = I for all u
So;
"Q #Q $Q 'D I
T"Q~/ #Q~/ $Q~/ V T'E V = TIV for all u
"Q~F #Q~F $Q~F '| I
The coefficient matrix above is called the Casorati matrix of signals. The determinant of
this matrix is called the Casoratian. If the Casorati matrix is invertible for at least one value
of u, then 'D = 'E = '| = I which means that the three signals are linearly independent.
Linear Difference Equations:

•q kl~R + •D kl~RND + G + •RND kl~D + •R kl = €l for all u, where •q and •R are nonzero and €l is
a signal. The equation above is called a linear difference equation (linear recurrence relation)
of order 6. If {€l } is the zero sequence, the equation is homogeneous; if not, the equation is
nonhomogeneous.
7
Solution Sets of Linear Difference Equations:
· If {€l } and •R J I are given; kl~R + •D kl~RND + G + •RND kl~D + •R kl = €l has a unique
solution as long as kq , kD , kE , … kRND are specified.
· The set - of all solutions to the equation kl~R + •D kl~RND + G + •RND kl~D + •R kl = I is
a 6-dimensional vector space.
Nonhomogeneous Equations:
The general solution of the nonhomogeneous linear equation kl~R + •D kl~RND + G +
•RND kl~D + •R kl = €l can be written as one particular solution of kl~R + •D kl~RND + G +
•RND kl~D + •R kl = €l plus an arbitrary linear combination of a fundamental set of solutions of
the related homogeneous equation kl~R + •D kl~RND + G + •RND kl~D + •R kl = I.
Reduction to Systems of First-Order Equations:

7Q~/ = 47Q for all u : 7Q vectors are in 9; and 4 is a square matrix with dimensions 6 × 6.
4.9 Applications to Markov Chains:

The probability vector is a vector with all entries adding up to 1 with all nonnegative entries.
A stochastic matrix is a square matrix with probability vectors as its columns. A Markov
chain is the combination of the stochastic matrix W and the probability vectors 7% , 7/ , 7F , …:
7/ = W7% , 7F = W7/ , 7• = W7F , …
‚ 7Q~/ = W7Q for u = I,1,ƒ …
7Q is also called a state vector.
Predicting the Distant Future:

We calculate 7/ , 7F , … by using 7Q~/ = W7Q with the stochastic matrix and the initial vector 7% .
After a while, we see that each entry of the vectors approach to the same values; the changes
within consecutive vectors get smaller. So, we reach to a steady state vector.
Steady-State Vectors:
W is a stochastic matrix with „ the steady-state (equilibrium) vector for W; then:
W„ = „
Every stochastic matrix has its steady-state vector.
· As u ‚ †, Markov chain {7Q } converges to „.

· 7/ = W7% , 7F = W7/ = WE 7% ….. So, in general: 7Q = Wl 7% for u = I,1, …
8
Questions 1-8 are true or false questions. If the statement is true, prove it. If
false, give a counterexample.
1. If A is a 2 × 2 matrix such that A(Ax) = 0 for all x R2, then A is the zero
matrix.
Solution:
False. If A(Ax) = 0 for all x, then the column space of A and the nullspace of A
must be the same space. In particular, consider the matrix
Hence, A(Ax) = 0 for all x, but A 6= 0, so A gives a counterexample to the

statement.
2. A system of 3 equations in 4 unknowns can never have a unique solution.
Solution:
True. We can realize such a system of equations as a single matrix equation Ax
= b, where A is a 3 × 4 matrix. Hence, rank(A) ≤ 3, so the dimension of the
nullspace of A is at least 1: dim nul(A) = 4 − rank(A) ≥ 4 − 3 = 1. Hence, there
must be at least one free variable in the system, meaning that, if the system is
solvable at all, it must have an infinite number of solutions.
3. If V is a vector space and S is a finite set of vectors in V , then some subset of

S forms a basis for V.
Solution:
False. Let V = R2, which is clearly a vector space, and let S be the singleton set {
[ 1 0 ] }. The single element of S does not span R2, so no subset of S can be a
basis for R2. Hence, this provides a counterexample to the statement
4. Suppose A is an m × n matrix such that Ax = b can be solved for any choice of
b Rm. Then the columns of A form a basis for Rm.
Solution:
False. Consider the matrix
.
Then A is already in reduced echelon form and clearly has 2 pivots, so rank(A) =
2. This implies that dim col(A) = 2, so the column space of A consists of all of R2.
Thus, the equation Ax = b can be solved for any b R2 (since any b is in col(A)).
However, the columns of A are clearly not linearly independent (no set containing
the zero vector can be linearly independent), so they cannot form a basis for R2.
A related but true statement would be the following: “Suppose A is an m × n
matrix such that Ax = b can be solved for any choice of b Rm. Then some
subset of the columns of A forms a basis for Rm.”
5. Given 3 equations in 4 unknowns, each describes a hyperplane in R4. If the

system of those 3 equations is consistent, then the intersection of the
hyperplanes contains a line.
Solution:
True. Translating the system of equations into a matrix equation Ax = b, the
nullspace of A must be at least one-dimensional, so the solution-space must be
at least one-dimensional. Since the solution space of the matrix equation
corresponds to the intersection of the hyperplanes, that intersection must be at
least one-dimensional, meaning it must contain a line.
6. If A is a symmetric matrix (i.e. A = AT), then A is invertible.
Solution:
False. Consider the symmetric matrix
.
Then A only has rank 1, meaning that A cannot be invertible, so this gives a
counterexample to the statement
7. If m < n and A is an m × n matrix such that Ax = b has a solution for all b R m,

then there exists z R m such that Ax = z has infinitely many solutions.
Solution:
True. The fact that Ax = b has a solution for all b Rm means that the column
space of A is equal to all of R m. Hence,
rank(A) = dim col(A) = m.
Since
dim nul(A) = n − rank(A) = n – m
and since m < n, we have that the nullspace of A has some positive dimension.
Since the nullspace of A consists precisely of those x Rn such that Ax = 0, this
equation has infinitely many solutions. Thus, letting z = 0, we see that the
statement is true.
8. The set of polynomials of degree ≤ 5 forms a vector space.
Solution:
True. You should check that the set of polynomials of degree ≤ 5 satisfies all the
rules for being a vector space. The important facts are this space is closed under
addition and scalar multiplication.
For questions 9 and 10, determine whether the given subset is a subspace of the
given vector space. Explain your answer.
9. Vector space: R4
Subset: vectors of the form
Solution:
Yes, this is a subspace. If we take two vectors in the subset, say
then their sum

is also in the subset, so this set is closed under addition.
Moreover, if c R, then
is in the set, so this set is closed under scalar multiplication. Thus, the set is
closed under both addition and scalar multiplication, and so is a subspace.
10. Vector Space: All polynomials.

Subset: The quadratic (i.e. degree 2) polynomials.
Solution:
No, this is not a subspace. To see that it is not closed under addition, notice that
if f(t) = t2 and g(t) = −t2, then f and g are both in the set of quadratic polynomials,
but, since (f + g)(t) = f(t) + g(t) = t2 + (−t2) = 0, the sum f + g is not a quadratic
polynomial.
Chapter 5 - Eigenvalues and Eigenvectors
5.1 Eigenvalues and Eigenvectors

Aim: Using eigenvalues and eigenvectors in solving differential equation systems.
If ! = "! is provided, here " is a scalar value and is the eigenvalue if only the equation
has a nontrivial solution, and ! is the eigenvector related to the eigenvalue#".
If we are asked if a vector ! is an eigenvector of a matrix or not; we need to try

! = "! and find out if ! satisfies the equation with any eigenvalue " or not. If it
·
does, we say that vector ! is an eigenvector of matrix# .

· If we are asked if a scalar " is an eigenvalue of a matrix# or not; we first need to
find out if the equation has a nontrivial solution or not. For this, we need to follow
! = "!
the steps below:
! $ "! = 0
( $ "%)! = 0 ... Here, if the columns of the matrix $ "% are linearly dependent,
we say that the equation ! = "! has a nontrivial solution. Then, we say that " is
an eigenvalue of matrix A.
0
If we have the eigenvalue(s) of a matrix, we find the corresponding eigenvectors by
performing row operations by inserting & ' as the last row next to the matrix#
·
0
# $ 7% . Row reductions can be used to find the eigenvectors, but not the
All eigenvectors including the zero vector as the solutions for ( $ "%)! = 0 form
eigenvalues.
·
the eigenspace of corresponding to#".
Hints on Eigenvalues and Eigenvectors:

· An eigenvector must be nonzero, but an eigenvalue may be zero.
· The elements on the main diagonal of a triangular matrix are the eigenvalues of
If the eigenvectors *+ , *- , … , *. are the eigenvectors related to the eigenvalues

that matrix.
"/ , "1 , …#"2 of the 3 × 3 matrix# , the set {*+ , *- , … , *. } is linearly independent.
·
A 3 × 3 square matrix has 3 eigenvalues, but they are not always distinct. When
they are all distinct, there will be 3 corresponding eigenvectors. If the matrix has
·
4 distinct eigenvalues, then there will be minimum 4 eigenvectors.
First-Order Difference Equations:

!5 = "5 !6 is a solution for !58/ = !5 ####(9 = 0, :, ;, … )
1
5.2 The Characteristic Equation:
Determinants:
· If the determinant of a matrix is not equal to zero and if “zero” is not an eigenvalue
of the matrix, then that matrix is invertible.
· If you perform row reduction operations on matrix# , the determinant of the last
form of the matrix will be the same as the determinant of matrix# . Note: Row
interchange changes the sign of the determinant! Also, a row scaling results in
($:)2 B CDEFGHIJK#FL Q ,####when# #is#invertible

factoring the determinant with the same scale.
>?@ = A DMNFJK#MO#P
0,###############################when# #is#not#invertible
·
Here, R is the number of row interchanges, S any echelon form of obtained by
det T = >?@ B >?@T

row replacements and interchanges without scaling.
>?@ U = det
·
·
· The determinant of a triangular matrix is the multiplication of the values on the
det( $ "%) = 0 is the characteristic equation of 3 × 3 matrix . The equation

main diagonal.
·
This polynomial is of 3VW degree and its roots are the eigenvalues of the matrix.
obtained by this operation is called the characteristic polynomial of matrix# .
There may be multiplicities within these roots.

· A matrix can have complex eigenvalues as well as real ones.
Similarity:
If both matrices and T are 3 × 3 dimensioned and if there is an invertible matrix
providing X Y/ X = T or = XTXY/ or#Z Y/ TZ = ; then we say that and T are
·
Similarity transformation is done by changing into#XY/ X.

similar.
·
· Similar matrices have the same characteristic polynomial which leads to exactly
same eigenvalues.
said to have the same eigenvalues. If is row equivalent to #T , there is the

· Row equivalent matrices cannot be said to be similar matrices, therefore cannot be
T = [ #equality where [ is an invertible matrix.
Application to Dynamical Systems:

Analyzing the long-term behavior of a dynamical system !58/ = !5 with given initial
value#!6 .
First, find the eigenvalues ( "/ , "1 , … , "\ ) by using the characteristic equation
obtained from#det( $ "%) = 0.
·
Find the eigenvectors (*+ , *- , … , *] ) corresponding to the eigenvalues.

Write !6 in terms of the eigenfactors:
·
·
2
^/
^1
!6 = ^/ *+ _ ^1 *- _ ` _ ^\ *] = a*+ *- ` *] c f g j
^\
·
Here, 3 is the dimension of the square matrix .

Find the values of the weights#^/ , ^1 , … , ^\ .
Starting from the given value#!6 ; by using# ! = "!, evaluate !/ , !1 #and so on until
·
·
you find the general formula for#!5 .
· The obtained formula gives the solution of the difference equation#!58/ = !5 .
5.3 Diagonalization:
If = XkXY/ where X is an invertible matrix and k is a diagonal matrix, then is
said to be diagonalizable because 5 = Xk 5 XY/ is obtainable.
·
· A 3 × 3 matrix is diagonalizable if only if it has 3 linearly independent
If a 3 × 3 matrix has 3 distinct eigenvalues, the matrix is certainly diagonalizable.

eigenvectors.
However, it is not necessary to find 3 eigenvalues to be diagonalizable.

·
The columns of matrix X should be these 3 linearly independent

eigenvectors#(*+ , *- , … , *] ).
·
· The diagonal entries of matrix k should be the eigenvalues ("/ , "1 , … , "\ ), written
in the order of the corresponding eigenvectors are written in matrix#X as the
columns.
Diagonalizing Matrices:
Find the corresponding 3 linearly independent eigenvectors of matrix# .

· Find the eigenvalues of matrix# .
Write matrix X using the eigenvectors as the columns.

·
Write matrix k using the eigenvalues as the diagonal entries. Eigenvalues and
·
·
Verify if# X = Xk (This is the equivalent of## = XkXY/ ). If it is verified, then X is

eigenvectors should be written in the same order corresponding each other.
·
invertible.
Matrices Whose Eigenvalues Are Not Distinct:

· Determine the eigenvalues (all including the multiplicities) of the matrix.
Check if the eigenvectors set is linearly independent. If it is, then matrix X is

· Find the corresponding eigenvectors.
invertible and matrix is diagonalizable:# = XkXY/ .

·
3
5.4 Eigenvectors and Linear Transformations:
The Matrix of a Linear Transformation:

m : n-dimensioned vector space #p : m-dimensioned vector space #q : linear
transformation from m to p. u and x: bases for m and#p, respectively.
·
R/
! = R/ y+ _ R1 y- _ ` _ R\ y] with the coordinate vector: azc| = ~ g •
R\
·
) ) )
q(!) = q(R/ y+ _ R1 y- _ ` _ R\ y] = R/ q(y+ _ R1 q(y- _ ` _ R\ q(y] )
aq(!)c€ = R/ aq(y+ )c€ _ R1 aq(y- )c€ _ ` _ R\ aq(y] )c€
·
aq(!)c€ = •a!c| and • = aaq(y+ )c€ aq(y- )c€ … aq(y] )c€ c ‚ the matrix for T
·
relative to the bases u and#x.

·
Linear Transformations from V into V:

p = m and x = #u.
•: matrix for q relative to u.
·
aq(!)c| = aqc| a!c|

·
·
Linear Transformations on#ƒ]
If we know that = XkXY/ where k is a diagonal matrix with 3 × 3 dimensions and u is

formed by the columns of X and is the basis for ƒ\ , then k is the u -matrix for the
transformation ! „ !.
Similarity of Matrix Representations:
If a transformation is defined from ƒ\ to ƒ\ by q(!) = !, and if u is a basis in ƒ\ , then

u-matrix for q is said to be similar to .
5.5 Complex Eigenvalues:

Complex roots of the characteristic equation give critical information about the matrix.
The corresponding eigenvector is also called complex eigenvector.
Real and Imaginary Parts of Vectors:

The real part of a vector ! is shown by †?#!, the imaginary part is shown by %‡#!.
4
Eigenvalues and Eigenvectors of a Real Matrix That Acts on#ˆ] :

· ! = Š!
‰‰‰‰ ‹= ! ‰‰‰ = "Š!
‹ = "! ‹ where is a 3 × 3 dimensioned matrix with real entries.
• $Ž
The complex eigenvalues of are found in conjugate pairs.
The eigenvalues of the matrix Œ = & ' are " = • ± Ž•. R = •"• = ‘•1 _ Ž 1 . So
Ž •
•“R $Ž“R R 0 ^•–— $–•3—
·
we can organize as: Œ = R ’ ”=& '=’ ” where — which is

Ž“R •“R 0 R –•3— ^•–—
called the argument of " = • _ Ž• is the angle lying between the _z-axis and the
ray connecting the origin to the point to (•, Ž).
If is a matrix with real entries, (†?#!) = †?# ! and (%‡#!) = %‡# !.
For a real ; × ; matrix , with a complex eigenvalue " = • $ Ž• with corresponding
·
• $Ž
eigenvalue *, X = a†?#* %‡#*c and Œ = & '; so = XkX Y/.
·
Ž •
5.6 Discrete Dynamical Systems:
behavior of the dynamic systems described by the difference equation#!58/ = !5 .

By the use of eigenvalues and eigenvectors, we have information about the long-term
Eigenvector decomposition: !6 = ^/ *+ _ ^1 *- _ ` _ ^\ *]
!/ = !˜ = ^/ *+ _ ^1 *- _ ` _ ^\ *] = !6 = ^/ "/ *+ _ ^1 "1 *- _ ` _ ^\ "\ *]
·
General form obtained: !5 = ^/ ("/ )5 *+ _ ^1 ("1 )5 *- _ ` _ ^\ ("\ )5 *] (9 = 0, :, ;, … )

·
We can examine the behavior of the dynamic system for large values of 9 by
·
making the assumption#9 ‚ ™.

·
· The trajectory of the dynamic system is obtained by graphing#!˜ , !+ , … , !].
Change of Variable
A new sequence: š› = XY/ !› or#!› = Xš› . So; since#!58/ = !5 ,#Xš›8+ = Xš› =
(XkXY/ )Xš› = Xkš› . Shortly;#Xš›8+ = Xkš› . Multiplying both sides by XY/ on the
·
left, we obtain#š›8+ = kš› .
5.6 Applications to Differential Equations:
z/ (@) z/ œ (@)
!œ (@)
= !(@) is a linear equation where#!(@) = ~ g •, ! (@) = ~ g •, and =
œ
z\ (@) z\ œ (@)
·
•// … •/\
~ g # g •
•\/ … •\\
· If the derivative of a function only depends on the function itself, then the system
If we know the value of !˜ , then it is an initial value problem in which !(@) is to be

is called decoupled.
calculated by knowing !(0) = !˜ and !œ = !.

·
A general solution of linear combination of functions in the form !(@) = *? •V is

suggested. Here, * is a fixed vector.
·
5
Decoupling a Dynamical System:

*+ ? •ž V , #*- ? •Ÿ V , … , *] ? • V are the eigenfunctions of matrix# . ¡/ , ¡1 , … , ¡\ are linearly
independent eigenvectors. X = a*+ … *] c . k is the diagonal matrix with the
·
eigenvalues "/ , "1 , … , "\ #as diagonal entries. So, = XkXY/ .

Change of variable: š(@) = XY/ !(@) or !(@) = Xš(@)
!œ = ! = ¢V (Xš) = (Xš) = (XkX ! )"#
= "$#. So, "#% = "$#&&& ' &&& #% = $#:
¢
·
*! +,-) 2! 3 … 3 *! ,-)
·
3 2. & / *. ,-)
( . +,-)1 = (
*
/ / & 4 3 1( / 1
*0 +,-) 3 … 3 20 *0 ,-)
·
The differential system is decoupled: the derivative of #5 depend on only #5 .

#%6 = 20 #6 &&&& 7 &&&& #6 ,-) = 80 9 :; > . So;
?,-) = "#,-) = 8! @A 9 :B > C 8. @D 9 :E > C F C 80 @6 9 :; >
·
Complex Eigenvalues:
9 G = H C I C I. C IK C F C I0 C F
! ! !
.J KJ 0J
·
9 :> = H C ,2-) C .J ,2-). C KJ ,2-)K C F C 0J ,2-)0 C F
! ! !
eigenvalue 2 = L C MN&&&& 7 &&&& 9 ,OPQR)> = 9 O> S 9 RQ> = 9 O> ,8TUM- C NUNVM-)

·
·
5.8 Iterative Estimates for Eigenvalues:

The Power Method:
· There should be a strictly larger eigenvalue in absolute value compared to the
WX ? = 8! ,2! )X @A C 8. ,2. )X @D C F C 80 ,20 )X @6 &&&& ' &&&& ,Y 5W ? = 8! @A C 8. ,:E )X @D C

others. The power method aims to find it.
A X :
A) B
·
: X
FC 80 Z :; [ @6
B
So, ,2! ) W ? ' 8! @A &LU&\ 7 ]

X X
· Choose an initial vector ?^ with largest entry 1.

Calculate&W?5 .
Find the largest entry in absolute value in&W?5 , name it as _X .
·
Calculate&?5PA = W?5 .
!
·
` a
·
· For all ?^ values chosen, _X approaches the dominant eigenvalue and ?5
approaches the corresponding eigenvector.
The Inverse Power Method:

The inverse power method aims to find an eigenvalue 2 of W.
Choose a b estimation that is sufficiently close to 2.
·
Choose an initial vector ?^ with largest entry 1.

·
Solve ,W c bd)#5 = ?5 .
·
Find the largest entry in absolute value in&#5 , name it as _X .

·
·
6
Calculate eX = b C .
!
` a
·
Calculate ?5PA = # .
!
`a 5
·
· For all ?^ values chosen, eX approaches the eigenvalue 2&and ?5
approaches the corresponding eigenvector.
7
1. Find the eigenvalues and eigenvectors of
Solution:
2. Find the eigenvalues and eigenvectors of
Solution:
3. Find the eigenvalues and eigenvectors of A and the stated power of A
Solution:
4. Obtain a cubic equation for the eigenvalues of the matrix\

and prove that the matrices
have the same eigenvalue equations as A.
Solution:
All matrices have the characteristic equation
5. Show that if A has eigenvalues λ1, λ2, . . . , λn then Am (where m is a positive

integer) has eigenvalues λm1 , λm2 , . . . λmn.
Solution:
We can diagonalise A by using the matrix of eigenvectors, L
D = LAL-1
If we rewrite this my multiplying it by L from the right, and L−1 from the left then
A = L-1DL
6. Express the quadratic form x2 + 16xy − 11y2 = 1 in matrix notation. By

calculating the eigenvalues and eigenvectors of the matrix. Write down the
equation of the conic referred to the new axes along the directions of the
eigenvectors.
Solution:
7. If
show that S is an orthogonal matrix. Hence show that if
then SPST is a diagonal matrix. Without further calculation give the eigenvalues
of P, and its corresponding eigenvectors.
Solution:
If S is orthogonal then SST = I. The diagonal elements should be the eigenvalues
4, −2. The rows of S can be read off to give the corresponding eigenvectors
8. Obtain eigenvalues and eigenvectors for
Solution:
Solution:
Solution:
Chapter – 6: Orthogonality and Least Squares
6.1 Inner Product, Length and Orthogonality
The Inner Product:
Consider two vectors u and v inside Rn, then u and v are regarded as n×1 matrices with the
transpose uT being a 1×n matrix and the matrix product uTv being a 1×1 matrix is represented as
a single real number (without brackets). This number obtained from uTv is known as the inner
product of u and v and is often written as u.v and sometimes referred to as dot product. If,
!% 0%
$ &, $0& ,
# +
= ! # ' + !-./!0 = ! ##0' ++
#(+ #(+
" )* "0) *
then the inner product of u and v is:
0%
$0& ,
# +
[ % & ' 1 ) ] #0' + = % 0% 2! & 0& 2! ' 0' 2!1 2! ) 0)
#(+
"0) *
Theorem – 1:
Considering vectors u, v and w present in Rn with c being a scalar quantity,
-3 43 5 = 53 4
63 74 2 583 9 = 43 9 2 53 9
:3 7:483 5 = :743 58 = 43 7:58

/3 43 4 < >!-./!43 4 = >!?@!-./!A.BC!?@!4 = >
From the above given properties, it can be further proved that,
7:% 4D 2 :& 4& 2 1 2 :E 4E 83 9 = ! :% 74D 3 98 2 ! 1 2 :E 74F 3 98
Length of a Vector:
For any v belonging to Rn with quantities v1, v2, … vn the length is a non-negative scalar
quantity ǁvǁ defined as;
GHG = IH3 H = Jv%& 2 v&& 2 1 2 v)& !-./!GHG& = H3 H
Also, for any scalar quantity c, the length of cv is c times the length of v, i.e.
G:HG = |:|GHG
A vector having the length of 1 is termed as a unit vector, by dividing a vector v by the scalar
quantity ǁvǁ, we get a unit vector u whose length is 1 and it is in the same direction as that of
vector v. This process is termed as normalizing a vector.
Distance in Rn:
For any two vectors, u and v belonging to Rn, the distance between the vectors is represented as
dist(u,v) and is equal to the length of the vector u-v.
dist7KL H8 = GK M HG
Orthogonal Vectors:
Two vectors are termed as orthogonal vectors if they are perpendicular to each other when
represented geometrically, this condition arises if and only if u.v = 0.
Theorem – 2:
Two vectors are orthogonal to each other if and only if,
GK 2 HG = !"! + !#!
If a vector z is orthogonal to each vector in a subspace W of Rn, then z is said to be
orthogonal to W and the set of all vectors z which are orthogonal to W is known as the
orthogonal complement of W and is denoted by W┴. A vector x can only belong to W┴ if and
only if it is orthogonal to each and every vector in W. Also, W┴ is a subspace of Rn.
Theorem – 3:
For a m×n matrix suppose A, the orthogonal complement of row space of A is the null space of
A and the orthogonal complement of column space of A is the null space of AT.
(Row$A)% = Nul$A$and$(Col$A)% = Nul$A&
If, u and v are two non-zero vectors, then the angle Ø between the geometric
representations of the two vectors can be found by the formula:
". # = !"!$!#!$'*, -
6.2 Orthogonal Sets:
A set of vectors (u1, u2, … un) belonging to Rn is an orthogonal set if each pair of distinct
vectors in the set is orthogonal to each other , i.e., if$"/ . "0 = 1$whenever$i 2 j.
Theorem – 4:
If S = (u1, u2, … un) is a set of orthogonal vectors which are non-zero and belong to Rn, then S is
linearly independent and hence is a basis for the subspace which is spanned by S.
Theorem – 5:
Let (u1, u2, … , un) be an orthogonal basis for a subspace W of Rn. For each y in W, the weights
in the linear combination
y = c1u1 + … + cnun
4."5
are given by c3 = $(7 = 89 … 9 :)
"6 ."5
Orthogonal Projection:
Consider two vectors u and y belonging to Rn. Now, y can be divided into two parts such that
one part is a multiple of u and the other part is orthogonal to u. This can be written as,
4= <+>
Where ŷ is termed as orthogonal projection of y onto u and the term z is termed as component
of y orthogonal to u.
!. "
= "
". "
Orthonormal Set:
A set (u1, u2, … , un) is said to be an orthonormal set if it is an orthogonal set which contains of
only unit vectors.

Theorem – 6:
A matrix U of m × n has orthonormal columns if and only if UTU = I.
Theorem – 7:
For an m×n matrix say U having orthonormal columns, and u and v belong to Rn, then:
· #U$# = #$#
· (U"). (U$) = ". $
· (U"). (U$) = %&if&and&only&if&". $ = %
An orthogonal matrix is a square matrix which is invertible such that U-1 = UT.
6.3 Orthogonal Projections:
The orthogonal projection of any point in R2 onto a line passing through the origin
has an important analogue in the Rn. Suppose there is a vector y along with a
subspace W in Rn then, there exists a vector ŷ in the subspace such that ŷ is the
unique vector in W closest to y. These two properties of the vector ŷ provide the
solution to finding the least squares solutions for linear systems. The terms in the
sum for y can be grouped into two parts such that y can be represented as:
Here, z1 and z2 represent the sum of identities present inside y.
Theorem – 8:
Orthogonal Decomposition Theorem:
Considering a subspace of Rn say W, then each y in Rn can be written in the unique form
Here, ŷ is in W and z is in W┴. If (u1, u2,… , up) is any orthogonal basis of W then,
And z = y – ŷ.
Properties of Orthogonal Projections:
Considering (u1, u2, … , up) is an orthogonal basis for W and y as a vector belonging to the
subspace W. Then,
Theorem – 9:
The best approximation theorem:
Considering a subspace of Rn say W and let a vector y be present in Rn and let ŷ be the
orthogonal projection of y onto W. Then the closest point in graphical representation to y is ŷ and
for any other v present in W and distinct from ŷ.

Theorem – 10:
Considering an orthonormal basis such as (u1, u2, … , up) belonging to the subspace W of Rn,
then we can say that,
In case U = [u1, u2, … up], then
Theorem – 11:
6.4 The Gram-Schmidt Process:
For a given basis say {x1,…,xn} for anon zero subspace W or Rn, define
In such a case, {v1,…, vn} is an orthogonal basis for W. In addition to this,
Span {v1,…vk} = Span {x1,…,xk} for 1 ≤ k ≤ p
Theorem – 12:
The QR Factorization:
Considering an m×n matrix say A which has linearly independent columns, then A can be
factored in the form of A = QR, where is an m×n matrix whose columns and an orthogonal
basis for Col A and R is an n×n upper triangular invertible matrix which has positive entries on
the diagonal.
6.5 Least-Squares Problems:
Considering A is an m×n matrix and b belongs to Rm, then a least-squares solution of Ay = b is
an ŷ belonging to Rn such that
! " #$ % & % ! " '(
For all the y belonging to Rn.
Each least squares solution of Ay = b satisfies the equation given below:
') '( = '* +
The equation given above is used to represent a system of equations known as the normal
equations for Ay = b and a solution for this is often denoted by ŷ.
Theorem – 13:
The set of least squares solutions for Ay = b is coincidental with the nonempty set of solutions of
the normal equations given by ATAy = ATb.
Theorem – 14:
Consideting A as a matrix of the order m×n. The following facts are subsequently true:
· The equation given by Ay = b has a unique least squares solution for every b belonging
to Rm.
· The columns present in A are linearly independent.
· The matrix given by ATA is invertible.
The above statements give rise to the least squares solution ŷ which is given by:
= (!" !)#$ !" %
Theorem – 15:
Take a matrix of the order m×n which has linearly independent columns say A and let A = QR
be a QR factorization of A. Then in that case, for each b in Rm, the equation given by Ax = b has
a unique least-squares solution which is given by:
Ŷ = R-1QTb
6.6 Applications to linear models:

Instead of writing Ax = b, one can write the equation as Xß = y and refer to X as the design
matrix, ß as the parameter vector, and y as the observation vector.
Least-Squares Lines:
The simplest relation between any two given variables say x and y is a linear equation denoted
by y = ß0 + ß1x. Suppose the experimental data has a close relevance to an equation for a line
then, the value yj is the observed value for y and ß0 + ß1xj the predicted value which is
determined by the line. And the difference between y-value and a predicted y-value is known as
residual.
The least-squares line is the line y = ß0 + ß1x which minimizes the sum of the squares of the
residuals. This line is also known as the line of regression of y on x, since any errors present in
the data are assumed to be just in the y-coordinates. The coefficients ß0, ß1 of the line are known
as regression coefficients.
In case the data points were on the line, the parameters ß0 and ß1 would satisfy the equations of
the line and can be represented as:
The General Linear Model:
In some applications, it is required to trace the data points graphically with something other than
a straight line. Usually, to deal with this situation, a residual vector is introduced and the
equation is written as:
An equation which is written in this form is referred to as a linear model. Once X and y are
determined, the objective becomes to minimize the length of E, which generates a least squares
solution of Xß = y.
Inner Product Spaces:

An inner product on a vector space V is a function that, to each pair of vectors u and v in V
associates a real number (u,v) and satisfies the following axioms for all u, v, w in V and all
scalars c:
A vector space with an inner product is called an inner product space.
Lengths, Distances, and Orthogonality:
Let V be an inner product space, with the inner product denoted by <u,v>. We can define the
length or norm of a vector v to be the scalar:&'& = *< ', ' >
A unit vector is a vector which has the length 1. The distance between u and vector v is ǁu-vǁ
and the vectors u and v are orthogonal if <u,v> = 0.
Theorem – 16:
The Cauchy-Schwarz Inequality:
For all u,v in V,
|<u,v>| ≤ ǁuǁ ǁvǁ
Theorem – 17:
The Triangle Inequality
For All u,v in V,
ǁ u + v ǁ ≤ ǁuǁ + ǁvǁ
An inner product for C[a,b]:
The inner product on Pn is given by:
This can also be written in a simplified manner as:
6.8 Applications for Inner Product Spaces:

The normal equation for the least-squares solution is:
In case a function is approximated by a curve which can be given using the quadratic equation,
y = ß 0 + ß 1 t + ß2 t 2 ,
the coefficient ß2 may not provide us with the desired values which are in par with the quadratic
trend in the data, since it might not be “independent” in a statistical sense from the other ßi. In
order to generate what is known as trend analysis of the given data we can introduce an inner
product on the space Pn. For p, q in Pn, define
<p, q> = p(t0)q(t0) + … + p(tn)q(tn)
Let p0, p1, p2, p3 denote an orthogonal basis for the subspace P3 belonging to Pn, obtained by
applying the Gram-Schmidt process for the polynomials which are 1, t, t2, and t3. Let ŷ be the
orthogonal projection as per the given inner product of y on P3 say
ŷ = c0p0 + c1p1 + c2p2 + c3p3
In such a case, ŷ is known as the cubic trend function, and c0, … , c3 are the trend coefficients
of the data. The coefficient c1 measures the linear trend, c2 the quadratic trend, and c3 the cubic
trend. It turns out that if the data have certain properties, these coefficients are statistically
independent.
Fourier Series (calculus required)
Continous functions can be approximated by using linear combinations of sine and cosine
finctions. Let us consider the functions on 0 ≤ t ≤ 2Π, and any function in C[0,2Π] can be
approximated as closely as the desired be a function of the form
a0/2 + a1cos t + … + an cos nt + b1 sin t + … + bn sin nt
for a largevalue of n. The function above is also known as a trigonometric polynomial. If an and
bn do not have zeroes as their value the polynomial is of the order n. The connection within the
trigonometric polynomials and other functions in C[0, 2Π] depends upon the given condition
stating for any n ≥ 1 the set:
{1, cos t , cos 2t, … , cos nt, sin t, sin 2t, … , sin nt}
Is orthogonal with respect to the inner product:

!
!, "# = $ % !(&)"(&)'&
"
The norm of the difference in between f and a Fourier approximation is known as the mean
square error in the approximation. For an expression given below, we can write the Fourier
series which is the expression for f(t) for f on [0,2Π].
.
&"
#($) = % + % ' (&* cos ,$ + % -* sin ,$)
2
*/0
Problems 1 and 2 are related to the following problem statement. Let L be the
line thru the origin in R2 that is parallel to the vector
1. Find the standard matrix of the orthogonal projection onto L.
Solution:
We know that projL: R2 → R2 is a linear transformation, so we can find the
columns of the standard matrix by plugging in the standard basis vectors. We
have the formula
2. Find the point on L which is closest to the point (7, 1) and find the point on L
which is closest to the point (−3, 5).
Solution:
projL(v) is the closest point to v in L.
So, (3, 4) is closest point on the line to (7, 1) and (33/25, 44/25) is the closest
point on the line to (−3, 5).
Problems 3 and 4 are related to the following problem statement. Let L be the
line thru the origin in R3 that is parallel to the vector
3. Find the standard matrix of the orthogonal projection onto L.
Solution:
We can also use the formula which says that A = UUT where U is the matrix
whose columns form an orthonormal basis of L. In the case of a line, an
orthonormal basis is just a vector with length one in the direction of L. To find this
we just normalize u. We first need to normalize the vector. The length
4. Find the point on L which is closest to the point (1, 0, 0).
Solution:
Problems 5-7 are related to the following problem statement. Let x1 = [1 2 1]T
and x2 = [3 0 3]T and let P be the plane thru the origin spanned by x1 and x2.
5. Find an orthonormal basis of P.
Solution:
6. Find the standard matrix of the orthogonal projection onto P.
Solution:
We let U be the matrix whose columns forms the orthonormal basis.
7. Find the point on P which is closest to the point (1, 0, 0).
Solution:
Problems 8-10 are related to the following problem statement. Let x1 = [1 0 0]T
and x2 = [1 1 1]T and let P be the plane thru the origin spanned by x1 and x2.
8. Find an orthonormal basis of P.
Solution:
We use the Gram-Schmidt process to find an orthogonal basis.
9. Find the standard matrix of the orthogonal projection onto P.
Solution:
We let U be the matrix whose columns forms the orthonormal basis
10. Find the point on P which is closest to the point (0, 0, 1).
Solution:
Chapter – 7: Symmetric Matrices and Quadratic Forms
7.1 – Diagonalization of Symmetric Matrices:
Symmetric Matrix:
A symmetric matrix say A is said to be a symmetric matrix if and only if A = AT and for this
property to be true, the matrix A should always be a square matrix in order to be a symmetric
matrix. While the values on the diagonal can remain any random values, the other values always
occur in pairs at opposite positions aligned as per the main diagonal.
Theorem – 1:
For a matrix A which is symmetrical, any two given eigenvectors from two different eigenspaces
will be orthogonal to each other.
A square matrix of say n × n order is called as orthogonally diagonalizable only if there is an
orthogonal matrix P such that the condition P-1 = PT holds true along with a diagonal matrix D
such that the given condition is true:
A = PDPT = PDP-1
For the completion of such a diagonalization we must first calculate the n eigenvectors which are
linearly independent and orthonormal. In case A is orthogonally diagonalizable as in the above
equation then the below equation holds true also stating the fact that A is symmetric,
AT = (PDPT) T = PTT DT PT = PDPT = A
Theorem – 2:
A square matrix let us suppose A of order n × n is orthogonally diagonalizable if and only if A is
a symmetric matrix.
Theorem – 3:
The Spectral Theorem for Symmetric Matrices:
The set consisting of eigenvalues of A can also be sometimes termed as the spectrum of A and
the following properties regarding the eigenvectors make this theorem the spectral theorem. A
symmetric matrix A of the order n × n has the following properties:
1. Counting the multiplicities, the number of real eigenvalues in A is n.
2. Each eigenvalue λ has an eigenspace whose dimensions are equal to λ as a root of the
characteristic equation.
3. Since the eigenvectors which correspond to different eigenvalues are orthogonal it is also
true that eigenspaces are mutually orthogonal.
4. A is orthogonally diagonalizable.
Spectral Decomposition:
Let us suppose that A = PDP-1 where the columns of P are orthogonal eigenvectors u1, … ,un for
a and the corresponding eigenvalues are inside the diagonal matrix D as λ1, … , λn. Now as it is
given P-1 = PT,
%" & ' !-" !-"

= !"! = [!" , … , !# ] $ ( ) ( * + ( . = / [%" !" & %# !# ] + ( .
' & %# !-# !-#
Thus we can also write the equation:

A = λ1u1uT1 + λ2u2uT2 + … + λnunuTn
The above expression for A is also known as the spectral decomposition of A as it breaks up the
matrix into small parts which are determined by the eigenvalues of A. Every matrix ujuTj is a
projection matrix because for each x in Rn, the vector denoted by (ujTj)x is the orthogonal
projection of x onto the subspace which is spanned by uj.
7.2 – Quadratic Forms:
A function say Q which is defined over Rn and the general value of that value at a vector x in Rn
can be generated through the help of an equation which states Q(x) = xTAx, and where A is an n
× n symmetric matrix then Q is also termed as a quadratic form on Rn. The matrix known as A
is known as the matrix of the quadratic form.
Change of Variable in a Quadratic Form:
An equation to change the variable in an equation say x with another variable say y the change
of variable equation comes into place which can be denoted by:
X = P y or can be equally written as y = P-1x
in the above equation, P is an invertible matrix and y belongs to Rn. Here y is the coordinate
vector of x which is a basis of Rn which is determined using the columns present in the matrix of
P.
If the change of variable is turned into a quadratic form xTAx in that case,
xTAx = (Py)TA(Py) = yTPTAPy = yT(PTAP) y
and the new matrix of the quadratic form is PTAP.
Theorem – 4:
The Principal Axes Theorem
For any symmetric matrix of the order n× n say A then there is an orthogonal change in variable,
x = Py such that the quadratic form xTAx is turned into a quadratic form yTDy without having
any cross-product term. The columns belonging to P are known as the principal axes of the
quadratic form xTAx. The vector y is the coordinate vector of x relative to the orthonormal basis
of the Rn which is given by these principal axes.
Geometric View of the Principal axes:
Suppose Q(x) = xTAx, where A is an invertible and symmetric matrix of the order 2 × 2. Taking
C to be a constant, all x in R2 satisfy the equation:
xT Ax = c
The matrix so formed is either an ellipse or a hyperbola as per the equation and if the equation is
not in standardized form, then the ellipse or hyperbola is rotated in either direction until it is as
per the equation.
Classifying Quadratic Forms:
A quadratic form Q is said to be:

1. Positive definite in case Q(x) > 0 for all x ≠ 0,
2. Negative definite in case Q(x) < 0 for all x ≠ 0,
3. Indefinite for cases where Q(x) has both positive as well as negative values.
4. Positive semidefinite is used for cases where Q(x) ≥ 0 for all values of x
5. Negative semidefinite is used in cases where Q(x) ≤ 0 for all values of x.
Theorem – 5:
Quadratic Forms and Eigenvalues:
Suppose A is an n × n symmetric matrix. Then a quadratic form xTA x is said to be
· Positive definite if and only if the eigenvalues belonging to A are all positive,
· Negative definite if and only if the eigenvalues which belong to A are all negative, or
· They are called indefinite when A has both positive as well as negative eigenvalues.
A positive definite matrix A is a symmetric matrix according to which the quadratic form xTAx
is positive definite. Similar derivation can also be done for a negative part also.
7.3 Constrained Optimization:
The need for having a unit vector x in Rn can be stated in a variety of similarly efficient ways as
given below:
ǁ x ǁ = 1, ǁ x ǁ2 = 1, xTx = 1
one of the commonly used versions (of xTx = 1) in applications is x12 + x22 + … + xn2 = 1
Theorem – 6:
Considering a symmetric matrix say A, it has the following values m and M respectively as the
least eigenvalue and the greatest eigenvalue λ1 of A. The values m and M are defined as:
m = min {xTAx : ǁ x ǁ = 1}, M = max {xTAx : ǁ x ǁ = 1}
The value of xTAx changes to M and m when x is a unit eigenvector say u1 which corresponds to
M and m respectively.
Theorem – 7:
Let A be a symmetric matrix and let λ1 be its greatest value and u1 be a unit eigenvector
corresponding to λ1 then the greatest value of xTAx is subjected to the restrictions,
xTx = 1, xTu1 = 0
this is the second greatest eigenvalue λ2 and this maximum is obtained when x is an eigenvector
u2 which corresponds to λ2.
Theorem – 8:
Supposing A to be a symmetric matrix of the order n × n along with an orthogonal
diagonalization done as A = PDP-1, where the entries on the diagonal of D are arranged in such a
manner that λ1 ≥ λ2 ≥ … ≥ λn and the places in which the columns of P are corresponding unit
eigenvectors u1, … , un. Then for k = 2, …, n, the maximum value of xTAx when subjected to
the constraints:
xTx = 1, xTu1 = 0, … xTuk-1 = 0
is the eigenvalue λk and this maximum value is attained at x = uk.

7.4 The Singular Value Decomposition:
A matrix say A of the order m × n can be factorized in the form A = QDP-1 and this type of
factorization is known as singular value decomposition is a quite useful matrix factorization
which is applied in linear algebra.
The singular values of an m × n matrix:
Let A be an m × n matrix. Then ATA is symmetric and also is orthogonally diagonalizable. Let
{v1, …, vn} be an orthonormal basis for Rn which consists of eigenvectors given by ATA, and let
λ1, …, λn be the associated eigenvalues of ATA. In that case, for 1 ≤ i ≤ n,
ǁAviǁ2 = (Avi)TAvi = viTATAvi
= viT(λivi)
= λi
The eigenvalues of ATA are all non-negative and can be easily arranged in the form:
λ1 ≥ λ2 ≥ ,…, ≥ λn ≥ 0
The singular values of A are the square roots of the eigenvalues of ATA.
Theorem – 9:
Assuming {v1, … , vn} is an orthonormal basis for Rn which has eigenvectors defined as ATA,
and these eigenvectors are arranged in such a manner that the corresponding eigenvalues given
by ATA adhere to the fact λ1 ≥ … ≥ λn, and assuming A to have r values which are non-zero
singular values then, {Av1, … , Avr} is an orthogonal basis for Col A as well as rank A = r.
Theorem – 10:
The Singular Value Decomposition:
Supposing A is an m × n order matrix which has rank r. Then there exists an m × n matrix Σ
according to which the diagonal entries in D are the first r singular values of A and there exists
an m × m orthogonal matrix also which can be given by U along with a n × n orthogonal matrix
V such that
A = U Σ VT
The columns of U in decomposition are termed as left singular vectors and the columns of the
matrix V are termed as right singular vectors.
Theorem – 11:
The Invertible Matrix Theorem:
Supposing A is a matrix of the order n × n then the following statements each prove that A is an
invertible matrix:
· (Col A)┴ = {0}
· (Nul A)┴ = Rn
· Row A = Rn
· A has n nonzero singular values

The above given factorization of A is known as a reduced singular decomposition of A. Since
the diagonal values present in D are not equal to zero, the matrix D is invertible, the following
matrix is known as the pseudoinverse of A:
A+ = VrD-1UrT
7.5 Applications to image processing and statistics:
Mean and Covariance
In order to ready the principal component analysis, suppose [X1 … XN] is a matrix of
observations of the order p × N. Then, the sample mean, M, of the observation vectors X1, … ,
XN is given by
For k = 1,…, N let
The columns of the p × N matrix
Have a sample mean and B is said to be in mean deviation form. The (sample) covariance
matrix is the p × p matrix S which is defined by the equation given below and also S is a matrix
of the form positive semidefinite:

For j = 1,… , p the diagonal entry sjj in S is called the variance of xj. The total variance of the
given data is the sum of the variances which are given on the diagonal of matrix S. As in general,
the sum of the diagonal terms in a square matrix S is known as the trace of the matrix, given by
tr(S). Thus
{total variance} = tr(S)
The value for sij belonging to the vector S such that i ≠ j is known as the covariance of xi and xj.
Principle Component Analysis:
Let us assume that a matrix given by [X1 … XN] is already in mean deviation form and the aim
for the principal component analysis is to get an orthogonal p × p matrix P = [u1 … up] that is
used in determining a change of variable given by, X = PY, or

With the property that new variables given by y1, … , yp are uncorrelated and are also arranged
in order of decreasing variance. The orthogonal change of variable X = PY can be deduced that
each observation vector receives a “new name” Yk such that Xk = PYk.

1. Let A be a square n × n matrix whose rows are orthonormal. Prove that the
columns of A are orthonormal.
Solution:
Since the rows of A are orthonormal AAT = I and hence ATAAT = AT. Since AT is
nonsingular it has an inverse (AT)−1 . Thus ATAAT(AT)−1 = AT(AT)−1 implying that
ATA = I, i.e., the columns of A are orthonormal.
2. Show that maximizing xTuuT(1−x) subject to xi {0, 1} is equivalent to

partitioning the coordinates of u into two subsets where the sum of the elements
in both subsets are equal.
Solution:
xTuuT(1−x) can be written as the product of two scalars (xTu)(uT(1 − x)) . The first
scalar is the sum of the coordinates of u corresponding to the subset S and the
second scalar is the sum of the complementary coordinates of u. To maximize
the product, one partitions the coordinates of u so that the two sums are as
equally as possible. Given the subset determined by the maximization, check if
xTu=uT(1 − x).
Questions 3 and 4 are related to the following problem statement. Let x1, x2, . . . ,
xn be n points in d-dimensional space and let X be the n×d matrix whose rows
are the n points. Suppose we know only the matrix D of pairwise distances
between points and not the coordinates of the points themselves. The xij are not
unique since any translation, rotation, or reflection of the coordinate system
leaves the distances invariant. Fix the origin of the coordinate system so that the
centroid of the set of points is at the origin.
3. Show that the elements of XTX are given by
Solution:
Since the centroid of the set of points is at the origin of the coordinate axes, ∑i=1
xij = 0.
Summing over j gives
Rearranging and substituting for xiTxi and xjTxj yields
Note that is D is the matrix of pairwise squared distances, then
and
are the averages of the square of the elements of the ith row, the square of the
elements of the jth column and all squared distances respectively.
4. Describe an algorithm for determining the matrix X whose rows are the xi.
Solution:
Having constructed XTX we can use an eigenvalue decomposition to determine
the coordinate matrix X. Clearly XTX is symmetric and if the distances come from
a set of n points in a d-dimensional space XTX will be positive definite and of rank
d. Thus we can decompose XTX as XTX = VTσV where the first d eigenvalues are
positive and the remainder are zero. Since the XTX = VTσ1/2σ1/2V and thus the
coordinates are given by X = VTσ1/2.
5 0
5. Let D = " #. Compute D2 and D3. In general, what is Dk, where k is a
0 4
positive integer?
Solution:
6 $1
6. Let A = " . Find a formula for Ak given that A = PDP−1 where " =
2 3
1 1 5 0 2 +1
# ,$ = # , %&'(")* = #
1 2 0 4 +1 1
Solution:
A square matrix A is said to be diagonalizable if A is similar to a diagonal matrix,

i.e. if A = PDP−1 where P is invertible and D is a diagonal matrix. When is A
diagonalizable? (The answer lies in examining the eigenvalues and eigenvectors
of A.)
7. Diagonalize the following matrix, if possible.
Solution:
8. Diagonalize the following matrix, if possible.
Solution:
Eigenvalues: −2 and 2 (each with multiplicity 2).
9. If possible, diagonalize the matrix and find an orthogonal basis in which it has
diagonal form:
Solution:
The characteristic polynomial is
(All roots are integers, so they can be found by trial and error, among the divisors
of 27.) The corresponding homogeneous systems for eigenvectors and their
solutions are:
Thus, the matrix can be diagonalized, and it has diagonal form in the basis {u1,
u2, u3}. Since A is symmetric, u1 is orthogonal to u2, u3, and we only need to
orthogonalize u2, u3. (By the way, the fact that A is symmetric also tells us that it
is diagonalizable, i.e., we must find three independent eigenvectors!) Applying
GramSchmidt to {u2, u3}, we replace u3 with [−1 2 1]T. Finally, the matrix has
diagonal form
10. Let A be a square matrix with integral entries. Prove that A−1 exists and has
integral entries if and only if det A = ±1.
Solution:
Clearly, the determinant of an integral matrix is an integer (as it is obtained from
the entries of the matrix by addition and multiplication only). Thus, for the only if
part it suffices to notice that det A det A−1 = det(AA−1) = det I = 1. Since the
product of two integers is 1, they must both be ±1. The if part follows from the
formula A−1 = (1/ det A) adj A and the fact that, if A is integral, so is adj A (as its
entries are determinants of integral matrices, see above).
Chapter – 8: The Geometry of vector Spaces
8.1: Affine Combinations
Say there are given vectors, v1, v2, …, vp which belong to Rn and scalars c1, c2, c3, … , cp an
affine combination of v1, v2, … , vp is a linear combination given by c1v1 + … + cpvp in such a
manner that the weights satisfy the condition c1 + … + cp = 1.
The set consisting of all affine combinations of points which are there in a set S is known as the
affine hull of S and is denoted by aff S.
Theorem – 1:
Any point y in Rn is an affine combination of v1, … , vp belonging to Rn if and only if y – v1 is a
linear combination of the translated points which are v2 – v1, …, vp – v1.
A set S is said to be affine in case p,q ϵ S implies that (1 – t)p + tq ϵ S for every real number t.
Theorem – 2:
A given set S is affine if and only if every affine combination of points of S belongs to S. Which
can be also understood as S is affine if and only if S = aff S.
Definition:
A translate of any given set say S in R n by a vector p is the set S + p = {s + p : s ϵ S}2. A flat in
Rn is a translate belonging to a subspace of Rn. Two flats are said to be parallel in case one of
the flats is a translate of the other. The dimension of a flat is also the dimension of the
corresponding parallel subspace and the dimension of a set S, symbolified as dim S, gives the
dimension for the smallest flat containing S. A line in Rn is a flat of dimension 1 whereas a
hyperplane in Rn is a flat of dimension n – 1.
Theorem – 3:
A non empty set say S is affine if and only if it is also a flat.
Definition:
#
For v belonging to Rn, the standard homogenous form of v is denoted by = ! " $ such that they
1
belong to Rn+1.
Theorem – 4:
Any given point let us say y in Rn is said to be an affine combination of v1, …, vp in Rn if and
only if the homogenous form of y is in Span {ṽ1. …, ṽp}. As a matter of fact, y = c1v1 + … + cpvp
along with c1 + … + cp = 1, if and only if ỹ = c1ṽ1 + … + cpṽp.
8.2 Affine Independence:
Definition:
An indexed set of points say {v1, …, vp} in Rn can be affinely dependent only in case there are
real numbers say c1 , …, cp which are not all zero in such a manner that c1 + … + cp = 0 and also
c1v1 + … + cpvp = 0 otherwise the set is said to be affinely independent.
Theorem – 5:
Let us say that S = {v1, …, vp}belonging to Rn where p ≥ 2 and S is an indexed set. In such a
case the given statements are all true at once:

· S is affinely dependent.
· One of the given points belonging to S is an affine combination of the other given points
belonging to S.
· The set given by {v2 – v1, …, vp – v1} ϵ Rn is linearly dependent.
· The set {ṽ1, …, ṽp} of homogenous forms in Rn+1 is said to be linearly dependent.
Barycentric Coordinates:
Theorem – 6:
Let us suppose an affinely independent set given by S = {v1, …, vk} in Rn. After that each p in
aff S has a unique representation as an affine combination of v1, …, vk. Which means that for
each and every p there exists a unique set of scalars c1 + … + ck = 1 and p = c1v1 + … + ckvk.
Definition:
Let us suppose S = {v1, …, vk} is an affinely independent set, then for each point p belonging to
aff S, the coefficients c1, …, cp of p as in p = c1v1 + … + ckvk are known as the barycentric
coordinates of p.
8.3 Convex Combinations:
Definition:
A convex combination of points say v1, v2, …, vk belonging to Rn is a linear combination given
by the equation c1v1 + c2v2 + … + ckvk in such a way that c1 + c2 + … + ck = 1 and ci ≥ 0 for all i.
Also, the set of all convex combinations of points in a set S is called the convex hull of S
denoted by conv S.
Definition:
A set say S is said to be convex if for each p,q ϵ S, the line segment pq is contained in S.
Theorem – 7:
A set say S is said to be convex if and only if every convex combination of points of S lies in S.
Which means that S is convex only when S = conv S.
Theorem – 8:
Let be a random collection of convex sets. Then is convex and for any
collection of affine sets which is given by then the value is affine.
Theorem – 9:
For any set say S, the convex hull of thei given set S is the place where all the convex sets meet
which are known to have S.
Theorem – 10:
(Caratheodory) In case S is a nonempty subset which belongs to Rn, then every single point in
conv S can be shown in the form of a convex combination of n+1 or lesser points from S.
8.4 – Hyperplanes:
A linear functional over Rn is a linear transformation f from Rn into R and for each scalar
quantity say d belonging to R, the symbol [f : d] gives the set for all x which are in Rn at which
the value of f becomes d. That is,
[ f : d ] is the set {x ϵ Rn : f(x) = d}

The zero functional is a form of transformation in such a manner that f(x) = 0 for all the x in Rn
and all other linear functions belonging to Rn are said to be nonzero.
Theorem – 11:
A subset H of Rn is said to be a hyperplane if and only if the condition H = [f : d] is true for any
nonzero linear functional f and some scalar quantity d belonging to R. This concludes that for H
to be a hyperplane, there exists a nonzero vector given be n along with a real number say d such
that H = {x : n.x = d}.
The convex hull of an open set is open whereas the convex hull in case of a compact set is
compact only.
Definition:
The hyperplane given by H = [f : d] separates two sets given by A and B now, if any one of the
given conditions is true:
· f(A) ≤ d and f(B) ≥ d, or
· f(A) ≥ d and f(B) ≤ d.
In the conditions given if in case all the weak inequalities are replaced by strict inequalities in
that case, H is said to be strictly separate A and B.
Theorem – 12:
Let us say that A and B are two non-empty convex sets in such a manner that A is a compact set
while B is closed. Then, there is said to exist a hyperplane H which strictly separates A from B
and this is true if and only if A ∩ B = ϕ

Theorem – 13:
Let us assume two non-empty compact sets given by A and B, then there exists a hyperplane
which strictly separates A and B if and only if (conv A) ∩ (conv B) = ϕ.
8.5 – Polytopes:
A polytope in Rn can be defined as a convex hull for a finite set of points in R2 the polytope is
simply a polygon.
Definition:
Say S is a compact convex subset belonging to Rn then, a non-empty subset F of S is known as a
face of S in case F is not equal to S and there is a hyperplane such that H = [f : d] in a way such
that F = S ∩ H and either f(S) ≤ d or else f(S) ≥ d. The hyperplane H is known as a supporting
hyperplane with respect to S and if the dimension of F is said to be k, then F is known as a k-
face of S. And in case P is a polytope of dimension k, then P is known to be a k-polytope. A 0-
face of P is known as a vertex, a 1-face is termed as an edge while a (k-1) dimensional face is a
facet of S.
Definition:
Let us assume that S us a convex set and a point p in S is known as an extreme point of S in
case p is not present on the inside of any given line segment which belongs to S. Specifically, if
x, y ϵ S and p belongs to the line xy then, p = x or p = y and the set consisting of all the extreme
points of S is known as the profile of S.
Definition:
The set {v1, …, vk} is a minimal representation of the polytope P in case P = conv {v1, …, vk}
and also for each i = 1, …, k, and also vi does not correspond to conv {vj : j ≠ i}.
Theorem – 14:
Let us say that M = {v1, …, vk} is the minimal representation of a polytope P. Then the
following three statements are also true:
· pϵM
· p is a vertex of P
· p is an extreme point of P
Theorem – 15:
Let us suppose that S is a nonempty and compact convex set. Then S is the convex hull of its
profile i.e., the set which consists of the extreme points present in S.
Theorem – 16:
Let us sat that f is a linear functional defined on a nonempty compact convex set given by S.
Then there are extreme points of S which are given as:
Simplex:
A convex hull of an affinely independent finite set of vectors is termed as a simplex and in order
to construct a k-dimensional simplex,

Hypercube:
Let Ii be a line segment starting from the origin at 0 to the standard basis vector ei in Rn and in
such a case for k, such that 1 ≤ k ≤ n, the vector sum2
Ck = I1 + I2 + … : Ik
This is known as a k-dimensional hypercube.
8.6 Curves and Surfaces:
Bezier Curves:
The control points for any curve can be denoted by p0, p1, p2 and p3 and these points can belong
to R2 or R3 and can also be represented by homogenous forms in R3 or R4. The standard
parametric descriptions of these curves for 0 ≤ t ≤ 1 are,
Connecting two Bezier Curves:

Two basic Bezier curves are often joined end to end for various calculations in such a way that
the ending point of the first curve say x(t) is the starting point for the second curve at p2
assuming that the second curve is y(t). At p2, the combined curves are said to have G0 or
geometric continuity since the curves are joining at that point but if the tangent to curve 1 at p2
has a different direction as per the tangent line to the second curve then an abrupt change of
direction or a corner is seen. In order to avoid a sharp bend or a corner the curves are adjusted to
G1 also known as geometric continuity and here both the vectors at the point p2 are pointing
towards the same direction and the derivatives x’(1) and y’(0) point in the same direction though
the magnitudes of them might be different. When the tangent vectors are equal at p2, then such a
tangent vector is termed as continuous at p2, and the combined curve is said to have C1
continuity also known as C1 parametric continuity.
Matrix Equations for Bezier Curves:
The formula for x(t) may be written as:

The matrix which has columns containing the four control points is known as a geometry
matrix, G. The 4 × 4 matrix consisting of polynomial coefficients is known as the Bezier basis
matrix, MB. In case u(t) is the column vector of powers of t, in that case, the Bezier curve is
defined by
x(t) = GMBu(t)
When the entry for MB is changed appropriately, the resulting curves are known as B-splines,
and although they are smoother than Bezier curves, they don’t pass through any of the control
points. When the columns of the geometry matrix have the starting as well as the ending points
of the curves and the tangent vector to those points, the resulting curve is known to be a Hermite
cubic curve and it can be attained by replacing the matrix MB with a Hermite basis matrix.
The patameter t can be replaced by a parameter s for easiness while dealing with surfaces as:
The matrix in the above equation consisting of the control points is known as the geometry
vector.
Bezier Surfaces:
A three dimensional surface patch can be constructed using the Bezier equations given below:
A Bezier curve is produced when any one of the above given geometric matrices is multiplied at
the right hand by the factor given below:
Now, each of the entity of the resulting 4 × 1 matrix is a Bezier curve, given as:
As we fix t, then the above given equation is a column vector which can be also applied as a
geometry vector for a Bezier curve for another variable in place of s. The Bezier bicubic surface
resulting is:
Recursive subdivision of Bezier curves and surfaces:
The midpoint of the original curve x(t) occurs at the position x(.5) when x(t) has the standard
parameterization given as:
x(t) = (1 – 3t + 3t2 – t3)p0 + (3t – 6t2 + 3t3)p1 + (3t2 – 3t3)p2 + t3p3
for 0 ≤ t ≤ 1, the new control points q3 and r0 are given by:
q3 = r0 = x(.5) = (1/8) * (p0 + 3p1 + 3p2 + p3)
for Bezier curve given above in general, the direction of x(t) can be given by x’(t) since it is the
tangent vector to the curve as:
And for the condition 0 ≤ t ≤ 1 in particular,
From the above given equations, we can also calculate the value for x’(.5) as:
1. Consider the hyperplane 5x1 − 2x2 + x3 − 3x4 = 5 of R4. Clearly explain why 5x1
− 2x2 + x3 − 3x4 = 5 is not a subspace of R4.
Solution:
Every subspace of R4 must contain the origin. The origin of R4 is (0, 0, 0, 0).
Since 5(0) − 2(0) + (0) − 3(0) ≠ 5, this hyperplane does not intersect the origin.
Hence, the equation does not define a subspace of R4.
2. Find a basis for the hyperplane 5x1 − 2x2 + x3 − 3x4 = 5 of R4.
Solution:
To find a basis, we need to convert this equation in to a vector equation. Solve
the equation for x1. (In truth, you can solve for any variable.)
Now define all other variables as parameters. Letting x2 = r, x3 = s, and x4 = t, we

find that any vector (x1, x2, x3, x4) that lies in the plane can be written as the
vector equation
Separating the parameters, we find that
Therefore, the set
is a basis for the hyperplane.
3. State any version of the Consistency Theorem
Solution:
Example. A linear system is consistent if and only if the row rank of the
coefficient matrix equals the row rank of the augmented matrix.
4. True or false, if the dimension of V is 4, then any set of 3 vectors in V must be

linearly independent.
Solution:
FALSE. There is no guarantee that the three vectors are not dependent. Three
vectors in a four dimensional space can form a line, a plane, or a three
dimensional hyperplane.
5. True or false, if a subspace W has two different bases B1 and B2, then B1 and
B2 can have a different number of vectors in them.
Solution:
FALSE. One of the properties of bases is that any basis for a vector space must
have the exact same number of vectors and this number of vectors determines
the dimension of the vector space.
Questions 6-10 require you to find a basis for the following vector spaces.
6. All vectors in R3 whose components are equal.
Solution:
Such vectors are of the form (x, x, x). They form a one dimensional subspace of
R3. A basis is given by (1, 1, 1). (Any nonzero vector (a, a, a) will give a basis.)
7. All vectors in R4 whose components add to zero and whose first two
components add to equal twice the fourth component.
Solution:
The subspace consists of vectors
1 1 1 1
It is the nullspace of the matrix ", a two dimensional subspace of
1 1 0 !2
R4, so any two independent vector gives a basis. For example, we can take as a
basis v1 = (1, 1, −3, 1), v2 = (1, −1, 0, 0).
8. All anti-symmetric 3 × 3 matrices.
Solution:
Any anti-symmetric 3 × 3 matrix is of the form
Thus it is a 3 dimensional subspace, and a basis is given by

9. All vectors in R4 that are perpendicular to (1, 0, 1, 0).
Solution:
This subspace is just the nullspace of the 1 × 4 matrix (1, 0, 1, 0). This is a three
dimensional hyperplane in R4. A basis can be read from the matrix (it is of row
reduced echelon form already!):
v1 = (0, 1, 0, 0), v2 = (−1, 0, 1, 0), v3 = (0, 0, 0, 1).
10. All polynomials p(x) whose degree is no more than 3 and satisfies p(0) = 0.
Solution:
We can write the polynomial p(x) as p(x) = ax3 + bx2 + cx + d. The condition p(0)
= 0 implies d = 0. Thus the space is {ax3 + bx2 + cx | a, b, c R}. It is a three
dimensional subspace in the vector space of polynomials, and a basis is given by
p1(x) = x3, p2(x) = x2, p3(x) = x.

LinearAlgebraStudyGuide PDF

Diunggah oleh

Informasi Dokumen

Judul Asli

Hak Cipta

Format Tersedia

Bagikan dokumen Ini

Bagikan atau Tanam Dokumen

Opsi Berbagi

Apakah menurut Anda dokumen ini bermanfaat?

Apakah konten ini tidak pantas?

Hak Cipta:

Format Tersedia

LinearAlgebraStudyGuide PDF

Diunggah oleh

Hak Cipta:

Format Tersedia

LINEAR ALGEBRA AND ITS APPLICATIONS – CHAPTER ONE - SUMMARY

Chapter 1 −4• + 5•" + 9•# = −9 , becomes:

The set of all linear combinations of ?, … ?@ ,

where these vectors are in ℝ3 , is denoted by

Span{?, … ?@ }. This is called the subset of ℝ3

spanned by ?, … ?@ . Span{?, … ?@ } is the

same solution set as the system of linear equations

a) For each b in ℝJ , the equation Ax = b has THEOREMS: An indexed set S = {K, … K@ } of

1.7 Linear Independence a. T(u + v) = T(u) + T(v) for all u, v in the

Terminology for mappings:

A mapping T: ℝ3 -> ℝJ is said to be onto ℝJ

A mapping T: ℝ3 -> ℝJ is said to be one-to-

Applying this to a matrix:

2. Find the reduced row echelon form of

4. Find the reduced row echelon form of

5. Compute the rank of

7. Compute the rank of

8. Compute the rank of

Chapter 2 - Matrix Algebra

Two main topics that help us solve equation systems:

2.1 Matrix Operations:

&' %%%%%%%%%%%%%%%&7 %%%%%%%%%%%%%%%%%&9

Sums and Scalar Multiples:

o If matrix%$ is in size ? × * and matrix%+ in size%* × @,%$+ multiplication

Properties of Matrix Multiplication:

Here' , !'and'" are matrices with appropriate sizes and # a scalar.

The Transpose of a Matrix:

2.2 The Inverse of a Matrix:

An Algorithm for Finding H6I J

Another View of Matrix Inversion:

2 = KI ,''''' 2 = KL ,''''… ,''''' 2 = KM

2.3 Characterization of Inverse Matrices:

Invertible Linear Transformations:

Addition and Scalar Multiplication:

multiplication is done block by block.

Multiplication of Partitioned Matrices:

If matrix # is in size / × 0 and matrix 1 in size 0 × 2:

= 345% (#)94;% (1) @ 3456 (#)94;6 (1) … @ 3457 (#)94;7 (1)

Inverses of Partitioned Matrices:

2.5 Matrix Factorizations:

Solve EO = M and find O.

2.6 The Leontief Input - Output Model:

Here, Q is the consumption matrix for an economy.

A Formula For (T S U)$V :

The Economic Importance of Entries in (T S U)$V :

Generally, the equation #" = M can always be written as (D S Q)" = M where

2.7 Applications to Computer Graphics:

The zero vector is in c.

For each g vector in c and each scalar 3, the vector 3g is in c.

Column Space and Null Space of a Matrix:

# = [iV … if ] j Col # = Span{iV , … , if }

The set of all solutions of #" = B of / homogeneous linear equations in 0

Basis for a Subspace:

2.9 Dimension and Rank:

The Dimension of a Subspace:

Rank Theorem: If # has 0 columns: rank # @ dim Nul # = 0

The columns of # form a basis of _f .

Thus, the solutions are x1 = 1, x2 = 2, x3 = 4

2. Simplify, or write “undefined” if the multiplication is not valid.

3. Simplify, or write “undefined” if the multiplication is not valid.

5. Simplify, or write “undefined” if the multiplication is not valid.

6. Simplify, or write “undefined” if the multiplication is not valid.

8. Simplify, or write “undefined” if the multiplication is not valid.

· ad^, = da^,4444444444and444444444444af + g, = af, + a*g,