Math 3412: Advanced Linear Algebra

1. Real Vector Spaces

Chapter 1: Real Vector Spaces
In this chapter we present some of the fundamental aspects of real
vector spaces, especially those with finite dimension. Such spaces are
1.1 Real Vector Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 more general than Rn but in many ways are essentially the same as Rn .
The central idea is that of a coordinate system. A coordinate system for
1.2 The Standard Basis in Rn . . . . . . . . . . . . . . . . . . . . . 4
a real vector space V is derived from a basis for V and allows one to
1.3 Spanning and Independent Sets . . . . . . . . . . . . . . . 7 represent vectors in V as ordinary columns of coordinates. Furthermore,
this representation is consistent with the two operations—vector addition
1.4 More on Spanning and Independent Sets . . . 12
and scalar multiplication—intrinsic to V. Thus for any two vectors in V
1.5 Basis and Dimension . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 the column for their sum is the sum of their columns, and the column
1.6 Changing Basis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 for a multiple of a vector is the multiple of its column. By representing
vectors as columns, we may work with vectors in any real n-dimensional
vector space V as if they were columns in Rn , at least as far as their
arithmetic is concerned.
R. McEachin
1.1. Real Vector Spaces
Department of Mathematics
University of the West Indies Let V be a collection of objects denoted by u, v, w, . . .. We say V
Mona Campus is a vector space over R to mean, first, that there is a vector addition
Kingston 7, Jamaica mapping V × V → V satisfying the properties:
1) (u + v) + w = u + (v + w) for all u, v, w ∈ V,
September, 2016
2) v + w = w + v for all v, w ∈ V,
3) there is some 0 ∈ V such that 0 + v = v + 0 = v for all v ∈ V, and
4) for any v ∈ V there is some −v ∈ V such that −v+v = v+(−v) = 0.
Second, there is a scalar multiplication mapping R × V → V satisfying
the properties:

1
 5) (α + β)v = αv + βv for all α, β ∈ R and v ∈ V, in Rn , we say u = v to mean that αi = βi for all i. Given any γ ∈ R as
6) α(v + w) = αv + αw for all α ∈ R and v, w ∈ V, well, we define u + v and γu by
7) (αβ)v = α(βv) for all α, β ∈ R and v ∈ V, and    
α 1 + β1 γα1
8) 1v = v for all v ∈ V.
 α 2 + β2   γα2 
u+v =
 .. ;
 γu =  
 ..  .
Thus we define a vector space in such a way that addition and scalar . .
multiplication are intrinsic. Furthermore, the properties listed here char- α n + βn γαn
acterize the fundamental aspects of these operations.
We will refer to a finite collection of vectors, such as {u, v, w}, many Using this arithmetic, one verifies that Rn satisfies Properties (1)–(8)
times in this book. We call such a collection a family to mean that it is an listed above. Thus Rn is a real vector space.
ordered k-tuple of vectors. A family differs from a set in two ways. First,
vectors may occur more than once in a family and we do not discard any Example 1.1.2. Let Zn consist of all polynomials with real coefficients
repetitions; otherwise we alter the family. Thus, for example, the family and having degree at most n, subject to the arithmetic described below.
{u, v, v} is not the same as {u, v}. Second, if we reorder the vectors in a Thus
family we consider the resulting family distinct. Thus, for example, the Zn = {α0 + α1 x + · · · + αn xn : α0 , α1 , . . . , αn ∈ R}
families {u, v, w} and {w, u, v} are distinct. We will also refer to a finite
subject to the arithmetic we presently describe. Note that Zn includes
collection of scalars many times in this book. We call such a collection a
the polynomial with αi = 0 for i = 0, 1, . . ., n. We call this the zero
family to mean that it is an ordered k-tuple of scalars. Pn
polynomial and usually denote it by 0. In Zn , given p(x) = i=0 αi xi
Pn
and q(x) = i=0 βi xi , we say that p = q to mean that αi = βi for i = 0,
Example 1.1.1. Let n be a positive integer. Then Rn denotes the set 1, . . ., n. Given any γ ∈ R as well, we define p + q and γp by
of all columns of real numbers of length n, subject to the arithmetic


described below. Thus p + q (x) = (α0 + β0 ) + (α1 + β1 )x + · · · + (αn + βn )xn ;


   γp (x) = γα0 + γα1 x + · · · + γαn xn
 α1 

 α2  

R = 
n
.  : α1 , α2 , . . . , αn ∈ R
 .  Using this arithmetic, one verifies that Zn satisfies Properties (1)–(8)

 

 .  listed above. Thus Zn is a real vector space.
αn

Example 1.1.3. Let Rm×n consist of all m×n matrices with real entries,
subject to the arithmetic we now describe. Given
subject to the arithmetic described below. Thus
   
α1 β1   
 α11 α12 · · · α1n 
 α2   β2   
u= 
 ...  , v= 
 ..  Rm×n
α
=  21
α22 · · · α2n 
 ij
: α ∈ R for all i and j
. 
 ···················· 

αn βn αm1 αm2 · · · αmn

2 3
subject to the arithmetic we now describe. Given Definition 1.2.1. Let A = {v1 , v2 , . . . , vk } be a family of vectors in a
    real vector space V. A linear combination of vectors in A is an expression
α11 α12 · · · α1n β11 β12 · · · β1n of the form α1 v1 + α2 v2 + · · · + αn vn , where α1 , α2 , . . . , αk ∈ R.
α α22 · · · α2n  β β22 · · · β2n 
A =  21 , B =  21 ,
···················· ···················· Thus linear combinations are produced by scaling each vi ∈ A by some αi
αm1 αm2 · · · αmn βm1 βm2 · · · βmn and summing the results. Note that some or even all of the αi can be 0.
For example, 0v1 + 0v2 + · · · + 0vk = 0 is a linear combination of vectors
we say A = B to mean that αij = βij for all i and j. Given any γ ∈ R
in A. Also, say, 0v1 + 1v2 + · · · + 0vk = v2 is such a linear combination.
as well, we define A + B and γA by
Furthermore, the sum of two linear combinations is a linear combination:
 
α11 + β11 α12 + β12 · · · α1n + β1n
(α1 v1 + α2 v2 + · · · + αk vk ) + (β1 v1 + β2 v2 + · · · + βk vk )
 α + β21 α22 + β22 · · · α2n + β2n 
A + B =  21 , = (α1 + β1 )v1 + (α2 + β2 )v2 + · · · + (αk + βk )vk
·········································
αm1 + βm1 αm2 + βm2 · · · αmn + βmn
  Similarly, a scalar multiple of a linear combination is a linear combination:
γα11 γα12 · · · γα1n
 γα21 γα22 · · · γα2n  γ(α1 v1 + α2 v2 + · · · + αk vk ) = (γα1 )v1 + (γα2 )v2 + · · · + (γαk )vk
γA =  
························
γαm1 γαm2 · · · γαmn
One particularly valuable use of linear combinations is to establish a
Using this arithmetic, one verifies that R m×n
satisfies Properties (1)–(8) coordinate system in a vector space V. To see how this may be done, we
listed above. Thus R m×n
is a real vector space. consider in some detail the system of standard coordinates in Rn . Let n
be a positive integer and define
Example 1.1.4. Let C[0, 1] consist of all continuous functions mapping      

1 0 0
[0, 1] → R. Given f, g ∈ C[0, 1], define f + g by f + g (x) = f (x) + g(x) 0 1 0



for all x. Given γ ∈ R as well, define γf by (γf (x) = γ f (x) . Using e1 =  
 ...  , e2 =  
 ...  , ..., en =  
 ...  ,
this arithmetic, one verifies that C[0, 1] satisfies Properties (1)–(8) listed
0 0 1
above. Thus C[0, 1] is a real vector space.
Thus each ej is the column of length n whose jth entry is 1 and whose
1.2. The Standard Basis in Rn other entries are 0. Define E = {e1 , e2 , . . . , en }. We call E the standard
basis for Rn . This set has two essential properties which allow it to be
In any real vector space, there are two basic arithmetical operations the basis for a coordinate system.
to be found: vector addition and scalar multiplication. The essential First, in Rn let
properties of these operations are given in Properties (1)-(8) listed at the  
α1
beginning of Section 1.1. These operations are fundamental to vector  α2 
x= ... 
 (1.1)
spaces and lead to the basic notion of a linear combination of a set of
vectors. αn

4 5
Then, using the vector arithmetic described in Example 1.1.1, Exercise Set 1.2
            1) Let A = {v1 , v2 , . . . , vk } and B = {w1 , w2 , . . . , wp } be families of
α1 0 0 1 0 0
 0   α2   0  0 1 0 vectors in a real vector space V. Let also v ∈ V and suppose v is a
x=    
 ...  +  ...  + · · · +  ..  = α1      
 ...  + α2  ...  + · · · + αn  ... 
.  linear combination of vectors in A. If each vi ∈ A is itself a linear
0 0 αn 0 0 1 combination of vectors in B, show that v is a linear combination of
vectors in B.
That is, x = α1 e1 + α2 e2 + · · · + αn en . Thus each x ∈ Rn can be written
as a linear combination of vectors in E in at least one way.
1.3. Spanning and Independent Sets
On the other hand, suppose x ∈ Rn and
Let B be a family of vectors in a real vector space V. We consider in
x = α1 e1 + α2 e2 + · · · + αn en , x = β1 e1 + β2 e2 + · · · + βn en more detail the properties B must have in order to serve as the basis for
a coordinate system in V. We also give examples of families of vectors
for some scalars α1 , α2 , . . ., αn , and β1 , β2 , . . ., βn . Rewriting each ej in a real vector space V which do or do not have some of the requisite
as a column of length n and using the arithmetic in Rn , one obtains properties. We begin with a definition.
  

α1 β1
Definition 1.3.1. Let B = {v1 , v2 , . . . , vk } be a family of vectors in a
 α2   β2 
x= 
 ...  , x= 
 ..  real vector space V. The span of B, denoted by span B, is the set of all
. linear combinations of vectors in B. Thus
αn βn

Thus αi = βi for all i. It follows that each x ∈ Rn can be written as a span B = {α1 v1 + α2 v2 + · · · + αk vk : α1 α2 , . . . , αk ∈ R}
linear combination of vectors in E in at most one way
In summary, each x as in (1.1) has a unique representation as a linear
Example 1.3.2. In R3 , let
combination of vectors in E. This is x = α1 e1 + α2 e2 + · · · + αn en . We
call the scalars α1 , α2 , . . ., αn the coordinates of x relative to E. Clearly,        
1 −1 −1 2
these n numbers (in the order listed) completely characterize x. v1 =  2  , v2 =  0  , v3 =  4  , v =  −6 
In general, given a real vector space V we may seek a family of vectors 0 1 3 −5
B = {v1 , v2 , . . . , vn } for use in establishing a coordinate system in V. It
is natural to require of B that each v ∈ V have a unique expression as a Setting B = {v1 , v2 , v3 }, we determine whether v ∈ span B. Thus we
linear combination of its vectors. In this case, if v = α1 v1 + α2 v2 + · · · + seek α1 , α2 , α3 such that α1 v1 + α2 v2 + α3 v3 = v. That is, we seek α1 ,
αn vn , we call α1 , α2 , . . ., αn the coordinates of v relative to B. We also α2 , α3 such that
say that B is the basis for this coordinate system.        
1 −1 −1 2
α1  2  + α2  0  + α3  4  =  −6 
0 1 3 −5

6 7
Using the arithmetic of vectors in R3 , this holds if and only if for a coordinate system in V we must first be able to write each v ∈ V
 as a linear combination of the vectors in B in at least one way. That is,
 α1 − α2 − α3 = 2
we must have v ∈ span B for all v ∈ V. In this case, we say that B is a
2α1 + 4α3 = −6
 spanning set for V or simply that B spans V.
α2 + 3α3 = −5
For B to be suitable as a basis for V, we’ve also seen that each v ∈ V
Using Gauss-Jordan elimination, one obtains the solutions must be expressible as a linear combination of vectors in B in at most
      one way. Thus if v ∈ span B and, say,
α1 −3 −2
 α2  =  −5  + t  −3  ,
α3 0 1 v = α 1 v1 + α 2 v2 + · · · + α k vk , v = β1 v1 + β2 v2 + · · · + βk vk , (1.2)

where t ∈ R is an arbitrary parameter. Thus v ∈ span B. In fact, v
can be represented as a linear combination of vectors in B in infinitely
many ways, one for each choice of t. For example (choosing t = 0),
v = −3v1 − 5v2 + 0v3 . Also (choosing t = 1), v = −5v1 − 8v2 + v3 .
Example 1.3.3. In Z2 , let
p1 (x) = 2 − 3x + x2 , p2 (x) = −1 + x + 2x2 , q(x) = −8 + 11x + 8x2 .
we must have αi = βi for all i. We now consider ways of reformulating
this property.
Suppose first that at least one v ∈ span B can be written as a linear
combination as in (1.2), and furthermore αi 6= βi for at least one index i.
Then, subtracting equations,
0 = (α1 − β1 )v1 + (α2 − β2 )v2 + · · · + (αk − βk )vk

Setting B = {p1 , p2 }, we determine whether q ∈ span B. Thus we seek In this case at least one coefficient, namely αi − βi , is nonzero. However,
scalars α1 and α2 such that α1 p1 + α2 p2 = q. That is, we seek α1 and α2 it is obvious that 0 = 0v1 +0v2 +· · ·+0vk as well. Thus 0 can be written
such that as a linear combination in more than one way. In other words, if some
v ∈ span B has more than one representation as a linear combination of
α1 (2 − 3x + x2 ) + α2 (−1 + x + 2x2 ) = −8 + 11x + 8x2 vectors in B then 0 has more than one such representation as well. It
(2α1 − α2 ) + (−3α1 + α2 )x + (α1 + 2α2 )x2 = −8 + 11x + 8x2 follows that if the representation for 0 is unique then the representation
for each v ∈ span B is unique. This motivates the following definition.
Equating coefficients, this last equation holds if and only if

 2α1 − α2 = −8 Definition 1.3.4. Let B = {v1 , v2 , . . . , vk } be a family of vectors in a
−3α1 + α2 = 11 real vector space V. We say B is linearly independent to mean that the

α1 + 2α2 = 8 only solution to the equation α1 v1 + α2 v2 + · · · + αk vk = 0 is the trivial
solution α1 = 0, α2 = 0, . . ., αk = 0.
Using Gauss-Jordan elimination, one determines that this system has no
solution. Thus q ∈
/ span{p1 , p2 }.
Proposition 1.3.5. Let B be a linearly independent family of vectors in a
Suppose now that V is a real vector space and B = {v1 , v2 , . . . , vk } real vector space V. Then each v ∈ span B can be written as a linear
is a family of vectors in B. We’ve seen that for B to be suitable as a basis combination of vectors in B in only one way.

8 9
This follows immediately from Definition 1.3.4 and the discussion preced- We determine whether A is linearly independent. To do this we seek all
ing it. solutions to the equation β1 B1 + β2 B2 + β3 B3 + β4 B4 = O, where O
Given a family B = {v1 , v2 , . . . , vk } in a real vector space V, we denotes the 2 × 2 zero matrix. Thus we seek β1 , β2 , β3 , β4 such that
say that B is linearly dependent to mean that it is not independent. In          
1 0 1 1 1 1 1 1 0 0
this case, the equation α1 v1 + α2 v2 + · · · + αk vk = 0 has at least one β1 + β2 + β3 + β4 =
0 0 0 0 1 0 1 1 0 0
nontrivial solution. In fact, it has infinitely many in this case.
Using the standard arithmetic of matrices, one obtains the linear system
Example 1.3.6. In R3 , let A = {v1 , v2 , v3 }, where of scalar equations
      
1 4 7  β1 + β2 + β3 + β4 = 0

v1 =  2  , v2 =  5  , v3 =  8  β2 + β3 + β4 = 0
3 6 9 
 β3 + β4 = 0
β4 = 0
We determine whether A is linearly independent. To do this we seek all
solutions to the equation α1 v1 + α2 v2 + α3 v3 = 0. That is, we seek α1 , Since this has unique solution β1 = β2 = β3 = β4 = 0, A is linearly
α2 , α3 such that independent.
        Exercise Set 1.3
1 4 7 0
α1  2  + α2  5  + α3  8  =  0  4
1) In R , let B = {b1 , b2 , . . . , b5 }, where b1 , b2 , b3 , b4 , and b5 are
3 6 9 0 the respective vectors
         
This leads to the system of equations 1 0 2 −3 1
  −2  1  0   4   5 
 ,  ,  ,  , and  .
 α1 + 4α2 + 7α3 = 0 3 2 14 −4 −1
2α1 + 5α2 + 8α3 = 0 2 0 4 −1 −8

3α1 + 6α2 + 9α3 = 0
a) Determine whether B is linearly independent.
The solution set is     b) Determine whether x1 = [1, 4, −3, −8]T is in span B. If so, de-
α1 1
 α2  = t  −2  termine whether its representation as a linear combination of
α3 1 the vectors in B is unique.
c) Determine whether x2 = [4, −1, 6, 2]T is in span B. If so, deter-
Since there are infinitely many solutions, A is linearly dependent.
mine whether its representation as a linear combination of the
Example 1.3.7. In R2×2 let A = {B1 , B2 , B3 , B4 }, where vectors in B is unique.

        2) In Z2 , let B = {p0 , p1 , p2 }, where

1 0 1 1 1 1 1 1
B1 = , B2 = , B3 = , B4 =
0 0 0 0 1 0 1 1 p0 (x) = x2 + 2x + 1, p1 (x) = x2 − 1, p2 (x) = x2 + x + 1

10 11
 Determine whether B is linearly independent. Proof. (⇒) Suppose that A is linearly dependent. Then the equation
3) Let A = {v1 , v2 , v3 } be the set vectors given in Example 1.3.6.
α 1 v1 + α 2 v2 + · · · + α k vk = 0
a) Write 0 as a linear combination of the vectors in A in two dis-
tinct ways. has at least one nontrivial solution. That is, there is some solution with
b) Define v = v1 + 2v2 − 3v3 . Write v as a linear combination at least one coefficient, say αi , nonzero. For this solution,
of the vectors in A in another way. What are all other ways of
−αi vi = α1 v1 + · · · + αi−1 vi−1 + αi+1 vi+1 + · · · + αk vk
writing v as a linear combination of the vectors in A?
α1 αi−1 αi+1 αk
4) Let B = {v1 , v2 , . . . , vk } be a family of vectors in a real vector vi = − v1 − · · · − vi−1 − vi+1 − · · · − vk
αi αi αi αi
space V.
a) If x, y ∈ span B, show that x + y ∈ span B. Thus vi ∈ span A0 , where A0 = {v1 , . . . , vi−1 , vi+1 , . . . , vk }.
(⇐) Suppose there is an index i such that vi is in the span of the
b) If x ∈ span B and γ ∈ R, show that γx ∈ span B.
remaining vectors in A. Thus there are scalars α1 , . . ., αi−1 , αi+1 , . . ., αk
5) Let {v1 , v2 } be a family in a real vector space V, and define w1 = such that
v1 + v2 and w2 = v1 − v2 . If {v1 , v2 } is linearly independent, show
that {w1 , w2 } is linearly independent. vi = α1 v1 + · · · + αi−1 vi−1 + αi+1 vi+1 + · · · + αn vn
6) Let V be a real vector space and let A = {v1 , v2 , . . . , vk } be a
Subtracting vi from both sides,
family of vectors in V. If vj = 0 for some j show that A is linearly
dependent. 0 = α1 v1 + · · · + αi−1 vi−1 − vi + αi+1 vi+1 + · · · + αn vn .

Since not all coefficients on the right side are 0 (the coefficient of vi is −1),
1.4. More on Spanning and Independent Sets
A is linearly dependent.

Let A = {v1 , v2 , . . . , vk } be a family of vectors in a real vector
space V. We’ve seen that the span of A is the set of all v ∈ V that can be
written as a linear combination of vectors of A in at least one way; A is
linearly independent if each v ∈ span A can be written so in at most one
way. Since the notions of span and independence both arise from that of
a linear combination, they are related to each other in subtle ways. In
this section, we give results establishing some of these relationships.
Corollary 1.4.2. Let A = {v1 , v2 , . . . , vk } be a linearly independent
family of vectors in a real vector space V. Let also v ∈ V and suppose
v∈/ span A. Then B = A ∪ {v} is linearly independent.
Proof. Suppose α1 , α2 , . . ., αk , β are scalars such that
α1 v1 + α2 v2 + · · · + αk vk + βv = 0. (1.3)

Proposition 1.4.1. Let A = {v1 , v2 , . . . , vk } be a family of vectors in a If β 6= 0 then it follows that

real vector space V. Then A is linearly dependent if and only if at least α1 α2 αk
one vector in A is in the span of the others. v=− v1 − v2 − · · · − vk
β β β

12 13
In particular, v ∈ span A. Since this is impossible, β = 0. Thus (1.3) Adding (1.4) and (1.5), one obtains
reduces to α1 v1 + α2 v2 + · · ·+ αk vk = 0. Since A is linearly independent,
the only solution here is α1 = 0, α2 = 0, . . ., αk = 0. Therefore all v = (α1 + β1 )v1 + · · · + (αi−1 + βi−1 )vi−1
coefficients in (1.3) must be 0, and B is linearly independent. + (αi+1 + βi+1 )vi+1 + · · · + (αk + βk )vk

Theorem 1.4.3. Let A = {v1 , v2 , . . . , vk } be a family of vectors in a real
vector space V. If A is linearly dependent then A contains some subset
A0 = {v1 , . . . , vi−1 , vi+1 , . . . , vk } such that span A0 = span A.
Proof. Suppose A is linearly dependent. By Proposition 1.4.1, some
vector, say vi , is in the span of the others. Define
0
A = {v1 , . . . , vi−1 , vi+1 , . . . , vk },
so that vi ∈ span A0 . We show that span A0 = span A.
First, let v ∈ span A0 . Then there are scalars α1 , . . ., αi−1 , αi+1 ,
. . ., αk such that
v = α1 v1 + · · · + αi−1 vi−1 + αi+1 vi+1 + · · · + αk vk .
Thus v ∈ span A0 , and hence span A ⊆ span A0 . It follows that span A0 =
span A.
Before giving our last results on independent and spanning sets, we
introduce a result we will use presently.
Proposition 1.4.4. Let A ∈ Rm×n . If n > m then the homogeneous
system Ax = 0 has at least one nontrivial solution. That is, there is
some x0 ∈ Rn with x0 6= 0 such that Ax0 = 0.
This result follows from the row-reduction algorithm for solving linear
systems of equations. An alternative, somewhat more formal proof can
be given by induction on m, the number of rows of A. We omit the
details.

Then, obviously, Theorem 1.4.5. Let A = {w1 , w2 , . . . , wp } be a spanning set for a real
vector space V. If B = {v1 , v2 , . . . , vk } is a family in V and k > p then
v = α1 v1 + · · · + αi−1 vi−1 + 0vi + αi+1 vi+1 + · · · + αk vk B is linearly dependent.

as well. Thus v ∈ span A, and hence span A0 ⊆ span A. Proof. Since A spans V and each vj is a vector in V, each vj is a linear
Suppose now that v ∈ span A. Then there are scalars α1 , α2 , . . ., αk combination of the vectors in A. Thus there are scalars αij with i = 1, 2,
such that v = α1 v1 + α2 v2 + · · · + αk vk . Thus . . . p and j = 1, 2, . . ., k such that

v − αi vi = α1 v1 + · · · + αi−1 vi−1 + αi+1 vi+1 + · · · + αk vk (1.4) 
 v1 = α11 w1 + α21 w2 + · · · + αp1 wp


v2 = α12 w1 + α22 w2 + · · · + αp2 wp
Furthermore, since vi ∈ span A0 one has αi vi ∈ span A0 as well. (See also 
 ·································


Problem 2 in Exercise Set 1.3.) Thus there are scalars β1 , . . ., βi−1 , βi+1 , vk = α1k w1 + α2k w2 + · · · + αpk wp
. . ., βk such that
(Note that the indices on the right sides above do not follow the usual
αi vi = β1 v1 + · · · + βi−1 vi−1 + βi+1 vi+1 + · · · + βk vk . (1.5) pattern for matrices.) Using these coefficients (but in the “transposed”

14 15
order), produce the homogeneous system of equations
 space V. If {v1 , v2 , . . . , vk } is a linearly independent family in V then

 α11 x1 + α12 x2 + · · · + α1k xk = 0


α21 x1 + α22 x2 + · · · + α2k xk = 0
 ······························
 Exercise Set 1.4


αp1 x1 + αp2 x2 + · · · + αpk xk = 0
Since k > p, our last proposition implies that this system has at least
one nontrivial solution. Let β1 , β2 , . . ., βk be such a solution. We show
that β1 v1 + β2 v2 + · · · + βk vk = 0. Since at least one βj is nonzero, this
proves that B is linearly dependent.
Note that
β1 v1 + β2 v2 + · · · + βk vk = β1 (α11 w1 + α21 w2 + · · · + αp1 wp )
+ β2 (α12 w1 + α22 w2 + · · · + αp2 wp )
+ ·······························
+ βk (α1k w1 + α2k w2 + · · · + αpk wp )
Regrouping terms on the right side above,
β1 v1 + β2 v2 + · · · + βk vk = (α11 β1 + α12 β2 + · · · + α1k βk )w1
+ (α21 β1 + α22 β2 + · · · + α2k βk )w2
+ ······························
+ (αp1 β1 + αp2 β2 + · · · + αpk βk )wp
Since αi1 β1 + αi2 β2 + · · · + αik βk = 0 for i = 1, 2, . . ., p, our last equation
reduces to
β1 v1 + β2 v2 + · · · + βk vk = 0w1 + 0w2 + · · · + 0wp = 0
Since at least one βj here is nonzero, B is linearly dependent.
The following corollary is just a restatement of the last result in a
slightly more natural form.
16
Corollary 1.4.6. Let {w1 , w2 , . . . , wp } be a spanning set for a real vector
k ≤ p.
1) Let A = {v1 , v2 , . . . , vk } be a linearly independent family of vectors
in a real vector space V. Let also v ∈ V and define B = A ∪ {v}. If
B is linearly dependent, show that v ∈ span A.
2) Let A = {v1 , v2 , . . . , vk } be a family of vectors in a real vector
space V, let α1 , α2 , . . ., αk be nonzero scalars, define wi = αi vi for
all i, and set B = {w1 , w2 , . . . , wk }.
a) Show that A is linearly independent if and only if B is linearly
independent.
b) Show that span A = span B.
3) Let A = {v1 , v2 , . . . , vm } be a family of vectors in a real vector
space V, let k < m, and set A0 = {v1 , v2 , . . . , vk }.
a) If A is linearly independent, show that A0 is linearly indepen-
dent.
b) If A0 is linearly dependent, show that A is linearly dependent.
4) Let A = {v1 , v2 , v3 } be a linearly independent family of vectors in
a real vector space V. Define w1 = 2v1 + v2 , w2 = 2v2 + v3 , and
w3 = 2v3 + v1 , and set B = {w1 , w2 , w3 }. Show that B is linearly
independent.
5) Let A = {v1 , v2 , . . . , vk } be a linearly independent family of vectors
in a real vector space V. Let also αij with i, j = 1, 2, . . . , k be real
scalars, define wj = α1j v1 + α2j v2 + · · · + αkj vk for all j, and set
B = {w1 , w2 , . . . , wk }. Assuming that A = [αij ]ni,j=1 is an invertible
matrix, show that B is linearly independent.
17
 1.5. Basis and Dimension
Let V be a real vector space. Clearly, any family which is linearly
independent and spans V has particularly desirable properties. In partic-
ular, it may serve as a basis for a coordinate system in V. In this section,
we consider such sets in more detail. We also formalize our notion of
coordinates.
Definition 1.5.1. Let B be a family of vectors in a real vector space V.
We say B is a basis for V to mean that B is linearly independent and
spans V.
Theorem 1.5.2. Let B = {v1 , v2 , . . . , vn } be a basis for a real vector
space V. For every v ∈ V, there are unique scalars α1 , α2 , . . ., αn such
that
v = α 1 v1 + α 2 v2 + · · · + α n vn . (1.6)
Proof. Let v ∈ V. Since B is a spanning set for V, every vector in V can
be written as a linear combination of vectors in B in at least one way.
(See also Definition 1.3.1.) Since B is linearly independent as well, each
vector can be written as such a linear combination in at most one way
(see Proposition 1.3.5 and the discussion preceding it.) Thus there are
unique scalars α1 , α2 , . . ., αn such that (1.6) holds.
Definition 1.5.3. Let B = {v1 , v2 , . . . , vn } be a basis for a real vector
space V. For each v ∈ V, the unique scalars α1 , α2 , . . ., αn of The-
orem 1.5.2 corresponding to v are called the coordinates of v relative
to B. We also denote the relation (1.6) by writing
 
α1
 α2 
v=  ... 

αn
B
Clearly, a basis is an extremely important set of vectors in a real
vector space V. Less clearly, perhaps, whenever V has a basis of the form
18
B = {v1 , v2 , . . . , vn } it has infinitely many such bases. (We will prove
this in Section 1.6.) Thus there are infinitely many coordinate systems
that may be produced in V. Despite this abundance of bases (and hence
of coordinate systems), however, there is one feature common to all of
them: if any basis contains, say, n vectors then all of them contain n
vectors.
Theorem 1.5.4. Let V be a real vector space and let B = {v1 , v2 , . . . , vn }
and D = {w1 , w2 , . . . , wm } be bases for V. Then n = m.
Proof. By definition, B and D are each linearly independent families
that span V. Since B is linearly independent and D spans V, n ≤ m by
Corollary 1.4.6. Since at the same time D is linearly independent and B
spans V, we also have m ≤ n. Therefore, n = m.
Definition 1.5.5. Let V be a vector space over R. We say that V is
n-dimensional, or V has dimension n, to mean that there is some basis
for V containing n vectors. In this case, we also write dim V = n.
In the special case that V = {0}, it is a common error to assume
that B = {0} is a basis for V. However, since any set containing the
zero vector is linearly dependent (see Problem 6 of Exercise Set 1.3), B
is not a basis for V. In fact, V has no basis (unless, as some argue, it
is ∅). At any rate, we say that V is 0-dimensional by convention. We
note in particular that V is not a 1-dimensional vector space as some first
assume.
The definition of dimensionality given above is unambiguous. If one
basis for V contains n vectors then every basis for V contains n vectors.
We may also observe the following facts about the dimension of a vector
space. Suppose V has dimension n and let B = {v1 , v2 , . . . , vn } be a
basis for V. If A is a linearly independent family of vectors in V then A
contains at most n vectors; if C is a spanning set for V then C contains at
least n vectors. Indeed, let A = {u1 , u2 , . . . , uk } be linearly independent.
By Corollary 1.4.6, since B is a spanning set for V we have k ≤ n.
19
Similarly, let C = {w1 , w2 , . . . , wp } be a spanning set for V. Since B is
linearly independent, p ≥ n by Corollary 1.4.6.
We now give two more results concerning properties of bases.
Theorem 1.5.6. Let V be a real n-dimensional vector space, let A =
{w1 , w2 , . . . , wk } be a family of vectors in V, and consider the following
three statements:
1) k = n;
2) A is linearly independent;
3) A spans V.
If any two of these statements are true then so is the third.
Proof. Note first that since V is n-dimensional it has some basis con-
taining n vectors. Let B = {v1 , v2 , . . . , vn } be such a basis.
[(1) and (2) ⇒ (3)] Suppose A contains n vectors and is linearly
independent. Suppose also that v ∈ V. Since B is a spanning set for V
containing n vectors and A∪{v} contains n+1 vectors, A∪{v} is linearly
dependent by Theorem 1.4.5. Thus there are scalars α1 , α2 , . . ., αn , β
with at least one of them nonzero such that
α1 v1 + α2 v2 + · · · + αn vn + βv = 0.
If β = 0, this equation reduces to α1 v1 + α2 v2 + · · · + αn vn = 0. Since
not all coefficients here are 0, this implies that A is linearly dependent.
Since this is impossible, β 6= 0. It follows that
α1 α2 αn
v=− v1 − v2 − · · · − vn
β β β k − n vectors from A in such a way that the resulting family is a
In particular, v ∈ span A. It follows that V ⊂ span A.
[(1) and (3) ⇒ (2)] Suppose A contains n vectors and spans V.
Suppose also, for the sake of contradiction, that A is linearly dependent.
By Theorem 1.4.3, it contains a subset A0 = {v1 , . . . , vi−1 , vi+1 . . . , vn }
such that span A0 = span A = V. But then the spanning set A0 contains
20
fewer vectors than the linearly independent set B. Since this is impossible
(see Corollary 1.4.6), A is linearly independent.
[(2) and (3) ⇒ (1)] Suppose A is linearly independent and spans V.
Then A is a basis for V. Since B is also a basis for V and all bases contain
the same number of vectors, k = n.
This result is useful for verifying that some family of vectors forms a
basis for a vector space V. Suppose, for example, that V is n-dimensional
and A = {v1 , v2 , . . . , vn } is a family of vectors in V. Thus, in particular,
A contains n vectors. If A is shown to be linearly independent then by
the foregoing result A also spans V. On the other hand, if A spans V
then by the foregoing result A is also linearly independent. In either case,
A forms a basis for V.
Suppose now that V is an n-dimensional vector space and A =
{v1 , v2 , . . . , vk } is a family in V. If A is linearly independent then k ≤ n.
If k = n then A spans V and hence is a basis for V. Otherwise, k < n
and it is impossible for A to span V. On the other hand, suppose A is a
spanning set for V. Then k ≥ n. If k = n then A is linearly independent
and hence a basis for V. Otherwise, k > n and it is impossible for A to
be linearly independent. Our last result elaborates further on these facts.
Theorem 1.5.7. Let V be an n-dimensional vector space and let A =
{v1 , v2 , . . . , vk } be a family in V.
1) If A is linearly independent but not a basis for V then it is possible
to add n − k vectors to A in such a way that the resulting family is
a basis for V.
2) If A spans V but is not a basis for V, then it is possible to discard
basis for V.
Proof. (1) Since A is linearly independent, k ≤ n; since A is not a basis
for V, k < n and A does not span V. Then we may choose vk+1 , vk+2 ,
. . ., vn in such a way that vk+i ∈ / span{v1 , v2 , . . . , vk+i−1 } for i = 1, 2,
. . ., n − k. To see this, note that all spanning sets for V contain at least
21
n vectors, but when i ≤ n − k the set {v1 , v2 , . . . , vk+i−1 } contains fewer also v, w ∈ V and suppose
than n vectors. Thus a vector vk+i ∈ / span{v1 , v2 , . . . , vk+i−1 } may be    
chosen as indicated. Furthermore, since vk+1 ∈ / span A and A is linearly β1 δ1
 β2   δ2 
independent, the set {v1 , v2 , . . . , vk+1 } is linearly independent as well. v= 
 ..  , w= 
 ...  .
Similarly, the sets {v1 , v2 , . . . , vk+2 } and so on, up to {v1 , v2 , . . . , vn }, .
are all linearly independent. Since this last set is linearly independent βn B δn B
and contains n vectors, it is a basis for V.
Show that, if γ ∈ R as well,
(2) We prove by induction on k − n. If k − n = 1 then, in particular,
k > n. Thus A is linearly dependent. By Theorem 1.4.3, there is some   

β1 + δ1 γβ1
set A0 = {v1 , . . . , vi−1 , vi+1 , . . . , vk } such that span A0 = span A = V.  β2 + δ2   γβ2 
Furthermore, A0 contains n vectors. Thus, by Theorem 1.5.6, A0 is a v+w =
 ..  ,
 γv =  
 ..  .
. .
basis for V. βn + δn γβn B
B
Suppose now that m is a natural number and the conclusion holds
whenever k − n = m. If k − n = m + 1 then, as before, k > n and hence
A is linearly dependent. By Theorem 1.4.3, there is some set 1.6. Changing Basis

A0 = {v1 , . . . , vi−1 , vi+1 , . . . , vn+m+1 }.
such that span A0 = span A = V. By our induction hypothesis, we may
discard m more vectors from A0 in such a way that the remaining vec-
tors form a basis for V. Since, in this case, the total number of vectors
discarded is m + 1, this proves our result. .
Exercise Set 1.5
1) What is the dimension of Rn ? [Justify your answer.]
2) Let n ∈ N and let Zn be as in Example 1.1.2. Find a basis for Zn and
show that your answer is correct. Hence determine the dimension
of Zn .
3) Let m, n ∈ N and let Rm×n be as in Example 1.1.3. Find a basis
for Rm×n and show that your answer is correct. Hence determine
the dimension of Rm×n .
4) Let B = {b1 , b2 , . . . , bn } be a basis for a real vector space V. Let
Let V be a real n-dimensional vector space. Thus V has some basis,
say B = {b1 , b2 , . . . , bn }, containing n vectors. We have mentioned in
Section 1.5 that in this case V has infinitely many bases. Thus there are
infinitely many coordinate systems for V as well. In this section, given a
basis B for V, we characterize all other bases for V. We also show how
to change from one coordinate system to another.
We begin by showing that V has infinitely many bases.
Theorem 1.6.1. Let B = {v1 , v2 , . . . , vn } be a basis for a real vector
space V and let D = {w1 , w2 , . . . , wm } be a family of vectors in V. Then
D is a basis for V if and only if m = n and there is an invertible matrix
A = [αij ]ni,j=1 such that wj = α1j v1 + α2j v2 + · · · + αnj vn for all j.
Before proving this result, we recall the following fact usually proved in
an introductory lineary algebra course. A square matrix A is invertible if
and only if the matrix-vector equation Ax = 0 has unique solution x = 0.
Proof. (⇒) Suppose D is a basis for V. Then m = n by Theorem 1.5.4.
Also, since B is a basis for V and wj ∈ V for all j, there are scalars αij

22 23
with i, j = 1, 2, . . ., n such that wj = α1j v1 + α2j v2 + · · · + αnj vn for Regrouping on the right side above,
all j. Define A = [αij ]ni,j=1 . To show that A is invertible we show that
the equation Ax = 0 has unique solution x = 0. 0 = (α11 β1 + α12 β2 + · · · + α1n βn )v1
Clearly, x = 0 is a solution. Suppose now that x = [β1 , β2 , . . . , βn ]T + (α21 β1 + α22 β2 + · · · + α2n βn )v2
is also a solution. Thus + ·······························
αi1 β1 + αi2 β2 + · · · + αin βn = 0 + (αn1 β1 + αn2 β2 + · · · + αnn βn )vn

for all i. Then Since B is linearly independent, all coefficients on the right side above
0 = 0v1 + 0v2 + · · · + 0vn are 0. That is, αi1 β1 + αi2 β2 + · · · + αin βn = 0 for i = 1, 2, . . ., n.
Equivalently,
= (α11 β1 + α12 β2 + · · · + α1n βn )v1    
β1 0
+ (α21 β1 + α22 β2 + · · · + α2n βn )v2  β2   0 
A   
 ..  =  ... 
+ ······························· .
+ (αn1 β1 + αn2 β2 + · · · + αnn βn )vn βn 0

Regrouping terms on the right side above, Since A is invertible, βi = 0 for all i. Thus D is linearly independent.
Since D also contains n vectors, D spans V by Theorem 1.5.6. Thus D
0 = β1 (α11 v1 + α21 v2 + · · · + αn1 vn ) is a basis for V.
+ β2 (α12 v1 + α22 v2 + · · · + αn2 vn )
Suppose now that V is a real n-dimensional vector space and B =
+ ······························· {v1 , v2 , . . . , vn } and D = {w1 , w2 , . . . , wn } are bases for V. Then for
+ βn (α1n v1 + α2n v2 + · · · + αnn vn ) each v ∈ V, there are scalars β1 , β2 , . . ., βn and δ1 , δ2 , . . ., δn such that
= β1 w1 + β2 w2 + · · · + βn wn
v = β1 v 1 + β2 v 2 + · · · + βn v n , v = δ1 w1 + δ2 w2 + · · · + δn wn .
Since D is a basis for V, it is linearly independent. Thus βi = 0 for all i.
Since the equation Ax = 0 has unique solution x = 0, A is invertible.
Alternatively, we may write
(⇒) Suppose now that m = n, there are scalars αij such that wj =
α1j v1 + α2j v2 + · · · + αnj vn for all j, and A = [αij ] is invertible. We   
β1 δ1
show first that D = {w1 , w2 , . . . , wn } is linearly independent. Suppose  β2   δ2 
β1 , β2 , . . ., βn are scalars such that β1 w1 + β2 w2 + · · · + βn wn = 0. Then v= 
 ..  , v= 
 ...  (1.7)
.
0 = β1 (α11 v1 + α21 v2 + · · · + αn1 vn ) βn B δn D

+ β2 (α12 v1 + α22 v2 + · · · + αn2 vn ) We now show how these columns of coordinates are related.
+ ······························· Since B is a basis for V it is, among other things, a spanning set
+ βn (α1n v1 + α2n v2 + · · · + αnn vn ) for V. Since wj ∈ V for all j, there are scalars αij with i, j = 1, 2, . . ., n

24 25
such that so that x and y are ordinary columns in Rn . Then the columns of
      coordinates for v relative to B and D, respectively, satisfy the relation
α11 α12 α1n
 α21   α22   α2n  S −1 y = x. Alternatively, Sx = y. We call S the change-of-basis matrix
w1 =  
 ...  , w2 =  
 ...  , ..., wn =  
 ...  (1.8) from the B- to the D-system of coordinates, while S −1 is the change-of-
αn1 αn2 αnn basis matrix from the D- to the B-system.
B B B
To summarize the foregoing discussion, we have the following al-
Let v ∈ V be a vector with B- and D-system coordinates as in (1.7). As
gorithm for changing from one system of coordinates to another. Let
noted above, the latter relation means that
B = {v1 , v2 , . . . , vn } be a given basis for V and let D = {w1 , w2 , . . . , wn }
δ1 w1 + δ2 w2 + · · · + δn wn = v. be another basis. To produce the change-of-basis matrix from the B- to
the D-system of coordinates:
Representing w1 , w2 , . . ., wn , and v in their B-system coordinates,
        1) For j = 1, 2, . . . , n, produce the family of coordinates α1j , α2j ,
α11 α12 α1n β1
 α21   α22   α2n   β2  . . ., αnj for wj relative to B. Thus each wj may be represented as
δ1      
 ...  + δ2  ...  + · · · + δn  ...  =  .. 
  in (1.8);
.
αn1 αn2 αnn βn 2) Define S −1 = [αij ]n×n . (This is the change-of-basis matrix for pass-
B B B B
ing from the D- to the B-system of coordinates.)
Following the arithmetic established in Problem 1 of Exercise Set 1.5, we
obtain 3) Compute S = (S −1 )−1 . This is the desired change-of-basis matrix.
   
α11 δ1 + α12 δ2 + · · · + α1n δn β1
 α21 δ1 + α22 δ2 + · · · + α2n δn   β2  Exercise Set 1.6
   
 · · · · · · · · · · · · · · · · · · · · · · · · · · · ·  =  .. 
. 2
1) In R let B = {v1 , v2 }, where
αn1 δ1 + αn2 δ2 + · · · + αnn δn B βn B    
Since the coordinates of v relative to B are unique, this yields the simul- −2 4
v1 = , v2 =
3 −3
taneous scalar equations

 α11 δ1 + α12 δ2 + · · · + α1n δn = β1 a) Find S, the change-of-basis matrix from the E- to the B-system


 of coordinates.
α21 δ1 + α22 δ2 + · · · + α2n δn = β2
 ······························ b) Find the B-coordinates for x and y, where


    
αn1 δ1 + αn2 δ2 + · · · + αnn δn = βn 5 7
x= , y=
Define S −1 = [αij ]n×n . Note that S −1 is, indeed, invertible by our last 3 −4
result. Define also
    2) In R3 let B = {v1 , v2 , v3 }, where
β1 δ1
 β2   δ2       
x=  y=  −2 −1 2
 ..  ,  ...  , v1 =  1  , v2 =  0  , v3 =  1 
.
βn δn 0 1 1

26 27
 a) Find S, the change-of-basis matrix from the E- to the B-system
of coordinates.
b) Find the B-coordinates for x and y, where
   
1 2
x = 2, y = 1
3 1

3) In Z2 , let p0 , p1 , and p2 denote the respective polynomials 1, x,

and x2 , and set B = {p0 , p1 , p2 }. Let also q0 , q1 , and q2 denote the
respective polynomials

1 + x − x2 , 1 − x + x2 , −1 + x + x2 .

and set D = {q0 , q1 , q2 }. Then B and D are each bases for Z2 , and
you may assume so here.
a) Find S, the change-of-basis matrix from the B- to the D-system
of coordinates.
b) Let r(x) = 3 + 4x + 5x2 . Hence find the coordinates of r relative
to D.

28 29