Linear Algebra
a1 x1 + a2 x2 + · · · + an xn = b
1
MATH10212 • Linear Algebra • Brief lecture notes 2
x−y−z = −4
y + 3z = 11
5z = 15
z = 3, y = 11 − 3 · 3 = 1,
x−y+z = −1
y+z = 1
2 1 −1
1 0 5
−1 3 −2
and the augmented matrix is
¯
2 1 −1 ¯ 3
¯
1 0 5 ¯ 1
¯
−1 3 −2 ¯ 0
Remarks
Proof. Suppose that one system (old) is transformed into a new one by
an elementary row operation (of one of the types 1, 2, 3). (Clearly, we only
need to consider one e.r.o.) Let S1 be the solution set of the old system, and
S2 the solution set of the new one. We need to show that S1 = S2 .
First it is almost obvious that S1 ⊆ S2 , that is, every solution of the old
system is a solution of the new one. Indeed, if it was type 1, then clearly
MATH10212 • Linear Algebra • Brief lecture notes 5
nothing changes, since the solution set does not depend on the order of
equations. If it was e.r.o. of type 2, then only the ith equation changes: if
ai1 u1 + ai2 u2 + · · · + ain un = bi (old), then kai1 u1 + kai2 u2 + · · · + kain un =
k(ai1 u1 + ai2 u2 + · · · + ain un ) = kbi (new), so a solution (u1 , . . . , un ) of the old
system remains a solution of the new one. Similarly, if it was type 3: only
the ith equation changes: if [u1 , . . . , un ] was a solution of the old system,
then both ai1 u1 + ai2 u2 + · · · + ain un = bi and aj1 u1 + aj2 u2 + · · · + ajn un = bj ,
whence by adding the second times k to the second and collecting terms
we get (ai1 + kaj1 )u1 + (ai2 + kaj2 )u2 + · · · + (ain + kajn )un = bi + kbj ,
so [u1 , . . . , un ] remains a solution of the new system. Thus, in each case,
S1 ⊆ S2 .
But by Lemma on inverses each e.r.o. has inverse, so the old system
can also be obtained from the new one by an elementary row operation.
Therefore, by the same argument, we also have S2 ⊆ S1 . Since now both
S2 ⊆ S1 and S1 ⊆ S2 , we have S2 = S1 , as required.
(b) Create a leading entry at the top of this column using type 1 e.r.o.
R1 ↔ Ri . (It helps if you make this leading entry = 1, if necessary
using type 2 e.r.o. (1/k)R1 .)
(c) Use the leading entry to create zeros below it: kill off all the entries of
this column below the leading, using type 3 e.r.o. Ri − aR1 .
(d) Cover (ignore) the first row containing the leading entry, and repeat
steps (a), (b), (c) on the remaining submatrix.
...And so on, every time in (d) ignoring several upper rows with the al-
ready created leading entries. Stop when the entire matrix is in row echelon
form.
MATH10212 • Linear Algebra • Brief lecture notes 6
It is fairly obvious that this procedure always works. There are no solu-
tions if and only if a “bad” row appears 0, 0, . . . , 0, b with b 6= 0: indeed, then
nothing can satisfy this equation 0x1 + · · · + 0xn = b 6= 0. Variables cor-
responding to leading coefficients are leading variables; all other variables
are free variables (possibly, none — then solution is unique). Clearly, when
we back-substitute, free variables can take any values (“free”), while lead-
ing variables are uniquely expressed in terms of free variables and “lower”
leading variables, which in turn are..., so in fact in the final form of solution
leading variables are uniquely expressed in terms of free variables only,
while free variables can take independently any values. In other words,
free variables are equal to independent parameters, and leading variables
are expressed in these parameters.
Gauss–Jordan Elimination:
1. Write the augmented matrix of the system of linear equations.
2. Use elementary row operations to reduce the augmented matrix to
reduced row echelon form. (In addition to (c) above, also kill off all
entries )i.e. create zeros) above the leading one in the same column.)
3. If there is a “bad row”, then there are no solutions. If there is no
“bad row” (i.e. the resulting system is consistent), then express the
leading variables in terms of the constant terms and any remaining
free variables.
A bit more work to r.r.e.f., but then much easier expressing leading vari-
ables in terms of the free variables.
Corollary Every consistent linear system over R has either a unique so-
lution (if there are no free variables, so all variables are leading), or in-
finitely many solutions (when there are free variables, which can take arbi-
trary values).
Remark If one needs a particular solution (that is, just any one solution),
simply set the parameters (leading var.) to any values (usually the simplest
is to 0s). E.g. general solution {[1 − t + 2u, t, 3 + u, u] | t, u ∈ R}; setting
t = u = 0 we get a particular solution [1, 0, 3, 0]; or we can set, say, t = 1
and u = 2, then we get a particular solution {[4, 1, 5, 2], etc.
Definition The rank of a matrix is the number of nonzero rows in its row
echelon form. We denote the rank of a matrix A by rank(A).
Homogeneous Systems
Definition A system of linear equations is called homogeneous if the con-
stant term in each equation is zero.
In other words, a homogeneous system has an augmented matrix of the
form [A|~0]. E.g., the following system is homogeneous:
x + 2y − 3z = 0
−x + y + 2z = 0
The next theorem says that the latter case must occur if the number of
variables is greater than the number of equations.
and
1 2 3 4
0 −1 −2 −3
0 0 0 1
are row equivalent.
Theorem 2.1 Matrices A and B are row equivalent if and only if they can
be reduced to the same row echelon form.
MATH10212 • Linear Algebra • Brief lecture notes 9
with coefficients c1 , . . . , ck ∈ R is
Obviously, to ask whether a vector ~b belongs to the span of vectors ~v1 , . . . , ~vk
is exactly the same as to ask whether ~b is a linear combination of the vectors
~v1 , . . . , ~vk ; see Theorem 2.4 and the method described above.
Linear (in)dependence
Definition. A set of vectors
is linearly dependent if there are scalars c1 , c2 , . . . , ck at least one of which is not zero,
such that
c1~v1 + c2~v2 + · · · + ck~vk = ~0
A set of vectors that is not linearly dependent is called linearly independent.
In other words, vectors {~v1 , ~v2 , . . . , ~vk } are linearly independent if equality
c1~v1 + c2~v2 + · · · + ck~vk = ~0 implies that all the ci are zeros (or: only the
trivial linear combination of the vi is equal to ~0).
Remarks.
• Since
0~v1 + 0~v2 + · · · + 0~vk = ~0
for any vectors ~v1 , ~v2 , . . . , ~vk , linear dependence essentially says that
the zero vector can be expressed as a nontrivial linear combination of
~v1 , ~v2 , . . . , ~vk . Thus, linear independence means that the zero vector
can be expressed as a linear combination of ~v1 , ~v2 , . . . , ~vk only in the
trivial way:
c1~v1 + c2~v2 + · · · + ck~vk = ~0
only if c1 = 0, c2 = 0, . . . , ck = 0.
MATH10212 • Linear Algebra • Brief lecture notes 11
Theorem 2.6. Let ~v1 , ~v2 , . . . , ~vm be (column) vectors in Rn and let A be the
n × m matrix
A = [~v1 |~v2 | · · · |~vm ]
with these vectors as its columns. Then ~v1 , ~v2 , . . . , ~vm are linearly dependent
if and only if the homogeneous linear system with augmented matrix [A|~0]
has a nontrivial solution.
Proof. ~v1 , ~v2 , . . . , ~vm are linearly dependent if and only if there are scalars
c1 , c2 , . . . , cm not all zero, such that
By Theorem 2.4, this is equivalent to saying that the system with the aug-
mented matrix [~v1 |~v2 | . . . |~vm |~0] has a non-trivial solution.
Method for determining if given vectors ~v1 , ~v2 , . . . , ~vm are linearly depen-
dent: form the homogeneous system as in Theorem 2.6 (unknowns are those
coefficients). Reduce its augmented matrix to r.e.f. If there are no non-
trivial solutions (= no free variables), then the vectors are linearly indepen-
dent. If there are free variables, then there are non-trivial solutions and
the vectors are dependent. To find a concrete dependence, find a particular
non-trivial solution, which gives required coefficients; for that set the free
variables to 1, say (not all to 0).
containing the zero vector is linearly dependent. For we can find a nontriv-
ial combination of the form
by setting c1 = 1 and c2 = c3 = · · · = cm = 0.
Theorem 2.5. Vectors ~v1 , ~v2 , . . . , ~vm in Rn are linearly dependent if and
only if at least one of the vectors can be expressed as a linear combination
of the others.
Proof. If one of the vectors, say, ~v1 , is a linear combination of the others,
then there are scalars c2 , . . . , cm such that
Rearranging, we obtain
which implies that ~v1 , ~v2 , . . . , ~vm are linearly dependent, since at least one
of the scalars (namely, the coefficient 1 of ~v1 ) is nonzero.
Conversely, suppose that ~v1 , ~v2 , . . . , ~vm are linearly dependent. Then
there are scalars c1 , c2 , . . . , cm not all zero, such that
Suppose c1 6= 0. Then
and we may multiply both sides by c11 to obtain ~v1 as a linear combination
of the other vectors:
µ ¶ µ ¶
c2 cm
~v1 = − ~v2 − · · · − ~vm .
c1 c1
E.g., vectors [1, 2, 1] and [1, 1, 3] are linearly independent, as they are not
proportional. Vectors [−1, 2, −1] and [2, −4, 2] are lin. dependent, since they
are proportional (with coeff. −2).
Theorem 2.7. Let ~v1 , ~v2 , . . . , ~vm be (row) vectors in Rn and let A be the
m × n matrix
− − ~v1 − −
− − ~v2 − −
..
.
− − ~vm − −
with these vectors as its rows. Then ~v1 , ~v2 , . . . , ~vm are linearly dependent if
and only if rank(A) < m.
Note that there is no linear system in Th.2.7 (although e.r.o.s must be used
to reduce A to r.e.f.; then rank(A) = number of non-zero rows of this r.e.f.)
MATH10212 • Linear Algebra • Brief lecture notes 13
Proof. “⇒” If ~v1 , ~v2 , . . . , ~vm are linearly dependent, then by Th. 2.5 one of
these vectors is equal to a linear combination of the others. Swapping rows
by type 1 e.r.o. if necessary, we can assume that
the resulting matrix will have m-th row consisting of zeros. Next, we apply
e.r.o.s to reduce the submatrix consisting of the upper m − 1 rows to r.e.f.
Clearly, together with the zero m-th row it will be r.e.f. of A, with at most
m − 1 non-zero rows. Thus, rank(A) ≤ m − 1.
“⇐” (assumed without proof) The idea is that if rank(A) ≤ m − 1, then
r.e.f. of A has zero row at the bottom. Analysing e.r.o.s that lead from A to
this r.e.f. one can show (we assume this without proof) that one of the
rows is a linear combination of the others; see the textbook, Example 2.25.
Row Method for deciding if vectors ~v1 , . . . , ~vm are linearly dependent.
Form the matrix A with rows ~vi (even if originally you were given columns,
just “lay them down”, rotate by 900 clockwise). Reduce A by e.r.o.s to r.e.f.,
the number of non-zero rows in this r.e.f. is =rank(A). The vectors are
linearly dependent if and only if rank(A) < m. (Again: note that there is
no linear system to solve here; no unknowns, it does not matter if there is
a bad row.)
Theorem on e.r.o.s and spans. E.r.o.s do not alter the span of rows of a
matrix.
Proof. Let ~v1 , ~v2 , . . . , ~vm be the rows of a matrix A, to which we apply e.r.o.s.
Clearly, it is sufficient to prove that the span of rows is not changed by
a single e.r.o. Let ~u1 , ~u2 , . . . , ~um be the rows of the new matrix. By the
definition of e.r.o., every ~ui is a linear combination of the ~vj (most rows are
even the same). Now, in every linear combination c1 ~u1 + c2 ~u2 + · · · + cm ~um
we can substitute those expressions of the ~ui via the ~vj . Expand brackets,
collect terms: this becomes a linear combination of the the ~vj . In other
words,
span(~v1 , . . . , ~vm ) ⊇ span(~u1 , . . . , ~um ).
By Lemma on inverse e.r.o., the old matrix is also obtained from the new
one by the inverse e.r.o. By the same argument,
As a result,
span(~v1 , . . . , ~vm ) = span(~u1 , . . . , ~um ).