LECTURE 3
DIRECT SOLUTIONS TO LINEAR ALGEBRAIC SYSTEMS - CONTINUED
Ill-conditioning of Matrices
There is no clear cut or precise definition of an ill-conditioned matrix.
Effects of ill-conditioning
Roundoff error accrues in the calculations
Can potentially result in very inaccurate solutions
Small variation in matrix coefficients causes large variations in the solution
Detection of ill-conditioning in a matrix
An inaccurate solution for X can satisfy an ill-conditioned matrix quite well!
Apply back substitution to check for ill-conditioning
Solve AX = B through Gauss or other direct method X poor
Back substitute AX poor B poor
Comparing we find that B poor B
p. 3.1
p. 3.2
aij2
i=1 j=1
where
Euclidean Norm of A
i=1
j=1
aij2
If the matrix A is diagonally dominant, i.e. the absolute values of the diagonal terms
the sum of the off-diagonal terms for each row, then the matrix is not ill-conditioned
N
a ii
i = 1, 2, , N
a ij
j = 1
i j
p. 3.3
Effects of ill-conditioning are most serious in large dense matrices (e.g. especially those
obtained in such problems as curve fitting by least squares)
Sparse banded matrices which result from Finite Difference and Finite Element methods
are typically much better conditioned (i.e. can solve fairly large sets of equations
without excessive roundoff error problems)
Ways to overcome ill-conditioning
Make sure you pivot!
Use large word size (use double precision)
Can use error correction schemes to improve the accuracy of the answers
Use iterative methods
p. 3.4
O ( N 3 ) + O ( N 2 ) steps
AX = B 2
O ( N 3 ) + O ( N 2 ) steps
.
.
.
AX = B R
.
.
.
O ( N 3 ) + O ( N 2 ) steps
Using Gauss elimination, O ( N 3 R ) operations, where N = size of the system of equation and R = the number of different load vectors which must be solved for
Concept of the factor method is to facilitate the solution of multiple right hand sides
without having to go through a re-triangulation process for each B r
p. 3.5
Factorization step
Given A , find P and Q such that
A = PQ
P Q 0
a 11 a 12 a 13
p 11
p 12
p 13
a 21 a 22 a 23 =
p 21
p 22
p 23 q 21 q 22 q 23
a 31 a 32 a 33
p 31
p 32
p 33
q 11 q 12 q 13
q 31 q 32 q 33
i = 1, N
Doolittle Method
q ii = 1
i = 1, N
Crout Method
Now we only have N 2 unknowns! We can solve for all unknown elements of P and Q by
proceeding from left to right and top to bottom
a 11 a 12 a 13
a 21 a 22 a 23
1
= p 21
a 31 a 32 a 33
p 31
0
1
q 11 q 12 q 13
0
0 0 q 22 q 23
p 32 1
q 33
q 11
q 12
q 13
p 21 q 11
p 21 q 12 + q 22
p 21 q 13 + q 23
p 31 q 11
p 31 q 12 + p 32 q 22
p 31 q 13 + p 32 q 23 + q 33
a 11 a 12 a 13
a 21 a 22 a 23 =
a 31 a 32 a 33
p. 3.7
Factorization proceeds from left to right and then top to bottom as:
Red current unknown being solved
Blue unknown value already solved
a 11 = q 11
a 12 = q 12
a 13 = q 13
a 21 = p 21 q 11
a 22 = p 21 q 12 + q 22
a 23 = p 21 q 13 + q 23
a 31 = p 31 q 11
a 32 = p 31 q 12 + p 32 q 22
a 33 = p 31 q 13 + p 32 q 23 + q 33
p. 3.8
We can compute the elements of P and Q and store them directly into the old A matrix
as the procedure progresses
A = PQ
where
rename L
rename U
p. 3.9
Now let
Y = UX
Apply a forward substitution sweep to solve for Y for the system of equations
LY = B
Apply a backward substitution sweep to solve for X for the system of equations
UX = Y
p. 3.11
p. 3.12
R O( N 3)
LU factorization costs
LU Factorization Cost = [ O ( N 3 ) ]
R O( N 2)
Gauss Elim.
O ( RN )
O ( RN )
O( N )
1,000
5,000
5x1012
5x109
1,000
105
106
1021
1016
100,000
LU Factorization
Ratio of Costs
p. 3.13
or
a ij = a ji
or
A = UT U
p. 3.14
We note that A contains 6 independent entries and that L contains 6 unknowns and therefore we can perform the factorization
a 11 a 12 a 13
A =
L = l 21 l 22
l 11
a 21 a 22 a 23
l 31 l 32 l 33
a 31 a 32 a 33
Substituting the factored form of the matrix and changing the order in which products
are taken
L( LT X ) = B
by forward substitution
L T X = Y by backward substitution
p. 3.15
___________
LDLT Method
Decompose A = LDL T
Where L is a lower triangular matrix
Where D = diagonal matrix
Set the diagonal terms of L to unity
Solving for the elements of L and D
A = LDL T
Substituting and changing the order in which the products are formed
L ( DL T X ) = B
Now let
DL T X = Y
p. 3.16
p. 3.17
A = LU
A = L U
The determinant of a triangularized matrix equals the product of the diagonal terms
In case of LU factorization
N
A =
lii
(Crout)
i=1
N
A =
uii
(Doolittle)
i=1
A = D =
d ii
i=1
For Gauss Elimination, simply keep a running product of the pivot elements.
p. 3.18