Special Matrices and Gauss-Seidel

Chapter 11

Credit: Prof. Lale Yurttas, Chemical Eng., Texas A&M University

1
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
• Certain matrices have particular structures
that can be exploited to develop efficient
solution schemes (e.g. banded, symmetric)

• A banded matrix is a square matrix that has

all elements equal to zero, with the exception
of a band centered on the main diagonal.

• Standard Gauss Elimination is inefficient in

solving banded equations because
unnecessary space and time would be
expended on the storage and manipulation of
zeros.

• There is no need to store or process the zeros

(off of the band)

2
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
 Solving Tridiagonal Systems
(Thomas Algorithm)
A tridiagonal system has a bandwidth of 3

 f1 g1   x1   r1  DECOMPOSITION
e   x  r 
 2 f2 g2   2    2 
 e3 f3 g 3   x3  r3  DO k = 2, n
     ek = ek / fk-1
 e4 f 4   x4  r4  fk = fk - ek gk-1
1 0 0 0  f1 g1  END DO
e' 1 0 0  f 2' g2 
A  L U   2 
0 e'3 1 0  f 3' g3  Time Complexity?
  '
0 0 e' 4 1  f4  O(n)
vs. O(n3)
3
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
 Tridiagonal Systems (cont.)
{d}
1 0 0 0  f1 g1   x1   r1 
 e' 1 0 0  f 2' g2   x  r 
 2  2    2 
0 e'3 1 0  f 3' g3   x3   r3 
     
0 0 e' 4 1  f 4'   x4  r4 

1 0 0 0  d1   r1   f1 g1   x1   d1 
e ' 0 d 2  r2     x  d 
 2 1 0
 f 2' g2  2    2 

0 e'3 1 0 d 3   r3   f 3' g3   x 3   d 3 
      '    
0 0 e' 4 1 d 4  r4   f 4   x 4  d 4 

Forward Substitution Back Substitution

d1 = r1 xn = dn /f’n
DO k = 2, n DO k = n-1, 1, -1
dk = rk – e’k dk-1 xk = (dk - gk . xk+1 )/f’k
END DO END DO 4
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
 Cholesky Decomposition
(for Symmetric Positive Definite† Matrices)
 a11 a21 a31 a41 
a a a a 
†A positive definite matrix is one for which the product [ A]  [ A]T   21 22 32 42 

{X}T[A]{X} is greater than zero for all nonzero vectors X

a31 a32 a33 a43 
 
 41 42
a a a 43 a 44 

l11  l11 l21 l31 l41 

l l  l22 l32 l42 
[ A]  [ A]  L  L  
T T 21 22  LT means Transpose of L
l31 l32 l33  l33 l43 
  
 41 42 43 44  
l l l l l44 

i 1
aki   lijlkj k 1
lkk  akk   lkj2
j 1
lki  for k  1,2,, n i  1,2,, k  1
lii j 1

Time Complexity:
O(n3) but requires half the number of operations as standard Gaussian elimination.
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
 Jacobi Iterative Method
Iterative methods provide an alternative to the elimination methods.

 a11 a12 a13  a11 0 0

Ax  b A  a21 a 22 a 23  D   0 a 22 0 
a31 a32 a33   0 0 a33 

[ D  ( A  D )] x  b  Dx  b  ( A  D ) x  x  D 1[b  ( A  D ) x ]

 x1  1/ a11 0 0    b1   0 a12 a13   x1  

x    0 1/ a 0  *  b    a a23   x2  

 2  22    2   21 0
 x3  k 1  0 0 1/ a33   b3   a31 a32 0   x3  k 

k 1 b1  a12 x2k  a13 x3k k 1 b2  a21 x1k  a23 x3k k 1 b3  a31 x1k  a32 x2k
x1  x 2  x 3 
a11 a22 a33
Choose an initial guess (i.e. all zeros) and Iterate until the equality is satisfied.
No guarantee for convergence! Each iteration takes O(n2) time! 6
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
 Gauss-Seidel
• The Gauss-Seidel method is a commonly used iterative method.

• It is same as Jacobi technique except with one important difference:

A newly computed x value (say xk) is substituted in the subsequent
equations (equations k+1, k+2, …, n) in the same iteration.

Example: Consider the 3x3 system below:

b1  a12 x2old  a13 x3old • First, choose initial guesses for the x’s.
xnew
1 
a11 • A simple way to obtain initial guesses is
to assume that they are all zero.
b2  a 21 x1new  a 23 x3old
x 2new  • Compute new x1 using the previous
a22 iteration values.
b3  a31 x1new  a32 x 2new • New x1 is substituted in the equations to
x3new 
a33 calculate x2 and x3
{ X }old  { X }new • The process is repeated for x2, x3, …
7
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
 Convergence Criterion for Gauss-Seidel Method
• Iterations are repeated until the convergence criterion is satisfied:
xij  xij 1 For all i, where j and j-1 are
 a ,i  j
100%   s the current and previous iterations.
xi

• As any other iterative method, the Gauss-Seidel method has problems:

– It may not converge or it converges very slowly.

• If the coefficient matrix A is Diagonally Dominant Gauss-Seidel is

guaranteed to converge.
For each equation i:
n
Diagonally Dominant  aii   ai , j
j 1
j i

• Note that this is not a necessary condition, i.e. the system may still have a
chance to converge even if A is not diagonally dominant.
Time Complexity: Each iteration takes O(n2) 8
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.