Numerical Methods

Process Systems Engineering

LINEAR EQUATIONS

Numerical methods in chemical

engineering
Edwin Zondervan

1
 Numerical Methods
Process Systems Engineering
For who was thinking that The Matrix is a product of Neo, Trinity and Morpheus …

2
 Numerical Methods
Process Systems Engineering
I have to say: It’s all about Leibniz, Cramer and Gauss

Leibniz Cramer
~1690 ~1720

Gauss
~1810

3
 Numerical Methods
Process Systems Engineering

OVERVIEW
• Different ways of looking at a system of
linear equations (Matrices)
• The inverse, determinant and rank of a
matrix
• The existence of a solution to a set of
linear equations
• Eigenvalues of a matrix

4
 Numerical Methods
Process Systems Engineering

MATLAB
• Vector and matrix • It is an interpreted
operations are an language (in
intrinsic part of matlab reference to a
• High level: don’t worry compiler): makes it
about memory sometimes slow
management/allocatio
n
• Close to ‘proper’
languages
• Command line
interface user friendly
5
 Numerical Methods
Process Systems Engineering

MATLAB SCREEN SHOT

6
 Numerical Methods
Process Systems Engineering

DIFFERENT VIEWS OF LINEAR SYSTEMS

Separate equations Matrix mapping Linear combination
x y z 4 1 1 1 x 4 1 1 1 4
2x y 3z 7 2 1 3 y 7 x 2 y1 z 3 7
3 1 6 z 5 3 1 6 5
3x y 6z 5
(2-1) (2-2) (2-3)

z T z
r x x, y , z T
r Mx 4,7,5

y y

x x

7
 Numerical Methods
Process Systems Engineering

INVERSE OF A MATRIX
• Inverse is defined such that:
1 1
MM I and M M I (2-4)

• Use the inverse to solve a set of linear

equations:
(2-5)
Mx b
1 1 (2-6)
M Mx M b
1
Ix M b (2-7)

1
x M b (2-8)

8
 Numerical Methods
Process Systems Engineering

HOW TO CALCULATE AN INVERSE?

• Inverse of an NxN matrix, can be calculated
using the co-factors of each element of the
matrix: T
C11 C12 C13
1 1
M C21 C22 C23 (2-9)
det(M )
C31 C32 C33

Det(M) = the determinant of M and

Cij is the co-factor of the ijth element in M.
9
 Numerical Methods
Process Systems Engineering

AN EXAMPLE
• The example:

1 1 1
(2-10)
M 2 1 3
3 1 6

• Let’s calculate C11

10
 Numerical Methods
Process Systems Engineering

CO-FACTORS
• A co-factor is defined to be the
determinant of the stuff left over when you
cover up the row and column of the
element in question:
1 * * (2-11)

* 1 3
* 1 6
11
 Numerical Methods
Process Systems Engineering

THE NEXT STEP

• Also multiply by +/-1, depending of the
position of the element, for the 3x3 matrix,
the following table holds:

(2-12)

12
 Numerical Methods
Process Systems Engineering

CALCULATING THE INVERSE

• So, the co-factor C11 can be calculated as:
1 3
C11 det 6 1 3 1 3
1 6

1 1 1 1 * *
M 2 1 3 * 1 3
3 1 6 * 1 6
13
 Numerical Methods
Process Systems Engineering

BACK TO THE EXAMPLE

• You’ll find:
1 T
1 1 1 3 5 2
1
2 1 3 3 3 1
det(M )
3 1 6 1 2 1

Determinant is very important.

Determinant = 0 The inverse does not exist!! (The
matrix is singular)

14
 Numerical Methods
Process Systems Engineering

CALCULATING THE INVERSE

• Compute the determinant by multiplication
of each element on a row (or column) by
its cofactor and adding the results:
1 1 1
1 3 2 3 2 1
det 2 1 3 det det det 1
1 6 3 6 3 1
3 1 6

1 1 1
2 1 1 1 1 1
det 2 1 3 det 3 det 6 det 1
3 1 3 1 2 1
3 1 6
15
 Numerical Methods
Process Systems Engineering

SOLVING A LINEAR SYSTEM

• Our example: x y z 4
2 x y 3z 7
3x y 6 z 5
• The solution is: The inverse exists, because det(M)=-1

x 3 5 2 4 13 13
1 1
y 3 3 1 7 4 4
1 1
z 1 2 1 5 5 5

16
 Numerical Methods
Process Systems Engineering

LARGE SYSTEMS
• Computation of determinants and inverses
of large matrices in this way is too difficult
(slow), so we need other methods to
calculate the inverse of a large matrix
(large systems).

17
 Numerical Methods
Process Systems Engineering

USEFUL PROPERTIES
• Triangular matrices:
(2-14) 5 3 2
n
det( M trian ) aii M 0 9 1 det( M ) 5 9 1 45
i 1 0 0 1
• Matrix multiplication:

det( AM ) det( A) * det(M )

a 0 0
A 0 1 0 det( AM ) det( A) * det(M ) a * det(M )
0 0 1

18
 Numerical Methods
Process Systems Engineering

USEFUL PROPERTIES
• Matrix multiplication:
1 1 0
A 0 1 0 ; det( A) 1
0 0 1

det( AM ) det( A) * det(M ) det(M )

Using rules like this, you can compute the determinant of a

matrix using row operations!

19
 Numerical Methods
Process Systems Engineering

MATRIX RANK
• Rank of a matrix: the number of linearly
independent columns (columns that can
not be expressed as a linear combination
of the other columns) of a matrix
Column 2 and 4 are not independent
5 3 2
1 2 1 0
M 0 9 1 M 0 0 1 1
0 0 1 0 0 0 0
3 independent columns: 2 independent columns:
rank(M)=3 Rank(M)=2

20
 Numerical Methods
Process Systems Engineering

SOLUTIONS OF LINEAR SYSTEMS

• The solution of a system of linear
equations may or may not exist, and it
may or may not be unique. Existence of
solutions can be determined by comparing
the rank of the Matrix M with the rank of
the augmented matrix Ma. Linear system

a11 a21 a31 b1 a11 a21 a31 b1

M a12 a22 a32 , b b2 Ma a12 a22 a32 b2
a13 a23 a33 b3 a13 a23 a33 b3
21
Mx b
 Numerical Methods
Process Systems Engineering

SOLUTIONS OF LINEAR SYSTEMS

• Rank(M)=n • Unique solution

• Rank(M) < n and • Infinite number of

rank(M)=rank(Ma) solutions

• Rank(M) < n and • No solution

rank(M)<rank(Ma)

22
 Numerical Methods
Process Systems Engineering

TWO EXAMPLES
1 1 2 17 1 1 2 17
M 0 3 1 ,b 11 Ma 0 3 1 11
0 0 2 4 0 0 2 4
Rank(M) = 3 = n  unique solution

Rank, solution??

1 1 2 17 1 1 2 17
M 0 3 1 ,b 11 Ma 0 3 1 11
0 0 0 0 0 0 0 0
Rank(M)=rank(Ma)=2 < n  infinite number of solutions
23
 Numerical Methods
Process Systems Engineering

EIGENVALUES AND
EIGENVECTORS
• Matrices have characteristic directions,
called eigenvectors, e:

Me e (2-15)

• is called the eigenvalue

24
 Numerical Methods
Process Systems Engineering

FINDING EIGENVALUES
• An example:
1 0 1
M 0 1 0
1 0 1
• From: Me= e

Me I e 0
M I e 0

25
 Numerical Methods
Process Systems Engineering

Calculating
• Find : previous equation has infinitely
many solutions for e, so:
det M I 0

1 0 1
det M I det 0 1 0 0
1 0 1

26
 Numerical Methods
Process Systems Engineering

Calculating , continued

det(M I )
1 0 0 0 0 1
(1 ) det 0 det 1det
0 1 1 1 1 0
(1 ){(1 ) 2 0} {0 1(1 )} 0

To find the eigenvalues, we need to find the roots of the characteristic

equation: in this case, 1 = 0, 2 =1 and 3 = 2, TRY FOR YOUR SELF!

27
 Numerical Methods
Process Systems Engineering

SPECTRAL DECOMPOSITION
Me1 e
1 1

M M M M M M 1 0 0
Me2 e
2 2 M e1 e2 e3 e1 e2 e3 0 2 0
M M M M M M 0 0 3

Me3 e
3 3

More compactly written as:

(2-17)
MU U
1
M U U (2-18)

28
 Numerical Methods
Process Systems Engineering

SUMMARY
• Linear equations can be written as
matrices
• A solution depends on the rank of a matrix
• Eigenvalues and eigenvectors are
characteristics of a matrix

29