Christian Rakotonirina
Institut Supérieur de Technologie d’Antananarivo, IST-T, BP 8122, Madagascar
e-mail: rakotopierre@refer.mg
June 9, 2009
Abstract
In this paper, we prove that if the matrix of the linear system is sym-
metric, the Cholesky decomposition can be obtained from the Gauss elim-
ination method without pivoting, without proving that the matrix of the
system is positive definite.
MSC : Primary 15A06
Keywords: Linear system, Gauss elimination, Cholesky decomposition
1 Introduction
The direct methods for solving linear system , Gauss elimination method, LU
decomposition and Cholesky method are well known. We agreed with some
authors [1],[2] that the LU decomposition and the Cholesky method are helpful
for solving many linear systems of the same matrix, whose difference is only the
constants at the right hand side.
The Gauss elimination method with or without pivoting can lead us to the
LU decomposition. The Gauss elimination method with pivoting applied to
linear system with symmetric matrix can’t lead us to the Cholesky decomposi-
tion because pivoting alters the symmetry. However, sometimes we don’t need
pivoting.
Since the Gauss elimination can lead us to the LU decomposition, we think
that it is better to solve at first one of these linear systems by the Gauss elim-
ination and the others by LU decomposition. So, we will be allowed to solve
at first one of these linear systems by the Gauss eliminaton and the others by
Cholesky method, in the case where the matrix is symmetric positive definite,
if Gauss elimination can lead us to the Cholesky method.
In this paper, we will see that there exist relation between Gauss elimination
without pivoting and the Cholesky method. That is Gauss elimination without
pivoting can lead us to Cholesky decomposition. We will prove this relation by
using the LU decomposition.
We organize the paper as the following. In the second section we represent
the LU decomposition by using the Gauss elimination method. In the third
1
section we will prove, with the help of the LU decomposition we can have the
Cholesky decomposition from the Gauss elimination method. At the last section
an example will be tread for clarify the method.
2 Gauss elimination
Let us consider the following system of linear n equations and n unknowns
a11 x1 + a12 x2 + . . . + a1n xn = b1
a21 x1 + a22 x2 + . . . + a2n xn = b2
...........................
a i1 x1 + ai2 x2 + . . . + ain xn = bi
. . . . . . . . . . . . . . . . . . . . . . . . . . .
an1 x1 + an2 x2 + . . . + ann xn = bn
which can be written under matricial form
AX = B (1)
where
x1 b1
x2 b2
A = (aij )1≤i,j≤n , X = . , B = .
.. ..
xn bn
If a11 6= 0, Gauss elimination in the first column is written
(0) (0) (0) (0)
a11 x1 + a12 x2 + . . . + a1n xn = b1
(1) (1) (1)
a22 x2 + . . . + a2n xn = b2
... ... ... ... ... ......
(1) (1) (1)
ai2 x2 + . . . + ain xn = bi
... ... ... ... ... ......
(1) (1) (1)
an2 x2 + . . . + ann xn = bn
(0) (1) ai1 a1j −a11 aij
with a1j = a1j and aij = − a11 ,
A1 X = B1 (2)
and can be obtained in multiplying (1) by the lower triangular matrix[3], [4]
2
1 0 0 ... 0
a(0)
− (0)21
1 0 ... 0
a11
(0)
G1 = − a(0)
a11
31
0 1 ... 0
.. .. .. . . .
. . . . ..
(0)
an1
− (0) 0 0 ... 1
a11
A1 = G1 A and B1 = G1 B.
A2 X = B2
with A2 = G2 G1 A and B2 = G2 G1 B.
U X = B′ (3)
3
1 0 0 0
... ...
0 1 ... 0 ... 0
.. .. .. .. ..
.
. . . .
G−1 =
l 0 0 ... 1 ... 0
(l−1)
a(l+1)l
0 0 ... (l−1) ... 0
all
.. .. .. .. ..
. . . . .
(l−1)
anl
0 0 ... (l−1) ... 1
all
So, Gn−1 Gn−2 · · · G2 G1 is an invertible matrix and the equation (3) becomes
LU X = B
with L = G−1 −1 −1 −1
1 G2 · · · Gn−2 Gn−1 and we have the decomposition of A as product
of lower triangular matrix by an upper triangular matrix
A = LU
4
3 Cholesky method
Now suppose A is a symmetric positive definite matrix. Then, the Cholesky
method consists to decompose A as the product
A = GT G
√
with uii a root squared of the complex number uii .
Proof.
(0) (0) (0)
a11 a12 ... a1n
(1) (1)
0 a22 ... a2n
U (A) = .. .. .. ..
. . . .
(n−1)
0 0 . . . ann
5
1 0 ... 0 ... 0
(0)
a21
1 0 0
(0) ... ...
a11
(0)
a31 a32
(1) .. .. ..
(0) (1)
. . .
a11 a22
L(A) =
.. ..
. . ... 1 ... 0
(0)
a(l+1)1
(1)
a(l+1)2
(l−1)
a(l+1)l ..
(0) (1) ... (l−1)
. 0
a11 a22 all
.. .. .. ..
. . . .
(0) (1) (l−1)
an1 a(n)2 anl
(0) (1) ... (l−1) ... 1
a11 a22 all
A = L(A)D−1 DU (A)
with
q1 0 0
(0)
...
a11
0 q1 ... 0
(1)
a22
D=
.. .. .. ..
. . . .
√ 1
0 0 ... (n−1)
ann
x1 − x2 + x4 =3
−x1 + 5x2 + 2x3 − 3x4 = −5
2x2 + 5x2 + x4 = −7
x1 − 3x2 + x3 + 4x4 =2
x1 − x2
+ x4 =3
−x1 + 5x2 + 2x3 − 3x4 =1
2x2 + 5x2 + x4 =2
x1 − 3x2 + x3 + 4x4 =2
whose difference is only the right hand side and their matrix is symmetric. So
let us solve the first one by the Gauss elimination method, and the second one
6
by the Cholesky method or LU method.
1 0 0 0 1 −1 0 1 x1 1 0 0 0 3
1 1 0 0 −1 5 2 −3 x2 1 1 0 0 −5
=
0 0 1 0 0 2 5 1 x3 0 0 1 0 −7
−1 0 0 1 1 −3 1 4 x4 −1 0 0 1 2
1 −1 0 1 x1 3
0 4 2 −2 x2 = −2
0 2 5 1 x3 −7
0 −2 1 3 x4 −1
1 0 0 0 1 −1 0 1 x1 1 0 0 0 3
0 1 0 0 0 4 2 −2 x2 0 1 0 0 −2
0 −1 1 0 0 2 x3 = 0 − 1 1
2 5 1 2 0 −7
0 12 0 1 0 −2 1 3 x4 0 12 0 1 −1
1 −1 0 1 x1 3
0 4 2 −2 x2 −2
=
0 0 4 2 x3 −6
0 0 2 2 x4 −2
1 0 0 0 1 −1 0 1 x1 1 0 0 0 3
0 1 0 0 0 4 2 −2 x2 0 1 0 0 −2
=
0 0 1 0 0 0 4 2 x3 0 0 1 0 −6
0 0 − 21 1 0 0 2 2 x4 0 0 − 21 1 −2
1 −1 0 1 x1 3
0 4 2 −2 x2 = −2
0 0 4 2 x3 −6
0 0 0 1 x4 1
x4 = 1, 4x3 + 2 = −6, x3 = −2, 4x2 − 4 − 2 = −2, x2 = 1, x1 − 1 + 1 = 3,
x1 = 3
Now, let us move to the second equation, which
can be written, after the
T GT Y = B
Proposition 2, G GX = B and solved in writing
GX = Y
1 0 0 0 y1 3
−1 2 0 0 y2 1
=
0 1 2 0 y3 2
1 −1 1 1 y4 2
y1 = 3, 3 + 2y2 = 1, y2 = 2, 2 + 2y3 = 2, y3 = 0, 3 − 2 + y4 = 2, y4 = 1
1 −1 0 1 x1 3
0 2 1 −1 x2 = 2
0 0 2 1 x3 0
0 0 0 1 x4 1
x4 = 1, 2x3 + 1 = 0, x3 = − 21 , 2x2 − 12 − 1 = 2, x2 = 47 , x1 − 47 + 1 = 3, x1 = 15
4
7
References
[1] B. Démidovitch and I. Maron. Eléments de Calcul Numérique.Editions Mir,
1979.
[2] J.P. Nougier. Méthodes de Calcul Numérique. Masson. Paris, 1987.
[3] David Kincaid and Ward Cheney. Numerical Mathematics and Computing.
Brooks/Cole Publishing Co., Pacific Grove, CA, fourth edition, 1999. ISBN
0-534-35184-0.
[4] E. Pav Steven. Numerical Methods Course Notes.
http://scicomp.ucsd.edu/∼spav/pub/numas.pdf