Notes from CHAPTER 5 Alpha Chiang LINEAR MODELS AND MATRIX ALGEBRA 5.

1 Conditions for Nonsingularity of a Matrix When squareness condition is already met, a sufficient condition for the nonsingularity of a matrix is that its rows be linearly independent. Squareness and linear independence constitute the necessary and sufficient condition for non-singularity. Nonsingularity squareness and linear independence An n x n coefficient matrix A can be considered as an ordered set of row vectors, ie., as a column vector whose elements are themselves row vectors: a11 a A = 21 L an1 a12 a22 L an1 L L L L a1n v '1 a2 n v '2 = L M an1 v 'n

where vi ' = [ ai1 ai 2 L ain ] , i = 1, 2,L , n For the rows to be linearly independent, none must be a linear combination of the rest. Example:
' 3 4 5 v1 ' A = 0 1 2 = v2 6 8 10 v3'

Since [6 8 10] = 2 [3 4 5], We have v3 = 2 v1 +2v1+0 v2. Thus the third row is expressible as a linear combination of the first two, the rows are not linearly independent. Rank of a Matrix Rank r = the maximum number of linearly independent rows that can be found in a matrix. (it also tells us the maximum number of linearly independent columns in the said matrix).

3/18/2012 Page 1 of 10

By definition, an n x n nonsingular matrix has n linearly independent rows (or columns); consequently, it must be of rank n. Conversely, an n x n matrix have a rank n must be nonsingular. TEST OF NONSINGULARITY BY USE OF DETERMINANT Determinants and Non-singularity The determinant of a square matrix A, denoted by |A|, is a uniquely defined scalar (number) associated with that matrix. Determinants are uniquely defined only for square matrices. a11 For a 2 x 2 matrix A = a21 as follows: A= Example: 10 4 3 5 Given A = and B = 0 1 8 5 their determinants are 10 4 A= = 10(5) 8(4) = 18 8 5 3 5 B = = 3(1) 0(5) = 3 0 1 Relationship between linear dependence of rows in matrix A vs. determinant c ' 3 8 d '1 2 6 C = 1 = and D = d ' = 8 24 c '2 3 8 2 both have linearly dependent rows because c1=c2 and d2=4d1. Determinants: C = D = 3 8 = 3(8) 3(8) = 0 3 8 2 6 = 2(24) 8(6) = 0 8 24 a11 a21 a12 , its determinant is defined to be the sum of two terms a22

a12 = a11a22 a21a12 a22

[= a scalar]

3/18/2012 Page 2 of 10

The result suggest that a vanishing determinant may have something to do with linear dependence. Evaluating a Third Order Determinant a11 A = a21 a31 a12 a22 a32 a13 a a23 = a11 22 a32 a33 a23 a a12 21 a33 a31 a23 a + a13 21 a33 a31 a22 a32

= a11a22 a33 a11a23 a32 + a12 a23 a31 a12 a21 a33 + a13 a21 a32 a13 a22 a31 a scalar Examples 2 1 3 4 5 6 = 9 7 8 9 7 0 3 9 1 4 = 295 0 6 5 Evaluating nth order Determinant by Laplace Expansion To sum up, the value of a determinant |A| of order n can be found by the Laplace expansion of any row or any column. (5.8) A = aij Cij
j =1 n n

[expansion by ith row] [expansion by jth row]

A = aij Cij
i =1

5.3 BASIC PROPERTIES OF DETERMINANTS Property I: The interchange of rows and columns does not affect the value of a determinant. Property II: The interchange of any two rows (or any two columns) will alter the sign, but not the numerical value of the determinant. Property III. The multiplication of any one row (or one column) by a scalar k will change the value of the determinant k-fold.

3/18/2012 Page 3 of 10

ka kb a b = kad kbc = k (ad bc) = k c d c d 15a 7b 5a 7b 5a 7b =3 = 3(2) = 6(5ad 14bc ) 12c 2d 4c 2d 2c d Property IV. The addition (subtraction) of a multiple of any row to (from) another row will leave the value of the determinant unaltered. The same holds true if we replace the word row by column in the above statement. a b a b = a (d + kb) b(c + ka ) = ad bc = c + ka d + kb c d

Property V. If one row (or column) is a multiple of another row (or column) the value of the determinant will be zero. As a special case, when two rows(or columns) are identical, the determinant will vanish. 2a 2b = 2ab 2ab = 0 a b c d c = cd cd = 0 d

Determinantal Criterion for Nonsingularity Summary: Given a linear equation system Ax=d where a is an n x n coefficient matrix, A 0 there is row (column) independence in matrix A A is non-singular A-1 exists a unique solution x = A1d exists

Caveat: we may sometimes get negative solution values that are economically inadmissible. Example: Does the equation system possess a unique solution? 7 x1 3 x2 3x3 = 7 2 x1 + 4 x2 + x3 = 0 2 x2 x3 = 2

3/18/2012 Page 4 of 10

7 3 3 2 4 1 = 8 0. 0 2 1 Therefore a unique solution exists. Rank of a Matrix Redefined Rank of matrix A is defined as the maximum number of linearly independent rows in A. In view of the link between row independence and the nonvanishing of the determinant, we can redefine the rank of an m x n matrix as the maximum order of non vanishing determinant that can be constructed from the rows and columns of that matrix. r (A) min {m,n} The rank of A is less than or equal to the minimum of the set of two numbers m and n. 5.4 FINDING THE INVERSE MATRIX Expansion of a determinant by Alien Co-factors Property VI. The expansion of a determinant by alien cofactors (the cofactors of a wrong row or column) always yields a value of zero. Example: If we expand the determinant
4 1 2 5 2 1 1 0 3

by using its first row elements but

the cofactors of the second row elements |c21|, |c22|, |c23| we get a11|c21| + a12 |c22| + a13 |c23| = 4(-3)+1(10)+2(1) More generally, if we have a determinant a11 a12 a13 A = a21 a22 a23 , will yield a zero sum as follows: a31 a32 a33

a
i =1

1j

c2 j = a11 c21 + a12 c22 + a13 c23 = a12 a32 a13 a + a12 11 a33 a31 a13 a a13 11 a33 a31 a12 a32 5.9

= a11

= a11a12 a33 + a11 a13 a32 + a11 a12 a33 a12 a13 a31 a11 a13 a32 + a12 a13 a31 = 0

3/18/2012 Page 5 of 10

Reason for this result: sum of products in 5.9 can be considered as the result of the regular expansion by the second order row of another determinant a11 a12 a13 A * = a11 a12 a13 which only differs from |A| only in its second row and whose first a31 a32 a33 two rows are identical. (5.10)

a
j =1 n

ij

Ci ' j = 0 (i i ')

[expansion by ith row and cofactors of i th

column]

a
j =1

ij

Ci ' j = 0 (i i ')

[expansion by jth row and cofactors of jth

column]

Matrix Inversion a11 a A = 21 ( nxn ) L an1 a12 a22 L an 2 L L L L a1 n a2 n L ann

( A 0)

Form a matrix of cofactors by replacing each element aij with its cofactor |Cij|. Such cofactor matrix denoted by C = [|Cij|] must also be n x n. Our interest is the transpose C commonly referred to as adjoint of A. C11 C C ' adjA 12 ( nxn ) L C1n C21 C22 L C2 n L L L L Cn1 Cn2 L Cnn

The matrices A and C are conformable for multiplication and their product AC is another matrix n x n matrix in which each element is the sum of products.

3/18/2012 Page 6 of 10

 n a1 j C1 j j =1 n a2 j C1 j AC ' = j =1 ( nxn ) M n anj C1 j j =1 A 0 = 0 0 0 A 0 0 0 0 A 0

a1 j C2 j
j =1 n

L L

a
j =1 n

2j

C2 j

a
j =1

nj

C2 j

Cnj j =1 n a2 j Cnj j =1 M n a1 j Cnj j =1

1j

by 5.8 and 5.10

0 1 0 0 = A 0 0 A 0

0 1 0 0

0 0 1 0

0 0 = A In 0 1

As the determinant |A| is a non zero scalar, it is permissible to divide both sides of the equation AC = |A| I. The result is AC ' =I A C' =I A

or

Premultiplying both sides of the last equation by A-1, and using the result that A-1A=I, C' 1 = A1 , or A1 = adj A we can get A A This is one way to invert matrix A!!! Example: 3 2 A= A = 2 0 inverse exists 1 0 C C12 0 1 C = 11 = C21 C22 2 3 0 2 C ' = adjA = 1 3 A1 = 0 2 0 1 1 adj A = 1 = 2 1 3 1 3 A 2 2

3/18/2012 Page 7 of 10

Example: 4 1 1 Find the inverse of B = 0 3 2 . Since |B|=99 0, the inverse of B-1 exists. The 3 0 7 cofactor matrix is 3 2 0 2 0 3 3 7 3 0 0 7 21 6 9 1 1 4 1 4 1 = 7 31 3 3 7 3 0 0 7 5 8 12 4 1 1 1 4 1 3 2 0 2 0 3 Therefore, 21 7 6 adj B = 6 31 8 9 3 12 and the desired inverse matrix is 21 7 5 1 1 1 B = adj B = 6 31 8 B 99 9 3 12 Check if AA-1 = A-1A = I 5.5 CRAMERS RULE a practical way of solving a linear equation system. Given an equation system Ax = d where A is nxn , the solution can be written as x = A1d = 1 (adj A) d A and BB-1 = B-1B = I.

provided A is non-singular. [p.108 provides the proof is which rather long] 1 A1 A |A1| is a new determinant were we replace the first column of |A| by the column vector d but keep all the other columns intact. x1 =

3/18/2012 Page 8 of 10

The expansion of the |A1| by its first column (the d column) will yield the expression

d
i =1

Ci1

because the elements di now take the place of elements aij.

x1 =

1 A1 A

In general, a11 a12 L Aj 1 a21 a22 L = (5.15) x j = A A M M an1 an 2 L Examples: Find the solution of 5 x1 + 3x2 = 30 6 x1 2 x2 = 8 A= 5 3 = 28 6 2 A1 84 = =3 A 28 A2 140 = =5 A 28 A1 = 30 3 = 84 8 2 A2 = 5 30 = 140 6 8 d1 L d2 L M dn L a1 n a2 n ann This is the statement of CramersRule

x1 = x1 =

Find the solution of the equation system: 7 x1 x2 x3 = 0 10 x1 2 x2 + x3 = 8 6 x1 + 3 x2 2 x3 = 7 A = 61, A1 = 61, Work this out!!!! A2 = 183, A3 = 244,

3/18/2012 Page 9 of 10

Solutions: x1 = x2 = x3 = A1 61 = =1 A 61 A2 183 = =3 A 61 A2 244 = =4 A 61

Note that |A| 0 is necessary condition for the application of Cramers Rule. Cramers rule is based upon the concept of the inverse matrix, even though in practice it bypasses the process of matrix inversion.

3/18/2012 Page 10 of 10