Matrix algebra is the link between regression and many multivariate techniques Much of the advanced stats literature is matrix-based Many advanced stats programs (e.g., SEM programs) are matrix-based
An element, aij would be the number in the ith row and jth column For example, element 2,3 is a23
Operation: Trace
Trace is the sum of the diagonal elements (main diagonal)
For a correlation matrix, the trace is the number of variables For a covariance matrix, the trace is the sum of the variances for all variables
Multiplication
To multiply a scalar by a matrix, merely multiply each element in the matrix by the scalar
Multiplication of Matrices
To multiply matrices (applies to vectors, too), the number of columns of the first matrix must equal the number of rows of the second matrix, otherwise the matrices are noncomformable Examples
A(3x2) * B(2x6) is conformable (a solution) A(3x2) * A(3x2) is nonconformable (no solution) A * A is conformable because transposing A () reverses the rows and columns (A(3x2) * A(2x3))
The result will be a matrix of order equal to the number of rows in the first matrix and the number of columns from the second matrix
A(3x2) * B(2x6) = a matrix with 3 rows and 6 columns
Multiplication Examples
Matrix Inversion
There is no straight division in matrix algebra. However, we can conduct division through multiplying a reciprocal In scalar algebra, know a-1 = 1/a, and a/a = 1 so a-1a = 1 In matrix algebra, we draw upon this last function to conduct division. That is:
A-1A = I where I is an identity matrix Also AA-1 = I (this is one case where pre or post multiplying does not matter) Matrix division merely requires us to find the matrix that, when multiplied by A, equals the identity matrix
So, if we think of this applied to a covariance matrix where the main diagonal is variance (like length) and the off diagonal is covariance (like correlation)
The end result is the volume of the geometric space for the matrix
Now, imagine if we had a cube (three dimensions, and the length of one side is zero)
The volume is zero, and so the determinant is zero
If the determinant equals 0, there is no inverse! This is what happens when you have multicollinearity Terms used for this are matrix is not positive definite matrix is singular or matrix is not invertable