MAE 107 Computational Methods Summer Session II 2009

Homework # 2 Solutions
4 problems 20 points

Problem 1 8 points
LU decomposition 1. The following MATLAB code performs the required tasks: function [L,U]=ludec(A) % Computes the LU decomposition of A using Gauss elimination % without pivoting % Input: % A: square matrix % Output: % L: lower triangular matrix with 1 on the diagonal % U: upper triangular matrix

[m,n]=size(A); if(n~=m) error(The matrix must be square) end % Forward Elimination with partial pivoting for j=1:n, %Loop on the columns of A

% Test if the pivot coefficient vanishes if(abs(A(j,j))<1e-6) error(One of the pivot coefficient is zero - Pivoting is necessary) end for k=j+1:n, % Loop on the rows, elements below the diagonal A(k,j)=A(k,j)/A(j,j); % A(k,j) now contains the factor %of elimination A(k,j+1:n)=A(k,j+1:n)-A(k,j)*A(j,j+1:n); % Row operation end 1

end % At the end of this loop, the elements below the diagonal in A are % the elimination factors (elements of L that are located below the diagonal) % U consists of the elements of A located on or above the main diagonal U=triu(A); % Extracts U from A L=tril(A); % Extracts L from A for j=1:n, L(j,j)=1; % Sets the diagonal elements of L to 1 end Note that the previous function stores both L and U in the array A until the very end: after elimination the subdiagonal elements of A are zeros and their value does not need to be stored. Instead we use the storage space to store the corresponding elements of L (elimination factor ij ). 2. We use the previous function at the command line to perform the requested decompositions: >> A1=[3 9 -1; -3 -11 8; 6 24 -24];[L1,U1]=ludec(A1) % Decompose A1 L1 = 1 -1 2 0 1 -3 0 0 1

U1 = 3 0 0 9 -2 0 -1 7 -1 %% Test that the product L1*U1 is indeed equal to A1

>> A1_test=L1*U1,A1 A1_test = 3 -3 6 9 -11 24 -1 8 -24

A1 = 3 -3 9 -11 -1 8 2

24

-24 % Decompose A2

>> A2=[4 2 0; 0 1 2; 20 10 4];[L2,U2]=ludec(A2) L2 = 1 0 5 0 1 0 0 0 1

U2 = 4 0 0 2 1 0 0 2 4 % Test that the product L2*U2 is indeed equal to A2

>> A2_test=L2*U2,A2 A2_test = 4 0 20 2 1 10 0 2 4

A2 = 4 0 20 >> 3. In this question, you needed to write a function that solves Ax = b using the LU decomposition of A. You were asked to use the previous function ludec.m to obtain this decomposition. You did not need to perform the full Gauss elimination here. The following function performs the required tasks. function x=linsys_LU(A,b) % This function solves the square system Ax=b using LU % decomposition. The system Ax=b is decomposed into two triangular systems Ly=b and Ux=y % The LU decomposition is obtained from the function ludec.m % Inputs: A: square matrix 3 2 1 10 0 2 4

b: column vector (right hand side_

% Outputs: x: column vector of the solution % Test if A and b have consistent sizes [m,n]=size(A); if(m~=n), error(A is not square); end if(length(b)~=n), error(b does not have the right length), end % Find the LU decomposition of A [L,U]=ludec(A); % Solve Ly =b using Forward Substitution y=zeros(n,1); for k=1:n, y(k)=b(k)-L(k,1:k-1)*y(1:k-1); end % Solve Ux=y using Backward Substitution x=zeros(n,1); for j=n:-1:1, x(j)=(y(j)-U(j,j+1:n)*x(j+1:n))/U(j,j); end 4 4. Using this function to solve A2 x = 2 : 16 >> A2=[4 2 0; 0 1 2; 20 10 4];b=[4;-2;16]; >> x=linsys_LU(A2,b) x = 1 0 -1 >> A2\b %Check your answer using the backslash operator in Matlab

ans = 4

1.0000 -0.0000 -1.0000 >> 5. The system can be put in standard form as: R1 0 R3 0 0 R1 R2 0 R4 0 0 0 R R ( R + R ) Ax = b, with A = 3 4 5 L 1 0 1 1 0 0 1 0 1 1

b=

Vsource 0 0 0 0

The following Matlab script voltage.m solves the system for the particular value of the voltage source and resistors using linsys_LU.m and computes the power consumption. clear all close all % Define the values of the resistors and voltage V=110; R1=200; R2=200; R3=300; R4=40; R5=1000; RL=400; % Define the matrix A and vector b b=[V;0;0;0;0]; A=[R1 0 R3 0 0; -R1 R2 0 -R4 0; 0 0 -R3 R4 RL+R5;1 0 -1 -1 0; 0 1 0 ... 1 -1]; % SOlve the linear system using linsys_LU x=linsys_LU(A,b); % Extract i5 i5=x(5); disp([The current in RL is ,num2str(i5,3), Amperes]) % Compute the dissipated power P=RL*i5^2; disp([The power dissipated in RL is ,num2str(P,3), Watts]) Executing this script leads to the following output: 5

>> voltage The current in RL is 0.0575 Amperes The power dissipated in RL is 1.32 Watts >> The power dissipated in the component RL is less than its melting threshold (5W) therefore this arrangement is safe for the component.

Problem 2 4 points
We are interested f1 e2 d3 0 0 0 here in solving such a system: g1 f2 e3 d4 .. . ... ... h1 g2 f3 e4 .. . 0 h2 g3 f4 .. . 0 0 0 0 h3 g4 .. . 0 0 0 0 0 h4 .. . 0 0 0 0 ... ... ... ... 0 0 0 0 . . . x1 x2 x3 . . . xn1 xn r1 r2 r3 . . .

dn1 en1 fn1 gn1 0 dn en fn

= rn1 rn

using Gauss elimination without pivoting but using the particular structure of the matrix. The function penta_sys.m should take advantage of the structure by not operating on a full square matrix A but on vectors that correspond to each of the diagonals of the matrix, stored respectively in the column vectors d, e, f , g and h. For convenience, these ve vectors are dened with length n but you notice from the denition of the matrix above, that some coecients are unused (so their value will actually be insignicant on the result). The dierent steps of your function are simply implementing Gauss elimination with the notation presented above. In the rst step, you are trying to cancel coecients e2 and d3 using multiples of the rst row and the coecient f1 as pivot. Do not forget to perform the same operations on the right hand side. Elimination of e2 : the elimination factor is 21 = e2 /f1 and the following operation is performed on the rows (2) (2) 21 (1) which translate in the notations of this problem as f2
(1)

= f2 21 g1 (1)

g2 = g2 21 h1 r2 = r2 21 r1 .
(1)

(1)

Note that h2 is not modied as the corresponding element in the rst row (same column) is zero. Elimination of d3 : the elimination factor is 31 = d3 /f1 and the following row operation is performed (3) (3) 31 (1), corresponding to the following element operations: e3 = e3 31 g1 f3
(1) (1) (1)

= f3 31 h1

(2)

r3 = r3 31 r1 . 6

And coecients g3 and h3 are not modied at this step since the corresponding element in the rst row are equal to zero. Note that we did not explicitly compute d3 and e2 as we know that they are equal to zero. Computing them is not a mistake but is inecient as it executes unnecessary operations. After this rst step, the system will look like: f1 g1 h1 0 0 0 0 ... 0 r1 x1 (1) (1) g2 h2 0 0 0 ... 0 0 f2 (1) r2 x2 (1) (1) 0 e3 (1) f3 g3 h3 0 0 ... 0 x r3 3 0 d4 e f g h 0 . . . 0 = 4 4 4 4 . . . . . . .. .. .. .. .. . . . . . . . . xn1 rn1 0 ... 0 0 dn1 en1 fn1 gn1 xn rn 0 ... 0 0 0 dn en fn In the second step of the elimination, similar operations will be performed to eliminate coecients e3 (1) and d4 by performing row operations using row (2) and f2 as pivot element. This operation is repeated until all the subdiagonal elements of the matrix are equal to zero. Note that for step j , 1 j n 2, 2 row operations are performed to cancel the 2 coecients ej +1 and dj +2 . In step n 1, only one subdiagonal coecient will be cancelled en (there is no dn+1 ). At the end of the Forward Elimination, the systems has the following form: 1 0 f 1 h 0 0 0 ... 0 1 g r 1 x1 0 f g 2 h2 0 0 0 ... 0 2 2 x2 r 0 0 f g h 0 0 . . . 0 3 3 3 (1) x3 r 3 0 0 0 f4 g 4 h4 0 . . . 0 . = . . . . . . .. .. .. .. .. . . . . . . . xn1 r n 1 0 ... 0 0 0 0 f n1 n1 g xn r n 0 ... 0 0 0 0 0 f n Then Backward Substitution can be implemented starting from the last row: xn = xn1 r n f n r n1 g n1 xn = f n1 j xj +2 r j g j xj +1 h for j = n 2, n 3, ...1 f j (3) (4) (5)
(1) (1) (1)

xj = =

The following function penta_sys.m implements this procedure: function x=penta_sys(d,e,f,g,h,r) % Solves penta-diagonal system Ax=r % Inputs: d,e,f,g,h: five column vectors of length n % f: main diagonal of A 7

% % %

e, g: first subdiagonal and superdiagonal d, h: second subdiagonal and superdiagonal r: right hand side vector (column vector of length n)

% Output: x: solution of the system Ax=r n=length(f); if(length(d)~=n|length(e)~=n|length(g)~=n|length(h)~=n| ... length(r)~=n) error(Inconsistent size of the diagonal vectors) end % Forward elimination for j=1:n-2, % Columns 1 to n-2 %eliminates the first subdiagonal coefficient in the column factor1=e(j+1)/f(j); f(j+1)=f(j+1)-factor1*g(j); g(j+1)=g(j+1)-factor1*h(j); r(j+1)=r(j+1)-factor1*r(j); % eliminates the second subdiagonal coefficient in the % column j factor2=d(j+2)/f(j); e(j+2)=e(j+2)-factor2*g(j); f(j+2)=f(j+2)-factor2*h(j); r(j+2)=r(j+2)-factor2*r(j); end % Column n-1 factor=e(n)/f(n-1); f(n)=f(n)-factor*g(n-1); r(n)=r(n)-factor*r(n-1); % Backward substitution x=zeros(n,1); x(n)=r(n)/f(n); x(n-1)=(r(n-1)-g(n-1)*x(n))/f(n-1); for j=n-2:-1:1, x(j)=(r(j)-g(j)*x(j+1)-h(j)*x(j+2))/f(j); end

We can check the validity of our 4 1 1 0 0 0

code by looking at the following pentadiagonal system: 1 1 0 0 0 1 2 5 1 1 0 0 1 4 1 1 0 x = 3 4 1 2 5 2 1 5 0 2 3 1 1 0 0 1 1 4 6

>> d=[0 0 -1 -1 2 -1];e=[0 1 -1 -2 -3 -1]; % Define the diagonals of A >> f=[-4 5 4 5 1 4];g=[1 1 -1 2 -1 0];h=[1 1 -1 -1 0 0]; >> r=[1:6]; % Defines the right-hand side vector % Reconstruct matrix A >> A=diag(d(3:end),-2)+diag(e(2:end),-1)+diag(f)+diag(g(1:end-1),1)+diag(h(1:end-2),2) %Solve Ax=r using penta_sys >> x=penta_sys(d,e,f,g,h,r) x = 0.2242 -0.1024 1.9991 0.2885 4.5860 2.7186 % Compares with the direct solution >> A\r ans = 0.2242 -0.1024 1.9991 0.2885 4.5860 2.7186 Extra-credit: We list here the number of operations performed at each step of the algorithm. For the Forward Elimination, During step 1 j n 2 of the Forward Elimination, the elimination of ej +1 requires 7 operations (1 division to compute the factor j +1,j , 1 subtraction and 1 multiplication to update coecients fj +1 , gj +1 and rj +1 (see for example system (1)). The elimination of dj +2 also requires 7 operations (1 division to compute the factor j +2,j , 1 subtraction and 1 multiplication to update coecients ej +2 , fj +2 and rj +2 ) (see for example system (2)). 9

 Step n 1 of Forward Elimination requires only to eliminate en and only 5 operations must be performed (1 division to compute n,n1 , 1 subtraction and 1 multiplication to update coecients fn and rn . The total cost of Forward Elimination is therefore 14(n 2) + 5 = 14n 23. For Backward Substitution, Computing xn requires only one operation (1 division) see equation (3) Computing xn1 requires 3 operations (1 multiplication, 1 subtraction and 1 division) see equation (4) Computing xj (1 j n 2) requires 5 operations (2 multiplications, 2 subtractions and 1 division) see equation (5) The total cost of Backward Substitution is therefore 5n 6. The total cost of solving eciently the pentadiagonal system is therefore 19n 29 or keeping only the leading order in n: the cost of solving the pentadiagonal system is 19n. Note the linear dependence in n compared to the cubic dependence of the cost of full Gauss elimination.

Problem 3 5 points
Consider the following system: 4x1 + 2x2 2x3 = 14 2x1 + 5x2 + 3x3 = 3 2x1 x2 + 4x3 = 8.5 The system is already organized so as to have the maximum coecients along the main diagonal. We can rewrite it as 7 + x3 x2 x1 = 2 2x1 3x3 3 x2 = (7) 5 x2 2x1 8.5 x3 = 4 (0) x1 (0) (0) 1. Starting from an initial guess x = x2 , the Jacobi algorithm is implemented by using the x3 system in the form (7) to express a new guess x(n+1) (on the left hand side) as a function of a previous guess x(n) on the right hand side. For the iteration n: x1 x2 x3
(n+1) (0)

(6)

(n+1)

(n+1)

7 + x3 x2 2 (n) (n) 2x1 3x3 3 = 5 (n) (n) x 2x1 8.5 = 2 4 = 10

(n)

(n)

(8)

and the iteration is repeated until the approximate relative error imate relative error is dened here as:
n

is less than 106 . The approx-

= max

xi

(n)

xi xi
(n)

(n1)

with 1 i 3.

The following Matlab script prob3_jacobi.m performs the right operations. (Note that you were asked here to write a script and not a function!). % Script file prob3_jacobi.m clear all close all

% Solves the system of problem 3 using the Jacobi method err_thres=1e-6; % Maximum relative error x0=[1;1;1]; % Initial vector x=x0; n=0; err=1; % Solve the system directly A=[4 2 -2; -2 5 3; 2 -1 4], b=[14; -3;-8.5], disp(True solution) xtrue=A\b N=2000; % Maximum number of iterations % Jacobi loop while(err>err_thres & n<N) n=n+1; xn(1,1)=(14-2*x(2)+2*x(3))/4; xn(2,1)=(-3+2*x(1)-3*x(3))/5; xn(3,1)=(-8.5-2*x(1)+x(2))/4; err=max(abs((xn-x)./xn)); x=xn; end % Output if(n<N) disp([Number of iterations required ,int2str(n)]) x else disp([The system did not converge in ,int2str(N), iterations]) end 11

The execution of this script produces the following output:

>> prob3_jacobi A = 4 -2 2 2 5 -1 -2 3 4

b = 14.0000 -3.0000 -8.5000 True solution xtrue = 1.5000 1.5000 -2.5000 Number of iterations required 64 x = 1.5000 1.5000 -2.5000

We see that our iterative scheme indeed converges to the correct solution and convergence is achieved after 64 iterations.

2. GaussSeidel method works in a very similar way as Jacobi algorithm. However, in (8), you observe (n+1) (n) that to compute x2 we use the value x1 on the right-hand side although a more accurate (n) (hopefully!) value x1 has been computed at the previous line. GaussSeidel method is based on adapting Jacobis algorithm to use the most recent values of each unknown. Starting from the same 12

initial vector x(0) , the GaussSeidel algorithm is implemented as follows: x1 x2

(n+1)

(n+1)

x3

(n+1)

7 + x3 x2 2 (n+1) (n) 2x1 3x3 3 = 5 (n+1) (n+1) x 2x1 8.5 = 2 4 =

(n)

(n)

where the new values of xi are used as soon as they are available. The corresponding script prob3_gauss.m is obtained as: % Script file prob3_gauss.m clear all close all

% Solves the system of problem 3 using the Gauss Seidel method err_thres=1e-6; % Maximum error x0=[1;1;1]; % initial vector x=x0; n=0; err=1; % Solves the system directly A=[4 2 -2; -2 5 3; 2 -1 4], b=[14; -3;-8.5], disp(True solution) xtrue=A\b N=2000; % Maximum number of iterations % Gauss Seidel loop while(err>err_thres & n<N) n=n+1;xo=x; x(1,1)=(14-2*x(2)+2*x(3))/4; x(2,1)=(-3+2*x(1)-3*x(3))/5; x(3,1)=(-8.5-2*x(1)+x(2))/4; err=max(abs((x-xo)./x)); end % Output if(n<N) disp([Number of iterations required ,int2str(n)]) x 13

else disp([The system did not converge in ,int2str(N), iterations]) end

The output of this script when executed at the command line is:

>> prob3_gauss A = 4 -2 2 2 5 -1 -2 3 4

b = 14.0000 -3.0000 -8.5000 True solution xtrue = 1.5000 1.5000 -2.5000 Number of iterations required 18 x = 1.5000 1.5000 -2.5000

We observe that we indeed have convergence to the correct solution and that convergence is achieved in only 18 iterations. Gauss-Seidel iterations converge in general faster than the Jacobi method as new values of the unknowns (that are expected to be closer from the true solutions) are used as soon as they are available. 14

3. If we replace the coecient of x1 in the rst equation by 1, the Gauss-Seidel iteration becomes: x1
(n+1)

= 14 + 2x3 2x2 2x1

(n+1) (n)

(n)

(n)

(n+1) x2

x3

(n+1)

3x3 3 = 5 (n+1) (n+1) x2 2x1 8.5 = 4

The modied script prob3_gauss2.m is % Script file prob3_gauss2.m clear all close all

% Solves the modified system of problem 3 using the Gauss Seidel method err_thres=1e-6; % Maximum error x0=[1;1;1]; % initial vector x=x0; n=0; err=1; % Solves the system directly A=[4 2 -2; -2 5 3; 2 -1 4], b=[14; -3;-8.5], disp(True solution) xtrue=A\b N=2000; % Maximum number of iterations % Gauss Seidel loop while(err>err_thres & n<N) n=n+1;xo=x; x(1,1)=14-2*x(2)+2*x(3); x(2,1)=(-3+2*x(1)-3*x(3))/5; x(3,1)=(-8.5-2*x(1)+x(2))/4; err=max(abs((x-xo)./x)); end % Output if(n<N) disp([Number of iterations required ,int2str(n)]) x else disp([The system did not converge in ,int2str(N), iterations]) end 15

and its execution leads to the following output: >> prob3_gauss2 A = 4 -2 2 2 5 -1 -2 3 4

b = 14.0000 -3.0000 -8.5000 True solution xtrue = 1.5000 1.5000 -2.5000 The system did not converge in 2000 iterations This new system can not be solved iteratively using GaussSeidel (or Jacobi) method as the iteration diverges. By replacing the coecient of x1 by 1 in the rst equation, we have made the matrix of the system being strongly not diagonally dominant which jeopardizes its ability to converge.

Problem 4 3 points
(adapted from Chapra, problem 9.4) Consider the following system: x2 2x3 = 3 +5x2 +3x3 = 3 4x2 =6

x1 2x1

0 1 2 3 . To compute the determinant of 1. The system has the form Ax = b with A = 1 5 2 4 0 16

|A| we can expand A along its rst column: 0 1 2 1 5 3 2 4 0

|A| =

=0

5 3 4 0

(1)

1 2 4 0

+2

1 2 5 3

= 8 + 2 13 = 18

|A| = 18.

2. The rst coecient in the rst row is equal to zero, therefore we need to use partial pivoting. Switching the rst and last rows of the system, 6 2 4 0 1 5 3 x = 3 3 0 1 2 One of the subdiagonal coecient is already equal to zero therefore we only need to perform an operation on row 2 to remove its rst coecient. The required operation is (2) (2) 21 (1) with 21 = 1/2. No operation is necessary in the rst step on row (3) (31 = 0). 2 4 0 6 0 3 3 x = 0 0 1 2 3 In the second step, we use the following operation to remove the last subdiagonal coecient: (3) (3) 32 (2) with 32 = 1/3: 2 4 0 6 0 3 3 x = 0 0 0 3 3 Using Backward substitution, we obtain the solution of the system:

1 x = 1 1

17

3. Using Cramers rules the solution is computed directly as: 3 1 2 3 5 3 6 4 0 x1 = =1 |A | 0 3 2 1 3 3 2 6 0 x2 = = 1 |A| 0 1 3 1 5 3 2 4 6 x3 = =1 |A | 1 This conrm that the solution of the system is x = 1 . 1

18