1 | P a g e

HOMEWORK
Chapter 6-7



Class: Linear System (Spring 2008)
Professor: Hansil Kim













Student: Ha Nguyen Vu
ID number: 20085165.
Wireless Communication Lab






Ulsan, June 16, 2008
2 | P a g e


Problem 1:
Describe: Singular Value Decomposition and LU Decomposition.


Singular Value Decomposition

In linear algebra, the singular value decomposition (SVD) is an important factorization of a
rectangular real or complex matrix, with several applications in signal processing and
statistics. Applications which employ the SVD include computing the pseudo inverse, least
squares fitting of data, matrix approximation, and determining the rank, range and null space
of a matrix.
Definition : Any real mxn matrix A can be decomposed uniquely by
T
A UDV =

Where:
U is a mxm orthogonal matrix (
T
UU I = ) and its columns are eigenvectors of
T
AA
V is a nxn orthogonal matrix (
T
VV I = ) and its columns are eigenvectors of
T
A A
D is a mxn diagonal matrix :
o
1 2
( , ,..., )
n
D diag =
ordered so that
1 2
...
n


o If 0
j
with j=1,2,n then
j
is called singular value and its square is
an eigenvalue of
T
A A.
o If
1 2
( , ,..., )
n
U u u u =
and
1 2
( , ,..., )
n
V v v v =
then
1
n
T
i i i
i
A u v
=
=


Applications of SVD

Computing the rank of a matrix using SVD
The rank of a matrix is equal to the number of non-zero singular values

Computing the inverse of a matrix using SVD
If with the matrix D of a matrix A has all values 0
j
for all j,
We have :
1 1 1
( ) ( )
T T T T
AA UD V V D U U DD U UU I

= = = =
And
1 1 1
( ) ( )
T T T T
A A VD U U DV V D D V VV I

= = = =
With
1
1 2
1 1 1
( , ,..., )
n
D diag

=

Thus, we can get the inverse of matrix A is :
1 1 T
A VD U

= .
3 | P a g e


Solving a system of linear equations :
We have linear equations for a system
Ax=b
Where A is a mxn matrix and b is a vector mx1.
The least-squares solution x with the smallest norm ||x|| is unique and it is given
by:

1
( )
T T
x A A A b A b
+
= =
Where

1
( )
T T
A A A A b
+
= (The More-Penrose pseudo inverse)

LU Decomposition

In linear algebra, the LU decomposition is a matrix decomposition which writes a matrix
as the product of a lower and upper triangular matrix. The product sometimes includes a
permutation matrix as well. This decomposition is used in numerical analysis to solve
systems of linear equations or calculate the determinant.

An LU decomposition is a decomposition of the form A = LU

Where A is an mxn matrix , L is a lower triangular mxm matrix and U an upper triangular
mxn matrix. This decomposition is used to solve systems of linear equations and to find the
inverse of a matrix. As we shall notice, if A is a band matrix, then the matrices L and U are
band matrices as well. This is a useful and memory-saving property.

Here we just concentrate on square matrices. The LU-decomposition of a matrix A is
gotten by left multiplying it by so called Gaussian matrices until we have an upper triangular
matrix.

Constructing LU-decomposition

Gaussian matrix Let x be an n-vector. We construct a square matrix Lk , such that first k
elements of the vector x remain fixed and the rest are changed to zero, And the Gaussian
matrix is:












4 | P a g e

It is easy to compute the inverse of a Gaussian matrix. The inverse is (show this)

Multiplying the system of equations by this matrix does not change the solution.
Suppose we have a matrix A that is of the following form

We transform this matrix to an upper triangular matrix by left multiplying it by suitable
Gaussian matrices. First we use a suitable matrix L
1
to transform the first column of A to
upper triangular form. Then we use a suitable L
2
to do the same thing for the second column
and so on. After n-1 steps we have an upper triangular matrix.

The 1
st
Column: We assume, that a
11
0, then choose a Gaussian matrix L
1
such that
it transforms the first row of A so that it is parallel with e
1
. Next we left multiply the
equation by the matrix

We have:

with component wise representation (with)

Where a
(1)
ij
are the new elements.

The 2
nd
Column: If
(1)
22
0 a we can use a Gaussian matrix that leaves the first two
elements of the second column in previous matrix intact and transforms the rest of the
elements to zero.

5 | P a g e

Where
(1)
2
2 (1)
22
3, 4,..., .
i
i
a
l i n
a
= =
and the component wise representation is

We get the product

That is:


The k
th
Column: We make a description of a general step after we have performed k-1
steps already. If
( 1)
0
k
kk
a

.Then we can find a Gaussian matrix, that leaves the first k
elements of the k
th
column intact and transforms the others to zero.
( 1)
( 1)
1, 2,..., .
k
ik
ik
k
kk
a
l i k k n
a

= = + +

so the Gaussian matrix has the following element wise representation

Finally k=n-1 and we have

Where U is upper triangular as we wanted

6 | P a g e

The product of two lower triangular matrices that have ones on the diagonal is a lower
triangular matrix that has ones on the diagonal (exercise 1.16.8), so the matrix

is lower triangular with ones on the diagonal. Now we can write
`
Where U is an upper diagonal matrix. Because the inverse of a lower triangular matrix
with ones on diagonal is a lower triangular matrix, we finally get the decomposition we
were after.
1

. A L U LU

= =

Existence of LU-decomposition

Now we want to know when the LU-decomposition of a given matrix exists. In construction of
an LU-decomposition we use the elements

as denominators. These elements and
( 1) n
nn
a

are called pivot elements. According to the
construction of LU-decomposition, it should be clear that if the n-1 first pivot elements differ
from zero then the LU-decomposition of a square matrix A exists.
Applications
Solution of a system (nxn) of linear equations by using LU-decomposition.
LU-decomposition is mainly used to solve matrix equations Ax=b. First we find the LU-
decomposition of a matrix A. partial pivoting does not change the solution, because only
the order of the equations is changed. In complete pivoting we might change the solution
by permuting it since we can change the order of the variables x
i
. Complete pivoting leads
to an equation of the form.

that is easy to solve. The inverse of a permutation matrix P is its transpose, so
we get the equation
LUx=Pb.
We can split this into two halves.
Ly=Pb,
Ux=y.
The first equation has a unique solution y=L-1Pb, because L is invertible. The matrix
U is an upper triangular matrix that has the pivot elements of A on its diagonal. The latter
equation also has a unique solution if all the pivot elements, including the last one, are
nonzero.
7 | P a g e


Finding the inverse of a matrix by using LU-decomposition.
LU-decomposition gives us new information about the inverse of a matrix: if det(A) 0,
then A has an inverse that can be computed by solving the system of equations,
1, 2,.., .
i i
Ax e i n = =

This could also be presented in the form

so AX = I. The properties of the determinant now say that
det(AX)=det(A)det(X)=1=det(I)
so det(X) 0, and its easy to find an nxnmatrix Y such satisfies the equation:
XY=I
We left multiply the equation by A:

So AX = XA = I, and that is X = A
-1
.



Problem 2:
Describe the adjoint operator and Least Square method.
Solve: problem 6.43 and 6.45


Adjoint Operator

Consider a linear transformation:
1 2
: A H H
Where:
1
H ,
2
H _ inner product spaces ( respectively,
1
, < > and
2
, < > _ inner products)
Its mean that:
1
x H
2
( ) A x y H = .
If z _ artritrary vector in
2
H then
2 2
, , ( ) z y z A x < > =< > is well defined. We can be used
it to define the adjoint transformation:
*
2 1
: A H H With
*
2 1
, ( ) ( ), ) z A x A z x < > =< >
It can be shown for finite dimensional spaces that
*
A is also a linear transformation.

Properties of adjoint transformations:

1. In all case we have:
*
A A = and
* *
( ) A A = .
If
1 2
: A H H and
*
2 1
: A H H . We have:
*
1 1
: A A H H and
*
2 2
: AA H H .
If
1
H =
2
H then
*
A A = and A is called self-adjoint.
If
* *
AA A A = are said to be normal transformation.

8 | P a g e

2. With any linear transformation
1 2
: A H H . Then the linear vector spaces
1
H and
2
H
can be written as direct sums :
*
1
( ) ( ) H N A R A = and
*
2
( ) ( ) H N A R A = (*)
Where:
o N(.) and R(.) are the null space and range of the transformations.
o (.) R denotes the closure of the range R(.).

It means if ( ) x N A and any vector
*
( ) y R A we have <x,y>=0.
Equation (*) provides an orthogonal decomposition for
1
H and
2
H .

Least Square Method

Consider the equation: Ax=y (where A is mxn matrix (m>n))
With this equation, the sub_equations are often inconsistent to each others and getting a
exactly solution is very difficult. In this case, the approximately solution is the best choice,
and the Least square method is the one of them.

Least Square method
To solve the problem, we set an nx1 error vector e : e = y-Ax.
The least-squares approach yields the one x which minimizes the sum of the squares of
the e(i) components .
That is, x is chosen to minimize :
2
( ) ( )
T T
J e e e y Ax y Ax = = =
To minimize
2
e , we must take the derivation of J and solve incase it equal zeros to get
the x value which makes
2
e is minimum. We have:
0 ( ) ( ) 0
T T
J
A y Ax A y Ax
x

= =

0
T T
A y A Ax =
Thus,

1
( )
T T
x A A A y

=
The matrix
1
( )
T T
R A A A

= is a particular example of the generalized or pseudo-inverse of

A written A
+
. If
1
A

exits, then
1
A A
+
= and J=0.

Weighted Least-Squares
Ignoring some equations or placing equal reliance on all equations represents two
extremes. If some equations are more reliable than others, but all equations are to be
retained, a weighted least-squares approximation can be used.
That is, x should minimize
1 1
( ) ( )
T T
J e R e y Ax R y Ax

= =
Take the derivation:
9 | P a g e

1 1
0
T T
J
A R Ax A R y
x

= =

We have:

1 1 1
( )
T T
x A R A A R y

=

Exercise 6.43 page 244



Call the current goes through the circuit is i(t)
Application the Kirchhoffs Currents Law, we have . '( ) . ( ) 0 Li t Ri t + =
Using Laplace transform:
. '( ) . ( ) 0 . . ( ) . (0 ) . ( ) 0
Laplace
Li t Ri t L s I s Li R I s + = + =
Where i(0) is the initial value of the current.
And:
. . ( ) . (0 ) . ( ) 0
. (0) (0)
( )
L s I s Li R I s
Li i
I s
R
R Ls
s
L
+ =
= =
+
+

Then:
_
(0)
( ) ( ) (0).
R
t
inv Laplace
L
i
I s i t i e
R
s
L

= =
+

So we have ( ) (0).
R
t
L
i t i e

= .
Thus the voltage value of R is
( ) ( ) . (0). (0).
R R
t t
L L
v t Ri t Ri e v e

= = =

With this equation we can get the form of the vector equation as:
V=E
t
.x
With V is a vector (nx1) of the array voltage value
E
t
is a vector (nx1) of the array value of the function
R
t
L
e

with the time change.

10 | P a g e

x is the matrix (1x1) present for the value of v(0), now we want to find.

From the table from the problem we have the value of v(t) with t=0,1,2,3 is.
| | 167.9 95.5 88.8 55.3
T
V =
And the vector E
t
we can get by calculating the value of the function
R
t
L
e

with t=0,1,2,3,
R=10 Ohm and L = 3.56 H , we have :
3 4
1 0.0602 3.631 10 2.188 10
T
t
E

( =


Apply the least-squares solution, we have:
1
( . ) . 0.996375. . 0.9964 173, 9837 173.3574
T T T
t T t t
x E E E V E V

= = = =
Finally, we have v(0)=173.3574 (V)
So the initial current is
(0) 173.3574
(0) 17.33574( )
10
v
i A
R
= = =



Exercise 6.45 page 244



We have the time (
i
T ) required to race a distance (
i
D ) can be expressed as:

) (
i i
D C T =
( from the exercise)
Taking logarithm for this equation, we have :
10 10 10
og
i i
Log T Log C L D = +

From the problem, we already have the array values of the (
i
T ) and (
i
D ), here we must
define the value of C and .
Calculate the values of
10 i
Log T from the above table of
i
T and
i
D we have:
| | _ 2.2672 1.9009 1.7782 1.5786 1.0607 Log time =
And
| | _ tan 1.4183 1.0934 0.9777 0.7924 0.3010 Log dis e =
Then we have the equations:
10
2.2672 1 1.4183
1.9009 1 1.0934
.log . 1.7782 1 0.9777
1.5786 1 0.7924
1.0607 1 0.3010
C
( ( (
( ( (
( ( (
( ( ( = +
( ( (
( ( (
( ( (


11 | P a g e

Set

10
log C
X

(
=
(

,
1 1.41830
1 1.09342
1 0.97772
1 0.79239
1 0.30103
A
(
(
(
(
=
(
(
(

and
2.26717
1.90091
1.77815
1.57863
1.06069
Y
(
(
(
( =
(
(
(


We have: Y=A.X.
Apply the least-squares solution, we have:
1
( )
T T
X A A A Y

= =
1
5.0000 4.5829 1.4335 1.3458
4.5829 4.8816 1.3458 1.4683
T T
A Y A Y

( (
=
( (

Thus
10
log 0.730
1.077
C
X

( (
= =
( (


Finally: C= 10
0.73
=5.37 and 1.077 =
From here we can achieve the equation:
1.077
5.37 ( istance) Time D =

So, with D=1 we have :
1.077
_1 5.37(1) 5, 37min 5min 22 T mile s = = =




Problem 3:
Describe how to calculate the eigen value and the eigen vector in each case.
Solve: problem 7.50 and 7.51


Eigenvalue And Eigenvector

Let A be any linear transformation with domain D(A) and range R(A), both contained
within the same linear vector space H. Let elements of H be denoted as
i
x . Those
particular elements 0
i
x and the particular scalars
i
F which satisfy ( )
i i i
A x x = are
called eigenvectors and eigenvalues, respectively.

Eigenvalue

In order to find eigenvalues of matrix A, we solve the equation det( ) 0 A I = and the
roots of this equation are eigenvalues.
The equation det( ) 0 A I = is called characteristic equation of the matrix A.

Example:
12 | P a g e

Find the eigenvalue of this matrix
0 1
5 6
A
(
=
(
(


We have the characteristic equation:
0 (6 ) 5 0 1 5 A I or = + = = =
So A has two eigenvalues, those are:
1
1 = and
2
5 =

Eigenvector

Two find the eivenvectors we must solve the equation .
i i i
A x x = (*)
but there are many case of the eigenvalue and sometime is hard to find the eigenvectors.

Case I : Distinct Eigenvalues

This is the simplest case, in this case, each eigenvalue has algebraic multiplicity of one,
so we have ( ) 1 rank A I n = (A_nxn matrix). This means that there is only one
independent nontrivial solution to the homogeneous equation .
i i i
A x x =

Example:
Find the eigenvector of matrix A (above).
weve already had
1
1 = and
2
5 = ..
o With
1
1 = , the equation (*) will be

1 0 1
1
1
5 6
a a a
k
b b b
(
( ( ( (
= =
(
( ( ( (
(

We chose
1
1
1
x
(
=
(


o With
2
3 = , the equation (*)will be
1 0 1
5
5
5 6
a a a
k
b b b
(
( ( ( (
= =
(
( ( ( (
(

We chose
2
1
5
x
(
=
(


So A has two eigenvectors respectively with the eigenvalues
1
1 = and
2
3 = those
are:
1
1
1
x
(
=
(

and
2
1
5
x
(
=
(



Case II: Repeated Eigenvalues

In this case, there are one or more eigenvalues are repeated roots of the characteristic
equation, a full set of eigenvectors may or may not exist. The number of linearly
independent eigenvectors associated with an eigenvalue
i
repeated with an algebraic
multiplicity
i
m is equal to the dimension of the null space of
i
A I . This dimension is
given by ( )
i i
q n rank A I = and is called the degeneracy of
i
A I .
13 | P a g e


Case II
1
: The fully degenerate case
i i
q m =
The fully degenerate case has a full set of
i
m eigenvectors associated with the
repeated root
i
. The
i
m independent eigenvectors associated with the repeated root
can be selected as independent columns of the differentiated adjoint matrix:
| |
1
1
1
( )
( 1)!
i
i
i
m
m
i
d
Adj A I
m d

)
(**)
Example:
Find the eigenvectors of matrix
1 0 0
0 1 0
0 0 2
A
(
(
=
(
(


Simply we can get there eigenvalues
1
2 = and
2 3
1 = =
o With
1
2 = we solve as case 1 then have the eigenvector is: | |
1
0 0 1
T
x =
o With
2 3
1 = = we have
2,3
0 0 0
( ) 0 0 0 1
0 0 1
rank A I rank
(
(
= =
(
(


So we have:
2 2
3 1 2 q m = = =
Apply (**):
1
(2 1)!
d
d
2
1
( 1)( 2) 0 0
0 ( 1)( 2) 0
0 0 ( 1)

=
(
(
=
(
(

1 0 0
0 1 0
0 0 0
(
(

(
(


Then we have more two eigenvectors:
2
1
0
0
x
(
(
=
(
(

;
3
0
1
0
x
(
(
=
(
(



Case II
2
: Simple Degeneracy, 1
i
q =
In this case, there is just one eigenvector for each eigenvector, regardless of the
algebraic multiplicity.
Assume that
i
has an algebraic multiplicity
i
m and 1
i
q = we can define all the
eigenvectors by the following method:
14 | P a g e

1 1
2 2 1
3 3 2
1




i
i
i
mi i mi mi
Ax x
Ax x x
Ax x x
Ax x x

=
= +
= +
= +

(***)
Example:
Find the eigenvalues and eigenvectors of following matrix:
0 1 0 0
0 0 1 0
0 0 0 1
8 20 18 7
A
(
(
(
=
(
(
(



The characteristic equation:

4 3 2
7 18 20 8 0 + + + + =
Solve the above equation we have: { } 1, 2, 2, 2 1, 4
i
i = = .
o With
1
1 = we solve as case 1 to get the eigenvector is
| |
1
1 1 1 1
T
x =
o With
2,3,4
2 = we have
2
3 m = and
2
1 q =
For
2
=-2 then we have
2
3 m = and
2
( 2 ) 1 q n rank A I = + =
So we must use (***) to find the eigenvectors:
Firstly, we find
2
x from equation :
2 2 i
Ax x =
2 2
2 2
2 2
2 2
0 1 0 0
0 0 1 0
2
0 0 0 1
8 20 18 7
a a
b b
c c
d d
(
( (
(
( (
(
( (
=
(
( (
(
( (
(



We have:

2
1 1 1
1
8 4 2
T
x
(
=
(


Second, we find
3
x from equation:
3 3 2 i
Ax x x = +
3 3
3 3
3 3
3 3
0 1 0 0 1/ 8
1/ 4 0 0 1 0
2
1/ 2 0 0 0 1
1 8 20 18 7
a a
b b
c c
d d
( (
( (
( (
( (

( (
( (
= +
( (
( (
( (
( (
( (


We have:
15 | P a g e

3
3 1 1
0
16 4 4
T
x
(
=
(


Finally, we find
4
x from equation:
4 4 3 i
Ax x x = +
4 4
4 4
4 4
4 4
0 1 0 0 3/ 16
1/ 4 0 0 1 0
2
1/ 4 0 0 0 1
0 8 20 18 7
a a
b b
c c
d d
( (
( (
( (
( (

( (
( (
= +
( (
( (
( (
( (
( (


We have:

4
3 3 1
0
16 16 8
T
x
(
=
(


Case II
3
1
i i
q m
This is the general case for both case II
1
and case II
2
and this case II
3
is really just a
combination of a previous two cases. There will be one Jordan block for each
eigenvector, that is ,
i
will have
i
q blocks associated with it.

Example:
Let A be a 8x8 matrix and assume that the eigenvalues have been found as
1 2
2 = = ,
3 4 5 6
3 = = = = ,
7 8
4 = = .
If
1,2
( ) 7 rank A I = ;
3,6
( ) 6 rank A I = ;
7,8
( ) 6 rank A I = we have:
o With
1 2
2 = = ,
1
8 7 1 q = = the simple degeneracy Case II
2
, so x
1
is a
eigenvector and x
2
is a generalized eigenvector.
o With
3 4 5 6
3 = = = = ,
3
8 6 2 q = = case II
3
ad there are two eigenvectors
and two generalized vectors here we will have two Jordan block with each
block has one eigenvector.
o With
7 8
4 = = ,
7
8 6 2 q = = case II
1
all the vectors are eigenvectors.
o Finally there are five eigenvectors (so there are five Jordan blocks) and three
generalized vectors.

The Jordan form can be: (there are two ways)



16 | P a g e

Exercise 7.50 page 281

This problem just only request to find the maximum value but here I will calculate both
minimum value and maximum value of Q.

We have

1
1 2
2
2 2
1 1 2 2
,

3 1

1 3
3 2 3
T
Q x Ax
x Ax
x
x x
x
x x x x
=< >
=
( (
( =
( (



= +

More over:

( )
( )
2 2 2 2 2 2 2
1 1 2 2 1 2 1 2 1 2
2 2 2 2 2
1 2 1 2 1 2
3 2 3 2 2 ( ) 2( )
4 4 ( ) 4( )
x x x x x x x x x x
x x x x x x
+ = + + +
= + + +

Thus,

2 2 2 2
1 2 1 2
2( ) , 4( ) x x Q x Ax x x + = +
In the other hand, we have x is a unit vector so , 1 x x < >= and
2 2
1 2
1 x x + = ( | |
1 2
T
x x x = )
Summarily, 2 4 Q
Then we have:
The minimum value of Q is 2 when
1 2
2 2
1 1
0
1
x x
x x
=

+ =

We have:
1 2
1 1 1 1
[ , ] [ , ];[ , ]
2 2 2 2
x x =
The maximum value of Q is 4 when

1 2
2 2
1 1
0
1
x x
x x
+ =

+ =

We have:
1 2
1 1 1 1
[ , ] [ , ];[ , ]
2 2 2 2
x x =

Plotting the Q value by using Matlab program to check the result:

x=[-1:0.01:1 1:-0.01:-1];
y=[];
q=[];
for ii=1:length(x)
if ii>length(x)/2
17 | P a g e

y=[y sqrt(1-x(ii)^2)];
else
y=[y -sqrt(1-x(ii)^2)];
end
q=[q 3*(x(ii)^2+y(ii)^2)-2*x(ii)*y(ii)];
end
plot3(x,y,q);



Conclusion:
In this problem we can solve by change

2 2 2 2 2 2 2 2
1 1 2 2 1 2 1 2 1 2 1 2
3 2 3 2 2 ( ) 4 4 ( ) x x x x x x x x x x x x + = + + = + +
And using the characteristic: the square of a real number is larger than zeros. But in some
case we can not do like that, we will be able to solve the same of this problem by using
Lagrange method as:
Get a function:
2 2
1 2
.( 1) L Q x x = + +
Then find the minimum and maximum by solving:
1
2
2 2
1 2
0
0
1 0
L
x
L
x
x x

=

+ =

18 | P a g e

Exercise 7.51 page 281:

Set:
The eigenvectors and eigenvalues of the simple matrix A is {
i
x
} and{ }
i
. The matrix A
is presented for the simple linear transformation.
The reciprocal basis vectors of {
i
x
} is {
i
r
}.
Its means:
. 1
T
i i
r x =
and
. 0 ( )
T
i j
r x i j =

`Let
i i i
E x r = ><
, we will prove that
i
E
is a projection onto
( )
i
N A I
.
We have:
Firstly,
i
E
is satisfy the condition
( ). 0
i i
A I E =
:
i i i
E x r = ><
Then
.
T
i i i
E x r =

So:

( ). ( ). .
( . . ).
( . . ). 0
T
i i i i i
T
i i i i
T
i i i i i
A I E A I x r
A x x r
x x r

=
=
= =

Thus
i
E
is onto the null space of
( )
i
A I
(mean
( )
i
N A I
)
Secondly,
. ( )( )
( . )( . ) .( . ).
.
i i i i i i
T T T T
i i i i i i i i
T
i i i i i
E E x r x r
x r x r x r x r
x r x r E
= >< ><
= =
= = >< =

So we have .
i i i
E E E =
And
. ( )( )
( . )( . ) .( . ).
.0. 0
i j i i j j
T T T T
i i j j i i j j
T
i i
E E x r x r
x r x r x r x r
x r
= >< ><
= =
= =

Then . 0
i i
E E =
With all conditions above, we can recognize easily that
i
E
is a projection onto
( )
i
N A I
.

Moreover, with any vector y goes through the linear transformation A we have:

1
,
n
i i
i
y r y x
=
= < >

19 | P a g e

And the transformation:

1
1
1
1
1
1
. ,
, .
, .
. , (because , and are the number (not a matrix))
. . .
.( . ).
.(
n
i i
i
n
i i
i
n
i i i
i
n
i i i i i
i
n
T
i i i
i
n
T
i i i
i
i i
Ay A r y x
r y A x
r y x
x r y r y
x r y
x r y
x

=
=
=
=
=
=
= < >
= < >
= < >
= < > < >
=
=
= >

1 1 1
). . . . .
n n n
i i i i i
i i i
r y E y E y
= = =
| |
< = =
|
\ .


So matrix A can be presented by
1
.
n
i i
i
A E
=
=

with
i
E
is a projection onto
( )
i
N A I
as we
have just proved.


Problem 4:
Describe the algorithm finding the trajectory of the linear or nonlinear differential equation.
Present 5 examples in the second or the third order differential equations, and calculate
the trajectory of them with any initial condition by simulation with C programming or Matlab.



The method to finding the trajectory of the linear differential equation:

The second order differential equation:
We consider the form of the second order differential equation of a linear system like:
'' . ' 0 x a x bx + + = with '
x
x
t

Here I will introduce the method to draw the trajectory of this equation.
We have:
'' . ' 0 x a x bx + + =
If we set:

1
x x = and
2
' x x =
20 | P a g e

We have:

1 2
2 2 1
'
'' . ' . .
x x x
x x a x b x a x b x
= =
= = =

So we have

1 1
2 2
0 1
.
x x
x x b a
( ( (
=
( ( (

Simply,
. X A X =

with
1
2
x
X
x
(
=
(

and
0 1
A
b a
(
=
(




From here we can solve the problem by three ways:

1. The first, we can calculate the eigenvalues of A and then find the form of x
1
and
x
2
like:

1 2
1 1 2 2
( ) . . . .
t t
x t k a e k a e

= +
Where
k
1
,k
2
are the constant.
a
1
,a
2
are the eigenvectors of matrix A.

1
,
2
are the eigenvalues of matrix A.
With the initial condition of X, we can define the value of k
1
and k
2
then we will
draw the trajectory of the system.

2. The second, we will not solve the equations, we will base on each equation to
find the relation between x
1
and x
2
and using the numerical method to find the
trajectory of the system:
The algorithm can be show follow:
We have:

1
1 2
dx
x x
dt
= = and
2
2 2 1
. .
dx
x a x b x
dt
= =
Divide two equations above we have:

2 2 1
1 2
. . dx a x b x
dx x

=
We using the approximately form:

2 2 1
1 2
. . x a x b x
x x

=

With the value of
1
x we can find the value
2
x by above formula.
Finally, with a recently position
1 2
( , ) x x we can calculate the value
(
1
x ,
2
x ) so then we can achieve the next position
1 1 2 2
( , ) x x x x + + and
then step by step we can draw the trajectory.


21 | P a g e

Algorithm:
o Get the initial condition [x
1
(0);x
2
(0)].
o Fix |
1
x |= the sign of
1
x is depend on the value of
2
x .
If
2
0 x > then
1
0 x >
If
2
0 x < then
1
0 x <
This to be sure its satisfied that x
1
and x
2
have the right variation
when time change.
o Calculate
2
( 1) x k + ,
1
( 1) x k + and
2
( 1) x k +

2 1
2 1
2
. ( ) . ( )
( 1) .
( )
a x k b x k
x k x
x k

+ =

1 1 1
( 1) ( ) x k x k x + = +

2 2 2
( 1) ( ) ( 1) x k x k x k + = + +
o Finish the repeat after a number step is large enough (N).

Matlab program:

function hw_ls2_2(a,b,m,n,N,d_x)
x1=[];
x1=[x1 m];
x2=[];
x2=[x2 n];
for ii=1:N
if x2(ii)>0
d_x1=d_x;
else
d_x1=-d_x;
end
d_x2=-(a*x2(ii)+b*x1(ii))*d_x1/x2(ii);
tam1=x1(ii)+d_x1;
tam2=x2(ii)+d_x2;
x1=[x1 tam1];
x2=[x2 tam2];
end
plot(x1,x2);

(a,b coefficient of the differential equation
m,n is the initial condition
N the number of repeat
d_x is the value of
1
x )

This algorithm is not good because we fix the value of the
1
x so the
trajectory is not smooth when the trajectory is conventional. To overcome
this problem we must find a good relation function between
1
( 1) x k + and
22 | P a g e

2
( ) x k so that
1
x and
2
x will come to zeros when there is the
conventional effect.
Another way to solve the above problem and I think it is better if we use a
variant time to define
1
x and
2
x and this way will be introduces

3. The third, we do as the second method, but we dont need to fin the relation
between the x
1
and x
2
, we just use the numerical method to define the next
position base on the relation between (
1
x ,
2
x ) and t .

We have:

1
1 2
dx
x x
dt
= = and
2
2 2 1
. .
dx
x a x b x
dt
= =
We use the approximation here to get:

1
2 1 2
.
x
x x x t
t

= =

2
2 1 2 2 1
. . ( . . )
x
a x b x x a x b x t
t

= = +

Then we can calculate easily the next value.

The Algorithm:
o Set t .
o Set the initial value [x
1
(0);x
2
(0)].
o Calculate:

1
( 1) x k + ,
1
( 1) x k + ,
2
( 1) x k + ,
2
( 1) x k + base on
1
( ) x k
2
( ) x k and t .
Do as follow:

1 2
( 1) ( ). x k x k t + =

2 2 1
( 1) ( . ( ) . ( )) x k a x k b x k t + = +

1 1 1
( 1) ( ) ( 1) x k x k x k + = + +

2 2 2
( 1) ( ) ( 1) x k x k x k + = + +
o Finish the repeat at the time value we want.

Matlab program

function hw_ls2_3(a,b,m,n,T,d_t)
t=0:d_t:T;
tam1=m;
tam2=n;
x1=[];
x2=[];
for ii=1:length(t)
if ii==1
x1=[x1 tam1];
x2=[x2 tam2];
23 | P a g e

else
k=tam2*d_t+tam1;
tam2=tam2-(a*tam2+b*tam1)*d_t;
tam1=k;
x1=[x1 tam1];
x2=[x2 tam2];
end
end
plot(x1,x2);


(a,b coefficient of the differential equation
m,n is the initial condition
T the time we want to draw the trajectory
d_t is the value of t )


Example 1: Draw the trajectory of this equation: x+4x+3x=0

Solving:
We set:

1 2
2 2 1
'
'' 4. ' 3 4. 3.
x x x
x x x x x x
= =
= = =

So we have

1 1
2 2
0 1
.
3 4
x x
x x
( ( (
=
( ( (

Here we draw by 3 method.

The first method we calculate to get the eigenvalue and eigenvector of A as:
The eigenvalue
1 = and 3 =
The eigenvector

1
1
(
(

and
1
3
(
(


So we have:
1 3
1 2
2
( ) 1 1
( )
( ) 1 3
t t
x t
x t c e c e
x t

( ( (
= = +
( ( (

With
1
2
(0)
(0)
(0)
x
x
x
(
=
(

we have
1 1 2
2 1 2
(0)
(0) 3
x c c
x c c
=
= +

Thus
1 2 1 2
1 2
3. (0) (0) (0) (0)
and
2 2
x x x x
c c
+ +
= =

The second and the third will be show by the graph

24 | P a g e

The trajectory of the equation x+4x+3x=0 with the initial condition is
1
2
(0) 3
(0)
(0) 3
x
x
x
( (
= =
( (





Conclusion:
o With the method 1 we can draw exactly the trajectory because it defines
exactly the function of each stage (x
1
;x
2
) but its complex with mathematic
solving the equations.
o The method 2 and method 3 are simpler than the method 1 because we
just use the numerical method to draw the trajectory. Thus, they are easy to
apply for the more complex equation.
o However, between method 2 and method 3 we can recognize that the
method 3 is better than the others because it is closer with the trajectory
(method 1) than the method 2 and it doesnt have the fluctuation when the
trajectory converges to the zeros point.
o So I will use the third method to draw all following examples.







25 | P a g e

Using the method 3 to draw some others case of equation above:

(x
1
,x
2
)= (-4,1);(-2,4) and (-3,3). (x
1
,x
2
)= (2,2);(2,4) and (3,3).


(x
1
,x
2
)= (-2,-2);(-2,-4) and (-3,-3). (x
1
,x
2
)= (1,-4);(2,-4) and (3,-3).










26 | P a g e

Example 2: Draw the trajectory of this equation: x+x+x=0

We use the matlab function of the method 3 with the coefficient is a=1, b=1:




The non-linear second order differential equation:

'' . ' x a x b xc + + =


Processing similar with the linear equation
We set:

1
x x = and
2
' x x =
We have:

1 2
2 2 1
'
'' . ' . .
x x x
x x a x b x c a x b x c
= =
= = + = +

Also:
27 | P a g e

1 1
2 2
0 1 0
.
1
x x
c
x x b a
( ( ( (
= +
( ( ( (

It has a form: . X A X Bc = +

with
1
2
x
X
x
(
=
(


0 1
A
b a
(
=
(


0
1
B
(
=
(


Here we can draw by method 3 with the algorithm:

We have:

1
1 2
dx
x x
dt
= = and
2
2 2 1
. .
dx
x a x b x c
dt
= = +
We use the approximation here to get:

1
2 1 2
.
x
x x x t
t

= =

2
2 1 2 2 1
. . ( . . )
x
a x b x c x a x b x c t
t

= + = +

Then we can calculate easily the next value.

The Algorithm:
o Set t .
o Set the initial value [x
1
(0);x
2
(0)].
o Calculate:

1
( 1) x k + ,
1
( 1) x k + ,
2
( 1) x k + ,
2
( 1) x k + base on
1
( ) x k
2
( ) x k and t .
Do as follow:

1 2
( 1) ( ). x k x k t + =

2 2 1
( 1) ( . ( ) . ( ) ) x k a x k b x k c t + = +

1 1 1
( 1) ( ) ( 1) x k x k x k + = + +

2 2 2
( 1) ( ) ( 1) x k x k x k + = + +
o Finish the repeat at the time value we want.

Matlab program is similar:
function hw_nls2_3(a,b,c,m,n,T,d_t)
t=0:d_t:T;
tam1=m;
tam2=n;
x1=[];
x2=[];
for ii=1:length(t)
if ii==1
x1=[x1 tam1];
x2=[x2 tam2];
else
k=tam2*d_t+tam1;
tam2=tam2-(a*tam2+b*tam1-c)*d_t;
tam1=k;
28 | P a g e

x1=[x1 tam1];
x2=[x2 tam2];
end
end
plot(x1,x2);


Example 3: Draw the trajectory of this equation x+3x+2x=1

We use the matlab function above with the coefficient is a=3, b=2, c=1:



The third order differential equation

I will solve for the non-linear system equation first and linear system is a special
case of non-linear.

29 | P a g e

We have the third order differential equation has the form as:

''' . '' ' x a x b xcx d + + + =


Processing similar with the second order differential equation
We set:

1
x x = ;
2
' x x = and
3
'' x x =
We have:

1 2
2 3
3 3 2 1
'
''
''' . '' ' . . .
x x x
x x x
x x a x b x cx d a x b x c x d
= =
= =
= = + = +

Matrix form:

1 1
2 2
3 3
0 1 0 0
0 0 1 . 0
1
x x
x x d
x c b a x
( ( ( (
( ( ( (
= +
( ( ( (
( ( ( (

It has a form:
. X A X Bd = +

with
1
2
3
x
X x
x
(
(
=
(
(


0 1 0
0 0 1 A
c b a
(
(
=
(
(

0
0
1
B
(
(
=
(
(


Here we can draw by method 3 with the algorithm:
We have:

1
1 2
dx
x x
dt
= =

2
2 3
dx
x x
dt
= =
and
2
3 3 2 1
. .
dx
x a x b x cx d
dt
= = +
We use the approximation here to get:

1
2 1 2
.
x
x x x t
t

= =

2
3 2 3
.
x
x x x t
t

= =

3
3 2 1 3 3 2 1
. . . ( . . . ).
x
a x b x c x d x a x b x c x d t
t

= + = + +

Then we can calculate easily the next value.

The Algorithm:
o Set t .
o Set the initial value [x
1
(0);x
2
(0);x
3
(0)].
o Calculate:

1
( 1) x k + ,
1
( 1) x k + ,
2
( 1) x k + ,
2
( 1) x k + ,
3
( 1) x k + ,
3
( 1) x k +
30 | P a g e

base on
1
( ) x k
2
( ) x k
3
( ) x k and t .
Do as follow:

1 2
( 1) ( ). x k x k t + =

2 3
( 1) ( ). x k x k t + =

3 3 2 1
( 1) ( . ( ) . ( ) ( ) ). x k a x k b x k cx k d t + = + +
and

1 1 1
( 1) ( ) ( 1) x k x k x k + = + +

2 2 2
( 1) ( ) ( 1) x k x k x k + = + +

3 3 3
( 1) ( ) ( 1) x k x k x k + = + +
o Finish the repeat at the time value we want.

Matlab program is similar:

function hw_nls3_3(a,b,c,d,x1_0,x2_0,x3_0,T,d_t)
t=0:d_t:T;
tam1=x1_0;
tam2=x2_0;
tam3=x3_0;
x1=[];
x2=[];
x3=[];
for ii=1:length(t)
if ii==1
x1=[x1 tam1];
x2=[x2 tam2];
x3=[x3 tam3];
else
k=tam2*d_t+tam1;
l=tam3*d_t+tam2;
tam3=tam3-(a*tam3+b*tam2+c*tam1-d)*d_t;
tam1=k;
tam2=l;
x1=[x1 tam1];
x2=[x2 tam2];
x3=[x3 tam3];
end
end
plot3(x1,x2,x3);







31 | P a g e

Example 4: Draw the trajectory of this equation x+2x+1.5x+x=0

This example is presented for the third-order linear differential equation
I chose d_t=0.01 and T=200




32 | P a g e

Example 5: Draw the trajectory of this equation x+3x+2x+1x=4

This example is presented for the third-order non-linear differential equation
I chose d_t=0.01 and T=200