DR Nazir A. Zafar Advanced Algorithms Analysis and Design

Advanced Algorithms Analysis
and Design
By

Dr. Nazir Ahmad Zafar
Dr Nazir A. Zafar Advanced Algorithms Analysis and Design
Lecture No. 7

Recurrence Relations

Mathematical Models, Analysis Techniques
What is Recursion?
Recursive Mathematical Models
Solving Recurrence Relations
First Order Linear Homogenous Recurrence
Relations, with Constant Coefficients
Second Order Linear Homogenous Recurrence
Relations, Constant Coefficients
Characteristics of Second Order Recurrences
Solution to Second order with distinct roots
Solution to Second order with repeated roots
General Homogenous Recurrences
Characteristics and solution
Today Covered
Some times problems are too difficult or too
complex to solve because these are too big.
A problem can be broken down into sub-problems
and find a way to solve these sub-problems
Then build up to a solution to the entire problem.
This is the idea behind recursion
Recursive algorithms break down problem in pieces
which you either already know the answer, or can
be solved applying same algorithm to each piece
And finally combine the results of these sub-
problems to find the final solution
What is Recursion?
More concisely, a recursive function or definition
is defined in terms of itself.
Recursion is a computer algorithm that calls itself
in a steps having a termination condition.
The successive repetitions are processed up to
the critical step where the condition is met.
In recursive algorithms, each repetition is
processed from the last one called to the first.
Recursion is a wonderful technique for dealing
with many problems where problems and sub-
problems have same mechanism to solve it.
What is Recursion?
Recursive solutions are much easier to conceive of
and code than their iterative counterparts.
Every problem is not solvable using this approach
What kinds of problems are solved with recursion?
Generally, problems which are defined in terms of
themselves are usually good candidates for
recursive techniques.

Example
Finding factorial of a number is an easiest example
one can think using recursion
Merits and Demerits of Recursion
Since n! can be computed as: 5! = 5*4*3*2*1.
If we have a close look at this, we notice that
5! = 5*4!.
Now if denote F(n) = n! then it can be written as
F(n) = n.F(n-1)
Assume 0! = 1 i.e. F(0) = 1, and solve till termination
F(n) = n.F(n-1) = n.(n-1).F(n-2) = n.(n-1).(n-2).F(n-3)
. . .
F(n) = n.(n-1).(n-2). . . 2.1.F(0) = n.(n-1).(n-2). . . 2.1.1

otherwise ) 1 ( .
0 if 1
) (
=
=
n F n
n
n F
Recursive Mathematical Model
The indispensable last step when analyzing an
algorithm is often to solve a recurrence equation.
With little experience recurrence equation can be
solved by intelligent guesswork.
However, there exist a powerful techniques that
can be use to solve certain classes of recurrence
almost automatically.
This is a main topic of this lecture, techniques
solving recurrence equations of form:
First order linear homogenous recurrences
Second order linear homogenous recurrences
Higher order recurrences
Solving Recurrence
First Order Linear Homogenous Recurrence
Relations with Constant Coefficients
(FOLHRRCC)

. 2
2
=
k k
a a
1 k
5 b

=
k
b

2
1
c
1 k
=
k
c k
No
YES
No
Definition and Examples of FOLHRRCC

1 1
=
k k k
a a a No
1
1
> =

k a A a
Definition
k k
0 A and numbers real a is where = A
Solve the recurrence: if b
0
= c

Solution
b
k
= a.b
k-1
b
k
= a.(a. b
k-2
) = a
2
. b
k-2

b
k
= a
2
.(a. b
k-3
) = a
3
. b
k-3

. . .
b
k
= a
k
. b
k-k
= a
k
. b
0

b
k
= a
k.
c
Now the explicit formula is c.a
k
which is a
geometric sequence.
Hence first order recurrence is nothing but G.S.
1 k
. b

=
k
b a
Solving First Order Recurrences
Example
Solve the recurrence: if b
0
= 3

Solution
b
k
= 5.b
k-1
b
k
= 5.(5. b
k-2
) = 5
2
. b
k-2

b
k
= 5
2
.(5. b
k-3
) = 5
3
. b
k-3

. . .
b
k
= 5
k
. b
k-k
= 5
k
. b
0
= 5
k.
3
Now the solution sequence becomes
3.5
0
, 3.5
1
, 3.5
2
, 3.5
3
, 3.5
4
, . . , 3.5
k
, . . .
3, 15, 75, 375, . . .geometric sequence
1 k
5 b

=
k
b
Solving First Order Recurrences
Second Order Linear Homogenous Recurrence
Relations with Constant Coefficients

2 3
2 1
+ =
k k k
a a a
b
3 2 1 k
+ + =
k k k
b b b

7
3
2
1
c
2 1 k
=
k k
c c
YES
NO
YES
Definition and Examples of SOLHRRCC
2

2 1
> + =

k a B a A a
Definition
k k k
0 with numbers real are and where = B B A
d
2 1
2
1 k
+ =
k k k
d d d
2 e
2 k
=
k
e
NO
YES
1 2 f
1 k
+ =
k
f
2 - k 1 k
g g g
k
+ =

2 1 k
) 1 ( ) 1 ( h

+ =
k k
h k h
NO
YES
NO
Some More Examples
2 1
+ =
k k k
Ba Aa a
0 ,.... ,.... , , 1
2
= t t t t
n
the satisfies if only and if no, real zero - non a is where t t
0 : equation
2
= B At t
Theorem: Let A and B are real numbers. A recurrence of the form
is satisfied by the sequence
Proof:
Let us suppose that the sequence
0 , ,..., ,..., , , 1
2
= t where t t t
n
Satisfies the recurrence relation
2 1
+ =
k k k
Ba Aa a
Theorem
2 1
+ =
k k k
Bt At t
0 0 since
2
= =
k
t t
2 k
t
It mean each form of sequence is equal to A times
the previous form plus B times the form before it

Hence
0
2
= B At t
Dividing both sides by
Conversely suppose that 0
2
= B At t
2 1
+ =
k k k
Bt At t
... ..., , , , 1 Hence
3 2 n
t t t t
satisfies the above
recurrence relation
Contd...
2
2 1
> + =

k a B a A a
k k k
0
2
= B At t
2 2
2 1
> + =

k a a a
k k k
Definition:
Given a second order linear homogeneous
recurrence relation with constant coefficients
The characteristic equation of the relation is
Example 1:
Using characteristic equation, find solution to a
recurrence relation
Find all sequences that satisfy relation and have the form
0 ,..., ,..., , , 1
2
= t t t t
n
Solution: Above recurrence relation is satisfied by a sequence
0 ,..., ,..., , , 1
2
= t t t t
n
Characteristic Equation
0 2 if, only and if
2
= t t
( )( ) 1 , 2 0 1 2 = = + t t t
Hence
,... 2 ,..., 2 , 2 , 2 , 2
3 2 0 n
and
( ) ( ) ( ) ( ) ,... 1 , 1 , 1 , 1
3 2 1 0

are both particular solutions for this recurrence relation
Contd...
0
2
= B At t
... ,..., 3 , 2 , 0, and ,..., , , 1
3 2 2 n n
nr r r r r r r
Theorem
Let A and B be real numbers, and suppose that
the characteristic equation
has a single root r, than the sequences
Repeated Roots
2 1
+ =
k k k
Ba Aa a
Both satisfy the recurrence relation
0 if
2
= B At t
( )
2
2 2 2
2
2
and 2
2

r B r A
r rt t B At t
r t B At t
= =
+ =
=
Repeated Roots
Proof
has a single repeated root r than
We know that r
n
is a solution. Now we prove that
n
n
nr S =
is also a solution, i.e.
2 1
+ =
k k k
Ba Aa a
n
S satisfies the recurrence
k
k k k
kr BS AS e i = + =
2 1
S ., .
( ) ( )
proved hence RHS, k.r
) 2 2 2 (
) 2 ( ) 1 ( 2
) 2 ( ) 1 ( 2
) 2 ( ) 1 (
side hand left he Consider t
k
2 2 1
2 1
2 1
= =
+ =
=
=
+ = +

k
k k
k k
k k
k k
r k k
r k r k
r k r r k r
r k B r k A BS AS
Repeated Roots
k
k k
r k S B S A . . .
that prove to have fact we In
2 1
= +

( )
conditions initial e satisfy th not do but relation, recurrence
same e satisfy th which sequences are 2 . and 2
root repeated is , 2 0 2
n
2
n
n
t t = =
3 and 1 b : condition initial with
2 4 4
1 0
2 1
= =
> =

b
k b b b
k k k
Repeated Roots
Example 1: Single root case
Suppose sequence, b0, b1, b2,. . . satisfies recurrence relation
then find the explicit formula for b0, b1, b2, . . .
Solution:
Characteristic equation
0 4 4
2
= + t t
1 1 2 0 , 0 or
3 , 1 Since
constants are D and C , recurrence original the satisfies which
2 . . 2 . b : solution general Suppose
0
1 0
n
= = + =
= =
+ =
C D C n F
b b
n D C
n n
Repeated Roots
roots. repeated for solution required the is 2
2
1
2
1 2 2
2
1
2 1 Hence,
2
1
2
2 3
3 2 1 2 3 2 2
2 1 2 , 1 For
1 1
1
n
n n n
n
n
n
n b
D D D C
D C b n
|
.
|
\
|
+ =
|
.
|
\
|
+ = + =
=
= = + = +
+ = =
b
n
= (1 + n/2).2
n

First term
b
0
= (1 + 0/2).2
0
= 1.1 = 1
Second term
b
1
= (1 + 1/2).2
1
= 3/2.2 = 3
Third term
b
2
= (1 + 2/2).2
2
= 2.4 = 8
Fourth term
b
3
= (1 + 3/2).2
3
= 5/2.8 = 20, and so on
Checking Explicit Formula
,... , ,
2 1 0
r r r
,... , ,
2 1 0
s s s
Theorem:
are sequences that satisfy the some second order
linear homogeneous recurrence relation with
constant coefficients, and if C and D are any
numbers then the sequence
also satisfies the same recurrence relation
0 > + = n Ds Cr a
n n n

if and
defined by the formula
,... , ,
2 1 0
a a a
Proof:
Since
and
,..., , ,
2 1 0
r r r
,... , ,
2 1 0
s s s
Satisfy the same second order LHRRWCC
s.t no. real constants and B A -
Theorem (Linear Combination is also a Solution)
2 1
+ =
k k k
Br Ar r
2 1
+ =
k k k
Bs As s
and
Then we have to prove that 2 1
+ =
k k k
Ba Aa a
Consider R. H. S
( ) ( )
2 2 1 1 2 1
+ + + = +
k k k k k k
Ds Cr B s D r C A Ba Aa
( ) ( )
2 1 2 1
+ + + =
k k k k
Bs As D Br Ar C
( )
k
a ) ( = + =
k k
s D r C
2 1

+ =
k k k
Ba Aa a
It proves that
n n n
Ds Cr a + =
satisfies same recurrence
Theorem
0 > + = n Ds Cr a If
n n n
2 2
2 1
> + =

k a a a
k k k
8 and 1
1 0
= = a a
2 2
2 1
> + =

k a a a
k k k
Example 1:
Find a sequence that satisfies the recurrence relation
And that also satisfies the initial condition
Solution: The characteristics equation of the relation
is
( )( ) 0 1 2 0 2
2
= + = t t t t
2 , 1 = t
Solution as Linear Combination
8 and 1
1 0
= = a a

n n n
Ds Cr a + =
o o o
Ds Cr a + = = get we 0 n For
Are both sequence which satisfy above relation but neither one is having
Now since also satisfies the same recurrence relation and
(1) 1 D C+ =
2 - D 3 get, we (2) and (1) equation Solving = = and C
Now
n n n
Ds Cr a + = ( ) ( ) 1 2 2 3
n n
n
a =
,... 2 , 2 , 2
2 1 0
=
n
r
( ) ( ) ( ) ,... 1 , 1 , 1
2 1 0
=
n
s
and
1 1 1
get we 1 n For Ds Cr a + = =
(2) 2 8 D C =
conditions given the satisfies which sequence required the is
Contd..
General Homogenous Recurrence Relations
(Constant Coefficients)
First order: linear combination of two cons. terms,
Second order: linear combination of three
consecutive terms,
We extend technique of recurrence equation with
resolution of homogeneous linear recurrence with
constant coefficient, that is recurrence of the form
a
0
t
n
+ a
1
t
n-1
+ ..+a
k
t
n-k
= 0 (1)
where t
i
are value we are looking for

In equation (1), values of t
i
such that (1 < i < k) are
need to determine a sequence.
Equation 1 typically has infinite many solutions,
because linear combination is also a solution.
K order: General Homogenous Recurrence
a
0
t
n
+ a
1
t
n-1
+ ..+a
k
t
n-k
= 0 (1)

If we put k = 1 then above equation becomes
a
0
t
n
+ a
1
t
n-1
= 0
t
n
= -(a
1
/ a
0
)t
n-1

The resultant equation becomes a recurrence
relation which is
first order
linear
homogenous
with constant coefficients.
K = 1: General Homogenous Recurrence
a
0
t
n
+ a
1
t
n-1
+ ..+a
k
t
n-k
= 0 (1)

a
0
t
n
+ a
1
t
n-1
+ a
2
t
n-2
= 0
t
n
= -(a
1
/ a
0
)t
n-1
- (a
2/
a
0
)t
n-2
= c
1
t
n-1
+ c
0
t
n-2

This time we have a recurrence relation which is
second order
linear
homogenous
a
0
t
n
+ a
1
t
n-1
+ ..+a
k
t
n-k
= 0 (1)
a
0
t
n
+ a
1
t
n-1
+ a
2
t
n-2
+ a
3
t
n-3
= 0
t
n
= -(a
1
/ a
0
)t
n-1
- (a
2/
a
0
)t
n-2
- (a
3/
a
0
)t
n-3

This is a recurrence relation which is
third order
linear
homogenous
Similarly we can have fourth order and higher
order recurrence relations.
a
0
t
n
+ a
1
t
n-1
+ ..+a
k
t
n-k
= 0 (1)
The recurrence is:
Linear because it does not contain terms of the
form t
n-i
.t
n-j
,t
2
n-i
and so on
Homogeneous because the linear combination of
the t
n-i
equal to zero
This is K
th
order because current term is linear
combination of previous k number of terms
Constant coefficients because all a
i
are constants
Example:
Consider Fibonacci sequence: f
n
= f
n-1
+ f
n-2

This recurrence fits Eq. (1) after obvious rewriting.
f
n
- f
n-1
- f
n-2
= 0
Characteristics of General Recurrence
f
n
- f
n-1
- f
n-2
= 0
The Fibonacci sequence corresponds to a second
homogeneous linear recurrence relation with
constant coefficient, where
k = 2,
a
0
=1,
a
1
= -1
a
2
= -1
Before we even start to look for solutions to
Equation 1, it would be interesting to note that any
linear combination of solutions is itself a solution.
Observation of Homogenous Recurrence
Statement: Prove that any linear combination of
solutions of equation given below is also a solution.
a
0
t
n
+ a
1
t
n-1
+ ..+a
k
t
n-k
= 0
Proof: Assume that f
n
and g
n
are solutions of above
equation and hence satisfy it, i.e.

If we set t
n
= c.f
n
+ d.g
n
for arbitrary constants c
and d, we have to prove that t
n
is also solution, i.e.,

a
0
t
n
+ a
1
t
n-1
++ a
k
t
n-k
= 0
0 and 0
0 0
= =

=

=

k
i
i n i
k
i
i n i
g a f a
Theorem
a
0
t
n
+ a
1
t
n-1
++ a
k
t
n-k
=
a
0
(cf
n
+ dg
n
) + a
1
(cf
n-1
+ dg
n-1
) + . . .+a
k
(cf
n-k
+dg
n-k
) =
c( a
0
f
n
+ a
1
f
n-1
+ . . .+ a
k
f
n-k
) +d(a
0
g
n
+ a
1
g
n-1
+ . . .+
a
k
g
n-k
) =
= c.0 + d.0 = 0
Hence a
0
t
n
+a
1
t
n-1
++ a
k
t
n-k
= 0

Note :
It is to be noted that this rule can be generalized to
linear combinations of any number of solutions.
Theorem
Statement:
If f
n
, g
n
and h
n
are solutions to recurrence below
then their linear combination is also a solution
a
0
t
n
+ a
1
t
n-1
+ ..+a
k
t
n-k
= 0
Proof: As f
n
, g
n
and h
n
are solutions of above, hence

Let t
n
= c.f
n
+ d.g
n
+ e.h
n
for arbitrary constants c, d
and e, now we prove that t
n
is also solution, i.e.,

a
0
t
n
+ a
1
t
n-1
++ a
k
t
n-k
= 0, easy to prove
0 and , 0 , 0
0 0 0
= = =

=

=

=

k
i
i n i
k
i
i n i
k
i
i n i
h a g a f a
More Generalized Results
Why recurrence?
Type of recurrences
Techniques solving recurrences
Generalized form of recurrence
Linear combination of solutions itself solution
Reusability of models when initial conditions are
changed but pattern remains same
It empowers the recursive models to be used in
more general phenomena
Easy implementation
Conclusion

DR Nazir A. Zafar Advanced Algorithms Analysis and Design

Diunggah oleh

Informasi Dokumen

Deskripsi Asli:

Judul Asli

Hak Cipta

Format Tersedia

Bagikan dokumen Ini

Bagikan atau Tanam Dokumen

Opsi Berbagi

Apakah menurut Anda dokumen ini bermanfaat?

Apakah konten ini tidak pantas?

Hak Cipta:

Format Tersedia

DR Nazir A. Zafar Advanced Algorithms Analysis and Design

Diunggah oleh

Hak Cipta:

Format Tersedia

Advanced Algorithms Analysis

Anda mungkin juga menyukai