4.

Recurrences
Methods for Solving
Recurrences

Computer Theory Lab.

Recurrences --T ( n ) aT ( n / b ) f ( n )

Chapter 4

A recurrence is an equation or inequality that

describes a function in terms of value on
smaller inputs.

P.2

Computer Theory Lab.

Recurrences and Running

Time

An equation or inequality that describes a

function in terms of its value on smaller inputs.
T(n) = T(n-1) + n

Recurrences arise when an algorithm contains

recursive calls to itself

What is the actual running time of the algorithm?

Need to solve the recurrence

Find an explicit formula of the expression

Bound the recurrence by an expression that involves n

Computer Theory Lab.

Example Recurrences

T(n) = T(n-1) + n (n2)

Recursive algorithm that loops through the input
to eliminate one item
T(n) = T(n/2) + c (lgn)
Recursive algorithm that halves the input in one
step
T(n) = T(n/2) + n (n)
Recursive algorithm that halves the input but
must examine every item in the input
T(n) = 2T(n/2) + 1
(n)
Recursive algorithm that splits the input into 2
halves and does a constant amount of other work

Computer Theory Lab.

Recurrences --T ( n ) aT ( n / b ) f ( n )

Chapter 4

Substitution method
Recursion-tree method
Master method

P.5

Computer Theory Lab.

Technicalities

Chapter 4

We neglect certain technical details when we

state and solve recurrences.
A good example of a detail that is often
glossed over is the assumption of integer
arguments(even or odd) to functions. Omit
floors, ceilings.
Boundary conditions is ignored.

P.6

Computer Theory Lab.

Methods for Solving

Recurrences

Iteration method

Substitution method

Recursion tree method

Master method

Computer Theory Lab.

The Iteration Method

Convert the recurrence into a summation

and try to bound it using known series

Iterate the recurrence until the initial

condition is reached.

Use back-substitution to express the

recurrence in terms of n and the initial
(boundary) condition.

Computer Theory Lab.

The Iteration Method

T(n) = c + T(n/2)
T(n) = c + T(n/2)
T(n/2) = c + T(n/4)
= c + c + T(n/4)
T(n/4) = c + T(n/8)
= c + c + c + T(n/8)
Assume n = 2k
T(n) = c + c + + c + T(1)
= clgn + T(1)
= (lgn)

Computer Theory Lab.

Iteration Method
Example
T(n) = n + 2T(n/2)

Assume: n = 2k

T(n) = n + 2T(n/2)
T(n/2) = n/2 + 2T(n/4)
= n + 2(n/2 + 2T(n/4))
= n + n + 4T(n/4)
= n + n + 4(n/4 + 2T(n/8))
= n + n + n + 8T(n/8)
= in + 2iT(n/2i)
= kn + 2kT(1)
10
= nlgn + nT(1) = (nlgn)
k = lgn & n = 2k

Computer Theory Lab.

4.1 The substitution method

: Mathematical induction

The substitution method for

solving recurrence entails two
steps:
1. Guess the form of the solution.
2. Use mathematical induction to find
the constants and show that the
solution works.

Chapter 4

P.11

Computer Theory Lab.

Substitution method

Guess a solution

T(n) = O(g(n))
Induction goal: apply the definition of the asymptotic
notation

T(n) c g(n), for some c > 0 and n n0

(strong induction)

Induction hypothesis: T(k) c g(k) for all k < n

Prove the induction goal

Use the induction hypothesis to find some values of the

constants c and n0 for which the induction goal holds

Computer Theory Lab.

Example: Binary Search

T(n) = c + T(n/2)

Guess: T(n) = O(lgn)

Induction goal: T(n) d lgn, for some d and n n0

Induction hypothesis: T(n/2) d lg(n/2)

Proof of induction goal:

T(n) = T(n/2) + c d lg(n/2) + c
= d lgn d + c d lgn
if: d + c 0, d c

Base case?

Computer Theory Lab.

Example: Binary Search

Mathematical induction requires our solution to holds for

boundary condition as base cases for inductive proof.

As we know that T(1) = 1

So if n = 1 then,
T(n) <= d log n
T(1) <=d log 1 as log2 1 = 0
T(1) <= 0
1<=0
Consequently, the base case of inductive proof fails to
hold.

Computer Theory Lab.

Example 2
T(n) = T(n-1) + n

Guess: T(n) = O(n2)

Induction goal: T(n) c n2, for some c and n n0

Induction hypothesis: T(n-1) c(n-1)2 for all k < n

Proof of induction goal:

T(n) = T(n-1) + n c (n-1)2 + n
= cn2 (2cn c - n) cn2
if: 2cn c n 0 c n/(2n-1) c 1/(2 1/n)

For n 1 2 1/n 1 any c 1 will work

Computer Theory Lab.

Example 3
T(n) = 2T(n/2) + n

Guess: T(n) = O(nlgn)

Induction goal: T(n) cn lgn, for some c and n n0

Induction hypothesis: T(n/2) cn/2 lg(n/2)

Proof of induction goal:

T(n) = 2T(n/2) + n 2c (n/2)lg(n/2) + n
= cn lgn cn + n cn lgn
if: - cn + n 0 c 1

Base Case ?

as cn log n/2 = cn logn cn log 2

Changing variables
T(n) = 2T( n ) + lgn

Rename: m = lgn n = 2m

T (2m) = 2T(2m/2) + m

Rename: S(m) = T(2m)

S(m) = 2S(m/2) + m S(m) = O(mlgm)

(demonstrated before)

Computer Theory Lab.

Computer Theory Lab.

Example
T ( n ) 2T ( n / 2 ) n

T (1 ) 1

(We may omit the initial condition

later.)
T (n) O(n log n)

Guess
Assume
Chapter 4

T ( n / 2 ) c n / 2 log n / 2

P.18

Computer Theory Lab.

n
T ( n ) 2( c n / 2 log n / 2 ) n cn log n
2
cn log n cn log 2 n cn log n (if c 1.)

Initial condition1 T (1) cn log1 0()

However, 4 T ( 2 ) cn log 2 ( if c 4 )

Chapter 4

P.19

Computer Theory Lab.

Making a good guess

T ( n ) 2T ( n / 2 17 ) n

We guessT ( n ) O( n log n )
Making guess provides loose upper
bound and lower bound. Then
improve the gap.
Chapter 4

P.20

Computer Theory Lab.

Subtleties
T ( n ) T ( n / 2 ) T ( n / 2 ) 1

Guess T ( n ) O( n )
AssumeT ( n ) cn
T ( n ) c n / 2 c n / 2 1 cn 1 cn

T ( n ) cn b
However, assume
T ( n ) ( c n / 2 b ) ( c n / 2 b ) 1
cn 2 b 1 cn b ( Choose b 1 )

Chapter 4

P.21

Computer Theory Lab.

Chapter 4

P.22

Computer Theory Lab.

Chapter 4

P.23

Computer Theory Lab.

Avoiding pitfalls
T ( n ) 2T ( n / 2 ) n

T (1 ) 1
AssumeT ( n ) O( n )

Hence T ( n ) cn
T (n) 2(c n / 2 ) n cn n O(n)
(Since c is a constant)

Chapter 4

(WRONG!) You cannot find such a

c.
P.24

Computer Theory Lab.

The recursion-tree method

Convert the recurrence into a tree:

Each node represents the cost incurred at

various levels of recursion

Sum up the costs of all levels

Used to guess a solution for the recurrence

25

Example 1
W(n) = 2W(n/2) + n2

Subproblem size at level i is: n/2i

Subproblem size hits 1 when 1 = n/2i i = lgn
Cost of the problem at level i = (n/2i)2
No. of nodes at level i = 2i
i
i
lg n 1 2
lg n 1

Total cost:
n
1
1
1

lg n
2
2
2
W (n)

2
i 0

W(n) = O(n2)

2 W (1) n

i 0

nn

i 0

O(n) n

1 1

O ( n) 2n 2

2
26

Computer Theory Lab.

4.2 the Recursion-tree

method

T (n) 3T ( n / 4 ) (n 2 )

Chapter 4

P.27

Computer Theory Lab.

Chapter 4

P.28

Computer Theory Lab.

The cost of the entire tree

Subproblem size at level i is: n/4i

Subproblem size hits 1 when 1 = n/4i i = log4n
Cost of a node at level i = c(n/4i)2
Total cost at level i = 3i c(n/4i)2 = (3/ 16)i cn2
Number of nodes at level i = 3i last level has 3log4n = nlog43 nodes
Total cost:

T (n) cn 2

log 4 n 1

i 0

3 2 3
3
2
cn
cn

...

16
16
16
i

3
log
2
cn

16

log 4 n 1

cn 2 (nlog 3 )
4

(3 / 16)log n 1 2

cn (n log 3 ).
(3 / 16) 1
4

By Equation A.5

Computer Theory Lab.

T ( n)

log 4 n 1

i 0
i

3
log
2

cn n
16

3
log 3
2

cn

i 0 16
1

cn 2 n log
1 (3 / 16)
16

cn 2 ( n log 3 )
13
4

By Theorem
A.6

O(n 2 )
Chapter 4

P.30

Computer Theory Lab.

substitution method
We want to Show that T(n) dn2 for some
constant d > 0. using the same constant c > 0
as before, we have
T ( n)
3T ( n / 4 ) cn 2

3d n / 4

cn 2

3d (n / 4) 2 cn 2
3 2

dn cn 2
16

dn 2 ,

Where the last step holds as long as d (16/13)c.

Chapter 4

P.31

Computer Theory Lab.

T (n) T (n / 3) T (2n / 3) cn

Chapter 4

P.32

Computer Theory Lab.

substitution method
T (n) T (n / 3) T (2n / 3) cn

d (n / 3) lg(n / 3) d (2n / 3) lg(2n / 3) cn

(d (n / 3) lg n d (n / 3) lg 3) (d (2n / 3) lg n d (2n / 3) lg(3 / 2)) cn

dn lg n d ((n / 3) lg 3 (2n / 3) lg(3 / 2)) cn

dn lg n d ((n / 3) lg 3 (2n / 3) lg 3 (2n / 3) lg 2 cn

dn lg n dn(lg 3 2 / 3) cn

dn lg n,

As long as d c/lg3 (2/3)).

Chapter 4

P.33

Computer Theory Lab.

Masters method

Cookbook for solving recurrences of the form:

n
T (n) aT f (n)
b

where, a 1, b > 1, and f(n) > 0

This recurrence describes the running time of an algorithm that
divides a problem of size n into a subproblems, each of size n/b.
The function f(n) encompass the cost of dividing of the problem
and combining the result of the subproblems.
Idea: compare f(n) with nlogba, the larger of two function determines the
solution

f(n) is asymptotically smaller or larger than nlogba by a polynomial factor

n

34

Computer Theory Lab.

Masters method

Cookbook for solving recurrences of the form:

n
T (n) aT f (n)
b
where, a 1, b > 1, and f(n) > 0
Case 1: if f(n) = O(nlogba -) for some > 0, then: T(n) = (nlogba)
Case 2: if f(n) = (nlogba), then: T(n) = (nlogba lgn)
Case 3: if f(n) = (nlogba +) for some > 0, and if
af(n/b) cf(n) for some c < 1 and all sufficiently large n, then:
regularity conditionT(n) = (f(n))

35

Computer Theory Lab.

Masters method

In case 1 f(n) must be smaller than nlogba. But also be polynomially

smaller. That is f(n) must be asymptotically smaller than nlogba by a
factor of n for some constant > 0.
In case 3 f(n) must be larger than nlogba. But also be polynomially
larger. That is f(n) must be asymptotically larger than nlogba and in
addition satisfy the regularity condition that af(n/b) cf(n).
When f(n) is not polynomally smaller or larger than nlogba. and f(n) is
polynomally smaller or larger but reularity condition fails to hold,
master theorem can not be used to solve the problem.

36

Computer Theory Lab.

Chapter 4

P.39

Computer Theory Lab.

Chapter 4

P.40

Examples
T(n) = 2T(n/2) + n
a = 2, b = 2, log22 = 1
Compare nlog22 with f(n) = n

As 1 < n case 3 but it is 1 hence case 2.

f(n) = (n) Case 2
T(n) = (nlgn)

41

Examples
T(n) = 2T(n/2) + n2
a = 2, b = 2, log22 = 1
Compare n with f(n) = n2
f(n) = (n1+) Case 3 verify regularity cond.
a f(n/b) c f(n)
2 n2/4 c n2 c = is a solution (c<1)
T(n) = (n2)
42

Examples (cont.)
T(n) = 2T(n/2) +

a = 2, b = 2, log22 = 1
Compare n with f(n) = n1/2
f(n) = O(n1-)

Case 1

T(n) = (n)

43

Examples
T(n) = 3T(n/4) + nlgn
a = 3, b = 4, log43 = 0.793
Compare n0.793 with f(n) = nlgn
f(n) = (nlog43+) Case 3
Check regularity condition:
3(n/4)lg(n/4) (3/4)nlgn = c f(n), c=3/4
T(n) = (nlgn)
44

Examples
T(n) = 2T(n/2) + nlgn
a = 2, b = 2, log22 = 1
Compare n with f(n) = nlgn
seems like case 3 should apply

f(n) must be polynomially larger by a factor of n

In this case it is only larger by a factor of lgn

45

Computer Theory Lab.

Chapter 4

The master method does not apply to

T (n) 2T (n / 2) n lg n,
the recurrence
even though it has the proper form: a
log b a
= 2, b=2, f(n)= n lgn,
n and n. It might
seem that case 3 should apply, since
log b a
f(n)= n lgn is asymptotically larger
n
n.
than
The problem is that it is not
polynomially larger.
P.46