Algorithms

Algorithms

This course aims to:

― give you a general understanding of the basic design of
different types of algorithms

3/1/2019 Algorithms 2
 Algorithms

This course aims to:

― give you a general understanding of the basic design of
different types of algorithms

3/1/2019 Algorithms 3
 Algorithms
Recommended Readings:

Text Book(s):
1. ‘Introduction to Algorithms’ by Cormen, Leiserson,
Rivest and Stein
2. ‘Fundamentals of Computer Algorithms’ by Horowitz,
Sahni, Rajasekaran

Recommended Online Reads:

1. https://www.tutorialspoint.com/computer_fundamen
tals/index.htm
2. http://ecomputernotes.com/fundamental

3/1/2019 Algorithms 4
 Algorithms

INTRODUCTION
 Introduction
― Algorithms have a long history and
the word can be traced back to the
9th century.

― Persian author Abu Ja'far Mohammed

ibn Musa al Khowarizmi (780-850
A.D) is often cited as “The father of
Algebra”.

― He wrote a textbook on mathematics

on solving linear and quadratic
equations. The book contained some
of the first algorithms.

3/1/2019 Algorithms 6
 Introduction

Image: https://www.visualcapitalist.com/intro-to-algorithms/

3/1/2019 Algorithms 7
 Algorithms
― Computational procedures for solving some problem

― An algorithm is a finite set of instructions that, if

followed, accomplishes a particular task

3/1/2019 Algorithms 8
 Properties of Algorithm

Input
Zero or more quantities that are externally supplied.
Output
At least one quantity is produced
Definiteness
Each instruction should be clear and unambiguous.
Finiteness
The process terminates after a finite number of steps
Effectiveness
Every instruction must be basic enough to be carried out
theoretically or by using paper and pencil.

3/1/2019 Algorithms 9
 Algorithms

GROWTH OF FUNCTIONS
AND
AYMPTOTIC NOTATION
 Asymptotic Notation

― When we study algorithms, we are interested in

characterizing them according to their efficiency.

― We are usually interesting in the order of growth of the

running time of an algorithm, not in the exact running
time. This is also referred to as the asymptotic running
time.

― Asymptotic notation gives us a method for classifying

functions according to their rate of growth.

3/1/2019 Algorithms 11
 Asymptotic Notation

― Different types of asymptotic notations are used to

represent the complexity of an algorithm. Following
asymptotic notations are used to calculate the running
time complexity of an algorithm.

 O − Big O

 Ω − Big omega

 θ − Big theta

3/1/2019 Algorithms 12
 Big O Notation
― The asymptotic upper bound

― The function 𝐟 𝐧 = 𝐎(𝐠(𝐧)) if and only if there exist positive

constants c and n0 such that f(n) ⩽ c.g(n) for any n > n0 .

― read as “f of n is big oh of g of n”

3/1/2019 Algorithms 13
 Big O Notation

Example: 3𝑛 + 2 = O 𝑛 ?

• Here we have 𝑓 𝑛 = 3𝑛 + 2, and 𝑔 𝑛 = 𝑛

• We need to prove that 𝟑𝒏 + 𝟐 ≤ 𝒄𝒏 for some constant c.

• We notice that when c = 4, we have 3𝑛 + 2 ≤ 4𝑛 for all

𝑛 ≥ 2.
• Therefore,
𝑓 𝑛 = O 𝑔(𝑛)
or 3𝑛 + 2 = O 𝑛
with c = 4 and 𝑛0 = 2

3/1/2019 Algorithms 14
 Big O Notation

Example: 𝑛2 + 𝑛 = 𝑂 𝑛3 ?

• Here we have 𝑓 𝑛 = 𝑛2 + 𝑛, and 𝑔 𝑛 = 𝑛3

• Notice that if 𝑛 > 1, we have 𝑛 < 𝑛3 .
• Also, notice that if 𝑛 > 1, we have n2 < 𝑛3
• Therefore,
𝑛2 + 𝑛 ≤ 𝑛3 + 𝑛3 = 2𝑛3
• We have just shown that
𝑛2 + 𝑛 ≤ 2𝑛3 for all 𝑛 ≥ 1

• Thus, we have shown that 𝑛2 + 𝑛 = 𝑂 𝑛3 with

c = 2 and 𝑛0 = 1

3/1/2019 Algorithms 15
 Big Ω Notation
― The asymptotic lower bound

― The function 𝐟 𝐧 = Ω(𝐠(𝐧)) if and only if there exist positive

constants c and n0 such that f(n) ⩾ c.g(n) for any n > n0 .

― read as “f of n is omega of g of n”

3/1/2019 Algorithms 16
 Big Ω Notation

Example: 3𝑛 + 2 = Ω 𝑛 ?

• Here we have 𝑓 𝑛 = 3𝑛 + 2, and 𝑔 𝑛 = 𝑛

• Notice that for any 𝑛 ≥ 1, we have 3𝑛 + 2 ≥ 𝑛.

• Therefore,
𝑓 𝑛 = Ω 𝑔(𝑛)
or 3𝑛 + 2 = Ω 𝑛
with c = 1 and 𝑛0 = 1

3/1/2019 Algorithms 17
 Big θ Notation
― The asymptotic tight bound

― The function 𝐟 𝐧 = 𝛉(𝐠(𝐧)) if and only if there exist positive

constants c1 , c2 and n0 such that c1 . 𝑔 𝑛 ≤ f 𝑛 ≤ c2 . 𝑔(𝑛) for
any n > n0 .

― read as “f of n is theta of g of n”

3/1/2019 Algorithms 18
 Big θ Notation

Example: n2 + 5n + 7 = θ(n2 ) ?

• Here we have 𝑓 𝑛 = n2 + 5n + 7, and 𝑔 𝑛 = n2

• When 𝑛 ≥ 1, we have
n2 + 5n + 7 ≤ n2 + 5n2 + 7n2
≤ 13n2
• Again, when n ≥ 1,
𝑛2 ≤ n2 + 5n + 7
• Thus, 𝑓 𝑛 = θ 𝑔(𝑛) or n2 + 5n + 7 = θ(n2 )
with 𝑛0 = 1, 𝑐1 = 1 and 𝑐2 = 13

3/1/2019 Algorithms 19
 Rate of Growth

• Consider the example of buying Laptop and Earphone:

Cost = cost_of_laptop + cost_of_earphone
Cost ≈ cost_of_laptop (approximation)

• Example: 2n2 + 4n + 6 ≈ n2

• The low order terms in a function are relatively

insignificant for large n

3/1/2019 Algorithms 20
 Rate of Growth

• We say that 2n2 + 4n + 6 and n2 have the same rate of

growth. Therefore, 2n2 + 4n + 6 = O(n2)

3/1/2019 Algorithms 21
 Common Rates of Growth

3/1/2019 Algorithms 22
 Common Rates of Growth

3/1/2019 Algorithms 23
 Summary

3/1/2019 Algorithms 24
 Try Yourself!

Show that:

• 3n2 + 4n - 2 = O(n2)
• 4n3 + 10n2 + 5n + 2 = Ω(n3)
• 3logn + 100 = O(logn)
• n + logn = θ(n)

• n3 ≠ O(n2)
• nlogn – 2n + 13 = Ω(nlogn)

3/1/2019 Algorithms 25
 Algorithms

TIME COMPLEXITY OF
ITERATIVE PROGRAMS
 Example - 1
Alg()
{
int i;
for i = 1 to n
{
print(“Hello!”);
}
}

3/1/2019 Algorithms 27
 Example - 1
Alg()
{
int i;
for i = 1 to n
{
print(“Hello!”);
}
}

𝑂(𝑛)
3/1/2019 Algorithms 28
 Example - 2
Alg()
{
int i, s = 0;
while (s < n)
{
i++;
s = s + i;
print(“Hello!”);
}
}

3/1/2019 Algorithms 29
 Example - 2
Alg()
{
int i, s = 0;
while (s < n)
{
i++;
s = s + i;
print(“Hello!”);
}
}

𝑂( 𝑛)
3/1/2019 Algorithms 30
 Example - 3
Alg()
{
int i;
for (𝒊 = 1; 𝒊𝟐 < 𝒏; 𝒊++)
{
print(“Hello!”);
}
}

3/1/2019 Algorithms 31
 Example - 3
Alg()
{
int i;
for (𝒊 = 1; 𝒊𝟐 < 𝒏; 𝒊++)
{
print(“Hello!”);
}
}

𝑂( 𝑛)
3/1/2019 Algorithms 32
 Example - 4
Alg()
{
int i, j, k;
for (𝒊 = 1; 𝒊 ≤ 𝒏; 𝒊++)
{
for (𝒋 = 𝟏; 𝒋 ≤ 𝒊; 𝒋++)
{
for (𝒌 = 𝟏; 𝒌 ≤ 𝟏𝟎𝟎; 𝒌++)
{
print(“Hello!”);
}
}
}
}

3/1/2019 Algorithms 33
 Example - 4
Alg()
{
int i, j, k;
for (𝒊 = 1; 𝒊 ≤ 𝒏; 𝒊++)
{
for (𝒋 = 𝟏; 𝒋 ≤ 𝒊; 𝒋++)
{
for (𝒌 = 𝟏; 𝒌 ≤ 𝟏𝟎𝟎; 𝒌++)
{
print(“Hello!”);
}
}
}
} 2
𝑂(𝑛 )
3/1/2019 Algorithms 34
 Example - 5
Alg()
{
int i, j, k;
for (𝒊 = 1; 𝒊 ≤ 𝒏; 𝒊++)
{
for (𝒋 = 𝟏; 𝒋 ≤ 𝒊𝟐 ; 𝒋++)
{
𝒏
for (𝒌 = 𝟏; 𝒌 ≤ ; 𝒌++)
𝟐
{
print(“Hello!”);
}
}
}
}

3/1/2019 Algorithms 35
 Example - 5
Alg()
{
int i, j, k;
for (𝒊 = 1; 𝒊 ≤ 𝒏; 𝒊++)
{
for (𝒋 = 𝟏; 𝒋 ≤ 𝒊𝟐 ; 𝒋++)
{
𝒏
for (𝒌 = 𝟏; 𝒌 ≤ ; 𝒌++)
𝟐
{
print(“Hello!”);
}
}
}
} 2
𝑂(𝑛 )
3/1/2019 Algorithms 36
 Example - 6
Alg()
{
int i;
for (𝒊 = 1; 𝒊 ≤ 𝒏; 𝒊 = 𝒊 ∗ 𝟐)
{
print(“Hello!”);

}
}

3/1/2019 Algorithms 37
 Example - 6
Alg()
{
int i;
for (𝒊 = 1; 𝒊 ≤ 𝒏; 𝒊 = 𝒊 ∗ 𝟐)
{
print(“Hello!”);

}
}

𝑂(𝑙𝑜𝑔2 𝑛)
3/1/2019 Algorithms 38
 Example - 6B
Alg()
{
int i;
for (𝒊 = 1; 𝒊 ≤ 𝒏; 𝒊 = 𝒊 ∗ 𝟒)
{
print(“Hello!”);

}
}

Try yourself! 
3/1/2019 Algorithms 39
 Example - 7
Alg()
{
int i, j, k;
𝒏
for (𝒊 = ; 𝒊 ≤ 𝒏; 𝒊++)
𝟐
{
𝒏
for (𝒋 = 𝟏; 𝒋 ≤ ; 𝒋++)
𝟐
{
for (𝒌 = 𝟏; 𝒌 ≤ 𝒏; 𝒌 = 𝒌 ∗ 𝟐)
{
print(“Hello!”);
}
}
}
}

3/1/2019 Algorithms 40
 Example - 7
Alg()
{
int i, j, k;
𝒏
for (𝒊 = ; 𝒊 ≤ 𝒏; 𝒊++)
𝟐
{
𝒏
for (𝒋 = 𝟏; 𝒋 ≤ ; 𝒋++)
𝟐
{
for (𝒌 = 𝟏; 𝒌 ≤ 𝒏; 𝒌 = 𝒌 ∗ 𝟐)
{
print(“Hello!”);
}
}
}
}
2
𝑂(𝑛 𝑙𝑜𝑔2 𝑛)
3/1/2019 Algorithms 41
 Example - 8
Alg()
{
int i, j, k;
𝒏
for (𝒊 = ; 𝒊 ≤ 𝒏; 𝒊++)
𝟐
{
for (𝒋 = 𝟏; 𝒋 ≤ 𝒏; 𝒋 = 𝒋 ∗ 𝟐)
{
for (𝒌 = 𝟏; 𝒌 ≤ 𝒏; 𝒌 = 𝒌 ∗ 𝟐)
{
print(“Hello!”);
}
}
}
}
Try yourself! 
3/1/2019 Algorithms 42
 Example - 9
Alg()
{
int i, j, k;
𝒏
for (𝒊 = ; 𝒊 ≤ 𝒏; 𝒊++)
𝟐
{
𝒏
for (𝒋 = 𝟏; 𝒋 ≤ ; 𝒋++)
𝟐
{
for (𝒌 = 𝟏; 𝒌 ≤ 𝒏; 𝒌 = 𝒌 ∗ 𝟐)
{
print(“Hello!”);
}
}
}
}

3/1/2019 Algorithms 43
 Example - 9
Alg()
{
int i, j, k;
𝒏
for (𝒊 = ; 𝒊 ≤ 𝒏; 𝒊++)
𝟐
{
𝒏
for (𝒋 = 𝟏; 𝒋 ≤ ; 𝒋++)
𝟐
{
for (𝒌 = 𝟏; 𝒌 ≤ 𝒏; 𝒌 = 𝒌 ∗ 𝟐)
{
print(“Hello!”);
}
}
}
}
2
𝑂(𝑛 𝑙𝑜𝑔2 𝑛)
3/1/2019 Algorithms 44