Analysis and Complexity:2

P. P. Chakrabarti

10-04-03

P.P.Chakrabarti, IIT Kharagpur

Analysis of Algorithms
? ?

Quantifying the resources required. Measures of resource utilization (efficiency):

? ?

Execution time ? Memory space ?

time complexity space complexity

Observation :
?

The larger the input data the more the resource requirement: Complexities are functions of the amount of input data (input size).
Refer to first few chapters of the book Introduction To Algorithms by Cormen, Leiserson, Rivest and Stein

10-04-03

P.P.Chakrabarti, IIT Kharagpur

Yardstick?
?The same algorithm will run at different speeds and will require different amounts of space when run on different computers, different programming languages, different compilers. ?Algorithms usually consume resources in some fashion that depends on the size of the problem they solve. ?Need a machine independent complexity measure that is a function of input size, n. (Also we are interested at asymptotic behaviour, that is, when n becomes large.
3
10-04-03 P.P.Chakrabarti, IIT Kharagpur

Analyzing SelSort
int max_loc(int x[], int k, int size) { int j, pos; pos = k; for (j=k+1; j<size; j++) if (x[j] > x[pos]) pos = j; return pos; } int selsort (int x[], int size) { int k, m, temp; for (k=0; k<size-1; k++) { m = max_loc(x, k, size); temp = x[k]; x[k] = x[m]; x[m] = temp; } }
?

Computer 1: f1(n) = 0.0007772 n2 + 0.00305 n + 0.001 ? Computer 2: f2(n) = 0.0001724 n2 + 0.00040 n + 0.100 Note: Both are quadratic functions of n ? The shape of the curve that expresses the running time as a function of the problem size stays the same.

We say that Selection Sort is of complexity Order n 2 or O(n2)

10-04-03 P.P.Chakrabarti, IIT Kharagpur

Complexity classes
?

The running time for different algorithms fall into different complexity classes.
Each complexity class is characterized by a different family of curves. ? All curves in a given complexity class share the same basic shape. (In the Asymptotic sense)
?

The O-notation is used for talking about the complexity classes of algorithms.
10-04-03 P.P.Chakrabarti, IIT Kharagpur

Order of Complexity, O- notation

?

For the quadratic function f(n) = an2 + bn + c we will say that f(n) is O(n2).
?

We focus on the dominant term, and ignore the lesser terms; then throw away the coefficient. [Asymptotic Analysis] Since constants are finally not important we may assume that each machine instruction takes one unit of time when we analyze the complexity of an algorithm. We may sometimes abstract this to unit time operators like add, sub, multiply, etc and do an operator count. We can perform worst case or average case analysis.

10-04-03

P.P.Chakrabarti, IIT Kharagpur

How execution time is affected by various complexity measures:

Assume speed S is 107 instructions per second.
size
10 .001 ms 20 .002 ms .008 ms .04 ms .8 ms .1 s 30 .003 ms .015 ms .09 ms 2.7 ms 100 s 50 .005 ms .03 ms .25 ms 12.5 ms 3y 100 .01 ms .07 ms 1 ms 100 ms 3x 10 c
13

1000 10000 .1 ms 1 ms 1 ms 100 ms 100 s inf 13 ms 10 s 28 h inf

C O M P L E X I T Y

nlogn .003 ms 2 .01 n ms 3 .1 n ms n .1 2 ms

10-04-03

P.P.Chakrabarti, IIT Kharagpur

Maximum size solvable within 1 hour

complexity

speed

S
N1 = 3.6x10 N2 =
10 9

100 S 1000 S
100 N1 85 N2 10 N3 N4+7 1000 N1 750 N2 30 N3 N4+10
P.P.Chakrabarti, IIT Kharagpur

n n log n n 2
8
2

1.2x10 N3 = 5 2x10 N4 = 35
10-04-03

Formal Definition
?

T(N) = O(f(N)) if there are positive constants c and n 0 such that T(N) ? c f(N) when N ? n0. Meaning : As N increases, T(N) grows no faster than f(N). The function T is eventually bounded by some multiple of f(N). f(N) gives an upper bound in the behavior of T(N).

logen = O(n) n2 = O(n2). n2 = O(n3). nlogn = O(n2). 3 n2 + 5n + 1 = O(n2)

9
10-04-03 P.P.Chakrabarti, IIT Kharagpur

Some Worst Case Complexities:

? ? ? ? ? ? ?

? ?

Linear Search in an array: ? Binary Search: ? Selection Sort: ? Mergesort: ? Quicksort: ? Stack ADT: Push and Pop are ? Queue using a singly linked list: ?? (What about a doubly linked list??) Multiplying 2 square matrices:? The primes that we wrote in class:?
10-04-03 P.P.Chakrabarti, IIT Kharagpur

10

Some Worst Case Complexities:

Linear Search in an array: O(n) ? Binary Search: O(log n) ? Selection Sort: O(n2) ? Mergesort:
?

T(n) = 2T(n/2) + cn ? Solves to O(n log n) ? T(n) = 2(2T(n/4) + cn/2) + cn = 4T(n/4) + 2cn = 2kT(n/2k) + kcn [Note that T(2) = 1] = O(nk) = O(n log n)
?

11

10-04-03

P.P.Chakrabarti, IIT Kharagpur

Some Worst Case Complexities:

Quicksort: O(n2) [This occurs when every partition breaks the array of n elements into 1 and n-1] ? Stack ADT: Push and Pop are O(1) ? Queue using a singly linked list:
?
? Insert

(Enqueue): O(1) ? Delete (Dequeue): O(n)

?

Queue using a doubly linked list: All O(1)

12

10-04-03

P.P.Chakrabarti, IIT Kharagpur

Primes:
?

? ?

For an integer of value k we loop at most sqrt(k) times. So is the complexity O(sqrt(n))? Remember that the input for value n is in log2 k bits. So the input size is n = log2 n The complexity in terms of input size for this algorithm is therefore O(2n) The latest proof of Dr Manindra Agarwal et al proves that prime detection can be done in O(nk) time for some constant k<5
10-04-03 P.P.Chakrabarti, IIT Kharagpur

13

More definitions
?

T(N) = ? (g(N)) if there are positive constants c and n0 such that T(N) ? c f(N) when N ? n0. Meaning : As N increases, T(N) grows no slower than g(N) ; T(N) grows at least as fast as g(N). T(N) = ?(h(N)) if and only if T(N) = O (h(N)) and T(N) = ? (h(N)) Meaning : As N increases, T(N) grows as fast as h(N). T(N) = o(p(N)) if T(N) = O(p(N)) and T(N) ? ?(p(N)) Meaning : As N increases, T(N) grows slower than p(N). lim n? ? T(N)/p(N) = 0.

logen = O(n) n10 = o(2 n) 3 n2 + 5n + 1 = ?(n2)

14
10-04-03 P.P.Chakrabarti, IIT Kharagpur

Space Complexity
?

Measures the work space or space required in addition to input storage ? Selection Sort: O(1) ? Binary Search: ? Recursive algorithm : O(log n) {Recursion Stack} ? Iterative algorithm: O(1) ? Mergesort: O(n) {Why: Additional array. Here recursion stack is of depth log n} ? Quicksort: O(n) {Why: recursion stack in the worst case}

15

10-04-03

P.P.Chakrabarti, IIT Kharagpur