Agenda: Worst/Best/Avg case analysis InsertionSort example Loop invariant Asymptotic Notations Textbook pages: 15-27, 41-57
times n n1 n1
n
tj (tj 1)
j =2 n
c5 c6
j =2
(tj 1) n1
c7
tj
=n1
n(n+1) 2
Key observation: equiprobable inputs imply for each key, rank among keys so far is equiprobable e.g., when j = 4, expected number of KC is
1+2+3+4 4
j
= 2 .5
i=1
j +1 2
j+1 1 = 2 2
n+1
=
j =3
1 2
(n + 1)(n + 2) 3 2
n2 + 3n 4 4
j =2
We will ignore the constant factors; we also only care about the dominant term, here it is n2 .
Maintenance: none from A[1..j ] moves beyond j ; sorted Termination #1: A[1..n] is sorted upon completion, j = n + 1 and by LI
Termination #2: for loop counter j increases by 1 at a time, and no change inside the loop 6
Denitions:
O(f (n)) is the set of functions h(n) that roughly, grow no faster than f (n), namely c0 , n0 , such that h(n) c0 f (n) for all n n0 (f (n)) is the set of functions h(n) that roughly, grow at least as fast as f (n), namely c0 , n0 , such that h(n) c0 f (n) for all n n0 (f (n)) is the set of functions h(n) that roughly, grow at the same rate as f (n), namely c0 , c1 , n0 , such that c0 f (n) h(n) c1 f (n) for all n n0 (f (n)) = O(f (n)) (f (n)) o(f (n)) is the set of functions h(n) that roughly, grow slower than f (n), namely
h(n) limn f =0 (n)
(f (n)) is the set of functions h(n) that roughly, grow faster than f (n), namely
h(n) limn f = (n)
Warning:
the textbook overloads = Textbook uses g (n) = O(f (n)) Incorrect !!! Because O(f (n)) is a set of functions. Correct: g (n) O(f (n)) You should use the correct notations.
Examples: which of the following belongs to O(n3 ), (n3 ), (n3 ), o(n3 ), (n3 ) ?
1. f1 (n) = 19n 2. f2 (n) = 77n2 3. f3 (n) = 6n3 + n2 log n 4. f4 (n) = 11n4
10
Answers:
1. 2. 3. 4. f1 (n) = 19n f2 (n) = 77n2 f3 (n) = 6n3 + n2 log n f4 (n) = 11n4
f1 , f2 , f3 O(n3 ) f1 (n) 19n3 , for all n 0 c0 = 19, n0 = 0 f2 (n) 77n3 , for all n 0 c0 = 77, n0 = 0 f3 (n) 6n3 + n2 n, for all n 1, since log n n if f4 (n) c0 n3 , then n f3 , f4 (n3 ) f3 (n) 6n3 , for all n 1, since n2 log n 0 f4 (n) 11n3 , for all n 0 f3 (n3 ) why? f1 , f2 o(n3 )
n f1 (n): limn 19 = limn 19 =0 n3 n2 n f2 (n): limn 77 = limn 77 =0 n3 n
2
c0 11
no such n0 exists
f3 (n): limn 6n
+n2 log n n3
4
= limn 6 +
log n n
=6
f4 (n3 ) 11
logarithm review:
Denition of logb n (b, n > 0): blogb n = n logb n as a function in n: increasing, one-to-one logb 1 = 0 logb xp = p logb x logb (xy ) = logb x + logb y xlogb y = y logb x logb x = (logb c)(logc x)
= (log n)
ln x =
1 x
. . . , then h (f ), (f ) . . . 0 < k < , then h (f ) . . . 0, then h O(f ), o(f ) LH opitals rules: if limn h(n) = , limn f (n) = , and h (n), f (n) exist, then h (n) h(n) = lim n f (n) n f (n) lim
n 1 e.g., limn ln = limn n =0 n
Still, we have h(n) O(n2 ), h(n) (1), etc. O(), (), (), o(), () JUST useful asymptotic notations
13
Example: The following functions are ordered in increasing order of growth (each is in big-Oh of next one). Those in the same group are in big-Theta of each other.
{n1/ log n ,
1} ,
ln ln n},
log n,
ln n, {n2 , en ,
log n
( 2)log n , {(log n)
{n log n, }, 3 2
log(n!)},
n
(log n)!,
log n
log log n
n.2n ,
(n!)2 ,
14