Sunu Wibirama
Referensi
!
Cormen, T.H., Leiserson, C.E., Rivest,
R.L., Stein, C., Introduction to Algorithms
2nd Edition, Massachusetts: MIT Press,
2002
!
Sedgewick, R., Algorithms in C++ Part
1-4, Massachusetts: Addison-Wesley
Publisher, 1998
!
Video lecture MIT Opencourseware
ke-1: Introduction
!
Video lecture IIT Kharagpur, India
ke-18: Complexity of Algorithm
Video lecture MIT
Opencourseware ke-1:
Introduction
Video lecture IIT
Kharagpur, India ke-18:
Complexity of Algorithm
Agenda Hari Ini
Contoh Implementasi
Kesimpulan
Fungsionalitas (functionality)
Keamanan (security)
Kesederhanaan (simplicity)
1
2
n
2
3n c
2
n
2
for all n n
0
. Dividing by n
2
yields
c
1
1
2
3
n
c
2
.
The right-hand inequality can be made to hold for any value of n 1 by choosing
c
2
1/2. Likewise, the left-hand inequality can be made to hold for any value
of n 7 by choosing c
1
1/14. Thus, by choosing c
1
= 1/14, c
2
= 1/2, and
n
0
= 7, we can verify that
1
2
n
2
3n = (n
2
). Certainly, other choices for the
constants exist, but the important thing is that some choice exists. Note that these
constants depend on the function
1
2
n
2
3n; a different function belonging to (n
2
)
would usually require different constants.
We can also use the formal denition to verify that 6n
3
= (n
2
). Suppose for
the purpose of contradiction that c
2
and n
0
exist such that 6n
3
c
2
n
2
for all n n
0
.
But then n c
2
/6, which cannot possibly hold for arbitrarily large n, since c
2
is
constant.
Intuitively, the lower-order terms of an asymptotically positive function can be
ignored in determining asymptotically tight bounds because they are insignicant
for large n. A tiny fraction of the highest-order term is enough to dominate the
Macam-macam Parameter N
1
Sebagian besar instruksi dieksekusi satu kali atau dalam
jumlah yang tidak terlalu banyak (waktu eksekusi
konstan)
Pertumbuhan waktu eksekusi program tidak terlalu cepat.
Waktu eksekusi ini terdapat pada program yang
memecahkan masalah dengan kapasitas yang cukup
besar, dipecah-pecah menjadi beberapa bagian
N
Waktu eksekusi program linier. Sebagian besar masukan
diproses dalam jumlah yang tidak terlalu banyak
Waktu eksekusi ini terdapat pada program yang
memecahkan masalah menjadi beberapa bagian,
menyelesaikannya secara terpisah, kemudian
menggabungkannya kembali
log N
N log N
Macam-macam Parameter N
(contd)
Biasanya digunakan untuk memecahkan masalah dalam
jumlah kecil. Biasanya terdapat pada program yang
memproses pasangan data (quadratic) atau array dua
dimensi secara bersamaan (double-nested loop)
Biasanya digunakan untuk memecahkan masalah dalam
jumlah kecil. Biasanya terdapat pada program yang
memproses tiga buah data (cubic) atau array tiga dimensi
secara bersamaan (triple-nested loop)
Waktu eksekusi program linier. Sebagian besar masukan
diproses dalam jumlah yang tidak terlalu banyak
N
2
N
3
2
N
Beberapa perbandingan
kompleksitas algoritma
Figure 1-1 shows how the various measures of complexity compare with one another. The horizontal
axis represents the size of the problemfor example, the number of records to process in a search algo-
rithm. The vertical axis represents the computational effort required by algorithms of each class. This is
not indicative of the running time or the CPU cycles consumed; it merely gives an indication of how the
computational resources will increase as the size of the problem to be solved increases.
Figure 1-1: Comparison of different orders of complexity.
Referring back at the list, you may have noticed that none of the orders contain constants. That is, if an
algorithms expected runtime performance is proportional to N, 2N, 3N, or even 100N, in all cases
the complexity is defined as being O(N). This may seem a little strange at first surely 2N is better than
100Nbut as mentioned earlier, the aim is not to determine the exact number of operations but rather
to provide a means of comparing different algorithms for relative efficiency. In other words, an algo-
rithm that runs in O(N) time will generally outperform another algorithm that runs in O(N
2
). Moreover,
when dealing with large values of N, constants make less of a difference: As a ration of the overall size,
the difference between 1,000,000,000 and 20,000,000,000 is almost insignificant even though one is actu-
ally 20 times bigger.
Of course, at some point you will want to compare the actual performance of different algorithms, espe-
cially if one takes 20 minutes to run and the other 3 hours, even if both are O(N). The thing to remember,
however, is that its usually much easier to halve the time of an algorithm that is O(N) than it is to
change an algorithm thats inherently O(N
2
) to one that is O(N).
O(N!)
O(N
2
)
O(N log N)
O(log N)
O(N)
O(1)
5
Getting Started
04_596748 ch01_2.qxd 11/12/07 4:06 PM Page 5
Running Time
(seconds)
Input size (N)
Formula Kondisi Periodik
n
j =2
t
j
6 do A[i +1] A[i ] c
6
n
j =2
(t
j
1)
7 i i 1 c
7
n
j =2
(t
j
1)
8 A[i +1] key c
8
n 1
The running time of the algorithm is the sum of running times for each statement
executed; a statement that takes c
i
steps to execute and is executed n times will
contribute c
i
n to the total running time.
5
To compute T(n), the running time of
INSERTION-SORT, we sum the products of the cost and times columns, obtaining
T(n) = c
1
n +c
2
(n 1) +c
4
(n 1) +c
5
n
j =2
t
j
+c
6
n
j =2
(t
j
1)
+c
7
n
j =2
(t
j
1) +c
8
(n 1) .
Even for inputs of a given size, an algorithms running time may depend on
which input of that size is given. For example, in INSERTION-SORT, the best
5
This characteristic does not necessarily hold for a resource such as memory. A statement that
references m words of memory and is executed n times does not necessarily consume mn words of
memory in total.
24 Chapter 2 Getting Started
In the following discussion, our expression for the running time of INSERTION-
SORT will evolve from a messy formula that uses all the statement costs c
i
to a
much simpler notation that is more concise and more easily manipulated. This
simpler notation will also make it easy to determine whether one algorithm is more
efcient than another.
We start by presenting the INSERTION-SORT procedure with the time cost
of each statement and the number of times each statement is executed. For each
j = 2, 3, . . . , n, where n = length[A], we let t
j
be the number of times the while
loop test in line 5 is executed for that value of j . When a for or while loop exits in
the usual way (i.e., due to the test in the loop header), the test is executed one time
more than the loop body. We assume that comments are not executable statements,
and so they take no time.
INSERTION-SORT(A) cost times
1 for j 2 to length[A] c
1
n
2 do key A[ j ] c
2
n 1
3 Insert A[ j ] into the sorted
sequence A[1 . . j 1]. 0 n 1
4 i j 1 c
4
n 1
5 while i > 0 and A[i ] > key c
5
n
j =2
t
j
6 do A[i +1] A[i ] c
6
n
j =2
(t
j
1)
7 i i 1 c
7
n
j =2
(t
j
1)
8 A[i +1] key c
8
n 1
The running time of the algorithm is the sum of running times for each statement
executed; a statement that takes c
i
steps to execute and is executed n times will
contribute c
i
n to the total running time.
5
To compute T(n), the running time of
INSERTION-SORT, we sum the products of the cost and times columns, obtaining
T(n) = c
1
n +c
2
(n 1) +c
4
(n 1) +c
5
n
j =2
t
j
+c
6
n
j =2
(t
j
1)
+c
7
n
j =2
(t
j
1) +c
8
(n 1) .
Even for inputs of a given size, an algorithms running time may depend on
which input of that size is given. For example, in INSERTION-SORT, the best
5
This characteristic does not necessarily hold for a resource such as memory. A statement that
references m words of memory and is executed n times does not necessarily consume mn words of
memory in total.
2.2 Analyzing algorithms 25
case occurs if the array is already sorted. For each j = 2, 3, . . . , n, we then nd
that A[i ] key in line 5 when i has its initial value of j 1. Thus t
j
= 1 for
j = 2, 3, . . . , n, and the best-case running time is
T(n) = c
1
n +c
2
(n 1) +c
4
(n 1) +c
5
(n 1) +c
8
(n 1)
= (c
1
+c
2
+c
4
+c
5
+c
8
)n (c
2
+c
4
+c
5
+c
8
) .
This running time can be expressed as an +b for constants a and b that depend on
the statement costs c
i
; it is thus a linear function of n.
If the array is in reverse sorted orderthat is, in decreasing orderthe worst
case results. We must compare each element A[ j ] with each element in the entire
sorted subarray A[1 . . j 1], and so t
j
= j for j = 2, 3, . . . , n. Noting that
n
j =2
j =
n(n +1)
2
1
and
n
j =2
( j 1) =
n(n 1)
2
(see Appendix A for a review of how to solve these summations), we nd that in
the worst case, the running time of INSERTION-SORT is
T(n) = c
1
n +c
2
(n 1) +c
4
(n 1) +c
5
n(n +1)
2
1
+c
6
n(n 1)
2
+c
7
n(n 1)
2
+c
8
(n 1)
=
c
5
2
+
c
6
2
+
c
7
2
n
2
+
c
1
+c
2
+c
4
+
c
5
2
c
6
2
c
7
2
+c
8
n
(c
2
+c
4
+c
5
+c
8
) .
This worst-case running time can be expressed as an
2
+bn +c for constants a, b,
and c that again depend on the statement costs c
i
; it is thus a quadratic function
of n.
Typically, as in insertion sort, the running time of an algorithm is xed for a
given input, although in later chapters we shall see some interesting randomized
algorithms whose behavior can vary even for a xed input.
Worst-case and average-case analysis
In our analysis of insertion sort, we looked at both the best case, in which the input
array was already sorted, and the worst case, in which the input array was reverse
sorted. For the remainder of this book, though, we shall usually concentrate on
2.2 Analyzing algorithms 25
case occurs if the array is already sorted. For each j = 2, 3, . . . , n, we then nd
that A[i ] key in line 5 when i has its initial value of j 1. Thus t
j
= 1 for
j = 2, 3, . . . , n, and the best-case running time is
T(n) = c
1
n +c
2
(n 1) +c
4
(n 1) +c
5
(n 1) +c
8
(n 1)
= (c
1
+c
2
+c
4
+c
5
+c
8
)n (c
2
+c
4
+c
5
+c
8
) .
This running time can be expressed as an +b for constants a and b that depend on
the statement costs c
i
; it is thus a linear function of n.
If the array is in reverse sorted orderthat is, in decreasing orderthe worst
case results. We must compare each element A[ j ] with each element in the entire
sorted subarray A[1 . . j 1], and so t
j
= j for j = 2, 3, . . . , n. Noting that
n
j =2
j =
n(n +1)
2
1
and
n
j =2
( j 1) =
n(n 1)
2
(see Appendix A for a review of how to solve these summations), we nd that in
the worst case, the running time of INSERTION-SORT is
T(n) = c
1
n +c
2
(n 1) +c
4
(n 1) +c
5
n(n +1)
2
1
+c
6
n(n 1)
2
+c
7
n(n 1)
2
+c
8
(n 1)
=
c
5
2
+
c
6
2
+
c
7
2
n
2
+
c
1
+c
2
+c
4
+
c
5
2
c
6
2
c
7
2
+c
8
n
(c
2
+c
4
+c
5
+c
8
) .
This worst-case running time can be expressed as an
2
+bn +c for constants a, b,
and c that again depend on the statement costs c
i
; it is thus a quadratic function
of n.
Typically, as in insertion sort, the running time of an algorithm is xed for a
given input, although in later chapters we shall see some interesting randomized
algorithms whose behavior can vary even for a xed input.
Worst-case and average-case analysis
In our analysis of insertion sort, we looked at both the best case, in which the input
array was already sorted, and the worst case, in which the input array was reverse
sorted. For the remainder of this book, though, we shall usually concentrate on
O(n
2
)
Cormen, T.H., Leiserson, C.E., Rivest, R.L., Stein, C., Introduction to Algorithms 2nd Edition, Massachusetts: MIT Press, 2002
Implementasi: Insertion Sort
Worst-case (seluruh input diurutkan terbalik):
Apakah Insertion Sort algoritma yang cepat?
- Ya, jika N kecil
- Tidak, jika N besar
Lihat perbandingannya dengan Merge Sort di
slide berikutnya
T(N) = O( j) = O(n
2
)
j =2
N
!
Deret aritmatika
Contoh komparasi
Komputer A:
Insertion Sort, lebih cepat untuk N kecil,
memproses N data dengan waktu eksekusi
Kecepatan eksekusi prosesor 10
9
instruksi/detik
Komputer B:
Merge Sort, lebih lambat untuk N kecil,
memproses N data dengan waktu eksekusi
Kecepatan eksekusi prosesor 10
7
instruksi/detik
Diketahui:
c
1
N
2
c
2
N log N
c
1
!c
2
Jika c
1
=2, c
2
=50, N = 10
6
,
mana algoritma yang lebih efisien?
Kesimpulan