12 Analisis Amortized

Mata Kuliah: Perancangan dan Analisis Algoritma

Dosen: Anny Yuniarti, S.Kom, M.Comp.Sc
Introduction
5/28/2012 Jurusan Teknik Informatika FTIF ITS 2
Amortized analysis is a tool for analyzing algorithms that perform a
sequence of similar operations.

Instead of bounding the cost of the sequence of operations by
bounding the actual cost of each operation separately, an amortized
analysis can be used to provide a bound on the actual cost of the
entire sequence.

One reason this idea can be effective is that it may be impossible in
a sequence of operations for all of the individual operations to run in
their known worst-case time bounds. While some operations are
expensive, many others might be cheap.

Amortized analysis is not just an analysis tool, however; it is also a
way of thinking about the design of algorithms, since the design
of an algorithm and the analysis of its running time are often closely
intertwined.
Amortized analysis
5/28/2012 Jurusan Teknik Informatika FTIF ITS 3
Analyze a sequence of operations on a data
structure.
Goal: Show that although some individual
operations may be expensive, on average the cost
per operation is small.

Average in this context does not mean that were
averaging over a distribution of inputs.

No probability is involved.
Were talking about average cost in the worst case.
Organization
5/28/2012 Jurusan Teknik Informatika FTIF ITS 4
Well look at 3 methods:
aggregate analysis
accounting method
potential method

The aggregate method, though simple, lacks the
precision of the other two methods. In particular, the
accounting and potential methods allow a specific
amortized cost to be allocated to each operation.
Aggregate analysis
5/28/2012 Jurusan Teknik Informatika FTIF ITS 5
Show that for all n, a sequence of n operations take
worst-case time T(n) in total

In the worst case, the average cost, or amortized
cost , per operation is T(n)/n.

The amortized cost applies to each operation, even
when there are several types of operations in the
sequence.
Stack Example
3 ops:
Push(S,x) Pop(S)
Multi-
pop(S,k)
Worst-case
cost:
O(1) O(1)
O(min(|S|,k)
= O(n)
Amortized cost: O(1) per op
PUSH(S, x)
1 top[S] top[S] + 1
2 S[top[S]] x
Stack Example
3 ops:
Push(S,x) Pop(S)
Multi-
pop(S,k)
Worst-case
cost:
O(1) O(1)
O(min(|S|,k)
= O(n)
Amortized cost: O(1) per op
STACK-EMPTY(S)
1 if top[S] = 0
2 then return TRUE
3 else return FALSE

POP(S)
1 if STACK-EMPTY(S)
2 then error "underflow"
3 else top[S] top[S] - 1
4 return S[top[S] + 1]
Stack Example
3 ops:
Push(S,x) Pop(S)
Multi-
pop(S,k)
Worst-case
cost:
O(1) O(1)
O(min(|S|,k)
= O(n)
Amortized cost: O(1) per op
MULTIPOP(S, k)
1 while not STACK-EMPTY(S) and k 0
2 do POP(S)
3 k k - 1
Stack Example : Aggregate Analysis
5/28/2012 Jurusan Teknik Informatika FTIF ITS 9
Sequence of n push, pop, Multipop operations
Worst-case cost of Multipop is O(n)
Have n operations
Therefore, worst-case cost of sequence is O(n
2
)
Observations
Each object can be popped only once per time that its
pushed
Have <= n pushes <= n pops, including those in
Multipop
Therefore total cost = O(n)
Average over n operations O(1) per operation on
average
Notice that no probability involved

Incrementing a Binary Counter
k-bit Binary Counter: A[0..k1]
INCREMENT(A)
1. i 0
2. while i < length[A] and A[i] = 1
3. do A[i] 0 reset a bit
4. i i + 1
5. if i < length[A]
6. then A[i] 1 set a bit

=

=
1
0
2 ] [
k
i
i
i A x
k-bit Binary Counter
Ctr A[4] A[3] A[2] A[1] A[0]
0 0 0 0 0 0
1 0 0 0 0 1
2 0 0 0 1 0
3 0 0 0 1 1
4 0 0 1 0 0
5 0 0 1 0 1
6 0 0 1 1 0
7 0 0 1 1 1
8 0 1 0 0 0
9 0 1 0 0 1
10 0 1 0 1 0
11 0 1 0 1 1
INCREMENT(A)
1. i 0
2. while i < length[A]
and A[i] = 1
3. do A[i] 0
4. i i + 1
5. if i < length[A]
6. then A[i] 1
k-bit Binary Counter
Ctr A[4] A[3] A[2] A[1] A[0] Cost
0 0 0 0 0 0 0
1 0 0 0 0 1 1
2 0 0 0 1 0 3
3 0 0 0 1 1 4
4 0 0 1 0 0 7
5 0 0 1 0 1 8
6 0 0 1 1 0 10
7 0 0 1 1 1 11
8 0 1 0 0 0 15
9 0 1 0 0 1 16
10 0 1 0 1 0 18
11 0 1 0 1 1 19
INCREMENT(A)
1. i 0
2. while i < length[A]
and A[i] = 1
3. do A[i] 0
4. i i + 1
5. if i < length[A]
6. then A[i] 1
The running cost for
flipping bits
INCREMENT(A)
1. i 0
2. while i < length[A] and A[i] = 1
3. do A[i] 0
4. i i + 1
5. if i < length[A]
6. then A[i] 1
Worst-case analysis
Consider a sequence of n insertions.
The worst-case time to execute one
insertion is O(k). Therefore, the worst-
case time for n insertions is n O(k) =
O(n k).
WRONG! In fact, the worst-case cost
for n insertions is only O(n) O(n k).
Lets see why.
Note: Youll be correct
If youd said O(n k).
But, its an overestimate.
Tighter analysis
Ctr A[4] A[3] A[2] A[1] A[0] Cost
0 0 0 0 0 0 0
1 0 0 0 0 1 1
2 0 0 0 1 0 3
3 0 0 0 1 1 4
4 0 0 1 0 0 7
5 0 0 1 0 1 8
6 0 0 1 1 0 10
7 0 0 1 1 1 11
8 0 1 0 0 0 15
9 0 1 0 0 1 16
10 0 1 0 1 0 18
11 0 1 0 1 1 19
A[0] flipped every op n
A[1] flipped every 2 ops
n/2
A[2] flipped every 4 ops
n/2
2

A[3] flipped every 8 ops
n/2
3


A[i] flipped every 2
i
ops
n/2
i

Total cost of n operations
i k never 0
Tighter analysis (continued)
) (
2
2 / 1 1
1

2
1


2

0
1
0
n O
n n n
n
i
i
k
i
i
=
=
|
.
|

\
|

= <
(

=
Cost of n increments
Thus, the average cost of each increment
operation is O(n)/n = O(1).
Accounting method
5/28/2012 Jurusan Teknik Informatika FTIF ITS 17
Assign different charges to different operations.
Some are charged more than actual cost.
Some are charged less.
Amortized cost = amount we charge.
When amortized cost > actual cost, store the
difference on specific objects in the data structure as
credit.
Use credit later to pay for operations whose actual
cost > amortized cost.
Differs from aggregate analysis:
In the accounting method, different operations can have
different costs.
In aggregate analysis, all operations have same cost.
Accounting method
5/28/2012 Jurusan Teknik Informatika FTIF ITS 18
Need credit to never go negative.
Otherwise, have a sequence of operations for which the
amortized cost is not an upper bound on actual cost.
Amortized cost would tell us nothing.
Let c
i
= actual cost of ith operation ,
= amortized cost of ith operation .

.
i
c

= =
s
n
i
i
n
i
i
c c
1 1

A Simple Example: Accounting

3 ops:
Push(S,x) Pop(S) Multi-pop(S,k)
Assigned
cost:
2 0 0
Actual cost: 1 1 min(|S|,k)
Push(S,x) pays for possible later pop of x.
1
1
1
1
1
1
1
1
1
0
1
1
0
0
0
Stack Example: Accounting Methods
When pushing an object, pay $2
$1 pays for the push
$1 is prepayment for it being popped by either
pop or Multipop
Since each object has $1, which is credit, the
credit can never go negative
Therefore, total amortized cost = O(n), is an
upper bound on total actual cost
Example:
Accounting analysis of INCREMENT
Charge an amortized cost of $2 every time a bit
is set from 0 to 1
$1 pays for the actual bit setting.
$1 is stored for later re-setting (from 1 to 0).
At any point, every 1 bit in the counter has $1 on
it that pays for resetting it. (reset is free)
0 0 0 1
$1
0 1
$1
0
0 0 0 1
$1
0 1
$1
1
$1

0 0 0 1
$1
1
$1
0 0
Cost = $2
Cost = $2
Incrementing a Binary Counter
When Incrementing,
Amortized cost for line 3 = $0
Amortized cost for line 6 = $2
Amortized cost for INCREMENT(A) = $2
Amortized cost for n INCREMENT(A) = $2n =O(n)
INCREMENT(A)
1. i 0
2. while i < length[A] and A[i] = 1
3. do A[i] 0 reset a bit
4. i i + 1
5. if i < length[A]
6. then A[i] 1 set a bit
The potential method
5/28/2012 Jurusan Teknik Informatika FTIF ITS 23
The potential method, or physicist's view, defines a function that
maps a data structure onto a real valued, nonnegative potential.
The potential stored in the data structure may be released to pay for
future operations.

The potential is associated with the data structure as a whole rather
than with specific objects within the data structure.

Let us refer to the data structure at time i (or alternatively, just after
operation i), as D
i.


The potential function (D
i
) maps D
i
onto a real value. Again, denote
the actual cost of operation i as c
i
.

In the potential method, the amortized cost of operation i is equal to
the actual cost plus the increase in potential due to the operation:
The potential method
5/28/2012 Jurusan Teknik Informatika FTIF ITS 24
Ensure that the total amortized cost of a sequence is
always an upper bound on the total actual cost
Example: Stack operations
5/28/2012 Jurusan Teknik Informatika FTIF ITS 25
A potential function on a stack is (D
i
) = the number
of items in the stack after operation i.
For the empty stack D
0
with which we start, we have
(D
0
) = 0.
If the ith operation on a stack containing s objects is
a PUSH operation, then the potential difference is
(D
i
) - (D
i-1
) = (s + 1) s = 1
The amortized cost of this PUSH operation is

Example: Stack operations (contd)
5/28/2012 Jurusan Teknik Informatika FTIF ITS 26
The amortized cost of each of the three operations is
O(1), and thus the total amortized cost of a
sequence of n operations is O(n).
Example: binary counter
5/28/2012 Jurusan Teknik Informatika FTIF ITS 27
The potential of the counter after the ith INCREMENT
operation to be b
i
, the number of 1s in the counter after the
ith operation.

Suppose that the ith INCREMENT operation resets t
i
bits. The
actual cost of the operation is therefore at most t
i
+ 1, since in
addition to resetting t
i
bits, it sets at most one bit to 1.

If b
i
= 0, then the ith operation resets all k bits, and so b
i-1
= t
i
=
k.

If b
i
> 0, then b
i
= b
i-1
- t
i
+ 1.

In either case, b
i
b
i-1
- t
i
+ 1, and the potential difference is
Example: binary counter (contd)
5/28/2012 Jurusan Teknik Informatika FTIF ITS 28
The amortized cost is therefore





If the counter starts at zero, then (D
0
) = 0. Since
(D
i
) 0 for all i, the total amortized cost of a
sequence of n INCREMENT operations is an upper
bound on the total actual cost, and so the worst-case
cost of n INCREMENT operations is O(n).
Comparing the accounting and potential
methods
5/28/2012 Jurusan Teknik Informatika FTIF ITS 29
The accounting method charges each operation a
certain amount according to its type (e.g., insert or
delete), putting the focus on each operation's
prepayment to offset future expensive operations.

The potential method instead puts the focus on the
effect of a particular operation at a particular point in
time, based on the effects of the operation on the
data structure and the corresponding change in
possibility for future operations to incur expense.
Comparing the accounting and potential
methods
5/28/2012 Jurusan Teknik Informatika FTIF ITS 30
The accounting method stores credit in particular
areas of a data structure, such as nodes of a tree or
items in a stack, whereas the potential method
generally considers properties of the entire data
structure (such as how many nodes are in a tree, or
how many items are in the stack) to compute the
potential.