Problems on Greedy

algorithms

By:
Tulika Garg
3rd CSE
400/05
The Greedy Method

 A Greedy algorithm always makes the

choice that looks best at the moment.
 This means it makes a locally optimal
choice that will lead to a globally optimal
solution.
The Greedy Method

 Only a few optimization problems can be

solved by the greedy method.
 Greedy algorithms do not always yield
optimal solutions, but for many problems
they need optimal solutions.
 The 2-way merging problem
 The problem: There are m sorted list,
each of length ni. What is the optimal
sequence of merging process to
merge these m lists into one sorted
list ?
 Ex: L1=50, L2=30, L3=10
Linear Merge Algorithm
 Input:2 sorted lists,L1=(a1,a2,…..am) and
L2=(b1,b2,…..bn)
 Output:a sorted list consisting of elements in L1 and
L2
 begin
i=1
 j=1
 Do
 Compare ai and bj
 If ai>bj then output bj and j=j+1
 Else output ai and i=i+1
 While (i<=m and j<=n)
 If i>m then output bj,bj+1,….bn
 Else output ai,ai+1,,,,,am
The 2-way merging problem
 Linear Merge Algorithm
two sorted lists, L1=(a1, a2, …, an) and
L2=(b1, b2, …, bm)
 The number of comparisons required
for the linear 2-way merge algorithm is
m+n-1 where m and n are the lengths
of the two sorted lists respectively.
2-way merging problem……..
 These merging processes are called 2-way
merge because each merging step only
merges two sorted lists.
 Suppose we are given 3 sorted lists l1,l2,l3
consisting of 50,30,10 elements
respectively.so we can merge l1 and l2 to
obtain l4.This merging step requires 50+30-
1=79 comparisons in the worst case,then
we merge l4 and l1 by using 80+10-1=89
comparisons.the no of comparisons become
168
2-way merging problem…..
 Alternatively we can first merge l2 and l3
and then l1 . This will require128
comparisons. so there will be many different
merging sequences that will require different
comparisons.so the question arises:
 What is the optimal sequence of merging
process to merge these sorted lists by
using the minimum no of comparisons?
 The 2-way merging problem
Different merging sequences require different numbers of
comparisons
Finding an Extended Binary Tree with minimum total
weighted external path length
We shall consider m+n instead of m+n-1for simplicity as it
won’t affect the design
di: the depth of an external node of the EBT
ni: the size of Li associated with this external node
k

The total number of comparisions = ∑ d n k: number of list

i =1
i i
 The 2-way merging problem

Extended Binary
Tree(EBT) Representing
a 2-way Merge
External node
Internal node
A Greedy Algorithm to generate
an optimal 2-way merge tree
 Input: m sorted lists Li,i=1,2,…m each Li consisting
of ni elements
 Output:An optimum two way merge tree
 Step 1. generate m trees, where each tree has
exactly 1 node(external node)with weight ni
 Step 2. choose two trees T1 and T2 with minimal
weights
 Step 3. create a new tree T whose root has T1 and
T2 as its subtrees and weight =the sum of weights
of T1 and T2.
 Step 4. replace T1 and T2 by T
 Step 5. If there is only 1 Tree left stop and
RETURN;otherwise goto step 2
Merging tree for the example

Here d2=d3=2,d=1.thus by using

L5
90 the formula total no of

comparisons=

40 L4 L1
50 30.2+10.2+50.1=130

L2 L3

30 10
The 2-way merging problem
 E.g. There are 6 sorted lists with
lengths 2, 3, 5, 7, 11 and 13.
The 2-way merging problem

 Time complexity of the greedy

algorithm to generate an
optimal extended binary
tree:O(n log n)
Huffman codes
 In telecommunication, how do we represent
a set of messages, each with an access
frequency, by a sequence of 0’s and 1’s?
 To minimize the transmission and decoding
costs, we may use short strings to represent
more frequently used messages.
 This problem can by solved by using an
extended binary tree which is used in the 2-
way merging problem.
 Huffman codes
 E.g.
symbols: A, B, C, D, E, F, G
freq. : 2, 3, 5, 8, 13, 15, 18

 Huffman codes:
A: 10100 B: 10101 C: 1011
D: 100 E: 00 F: 01
G: 11

A Huffman code Tree

 Algorithm for Huffman codes
1 n ← |C|
2 Q←C ; Characters are in a priority queue
3 for i ←1 to n-1
4 do z ← ALLOCATE-NODE()
5 x ← left[z] ← EXTRACT-MIN(Q)
6 y ← right[z] ← EXTRACT-MIN(Q)
7 f[z] ← f[x] + f[y]
8 INSERT(Q,z)
9 return EXTRACT-MIN(Q)
The minimal cycle basis
problem
 Given a graph, find a minimal cycle basis of
this graph.
 Def : A cycle basis of a graph is a set of
cycles such that every cycle in the graph
can be generated by applying ⊕ on some
cycles of this basis.
 Weight of a cycle: smallest total weight of all
edges in this cycle
 Minimal cycle basis : smallest total weight of
all edges in this cycle basis.
The minimal cycle basis
problem
 3 cycles:
A1 = {ab, bc, ca}
A2 = {ac, cd, da}
A3 = {ab, bc, cd, da}
where A3 = A1 ⊕ A2
(A ⊕ B = (A∪B)-(A∩B))
A2 = A1 ⊕ A3
A1 = A2 ⊕ A3
Cycle basis : {A1, A2} or {A1, A3} or {A2, A3}
The minimal cycle basis
problem
 Algorithm for finding a minimal cycle basis:
 Step 1: Determine the size of the minimal cycle
basis, demoted as k.
 Step 2: Find all of the cycles. Sort all cycles( by
weight).
 Step 3: Add cycles to the cycle basis one by
one. Check if the added cycle is already
combination of some cycles already existing in
the basis. If it is, delete this cycle.
 Step 4: Stop if the cycle basis has k cycles.
Minimum cycle basis problem
solved by greedy algorithm
 We can determine the minimum size K
of this set and then use the greedy
method to add objects one by one to
this set until the size of this set
becomes K.
 The minimal cycle basis
problem – detail description
 A cycle basis corresponds to the fundamental set of
cycles with respect to a spanning tree.

a graph a spanning a fundamental

tree set of cycles
Minimum cycle basis…..
 Thus each addition of an edge creates a
new independent cycle.The no of
independent cycles is equal to the no of
edges which can be added to a spanning
tree
 No of edges in a spanning tree=V-1,the total
no of edges that can be added is=E-(V-
1)=E-V+1=K=size of the minimum cycle
basis
 The minimal cycle basis
problem – detail description

EXAMPLE

Here K=E-V+1=5-4+1=2

The action can be done by representing the cycles with an incidence

matrix.Each row corresponds to a cycle and each column corresponds to an

edge.There are three cycles:

 The minimal cycle basis
problem – detail description
 E.g.  Add C3
e1 e2 e3 e4 e5
C1 1 1 1
C2 1 1 1
C3 1 1 1 1

 ⊕ on rows 1 and
3 e1 e2 e3 e4 e5
C1 1 1 1
C2 1 1 1
 2 cycles C1 and C2 are C3 1 1 1
represented by a 0/1 matrix
e1 e2 e3 e4 e5 ⊕ on rows 2 and 3 : empty
C1 1 1 1 row
C2 1 1 1
∵C = C ⊕ C
Minimum Cycle Basis
problem…
 The exclusive operation on C2 and C3
produces an empty row which shows
that C3 is a linear combination of C1
and C2.
 Thus C3 is discarded .since now the
cycle basis already has two cycles,we
stop as K=2.A minimum cycle basis is
found to be {C1,C2}.
THANK YOU!!