Anda di halaman 1dari 7

EECS 4101 Advanced Data Structures EECS 4101 Advanced Data Structures

MINIMUM SPANNING TREE ALGORITHMS 1 MINIMUM SPANNING TREE CLRS 23 2

NP-COMPLETENESS

ASSIGNMENT 2 ANSWERS • Given: G=(V,E) - an undirected connected graph with vertices V,
edges E and an edge weight function (natural numbers≥0, don’t
have to be distinct) w(u,v) = weight of edge (u,v)

Lec 21: Minimum Spanning Trees, CLRS 23 • Spanning tree of a graph - subset of |V|-1 edges forming a tree, i.e no
NP CLRS 34 cycles, every vertex connected by path in tree
Lec 22, 23 : NP-completeness and • Weight of spanning tree w(T) = ∑ (u,v) in T w(u,v)
NP Approximation CLRS 34-35
• Minimum spanning tree (MST) is a spanning tree with minimum
weight

• Used for pipelines, traintracks, Moravia electrical system (Boruvka in


20’s), Prim (+ Jarnik, Djisktra) and Kruskal have famous algorithms
for MST

Kevin Wayne, Princeton University


Applications Prim's algorithm (node oriented)

MST is fundamental problem with diverse applications. Initialize S = any node, A = ∅

・Dithering. Repeat n – 1 times:

・Cluster analysis. ・Add to tree A the min weight edge e with one endpoint in S
・Max bottleneck paths. ・Add new endpoint of e to S
・Real-time face verification. 


・LDPC codes for error correction. 


・Image registration with Renyi entropy. Theorem. Prim's algorithm computes the MST.

・Find road networks in satellite and aerial imagery.


・Reducing data storage in sequencing amino acids in a protein.
・Model locality of particle interactions in turbulent fluid flows.
・Autoconfig protocol for Ethernet bridging to avoid cycles in a network. S

・Approximation algorithms for NP-hard problems (e.g., TSP, Steiner tree).


・Network design (communication, electrical, hydraulic, computer, road).

3
Kevin Wayne, Princeton University 4
EECS 4101 Advanced Data Structures EECS 4101 Advanced Data Structures
CONSOLIDATE 5 PRIM’S ALGORITHM IMPLEMENTATION 6

• PRIM(G,w,v) // G=(V,E) is graph, v nodes, e edges, w is weight fn


• Q= ∅ A=∅
• for each u in V
• u.key = ∞ // distance
• u.link = nil // link from u to “closest” neighbour in A
• insert(Q, u)
• decrease-key(Q, r, 0) //r.key = 0
• while Q ≠ ∅
• u= extract-min(Q) // and add edge (u, u.link) to A
• for each v ∈ V where (u,v) ∈ E // neighbours of u
• if v ∈Q and w(u,v) < v.key
• v.link =u
CLRSFig-23-5 - Prim’s MST
• decrease-key (Q, v, w(u,v))

EECS 4101 Advanced Data Structures EECS 4101 Advanced Data Structures
CORRECTNESS OF PRIM’S ALGORITHM
• Given connected undirected graph G=(V,E) and weight function w, let T⊆E be7 EXCHANGE PROOF FOR PRIM’S ALGORITHM 8

a MST of G • Let T be a MST of G=(V,E) and let A ⊂ T be any proper subtree of T, not
• Let A ⊂ T be a any proper non-empty subset of T spanning all vertices.
• Let VA be the vertices in A. VA partitions V into blocks VA and V - VA • Let VA be the vertices in A
• Theorem: Any minimum weight edge e = (u,v) with u ∈ VA and v ∈ V-VA forms
part of a minimum spanning tree, thus can be added safely to A • Let (u,v ) be a minimum weight edge in G connecting any vertex u in VA to a
• Proof: If MST uses different edges, it still must have a path from u to v with a vertex v in VA-A. Claim is that A ∪ {(u,v)} is in a minimum spanning tree T of G
crossing f between VA and V - VA. Exchanging e and f yields MST that costs
• Proof: If (u,v) ∈ T we are done, so assume (u,v) is not in T. Consider the simple
no more, because the weight of w(e) ≤ w(f)
path P from u to v in T (there must be one in a MST). Let T’ be T modified so
A={(c,f),(c,d)(b,c)} that the first edge in P that has an endpoint not in A (i.e. an edge from VA to V-
VA = {b,c,d,e} VA) is replaced by (u,v). This is still a spanning tree since it spans all the same
red edges nodes, and it’s minimum because (u,v) can’t cost any more than the removed
crossing from VA edge, since (u,v)’s weight is minimum for all edges connecting VA to V-VA.
𝑎 So A ∪ {(u,v)} is a subtree of some MST
to V - VA

safe edges for A • In fact, can use any cut set V that includes all vertices in A (“respects A” in
CLRS parlance) and this leads to Kruskal’s algorithm
EECS 4101 Advanced Data Structures EECS 4101 Advanced Data Structures
PRIM’S ALGORITHM ANALYSIS 9 KRUSKAL’S MST (EDGE-ORIENTED) 10

• PRIM(G,w,v) // G=(V,E) is graph, v nodes, e edges, w is weight fn • Kruskal’s MST algorithm


• Q= ∅; A = ∅
• for each u in V //O(v lg v)
• Given connected graph G = (V,E) with |V | = v and |E| = e, plus w:E→N
• u.key = ∞ // distance A=∅
• u.link = nil // link from u to “closest” neighbour in A for each vertex v ∈ V
• insert(Q, u)
• decrease-key(Q, r, 0) //r.key = 0 //O(lg v) makeset(v)
• while Q ≠ ∅ //O(v lg v) sort edges of E into nondecreasing order by weight w
• u= extract-min(Q) // and add edge (u, u.link) to A for each (u,v) in E, in sorted order
• for each v ∈ V where (u,v) ∈ E // neighbours of u
• if v ∈Q and w(u,v) < v.key
if find(u) ≠ find(v)
• v.link =u A = A ∪ {(u,v)}
• decrease-key (Q, v, w(u,v)) //O(e lg v) union(u,v)
• Using Fibonacci heap decrease-key is O(1) amortized so O(v lg v + e)
• But in practice it seems better to use binary heap unless really large
• Note this is almost linear if edges are presorted

EECS 4101 Advanced Data Structures EECS 4101 Advanced Data Structures
CONSOLIDATE 11 CONSOLIDATE 12

CLRS Fig-23-4-L

CLRS Fig23-4-R
Kruskal’s MST
EECS 4101 Advanced Data Structures Does a linear-time MST algorithm exist?

KRUSKAL’S MST (EDGE-ORIENTED) 13



deterministic compare-based MST algorithms
• Kruskal’s MST algorithm 


 year worst case discovered by
• Given connected graph G = (V,E) with |V | = v and |E| = e, plus w:E→N 
 1975 O(m log log n) Yao

A=∅ 
 1976 O(m log log n) Cheriton-Tarjan

for each vertex v ∈ V // O(v) 
 1984 O(m log*n) O(m + n log n) Fredman-Tarjan

makeset(v) 1986 O(m log (log* n)) Gabow-Galil-Spencer-Tarjan

sort edges of E into nondecreasing order by weight w // O(e lg e) 

1997 O(m α(n) log α(n)) Chazelle

2000 O(m α(n)) Chazelle


for each (u,v) in E, in sorted order // O(e α(e)) 

2002 optimal Pettie-Ramachandran
if find(u) ≠ find(v) 


 20xx O(m) ???
A = A ∪ {(u,v)}

union(u,v) 

Remark 1. O(m) randomized MST algorithm. [Karger-Klein-Tarjan 1995]
//O(e lg e + eα(e))
• Note this is almost linear if edges are presorted Remark 2. O(m) MST verification algorithm. [Dixon-Rauch-Tarjan 1992]

14

8. INTRACTABILITY I 8. INTRACTABILITY I

‣ poly-time reductions ‣ poly-time reductions


‣ packing and covering problems ‣ packing and covering problems
‣ constraint satisfaction problems ‣ constraint satisfaction problems
‣ sequencing problems ‣ sequencing problems
‣ partitioning problems ‣ partitioning problems
‣ graph coloring ‣ graph coloring
‣ numerical problems ‣ numerical problems
Lecture slides by Kevin Wayne

Copyright © 2005 Pearson-Addison Wesley

http://www.cs.princeton.edu/~wayne/kleinberg-tardos

Last updated on 2017-03-27 1:23 AM


Algorithm design patterns and antipatterns Classify problems according to computational requirements

Algorithm design patterns. Q. Which problems will we be able to solve in practice?


・Greedy. 

・Divide and conquer. A working definition. Those with polynomial-time algorithms.
・Dynamic programming. 

・Duality. 

・Reductions. 

・Local search. 

・Randomization. 

von Neumann Nash Gödel Cobham Edmonds Rabin

 
 (1953) (1955) (1956) (1964) (1965) (1966)

 

Algorithm design antipatterns. 

・NP-completeness. algorithm unlikely.
O(nk) 

・PSPACE-completeness. O(nk) certification algorithm unlikely. Theory. Definition is broad and robust.
・Undecidability. No algorithm possible. 

constants tend to be small, e.g., 3 n 2

Practice. Poly-time algorithms scale to huge problems.

17 18

Classify problems according to computational requirements Classify problems

Q. Which problems will we be able to solve in practice? Desiderata. Classify problems according to those that can be solved in polynomial
time and those that cannot.
A working definition. Those with polynomial-time algorithms. 
 input size = c + lg k
Provably requires exponential time.
・Given a constant-size program, does it halt in at most k steps?
yes probably no ・Given a board position in an n-by-n generalization of checkers,

can black guarantee a win?
shortest path longest path using forced capture rule

min cut max cut

2-satisfiability 3-satisfiability 

planar 4-colorability planar 3-colorability 


bipartite vertex cover vertex cover

matching 3d-matching

primality testing factoring Frustrating news. Huge number of fundamental problems have defied classification
linear programming integer linear programming for decades.

19 20
Polynomial-time reductions Polynomial-time reductions

Desiderata'. Suppose we could solve problem Y in polynomial-time.
 Desiderata'. Suppose we could solve problem Y in polynomial-time.

What else could we solve in polynomial time? What else could we solve in polynomial time?

 

Reduction. Problem X polynomial-time (Cook) reduces to problem Y if arbitrary Reduction. Problem X polynomial-time (Cook) reduces to problem Y if arbitrary
instances of problem X can be solved using: instances of problem X can be solved using:
・ Polynomial number of standard computational steps, plus ・Polynomial number of standard computational steps, plus
・Polynomial number of calls to oracle that solves problem Y.
 ・Polynomial number of calls to oracle that solves problem Y.


computational model supplemented by special piece

of hardware that solves instances of Y in a single step
Notation. X ≤ P Y.

Note. We pay for time to write down instances sent to oracle ⇒


 instances of Y must be of polynomial size.
Algorithm

instance I
 solution S to I 

for Y
(of X)
Caveat. Don't mistake X ≤ P Y with Y ≤ P X.

Algorithm for X

21 22

Polynomial-time reductions Independent set

Design algorithms. If X ≤ P Y and Y can be solved in polynomial time,
 INDEPENDENT-SET. Given a graph G = (V, E) and an integer k, is there a subset of
then X can be solved in polynomial time. vertices S ⊆ V such that | S | ≥ k, and for each edge at most one of its endpoints is in

 S?
Establish intractability. If X ≤ P Y and X cannot be solved in polynomial time, then Y 

cannot be solved in polynomial time. Ex. Is there an independent set of size ≥ 6 ?

 Ex. Is there an independent set of size ≥ 7 ?
Establish equivalence. If both X ≤ P Y and Y ≤ P X, we use notation X ≡ P Y.

In this case, X can be solved in polynomial time iff Y can be.





independent set of size 6


Bottom line. Reductions classify problems according to relative difficulty.

23 24
Vertex cover Vertex cover and independent set reduce to one another

VERTEX-COVER. Given a graph G = (V, E) and an integer k, is there a subset of Theorem. VERTEX-COVER ≡P INDEPENDENT-SET.
vertices S ⊆ V such that | S | ≤ k, and for each edge, at least one of its endpoints is Pf. We show S is an independent set of size k iff V − S is a vertex cover

in S ? of size n – k.

Ex. Is there a vertex cover of size ≤ 4 ?
Ex. Is there a vertex cover of size ≤ 3 ?

independent set of size 6 independent set of size 6

vertex cover of size 4 vertex cover of size 4

25 26

Vertex cover and independent set reduce to one another Vertex cover and independent set reduce to one another (END 21)

Theorem. VERTEX-COVER ≡P INDEPENDENT-SET. Theorem. VERTEX-COVER ≡P INDEPENDENT-SET.


Pf. We show S is an independent set of size k iff V − S is a vertex cover
 Pf. We show S is an independent set of size k iff V − S is a vertex cover

of size n – k.
 of size n – k.


 

⇒ ⇐
・ Let S be any independent set of size k. ・Let V − S be any vertex cover of size n – k.
・V − S is of size n – k. ・S is of size k.
・Consider an arbitrary edge (u, v). ・Consider two nodes u ∈ S and v ∈ S.
・S independent ⇒ either u ∉ S or v ∉ S (or both) ・Observe that (u, v) ∉ E since V − S is a vertex cover.
⇒ either u ∈ V − S or v ∈ V − S (or both). ・Thus, no two nodes in S are joined by an edge ⇒ S independent set. ▪
・ Thus, V − S covers (u, v).


27 28