RAIK 283: Data Structures & Algorithms

Dynamic Programming Warshalls and FloydsAlgorithm

Dr. Ying Lu ylu@cse.unl.edu

Design and Analysis of Algorithms - Chapter 8

RAIK 283: Data Structures & Algorithms

Giving credit where credit is due:

Most of the lecture notes are based on the slides from the Textbooks companion website http://www.aw-bc.com/info/levitin Some of the notes are based on slides created by Dr. Philippe Laborie, ILOG and Mark Llewellyn, University of Central Florida. I have modified many of their slides and added new slides.

Design and Analysis of Algorithms - Chapter 8

Examples of dynamic programming algorithms

Coin-row problem Computing binomial coefficients Longest Common Subsequence (LCS) Warshalls algorithm for transitive closure Floyds algorithm for all-pairs shortest paths

Some instances of difficult discrete optimization problems: 0-1 knapsack

Warshalls algorithm: transitive closure

Computes the transitive closure of a relation (Alternatively: all paths in a directed graph)
Example of transitive closure: 3 1 1 3

0 1 0 0

0 0 0 1

1 0 0 0

0 1 0 0

0 1 0 1

0 1 0 1

1 1 0 1

0 1 0 1

Design and Analysis of Algorithms - Chapter 8

Warshalls algorithm
Main

idea: a path exists between two vertices i, j, iff

there is an edge from i to j; or there is a path from i to j going through intermediate vertices which are drawn from set {vertex 1}; or

there is a path from i to j going through intermediate vertices which are drawn from set {vertex 1, 2}; or

Design and Analysis of Algorithms - Chapter 8

Warshalls algorithm
Main

idea: a path exists between two vertices i, j, iff

there is a path from i to j going through intermediate vertices which are drawn from set {vertex 1, 2, k-1}; or there is a path from i to j going through intermediate vertices which are drawn from set {vertex 1, 2, k}; or ... there is a path from i to j going through any of the other vertices

Design and Analysis of Algorithms - Chapter 8

Warshalls algorithm
Idea:

dynamic programming

Let V={1, , n} and for kn, Vk={1, , k} For any pair of vertices i, jV, identify all paths from i to j whose intermediate vertices are all drawn from Vk: Pijk={p1, p2, }, if Pijk then Rk[i, j]=1 V
k

P1 p2

For any pair of vertices i, j: Rn[i, j], that is Rn Starting with R0=A, the adjacency matrix, how to get R1 Rk-1 Rk Rn
Design and Analysis of Algorithms - Chapter 8 7

Warshalls algorithm
Idea:

dynamic programming

pPijk: p is a path from i to j with all intermediate vertices in Vk If k is not on p, then p is also a path from i to j with all intermediate vertices in Vk-1: pPijk-1

Vk-1

Vk

Design and Analysis of Algorithms - Chapter 8

Warshalls algorithm
Idea:

dynamic programming

pPijk: p is a path from i to j with all intermediate vertices in Vk If k is on p, then we break down p into p1 and p2
What are P1 and P2?

p p1

k
Vk-1

Vk p2

i
Design and Analysis of Algorithms - Chapter 8

j
9

Warshalls algorithm
Idea:

dynamic programming

pPijk: p is a path from i to j with all intermediate vertices in Vk If k is on p, then we break down p into p1 and p2 where
p1 is a path from i to k with all intermediate vertices in Vk-1 p2 is a path from k to j with all intermediate vertices in Vk-1

p1

k
Vk-1

Vk p2

i
Design and Analysis of Algorithms - Chapter 8

j
10

Warshalls algorithm
In the kth stage determine if a path exists between two vertices i, j using just vertices among 1, , k

R(k)[i,j] =

R(k-1)[i,j] (path using just 1, , k-1) or (R(k-1)[i,k] and R(k-1)[k,j]) (path from i to k and from k to j k using just 1, , k-1) kth stage

j
Design and Analysis of Algorithms - Chapter 8 11

Warshalls algorithm
3 3 1 4 R0 0 1 0 0 0 0 1 0 0 1 0 0 3 1 4 2 R1 0 0 1 0 0 0 0 1 4 3 1 1 1 0 0 0 1 0 0 R3 0 0 1 0 0 0 1 1 4 R2 0 0 1 0 0 0 1 1 1 1 0 1 0 1 0 1 1

2 0 1 0 0

3
1

1 1 0 1

0 1 0 1

R4 0 0 1 1 0 0 1 1

1 1 0 1

0 1 0 1
12

Design and Analysis of Algorithms - Chapter 8

Warshalls algorithm

R0 = A 0 0 1 0 1 0 0 1 0 0 0 0 0 1 0 0

R1 0 0 1 0 0 0 0 1

1 1 0 0

0 1 0 0

R2 0 0 1 0 0 0 1 1

1 1 0 1

0 1 0 1

R3 0 0 1 0 0 0 1 1

1 1 0 1

0 1 0 1

R4 0 0 1 1 0 0 1 1

1 1 0 1

0 1 0 1

Design and Analysis of Algorithms - Chapter 8

13

In-class exercises

8.4.1 Apply Warshalls algorithm to find the transitive closure of the digraph defined by the following adjacency matrix

0 0 0 0

1 0 0 0

0 1 0 0

0 0 1 0

Design and Analysis of Algorithms - Chapter 8

14

In-class exercises

Problem 2 a. What is the time efficiency of Warshalls algorithm?

Design and Analysis of Algorithms - Chapter 8

15

In-class exercises

Problem 2 a. What is the time efficiency of Warshalls algorithm? b. How to solve this finding all paths in a directed graph problem by a traversal-based algorithm (BFS-based or DFS-based)?

Design and Analysis of Algorithms - Chapter 8

16

In-class exercises

Problem 2 a. What is the time efficiency of Warshalls algorithm? b. How to solve this finding all paths in a directed graph problem by a traversal-based algorithm (BFS-based or DFS-based)? c. Explain why the time efficiency of Warshalls algorithm is inferior to that of traversal-based algorithm for sparse graphs represented by their adjacency lists.

Design and Analysis of Algorithms - Chapter 8

17

Floyds algorithm: all pairs shortest paths

In a weighted graph, find shortest paths between every pair of vertices Same idea: construct solution through series of matrices D(0), D(1), using an initial subset of the vertices as intermediaries.

In D(k), dij(k): weight of the shortest path from ui to uj with all intermediate vertices in an initial subset {u1, u2, uk} 4 3 1 1 Example: 6
1 5

Design and Analysis of Algorithms - Chapter 8

4
18

Floyds algorithm: all pairs shortest paths

Idea:

dynamic programming

Let V={u1,,un} and for kn, Vk={u1,,uk} To construct D(k) , we need to get dijk For any pair of vertices ui, ujV, consider all paths from ui to uj whose intermediate vertices are all drawn from Vk and find p the shortest path among them, weight of p is dijk Vk

ui

uj
19

Design and Analysis of Algorithms - Chapter 8

Floyds algorithm: all pairs shortest paths

Idea:

dynamic programming

Let V={u1,,un} and for kn, Vk={u1,,uk} To construct D(k) , we need to get dijk For any pair of vertices ui, ujV, consider all paths from ui to uj whose intermediate vertices are all drawn from Vk and find p the shortest path among them, weight of p is dijk If uk is not on p, then what is dijk equal to? Vk

ui

uj
20

Design and Analysis of Algorithms - Chapter 8

Floyds algorithm: all pairs shortest paths

Idea:

dynamic programming

If uk is not on p, then a shortest path from ui to uj with all intermediate vertices in Vk-1 is also a shortest path in Vk , i.e., dij(k) = dij(k-1).

If uk is on p, then we break down p into p1 and p2

What are p1, p2?

Design and Analysis of Algorithms - Chapter 8

21

Floyds algorithm: all pairs shortest paths

Idea:

dynamic programming

If uk is not in p, then a shortest path from ui to uj with all intermediate vertices in Vk-1 is also a shortest path in Vk , i.e., dij(k) = dij(k-1).

If uk is in p, then we break down p into p1 and p2 where

p1 is the shortest path from ui to uk with all intermediate vertices in Vk-1 p2 is the shortest path from uk to uj with all intermediate vertices in Vk-1

i.e., dij(k) = dik(k-1)+ dkj(k-1).

Design and Analysis of Algorithms - Chapter 8 22

Floyds algorithm: all pairs shortest paths

Idea:

dynamic programming

Construct matrices D(0), D(1), D(k-1), D(k) D(n) dij(k): weight of the shortest path from ui to uj with all intermediate vertices in Vk dij(0)=wij dij(k)=min (dij(k-1), dik(k-1)+ dkj(k-1)) for k1

Design and Analysis of Algorithms - Chapter 8

23

Floyds algorithm: all pairs shortest paths

FLOYD(G) for i,j in [1..n] d[i,j]=w(ui,uj) for k in [1..n] for i in [1..n] for j in [1..n] d[i,j]=min(d[i,j],d[i,k]+d[k,j])

Time complexity: O(n3)

Design and Analysis of Algorithms - Chapter 8

24

In-Class Exercises

Apply the Floyds algorithm to the following problem instance: D(0) = 0 3 2 0 7 0 1 6 0

How to enhance Floyds algorithm to actually output the shortest path from node i to node j?

Design and Analysis of Algorithms - Chapter 8

25

Enhanced Floyds Algorithm

FloydEnhanced(w[1..n, 1..n]) for i,j in [1..n] d[i,j]=w(ui,uj) for i in [1..n] for j in [1..n] p[i,j]=0 for k in [1..n] for i in [1..n] for j in [1..n] if d[i,j] >d[i,k]+d[k,j] d[i,j]=d[i,k]+d[k,j] p[i,j] = k ShortestPath(i, j, p[1..n, 1..n]) k = p[i, j] if k 0 ShortestPath(i, k) print(k) ShortestPath(k, j)

Design and Analysis of Algorithms - Chapter 8

26

In-Class Exercises

Apply the enhanced Floyds algorithm, which finds the shortest paths and their lengths, to the following problem instance: D(0) = 0 3 2 0 7 0 1 6 0

Design and Analysis of Algorithms - Chapter 8

27

Dynamic programming

Dynamic programming is a technique for solving problems with overlapping subproblems. It suggests solving each smaller subproblem once and recording the results in a table from which a solution to the original problem can be then obtained.

Design and Analysis of Algorithms - Chapter 8

28

Dynamic programming

Dynamic programming is a technique for solving problems with overlapping subproblems. It suggests solving each smaller subproblem once and recording the results in a table from which a solution to the original problem can be then obtained. What are the overlapping subproblems in Floyds algorithm?
FLOYD(G) for i,j in [1..n] d[i,j]=w(ui,uj) for k in [1..n] for i in [1..n] for j in [1..n] d[i,j]=min(d[i,j],d[i,k]+d[k,j])
Design and Analysis of Algorithms - Chapter 8 29

Floyds Algorithm
dij(k)=min (dij(k-1), dik(k-1)+ dkj(k-1)) for k1 p p1

k
Vk-1

Vk p2

solutions for smaller subproblems solution for a larger subproblem

Floyds Algorithm
dij(k)=min (dij(k-1), dik(k-1)+ dkj(k-1)) for k1 dil(k)=min (dil(k-1), dik(k-1)+ dkl(k-1)) for k1 p p1

k
Vk-1
p3

Vk p2

l
solution for a smaller subproblem is used for getting solutions for multiple bigger subproblems

In-class exercise

What does dynamic programming have in common with divide-and-conquer? What is a principal difference between them?

Design and Analysis of Algorithms - Chapter 8

32