EECS 4101 Advanced Data Structures Polynomial-time reductions

1
Desiderata'. Suppose we could solve problem Y in polynomial-time.

NP APPROXIMATION What else could we solve in polynomial time?


Reduction. Problem X polynomial-time (Cook) reduces to problem Y if arbitrary
instances of problem X can be solved using:
・Polynomial number of standard computational steps, plus
・Polynomial number of calls to oracle that solves problem Y.

Lec 23 : NP- Approximation CLRS 34-35
computational model supplemented by special piece

(mostly Kevin Wayne slides) of hardware that solves instances of Y in a single step



Algorithm

instance I
 solution S to I
for Y
(of X)

Algorithm for X

Recap Decision problems

Decision problem.
3-SAT ・Problem X is a set of strings.
・Instance s is one string.


es
uc
red

・Algorithm A solves problem X:

e T
tim -SE
l y- NT
T po
EN
DE A(s) = yes iff s ∈ X.
SA
3- EP
I ND
to 

Def. Algorithm A runs in polynomial time if for every string s, A(s) terminates in at
INDEPENDENT-SET DIR-HAM-CYCLE GRAPH-3-COLOR SUBSET-SUM
most p( | s | ) "steps", where p(⋅) is some polynomial function.



 length of s



VERTEX-COVER HAM-CYCLE PLANAR-3-COLOR SCHEDULING Ex.
・Problem PRIMES = { 2, 3, 5, 7, 11, 13, 17, 23, 29, 31, 37, …. }.
・Instance s = 592335744548702854681.
・AKS algorithm: solves PRIMES in O( | s | 8 ) steps.
SET-COVER TSP 3-SAT poly-time reduces to all of
these problems (and many, many more)

3 4
Definition of P NP

P. Decision problems for which there is a poly-time algorithm. Certification algorithm intuition.
・Certifier views things from "managerial" viewpoint.
・Certifier doesn't determine whether s ∈ X on its own;

Problem Description Algorithm yes no rather, it checks a proposed proof t that s ∈ X.


MULTIPLE Is x a multiple of y ? grade-school division 51, 17 51, 16 Def. Algorithm C(s, t) is a certifier for problem X if for every string s,

s ∈ X iff there exists a string t such that C(s, t) = yes.
REL-PRIME Are x and y relatively prime ? Euclid (300 BCE) 34, 39 34, 51 

"certificate" or "witness"


PRIMES Is x prime ? AKS (2002) 53 51 Def. NP is the set of problems for which there exists a poly-time certifier.
・C(s, t) is a poly-time algorithm.
・Certificate t is of polynomial size:
Is the edit distance between x
EDIT-DISTANCE dynamic programming niether neither acgggt ttttta | t | ≤ p( | s | ) for some polynomial p(⋅).
and y less than 5 ?


Is there a vector x that Gauss-Edmonds
L-SOLVE satisfies Ax = b ? 

elimination
Remark. NP stands for nondeterministic polynomial time.
Is an undirected graph
 depth-first search
U-CONN G connected? (Theseus)

5 6

Certifiers and certificates: composite Certifiers and certificates: satisfiability

COMPOSITES. Given an integer s, is s composite? SAT. Given a CNF formula Φ, does it have a satisfying truth assignment?

 3-SAT. SAT where each clause contains exactly 3 literals.
Certificate. A nontrivial factor t of s. Such a certificate exists iff s is composite. 

Moreover | t | ≤ | s |. Certificate. An assignment of truth values to the Boolean variables.

 

Certifier. Check that 1 < t < s and that s is a multiple of t. Certifier. Check that each clause in Φ has at least one true literal.

 


 


 

instance s 437669

 
 instance s
certificate t 541 or 809 437,669 = 541 × 809

 

certificate t x1 = true, x2 = true, x3 = false, x4 = false

 


 


 

Conclusion. COMPOSITES ∈ NP. in fact, COMPOSITES ∈ P 



Conclusions. SAT ∈ NP, 3-SAT ∈ NP.
7 8
Certifiers and certificates: Hamilton path Definition of NP

HAM-PATH. Given an undirected graph G = (V, E), does there exist a simple path P NP. Decision problems for which there is a poly-time certifier/verifier
that visits every node?


Problem Description Algorithm yes no
Certificate. A permutation of the n nodes.

 Is there a vector x
 Gauss–Edmonds
L-SOLVE that satisfies Ax = b ?
Certifier. Check that the permutation contains each node in V exactly once, and that elimination

there is an edge between each pair of adjacent nodes.

COMPOSITES Is x composite ? AKS (2002) 51 53



 Does x have a nontrivial factor less
FACTOR than y ?
? (56159, 50) (55687, 50)



 Given a CNF formula, does it have a ¬ x1 ∨ ¬ x2 ∨ ¬ x3 ¬ x1 ∨ ¬ x2
SAT ? ¬ x1 ∨ ¬ x2 ∨ ¬ x3 ¬ x1 ∨ ¬ x2

 satisfying truth assignment? ¬ x1 ∨ ¬ x2 ∨ ¬ x3 ¬ x1 ∨ ¬ x2


 Can the nodes of a graph G


3-COLOR ?

 be colored with 3 colors?
instance s certificate t

 Is there a simple path between

HAM-PATH ?
Conclusion. HAM-PATH ∈ NP. u and v that visits every node?

9 10

Definition of NP P, NP, and EXP

NP. Decision problems for which there is a poly-time certifier. P. Decision problems for which there is a poly-time algorithm.
NP. Decision problems for which there is a poly-time certifier.
EXP. Decision problems for which there is an exponential-time algorithm.


Claim. P ⊆ NP.
“ NP captures vast domains of computational, scientific, and mathematical
 Pf. Consider any problem X ∈ P.
endeavors, and seems to roughly delimit what mathematicians and scientists
 ・By definition, there exists a poly-time algorithm A(s) that solves X.
have been aspiring to compute feasibly. ” — Christos Papadimitriou ・Certificate t = ε, certifier C(s, t) = A(s). ▪


Claim. NP ⊆ EXP.
Pf. Consider any problem X ∈ NP.

“ In an ideal world it would be renamed P vs VP. ”
— Clyde Kruskal ・By definition, there exists a poly-time certifier C(s, t) for X,

where certificate t satisfies | t | ≤ p( | s | ) for some polynomial p(⋅).
・To solve input s, run C(s, t) on all strings t with | t | ≤ p(| s |).
・Return yes if C(s, t) returns yes for any of these potential certificates. ▪
Remark. Time-hierarchy theorem implies P ⊊ EXP.
11 12
The main question: P vs. NP The main question: P vs. NP

Q. How to solve an instance of 3-SAT with n variables? Does P = NP? [Cook 1971, Edmonds, Levin, Yablonski, Gödel]

A. Exhaustive search: try all 2n truth assignments. Is the decision problem as easy as the certification problem?

 

Q. Can we do anything substantially more clever? 

Conjecture. No poly-time algorithm for 3-SAT. 

EXP NP EXP


"intractable"

 P P = NP






 If P ≠ NP If P = NP




If yes. Efficient algorithms for 3-SAT, TSP, 3-COLOR, FACTOR, …
If no. No efficient algorithms possible for 3-SAT, TSP, 3-COLOR, …


Consensus opinion. Probably no.

13 14

Possible outcomes Possible outcomes

P ≠ NP. P ≠ NP.

“ In my view, there is no way to even make intelligent guesses about the

answer to any of these questions. If I had to bet now, I would bet that
“ I conjecture that there is no good algorithm for the traveling salesman
P is not equal to NP. I estimate the half-life of this problem at 25–50
problem. My reasons are the same as for any mathematical conjecture:
more years, but I wouldn’t bet on it being solved before 2100. ”
(i) It is a legitimate mathematical possibility and (ii) I do not know.”
— Bob Tarjan (2002)
— Jack Edmonds 1966

“ We seem to be missing even the most basic understanding of the

nature of its difficulty…. All approaches tried so far probably (in

some cases, provably) have failed. In this sense P =NP is different
from many other major mathematical problems on which a gradual
progress was being constantly done (sometimes for centuries)
whereupon they yielded, either completely or partially. ”
— Alexander Razborov (2002)
15 16
Possible outcomes
P ≠ NP.
“ In my opinion this shouldn’t really be a hard problem; it’s just
that we came late to this theory, and haven’t yet developed any
techniques for proving computations to be hard. Eventually, it will
just be a footnote in the books. ” — John Conway
17
Other possible outcomes
P = NP, but only Ω(n100) algorithm for 3-SAT.
P ≠ NP, but with O(nlog*n) algorithm for 3-SAT.
P = NP is independent (of ZFC axiomatic set theory).
“ It will be solved by either 2048 or 4096. I am currently somewhat
pessimistic. The outcome will be the truly worst case scenario:
namely that someone will prove “P = NP because there are only
finitely many obstructions to the opposite hypothesis”; hence there
exists a polynomial time solution to SAT but we will never know

its complexity! ” — Donald Knuth
19
Possible outcomes
P = NP.
“ I think that in this respect I am on the loony fringe of the mathematical
community: I think (not too strongly!) that P=NP and this will be
proved within twenty years. Some years ago, Charles Read and I
worked on it quite bit, and we even had a celebratory dinner in a
good restaurant before we found an absolutely fatal mistake. ”

— Béla Bollobás (2002)
18
Millennium prize
Millennium prize. $1 million for resolution of P = NP problem.
20
NP-complete 3-satisfiability is NP-complete

NP-complete. A problem Y ∈ NP with the property that for every
 Theorem. 3-SAT ∈ NP-complete.
problem X ∈ NP, X ≤ p Y. Pf. Cook, 1971; Levin, 1973


Theorem. Suppose Y ∈ NP-complete. Then Y ∈ P iff P = NP.
Pf. ⇐ If P = NP, then Y ∈ P because Y ∈ NP.
Pf. ⇒ Suppose Y ∈ P.
・Consider any problem X ∈ NP. Since X ≤ p Y, we have X ∈ P.
・This implies NP ⊆ P.
・We already know P ⊆ NP. Thus P = NP. ▪








Fundamental question. Do there exist "natural" NP-complete problems?

21 22

Establishing NP-completeness More hard computational problems

Aerospace engineering. Optimal mesh partitioning for finite elements.

Remark. Once we establish first "natural" NP-complete problem,

Biology. Phylogeny reconstruction.
others fall like dominoes.
Chemical engineering. Heat exchanger network synthesis.


Chemistry. Protein folding.
Recipe. To prove that Y ∈ NP-complete:
・Step 1.
Civil engineering. Equilibrium of urban traffic flow.
Show that Y ∈ NP.
・Step 2.
Economics. Computation of arbitrage in financial markets with friction.
Choose an NP-complete problem X. Electrical engineering. VLSI layout.

・Step 3. Prove that X ≤ p Y. Environmental engineering. Optimal placement of contaminant sensors.


 Financial engineering. Minimum risk portfolio of given return.

 Game theory. Nash equilibrium that maximizes social welfare.

Theorem. If X ∈ NP-complete, Y ∈ NP, and X ≤ p Y, then Y ∈ NP-complete. Mathematics. Given integer a1, …, an, compute

Pf. Consider any problem W ∈ NP. Then, both W ≤ p X and X ≤ p Y. Mechanical engineering. Structure of turbulence in sheared flows.

・By transitivity, W ≤ p Y. Medicine. Reconstructing 3d shape from biplane angiocardiogram.

・Hence Y ∈ NP-complete. ▪ by definition of


NP-complete
by assumption Operations research. Traveling salesperson problem.
Physics. Partition function of 3d Ising model.
Politics. Shapley–Shubik voting power.
Recreation. Versions of Sudoku, Checkers, Minesweeper, Tetris.
Statistics. Optimal experimental design.
23 24
Extent and impact of NP-completeness P vs. NP revisited

Extent of NP-completeness. [Papadimitriou 1995] Overwhelming consensus (still). P ≠ NP.

・Prime intellectual export of CS to other disciplines. 

・6,000 citations per year (more than "compiler", "OS", "database"). 

・Broad applicability and classification power. 
 NP

 P NPC P = NP


NP-completeness can guide scientific inquiry. 

・1926: Ising introduces simple model for phase transitions. 

・1944: Onsager finds closed-form solution to 2D-ISING in tour de force. 
 P ≠ NP P = NP

・19xx: Feynman and other top minds seek solution to 3D-ISING. 


・2000: Istrail proves 3D-ISING ∈ NP-complete. a holy grail of Why we believe P ≠ NP.
statistical mechanics

search for closed formula appears doomed

“ We admire Wiles' proof of Fermat's last theorem, the scientific theories of Newton,
Einstein, Darwin, Watson and Crick, the design of the Golden Gate bridge and the
Pyramids, precisely because they seem to require a leap which cannot be made by
everyone, let alone by a simple mechanical device. ” — Avi Wigderson

25 26

Is factoring in P ? Exploiting intractability

Fundamental question. Is FACTOR ∈ P ? Modern cryptography.


 ・Ex. Send your credit card to Amazon.
Challenge. Factor this number. ・Ex. Digitally sign an e-document.
・Enables freedom of privacy, speech, press, political association.


7403756347956171282804679609742957314259318888923128908493 RSA. Based on dichotomy between complexity of two problems.
6232638972765034028266276891996419625117843995894330502127 ・To use: generate two random n-bit primes and multiply.
5853701189680982867331732731089309005525051168770632990723 ・To break: suffices to factor a 2n-bit integer.
96380786710086096962537934650563796359
RSA-704

($30,000 prize if you can factor)

RSA sold

RSA algorithm for $2.1 billion or design a t-shirt
27 28
Coping with NP-completeness Vertex cover

Q. Suppose I need to solve an NP-complete problem. What should I do? Given a graph G = (V, E) and an integer k, is there a subset of vertices S ⊆ V such
A. Theory says you're unlikely to find poly-time algorithm. that | S | ≤ k, and for each edge (u, v) either u ∈ S or v ∈ S or both?

Must sacrifice one of three desired features.

・Solve problem to optimality.
・Solve problem in polynomial time. 1 6

・Solve arbitrary instances of the problem.

2 7

This lecture. Solve some special cases of NP-complete problems. 3 8

4 9

5 10

S = { 3, 6, 7, 10 } is a vertex cover of size k = 4

29 30

Subset sum algorithm

Dealing with NP-completeness • In the subset-sum problem you are given a list S of positive numbers
and a target value t. You must decide if there is any subset of the
numbers that would add up to the target.
– Solve P=NP with an efficient algorithm for SAT or: • Example: the set {4,5} and target 19?
• In this case the answer is no.
– Give up on one of:
• One way to store information about what possible sums are reachable
– finding a general algorithm that solves any from a given set of numbers is a boolean array with indices in range
instance of the problem efficiently 0…t
• Entries in the array would be set to true if there were a subset of the
– solving the problem correctly or exactly (i.e. must given numbers that sums up the value of the subscript
settle for approximate solution) • Example: Suppose we have the numbers above, the boolean array
would indicate every possible value reachable by choosing any of the
– having an algorithm that can solve the problem subsets of the set {4,5}
in polynomial time. But you can still try to get a • table indexed from 0 to 19, entries 0, 4, 5, 9 are true, rest are false
“good” exponential algorithm • The key idea in developing an algorithm for the general problem is to
31
consider what happens to the above table if a new number is added to
the set {4,5} 32
 SUBSETSUM algorithm - basic idea Algorithm for subset sum
• Suppose we have a big array S of bits, indexed from 0 to t to represent all sums – SUBSETSUM problem with n numbers x1, x2, ...,xn and target t
less than or equal to the target that can be reached using any subset of the given – Bit Vector: Tables of length t +1 indexed from 0 to t so that:
numbers. For this example the target =14, and the numbers are {4,5} i
– S [ j ] = true if there is a subset of first i numbers that sums to j
T t T T t t T t – for i = 0, S0[0] = true and for all 1≦ j ≦ t, S0[ j ] = false;
S=
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 – for 1 ≦ i ≦ n, for all 0 ≦ j ≦ t S i+1 [ j ] = Si [ j ] ⋁ Si [ j - xi+1 ] *see
note below

• S[j] is set true if there is any subset of the numbers that sums to j, false otherwise
– Correctness: The sums that can be reached using any subset of
(We are showing only the true ones above)
the first i+1 variables are (a) the sums that can be reached using
• S is shown for the numbers { 4, 5 } with target 14. By choosing various subsets of
any subset of the first i of the x values (i.e. the subset doesn’t
the numbers {4,5 } any of the sums marked by a ( black, upper-case) T can be
reached include the use of xi+1) and (b) the sums that can be reached by
• How would S change if a new number were added to the set, e.g. the set of
using xi+1 added to any of the sums that can be reached using only
numbers becomes {4,5,3}? a subset of the first i of the numbers
• The new set of reachable sums includes everything in the old set, plus any sum ( * and interpret any out of range subscripts as false)
that can be reached by adding the new number to any of the sums that could be Don’t really need n tables, but must be careful overwriting - go from
reached using the old set ( these new values are shown in lower-case blue) top end to small end to avoid confusion between old and new values
33
34

Analysis of Pseudo-polynomial
Algorithm for subset sum
– SUBSETSUM with n numbers x1, x2, ...,xn and target t
algorithm for subset sum
– Bit Vector: Tables of length t – So we have n tables to fill out, each of length t
– Want Ti [ j ] = 1 if there is a subset of first i numbers that – Simple bit operation and table lookup for each entry on a
Random Access Machine
sums to j
– O(nt ) - is this polynomial time ?
– for i = 0, for all 1≦ j ≦ t, T0[ j ] = 0
– Certainly the O(n) part is fine, but what about the t?
– for 1 ≦ i ≦ n, for all 1≦ j ≦ t T i+1 [ j ] = Ti [ j ] ⋁ Ti [ j - xi+1
– How does t compare to the size of the input?
]
– t itself takes O(log2 t) bits to record, so its value t is
– Correctness: The sums that can be reached using the actually exponential in the size of its representation
first i+1 values of x are the same as the sums that can
– Example: if t were an n-bit number, t could have a value
be reached using the first i values (i.e. they don’t use
around 2n so the algorithm running in time t is running in
xi+1) along with the sums that can be reached using xi+1
time exponential in size of the input n
plus sums that are reachable using the first i values of x
– This is called a pseudo-polynomial algorithm
35SUBSETSUM with n numbers x1, x2, ...,xn and target t 36

– Bit Vector: Tables of length t

Pseudo-polynomial algorithm for subset
sum
– So we have n tables to fill out, each of length t
– Simple bit operation and table lookup for each entry
– O(nt ) - polynomial time ?
– Certainly the O(n) part is fine, but what about the t?
– How does t compare to the size of the input?
– t itself takes log2 t bits to record, so its value is
actually exponential in its size
– Example if t were an n-bit number, our algorithm runs
in t steps, which is exponential time in the size of the
input ≥ number of bits of t
37
Approximation Schemes
– Approximate vertex cover
– Can get a solution within twice the optimal
– Approximate ̗TSP
– for any ɛ can get solution within factor of (1+ɛ)
– But for general TSP, existence of a poly time algorithm
that produces a solution with cost that is guaranteed
to be within twice the optimal cost implies that P=NP
– Exercise: Prove the above. Hint: Show how to use
such an algorithm to solve HAMCYCLE 39
Pseudo-polynomial algorithms
– Such algorithms are called “pseudo-polynomial” because they
are polynomial in the value of one of the inputs, not its size
– We would not want an addition algorithm that worked in time
proportional to the values of the operands, we want time
proportional to the size of their representations (in binary, not
unary), and this is exponentially smaller
– NP-complete problems that remain NP-complete, requiring
time that is exponential even in the values (as opposed to
number of bits) of any numbers, as well as in the overall size
of the problem instance, are called strongly-NP-complete,
and of course, no polynomial or pseudo-polynomial
algorithms are known for them
– Of problems we have studied, all are strongly NP-complete
except SUBSETSUM (and KNAPSACK). (remember those
big numbers in the reduction? formula of length n generates
n-bit numbers, i.e. number values as big as 2n)
38
Approximating min vertex cover
– Algorithm 1: repeat until no edges remain in G
– Take remaining vertex of highest degree, add it to VC, and
remove it and all edges it touches from G
– This guarantees a vertex cover of size no more than O(log n) *
size of minimum vertex cover
– Algorithm 2: repeat until no edges remain in G
– Take any edge remaining and add both endpoints to VC and
remove them and all incident edges from G
– This guarantees a vertex cover of size no worse than two times
the size of minimum vertex cover
– Reason: any maximal matching must be a vertex cover and the
min vertex cover must include at least one vertex from every
edge in the matching
40