Intractability/NP-Completeness

Lesson 12.23,651 words

NP-Completeness

Some problems in NP\mathsf{NP} are universally hardest: every other problem in NP\mathsf{NP} reduces to them. This lesson defines NP\mathsf{NP}-hard and NP\mathsf{NP}-complete, states the Cook–Levin theorem that anchors the theory on SAT, walks the web of reductions that grows from it, and gives the four-step recipe for proving a brand-new problem NP\mathsf{NP}-complete.

╌╌╌╌

The previous lesson left us with a claim: inside there are problems that are universally hardest, and they decide the versus question. This lesson makes that claim precise. We define what it means to be hardest, identify the first such problem (SAT, via the Cook–Levin theorem), and then show how a single starting problem yields many more equally hard problems through reductions. Finally we state the four-step recipe for proving a new problem -complete.

NP-hard and NP-complete

Recall that means is no harder than . Now imagine a problem that every problem in is no harder than. Such a is at least as hard as everything in , a ceiling on the whole class.

The distinction matters. -hardness is a pure lower bound: is as hard as anything in , but need not itself be in ; it could be far harder, even undecidable. -complete problems are the ones that are hardest and still belong to : they sit exactly at the frontier. They are the hardest problems whose solutions we can still efficiently check.1

-complete is the intersection of and -hard; some -hard problems lie outside .

Two consequences follow.

  • All -complete problems stand or fall together. Suppose is -complete and someone finds a polynomial-time algorithm for . Then for any we have , so is solvable in polynomial time too, every at once. Hence Conversely, proving any single -complete problem requires super-polynomial time would prove . In this sense the thousands of known -complete problems are a single problem in many guises.
  • Transitivity bootstraps the class. If is -complete and we show for some , then is -complete too: every satisfies , and composes. This is how one-complete problem yields many.
Transitivity bootstrap: every already reduces to the known-complete ; one new reduction extends the chain, so inherits hardness from the whole class.

Transitivity spreads -completeness from one problem to the next, but it presupposes we already have a first -complete problem to start from. Where does the first one come from?

The first one: Cook–Levin

The breakthrough, proved independently by Stephen Cook (1971) and Leonid Levin, is that a natural problem is -complete from scratch, without reducing from anything, by reasoning directly about computation itself.

The problem is boolean satisfiability, or SAT.

Cook–Levin encodes a verifier's whole computation on as one formula , satisfiable iff some accepts.
sat.pypython
from itertools import product
from typing import Iterable, Iterator, NamedTuple, Optional

# A truth assignment maps each variable name to True or False.
Assignment = dict[str, bool]

class Literal(NamedTuple):
  """
    One occurrence of a variable in a clause: its name and whether it is\n
    negated. `Literal("x", False)` is the literal x; `Literal("x", True)`\n
    is the literal not-x.\n
  """
  variable: str
  negated: bool

  def is_satisfied_by(self, assignment: Assignment) -> bool:
    """
      Whether this literal evaluates to true under `assignment`.\n
    """
    value: bool = assignment[self.variable]
    return (not value) if self.negated else value

  def __repr__(self) -> str:
    return f"{'¬' if self.negated else ''}{self.variable}"

class Clause(NamedTuple):
  """
    A disjunction (OR) of literals; true when any one literal is true.\n
  """
  literals: tuple[Literal, ...]

  def is_satisfied_by(self, assignment: Assignment) -> bool:
    """
      Whether some literal in this clause is true under `assignment`.\n
    """
    return any(literal.is_satisfied_by(assignment) for literal in self.literals)

class Formula(NamedTuple):
  """
    A CNF formula: a conjunction (AND) of clauses, true only when every\n
    clause is true.\n
  """
  clauses: tuple[Clause, ...]

  @property
  def variables(self) -> list[str]:
    """
      Every distinct variable name, in first-seen order.\n
    """
    # a dict preserves first-seen order while collapsing duplicates.
    seen: dict[str, None] = {}
    for clause in self.clauses:
      for literal in clause.literals:
        seen.setdefault(literal.variable, None)

    return list(seen)

  def is_three_cnf(self) -> bool:
    """
      Whether every clause holds exactly three literals (the 3-SAT shape).\n
    """
    return all(len(clause.literals) == 3 for clause in self.clauses)

  def is_satisfied_by(self, assignment: Assignment) -> bool:
    """
      Whether `assignment` makes every clause — and so the whole formula —\n
      true.\n
    """
    return all(clause.is_satisfied_by(assignment) for clause in self.clauses)

def make_formula(clauses: Iterable[Iterable[tuple[str, bool]]]) -> Formula:
  """
    Build a Formula from a lightweight description: an iterable of clauses,\n
    each an iterable of `(variable, negated)` pairs. Convenient for tests\n
    and callers that would rather not nest the NamedTuples by hand.\n
  """
  # wrap each (variable, negated) pair into a Literal, each clause into a Clause.
  built: list[Clause] = []
  for raw_clause in clauses:
    literals: tuple[Literal, ...] = tuple(
      Literal(variable, negated) for variable, negated in raw_clause
    )
    built.append(Clause(literals))

  return Formula(tuple(built))

def verify(formula: Formula, assignment: Assignment) -> bool:
  """
    The SAT verifier: check that `assignment` is a satisfying certificate\n
    for `formula`. Runs in time linear in the formula's size — this is the\n
    proof that SAT lies in NP.\n
  """
  return formula.is_satisfied_by(assignment)

def all_assignments(variables: list[str]) -> Iterator[Assignment]:
  """
    Yield every truth assignment over `variables` — all 2^k of them.\n
  """
  for values in product((False, True), repeat=len(variables)):
    yield dict(zip(variables, values))

def satisfiable(formula: Formula) -> Optional[Assignment]:
  """
    Decide SAT by exhaustive search: return a satisfying assignment if one\n
    exists, otherwise None. Takes O(2^k) over k variables — the brute force\n
    no polynomial algorithm is known to beat. The empty formula (no clauses)\n
    is vacuously satisfiable, witnessed by the empty assignment.\n
  """
  # try every truth assignment; return the first that satisfies the formula.
  for assignment in all_assignments(formula.variables):
    if formula.is_satisfied_by(assignment):
      return assignment

  return None

With one -complete problem in hand, transitivity takes over.

The reduction web

Karp's 1972 paper reduced SAT to twenty-one other problems, showing them all -complete and launching the field.3 A convenient intermediate is 3-SAT, the special case of SAT in which the formula is in conjunctive normal form with exactly three literals per clause, a big AND of small ORs such as . One can show , so 3-SAT is itself -complete, and its rigid structure makes it the favorite starting point for further reductions.

The reduction is a small gadget worth seeing, because it shows how a clause of the wrong width is padded or split to exactly three literals using fresh variables. A clause with one literal becomes with new variables : the four clauses together force true regardless of . A long clause is chained with fresh variables into : a satisfying assignment of the original picks some true , and the can be set to carry the still unsatisfied signal down the chain until that discharges it. Each new clause has exactly three literals, the blow-up is linear, and satisfiability is preserved in both directions — a clean polynomial reduction.

From 3-SAT the web branches out. A small portion:

Reduction web of NP-complete problems branching out from SAT and 3-SAT.

Every arrow is a reduction ; following arrows back to SAT certifies each box as -complete. (The arrows above record one route to each result, not the only one; many of these problems also reduce to each other directly, as we saw with and Clique last lesson.)

A worked reduction: 3-SAT to Independent-Set

Let us actually build one arrow, the classic . We are given a 3-CNF formula with clauses and must produce a graph and integer such that has an independent set of size exactly when is satisfiable.

The construction. For each clause, create a triangle of three vertices, one per literal. So clause becomes three mutually-connected vertices labeled , , . Then add a conflict edge between any two vertices in different triangles that hold contradictory literals: one labeled and another labeled . Finally set , the number of clauses. This is clearly polynomial: vertices and at most edges. The construction is correct in both directions.

Clause triangles with dashed conflict edges for the 3-SAT to Independent-Set reduction.

Solid edges are the per-clause triangles; dashed edges connect contradictory literals ( vs. , and vs. ). Picking one non-conflicting vertex per triangle amounts to a consistent satisfying choice.

A fully worked instance. Take the concrete formula It has clauses, so the construction builds vertices in two triangles and sets . The conflict edges join (clause 1) to (clause 2), and (clause 1) to (clause 2) — the two variable/negation collisions across the triangles. Now find a size- independent set: pick from the first triangle and from the second. These two are not joined by a conflict edge (the conflicts are and , neither of which is the pair ), and they sit in different triangles, so the two vertices are non-adjacent — an independent set of size . Reading the chosen literals back as an assignment gives , (and free): the first clause is satisfied by , the second by , so is satisfiable. The reduction turned a logic question into a graph question and back, exactly as the correctness proof requires.

Choosing — one vertex per triangle, no conflict edge between them — is a size- independent set, i.e. a satisfying assignment.
three_sat_to_independent_set.pypython
from itertools import combinations
from typing import NamedTuple, Optional

from graph import Graph

from sat import Assignment, Formula, Literal

# A vertex label names the clause index, the literal's position within that
# clause, and the literal itself. Including the position keeps the three corners
# of every triangle distinct even when a clause repeats a literal (e.g.
# x ∨ x ∨ x), so each clause always contributes exactly three vertices.
VertexLabel = tuple[int, int, Literal]

class Reduction(NamedTuple):
  """
    The output of the reduction: the constructed graph, the target size k,\n
    and the source formula kept for decoding a witness back to an assignment.\n
  """
  graph: Graph[VertexLabel]
  target_size: int
  formula: Formula

def reduce_three_sat_to_independent_set(formula: Formula) -> Reduction:
  """
    Build the Independent-Set instance for a 3-CNF `formula`. Returns the\n
    graph, the target k = m (number of clauses), and the formula. Polynomial:\n
    3m vertices and O(m²) edges.\n
  """
  graph: Graph[VertexLabel] = Graph(directed=False)

  # one vertex per literal occurrence; remember each clause's three labels.
  clause_labels: list[list[VertexLabel]] = []
  for clause_index, clause in enumerate(formula.clauses):
    labels: list[VertexLabel] = [
      (clause_index, position, literal)
      for position, literal in enumerate(clause.literals)
    ]
    for label in labels:
      graph.add_vertex(label)
    clause_labels.append(labels)

  # triangle edges: every pair of vertices inside the same clause.
  for labels in clause_labels:
    for first_label, second_label in combinations(labels, 2):
      graph.add_edge(first_label, second_label)

  # conflict edges: pair up vertices from two different clauses.
  for first_clause, second_clause in combinations(clause_labels, 2):
    for first_label in first_clause:
      for second_label in second_clause:
        # join contradictory literals x and ¬x on the same variable.
        first_literal, second_literal = first_label[2], second_label[2]
        if (
          first_literal.variable == second_literal.variable
          and first_literal.negated != second_literal.negated
        ):
          graph.add_edge(first_label, second_label)

  return Reduction(graph, len(formula.clauses), formula)

def is_independent_set(
  graph: Graph[VertexLabel],
  chosen: set[VertexLabel],
) -> bool:
  """
    Whether `chosen` is an independent set: no edge joins two of its members.\n
  """
  # reject if any edge has both endpoints inside the chosen set.
  for edge in graph.edges():
    if edge.source.label in chosen and edge.target.label in chosen:
      return False

  return True

def maximum_independent_set(
  graph: Graph[VertexLabel],
) -> set[VertexLabel]:
  """
    A largest independent set, found by brute force over all vertex subsets.\n
    Exponential — Independent-Set is itself NP-complete — and used here only\n
    to exercise the reduction on small graphs.\n
  """
  labels: list[VertexLabel] = [vertex.label for vertex in graph.vertices]

  # scan subsets largest-first; the first independent one is a maximum set.
  for size in range(len(labels), -1, -1):
    for candidate in combinations(labels, size):
      chosen: set[VertexLabel] = set(candidate)
      if is_independent_set(graph, chosen):
        return chosen

  # unreachable: the empty set is always independent and is tried at size 0.
  return set()

def assignment_from_independent_set(
  reduction: Reduction,
  chosen: set[VertexLabel],
) -> Assignment:
  """
    Decode a size-k independent set back into a satisfying assignment: set\n
    each chosen literal true, then fill any untouched variable arbitrarily.\n
    Assumes `chosen` is a valid witness (one consistent literal per clause).\n
  """
  # each chosen literal pins its variable to whatever makes that literal true.
  assignment: Assignment = {}
  for _clause_index, _position, literal in chosen:
    assignment[literal.variable] = not literal.negated

  # fill any variable the witness never touched with an arbitrary value.
  for variable in reduction.formula.variables:
    assignment.setdefault(variable, False)

  return assignment

def solve_via_independent_set(formula: Formula) -> Optional[Assignment]:
  """
    Decide 3-SAT through the reduction: build the graph, find a maximum\n
    independent set, and accept iff it reaches size k = m, decoding it into\n
    a satisfying assignment. Returns None when no such set exists.\n
  """
  # build the graph and find its largest independent set.
  reduction: Reduction = reduce_three_sat_to_independent_set(formula)
  chosen: set[VertexLabel] = maximum_independent_set(reduction.graph)

  # accept iff the set reaches size k = m, then decode it into an assignment.
  if len(chosen) < reduction.target_size:
    return None

  return assignment_from_independent_set(reduction, chosen)
graph.pypython
from collections.abc import Hashable, Iterator
from typing import Generic, Optional, TypeVar


Label = TypeVar("Label", bound=Hashable)


class Edge(Generic[Label]):
  """
    A directed connection from `source` to `target`, carrying a weight.\n
  """

  def __init__(
    self,
    source: Vertex[Label],
    target: Vertex[Label],
    weight: float = 1.0,
  ) -> None:
    self.source: Vertex[Label] = source
    self.target: Vertex[Label] = target
    self.weight: float = weight

  def __repr__(self) -> str:
    return f"Edge({self.source.label!r} -> {self.target.label!r}, w={self.weight})"


class Vertex(Generic[Label]):
  """
    A graph vertex: a label plus the list of edges leaving it.\n
  """

  def __init__(self, label: Label) -> None:
    self.label: Label = label
    self.outgoing: list[Edge[Label]] = []

  def neighbors(self) -> list[Vertex[Label]]:
    """
      The vertices reachable from this one by a single edge.\n
    """
    return [edge.target for edge in self.outgoing]

  def edge_to(self, label: Label) -> Optional[Edge[Label]]:
    """
      The outgoing edge to the vertex with `label`, or None.\n
    """
    for edge in self.outgoing:
      if edge.target.label == label:
        return edge
    return None

  def __repr__(self) -> str:
    return f"Vertex({self.label!r})"


class Graph(Generic[Label]):
  """
    A graph of Vertex objects linked by Edge objects.\n
    Pass `directed=True` for a digraph; otherwise each `add_edge` inserts\n
    the reverse edge too.\n
  """

  def __init__(self, directed: bool = False) -> None:
    self.directed: bool = directed
    self._vertices: dict[Label, Vertex[Label]] = {}

  def add_vertex(self, label: Label) -> Vertex[Label]:
    """
      Return the vertex for `label`, creating it if it is absent.\n
    """
    # reuse the existing vertex, or mint and register a fresh one.
    vertex = self._vertices.get(label)
    if vertex is None:
      vertex = Vertex(label)
      self._vertices[label] = vertex
    return vertex

  def add_edge(
    self,
    source_label: Label,
    target_label: Label,
    weight: float = 1.0,
  ) -> None:
    """
      Connect two labels (creating either vertex as needed).\n
      Adds the reverse edge as well when the graph is undirected.\n
    """
    source = self.add_vertex(source_label)
    target = self.add_vertex(target_label)

    # link source to target, and mirror it back when undirected.
    source.outgoing.append(Edge(source, target, weight))
    if not self.directed:
      target.outgoing.append(Edge(target, source, weight))

  def vertex(self, label: Label) -> Vertex[Label]:
    """
      The vertex carrying `label` (raises KeyError if absent).\n
    """
    return self._vertices[label]

  @property
  def vertices(self) -> list[Vertex[Label]]:
    """
      Every vertex, in insertion order.\n
    """
    return list(self._vertices.values())

  @property
  def labels(self) -> list[Label]:
    """
      Every vertex label, in insertion order.\n
    """
    return list(self._vertices)

  def edges(self) -> Iterator[Edge[Label]]:
    """
      Each edge once — an undirected edge is yielded a single time.\n
    """
    # track undirected endpoint pairs so each is emitted only once.
    seen: set[frozenset[Label]] = set()

    for vertex in self._vertices.values():
      for edge in vertex.outgoing:
        # skip an undirected edge already yielded from the other endpoint.
        if not self.directed:
          endpoints = frozenset((edge.source.label, edge.target.label))
          if endpoints in seen:
            continue
          seen.add(endpoints)

        yield edge

  def __contains__(self, label: Label) -> bool:
    return label in self._vertices

  def __iter__(self) -> Iterator[Vertex[Label]]:
    return iter(self._vertices.values())

  def __len__(self) -> int:
    return len(self._vertices)

The recipe: proving a new problem NP-complete

Once a stockpile of -complete problems exists, classifying a new problem follows a fixed procedure. To prove is -complete, carry out four steps.

  1. Show . Describe a polynomial-size certificate for yes-instances and argue it can be checked in polynomial time. This is usually the easy step, but skipping it is a real error: an -hard problem outside is hard but not complete.
  2. Choose a known -complete problem to reduce from. Pick one whose structure resembles 3-SAT for logical or gadget-style constraints, or for graph selection, or for numeric targets, for routing.
  3. Give a polynomial-time reduction . Construct, from an arbitrary instance of , an instance of . This is the creative step of the proof. Mind the direction: you transform 's instance into 's, so that solving would solve . Reducing the wrong way () proves nothing about 's hardness.
  4. Prove the reduction correct. Establish the if and only if, which is exactly the completeness and soundness of the map. Completeness: a yes-instance of maps to a yes-instance of (no true case lost). Soundness: conversely every yes-instance of comes only from a yes-instance of (no false yes invented). Both directions are mandatory; dropping completeness lets false negatives slip through, dropping soundness lets false positives.

Steps 1 and 2 are bookkeeping; steps 3 and 4 are the substance.5 The worked reduction above is this recipe applied with and : the triangles-and-conflicts gadget is step 3, and the two-direction argument is step 4.

The tractability boundary and what hard costs

The reduction web draws a line, and the sharpest way to see where it falls is a pair of problems that look nearly identical yet land on opposite sides. 2-SAT, the restriction of SAT to two literals per clause, is solvable in linear time: build the implication graph (each clause becomes and ), and the formula is satisfiable exactly when no variable and its negation share a strongly connected component — one pass of the SCC algorithm, worked out in full in the 2-SAT lesson. Add one literal per clause and 3-SAT becomes -complete. The jump from to is the whole story, and it is not an accident: Schaefer's dichotomy theorem (1978) proves that every boolean constraint-satisfaction problem is either in or -complete, with nothing in between, and it names the exact conditions that put a problem in .6

Trace the linear-time test on . Each clause contributes both implications and , giving six directed edges: from the first clause, and ; from the second, and ; from the third, and . Every path in this graph forces truth values: following the edges out of , we reach then , and no path ever leads from a literal to its own negation, so no variable shares an SCC with its complement. The formula is satisfiable — indeed works. Had we instead added the clauses , the graph would route , collapsing and into one SCC and certifying unsatisfiability — all detected by a single SCC pass, never any search.

A knife-edge boundary: 2-SAT is linear-time, 3-SAT is NP-complete; Schaefer's theorem says CSPs are always one or the other, never in between.

Two deeper results sharpen how hard. The Exponential Time Hypothesis (ETH), conjectured by Impagliazzo and Paturi, says 3-SAT cannot be solved in time — not merely that it is not polynomial, but that even sub-exponential time is out of reach.7 ETH is now the standard tool for proving that specific problems need or time, giving fine-grained lower bounds far below the crude -versus- divide. And the PCP theorem (Arora–Safra and Arora–Lund–Motwani–Sudan–Szegedy, 1998) recharacterizes so strongly that it makes approximation itself hard: for many problems, even finding a solution within some constant factor of optimal is -complete.8 That result leads to the approximation lesson, where we ask not can we solve it exactly but how close can we provably get.

Takeaways

  • is -hard if every problem in reduces to it (a lower bound); it is -complete if it is also in, hardest among the efficiently-checkable problems.
  • All -complete problems share one fate: a polynomial-time algorithm for any of them would prove and solve them all.
  • Cook–Levin anchors the theory: SAT is -complete, proved directly by encoding any verifier's computation as a boolean formula. From it, 3-SAT and a vast reduction web follow by transitivity.
  • The reduction (a triangle per clause plus conflict edges, with ) is the model gadget reduction.
  • To prove a new problem -complete: (1) show membership in , (2) pick a known -complete , (3) reduce in polynomial time, (4) prove the equivalence both ways, always reducing from the hard problem.

Footnotes

  1. CLRS, Ch. 34 — NP-Completeness (§34.1): the definitions of -hard (a lower bound) and -complete (hard and in ).
  2. CLRS, Ch. 34 — NP-Completeness (§34.3): the Cook–Levin theorem that SAT is -complete, proved by encoding a verifier's computation as a boolean formula.
  3. Skiena, §11 — NP-Completeness: Karp's 1972 reductions and the web of -complete problems growing from satisfiability.
  4. Erickson, Ch. 12 — NP-Hardness: the gadget reduction (clause triangles plus conflict edges, ).
  5. CLRS, Ch. 34 — NP-Completeness (§34.4): the standard four-step recipe for proving a new problem -complete.
  6. Thomas J. Schaefer, The Complexity of Satisfiability Problems, STOC 1978 — every boolean CSP is either in or -complete, with the exact tractable cases characterized (2-SAT, Horn-SAT, affine, and duals).
  7. Russell Impagliazzo and Ramamohan Paturi, On the Complexity of -SAT, Journal of Computer and System Sciences 62(2), 2001 — the Exponential Time Hypothesis, that 3-SAT has no algorithm.
  8. Sanjeev Arora and Shmuel Safra, Probabilistic Checking of Proofs, and Arora, Lund, Motwani, Sudan, Szegedy, Proof Verification and the Hardness of Approximation Problems, Journal of the ACM, 1998 — the PCP theorem and inapproximability.
Practice

╌╌ END ╌╌