Branch & Bound and Meet in the Middle
Plain backtracking prunes a search tree by feasibility; for optimization problems we can prune far more aggressively by value. Branch and bound keeps the best complete solution found so far and discards any partial solution whose optimistic bound cannot beat it.
╌╌╌╌
The previous lessons built backtracking: a depth-first walk over a tree of partial solutions that prunes a branch the moment it becomes infeasible, such as a queen attacking another or a graph coloring conflict. That kind of pruning asks a yes/no question: can this partial solution still be completed at all? For optimization problems, maximize this or minimize that, we can ask a sharper question: even in the best case, can completing this partial solution beat the best answer I already have? If not, the entire subtree is dead, feasible or not. Pruning by value rather than mere feasibility is the idea behind branch and bound, and it often cuts the running time by many orders of magnitude.
When even aggressive pruning is not enough, when the search tree is genuinely -shaped and is, say, , a second technique buys a square root of the running time outright. Meet in the middle splits the instance, enumerates each half independently, and stitches the halves back together with sorting and binary search. Both techniques attack the exponential running time, from different sides.
Branch and bound: pruning by value
Branch and bound is backtracking with two extra pieces of bookkeeping. The first is the incumbent: the value (and witness) of the best complete solution found anywhere in the search so far. The second is a bound computed at every node, an optimistic estimate of the best objective achievable by any completion of that partial solution. For a maximization problem the bound is an upper bound (no completion can do better than this); for minimization it is a lower bound.
A bound is sound exactly when this holds — when it never prunes a subtree that contains an optimum. An optimistic bound guarantees soundness, so branch and bound stays complete for the optimization problem: the optimal solution survives every cut and is eventually reported. A bound that could under-estimate (maximization) would be unsound, silently discarding the answer.
The method's effectiveness depends entirely on two design choices. A tighter bound prunes more nodes; a bound equal to the true optimum would prune everything but the answer. And a better search order, finding a strong incumbent early, raises sooner, which retroactively prunes more of the tree. The two interact: a good incumbent makes a mediocre bound effective.
The right subtree carries bound , so it is pruned without ever being expanded; the blue path is the active best completion that established the incumbent.
Worked example: 0/1 knapsack by branch and bound
We have items with values and weights and a capacity ; choose a subset of maximum total value with total weight . The decision tree is binary, take item or skip it, so it has leaves. To bound a node, we use the LP relaxation: relax the integrality constraint and allow a fraction of the next item. First order all items by value density, descending. At a node that has fixed a prefix of decisions, with accumulated value and remaining capacity , greedily fill with the not-yet-decided items in density order, taking the last one fractionally:
where are the remaining items that fit wholly and is the first item that overflows (filled fractionally). This fractional fill is the optimal solution to the relaxed problem, so it can only over-estimate the integral optimum, which is the optimism a sound bound requires.2
- 1incumbent value
- 2:
- 3if then returninfeasible
- 4if thennew incumbent
- 5if then return
- 6if then returnprune by value
- 7take item
- 8skip item
- 9
- 10:
- 11
- 12for to do
- 13if then
- 14else returnfractional fill
- 15return
Branching on take before skip tends to find a heavy, valuable incumbent early, which makes the bound bite sooner. Compare this against the dynamic-programming solution. The DP runs in time, pseudo-polynomial, because enters as a magnitude, not a bit-length. When is enormous (say weights are -digit numbers) the DP table is hopeless, yet if is moderate the branch-and-bound tree, heavily pruned, finishes quickly. The two methods are complementary: DP wins when is small; branch and bound wins when is huge but is moderate.
Branching take-first dives straight to the incumbent with value . The skip- subtree's optimistic LP bound is only (take whole, then of : ), which cannot beat , so the entire right half is discarded before a single completion is built.
Search order: depth-first vs best-first
The skeleton above is depth-first branch and bound: it recurses to a leaf fast, so it finds some complete solution, an incumbent, almost immediately, and it uses only stack. The cost is that the first incumbent may be poor, weakening early pruning. The alternative is best-first search: keep a priority queue of live nodes keyed by their bound, and always expand the node with the most promising bound. Best-first tends to drive toward the optimum with the fewest expansions and, for many problems, expands the optimal node first, but it can hold an exponential frontier of live nodes in the queue, so its memory is the liability. The practical compromise is to seed the incumbent with a quick greedy solution, then run depth-first with strong bounds: cheap memory, and a floor high enough that the bound prunes hard from the start.
Depth-first (left) follows one accented path to a leaf, banking an incumbent fast while holding only the current root-to-node stack. Best-first (right) instead pops the live node of highest bound () from a priority queue, steering toward the optimum in fewer expansions at the cost of keeping the whole frontier in memory.
Meet in the middle
Some problems resist pruning entirely: the bound is weak, the structure symmetric, every branch genuinely live. If the instance is a subset problem over items and , meet in the middle sidesteps pruning and attacks the exponent directly. Split the items into two halves and of size . Enumerate all subset sums of into a list , and likewise all subset sums of into . Every subset of the whole is one choice from paired with one from , so the full answer is recovered by combining one element of with one of , but we perform that combination efficiently, not by trying all pairs.
For the canonical task, find a subset whose sum is closest to a target (this is the minimum-partition-difference problem with ), sort , then for each binary-search for the value nearest . Each query is , so the whole combine is .
- 1split items into halves (size ) and
- 2all subset sums of
- 3all subset sums of
- 4sort
- 5
- 6for each in do
- 7complement from
- 8value in nearest (binary search: floor and ceiling of )
- 9
- 10return
The enumeration is per half, the sort is , and the combine is , so the whole algorithm is , a quadratic improvement over the brute force. Concretely, is about (out of reach) while is about (instant). The technique is exact, no approximation and no pruning luck, and it is the intended solution to every Hard subset problem in this lesson's practice set.
For each sum on the top we binary-search the sorted bottom list for ; the blue pair hits the target exactly. To see the whole method end to end, take the eight numbers and target . Split into and . Enumerating every subset sum:
- (the sums of ).
- (sorted).
Now walk : for we seek in , and is present, so hits the target exactly — the subset from plus from . The combine did binary searches of a -element list instead of scanning all subsets, and it scales: at it is searches rather than subsets.
The same split-and-recombine idea is the graph analog bidirectional search: to find a shortest path, run BFS forward from the source and backward from the target simultaneously and stop when the two frontiers meet, exploring nodes instead of .
When to reach for which
The three pruning disciplines line up neatly along one axis: what justifies discarding a branch.
- Backtracking prunes by feasibility: a partial solution that violates a constraint can never be completed, so cut it.
- Branch and bound prunes by value: a partial solution whose optimistic bound cannot beat the incumbent is pointless to complete, so cut it.
- Meet in the middle prunes nothing; it instead trades exponential time for the square root of it, , paying with memory to store the enumerated half.
Branch and bound works best when a cheap, tight optimistic bound exists (knapsack's LP fill, a TSP node's spanning-tree lower bound). Meet in the middle works best when no such bound exists but is small enough that is affordable. Both are exact; neither changes the worst-case exponential complexity; both routinely turn an infeasible instance into a feasible one.
How the world actually solves hard optimization
Branch and bound solves the large integer programs behind logistics, scheduling, and network design every day.
Branch and cut. Modern integer-programming solvers — CPLEX, Gurobi, the open-source SCIP — run branch and cut: branch and bound whose LP-relaxation bound (exactly the knapsack bound of this lesson, generalized) is tightened at each node by adding cutting planes, linear inequalities valid for all integer solutions but violated by the current fractional optimum.3 Gomory's cuts (1958) and the Padberg–Rinaldi cuts for the traveling salesman turned instances once deemed hopeless into routine ones; a combination of branch and cut solved a TSP over all cities of a VLSI application to proven optimality.4 The engineering lesson matches this lesson's theory: a tighter bound (better cuts) and a stronger incumbent (better heuristics) each prune more of the tree.
A*: the same idea. Best-first branch and bound is, essentially,
the A* search algorithm (Hart, Nilsson & Raphael, 1968): expand the live node
minimizing , where is the cost so far and is an admissible
heuristic — a bound that never overestimates the remaining cost.5
Admissibility supplies the optimistic-bound condition that makes pruning sound
here; A* is branch and bound with the objective shortest path
and the bound
heuristic-to-goal.
Meet in the middle in cryptanalysis. The meet-in-the-middle split is older than its algorithmic-puzzle use: Diffie and Hellman (1977) introduced it to attack double encryption, showing that encrypting twice with two keys gives far less than double the security because an attacker enumerates each key-half and matches in the middle — the same collapse, applied to key search.6 The subset trick and the cryptographic attack are the same idea.
Takeaways
- Branch and bound is backtracking for optimization: maintain an incumbent (best complete solution so far) and a bound (optimistic estimate per node), and prune any node whose bound cannot beat the incumbent.
- The method's power is all in the bound tightness and search order: a tighter bound and an earlier strong incumbent each prune more of the tree.
- For 0/1 knapsack, order by density and bound by the LP-relaxation fractional fill; branch and bound beats the DP when is huge but is moderate.
- Depth-first branch and bound finds an incumbent fast with memory; best-first (priority queue on bound) targets the optimum with fewer expansions but can hold an exponential frontier.
- Meet in the middle enumerates each of two halves ( subset sums) and recombines by sorting + binary search, giving , exact search up to ; bidirectional search is the graph analog.
- One axis: backtracking prunes by feasibility, branch and bound by value, meet in the middle trades exponential time for of it at the cost of memory.
Footnotes
- Erickson, Ch. — Backtracking: branch and bound as backtracking augmented with a value bound; the optimism of the bound is what makes pruning sound. ↩
- Skiena, § — Combinatorial Search / Heuristics: pruning a combinatorial search by bounding the best achievable completion, illustrated on knapsack-style problems. ↩
- Padberg, M. & Rinaldi, G. (1991),
A branch-and-cut algorithm for the resolution of large-scale symmetric traveling salesman problems,
SIAM Review 33(1), 60–100 — branch and bound tightened by cutting planes, the template of modern IP solvers; cutting planes trace to Gomory, R. E. (1958). ↩ - Applegate, D. L., Bixby, R. E., Chvátal, V. & Cook, W. J. (2006), The Traveling Salesman Problem: A Computational Study, Princeton University Press — solving TSP instances with tens of thousands of cities to proven optimality by branch and cut. ↩
- Hart, P. E., Nilsson, N. J. & Raphael, B. (1968),
A formal basis for the heuristic determination of minimum cost paths,
IEEE Transactions on Systems Science and Cybernetics 4(2), 100–107 — A* as best-first search with an admissible (optimistic) heuristic, i.e. branch and bound for shortest paths. ↩ - Diffie, W. & Hellman, M. E. (1977),
Exhaustive cryptanalysis of the NBS data encryption standard,
Computer 10(6), 74–84 — the meet-in-the-middle attack on double encryption, the same split used for subset problems. ↩
╌╌ END ╌╌