Mathematical Algorithms/Sieves & Factorization

Lesson 10.33,443 words

Sieves & Factorization

The previous lesson tested one number for primality; here we ask for all primes up to nn at once. The sieve of Eratosthenes cross-cuts composites in O(nloglogn)O(n\log\log n), and a linear sieve does it in O(n)O(n) while recording each number's smallest prime factor, which then factors any xnx \le n in O(logx)O(\log x).

╌╌╌╌

The previous lesson handed us a fast test for whether a single number is prime. Many problems instead need the primes en masse: every prime below , or the factorization of each of many queries, and testing each number independently wastes the structure shared across them. A sieve inverts the computation: rather than test one number at a time, it strikes out the multiples of each prime, eliminating the composites collectively. The result is a precomputed table over that answers is prime? in and, with one more field, factors any in .

The sieve of Eratosthenes

The idea is ancient and simple. Write the integers . The smallest unmarked number, , is prime; cross out all of its multiples . The next still-unmarked number, , is prime; cross out . Repeat. Whenever we reach an unmarked number, no smaller prime struck it, so it has no smaller divisor, hence it is prime, and we strike its multiples in turn. When we are done, the unmarked numbers are exactly the primes.

In the grid below, is laid out ten per row: composites are shaded out (grey), is left blank, and the survivors — the primes — are highlighted.

The sieve over : composites shaded out, blank, the 25 surviving primes highlighted.

Two optimizations make the sieve fast and are worth stating precisely.

Algorithm:Sieve(n)\textsc{Sieve}(n) — mark composites, return the prime indicator array
  1. 1
    P[0..n]trueP[0..n] \gets \text{true}; P[0]falseP[0] \gets \text{false}; P[1]falseP[1] \gets \text{false}
  2. 2
    for c2c \gets 2 to n\lfloor\sqrt{n}\rfloor do
  3. 3
    if P[c]P[c] then
    if candidate is prime
  4. 4
    fc2f \gets c^2
    skip factors of smaller primes
  5. 5
    while fnf \le n do
  6. 6
    P[f]falseP[f] \gets \text{false}
    sieve out the factor
  7. 7
    ff+cf \gets f + c
    stride by prime
  8. 8
    return PP
sieve_of_eratosthenes.pypython
def prime_sieve(limit: int) -> list[bool]:
  """
    The prime-indicator array over `0..limit`: `is_prime[k]` is True exactly\n
    when k is prime. Indices 0 and 1 are never prime.\n
  """
  if limit < 0:
    raise ValueError("limit must be non-negative")

  # 0 and 1 are never prime; index 0 always exists here, 1 only when limit >= 1.
  is_prime: list[bool] = [True for _ in range(limit + 1)]
  is_prime[0] = False
  if limit >= 1:
    is_prime[1] = False

  candidate: int = 2
  while candidate * candidate <= limit:
    if is_prime[candidate]:

      # smaller multiples of `candidate` already fell to smaller primes,
      # so the first new composite is candidate^2; stride by `candidate`.
      multiple: int = candidate * candidate
      while multiple <= limit:
        is_prime[multiple] = False
        multiple += candidate
    candidate += 1
  return is_prime

def primes_up_to(limit: int) -> list[int]:
  """
    The list of primes `<= limit`, ascending.\n
  """
  is_prime: list[bool] = prime_sieve(limit)
  return [number for number in range(2, limit + 1) if is_prime[number]]

def count_primes(limit: int) -> int:
  """
    The number of primes strictly less than `limit` (the `Count Primes`\n
    convention: primes in `[2, limit)`).\n
  """
  if limit <= 2:
    return 0
  return sum(prime_sieve(limit - 1))

Why it is

The work is dominated by the inner loop, which for each prime strikes multiples. Summing over primes,

The naive worry is that behaves like the harmonic series , which would give . But the sum runs over primes only, which are sparse, and a classical theorem of Mertens says the reciprocal sum of primes grows far more slowly:

Hence the total work is .1 The factor is, for all practical , a small constant (under for ), so the sieve is effectively linear. Space is for the array (one bit per number if packed). Starting at rather than does not change the asymptotics but roughly halves the constant.

The linear sieve and smallest prime factors

The Eratosthenes sieve strikes some composites more than once: is hit by (as ) and by (as ). That redundancy accounts for the factor. A linear sieve removes it by guaranteeing that every composite is crossed out exactly once, by its smallest prime factor (SPF). As a bonus it records that smallest prime factor, which is the key to fast factorization below.

Maintain a growing list of primes found so far. For each from to , and for each known prime in increasing order, mark the product as composite with smallest prime factor . The subtle line is the termination: as soon as divides , we break.

Algorithm:LinearSieve(n)\textsc{LinearSieve}(n) — compute spf[x]\text{spf}[x] for every xnx \le n
  1. 1
    spf[0..n]0\text{spf}[0..n] \gets 0; primes[]\text{primes} \gets [\,]
  2. 2
    for i2i \gets 2 to nn do
  3. 3
    if spf[i]=0\text{spf}[i] = 0 then
    ii is prime
  4. 4
    spf[i]i\text{spf}[i] \gets i; append ii to primes\text{primes}
  5. 5
    for each pp in primes\text{primes} do
  6. 6
    if p>spf[i]p > \text{spf}[i] or ip>ni \cdot p > n then break
  7. 7
    spf[ip]p\text{spf}[i \cdot p] \gets p
  8. 8
    return spf,primes\text{spf}, \text{primes}
linear_sieve.pypython
from typing import NamedTuple

class SieveResult(NamedTuple):
  """
    The output of the linear sieve: the smallest-prime-factor table over\n
    `0..limit` (0 for 0 and 1, the prime itself for a prime), and the list\n
    of primes found in ascending order.\n
  """
  smallest_prime_factor: list[int]
  primes: list[int]

def linear_sieve(limit: int) -> SieveResult:
  """
    Compute `smallest_prime_factor[x]` for every `x <= limit` in linear time,\n
    alongside the ascending list of primes up to `limit`.\n
  """
  if limit < 0:
    raise ValueError("limit must be non-negative")

  smallest_prime_factor: list[int] = [0 for _ in range(limit + 1)]
  primes: list[int] = []

  for number in range(2, limit + 1):
    if smallest_prime_factor[number] == 0:

      # nothing smaller struck it, so `number` is prime.
      smallest_prime_factor[number] = number
      primes.append(number)

    for prime in primes:

      # only multiples with `prime` as their smallest factor belong here;
      # stop once `prime` exceeds spf[number] or the product overflows.
      if prime > smallest_prime_factor[number] or number * prime > limit:
        break
      smallest_prime_factor[number * prime] = prime

  return SieveResult(smallest_prime_factor, primes)

Each composite is written once, when and , so the total number of marking operations equals the number of composites: the running time is , with space.2 The classic Count Primes problem is solved by either sieve; the linear sieve is the right tool whenever you also need per-number factor data downstream.

The payoff is the table itself: every prime maps to itself, every composite to its smallest prime factor, each entry written exactly once. The composite , for instance, is struck only when , never again by the larger prime .

Linear sieve: each composite carries its smallest prime factor , written once

Factorization

With a precomputed SPF table:

Given the array from the linear sieve, any factors by peeling off its smallest prime factor and dividing it out, repeatedly, until remains.

Algorithm:Factor(x)\textsc{Factor}(x) — full prime factorization of xnx \le n via spf\text{spf}
  1. 1
    F{}F \gets \{\}
    map prime \to exponent
  2. 2
    while x>1x > 1 do
  3. 3
    pspf[x]p \gets \text{spf}[x]
  4. 4
    while xmodp=0x \bmod p = 0 do
  5. 5
    xx/px \gets x / p; F[p]F[p]+1F[p] \gets F[p] + 1
  6. 6
    return FF
spf_factorization.pypython
def factorize_with_spf(value: int, smallest_prime_factor: list[int]) -> dict[int, int]:
  """
    The prime factorization of `value` as a map from prime to exponent,\n
    using a precomputed `smallest_prime_factor` table that covers `value`.\n
    `value` must satisfy `2 <= value < len(smallest_prime_factor)`.\n
  """
  if value < 2:
    return {}
  if value >= len(smallest_prime_factor):
    raise ValueError("value exceeds the smallest-prime-factor table")

  factorization: dict[int, int] = {}
  remaining: int = value

  # read off spf[remaining], then strip every copy before moving on.
  while remaining > 1:
    prime: int = smallest_prime_factor[remaining]
    while remaining % prime == 0:
      remaining //= prime
      factorization[prime] = factorization.get(prime, 0) + 1
  return factorization

def distinct_prime_factors(value: int, smallest_prime_factor: list[int]) -> list[int]:
  """
    The distinct prime factors of `value`, ascending, via the SPF table.\n
  """
  return sorted(factorize_with_spf(value, smallest_prime_factor))

Each division by a prime at least halves , so the outer process runs at most times: factorization is once the table is built. This is what makes problems like Distinct Prime Factors of Product of Array and Smallest Value After Replacing With Sum of Prime Factors tractable across many values: sieve once, then factor each query in logarithmic time.

Divide by until ; the chain of factors collects in the accent box

Without preprocessing: trial division and beyond

When is a one-off, or larger than any sieve we can afford, fall back to trial division: try each candidate divisor up to , dividing it out whenever it divides. The bound is the same observation as in the primality lesson: if with then , so the smallest nontrivial factor appears by ; any factor left after the loop is the final large prime.

Algorithm:TrialFactor(x)\textsc{TrialFactor}(x) — factor a single xx in O(x)O(\sqrt{x})
  1. 1
    F{}F \gets \{\}; d2d \gets 2
  2. 2
    while ddxd \cdot d \le x do
  3. 3
    while xmodd=0x \bmod d = 0 do
  4. 4
    xx/dx \gets x / d; F[d]F[d]+1F[d] \gets F[d] + 1
  5. 5
    dd+1d \gets d + 1
  6. 6
    if x>1x > 1 then F[x]F[x]+1F[x] \gets F[x] + 1
    leftover prime >x> \sqrt{x}
  7. 7
    return FF
trial_division.pypython
def trial_factor(value: int) -> dict[int, int]:
  """
    The prime factorization of `value >= 1` as a map from prime to exponent,\n
    by trial division up to sqrt(value). The empty map represents 1.\n
  """
  if value < 1:
    raise ValueError("value must be a positive integer")

  factorization: dict[int, int] = {}
  remaining: int = value

  divisor: int = 2
  while divisor * divisor <= remaining:

    # strip every copy of `divisor` before advancing.
    while remaining % divisor == 0:
      remaining //= divisor
      factorization[divisor] = factorization.get(divisor, 0) + 1
    divisor += 1

  # a leftover above 1 is a prime larger than sqrt(value).
  if remaining > 1:
    factorization[remaining] = factorization.get(remaining, 0) + 1
  return factorization

This costs . For genuinely large (say -bit and beyond), is too slow, and one reaches for Pollard's rho, a randomized factoring algorithm that finds a nontrivial factor in expected time via cycle-detection on a pseudorandom map, paired with the Miller–Rabin primality test to know when a factor is itself prime and recursion can stop.3

The name comes from the shape of the orbit. Iterating from a seed eventually repeats, so the trajectory runs down a tail and then loops a cycle — drawn out, it looks like the Greek letter . On the seed feeds a four-step cycle; because the cycle's residues modulo collide before they collide modulo , a difference shares the factor with , which then extracts.

Pollard's rho on with : the orbit of forms a tail into a cycle — the shape
pollard_rho.pypython
import random
from math import gcd

# deterministic Miller-Rabin witnesses covering all 64-bit integers.
_WITNESSES: tuple[int, ...] = (2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37)

def is_prime(number: int) -> bool:
  """
    Whether `number` is prime, by deterministic Miller-Rabin. The fixed\n
    witness set is exact for every `number < 3.3 * 10^24`.\n
  """
  if number < 2:
    return False

  # settle small primes directly and reject their larger multiples.
  for small_prime in _WITNESSES:
    if number == small_prime:
      return True
    if number % small_prime == 0:
      return False

  # write number - 1 = odd_part * 2^power_of_two.
  odd_part: int = number - 1
  power_of_two: int = 0
  while odd_part % 2 == 0:
    odd_part //= 2
    power_of_two += 1

  # each witness must reach -1 or already sit at +-1, else `number` is composite.
  for witness in _WITNESSES:
    residue: int = pow(witness, odd_part, number)
    if residue == 1 or residue == number - 1:
      continue

    # square up to `power_of_two - 1` times, hunting for -1.
    is_composite: bool = True
    for _ in range(power_of_two - 1):
      residue = (residue * residue) % number
      if residue == number - 1:
        is_composite = False
        break

    if is_composite:
      return False
  return True

def _pollard_rho_factor(number: int) -> int:
  """
    A single nontrivial factor of composite `number` via Pollard's rho,\n
    retrying with fresh parameters until the cycle yields a divisor.\n
  """
  if number % 2 == 0:
    return 2

  # retry with fresh c and start until a proper divisor falls out.
  while True:
    increment: int = random.randrange(1, number)
    slow: int = random.randrange(2, number)
    fast: int = slow
    divisor: int = 1

    # Floyd-style: advance `slow` once and `fast` twice per step, tracking
    # gcd of their difference with `number`.
    while divisor == 1:
      slow = (slow * slow + increment) % number
      fast = (fast * fast + increment) % number
      fast = (fast * fast + increment) % number
      divisor = gcd(abs(slow - fast), number)

    if divisor != number:
      return divisor

def factorize(number: int) -> dict[int, int]:
  """
    The prime factorization of `number >= 1` as a map from prime to exponent,\n
    recursively splitting composites with Pollard's rho until every part is\n
    prime by Miller-Rabin. The empty map represents 1.\n
  """
  if number < 1:
    raise ValueError("number must be a positive integer")

  factorization: dict[int, int] = {}

  def split(value: int) -> None:
    if value == 1:
      return

    # a prime part contributes one exponent and stops the recursion.
    if is_prime(value):
      factorization[value] = factorization.get(value, 0) + 1
      return

    # otherwise carve off a factor and recurse on both halves.
    factor: int = _pollard_rho_factor(value)
    split(factor)
    split(value // factor)

  split(number)
  return factorization

Worked example (rho splits ). Iterate from , running a slow pointer (one step) against a fast pointer (two steps) — Floyd's tortoise-and-hare cycle detector — and testing at each round. After one round the slow pointer is at and the fast pointer at , so we test , a nontrivial factor, so . The reason it works: modulo the hidden factor , the sequence collides after only a few steps (there are just residues), while modulo it has not yet repeated — so but , the gap that makes land on . The expected number of steps to a collision modulo a factor is by the birthday bound, giving the expected running time.

Multiplicative functions from the factorization

Once is in hand, a family of useful quantities are read straight off the exponents. Each is multiplicative, meaning its value on a product of coprimes is the product of its values, which is why each factors as a product over the distinct primes.

Number of divisors . A divisor of chooses, independently for each prime , an exponent between and — that is choices. Multiplying the independent counts,

For this is divisors. The product literally counts cells of a grid: the exponents of and index a block of divisors of , and the choice of the factor or stacks a second identical block behind it, so .

counts cells of an exponent grid: a block of , doubled by the factor or .

Four Divisors and Closest Divisors are direct applications: the former asks for numbers with , the latter searches divisor pairs near .

Sum of divisors . The divisors of are obtained by expanding ; each bracket is a geometric series, so

Euler's totient counts the integers in coprime to . By inclusion–exclusion over the distinct prime factors, removing the fraction of integers each prime divides, it reduces to a product:

For example .

multiplicative_functions.pypython
from collections.abc import Mapping

from trial_division import trial_factor

def _factorization(value: int, factors: Mapping[int, int] | None) -> Mapping[int, int]:
  """
    Use the supplied factorization of `value`, or compute one by trial\n
    division when none is given.\n
  """
  if factors is not None:
    return factors
  return trial_factor(value)

def divisor_count(value: int, factors: Mapping[int, int] | None = None) -> int:
  """
    tau(value): the number of positive divisors, prod (exponent + 1).\n
    Pass a precomputed `factors` map to skip refactoring.\n
  """
  if value < 1:
    raise ValueError("value must be a positive integer")

  # tau = prod (e_i + 1) over the exponents of distinct primes.
  count: int = 1
  for exponent in _factorization(value, factors).values():
    count *= exponent + 1
  return count

def divisor_sum(value: int, factors: Mapping[int, int] | None = None) -> int:
  """
    sigma(value): the sum of all positive divisors, by the geometric-series\n
    product prod (p^(e+1) - 1)/(p - 1).\n
  """
  if value < 1:
    raise ValueError("value must be a positive integer")

  # multiply the geometric-series term (p^(e+1) - 1)/(p - 1) per prime.
  total: int = 1
  for prime, exponent in _factorization(value, factors).items():
    total *= (prime ** (exponent + 1) - 1) // (prime - 1)
  return total

def euler_totient(value: int, factors: Mapping[int, int] | None = None) -> int:
  """
    phi(value): the count of integers in `[1, value]` coprime to `value`,\n
    via value * prod (1 - 1/p) = value * prod (p - 1)/p over distinct primes.\n
  """
  if value < 1:
    raise ValueError("value must be a positive integer")

  # apply the (p - 1)/p factor per distinct prime in exact integer math.
  result: int = value
  for prime in _factorization(value, factors):
    result -= result // prime
  return result

When is needed for every number up to , do not factor each one; sieve directly. Initialize , then for each prime sweep its multiples and apply the factor once, i.e. for each multiple of :

Algorithm:TotientSieve(n)\textsc{TotientSieve}(n) — compute φ(x)\varphi(x) for all xnx \le n
  1. 1
    for i0i \gets 0 to nn do φ[i]i\varphi[i] \gets i
  2. 2
    for p2p \gets 2 to nn do
  3. 3
    if φ[p]=p\varphi[p] = p then
    pp is prime
  4. 4
    mpm \gets p
  5. 5
    while mnm \le n do
  6. 6
    φ[m]φ[m]φ[m]/p\varphi[m] \gets \varphi[m] - \varphi[m] / p
    apply (11/p)(1-1/p)
  7. 7
    mm+pm \gets m + p
  8. 8
    return φ\varphi
totient_sieve.pypython
def totient_sieve(limit: int) -> list[int]:
  """
    The array `phi[0..limit]` of Euler totients. `phi[0] = 0`, `phi[1] = 1`,\n
    and `phi[x]` counts the integers in `[1, x]` coprime to x for x >= 1.\n
  """
  if limit < 0:
    raise ValueError("limit must be non-negative")

  totient: list[int] = list(range(limit + 1))
  for candidate in range(2, limit + 1):

    # an untouched entry equal to itself marks a prime; sweep its multiples.
    if totient[candidate] == candidate:
      multiple: int = candidate
      while multiple <= limit:
        totient[multiple] -= totient[multiple] // candidate
        multiple += candidate
  return totient

This runs in , the same harmonic-over-primes sum as the plain sieve, and gives every totient at once.4

Segmented sieving and counting the primes

Two extensions matter at scale: sieving beyond available memory, and counting primes without enumerating them.

Segmented sieving. The plain sieve needs an array of size , which fails when is, say, — no machine holds a trillion-bit array. The segmented sieve fixes this: compute the primes up to once (a small sieve), then process in cache-sized windows , marking each window with the multiples of every prime . Memory drops to while the total work stays , and because each window fits in cache the constant factor improves. This is how record prime enumerations (all primes below ) are actually run. To count primes in a range — the shape of Closest Prime Numbers in Range — sieve just that window against the small primes below .

Segmented sieve: small primes up to (left) mark each cache-sized window of in turn, so only memory is live.

How many primes are there? The sieve enumerates primes; the prime number theorem counts them: , so a random integer near is prime with probability about .5 This is what tells RSA key generation how many random candidates it must test before finding a 1024-bit prime (about on average). Counting without listing every prime — the Meissel–Lehmer method and its modern refinement by Lagarias, Miller, and Odlyzko — computes in roughly time and far less space, reaching values of well beyond what any sieve could enumerate.6

The linear sieve's lineage. The once-per-composite linear sieve is usually credited to Paul Pritchard's sublinear wheel sieves and to Gries and Misra (1978), who gave the SPF-recording form used here.7 Its main value is the smallest-prime-factor table it produces, not the marginal speedup over Eratosthenes: the table turns every subsequent factorization query into an table walk.

Takeaways

  • The sieve of Eratosthenes marks composites by striking each prime's multiples from with stride ; survivors are prime. The cost is by Mertens' theorem, space .
  • The linear sieve strikes each composite exactly once — by its smallest prime factor — running in while recording ; the break when is what enforces the once-only invariant.
  • With an SPF table, any factors in by repeatedly dividing by ; without preprocessing, trial division to costs , and Pollard's rho + Miller–Rabin handle large .
  • From the multiplicative functions follow: , , and Euler's totient .
  • over an entire range is itself sieved in ; never factor each number when a sieve will compute them all together.

Footnotes

  1. CLRS, Ch. 31 — Number-Theoretic Algorithms: divisibility, primes, and the cost of generating them; the sieve bound follows from .
  2. Skiena, § — Number Theory / Primes: the sieve of Eratosthenes and its linear refinement that records smallest prime factors for factorization.
  3. CLRS, Ch. 31 — Number-Theoretic Algorithms (§31.9): Pollard's rho heuristic for factoring large integers, with Miller–Rabin (§31.8) as the companion primality test.
  4. Erickson, Ch. — (number theory): multiplicative functions , , read off the prime factorization, and sieving over a range.
  5. The prime number theorem, (Hadamard and de la Vallée Poussin, 1896). See Skiena, § — Number Theory / Primes, for the algorithmic consequence for random-prime generation.
  6. J. C. Lagarias, V. S. Miller, A. M. Odlyzko, Computing : the Meissel–Lehmer method, Mathematics of Computation 44(170), 1985.
  7. D. Gries and J. Misra, A linear sieve algorithm for finding prime numbers, Communications of the ACM 21(12), 1978; P. Pritchard, A sublinear additive sieve for finding prime numbers, CACM 24(1), 1981.
Practice

╌╌ END ╌╌