---
title: Text Summarization
module: Applications
moduleNumber: 7
lessonNumber: 7
order: 707
summary: >
  Summarization compresses a document to its essential meaning, by either
  selecting sentences to keep (extractive) or writing new ones (abstractive).
  This part fixes the task and its flavors — single vs. multi-document, generic
  vs. query-focused, extractive vs. abstractive — then works through extractive
  summarization in full: scoring by position and centrality, the TextRank/LexRank
  graph algorithm run as PageRank over a sentence-similarity graph with a worked
  iteration, and supervised sentence selection.
topics: [Applications]
sources:
  - book: Jurafsky
    ref: "Ch. 9 — Deep Learning Architectures for Sequence Processing; §9.9 Contextual Generation and Summarization (extractive)"
  - book: Jurafsky
    ref: "§9.9.1 Applying Transformers to other NLP tasks (supervised sentence selection)"
---

**Text summarization** is the task of taking a full-length document and producing
a shorter text that keeps its most important information. Stated as a mapping it
is the same shape as translation — a source $\mathbf{x} = x_1, \ldots, x_m$ goes
in and a shorter target $\mathbf{y} = y_1, \ldots, y_n$ with $n \ll m$ comes out —
but where translation preserves meaning across a change of _language_,
summarization preserves the _essential_ meaning across a change of _length_. The
modern method is context-based [autoregressive
generation](/natural-language-processing/transformers/large-language-models): prime a
language model with the document and let it generate the summary, exactly the
[sequence-to-sequence](/natural-language-processing/applications/machine-translation)
recipe translation established.[^jm-ctxgen] This lesson covers what the task is,
the two families of methods that solve it, and how to tell whether a summary is
any good.

## The task and its flavors

"Summarize this" is really a family of tasks that differ along a few axes.
Identifying which variant applies comes first, because it determines both the
method and the evaluation.

$$
% caption: The three axes of a summarization task: how many documents come in
% (single vs. multi), what shapes the summary (generic vs. query-focused), and
% how the summary is produced (extractive vs. abstractive). This lesson turns on
% the last axis.
\begin{tikzpicture}[>=stealth, font=\footnotesize,
  ax/.style={draw, minimum width=30mm, minimum height=10mm, align=center, font=\scriptsize},
  opt/.style={draw, minimum width=30mm, minimum height=10mm, align=center, font=\scriptsize}]
  \definecolor{acc}{HTML}{2348F2}
  \node[ax] (a1) at (0,3.4)   {input};
  \node[ax] (a2) at (0,0)     {focus};
  \node[ax, draw=acc, text=acc] (a3) at (0,-3.4) {method};
  \node[opt] (i1) at (5.4,4.2)  {single-document};
  \node[opt] (i2) at (5.4,2.6)  {multi-document};
  \node[opt] (f1) at (5.4,0.8)  {generic};
  \node[opt] (f2) at (5.4,-0.8) {query-focused};
  \node[opt, draw=acc, text=acc] (m1) at (5.4,-2.6) {extractive};
  \node[opt, draw=acc, text=acc] (m2) at (5.4,-4.2) {abstractive};
  \draw[->, black] (a1) -- (i1);
  \draw[->, black] (a1) -- (i2);
  \draw[->, black] (a2) -- (f1);
  \draw[->, black] (a2) -- (f2);
  \draw[->, acc, thick] (a3) -- (m1);
  \draw[->, acc, thick] (a3) -- (m2);
\end{tikzpicture}
$$

**Single- vs. multi-document.** A **single-document** summarizer condenses one
article. A **multi-document** summarizer fuses many sources on the same topic —
several news stories about one event, a cluster of reviews of one product — into a
single summary, which adds the problem of detecting and removing redundancy across
documents.[^jm-ctxgen]

**Generic vs. query-focused.** A **generic** summary answers "what is this
document about?" with no further guidance. A **query-focused** (or
_topic-focused_) summary answers "what does this document say about $q$?" for a
user query $q$, keeping only the material relevant to $q$. Query-focused
summarization is essentially long-form
[question answering](/natural-language-processing/applications/question-answering):
retrieve or condition on the query, then generate the answer from the source.

**Extractive vs. abstractive.** This is the axis that shapes the algorithm.

> **Definition (Extractive summarization).** Build the summary by **selecting**
> and concatenating spans — usually whole sentences — copied verbatim from the
> source document. The output is a subset of the input; nothing is rewritten.

> **Definition (Abstractive summarization).** **Generate** new text that conveys
> the source's meaning, free to paraphrase, compress two sentences into one, and
> use words absent from the source. The output need not appear anywhere in the
> input.

$$
% caption: Extractive vs. abstractive. Extractive selects a subset of the source
% sentences and copies them verbatim (blue, kept; faded, dropped). Abstractive
% writes a fresh sentence that fuses and rephrases the content, using words the
% source never used.
\begin{tikzpicture}[>=stealth, font=\footnotesize,
  keep/.style={draw=acc, text=acc, fill=acc!8, minimum width=44mm, minimum height=6mm, inner sep=2pt, font=\scriptsize},
  drop/.style={draw, text=black, minimum width=44mm, minimum height=6mm, inner sep=2pt, font=\scriptsize},
  gen/.style={draw=acc, thick, fill=acc!5, minimum width=44mm, minimum height=6mm, inner sep=2pt, font=\scriptsize}]
  \definecolor{acc}{HTML}{2348F2}
  % source document
  \node[font=\scriptsize, anchor=south] at (0,2.55) {source document};
  \node[keep] (s1) at (0,2.0)  {S1: Waring ships Boston snow for 89 dollars.};
  \node[drop] (s2) at (0,1.3)  {S2: Each box holds 10 to 15 snowballs.};
  \node[keep] (s3) at (0,0.6)  {S3: He will not ship to the northeast.};
  \node[drop] (s4) at (0,-0.1) {S4: Boston set a record for snowiest month.};
  % extractive output
  \node[font=\scriptsize, anchor=south, text=acc] at (7.0,1.55) {extractive: keep S1, S3};
  \node[keep] (e1) at (7.0,1.0)  {Waring ships Boston snow for 89 dollars.};
  \node[keep] (e2) at (7.0,0.3)  {He will not ship to the northeast.};
  \draw[->, acc] (s1.east) to[out=0,in=180] (e1.west);
  \draw[->, acc] (s3.east) to[out=0,in=180] (e2.west);
  % abstractive output
  \node[font=\scriptsize, anchor=south] at (7.0,-1.05) {abstractive: rewrite};
  \node[gen] (g1) at (7.0,-1.6) {A man sells Boston snow online, not to the northeast.};
  \draw[->, acc, dashed] (s1.east) to[out=-30,in=170] (g1.west);
  \draw[->, acc, dashed] (s3.east) to[out=-20,in=175] (g1.west);
\end{tikzpicture}
$$

Extractive summaries are guaranteed grammatical (every sentence was written by a
human) and factually faithful to the source (they only copy), but they read
choppily and cannot compress within a sentence or fuse information across
sentences. Abstractive summaries read like something a person would write and can
compress far more, at the cost of two hazards: disfluency, which modern neural
models solved, and **factual errors** — a summary
that asserts something the source does not — which remain the open problem. Historically
extractive methods came first and are still the more reliable choice when
faithfulness matters most; neural sequence-to-sequence models made abstractive
summarization practical and are now the dominant approach.[^jm-ctxgen]

## Extractive summarization

Extractive summarization reduces to one question: **which sentences are worth
keeping?** Score every sentence in the document for importance, then take the
top-scoring few (up to a length budget), and emit them in their original order.
The methods differ only in how they compute the score.

$$
% caption: The extractive pipeline. Split the document into sentences, score each
% for importance, sort by score, and take the top few within a length budget,
% emitted in document order. Every method below is a different scoring rule.
\begin{tikzpicture}[>=stealth, font=\footnotesize,
  box/.style={draw, minimum width=22mm, minimum height=9mm, align=center, font=\scriptsize}]
  \definecolor{acc}{HTML}{2348F2}
  \node[box] (split) at (0,0)    {split into\\sentences};
  \node[box, draw=acc, text=acc] (score) at (3.1,0) {score each\\sentence};
  \node[box] (sort)  at (6.2,0)  {sort by\\score};
  \node[box] (take)  at (9.3,0)  {take top k\\(in order)};
  \draw[->, black] (split) -- (score);
  \draw[->, acc, thick] (score) -- (sort);
  \draw[->, black] (sort) -- (take);
\end{tikzpicture}
$$

### Scoring by position and content

Two simple signals go a long way. The first is **position**. In news writing the
lead sentences carry the gist — the inverted-pyramid convention front-loads the
who-what-where — so sentence position alone is a strong baseline: keep the first
few sentences and you have a summary that is hard to beat cheaply. The strength of
the position signal is genre-dependent (it is powerful for news, weaker for
scientific papers whose contributions surface in the abstract and conclusion), so
a position score is really a learned prior over where important sentences live in
a given kind of document.

The second signal is **content**: a sentence is important if it contains the
document's important **words**. Score a word by
[TF-IDF](/natural-language-processing/semantics/vector-semantics-and-embeddings) —
high if it is frequent in _this_ document but rare across the corpus, which zeroes
out words like _the_, _of_, and _and_ — and score a sentence by the weight of the
words it carries. A clean version of this is the **centroid** method: represent the
document by the average (centroid) of its sentence vectors, and score each
sentence by its cosine similarity to that centroid. A sentence close to the
centroid is close to the document's overall topic; a sentence far from it is a
digression.

$$
% caption: Centroid scoring. Each sentence is a vector (dot); the document
% centroid (star) is their average. A sentence's score is its cosine similarity
% to the centroid, so central sentences (near the star) score high and outliers
% score low.
\begin{tikzpicture}[>=stealth, font=\footnotesize]
  \definecolor{acc}{HTML}{2348F2}
  \definecolor{red}{HTML}{C0392B}
  % sentence points
  \foreach \p in {(0.4,0.5),(1.1,1.9),(2.0,0.9),(1.5,2.4),(2.9,1.9),(0.9,2.7),(4.0,0.5)}
    \fill[acc] \p circle (2.2pt);
  % centroid marker (drawn diamond, since star glyph is unavailable)
  \fill[red] (1.7,1.4) ++(0,3.3pt) -- ++(3.3pt,-3.3pt) -- ++(-3.3pt,-3.3pt) -- ++(-3.3pt,3.3pt) -- cycle;
  \node[anchor=south, font=\scriptsize, text=red] at (1.7,1.62) {centroid};
  % a central sentence (high score) and an outlier (low score)
  \draw[acc, thick] (1.7,1.4) -- (2.0,0.9);
  \node[anchor=north west, font=\scriptsize, text=acc] at (2.05,0.9) {high score};
  \draw[black, thick, dashed] (1.7,1.4) -- (4.0,0.5);
  \node[anchor=north, font=\scriptsize, text=black] at (4.0,0.4) {outlier: low score};
\end{tikzpicture}
$$

A summary should not just be important sentences — it should be _non-redundant_
important sentences, since two sentences that say the same thing waste half the
budget. **Maximal marginal relevance** (MMR) handles this greedily: add sentences
one at a time, and at each step pick the sentence that best trades off high
importance against low similarity to what is already in the summary,

$$
\text{next} = \argmax_{s \,\notin\, \text{summary}}
\;\big[\, \lambda\,\text{importance}(s) \;-\; (1-\lambda)\,\max_{s' \in \text{summary}} \text{sim}(s, s') \,\big],
$$

so a sentence that duplicates one already chosen is penalized by the second term.
Redundancy removal is what makes MMR the natural choice for **multi-document**
summarization, where the same fact appears in many sources.[^jm-ctxgen]

### Graph methods: TextRank and LexRank

The centroid method measures whether a sentence is close to the _document
average_; a graph method measures whether it is close to _many other sentences_. The
idea is to build a graph whose nodes are sentences and whose edges join **similar**
sentences, then rank a sentence by how **central** it is in that graph — a
sentence that many others resemble is, by a kind of vote, a good representative of
the document.

$$
% caption: A sentence-similarity graph. Nodes are sentences; an edge joins two
% sentences whose similarity exceeds a threshold, weighted by that similarity. S3
% is central (linked to S1, S2, S4), so it collects the most PageRank and is
% ranked first. S5 is peripheral and ranks low.
\begin{tikzpicture}[>=stealth, font=\footnotesize,
  n/.style={circle, draw, minimum size=8mm, inner sep=0pt, font=\scriptsize},
  hub/.style={circle, draw=acc, text=acc, thick, fill=acc!8, minimum size=9mm, inner sep=0pt, font=\scriptsize}]
  \definecolor{acc}{HTML}{2348F2}
  \node[n]   (s1) at (0,2)      {S1};
  \node[n]   (s2) at (2.6,2.4)  {S2};
  \node[hub] (s3) at (1.7,0.6)  {S3};
  \node[n]   (s4) at (3.6,0.6)  {S4};
  \node[n]   (s5) at (-0.5,0.2) {S5};
  \draw[black] (s1) -- (s3);
  \draw[black] (s2) -- (s3);
  \draw[black] (s4) -- (s3);
  \draw[black] (s1) -- (s2);
  \draw[black, dashed] (s5) -- (s1);
  \node[anchor=north, font=\scriptsize, text=acc] at (1.7,0.1) {central: rank 1};
  \node[anchor=north, font=\scriptsize, text=black] at (-0.5,-0.25) {peripheral};
\end{tikzpicture}
$$

Centrality is measured by **PageRank**, the same algorithm that ranks web pages by
treating links as votes.[^jm-graph] Think of a random walker hopping between
sentences along similarity edges: it lands most often on sentences that many
edges point to, and those hitting-probabilities _are_ the centrality scores.
Because the score of a sentence depends on the scores of its neighbors, which
depend on _their_ neighbors, PageRank is computed by iterating a fixed-point update
until the scores stop changing.

> **Definition (Sentence centrality).** Given a graph over sentences with
> nonnegative edge weights $w_{ij} = \text{sim}(S_i, S_j)$, the centrality of
> $S_i$ is its PageRank: a node inherits a share of each neighbor's score in
> proportion to the edge weight, plus a small uniform term so the walk can restart
> anywhere. Central sentences are those that are similar to many other central
> sentences.

The recipe is called **TextRank** when the similarity is a simple word-overlap
count and **LexRank** when it is cosine similarity of
[TF-IDF](/natural-language-processing/semantics/vector-semantics-and-embeddings) (or
embedding) sentence vectors; the graph algorithm is identical. Written out, with a
damping factor $d$ (usually $0.85$) that mixes the neighbor-vote term with a
uniform restart:

```algorithm
caption: $\textsc{TextRank}$ — extractive summarization by PageRank centrality
input: sentences $S_1, \ldots, S_N$; similarity threshold $\tau$; damping $d$; budget $k$
for each pair $i, j$ with $i \neq j$ do
  $w_{ij} \gets \text{sim}(S_i, S_j)$
  if $w_{ij} < \tau$ then
    $w_{ij} \gets 0$ // prune weak edges
$\text{score}_i \gets 1/N$ for all $i$ // uniform initialization
repeat
  for each sentence $i$ do
    $\text{score}_i \gets \dfrac{1-d}{N} + d \displaystyle\sum_{j \neq i} \dfrac{w_{ij}}{\sum_{k} w_{jk}}\,\text{score}_j$
until scores converge
sort sentences by $\text{score}_i$ descending
return the top $k$ sentences, emitted in document order
```

In the one-line update, sentence $i$'s new score is a small
uniform floor $(1-d)/N$ plus $d$ times a weighted sum of its neighbors' scores,
where each neighbor $j$ distributes its score across its edges in proportion to
their weights. Iterate and the scores settle into the stationary distribution of
the random walk. Graph methods are **unsupervised** — no training data, just the
document — which is their great advantage, and why TextRank/LexRank were standard
extractive summarizers for years.[^jm-graph]

#### A worked TextRank iteration

For a concrete run of the update, take the
five-sentence graph above and read off its weighted adjacency: $S_3$ links to
$S_1, S_2, S_4$; $S_1$ links to $S_2, S_3$; $S_2$ links to $S_1, S_3$; $S_4$ links
only to $S_3$; and $S_5$ links only to $S_1$. Give every edge unit weight for
clarity, use damping $d = 0.85$, and start each sentence at the uniform score
$1/5 = 0.20$. The floor term is fixed at $(1-d)/N = 0.15/5 = 0.03$ for every
sentence, every iteration.

The neighbor-vote term for $S_3$ collects, from each neighbor $j$ that points at
$S_3$, that neighbor's current score split across $j$'s out-edges. $S_1$ has two
out-edges so it sends $0.20/2 = 0.10$; $S_2$ likewise sends $0.10$; $S_4$ has one
out-edge so it sends its whole $0.20$. The first update for $S_3$ is therefore

$$
\text{score}_3 \;=\; 0.03 \;+\; 0.85\,(0.10 + 0.10 + 0.20) \;=\; 0.03 + 0.85\,(0.40) \;=\; 0.37.
$$

Doing the same arithmetic for the others in the same sweep gives $S_1 = 0.03 +
0.85(0.10 + 0.20) = 0.29$ (votes from $S_2$'s split half and $S_5$'s whole
score), $S_2 = 0.29$, $S_4 = 0.03 + 0.85(0.20/3) = 0.09$, and $S_5 = 0.03$
(nothing points at it). Central $S_3$ has already pulled ahead; peripheral $S_5$,
a pure sink for probability, has collapsed to the floor.

$$
% caption: Three TextRank sweeps over the five-sentence graph, unit edge weights
% and damping 0.85. Central S3 climbs and holds the top rank; the peripheral S5
% sinks to the uniform floor. The ranking, not the exact values, is what selects
% the summary.
\begin{tikzpicture}[>=stealth, font=\footnotesize]
  \definecolor{acc}{HTML}{2348F2}
  \definecolor{red}{HTML}{C0392B}
  \node[font=\scriptsize, anchor=west] at (0,3.3)   {sentence};
  \node[font=\scriptsize] at (3.2,3.3) {init};
  \node[font=\scriptsize] at (4.9,3.3) {sweep 1};
  \node[font=\scriptsize] at (6.6,3.3) {sweep 2};
  \node[font=\scriptsize] at (8.3,3.3) {sweep 3};
  \draw[black] (-0.1,3.05) -- (9.2,3.05);
  \foreach \s/\r/\a/\b/\c/\y in {
    S1/0.20/0.29/0.31/0.31/2.5,
    S2/0.20/0.29/0.31/0.31/1.9,
    S3/0.20/0.37/0.40/0.41/1.3,
    S4/0.20/0.09/0.13/0.14/0.7,
    S5/0.20/0.03/0.03/0.03/0.1} {
    \node[font=\scriptsize, anchor=west] at (0,\y) {\s};
    \node[font=\scriptsize] at (3.2,\y) {\r};
    \node[font=\scriptsize] at (4.9,\y) {\a};
    \node[font=\scriptsize] at (6.6,\y) {\b};
    \node[font=\scriptsize] at (8.3,\y) {\c};
  }
  \node[font=\scriptsize, text=acc, anchor=west] at (8.7,1.3) {rank 1};
  \node[font=\scriptsize, text=red, anchor=west] at (8.7,0.1) {f\/loor};
\end{tikzpicture}
$$

The exact numbers drift for a sweep or two and then settle; what matters is that
the _order_ stabilizes almost immediately. $S_3$ leads, $S_1$ and $S_2$ tie behind
it, $S_4$ trails, and $S_5$ is last. With a budget of two sentences the summarizer
returns $S_3$ and one of the tied pair, emitted in document order. Notice the
mechanism that makes centrality work: $S_4$ contributes little on its own, but by
pointing all its score at $S_3$ it _amplifies_ the sentence many others already
favor. Centrality is votes weighted by the voter's own standing, exactly PageRank.

### Supervised sentence selection

If you have documents paired with human summaries, sentence selection can be
learned rather than scored by hand. Frame it as **binary classification**: for
each sentence, predict include-in-summary or not. To build the training labels,
align each human-summary sentence to the source sentence it most overlaps and mark
that source sentence positive; everything else is negative. A classifier — today a
transformer that reads each sentence in the context of the whole document and
[fine-tunes](/natural-language-processing/transformers/fine-tuning-and-prompting) a
per-sentence keep/drop head — then learns which combination of position, content
weight, and centrality signals predicts a kept sentence, folding the hand-designed
scores above into learned features.[^jm-pretrain]

Extractive selection is the safe half of summarization: it can rank and keep the
sentences that carry the meaning, but it can never compress within a sentence, fuse
two sentences, or paraphrase. Doing any of those means _generating_ new text. This
continues in [Abstractive Summarization and Evaluation](/natural-language-processing/applications/abstractive-summarization-and-evaluation),
which builds the generative summarizer and the metrics that score every summary.

[^jm-ctxgen]: **Jurafsky & Martin**, _Speech and Language Processing_ (3rd ed.), §9.9 — Contextual Generation and Summarization: text summarization as context-based autoregressive generation; the append-a-summary-with-a-separator scheme converting each article–summary pair $(x_1,\ldots,x_m),(y_1,\ldots,y_n)$ into one training instance $(x_1,\ldots,x_m,\delta,y_1,\ldots,y_n)$ trained with teacher forcing; priming with the article at inference; the CNN/DailyMail news corpus; and single- vs. multi-document and generic vs. query-focused as task variants.
[^jm-graph]: **Jurafsky & Martin**, §9.9 (extractive summarization) — extractive summarization by scoring and selecting sentences, and unsupervised graph-based centrality (TextRank/LexRank) that ranks sentences by PageRank over a sentence-similarity graph, presented here at that level of detail; PageRank centrality as similarity-weighted neighbor voting with a damping/restart term.
[^jm-pretrain]: **Jurafsky & Martin**, §9.9.1 — Applying Transformers to other NLP tasks: pretraining a transformer language model self-supervised on a large corpus and then finetuning on a smaller task-specific dataset; denoising pretraining objectives (BART-style corruption/reconstruction, PEGASUS-style gap-sentence generation) and zero-/few-shot prompting of large language models as the modern route to summarization.