Text Summarization

Text summarization is the task of taking a full-length document and producing a shorter text that keeps its most important information. Stated as a mapping it is the same shape as translation — a source $\mathbf{x}=x_1,\dots ,x_m$ goes in and a shorter target $\mathbf{y}=y_1,\dots ,y_n$ with $n\ll m$ comes out — but where translation preserves meaning across a change of language, summarization preserves the essential meaning across a change of length. The modern method is context-based autoregressive generation: prime a language model with the document and let it generate the summary, exactly the sequence-to-sequence recipe translation established.¹ This lesson covers what the task is, the two families of methods that solve it, and how to tell whether a summary is any good.

The task and its flavors

Summarize this is really a family of tasks that differ along a few axes. Identifying which variant applies comes first, because it determines both the method and the evaluation.

Single- vs. multi-document. A single-document summarizer condenses one article. A multi-document summarizer fuses many sources on the same topic — several news stories about one event, a cluster of reviews of one product — into a single summary, which adds the problem of detecting and removing redundancy across documents.¹

Generic vs. query-focused. A generic summary answers what is this document about? with no further guidance. A query-focused (or topic-focused) summary answers what does this document say about $q$? for a user query $q$, keeping only the material relevant to $q$. Query-focused summarization is essentially long-form question answering: retrieve or condition on the query, then generate the answer from the source.

Extractive vs. abstractive. This is the axis that shapes the algorithm.

Extractive summaries are guaranteed grammatical (every sentence was written by a human) and factually faithful to the source (they only copy), but they read choppily and cannot compress within a sentence or fuse information across sentences. Abstractive summaries read like something a person would write and can compress far more, at the cost of two hazards: disfluency, which modern neural models solved, and factual errors — a summary that asserts something the source does not — which remain the open problem. Historically extractive methods came first and are still the more reliable choice when faithfulness matters most; neural sequence-to-sequence models made abstractive summarization practical and are now the dominant approach.¹

Extractive summarization

Extractive summarization reduces to one question: which sentences are worth keeping? Score every sentence in the document for importance, then take the top-scoring few (up to a length budget), and emit them in their original order. The methods differ only in how they compute the score.

Scoring by position and content

Two simple signals go a long way. The first is position. In news writing the lead sentences carry the gist — the inverted-pyramid convention front-loads the who-what-where — so sentence position alone is a strong baseline: keep the first few sentences and you have a summary that is hard to beat cheaply. The strength of the position signal is genre-dependent (it is powerful for news, weaker for scientific papers whose contributions surface in the abstract and conclusion), so a position score is really a learned prior over where important sentences live in a given kind of document.

The second signal is content: a sentence is important if it contains the document's important words. Score a word by TF-IDF — high if it is frequent in this document but rare across the corpus, which zeroes out words like the, of, and and — and score a sentence by the weight of the words it carries. A clean version of this is the centroid method: represent the document by the average (centroid) of its sentence vectors, and score each sentence by its cosine similarity to that centroid. A sentence close to the centroid is close to the document's overall topic; a sentence far from it is a digression.

A summary should not just be important sentences — it should be non-redundant important sentences, since two sentences that say the same thing waste half the budget. Maximal marginal relevance (MMR) handles this greedily: add sentences one at a time, and at each step pick the sentence that best trades off high importance against low similarity to what is already in the summary,

so a sentence that duplicates one already chosen is penalized by the second term. Redundancy removal is what makes MMR the natural choice for multi-document summarization, where the same fact appears in many sources.¹

Graph methods: TextRank and LexRank

The centroid method measures whether a sentence is close to the document average; a graph method measures whether it is close to many other sentences. The idea is to build a graph whose nodes are sentences and whose edges join similar sentences, then rank a sentence by how central it is in that graph — a sentence that many others resemble is, by a kind of vote, a good representative of the document.

Centrality is measured by PageRank, the same algorithm that ranks web pages by treating links as votes.² Think of a random walker hopping between sentences along similarity edges: it lands most often on sentences that many edges point to, and those hitting-probabilities are the centrality scores. Because the score of a sentence depends on the scores of its neighbors, which depend on their neighbors, PageRank is computed by iterating a fixed-point update until the scores stop changing.

The recipe is called TextRank when the similarity is a simple word-overlap count and LexRank when it is cosine similarity of TF-IDF (or embedding) sentence vectors; the graph algorithm is identical. Written out, with a damping factor $d$ (usually $0.85$) that mixes the neighbor-vote term with a uniform restart:

In the one-line update, sentence $i$'s new score is a small uniform floor $(1-d)/N$ plus $d$ times a weighted sum of its neighbors' scores, where each neighbor $j$ distributes its score across its edges in proportion to their weights. Iterate and the scores settle into the stationary distribution of the random walk. Graph methods are unsupervised — no training data, just the document — which is their great advantage, and why TextRank/LexRank were standard extractive summarizers for years.²

A worked TextRank iteration

For a concrete run of the update, take the five-sentence graph above and read off its weighted adjacency: $S_3$ links to $S_1,S_2,S_4$; $S_1$ links to $S_2,S_3$; $S_2$ links to $S_1,S_3$; $S_4$ links only to $S_3$; and $S_5$ links only to $S_1$. Give every edge unit weight for clarity, use damping $d=0.85$, and start each sentence at the uniform score $1/5=0.20$. The floor term is fixed at $(1-d)/N=0.15/5=0.03$ for every sentence, every iteration.

The neighbor-vote term for $S_3$ collects, from each neighbor $j$ that points at $S_3$, that neighbor's current score split across $j$'s out-edges. $S_1$ has two out-edges so it sends $0.20/2=0.10$; $S_2$ likewise sends $0.10$; $S_4$ has one out-edge so it sends its whole $0.20$. The first update for $S_3$ is therefore

Doing the same arithmetic for the others in the same sweep gives $S_1=0.03+0.85(0.10+0.20)=0.29$ (votes from $S_2$'s split half and $S_5$'s whole score), $S_2=0.29$, $S_4=0.03+0.85(0.20/3)=0.09$, and $S_5=0.03$ (nothing points at it). Central $S_3$ has already pulled ahead; peripheral $S_5$, a pure sink for probability, has collapsed to the floor.

The exact numbers drift for a sweep or two and then settle; what matters is that the order stabilizes almost immediately. $S_3$ leads, $S_1$ and $S_2$ tie behind it, $S_4$ trails, and $S_5$ is last. With a budget of two sentences the summarizer returns $S_3$ and one of the tied pair, emitted in document order. Notice the mechanism that makes centrality work: $S_4$ contributes little on its own, but by pointing all its score at $S_3$ it amplifies the sentence many others already favor. Centrality is votes weighted by the voter's own standing, exactly PageRank.

Supervised sentence selection

If you have documents paired with human summaries, sentence selection can be learned rather than scored by hand. Frame it as binary classification: for each sentence, predict include-in-summary or not. To build the training labels, align each human-summary sentence to the source sentence it most overlaps and mark that source sentence positive; everything else is negative. A classifier — today a transformer that reads each sentence in the context of the whole document and fine-tunes a per-sentence keep/drop head — then learns which combination of position, content weight, and centrality signals predicts a kept sentence, folding the hand-designed scores above into learned features.³

Extractive selection is the safe half of summarization: it can rank and keep the sentences that carry the meaning, but it can never compress within a sentence, fuse two sentences, or paraphrase. Doing any of those means generating new text. This continues in Abstractive Summarization and Evaluation, which builds the generative summarizer and the metrics that score every summary.