Discourse Coherence

Compare these two short passages:

(a) Jane took a train from Paris to Istanbul. She likes spinach.
(b) Jane took a train from Paris to Istanbul. She had to attend a conference.

Both are grammatical, both are about Jane, and both pair a travel sentence with a personal fact. Yet (b) reads as a small story and (a) reads as a non sequitur. The difference is that in (b) the reader can build a connection — the second sentence supplies a reason for the first — while in (a) no such link is available, so the reader strains to invent one.¹ A discourse is a structured group of sentences, not a bare collection, and coherence is the property that makes the group hang together.

We met discourse once already, from the side of the entities: coreference resolution recognizes one entity under its many names across a text. This lesson takes the whole discourse as the object of study. What makes a sequence of sentences a coherent text, and how do we model it computationally? The answer has several strands — relations between clauses, threads of entities, shared topic, and genre-level structure — and each has spawned its own corpora and algorithms.

What makes a text cohere

Coherence comes in a local flavor (links between adjacent or nearby sentences) and a global flavor (conventional structure of a whole genre). Local coherence itself has three sources.¹

Relational coherence: adjacent clauses are tied by a coherence relation like Reason, Elaboration, or Contrast. Passage (b) coheres because Reason holds between its two sentences.
Entity-based coherence: the discourse stays about the same salient entities rather than lurching among them. A passage that swings from John to Jenny to the piano store to the living room and back reads as incoherent even when every sentence is fine.²
Topical coherence: nearby sentences discuss the same topic and draw on the same vocabulary — lexical cohesion, the sharing of identical or related words (chimney, house, garret, closet, window all from one semantic field).¹

The two-sentence Hobbs pair. Passage (a) is incoherent (no relation links the sentences); passage (b) coheres because a Reason relation runs from the second sentence to the first. The reader constructs the relation, so coherence is something comprehension builds, not something the words state.

Global coherence sits above all three: genres carry conventional macro-structure. Academic articles have Methods and Results; folktales follow plot motifs; persuasive essays state a claim and marshal premises for and against it. Coherence matters wherever text quality matters — essay grading, summarization (a summary should keep the central material and drop the peripheral), and even clinical measures that detect disordered language as a symptom of schizophrenia.¹ For summarization in particular, knowing the coherence relations tells the system which spans are central and which can be cut.³

Coherence relations

A coherence relation is a structured link between two spans of text. The dominant model is Rhetorical Structure Theory (RST).⁴ In RST a relation holds between two spans, generally a nucleus and a satellite. The nucleus is the span central to the writer's purpose and interpretable on its own; the satellite is peripheral and interpretable only relative to the nucleus. Some symmetric relations instead hold between two nuclei. A handful of the relations, with definitions adapted from the RST Treebank manual:⁴

Relation	Nucleus / satellite	Example
Reason	NUC an action by an animate agent, SAT its reason	Jane took a train from Paris to Istanbul. / She had to attend a conference.
Elaboration	SAT adds detail about the nucleus	Dorothy was from Kansas. / She lived in the midst of the great Kansas prairies.
Evidence	SAT presents information to make the reader accept the nucleus	Kevin must be here. / His car is parked outside.
Attribution	SAT gives the source of reported speech in the nucleus	Analysts estimated / that sales at U.S. stores declined in the quarter.
List	multinuclear series, no contrast or explicit comparison	Billy Bones was the mate; / Long John, he was quartermaster.

Asymmetric relations are drawn graphically with an arrow from the satellite to the nucleus.

The RST Evidence relation drawn in the standard way. The satellite (the evidence) points with an arrow to the nucleus (the claim it supports). The nucleus stands alone; the satellite only makes sense relative to it.

The base units — the leaves of an RST analysis — are elementary discourse units (EDUs), roughly clauses, also called discourse segments.⁴ Because EDUs can span arbitrary text, finding their boundaries is itself a task (below). Relations compose hierarchically: a relation can hold between an EDU and another EDU, or between an EDU and a whole span already built from smaller relations, all the way up to a single tree over the entire text. The RST Discourse Treebank is the largest such corpus, 385 documents from the Penn Treebank with full RST parses using 78 relations grouped into 16 classes.³

An RST discourse tree over a Scientific American paragraph about Mars. Leaves are EDUs; each satellite arrow points to its nucleus, and relations nest: an Evidence over spans 2-9 sits under the whole, itself built from a Background and an Elaboration, which decomposes into a List and a Contrast.

The Penn Discourse Treebank

A second, more lexically grounded model is the Penn Discourse Treebank (PDTB).⁵ Rather than ask annotators to tag a relation directly, PDTB labeling starts from the surface. Annotators are given a list of discourse connectives — words like because, although, when, since, as a result — that signal a coherence relation, and they mark the connective together with its two arguments, Arg1 and Arg2. In

(22.13) Jewelry displays in department stores were often cluttered and uninspired. And the merchandise was, well, fake. As a result, marketers of faux gems steadily lost space in department stores.

the connective as a result signals a causal relation between Arg1 (the first two sentences, its antecedent) and Arg2 (the sentence it introduces, in bold).

Not every relation is marked by an explicit connective. Where none appears, the annotator supplies the word that could have been there and marks the relation implicit, then labels its sense.

Explicit vs. implicit relations in the PDTB. In the explicit case a connective (as a result) links Arg1 and Arg2 on the surface. In the implicit case no connective appears, so the annotator inserts the one that could have (implicit=as a result) and labels the sense of the relation between the two spans.

PDTB senses form a four-way top level — Temporal, Contingency, Comparison, Expansion — refined into 16 types and 23 subtypes.⁵ One example per top class:

Class	Type	Example (connective underlined)
Temporal	Synchronous	The parishioners stop to chat at the church door, (implicit while) as members always have.
Contingency	Reason	Mr. Breeden is in a position to get somewhere with his agenda (implicit because) he is savvy in the ways of Washington.
Comparison	Contrast	The U.S. wants the removal of barriers to investment; Japan denies there are barriers.
Expansion	Conjunction	the actors stand outside their characters … but they literally stand on their heads.

The final corpus holds roughly 18,000 explicit and 16,000 implicit relations. Unlike RST, the PDTB annotates only span pairs and makes no commitment above the span-pair level — it builds no global tree.⁵ Corpora in this style exist for other languages too; Chinese marks only about 22% of relations with explicit connectives (against 47% for English), so its treebank maps sentence pairs directly to sense tags.⁵

Discourse structure parsing

Discourse parsing recovers these relations automatically. For RST it runs in two stages: segment the text into EDUs, then build the tree.⁶

EDU segmentation

The first stage, EDU segmentation, marks the start and end of each EDU. Since EDUs roughly correspond to clauses, early models ran a syntactic parser and post-processed its output; modern systems use a neural sequence model that predicts, for each token, whether an EDU begins there. The target labeling looks like

Mr. Rambo says that a 3.2-acre property overlooking the San Fernando Valley is priced at $4 million because the late actor Erroll Flynn once lived there.

The input passes through an encoder; a linear layer and softmax over each token produce a $0/1$ sequence where $1$ marks the start of an EDU.⁶

EDU segmentation as neural sequence labeling. An encoder produces one vector per token; a linear layer and softmax emit 0 or 1 per token, with 1 marking the start of a new elementary discourse unit. Here a break is predicted at "that", the start of the next EDU.

Building the RST tree

With EDUs in hand, an RST parser assembles the tree. Modern parsers are shift-reduce transition systems built on a neural encoder.⁷ The parser state is a stack and a queue of EDUs, and it produces the tree by a sequence of actions:

shift — push the front EDU of the queue onto the stack as a single-node subtree.
reduce $(l, d)$ — merge the top two subtrees on the stack, labeling the coherence relation $l$ and the nuclearity direction $d \in {NN, NS, SN}$ (nucleus-nucleus, nucleus-satellite, satellite-nucleus).
pop-root — remove the final finished tree from the stack.

The parser of Yu et al. encodes the input with a hierarchical biLSTM: a word-level biLSTM builds a representation of the words inside each EDU (average-pooled to one vector per EDU), and an EDU-level biLSTM contextualizes the EDU sequence. A feedforward decoder then chooses each action from a concatenation of the top three stack subtrees $s_{0}, s_{1}, s_{2}$ and the front-of-queue EDU $q_{0}$ ,⁷

o = W [h_{s_{0}}^{t}, h_{s_{1}}^{t}, h_{s_{2}}^{t}, h_{q_{0}}^{e}],

where each subtree vector $h^{t}$ is the average pool over that subtree's EDU encodings. A softmax over $o$ gives the action distribution. The trace below builds a four-EDU tree.

A shift-reduce trace building an RST tree over four EDUs. Each row is a parser state (stack and queue) and the action taken: SH shifts the front EDU, RD reduces the top two subtrees with a labeled, directed relation, PR pops the finished root. The tree it builds attaches e1:e2 by attribution then elaborates.

Training maps each gold tree to its oracle action sequence and fits the decoder with cross-entropy (plus $ℓ_{2}$ regularization):⁷

p_{a} = \frac{exp ( o _{a} )}{\sum _{a^{'} \in A} exp ( o _{a^{'}} )}, L_{CE} = - log p_{a} + \frac{λ}{2} ∥Θ ∥^{2} .

RST parsers report four F1 scores of increasing strictness, computed after binarizing the gold trees: Span (bare tree, no labels), Nuclearity (with nucleus/satellite marks), Relation (with relation labels), and Full (both).⁷

PDTB relation classification

PDTB parsing is shallower — flat relations, no tree — hence also called shallow discourse parsing. It splits into four subtasks:⁸

Find the discourse connectives, disambiguating and the connective from and the NP conjunction.
Find each connective's two argument spans.
Label the sense of each explicit relation.
Assign a relation between every adjacent pair of sentences (the implicit case).

Task 4 is the hardest and most studied: take a pair of adjacent sentences and predict a sense label, usually one of the 11 common second-level PDTB tags. A simple but strong baseline encodes each span with BERT, takes the [CLS] hidden state, and passes it through a one-layer tanh feedforward net and softmax for sense classification.⁸ Task 1 is a special case of word-sense disambiguation: and is a discourse connective in

(22.24) Selling picked up as previous buyers bailed out of their positions andaggressive short sellers moved in.

but a non-discourse NP conjunction in My favorite colors are blue and green. Similarly once marks a temporal relation in one context but merely means formerly in a form of asbestos once used to make cigarette filters. Modern systems perform task 1 end-to-end with a biLSTM-CRF that emits B-CONN/I-CONN/O tags, and use the same BERT sequence models for task 2's span-finding.⁸

Where this continues

We have the first source of coherence: relations between clauses. Rhetorical Structure Theory ties spans as nucleus and satellite; the Penn Discourse Treebank grounds the same links in explicit and implicit connectives; and discourse parsers recover both — a shift-reduce transition system that builds the RST tree from EDUs, and a shallow classifier that labels the sense of each adjacent-sentence pair.

Relational coherence is only one of three local sources, and it says nothing about the global structure a genre imposes. Being about stable entities, sharing a topic, and obeying a narrative or argumentative macro-structure continue in entity-based and global coherence, which also gathers the neural models that learn each.

Jurafsky & Martin, Speech and Language Processing (3rd ed.), Ch. 22 intro — Discourse Coherence: discourse and coherence; the Hobbs spinach/conference minimal pair; local coherence in three forms (relational coherence relations, entity-based coherence, topical coherence via lexical cohesion) versus global coherence; the roles of coherence in essay grading, summarization, and detecting disordered language. ↩ ↩² ↩³ ↩⁴
Jurafsky & Martin, Ch. 22 intro; §22.3 — entity-based coherence: the incoherent passage that swings among John, Jenny, the piano store, and the living room; a discourse is locally coherent by being about salient entities that do not lurch. ↩
Jurafsky & Martin, §22.1.1 — the RST Discourse Treebank (Carlson et al. 2001): 385 documents from the Penn Treebank with full RST parses, 78 relations in 16 classes; the observation that nuclei carry more important information than satellites, so a summary can drop satellites. ↩ ↩²
Jurafsky & Martin, §22.1.1 — Rhetorical Structure Theory (Mann and Thompson 1987): nucleus/satellite relations (Reason, Elaboration, Evidence, Attribution, List); satellite-to-nucleus arrows; elementary discourse units (EDUs), also called discourse segments; hierarchical composition into a tree. ↩ ↩² ↩³
Jurafsky & Martin, §22.1.2 — Penn Discourse TreeBank (Prasad et al.): lexically grounded annotation via discourse connectives linking Arg1 and Arg2; explicit versus implicit relations, where the annotator inserts the connective that could have appeared; the four-class sense hierarchy (Temporal, Contingency, Comparison, Expansion) with 16 types and 23 subtypes; ~18,000 explicit and 16,000 implicit relations; no commitment above the span-pair level; the Chinese Discourse TreeBank and its 22% explicit rate. ↩ ↩² ↩³ ↩⁴
Jurafsky & Martin, §22.2.1 — EDU segmentation for RST parsing: early syntactic-parser-plus-postprocessing versus modern neural sequence models (simplified from Lukasik et al. 2020); the encoder-plus-linear-plus-softmax predicting a 0/1 sequence where 1 marks an EDU start. ↩ ↩²
Jurafsky & Martin, §22.2.2 — RST parsing: the shift-reduce parser of Yu et al. (2018) with shift, reduce( $l$ , $d$ ) over $d \in {$ NN,NS,SN $}$ , and pop-root; the hierarchical biLSTM word- and EDU-level encoder with average pooling; the feedforward decoder over the top three stack subtrees and the front-of-queue EDU; oracle-action training with cross-entropy and $ℓ_{2}$ regularization; the Span/Nuclearity/Relation/Full RST-Pareval metrics on binarized trees. ↩ ↩² ↩³ ↩⁴
Jurafsky & Martin, §22.2.3 — PDTB (shallow) discourse parsing: the four subtasks (find connectives, find their spans, label explicit senses, label implicit adjacent-sentence relations) from Lin et al. (2014); the BERT [CLS] plus one-layer feedforward baseline for implicit sense classification (Nie et al. 2019); connective disambiguation as word-sense disambiguation (and, once) with a biLSTM-CRF over B-CONN/I-CONN/O tags. ↩ ↩² ↩³