Coherence and Discourse Structure

This builds on coreference resolution, which recognized one entity under its many names across a text. Coreference is only the entity half of tracking a discourse; the other half is what makes a sequence of sentences hang together at all — the subject of this lesson.

Discourse: what makes a text cohere

Language comes in collocated, structured groups of sentences — a discourse — and coherence is the relationship among sentences that separates a real discourse from a random pile.¹ Compare Hobbs's minimal pair:

(a) John took a train from Paris to Istanbul. He likes spinach. (b) Jane took a train from Paris to Istanbul. She had to attend a conference.

(a) is incoherent — what has spinach to do with train trips? — while (b) coheres because the second sentence gives a reason for the first.¹ That readers strain to find a connection in (a) shows coherence is something comprehension actively constructs. Real discourses are coherent in three local ways and one global way:¹

Coherence relations

A coherence relation is a structured link between two spans of text.² The dominant model is Rhetorical Structure Theory (RST). RST relations hold between a nucleus (central, independently interpretable) and a satellite (peripheral, interpretable only relative to the nucleus); some symmetric relations hold between two nuclei.² A few examples:

The base units — leaves of an RST tree — are elementary discourse units (EDUs), roughly clauses.² Relations compose hierarchically into a tree over the whole text; asymmetric relations are drawn as an arrow from satellite to nucleus.

A second, more lexically grounded corpus is the Penn Discourse TreeBank (PDTB).³ Rather than tag relations directly, annotators start from discourse connectives — words like because, although, since, as a result — that signal a relation between two arguments Arg1 and Arg2. Where a relation has no explicit connective, the annotator supplies the one that could have appeared (marked implicit) and labels its sense. PDTB senses form a four-way top level — Temporal, Contingency, Comparison, Expansion — refined into types and subtypes.³ Unlike RST, PDTB annotates only span pairs and makes no commitment to a global tree.³

Discourse structure parsing

Discourse parsing recovers these relations automatically.⁴ For RST it runs in two stages. First, EDU segmentation labels each token as a break or not, typically with a neural sequence model over an encoder:

Mr. Rambo says that a 3.2-acre property overlooking the San Fernando Valley is priced at $4 million because the late actor Erroll Flynn once lived there.

Second, an RST parser builds the tree. Modern parsers are shift-reduce transition systems with a stack and a queue and two core actions:⁴

• shift — pop the front EDU off the queue onto the stack as a single-node subtree.
• reduce$(l,d)$ — merge the top two subtrees, labeling the relation $l$ and the nuclearity direction $d\in \{\text{NN},\text{NS},\text{SN}\}$.

plus a pop-root that removes the final tree. The Yu et al. parser encodes words and EDUs with a hierarchical biLSTM, then a feedforward decoder chooses each action from the top few stack subtrees and the front-of-queue EDU. Below is the action trace that builds a four-EDU tree.

Training maps each gold tree to its oracle action sequence and fits the decoder with cross-entropy. RST parsers report four $F_1$ scores of increasing strictness: Span (bracketing only) $\le$ Nuclearity (bracketing + N/S direction) $\le$ Relation (+ label) $\le$ Full (all three).⁴

PDTB parsing is shallower — flat relations, no tree — and splits into four subtasks: find discourse connectives (disambiguating and the connective from and the NP conjunction), find each connective's two spans, label explicit relations, and label the relation between every adjacent sentence pair (the implicit case).⁴ A strong baseline for the implicit task encodes each span with BERT, takes the [CLS] state, and passes it through a small feedforward net and softmax.⁴

Centering and entity-based coherence

The second flavor of coherence is being about something. A discourse coheres by keeping the same salient entities in focus rather than lurching among them. Consider Grosz et al.'s minimal pair, same propositions, different realization:⁵

(i) John went to his favorite music store to buy a piano. He had frequented the store for years. He was excited that he could finally buy a piano. He arrived just as the store was closing.

(ii) John went to his favorite music store to buy a piano. It was a store John had frequented for years. He was excited that he could finally buy a piano. It was closing just as John arrived.

Both are grammatical, but (i) is more coherent — it is steadily about John, while (ii) swings John / store / John / store.⁵ Centering Theory formalizes this. Each utterance $U_n$ carries two structures: a backward-looking center$C_b(U_n)$, the single entity currently in focus, and a set of forward-looking centers $C_f(U_n)$, the entities $U_n$ evokes, ranked by grammatical role (subject $>$ object $>$ other).⁵ The top-ranked forward center is the preferred center $C_p(U_n)=\max C_f(U_n)$, a prediction of what comes next. The relationship between successive utterances falls into four transition types, ordered by how coherent they are:

Two rules drive the theory. Rule 1: if any forward center is pronominalized in the next utterance, then $C_b$ must be pronominalized too — pronouns mark the most salient entity. Rule 2: transitions are preferred in the order Continue > Retain

Smooth-Shift > Rough-Shift.⁵ Discourse (i) is a chain of Continues on John; (ii) forces Shifts, which is why it reads worse.

The entity grid

Centering is a top-down theory. The entity grid learns the same intuition bottom-up.⁶ A grid is a table: one row per sentence, one column per entity, each cell recording the entity's grammatical role in that sentence — Subject, Object, X (other), or – (absent). Columns come from coreference resolution, so this stage depends on the first half of the lesson.

Coherence is read off the local transitions — a length-2 slice of a column, like $[\text{S}\text{–}]$ (subject then absent). With role set $R=\{\text{S},\text{O},\text{X},\text{–}\}$, each of the $|R{|}^2=16$ transition types $(r,r^{\prime })$ becomes a feature: its frequency across all columns,

collected into a document vector $\mathbf{\varphi }$ fed to a classifier.⁶ Coherent texts have a characteristic topology: a few dense columns (entities mentioned throughout, often as S or O) and many sparse ones.⁶

Labels are expensive, so entity-grid and neural coherence models train by self-supervision: a naturally-ordered discourse is more coherent than a permutation of its sentences, so the model learns to prefer the original.⁶ This is realized as sentence-order discrimination (rank original above a permutation), insertion (find the original slot for a removed sentence), or reconstruction (put shuffled sentences back in order).⁷

Local vs. global coherence

Everything above is local coherence — links between adjacent or nearby sentences. Discourses also exhibit global coherence: genres carry conventional structures. Academic articles have Methods and Results; stories follow plot motifs; persuasive essays state a claim and marshal premises for and against it.¹ Coherence detection matters wherever text quality matters — essay grading, summarization (drop satellites, keep nuclei), even clinical measures of disordered language.¹

The through-line

A text is more than a bag of sentences. Coreference is the machinery that recognizes one entity under its many names; coherence is what ties clauses and entities across a whole document. The two halves meet in the entity grid, where coherence is measured over the coreference chains: whether a text stays about John cannot be assessed until the mentions of John are identified. Both take the entities and relations of semantic roles and information extraction and lift them from the sentence to the discourse.