--- title: Discourse Coherence module: Linguistic Structure moduleNumber: 6 lessonNumber: 17 order: 617 summary: > A text is more than a set of sentences. What binds a run of sentences into a discourse is coherence, and one of its sources is structured relations between clauses. This lesson develops relational coherence — RST and the PDTB models of coherence relations — and discourse-structure parsing: EDU segmentation and shift-reduce RST parsing, then PDTB relation classification. Entity-based and global coherence follow in the companion lesson. topics: [Structure] sources: - book: Jurafsky ref: "Ch. 22 — Discourse Coherence; §22.1 Coherence Relations; §22.2 Discourse Structure Parsing" --- Compare these two short passages: > (a) Jane took a train from Paris to Istanbul. She likes spinach. > > (b) Jane took a train from Paris to Istanbul. She had to attend a conference. Both are grammatical, both are about Jane, and both pair a travel sentence with a personal fact. Yet (b) reads as a small story and (a) reads as a non sequitur. The difference is that in (b) the reader can build a connection — the second sentence supplies a **reason** for the first — while in (a) no such link is available, so the reader strains to invent one.[^jm-intro] A **discourse** is a structured group of sentences, not a bare collection, and **coherence** is the property that makes the group hang together. We met discourse once already, from the side of the entities: [coreference resolution](/natural-language-processing/linguistic-structure/coreference-and-discourse) recognizes one entity under its many names across a text. This lesson takes the whole discourse as the object of study. What makes a sequence of sentences a coherent text, and how do we model it computationally? The answer has several strands — relations between clauses, threads of entities, shared topic, and genre-level structure — and each has spawned its own corpora and algorithms. ## What makes a text cohere Coherence comes in a **local** flavor (links between adjacent or nearby sentences) and a **global** flavor (conventional structure of a whole genre). Local coherence itself has three sources.[^jm-intro] - **Relational** coherence: adjacent clauses are tied by a **coherence relation** like Reason, Elaboration, or Contrast. Passage (b) coheres because Reason holds between its two sentences. - **Entity-based** coherence: the discourse stays **about** the same salient entities rather than lurching among them. A passage that swings from John to Jenny to the piano store to the living room and back reads as incoherent even when every sentence is fine.[^jm-entity] - **Topical** coherence: nearby sentences discuss the same topic and draw on the same vocabulary — **lexical cohesion**, the sharing of identical or related words (_chimney_, _house_, _garret_, _closet_, _window_ all from one semantic field).[^jm-intro] $$ % caption: The two-sentence Hobbs pair. Passage (a) is incoherent (no relation % links the sentences); passage (b) coheres because a Reason relation runs from % the second sentence to the first. The reader constructs the relation, so % coherence is something comprehension builds, not something the words state. \begin{tikzpicture}[>=stealth, font=\footnotesize, s/.style={draw, minimum width=48mm, minimum height=8mm, align=left, font=\scriptsize, inner sep=3pt}] \definecolor{acc}{HTML}{2348F2} \definecolor{red}{HTML}{C0392B} % incoherent (a) \node[s] (a1) at (0,2.3) {Jane took a train to Istanbul.}; \node[s] (a2) at (0,1.2) {She likes spinach.}; \node[red, font=\scriptsize, anchor=west] at (2.9,1.75) {no relation}; \node[red, font=\scriptsize, anchor=west] at (2.9,1.3) {(a) incoherent}; \draw[red, dashed] (a1.east) -- (2.75,1.75) -- (a2.east); % coherent (b) \node[s] (b1) at (0,-0.6) {Jane took a train to Istanbul.}; \node[s] (b2) at (0,-1.7) {She had to attend a conference.}; \draw[->, acc, thick] (b2.east) to[bend right=30] node[pos=0.5, right, font=\scriptsize, text=acc] {Reason} (b1.east); \node[acc, font=\scriptsize, anchor=west] at (3.6,-1.15) {(b) coherent}; \end{tikzpicture} $$ Global coherence sits above all three: genres carry conventional macro-structure. Academic articles have Methods and Results; folktales follow plot motifs; persuasive essays state a claim and marshal premises for and against it. Coherence matters wherever text **quality** matters — essay grading, summarization (a summary should keep the central material and drop the peripheral), and even clinical measures that detect disordered language as a symptom of schizophrenia.[^jm-intro] For summarization in particular, knowing the coherence relations tells the system which spans are central and which can be cut.[^jm-relcorp] ## Coherence relations A **coherence relation** is a structured link between two spans of text. The dominant model is **Rhetorical Structure Theory (RST)**.[^jm-rst] In RST a relation holds between two spans, generally a **nucleus** and a **satellite**. The nucleus is the span central to the writer's purpose and interpretable on its own; the satellite is peripheral and interpretable only relative to the nucleus. Some symmetric relations instead hold between two nuclei. A handful of the relations, with definitions adapted from the RST Treebank manual:[^jm-rst] | Relation | Nucleus / satellite | Example | | --- | --- | --- | | Reason | NUC an action by an animate agent, SAT its reason | _Jane took a train from Paris to Istanbul._ / _She had to attend a conference._ | | Elaboration | SAT adds detail about the nucleus | _Dorothy was from Kansas._ / _She lived in the midst of the great Kansas prairies._ | | Evidence | SAT presents information to make the reader accept the nucleus | _Kevin must be here._ / _His car is parked outside._ | | Attribution | SAT gives the source of reported speech in the nucleus | _Analysts estimated_ / _that sales at U.S. stores declined in the quarter._ | | List | multinuclear series, no contrast or explicit comparison | _Billy Bones was the mate;_ / _Long John, he was quartermaster._ | Asymmetric relations are drawn graphically with an arrow **from the satellite to the nucleus**. $$ % caption: The RST Evidence relation drawn in the standard way. The satellite % (the evidence) points with an arrow to the nucleus (the claim it supports). The % nucleus stands alone; the satellite only makes sense relative to it. \begin{tikzpicture}[>=stealth, font=\footnotesize, edu/.style={draw, minimum width=38mm, minimum height=9mm, align=center, font=\scriptsize}] \definecolor{acc}{HTML}{2348F2} \node[edu] (nuc) at (0,0) {Kevin must be here. (NUC)}; \node[edu] (sat) at (5.4,0) {His car is parked outside. (SAT)}; \coordinate (apex) at (2.7,1.6); \draw[black] (nuc.north) -- (apex); \draw[->, acc, thick] (sat.north) -- (apex) node[pos=0.55, right, font=\scriptsize, text=acc] {evidence}; \end{tikzpicture} $$ The base units — the leaves of an RST analysis — are **elementary discourse units (EDUs)**, roughly clauses, also called discourse segments.[^jm-rst] Because EDUs can span arbitrary text, finding their boundaries is itself a task (below). Relations compose hierarchically: a relation can hold between an EDU and another EDU, or between an EDU and a whole span already built from smaller relations, all the way up to a single tree over the entire text. The **RST Discourse Treebank** is the largest such corpus, 385 documents from the Penn Treebank with full RST parses using 78 relations grouped into 16 classes.[^jm-relcorp] $$ % caption: An RST discourse tree over a Scientific American paragraph about Mars. % Leaves are EDUs; each satellite arrow points to its nucleus, and relations nest: % an Evidence over spans 2-9 sits under the whole, itself built from a Background % and an Elaboration, which decomposes into a List and a Contrast. \begin{tikzpicture}[>=stealth, font=\footnotesize, edu/.style={draw, minimum width=15mm, minimum height=13mm, align=center, font=\scriptsize, inner sep=1.5pt}, rel/.style={font=\scriptsize, text=acc}] \definecolor{acc}{HTML}{2348F2} % leaves \node[edu] (e2) at (0,0) {(2)\\distant\\orbit}; \node[edu] (e3) at (1.7,0) {(3)\\frigid\\weather}; \node[edu] (e4) at (3.7,0) {(4)\\avg\\-60 C}; \node[edu] (e5) at (5.4,0) {(5)\\-123 C\\at poles}; \node[edu] (e8) at (7.4,0) {(8)\\water\\evaporates}; \node[edu] (e9) at (9.1,0) {(9)\\low\\pressure}; % span 2-3 : background under 4-9? no: 2-3 background of 4-9 \coordinate (n23) at (0.85,1.4); \draw[->, acc] (e2.north) -- (n23) node[pos=0.5, left, rel] {list}; \draw[black] (e3.north) -- (n23); \coordinate (n45) at (4.55,1.4); \draw[black] (e4.north) -- (n45); \draw[->, acc] (e5.north) -- (n45) node[pos=0.5, right, rel] {list}; \coordinate (n89) at (8.25,1.4); \draw[black] (e8.north) -- (n89); \draw[->, acc] (e9.north) -- (n89) node[pos=0.5, right, rel] {contrast}; % 4-9 elaboration \coordinate (n49) at (6.4,2.7); \draw[black] (n45) -- (n49); \draw[->, acc] (n89) -- (n49) node[pos=0.5, right, rel] {elaboration}; % background 2-3 to 4-9 \coordinate (root) at (3.6,3.9); \draw[->, acc] (n23) to[out=90,in=200] node[pos=0.55, above, rel] {background} (root); \draw[black] (n49) to[out=110,in=-20] (root); \node[anchor=south, black, font=\scriptsize] at (3.6,3.95) {Mars (whole text)}; \end{tikzpicture} $$ ### The Penn Discourse Treebank A second, more lexically grounded model is the **Penn Discourse Treebank (PDTB)**.[^jm-pdtb] Rather than ask annotators to tag a relation directly, PDTB labeling starts from the surface. Annotators are given a list of **discourse connectives** — words like _because_, _although_, _when_, _since_, _as a result_ — that signal a coherence relation, and they mark the connective together with its two arguments, **Arg1** and **Arg2**. In > (22.13) _Jewelry displays in department stores were often cluttered and > uninspired. And the merchandise was, well, fake._ **As a result, marketers of > faux gems steadily lost space in department stores.** the connective _as a result_ signals a causal relation between Arg1 (the first two sentences, its antecedent) and Arg2 (the sentence it introduces, in bold). Not every relation is marked by an explicit connective. Where none appears, the annotator supplies the word that _could_ have been there and marks the relation **implicit**, then labels its sense. $$ % caption: Explicit vs. implicit relations in the PDTB. In the explicit case a % connective (as a result) links Arg1 and Arg2 on the surface. In the implicit % case no connective appears, so the annotator inserts the one that could have % (implicit=as a result) and labels the sense of the relation between the two spans. \begin{tikzpicture}[>=stealth, font=\footnotesize, arg/.style={draw, minimum width=40mm, minimum height=8mm, align=center, font=\scriptsize}, con/.style={draw, minimum width=26mm, minimum height=8mm, align=center, font=\scriptsize}] \definecolor{acc}{HTML}{2348F2} \definecolor{red}{HTML}{C0392B} % explicit \node[anchor=west, acc, font=\scriptsize] at (-0.2,2.9) {explicit}; \node[arg] (a1) at (0,2.2) {Arg1: merchandise was fake}; \node[con, draw=acc, text=acc] (c1) at (5.6,2.2) {as a result}; \node[arg] (a2) at (10.2,2.2) {Arg2: lost shelf space}; \draw[->, acc] (a1) -- (c1); \draw[->, acc] (c1) -- (a2); % implicit \node[anchor=west, red, font=\scriptsize] at (-0.2,0.9) {implicit}; \node[arg] (b1) at (0,0.2) {Arg1: EPA banned asbestos}; \node[con, draw=red, text=red, dashed] (c2) at (5.6,0.2) {(as a result)}; \node[arg] (b2) at (10.2,0.2) {Arg2: uses outlawed by 1997}; \draw[->, red, dashed] (b1) -- (c2); \draw[->, red, dashed] (c2) -- (b2); \node[anchor=north, black, font=\scriptsize] at (5.6,-0.35) {connective inserted, then sense labeled}; \end{tikzpicture} $$ PDTB senses form a four-way top level — **Temporal**, **Contingency**, **Comparison**, **Expansion** — refined into 16 types and 23 subtypes.[^jm-pdtb] One example per top class: | Class | Type | Example (connective underlined) | | --- | --- | --- | | Temporal | Synchronous | _The parishioners stop to chat at the church door,_ (implicit _while_) _as members always have._ | | Contingency | Reason | _Mr. Breeden is in a position to get somewhere with his agenda_ (implicit _because_) _he is savvy in the ways of Washington._ | | Comparison | Contrast | _The U.S. wants the removal of barriers to investment;_ _Japan denies there are barriers._ | | Expansion | Conjunction | _the actors stand outside their characters …_ _but they literally stand on their heads._ | The final corpus holds roughly 18,000 explicit and 16,000 implicit relations. Unlike RST, the PDTB annotates **only span pairs** and makes no commitment above the span-pair level — it builds no global tree.[^jm-pdtb] Corpora in this style exist for other languages too; Chinese marks only about 22% of relations with explicit connectives (against 47% for English), so its treebank maps sentence pairs directly to sense tags.[^jm-pdtb] ## Discourse structure parsing **Discourse parsing** recovers these relations automatically. For RST it runs in two stages: segment the text into EDUs, then build the tree.[^jm-seg] ### EDU segmentation The first stage, **EDU segmentation**, marks the start and end of each EDU. Since EDUs roughly correspond to clauses, early models ran a syntactic parser and post-processed its output; modern systems use a neural sequence model that predicts, for each token, whether an EDU begins there. The target labeling looks like > _[Mr. Rambo says]_ _[that a 3.2-acre property]_ _[overlooking the San Fernando > Valley]_ _[is priced at \$4 million]_ _[because the late actor Erroll Flynn once > lived there.]_ The input passes through an encoder; a linear layer and softmax over each token produce a $0/1$ sequence where $1$ marks the start of an EDU.[^jm-seg] $$ % caption: EDU segmentation as neural sequence labeling. An encoder produces one % vector per token; a linear layer and softmax emit 0 or 1 per token, with 1 % marking the start of a new elementary discourse unit. Here a break is predicted % at "that", the start of the next EDU. \begin{tikzpicture}[>=stealth, font=\footnotesize, tok/.style={minimum width=13mm, minimum height=6mm, align=center, font=\scriptsize}, cell/.style={draw, minimum width=7mm, minimum height=6mm, align=center, font=\scriptsize}, enc/.style={draw, fill=black!5, minimum width=64mm, minimum height=8mm, align=center}] \definecolor{acc}{HTML}{2348F2} \def\dx{1.7} % tokens \foreach \i/\w in {0/Mr., 1/Rambo, 2/says, 3/that} \node[tok] (t\i) at (\i*\dx,0) {\w}; \node[anchor=east, black, font=\scriptsize] at (-1.0,0) {tokens}; % encoder \node[enc] (enc) at (2.55,1.0) {ENCODER}; \foreach \i in {0,1,2,3} \draw[->, black] (t\i.north) -- (t\i.north |- enc.south); % linear + softmax outputs \foreach \i/\v in {0/0, 1/0, 2/0, 3/1} { \node[cell, draw=acc, text=acc] (o\i) at (\i*\dx,2.3) {\v}; \draw[->, acc] (\i*\dx,1.42) -- (o\i.south); } \node[anchor=east, acc, font=\scriptsize] at (-1.0,2.3) {EDU break}; \node[anchor=west, black, font=\scriptsize] at (3*\dx+0.6,2.3) {1 = new EDU}; \end{tikzpicture} $$ ### Building the RST tree With EDUs in hand, an RST **parser** assembles the tree. Modern parsers are **shift-reduce** transition systems built on a neural encoder.[^jm-rstparse] The parser state is a **stack** and a **queue** of EDUs, and it produces the tree by a sequence of actions: - **shift** — push the front EDU of the queue onto the stack as a single-node subtree. - **reduce**$(l, d)$ — merge the top two subtrees on the stack, labeling the coherence relation $l$ and the nuclearity direction $d \in \{\text{NN}, \text{NS}, \text{SN}\}$ (nucleus-nucleus, nucleus-satellite, satellite-nucleus). - **pop-root** — remove the final finished tree from the stack. The parser of Yu et al. encodes the input with a **hierarchical biLSTM**: a word-level biLSTM builds a representation of the words inside each EDU (average-pooled to one vector per EDU), and an EDU-level biLSTM contextualizes the EDU sequence. A feedforward decoder then chooses each action from a concatenation of the top three stack subtrees $s_0, s_1, s_2$ and the front-of-queue EDU $q_0$,[^jm-rstparse] $$ \mathbf{o} = \mathbf{W}\big[\, \mathbf{h}^{t}_{s_0},\; \mathbf{h}^{t}_{s_1},\; \mathbf{h}^{t}_{s_2},\; \mathbf{h}^{e}_{q_0} \,\big], $$ where each subtree vector $\mathbf{h}^{t}$ is the average pool over that subtree's EDU encodings. A softmax over $\mathbf{o}$ gives the action distribution. The trace below builds a four-EDU tree. $$ % caption: A shift-reduce trace building an RST tree over four EDUs. Each row is a % parser state (stack and queue) and the action taken: SH shifts the front EDU, RD % reduces the top two subtrees with a labeled, directed relation, PR pops the % finished root. The tree it builds attaches e1:e2 by attribution then elaborates. \begin{tikzpicture}[>=stealth, font=\footnotesize, row/.style={font=\scriptsize}] \definecolor{acc}{HTML}{2348F2} \node[row, anchor=west] at (0,3.6) {\textbf{stack}}; \node[row, anchor=west] at (3.7,3.6) {\textbf{queue}}; \node[row, anchor=west] at (7.2,3.6) {\textbf{action}}; \draw[black] (-0.1,3.35) -- (10.4,3.35); \node[row, anchor=west] at (0,3.0) {(empty)}; \node[row, anchor=west] at (3.7,3.0) {e1 e2 e3 e4}; \node[row, anchor=west, text=acc] at (7.2,3.0) {SH}; \node[row, anchor=west] at (0,2.4) {e1}; \node[row, anchor=west] at (3.7,2.4) {e2 e3 e4}; \node[row, anchor=west, text=acc] at (7.2,2.4) {SH}; \node[row, anchor=west] at (0,1.8) {e1 e2}; \node[row, anchor=west] at (3.7,1.8) {e3 e4}; \node[row, anchor=west, text=acc] at (7.2,1.8) {RD(attr, SN)}; \node[row, anchor=west] at (0,1.2) {e1:2}; \node[row, anchor=west] at (3.7,1.2) {e3 e4}; \node[row, anchor=west, text=acc] at (7.2,1.2) {SH SH}; \node[row, anchor=west] at (0,0.6) {e1:2 e3 e4}; \node[row, anchor=west] at (3.7,0.6) {(empty)}; \node[row, anchor=west, text=acc] at (7.2,0.6) {RD(elab, NS)}; \node[row, anchor=west] at (0,0.0) {e1:2 e3:4}; \node[row, anchor=west] at (3.7,0.0) {(empty)}; \node[row, anchor=west, text=acc] at (7.2,0.0) {RD(elab, SN)}; \node[row, anchor=west] at (0,-0.6) {e1:4}; \node[row, anchor=west] at (3.7,-0.6) {(empty)}; \node[row, anchor=west, text=acc] at (7.2,-0.6) {PR}; \end{tikzpicture} $$ Training maps each gold tree to its **oracle** action sequence and fits the decoder with cross-entropy (plus $\ell_2$ regularization):[^jm-rstparse] $$ p_a = \frac{\exp(\mathbf{o}_a)}{\sum_{a' \in A} \exp(\mathbf{o}_{a'})}, \qquad L_{\text{CE}} = -\log p_a + \frac{\lambda}{2}\,\|\Theta\|^2 . $$ RST parsers report four F1 scores of increasing strictness, computed after binarizing the gold trees: **Span** (bare tree, no labels), **Nuclearity** (with nucleus/satellite marks), **Relation** (with relation labels), and **Full** (both).[^jm-rstparse] ### PDTB relation classification **PDTB parsing** is shallower — flat relations, no tree — hence also called **shallow discourse parsing**. It splits into four subtasks:[^jm-pdtbparse] 1. Find the discourse connectives, disambiguating _and_ the connective from _and_ the NP conjunction. 2. Find each connective's two argument spans. 3. Label the sense of each explicit relation. 4. Assign a relation between every adjacent pair of sentences (the implicit case). Task 4 is the hardest and most studied: take a pair of adjacent sentences and predict a sense label, usually one of the 11 common second-level PDTB tags. A simple but strong baseline encodes each span with BERT, takes the `[CLS]` hidden state, and passes it through a one-layer tanh feedforward net and softmax for sense classification.[^jm-pdtbparse] Task 1 is a special case of word-sense disambiguation: _and_ is a discourse connective in > (22.24) _Selling picked up as previous buyers bailed out of their positions_ **and** > _aggressive short sellers moved in._ but a non-discourse NP conjunction in _My favorite colors are blue **and** green._ Similarly _once_ marks a temporal relation in one context but merely means "formerly" in _a form of asbestos once used to make cigarette filters._ Modern systems perform task 1 end-to-end with a biLSTM-CRF that emits `B-CONN`/`I-CONN`/`O` tags, and use the same BERT sequence models for task 2's span-finding.[^jm-pdtbparse] ## Where this continues We have the first source of coherence: **relations** between clauses. Rhetorical Structure Theory ties spans as nucleus and satellite; the Penn Discourse Treebank grounds the same links in explicit and implicit connectives; and discourse parsers recover both — a shift-reduce transition system that builds the RST tree from EDUs, and a shallow classifier that labels the sense of each adjacent-sentence pair. Relational coherence is only one of three local sources, and it says nothing about the global structure a genre imposes. Being **about** stable entities, sharing a **topic**, and obeying a narrative or argumentative macro-structure continue in [entity-based and global coherence](/natural-language-processing/linguistic-structure/entity-based-and-global-coherence), which also gathers the neural models that learn each. [^jm-intro]: **Jurafsky & Martin**, _Speech and Language Processing_ (3rd ed.), Ch. 22 intro — Discourse Coherence: discourse and coherence; the Hobbs spinach/conference minimal pair; local coherence in three forms (relational coherence relations, entity-based coherence, topical coherence via lexical cohesion) versus global coherence; the roles of coherence in essay grading, summarization, and detecting disordered language. [^jm-entity]: **Jurafsky & Martin**, Ch. 22 intro; §22.3 — entity-based coherence: the incoherent passage that swings among John, Jenny, the piano store, and the living room; a discourse is locally coherent by being "about" salient entities that do not lurch. [^jm-relcorp]: **Jurafsky & Martin**, §22.1.1 — the RST Discourse Treebank (Carlson et al. 2001): 385 documents from the Penn Treebank with full RST parses, 78 relations in 16 classes; the observation that nuclei carry more important information than satellites, so a summary can drop satellites. [^jm-rst]: **Jurafsky & Martin**, §22.1.1 — Rhetorical Structure Theory (Mann and Thompson 1987): nucleus/satellite relations (Reason, Elaboration, Evidence, Attribution, List); satellite-to-nucleus arrows; elementary discourse units (EDUs), also called discourse segments; hierarchical composition into a tree. [^jm-pdtb]: **Jurafsky & Martin**, §22.1.2 — Penn Discourse TreeBank (Prasad et al.): lexically grounded annotation via discourse connectives linking Arg1 and Arg2; explicit versus implicit relations, where the annotator inserts the connective that could have appeared; the four-class sense hierarchy (Temporal, Contingency, Comparison, Expansion) with 16 types and 23 subtypes; ~18,000 explicit and 16,000 implicit relations; no commitment above the span-pair level; the Chinese Discourse TreeBank and its 22% explicit rate. [^jm-seg]: **Jurafsky & Martin**, §22.2.1 — EDU segmentation for RST parsing: early syntactic-parser-plus-postprocessing versus modern neural sequence models (simplified from Lukasik et al. 2020); the encoder-plus-linear-plus-softmax predicting a 0/1 sequence where 1 marks an EDU start. [^jm-rstparse]: **Jurafsky & Martin**, §22.2.2 — RST parsing: the shift-reduce parser of Yu et al. (2018) with shift, reduce($l$,$d$) over $d \in \{$NN,NS,SN$\}$, and pop-root; the hierarchical biLSTM word- and EDU-level encoder with average pooling; the feedforward decoder over the top three stack subtrees and the front-of-queue EDU; oracle-action training with cross-entropy and $\ell_2$ regularization; the Span/Nuclearity/Relation/Full RST-Pareval metrics on binarized trees. [^jm-pdtbparse]: **Jurafsky & Martin**, §22.2.3 — PDTB (shallow) discourse parsing: the four subtasks (find connectives, find their spans, label explicit senses, label implicit adjacent-sentence relations) from Lin et al. (2014); the BERT `[CLS]` plus one-layer feedforward baseline for implicit sense classification (Nie et al. 2019); connective disambiguation as word-sense disambiguation (_and_, _once_) with a biLSTM-CRF over B-CONN/I-CONN/O tags.