---
title: "Entity-Based and Global Coherence"
module: Linguistic Structure
moduleNumber: 6
lessonNumber: 18
order: 618
summary: >
  A text coheres not only through relations between clauses but by staying about
  the same entities and the same topic, and by obeying the macro-structure of its
  genre. This lesson develops Centering Theory and the entity grid for entity-based
  coherence, representation-learning models of local coherence, and global coherence —
  topic segmentation, narrative and argumentation structure, and scientific discourse —
  then the neural models that learn each.
topics: [Structure]
sources:
  - book: Jurafsky
    ref: "§22.3 Centering and Entity-Based Coherence; §22.4 Representation Learning Models for Local Coherence; §22.5 Global Coherence"
---

This builds on [discourse coherence: relations and parsing](/natural-language-processing/linguistic-structure/discourse-coherence),
which tied clauses together with coherence relations (RST and the PDTB) and
recovered them with discourse parsers. Relational coherence is one of three local
sources; here we take up the other two — being **about** stable entities and
sharing a **topic** — and then the global structure a whole genre imposes.

## Centering and entity-based coherence

The second flavor of coherence is being **about** something. A discourse coheres by
keeping the same salient entities in focus rather than swinging among them. Grosz et
al.'s minimal pair makes the point with two passages of identical propositional
content but different **saliences**:[^jm-center]

> (i) John went to his favorite music store to buy a piano. He had frequented the
> store for years. He was excited that he could finally buy a piano. He arrived just
> as the store was closing.
>
> (ii) John went to his favorite music store to buy a piano. It was a store John had
> frequented for years. He was excited that he could finally buy a piano. It was
> closing just as John arrived.

Both are grammatical, but (i) is more coherent: it is steadily _about John_, whereas
(ii) swings John / store / John / store and lacks a stable "aboutness."[^jm-center]

**Centering Theory** formalizes this as a theory of both salience and coherence. Each
utterance $U_n$ carries two structures:[^jm-center]

- The **backward-looking center** $C_b(U_n)$: the single entity currently in focus,
  the one being centered on.
- The **forward-looking centers** $C_f(U_n)$: the set of entities $U_n$ evokes, any
  of which could be the center of the next utterance, **ranked** by grammatical role
  (subject > object > everything else).

The top-ranked forward center is the **preferred center** $C_p(U_n)$ — a prediction
of what the next utterance will be about. The relationship between successive
utterances falls into four **transition types**, classified by two questions: does
$C_b$ stay the same from $U_n$ to $U_{n+1}$, and does the new $C_b$ equal the
predicted $C_p$?

$$
% caption: The four Centering transitions of Brennan et al., classified by whether
% the backward-looking center stays the same across utterances (columns) and
% whether it equals the predicted preferred center (rows). Continue is the most
% coherent, Rough-Shift the least; Rule 2 prefers them in that order.
\begin{tikzpicture}[>=stealth, font=\footnotesize,
  cell/.style={draw, minimum width=36mm, minimum height=11mm, align=center, font=\scriptsize}]
  \definecolor{acc}{HTML}{2348F2}
  \definecolor{red}{HTML}{C0392B}
  \node[anchor=south, font=\scriptsize, align=center] at (2.1,1.5)  {Cb(n+1) = Cb(n)\\(or Cb(n) undef)};
  \node[anchor=south, font=\scriptsize, align=center] at (6.3,1.5)  {Cb(n+1) != Cb(n)};
  \node[anchor=east, font=\scriptsize, align=right] at (-0.3,0.5)  {Cb(n+1) = Cp(n+1)};
  \node[anchor=east, font=\scriptsize, align=right] at (-0.3,-1.1) {Cb(n+1) != Cp(n+1)};
  \node[cell, draw=acc, text=acc] (c11) at (2.1,0.5)   {Continue};
  \node[cell] (c12) at (6.3,0.5)   {Smooth-Shift};
  \node[cell] (c21) at (2.1,-1.1)  {Retain};
  \node[cell, draw=red, text=red] (c22) at (6.3,-1.1)  {Rough-Shift};
  \node[anchor=west, black, font=\scriptsize] at (-0.4,-2.3) {preference: Continue $>$ Retain $>$ Smooth-Shift $>$ Rough-Shift};
\end{tikzpicture}
$$

Two rules drive the algorithm:[^jm-center]

> **Rule 1 (Pronominalization).** If any element of $C_f(U_n)$ is realized by a
> pronoun in $U_{n+1}$, then $C_b(U_{n+1})$ must be realized by a pronoun too. Since
> pronouns mark the most salient entity, if only one thing is pronominalized it must
> be the center.

> **Rule 2 (Transition ordering).** Transitions are preferred in the order Continue
> $>$ Retain $>$ Smooth-Shift $>$ Rough-Shift. A discourse that keeps centering on
> the same entity is more coherent than one that repeatedly shifts.

### A worked centering trace

Run the algorithm on the coherent passage (i), taking its first two utterances. Rank
each $C_f$ by grammatical role.

$$
% caption: A worked Centering trace over the first two utterances of the coherent
% piano passage. Forward centers are ranked subject-first; Cp is the top-ranked
% one. U1 has undefined Cb (nothing precedes it); U2 keeps Cb = John = Cp, so the
% transition is Continue, the most coherent type.
\begin{tikzpicture}[>=stealth, font=\footnotesize, row/.style={font=\scriptsize}]
  \definecolor{acc}{HTML}{2348F2}
  \node[row, anchor=west, align=left] at (0,3.6)
    {U1: John went to his favorite music store to buy a piano.};
  \node[row, anchor=west] at (0.5,3.0)  {Cf(U1) = John, store, piano};
  \node[row, anchor=west] at (0.5,2.5)  {Cp(U1) = John};
  \node[row, anchor=west] at (0.5,2.0)  {Cb(U1) = undef\/ined};
  \draw[black] (-0.1,1.65) -- (9.3,1.65);
  \node[row, anchor=west, align=left] at (0,1.2)
    {U2: He was excited that he could f\/inally buy a piano.};
  \node[row, anchor=west] at (0.5,0.6)  {Cf(U2) = John, piano};
  \node[row, anchor=west] at (0.5,0.1)  {Cp(U2) = John};
  \node[row, anchor=west] at (0.5,-0.4) {Cb(U2) = John};
  \node[row, anchor=west, text=acc] at (0.5,-1.05)
    {Result: Cb(U2) = Cp(U2), Cb(U1) undef\/ined  $\Rightarrow$  CONTINUE};
\end{tikzpicture}
$$

Because $C_b(U_2) = C_p(U_2) = \text{John}$ and $C_b(U_1)$ is undefined, the
transition is a Continue — the most coherent type. Passage (i) is a chain of
Continues on John; passage (ii), which realizes John and the store to keep swapping
which is the subject, forces Shifts, and those Shifts are why it reads worse.

### The entity grid

Centering is a **top-down** theory: it stipulates that salient entities appear as
subjects or pronouns and that continuing on them is coherent. The **entity grid**
model of Barzilay and Lapata learns the same intuition **bottom-up**, inducing the
patterns of entity mention that make a discourse coherent.[^jm-grid]

An entity grid is a two-dimensional array: one **row per sentence**, one **column per
discourse entity**. Each cell records the entity's grammatical role in that sentence —
**S**ubject, **O**bject, **X** (neither), or **–** (absent). An entity appearing more
than once in a sentence takes its highest-ranked role. Because the columns are
discourse entities, building a grid requires coreference resolution to cluster
mentions and parsing to read off grammatical roles — so this stage depends directly on
[coreference](/natural-language-processing/linguistic-structure/coreference-and-discourse).

$$
% caption: An entity grid fragment for a text about a Microsoft antitrust case.
% Rows are sentences, columns entities, cells the grammatical role: S subject, O
% object, X neither, dash absent. Dense columns (Microsoft) mark salient entities;
% a length-2 column slice like [S -] is a local transition used as a feature.
\begin{tikzpicture}[>=stealth, font=\footnotesize]
  \definecolor{acc}{HTML}{2348F2}
  \def\cw{1.75}  \def\rh{0.6}
  % headers
  \foreach \c/\name in {1/Dept, 2/Trial, 3/Microsoft, 4/Netscape, 5/Suit}
    \node[font=\scriptsize] at (\c*\cw, 0.55) {\name};
  \foreach \r in {1,...,6} \node[font=\scriptsize, anchor=east] at (0.55,-\r*\rh) {S\r};
  % rows
  \foreach \c/\v in {1/S, 2/O, 3/S, 4/-, 5/-}  \node[font=\scriptsize] at (\c*\cw,-1*\rh) {\v};
  \foreach \c/\v in {1/-, 2/-, 3/O, 4/-, 5/-}  \node[font=\scriptsize] at (\c*\cw,-2*\rh) {\v};
  \foreach \c/\v in {1/-, 2/-, 3/S, 4/O, 5/-}  \node[font=\scriptsize] at (\c*\cw,-3*\rh) {\v};
  \foreach \c/\v in {1/-, 2/-, 3/S, 4/-, 5/-}  \node[font=\scriptsize] at (\c*\cw,-4*\rh) {\v};
  \foreach \c/\v in {1/-, 2/-, 3/-, 4/-, 5/O}  \node[font=\scriptsize] at (\c*\cw,-5*\rh) {\v};
  \foreach \c/\v in {1/-, 2/X, 3/S, 4/-, 5/-}  \node[font=\scriptsize] at (\c*\cw,-6*\rh) {\v};
  % highlight dense Microsoft column
  \draw[acc, thick] (3*\cw-0.6,-6*\rh-0.3) rectangle (3*\cw+0.6,-1*\rh+0.3);
  \node[acc, anchor=west, font=\scriptsize] at (5*\cw+0.6,-3.5*\rh) {dense = salient};
  \draw[->, acc] (5*\cw+0.55,-3.5*\rh) -- (3*\cw+0.65,-3.5*\rh);
\end{tikzpicture}
$$

Coherence is read off the **local entity transitions** — a length-2 vertical slice of
a column, like $[\text{S}\ \text{–}]$ (subject then absent) or $[\text{O}\
\text{X}]$. There are $4^2 = 16$ possible transitions, and each document becomes a
feature vector of their **probabilities**: the fraction of column-slices that take
each value. That vector feeds a machine-learning classifier.
Coherent texts have a characteristic topology — a few **dense** columns (entities
mentioned throughout, often as S or O) and many **sparse** ones.[^jm-grid]

To see the vector come out of the grid, count transitions column by column. Each
column of $r$ rows yields $r - 1$ length-2 slices; a $6$-row, $5$-column grid gives
$5 \times 5 = 25$ slices total. Slide down the Microsoft column
$[\text{S},\text{O},\text{S},\text{S},\text{–},\text{S}]$ and read off its five
adjacent pairs: $[\text{S}\,\text{O}]$, $[\text{O}\,\text{S}]$, $[\text{S}\,\text{S}]$,
$[\text{S}\,\text{–}]$, $[\text{–}\,\text{S}]$. Do the same for every column and
divide each transition's count by $25$.

$$
% caption: Reading the entity-grid feature vector off the grid. Each column of six
% rows contributes five adjacent role pairs; over 25 pairs total, the fraction
% taking each of the 16 possible values is one feature. The dense Microsoft column
% supplies most of the S-involving transitions; the sparse columns supply the many
% dash-dash pairs.
\begin{tikzpicture}[>=stealth, font=\footnotesize,
  bar/.style={draw=acc, fill=acc!18}]
  \definecolor{acc}{HTML}{2348F2}
  \draw[->, black] (0,0) -- (0,2.6) node[above, font=\scriptsize] {probability};
  \draw[->, black] (0,0) -- (7.6,0) node[right, font=\scriptsize] {transition type};
  \foreach \x/\h/\lab in {0.5/0.44/dd, 1.5/0.16/Sd, 2.5/0.12/dS, 3.5/0.08/SS, 4.5/0.08/SO, 5.5/0.08/OS, 6.5/0.04/oth} {
    \fill[acc!18, draw=acc] (\x-0.28,0) rectangle (\x+0.28,\h*5);
    \node[font=\scriptsize, anchor=north] at (\x,-0.05) {\lab};
  }
  \node[anchor=west, font=\scriptsize, black] at (2.6,2.2)
    {dd = both absent; Sd = subject-then-absent};
\end{tikzpicture}
$$

The vector is dominated by the $[\text{–}\,\text{–}]$ transition — most
entity-sentence cells are empty — with a long tail of role-bearing transitions.
Barzilay and Lapata found that giving different weights to salient versus
non-salient entities (split by mention frequency) and adding the coreference-based
columns beats a bag-of-words baseline on the ordering tasks below; the grid captures
something a word-overlap model cannot.[^jm-grid]

## Evaluating and training coherence models

Labels for coherence are expensive, so most entity-grid and neural models train and
evaluate one of two ways.[^jm-eval] The first uses **human ratings**: have annotators
score each document (high/mid/low, or continuous) and fit a classifier to predict
those scores — best when an end task like essay grading fixes what the label should
mean. The second exploits **self-supervision**: a naturally ordered discourse is more
coherent than a random permutation of its sentences, so a good model should prefer
the original. This gives three concrete tasks:[^jm-eval]

- **Sentence-order discrimination** — rank the original document above a random
  permutation of its sentences.
- **Sentence insertion** — remove one sentence and find its original slot among the
  $n$ positions. Harder than discrimination, since the compared documents differ in
  just one sentence.
- **Sentence-order reconstruction** — shuffle a document and put the sentences back
  in the correct order. Hardest of the three, being a reordering rather than a
  classification.

## Representation-learning models for local coherence

The third local flavor — **topical** coherence — rests on **lexical cohesion**:
discourses cohere by discussing the same topics with words from the same semantic
fields.[^jm-repl] A line of unsupervised models from the 1990s measured this directly.
**Lexical chains** track words in a discourse that are related through a thesaurus
(_pine_, _bush_, _trees_, _trunk_); their number and density correlate with the topic
structure. **TextTiling** computes the cosine between vectors of raw word counts in
neighboring text spans, high within a subtopic and low across a subtopic boundary
(we use it for segmentation below). **LSA coherence** was the first to use embeddings:
it represents a sentence $s$ by summing its word vectors and scores overall coherence
as the average cosine between adjacent sentences,[^jm-repl]

$$
\text{sim}(s, t) = \cos\!\Big( \sum_{w \in s}\mathbf{w},\; \sum_{w \in t}\mathbf{w} \Big),
\qquad
\text{coherence}(T) = \frac{1}{n-1}\sum_{i=1}^{n-1} \cos(s_i, s_{i+1}).
$$

Modern neural models keep the "how much do neighboring sentences change" intuition
but add self-supervision, training on natural-vs-scrambled orderings to discover the
features that matter for coherence. One such model is the **local coherence
discriminator (LCD)**.[^jm-repl] LCD scores a text as the average coherence over
consecutive sentence pairs. It embeds a pair $(s, t)$ with any sentence encoder and
concatenates four combinations — the two vectors, their difference $\mathbf{s} -
\mathbf{t}$, its absolute value $|\mathbf{s} - \mathbf{t}|$, and the element-wise
product $\mathbf{s} \odot \mathbf{t}$ — then passes them through a one-layer
feedforward net to a coherence score.

$$
% caption: The local coherence discriminator (LCD). Two sentence encoders produce
% vectors s and t; the model concatenates s, t, their difference, and their
% element-wise product, then a one-layer MLP outputs a coherence score. It is
% trained so true consecutive pairs score above pairs with a random second sentence.
\begin{tikzpicture}[>=stealth, font=\footnotesize,
  enc/.style={draw, fill=black!5, minimum width=32mm, minimum height=8mm, align=center, font=\scriptsize},
  box/.style={draw, minimum width=42mm, minimum height=8mm, align=center, font=\scriptsize}]
  \definecolor{acc}{HTML}{2348F2}
  \node[enc] (es) at (0,0)   {encode s (sentence i)};
  \node[enc] (et) at (5.0,0) {encode t (sentence i+1)};
  \node[box] (feat) at (2.5,1.4) {[ s, t, s-t, s (*) t ]};
  \draw[->, black] (es.north) -- (feat.south);
  \draw[->, black] (et.north) -- (feat.south);
  \node[box, draw=acc, text=acc] (mlp) at (2.5,2.8) {one-layer MLP};
  \draw[->, acc] (feat.north) -- (mlp.south);
  \node[box, draw=acc, text=acc] (score) at (2.5,4.1) {coherence score};
  \draw[->, acc] (mlp.north) -- (score.south);
\end{tikzpicture}
$$

Training makes true consecutive pairs $(s_i, s_{i+1})$ score above negative pairs
$(s_i, s')$ with $s'$ a sentence sampled uniformly from elsewhere in the same
document. With a margin loss $\ell(f^+, f^-) = \max(0,\, \eta - f^+ + f^-)$ over
positive $f^+ = f_\theta(s_i, s_{i+1})$ and negative $f^- = f_\theta(s_i, s')$, the
objective is[^jm-repl]

$$
L_\theta = \sum_{d \in C}\sum_{s_i \in d}\; \mathbb{E}_{p(s' \mid s_i)}\big[\, \ell\big(f_\theta(s_i, s_{i+1}),\; f_\theta(s_i, s')\big) \,\big].
$$

A related predictability baseline trains an RNN language model and measures how much
the preceding context raises the log-likelihood of $s_i$ (conditional minus marginal
log-likelihood) — a coherence signal for free. Pushing beyond consecutive pairs helps:
a Transformer trained with a contrastive objective to predict text up to $\pm 2$
sentences away yields stronger discourse representations still.[^jm-repl]

## Global coherence

A discourse must cohere **globally**, not just between neighboring sentences. Two
kinds of global structure show up: the topic organization of a text, and the
conventional macro-structure of a genre.

### Topic segmentation

Long documents move through subtopics, and finding the boundaries is **topic
segmentation**. The classic unsupervised method is **TextTiling**: slide a window
across the text, represent the block of sentences on each side of a candidate gap as a
word-count vector, and compute their cosine.[^jm-repl] Similarity stays high inside a
subtopic (neighbors share vocabulary) and dips at a subtopic boundary. Boundaries are
placed at the deep valleys of the similarity-versus-position curve.

$$
% caption: TextTiling for topic segmentation. The curve is the lexical similarity
% (cosine of word-count vectors) between the text on either side of each gap
% between sentences. High plateaus are coherent subtopics; the deep valley marks a
% topic boundary, placed where neighboring blocks share the least vocabulary.
\begin{tikzpicture}[>=stealth, font=\footnotesize]
  \definecolor{acc}{HTML}{2348F2}
  \definecolor{red}{HTML}{C0392B}
  % axes
  \draw[->, black] (0,0) -- (9.2,0) node[right, font=\scriptsize, black] {gap position};
  \draw[->, black] (0,0) -- (0,3.3) node[above, font=\scriptsize, black] {similarity};
  % similarity curve: high, dips low in the middle, high again
  \draw[acc, thick]
    (0.3,2.6) .. controls (1.2,2.7) and (2.0,2.5) .. (2.8,2.4)
    .. controls (3.4,2.3) and (3.8,0.7) .. (4.4,0.5)
    .. controls (5.0,0.7) and (5.4,2.3) .. (6.0,2.5)
    .. controls (6.9,2.6) and (7.9,2.5) .. (8.8,2.6);
  % boundary marker at the valley
  \draw[red, dashed, thick] (4.4,0) -- (4.4,3.0);
  \node[red, anchor=south, font=\scriptsize] at (4.4,3.0) {topic boundary};
  \node[acc, anchor=west, font=\scriptsize] at (1.0,2.85) {subtopic A};
  \node[acc, anchor=west, font=\scriptsize] at (6.2,2.85) {subtopic B};
\end{tikzpicture}
$$

### Discourse macro-structure: schemas

Beyond topic flow, whole genres impose conventional structure. The oldest studied
case is narrative: in his _Morphology of the Folktale_, Propp models Russian
folktales with a plot grammar of **dramatis personae** (character roles like Hero,
Villain, Donor, Helper) and **functions** (events like "Villain commits kidnapping,"
"Hero is pursued") that occur in a fixed order.[^jm-global] Every tale is a subset of
the functions in that order — a discourse grammar of stories. Recent work induces such
Proppian functions from folktale corpora, and learns latent personae from movie plot
summaries using features like the actions a character takes (Villains _strangle_),
what is done to them (Villains are _foiled_), and how they are described (Villains are
_evil_).[^jm-global]

### Argumentation structure

A second, widely studied kind of global structure is the structure of **arguments**;
analyzing it computationally is called **argumentation mining**.[^jm-arg] Aristotle
named three appeals — **pathos** (emotion), **ethos** (character), and **logos**
(logic) — and most computational work targets logos. Persuasive-essay corpora annotate
argumentative components: **claims** (the controversial central assertions that need
support) and **premises** (reasons offered to support or attack a claim), plus the
**argumentative relations** between them, `SUPPORT` and `ATTACK`. In the essay

> "(1) **Museums and art galleries provide a better understanding of arts than
> Internet.** (2) In most museums and art galleries, detailed descriptions of the
> background, history and author are provided. (3) Seeing an artwork online is not
> the same as watching it with our own eyes, as (4) the picture online does not show
> the texture or three-dimensional structure of the art."

sentence (1) is the claim, (2) and (3) are premises supporting it, and (4) is a
premise supporting (3): three argumentative relations, `SUPPORT(2,1)`, `SUPPORT(3,1)`,
`SUPPORT(4,3)`.

$$
% caption: The argumentation structure of a persuasive essay. A single major claim
% is supported (arrowhead) or attacked (circle head) by claims, which premises
% (P) in turn support. Unlike coherence relations, argument structure is not
% local: one claim governs premises spread across the whole essay.
\begin{tikzpicture}[>=stealth, font=\footnotesize,
  claim/.style={draw, minimum width=17mm, minimum height=8mm, align=center, font=\scriptsize},
  prem/.style={draw, minimum width=9mm, minimum height=6mm, align=center, font=\scriptsize}]
  \definecolor{acc}{HTML}{2348F2}
  \definecolor{red}{HTML}{C0392B}
  \node[claim, draw=acc, text=acc, minimum width=90mm] (major) at (3.6,3.2) {Major Claim};
  \node[claim] (c1) at (0,1.5)  {Claim 1 (for)};
  \node[claim] (c2) at (3.0,1.5) {Claim 2 (for)};
  \node[claim] (c3) at (6.2,1.5) {Claim 3 (against)};
  \draw[->, acc] (c1.north) -- (c1.north |- major.south);
  \draw[->, acc] (c2.north) -- (c2.north |- major.south);
  \draw[red] (c3.north) -- ++(0,0.55);
  \fill[red] (c3.north)++(0,0.65) circle (1.6pt);
  % premises
  \node[prem] (p1) at (2.3,0.1) {P1};
  \node[prem] (p2) at (3.7,0.1) {P2};
  \node[prem] (p3) at (5.5,0.1) {P6};
  \draw[->, acc] (p1.north) -- (p1.north |- c2.south);
  \draw[->, acc] (p2.north) -- (p2.north |- c2.south);
  \draw[->, acc] (p3.north) -- (p3.north |- c3.south);
\end{tikzpicture}
$$

Detection systems combine a component classifier (claim / premise / non-argument)
with a relation classifier (`SUPPORT` / `ATTACK` / neither).[^jm-arg] Richer work
labels **argumentation schemes** — larger patterns like _argument from example_ or
_argument from cause to effect_ — and studies which features make an argument
**persuasive**, connecting logos back to Aristotle's ethos and pathos through social
cues like reciprocity, authority, and scarcity.[^jm-arg] Argument structure is
markedly **less local** than coherence relations: a persuasive essay has one main
claim with premises scattered throughout, without the adjacent-clause linking of RST.

### Scientific discourse structure

Scientific papers carry their own conventional macro-structure: somewhere the authors
state a goal, develop a method, present evidence, and compare to prior work.[^jm-arg]
The **argumentative zoning** scheme captures these rhetorical roles by assigning each
sentence one of 15 labels — `AIM`, `OWN_METHOD`, `OWN_RESULTS`, `USE`, `GAP_WEAK`,
`SUPPORT`, `ANTISUPPORT`, and others — reflecting the idea that a paper makes a
**knowledge claim** and positions it against the field. Labeled corpora of
computational-linguistics and chemistry articles supervise a standard
sentence-classification model to assign the tags.[^jm-arg]

## Neural discourse parsing and coherence

The models above are the classical backbone; the last decade reshaped each of them
around learned representations. Four public lines of work stand out.

**Representation learning for RST.** Ji and Eisenstein (2014), _"Representation
Learning for Text-level Discourse Parsing"_ (ACL), replaced hand-built lexical and
syntactic features with learned vector representations of each discourse span,
projecting the surface features into a latent space jointly with the parsing
objective (their DPLP parser). Learned span vectors, rather than sparse features,
have driven RST parsing accuracy ever since and are the reason the
[shift-reduce RST parser](/natural-language-processing/linguistic-structure/discourse-coherence)
of the companion lesson uses a hierarchical biLSTM to encode EDUs.[^jieisen]

**Top-down parsing with pointer networks.** That shift-reduce parser builds the tree
bottom-up. Lin et al. (2019) and Kobayashi et al. (2020) showed a **top-down**
alternative: a pointer-network decoder repeatedly splits a span by pointing at the
boundary where it divides into two subtrees, recursing until it reaches EDUs. Top-down
splitting sees the whole span before committing to a division, which helps on the
long-range attachments that trip a greedy bottom-up parser, and these systems set
strong numbers on the RST Discourse Treebank.[^topdown]

$$
% caption: Bottom-up vs. top-down RST parsing. The shift-reduce parser merges the
% smallest spans first and grows the tree upward; a pointer-network parser starts
% from the whole span and points at each split boundary, recursing down to the EDUs.
% Top-down sees the full span before choosing where to divide it.
\begin{tikzpicture}[>=stealth, font=\footnotesize,
  box/.style={draw, minimum width=8mm, minimum height=6mm, font=\scriptsize}]
  \definecolor{acc}{HTML}{2348F2}
  % bottom-up
  \node[font=\scriptsize, anchor=west] at (-0.5,2.3) {bottom-up (shift-reduce)};
  \foreach \i in {1,...,4} \node[box] (bu\i) at (\i*0.9,0) {e\i};
  \coordinate (bu12) at (1.35,0.8);
  \coordinate (bu34) at (3.15,0.8);
  \coordinate (bur) at (2.25,1.6);
  \draw[black] (bu1.north) -- (bu12) -- (bu2.north);
  \draw[black] (bu3.north) -- (bu34) -- (bu4.north);
  \draw[acc, thick] (bu12) -- (bur) -- (bu34);
  \node[font=\scriptsize, text=acc, anchor=west] at (3.7,1.4) {merge upward};
  % top-down
  \begin{scope}[xshift=7.2cm]
    \node[font=\scriptsize, anchor=west] at (-0.5,2.3) {top-down (pointer split)};
    \foreach \i in {1,...,4} \node[box] (td\i) at (\i*0.9,0) {e\i};
    \coordinate (td12) at (1.35,0.8);
    \coordinate (td34) at (3.15,0.8);
    \coordinate (tdr) at (2.25,1.6);
    \draw[acc, thick] (tdr) -- (td12);
    \draw[acc, thick] (tdr) -- (td34);
    \draw[black] (td12) -- (td1.north);
    \draw[black] (td12) -- (td2.north);
    \draw[black] (td34) -- (td3.north);
    \draw[black] (td34) -- (td4.north);
    \node[font=\scriptsize, text=acc, anchor=west] at (3.7,1.4) {split downward};
  \end{scope}
\end{tikzpicture}
$$

**Implicit relations under pretrained models.** Task 4 of shallow parsing — labeling
the implicit sense of an adjacent sentence pair — was for years the field's hardest
number. Feeding both arguments to a pretrained masked language model and classifying
off the pooled representation (the BERT `[CLS]` baseline the
[companion lesson](/natural-language-processing/linguistic-structure/discourse-coherence)
describes, from Nie et al. 2019) jumped implicit-relation accuracy well past the feature-engineered
systems that preceded it, and later work adds the connective as an auxiliary
prediction target so the model learns the word that _could_ have appeared. Implicit
relation recognition is still open — the four-way top-level split is far from
solved — but pretrained encoders reset the baseline.[^plmimpl]

**Neural coherence models and their benchmark.** Nguyen and Joty (2017) gave the
entity grid a neural form, running a **convolutional network** over the grid so the
model learns which local transition patterns matter rather than enumerating all
sixteen, and it beat the classic grid on the ordering tasks. More broadly, the
self-supervised discriminators the lesson describes (the LCD, the contrastive
Transformer) are now evaluated on shared discourse benchmarks such as **DiscoEval**
(Chen et al. 2019), a suite of probing tasks — sentence position, binary sentence
ordering, discourse coherence, RST-relation prediction — that measure how much
discourse structure a sentence encoder captures. The through-line is the same in
every case: the classical model states _what_ to measure (relations, entity
transitions, adjacent-sentence continuity) and the neural version learns the
features that measure it.[^neuralcoh]


## The through-line

A text is more than a set of sentences, and these two lessons are the catalog of
what binds them. [Coherence relations](/natural-language-processing/linguistic-structure/discourse-coherence)
tie clauses; Centering and the entity grid thread salient entities through the
sentences; lexical cohesion and neural discriminators track shared topic; and above
all of it, genres impose narrative, argumentative, and scientific macro-structure.
The entity-based half depends squarely on the earlier
[coreference](/natural-language-processing/linguistic-structure/coreference-and-discourse)
machinery — whether a text stays "about John" cannot be checked until the mentions
of John are identified — and lifts it from the sentence to the discourse. Downstream,
the same structure is used directly: a summarizer keeps nuclei and drops satellites,
an essay grader reads the argument structure, and a coherence score flags text that
fails to hang together.

[^jm-center]: **Jurafsky & Martin**, §22.3.1 — Centering Theory (Grosz et al. 1995): the piano-store minimal pair; the backward-looking center $C_b$, the forward-looking centers $C_f$ ranked by grammatical role, and the preferred center $C_p$; the four transitions (Continue, Retain, Smooth-Shift, Rough-Shift) of Brennan et al. (1987); Rule 1 (pronominalization) and Rule 2 (transition ordering); the worked trace showing $U_1 \to U_2$ as a Continue.
[^jm-grid]: **Jurafsky & Martin**, §22.3.2 — the Entity Grid model (Barzilay and Lapata 2008): the sentence-by-entity grid with S/O/X/– roles, requiring coreference and parsing; local entity transitions as length-2 column slices; the 16-dimensional transition-probability feature vector (e.g. $[\text{S}\ \text{–}]$ at 0.08); dense versus sparse columns; feeding the vector to a classifier.
[^jm-eval]: **Jurafsky & Martin**, §22.3.3 — Evaluating neural and entity-based coherence: human-rating classifiers versus self-supervision from natural-versus-permuted orderings; the sentence-order discrimination, sentence insertion, and sentence-order reconstruction tasks and their relative difficulty.
[^jm-repl]: **Jurafsky & Martin**, §22.4 — Representation learning models for local coherence: lexical cohesion; lexical chains (Morris and Hirst 1991), TextTiling (Hearst 1997) as cosine of neighboring word-count spans, and LSA coherence (Foltz et al. 1998) as average adjacent-sentence cosine; the self-supervised local coherence discriminator (Xu et al. 2019) with its four sentence-pair features (concatenation, difference, absolute difference, element-wise product), the margin loss, the RNN-predictability baseline, and the $\pm 2$-sentence contrastive Transformer (Iter et al. 2020).
[^jm-global]: **Jurafsky & Martin**, §22.5 — Global Coherence: narrative structure via Propp's _Morphology of the Folktale_, dramatis personae and functions as a plot grammar; computational induction of Proppian functions and of latent personae from movie plot summaries.
[^jm-arg]: **Jurafsky & Martin**, §22.5.1–§22.5.2 — Argumentation structure and scientific discourse: argumentation mining; Aristotle's pathos/ethos/logos; claims, premises, and SUPPORT/ATTACK argumentative relations in persuasive essays; component and relation classifiers; argumentation schemes and persuasion cues (reciprocity, authority, scarcity); the argumentative zoning scheme with its 15 knowledge-claim labels for scientific papers.
[^jieisen]: **Ji and Eisenstein (2014)**, "Representation Learning for Text-level Discourse Parsing," _ACL 2014_ — the DPLP parser, which learns latent vector representations of discourse spans jointly with the shift-reduce parsing objective, replacing hand-built features and improving RST parsing accuracy.
[^topdown]: **Lin, Ng, and Kan (2019)** and **Kobayashi et al. (2020)** — top-down RST parsing with pointer-network decoders that recursively split a span at a predicted boundary, seeing the whole span before dividing it and improving long-range attachment over bottom-up shift-reduce parsers.
[^plmimpl]: **Nie, Bennett, and Goodman (2019)** and follow-on work — using pretrained masked language models (a BERT `[CLS]` classifier over the two argument spans) for implicit PDTB relation sense classification, resetting the baseline above feature-engineered systems, with later systems adding connective prediction as an auxiliary task.
[^neuralcoh]: **Nguyen and Joty (2017)**, "A Neural Local Coherence Model," _ACL 2017_ — a convolutional network over the entity grid that learns salient local transition patterns and beats the classic grid on ordering tasks; and **Chen, Chen, and Durrett (2019)**, the **DiscoEval** benchmark of discourse probing tasks (sentence position, ordering, coherence, RST-relation prediction) for evaluating discourse representations in sentence encoders.