---
title: Semantic Roles and Information Extraction
module: Linguistic Structure
moduleNumber: 6
lessonNumber: 7
order: 607
summary: >
  Semantic roles answer "who did what to whom" for a single event, abstracting away
  the syntax that expresses it. We show why syntax alone is not enough, generalize
  over diathesis alternations with thematic roles, number a predicate's arguments
  with PropBank and group predicates into frames with FrameNet, tag each argument
  automatically with semantic role labeling, and factor predicates into primitives.
  Information extraction scales the idea to a corpus in the companion lesson.
topics: [Structure]
sources:
  - book: Jurafsky
    ref: "Ch. 19 — Semantic Role Labeling; §19.1 Semantic Roles; §19.2 Diathesis Alternations; §19.4 PropBank; §19.5 FrameNet; §19.6 Semantic Role Labeling; §19.7 Selectional Restrictions; §19.8 Primitive Decomposition of Predicates"
---

A [sequence labeler](/natural-language-processing/sequences/sequence-labeling)
tells you that _United Airlines_ is an organization and _Friday_ is a time. It
does not tell you that United _raised_ a fare, that the raise took effect Friday,
or that American _followed_. Those are relations between the entities — the
structured content buried in the sentence — and recovering them is the subject of
these two lessons. We work outward in two linked steps. First, **semantic roles**
— the subject here: a level of representation that says who played what part in a
single event, abstracting away the syntax that expresses it. Then **information
extraction** — the companion lesson,
[relations, events, and templates](/natural-language-processing/linguistic-structure/relations-events-and-templates)
— running that idea across a corpus to fill a database of relations, events, and
times that a downstream system can query.

The connecting thread is the same question the Roman rhetoricians already asked —
_who, what, where, when, with what, why, how_ — and the same output: free text in,
predicate-argument and relational structure out.

## Why syntax is not enough

Consider a single event, a purchase of stock by XYZ Corporation, and the many
surface forms that describe it:[^jm-roles]

- _XYZ Corp bought the stock._
- _They sold the stock to XYZ Corp._
- _The stock was bought by XYZ Corp._
- _The purchase of the stock by XYZ Corp…_

In every one, XYZ Corp is the buyer and the stock is the thing bought — the same
two participants in the same roles. Yet XYZ Corp appears as a subject, an indirect
object, and the object of a preposition; the event is named by a verb (_bought_,
_sold_) and by a noun (_purchase_). A parse tree records these differences
faithfully, and that is the problem: the differences are what we want to
throw away. If a document says _Company A acquired Company B_, we want to answer
"Was Company B acquired?" — and the parse offers no direct route from one to the
other.

**Thematic roles** supply the missing generalization. The subject of both _break_
and _open_ is a **volitional causer** of the event; we call that role the
**agent**. The direct object of both is the **participant most directly affected**;
that is the **theme**. Roles like these are shared across verbs, so they let us
state what two syntactically different sentences have in common.

$$
% caption: Five syntactic realizations of one $\textsc{break}$ event all map to the
% same small set of thematic roles. The syntactic position of an argument (subject,
% object, PP) varies; its role (AGENT, THEME, INSTRUMENT) does not.
\begin{tikzpicture}[>=stealth, font=\footnotesize,
  sent/.style={draw, minimum width=52mm, minimum height=7mm, align=left, font=\scriptsize, anchor=west},
  role/.style={draw, minimum width=24mm, minimum height=8mm, align=center, font=\scriptsize}]
  \definecolor{acc}{HTML}{2348F2}
  \definecolor{red}{HTML}{C0392B}
  \node[sent] (s1) at (0,3.0)  {John broke the window.};
  \node[sent] (s2) at (0,2.1)  {John broke the window with a rock.};
  \node[sent] (s3) at (0,1.2)  {The rock broke the window.};
  \node[sent] (s4) at (0,0.3)  {The window broke.};
  \node[sent] (s5) at (0,-0.6) {The window was broken by John.};
  \node[role, draw=acc, text=acc] (ag) at (9.3,2.4)  {AGENT};
  \node[role] (th)                 at (9.3,1.2)  {THEME};
  \node[role, draw=red, text=red]  (in) at (9.3,0.0)  {INSTRUMENT};
  \draw[->, acc]  (s1.east) -- (ag.west);
  \draw[->, acc]  (s2.east) -- (ag.west);
  \draw[->, acc]  (s5.east) to[out=0,in=200] (ag.west);
  \draw[->, black] (s1.east) to[out=0,in=170] (th.west);
  \draw[->, black] (s3.east) -- (th.west);
  \draw[->, black] (s4.east) -- (th.west);
  \draw[->, red]  (s2.east) to[out=-20,in=150] (in.west);
  \draw[->, red]  (s3.east) to[out=-20,in=170] (in.west);
\end{tikzpicture}
$$

The verb _break_ takes (at least) an agent, a theme, and an instrument, and it can
realize them in several syntactic slots — a flexibility the next section makes
precise. What matters here is that the role stays fixed while the slot moves, which is
why a level of representation above the parse tree is needed at all.

Here is the standard inventory of thematic roles used across the computational
literature, with a defining example for each:[^jm-roles]

| Role | Definition | Example |
| --- | --- | --- |
| Agent | volitional causer of an event | **The waiter** spilled the soup |
| Experiencer | the one who experiences an event | **John** has a headache |
| Force | non-volitional causer | **The wind** blew debris into the yard |
| Theme | participant most directly affected | Franklin broke **the ice** |
| Result | end product of an event | The city built **a baseball diamond** |
| Content | proposition of a propositional event | Mona asked **"You met Mary Ann?"** |
| Instrument | instrument used in an event | He stunned them **with a device** |
| Beneficiary | the one an event is done for | Ann makes reservations **for her boss** |
| Source | origin of a transferred object | I flew in **from Boston** |
| Goal | destination of a transferred object | I drove **to Portland** |

> **Definition (Thematic role).** An abstract label for the part an argument plays
> in the event named by a predicate — agent, theme, instrument, and so on —
> shared across predicates and independent of the syntactic position (subject,
> object, oblique) in which the argument surfaces.

## Diathesis alternations

The reason to bother with thematic roles at all is that they let us make inferences
the surface string and even the parse tree cannot. If a document says _Company A
acquired Company B_, we want to answer _Was Company B acquired?_ — and the two
sentences share no useful syntactic structure. What they share is the roles. The same
role can surface in different syntactic positions, and a single verb often licenses
several such mappings.

Take _break_. Its arguments — agent, theme, instrument — can be realized in many
ways:[^jm-diath]

- _John_ **broke** _the window._ — agent/subject, theme/object
- _John_ **broke** _the window_ _with a rock._ — agent/subject, theme/object, instrument/PP
- _The rock_ **broke** _the window._ — instrument/subject, theme/object
- _The window_ **broke.** — theme/subject
- _The window_ _was_ **broken** _by John._ — theme/subject, agent/PP

So _break_ has (at least) the arguments agent, theme, and instrument, and admits the
realizations agent/subject with theme/object; instrument/subject with theme/object;
and theme/subject alone. The set of thematic arguments a verb takes is its **thematic
grid** (or _θ-grid_, or **case frame**), and the fact that a verb permits multiple
mappings from that grid to syntax is a **diathesis alternation**.

> **Definition (Diathesis alternation).** A regular pattern in which a verb realizes
> the same thematic arguments in more than one syntactic configuration — for
> instance, letting the agent, the instrument, or the theme appear as the subject.
> Alternations are what force thematic roles to abstract over syntactic position.

$$
% caption: The causative/inchoative alternation of $\textsc{break}$. The same THEME
% (the window) surfaces as object in the causative frame (an AGENT breaks it) and as
% subject in the inchoative frame (it just breaks); the role is constant, the
% syntactic slot is not.
\begin{tikzpicture}[>=stealth, font=\footnotesize,
  sent/.style={draw, minimum width=44mm, minimum height=8mm, align=center, font=\scriptsize},
  role/.style={draw, minimum width=22mm, minimum height=7mm, align=center, font=\scriptsize}]
  \definecolor{acc}{HTML}{2348F2}
  \definecolor{red}{HTML}{C0392B}
  % causative
  \node[sent] (cau) at (0,1.2) {John broke the window};
  \node[anchor=east, font=\scriptsize, text=black] at (-2.5,1.2) {causative};
  % inchoative
  \node[sent] (inc) at (0,-1.2) {The window broke};
  \node[anchor=east, font=\scriptsize, text=black] at (-2.5,-1.2) {inchoative};
  % shared THEME role node
  \node[role, draw=acc, text=acc] (th) at (5.6,0) {THEME = window};
  \node[role] (ag) at (5.6,1.6) {AGENT = John};
  \draw[->, acc] (cau.east) to[out=0,in=170] node[above, font=\scriptsize, pos=0.6] {object} (th.west);
  \draw[->, acc] (inc.east) to[out=0,in=190] node[below, font=\scriptsize, pos=0.6] {subject} (th.west);
  \draw[->, black] (cau.east) to[out=0,in=200] (ag.west);
\end{tikzpicture}
$$

Alternations come in named families tied to semantic verb classes. The one above is
the **causative/inchoative** alternation — the window can be broken by an agent or can
simply break. Another is the **dative** alternation of _give_: _Doris gave the book to
Cary_ (theme, then goal in a PP) versus _Doris gave Cary the book_ (goal, then theme,
both bare objects) realize the same theme and goal in opposite orders. The dative
alternation is not arbitrary; it recurs across whole semantic classes — "verbs of
future having" (_advance, allocate, offer, owe_), "send verbs" (_forward, hand,
mail_), "verbs of throwing" (_kick, pass, throw_). Levin's catalog lists 3,100 English
verbs sorted into classes by exactly which alternations they permit, a resource folded
into VerbNet, which links each verb to WordNet and FrameNet. Because a verb's roles
move around the sentence, a level of description _above_ syntax is required.

### The trouble with a universal list

A single fixed list of a dozen roles is attractive, but every attempt to draw
one up runs into trouble. Roles fragment under scrutiny: there seem to be at least two kinds of **instrument** — an
_intermediary_ one that can be the subject (_The new gadget opened the jar_) and an
_enabling_ one that cannot (_\*The fork ate the banana_). And no crisp definition
captures **agent**: agents are typically animate, volitional, sentient, and causal,
but any given noun phrase might have only some of these properties.[^jm-proto]

Two responses have shaped the field's two major resources. One is to use
**generalized roles** — **proto-agent** and **proto-patient** — defined not by
necessary conditions but by a bundle of agent-like features (volitionally
involved, causing a change of state, sentient, moving) and patient-like features
(undergoing a change of state, causally affected, stationary). The more of the
bundle an argument exhibits, the more it counts as that proto-role. The other route
is to define roles that are **specific** to a verb or to a group of related
predicates. PropBank uses both proto-roles and per-verb roles; FrameNet uses roles
specific to a shared situation called a frame.

## PropBank

The **Proposition Bank** (PropBank) annotates the sentences of the Penn Treebank
with semantic roles. Because defining a universal role set is so hard, PropBank
defines roles **per verb sense**, and names them only with numbers: `Arg0`, `Arg1`,
`Arg2`, and so on. Two conventions carry across verbs: `Arg0` is the proto-agent
and `Arg1` the proto-patient. The higher-numbered arguments are looser — `Arg2` is
often a benefactive, instrument, attribute, or end state — and their meaning is
defined verb by verb in a **frame file**.[^jm-propbank]

A frame file for one sense of `increase` reads:

```
increase.01  "go up incrementally"
  Arg0: causer of increase
  Arg1: thing increasing
  Arg2: amount increased by
  Arg3: start point
  Arg4: end point
```

The payoff is that three syntactically different sentences receive the _same_
argument labels, so a system can infer they describe the same event structure.

$$
% caption: PropBank labels three surface forms of the same $\textsc{increase.01}$
% event. In each, Big Fruit Co. is $\text{Arg0}$ (the proto-agent) and the price of
% bananas is $\text{Arg1}$ (the proto-patient), regardless of active/passive voice
% or which argument surfaces as subject.
\begin{tikzpicture}[>=stealth, font=\scriptsize,
  arg0/.style={draw=acc, text=acc, inner sep=2.5pt, font=\scriptsize},
  arg1/.style={draw=black, inner sep=2.5pt, font=\scriptsize},
  arg2/.style={draw=red, text=red, inner sep=2.5pt, font=\scriptsize},
  pred/.style={font=\scriptsize\itshape}]
  \definecolor{acc}{HTML}{2348F2}
  \definecolor{red}{HTML}{C0392B}
  % row 1
  \node[arg0] (a) at (0,2.0) {Big Fruit Co.};
  \node[pred, right=3mm of a] (p1) {increased};
  \node[arg1, right=3mm of p1] (b) {the price of bananas};
  \node[left=1mm of a, font=\scriptsize] {(a)};
  % row 2
  \node[arg1] (c) at (0,1.0) {The price of bananas};
  \node[pred, right=3mm of c] (p2) {was increased by};
  \node[arg0, right=3mm of p2] (d) {Big Fruit Co.};
  \node[left=1mm of c, font=\scriptsize] {(b)};
  % row 3
  \node[arg1] (e) at (0,0.0) {The price of bananas};
  \node[pred, right=3mm of e] (p3) {increased};
  \node[arg2, right=3mm of p3] (f) {5\%};
  \node[left=1mm of e, font=\scriptsize] {(c)};
  % annotations
  \node[acc, anchor=west, font=\scriptsize] at (8.7,2.0) {Arg0 = causer};
  \node[black, anchor=west, font=\scriptsize] at (8.7,1.0) {Arg1 = thing increasing};
  \node[red, anchor=west, font=\scriptsize] at (8.7,0.0) {Arg2 = amount};
\end{tikzpicture}
$$

PropBank also carries a set of **non-numbered modifiers**, the `ArgM` labels,
stable across predicates: `ArgM-TMP` (when? _yesterday_), `ArgM-LOC` (where? _at
the museum_), `ArgM-DIR` (where to/from? _to Bangkok_), `ArgM-MNR` (how?
_clearly_), `ArgM-CAU` (why? _because…_). A companion project, **NomBank**, adds
the same treatment to noun predicates, so _Apple's agreement with IBM_ labels
_Apple_ as `Arg0` and _IBM_ as `Arg2`.

For example, take the sentence

> The San Francisco Examiner issued a special edition around noon yesterday.

Its predicate is _issued_, whose frame file (`issue.01`, "put out, publish") declares
`Arg0` = the publisher and `Arg1` = the thing published. The two numbered arguments are
filled by the subject and object; the two remaining phrases are modifiers — a temporal
`ArgM-TMP` for _yesterday_ and another temporal for _around noon_ — so the complete
label is:

$$
% caption: A full PropBank labeling of one sentence. The two numbered arguments
% (Arg0 publisher, Arg1 thing published) attach to the subject and object; the
% temporal phrases are non-numbered ArgM-TMP modifiers, stable across predicates.
\begin{tikzpicture}[>=stealth, font=\scriptsize,
  a0/.style={draw=acc, text=acc, inner sep=2.5pt, font=\scriptsize},
  a1/.style={draw=black, inner sep=2.5pt, font=\scriptsize},
  am/.style={draw=red, text=red, inner sep=2.5pt, font=\scriptsize},
  pr/.style={font=\scriptsize\itshape}]
  \definecolor{acc}{HTML}{2348F2}
  \definecolor{red}{HTML}{C0392B}
  \node[a0] (a) at (0,0) {The San Francisco Examiner};
  \node[pr, right=3mm of a] (p) {issued};
  \node[a1, right=3mm of p] (b) {a special edition};
  \node[am, right=3mm of b] (c) {around noon};
  \node[am, right=3mm of c] (d) {yesterday};
  % role labels underneath
  \node[acc, anchor=north, font=\scriptsize] at (a.south) {Arg0};
  \node[black, anchor=north, font=\scriptsize] at (b.south) {Arg1};
  \node[red, anchor=north, font=\scriptsize] at (c.south) {ArgM-TMP};
  \node[red, anchor=north, font=\scriptsize] at (d.south) {ArgM-TMP};
\end{tikzpicture}
$$

The value of the numbering shows when the same event is rephrased. _A special edition
was issued by the Examiner around noon_ moves the publisher into a _by_-phrase and the
thing published into subject position, yet both keep their labels — `Arg0` on the
_Examiner_, `Arg1` on _a special edition_ — because PropBank labels the role, not the
syntactic slot. That is the same invariance the `increase.01` figure showed, now with
a modifier in play: the `ArgM-TMP` phrases travel with the event regardless of voice.

## FrameNet

PropBank's roles are tied to individual verbs, so it still cannot relate _increase_,
_rise_ (a different verb), and _rise_ (the noun). **FrameNet** raises the level of
abstraction: it defines roles relative to a **frame**, a chunk of background
knowledge that a whole family of words evokes.[^jm-framenet] The words
_reservation, flight, travel, fare, meal, plane_ are not related by simple synonymy
or hyponymy; they cohere because they all presuppose the same background — air
travel. That background is a frame.

> **Definition (Frame).** A background-knowledge structure that defines a set of
> frame-specific semantic roles, called **frame elements**, together with the set
> of predicates (verbs, nouns, adjectives) that evoke it. Each word _profiles_ some
> aspect of its frame; the frame's elements are the roles its predicates share.

The `change_position_on_a_scale` frame covers the change of an `Item`'s position on
a scale (the `Attribute`) from an `Initial_value` to a `Final_value`. Its predicates
include verbs (_rise, fall, increase, climb, plummet, soar_), nouns (_increase,
decline, hike_), and adverbs (_increasingly_). Every one of them shares the same
frame elements.

$$
% caption: The $\textsc{change-position-on-a-scale}$ frame. A single frame is
% evoked by many predicates (verbs and nouns alike) and defines shared core roles
% (frame elements) — ITEM, ATTRIBUTE, INITIAL/FINAL VALUE, DIFFERENCE — plus
% non-core roles for time and group.
\begin{tikzpicture}[>=stealth, font=\scriptsize,
  frame/.style={draw=acc, text=acc, thick, minimum width=52mm, minimum height=9mm, align=center, font=\small},
  pred/.style={draw, minimum width=15mm, minimum height=6mm, align=center, font=\scriptsize},
  fe/.style={draw, fill=black!4, minimum width=22mm, minimum height=6mm, align=center, font=\scriptsize}]
  \definecolor{acc}{HTML}{2348F2}
  \node[frame] (fr) at (0,0) {frame:\\CHANGE-POSITION-ON-A-SCALE};
  % predicates that evoke it (top)
  \node[pred] (v1) at (-4.6,2.4) {rise (v)};
  \node[pred] (v2) at (-2.3,2.4) {fall (v)};
  \node[pred] (v3) at (0,2.4)    {increase (v)};
  \node[pred] (v4) at (2.5,2.4)  {increase (n)};
  \node[pred] (v5) at (4.8,2.4)  {decline (n)};
  \foreach \v in {v1,v2,v3,v4,v5} \draw[->, acc] (\v) -- (fr.north);
  \node[anchor=south, font=\scriptsize] at (0,3.0) {predicates evoke the frame};
  % frame elements (bottom)
  \node[fe] (e1) at (-4.4,-2.2) {ITEM};
  \node[fe] (e2) at (-1.9,-2.2) {ATTRIBUTE};
  \node[fe] (e3) at (0.6,-2.2)  {INITIAL VALUE};
  \node[fe] (e4) at (3.1,-2.2)  {FINAL VALUE};
  \node[fe] (e5) at (5.4,-2.2)  {DIFFERENCE};
  \foreach \e in {e1,e2,e3,e4,e5} \draw[->, black] (fr.south) -- (\e);
  \node[anchor=north, font=\scriptsize] at (0.5,-2.9) {frame def\/ines shared roles (frame elements)};
\end{tikzpicture}
$$

Labeled examples:

- _[ITEM Oil] rose [ATTRIBUTE in price] [DIFFERENCE by 2%]._
- _[ITEM Microsoft shares] fell [FINAL_VALUE to 7 5/8]._
- _a steady increase [INITIAL_VALUE from 9.5] [FINAL_VALUE to 14.3] [ITEM in dividends]._

Because the noun _increase_ and the verb _rose_ evoke the same frame, a system can
recognize that _the price of bananas_ is the `Item` and _5%_ the `Difference`
whether the sentence uses the verb or the noun. FrameNet also encodes relations
_between_ frames: a `cause_change_position_on_a_scale` frame **inherits** from the
plain frame and adds an `Agent` role, so _[AGENT They] raised [ITEM the price]
[DIFFERENCE by 2%]_ shares its event semantics with the non-causal versions.

The two resources trade off breadth against granularity:

| | PropBank | FrameNet |
| --- | --- | --- |
| Role scope | per verb sense | per frame (many predicates) |
| Role names | numbered: `Arg0`…`Arg4`, `ArgM-*` | named frame elements: `Item`, `Attribute`, … |
| Generalizes across | inflections of one verb | verbs, nouns, adjectives of one situation |
| Annotation base | Penn Treebank sentences | frame-by-frame example sentences |
| Same sentence | `[Arg0 …] issued [Arg1 …] [ArgM-TMP …]` | `[Cognizer …] blame [Evaluee …] [Reason …]` |

## Semantic role labeling

**Semantic role labeling (SRL)** is the task of automatically finding the semantic
role of each argument of each predicate in a sentence.[^jm-srl] Modern systems are
supervised, trained on PropBank or FrameNet. The classical pipeline begins with a
syntactic parse and then, for each predicate, decides the role (if any) that each
parse-tree constituent fills.

The generic procedure parses, then visits every predicate and every node:

```algorithm
caption: $\textsc{Semantic-Role-Label}(\text{words})$ — parse, then classify each node per predicate
$\text{parse} \gets \textsc{Parse}(\text{words})$
for each $\text{predicate}$ in $\text{parse}$ do
  for each $\text{node}$ in $\text{parse}$ do
    $\text{features} \gets \textsc{Extract-Features}(\text{node}, \text{predicate}, \text{parse})$
    $\textsc{Classify-Node}(\text{node}, \text{features}, \text{parse})$
```

Here `Classify-Node` is a **1-of-N classifier**: it assigns one of the semantic
roles, or a special `NONE` for a constituent that fills no role. Rather than one
monolithic decision, most systems break the per-node task into three stages, which
lets each stage use its own features and prunes the search early.

$$
% caption: The three-stage SRL pipeline for a fixed predicate. Pruning drops
% unlikely constituents by heuristic; identification is a binary argument/NONE
% decision; classification is a 1-of-N assignment of a role label to each surviving
% argument.
\begin{tikzpicture}[>=stealth, font=\footnotesize,
  stage/.style={draw, minimum width=30mm, minimum height=13mm, align=center, font=\scriptsize}]
  \definecolor{acc}{HTML}{2348F2}
  \node[stage] (pr)  at (0,0)    {1. Pruning\\drop unlikely nodes};
  \node[stage] (id)  at (4.2,0)  {2. Identif\/ication\\argument vs. NONE};
  \node[stage, draw=acc, text=acc] (cl) at (8.4,0) {3. Classif\/ication\\1-of-N role label};
  \draw[->, acc, thick] (pr) -- (id);
  \draw[->, acc, thick] (id) -- (cl);
  \node[anchor=north, font=\scriptsize] at (0,-1.0)   {heuristic};
  \node[anchor=north, font=\scriptsize] at (4.2,-1.0) {binary classif\/ier};
  \node[anchor=north, font=\scriptsize] at (8.4,-1.0) {N-way classif\/ier};
\end{tikzpicture}
$$

The feature-based classifier at the heart of the pipeline uses a small,
now-standard template set, computed for each constituent relative to the
predicate. For the subject _The San Francisco Examiner_ in _The San Francisco
Examiner issued a special edition yesterday_, the features include the governing
predicate (_issued_), the constituent's phrase type (`NP`), its headword
(_Examiner_) and headword POS (`NNP`), the **path** through the parse tree from the
constituent to the predicate (`NP↑S↓VP↓VBD`), the **voice** of the clause (active),
the **position** relative to the predicate (before), and the predicate's
**subcategorization** (`VP→VBD NP PP`).[^jm-features] The path feature is the
most useful of the set: it compactly encodes the grammatical relation between the
argument and its predicate.

Because the local classifier labels each argument independently, systems often add
a fourth, **global** step: constituents must not overlap, and a verb cannot have two
`Arg0` arguments. A second Viterbi pass or an integer-linear-program over the
per-constituent label distributions enforces these joint constraints.

### Neural SRL as sequence labeling

The neural approach discards the parse and recasts SRL as
[sequence labeling](/natural-language-processing/sequences/sequence-labeling), the
same BIO scheme used for named-entity recognition. Given the predicate, each role
gets a begin/inside pair (`B-ARG0`, `I-ARG0`, `B-ARG1`, `I-ARG1`, …) plus an outside
tag `O`, and the model tags the whole sentence at once.[^jm-neural-srl] The sentence
is fed to a pretrained encoder; each token's contextual embedding is concatenated
with an indicator of whether it is the predicate; and a feedforward network with a
softmax emits a label distribution per token.

$$
% caption: A neural SRL tagger. Each token's encoder output is concatenated with a
% predicate indicator, then a feedforward net with a softmax emits a BIO role tag.
% The goal is the highest-probability tag sequence; a CRF layer can be swapped in
% for global decoding.
\begin{tikzpicture}[>=stealth, font=\scriptsize,
  tok/.style={draw, minimum width=12mm, minimum height=6mm, align=center, font=\scriptsize},
  enc/.style={draw=acc, text=acc, thick, minimum width=52mm, minimum height=8mm, align=center, font=\small},
  ffn/.style={draw, minimum width=12mm, minimum height=5mm, align=center, font=\scriptsize},
  tag/.style={draw=acc, text=acc, minimum width=12mm, minimum height=6mm, align=center, font=\scriptsize}]
  \definecolor{acc}{HTML}{2348F2}
  % input tokens
  \node[tok] (t1) at (0,0)   {the};
  \node[tok] (t2) at (1.3,0) {cats};
  \node[tok] (t3) at (2.6,0) {love};
  \node[tok] (t4) at (3.9,0) {hats};
  \node[tok] (pi) at (5.6,0) {love=pred};
  % encoder
  \node[enc] (E) at (2.2,1.3) {ENCODER};
  \foreach \t in {t1,t2,t3,t4} \draw[->, acc] (\t) -- (E.south);
  \draw[->, acc] (pi) to[out=90,in=-15] (E.south east);
  % concat + ffn
  \node[ffn] (f1) at (0,2.6)   {FFN};
  \node[ffn] (f2) at (1.3,2.6) {FFN};
  \node[ffn] (f3) at (2.6,2.6) {FFN};
  \node[ffn] (f4) at (3.9,2.6) {FFN};
  \foreach \i/\x in {f1/0,f2/1.3,f3/2.6,f4/3.9} \draw[->, black] (\x,1.72) -- (\i.south);
  \node[anchor=west, font=\scriptsize] at (4.7,2.6) {concat predicate f\/lag};
  % output tags
  \node[tag] (g1) at (0,3.8)   {O};
  \node[tag] (g2) at (1.3,3.8) {B-ARG0};
  \node[tag] (g3) at (2.6,3.8) {B-PRED};
  \node[tag] (g4) at (3.9,3.8) {B-ARG1};
  \foreach \f/\g in {f1/g1,f2/g2,f3/g3,f4/g4} \draw[->, acc] (\f) -- (\g);
\end{tikzpicture}
$$

As with any tagger, the objective is the highest-probability tag sequence

$$
\hat{y} = \argmax_{y \in T} P(y \mid w),
$$

over the input words $w$. A CRF layer can replace the per-token softmax to enforce
valid label transitions globally, though in practice the gain is small once the
encoder is strong. SRL is evaluated with precision, recall, and F-measure, scoring
an argument correct only when its label spans exactly the right word sequence.

### Deep, span-based, and syntax-aware SRL

The BIO tagger just described is the starting point of a research line that reshaped
SRL between 2017 and 2018, and the three steps of that line answer a question Jurafsky & Martin
leave open: does throwing away the parse tree actually help, and if so, what replaces
the grammatical information the path feature used to carry?

**Deep tagging without syntax.** He, Lee, Lewis, and Zettlemoyer (ACL 2017) took the
BIO formulation to its limit with an eight-layer highway BiLSTM and constrained
decoding — no parse tree, no hand-built path or subcategorization features, just deep
sequence labeling over word embeddings. Their ensemble reached 83.2 F1 on CoNLL-2005,
about a 10% relative error reduction over the previous, syntax-dependent state of the
art.[^deep-srl] Two findings accompanied the number.
First, the deep stack recovers **long-distance dependencies** — an argument many words
from its predicate — far better than shallow models, recovering the information
the tree-path feature existed to supply, now learned implicitly. Second, the model still
makes **obvious structural errors** (overlapping arguments, two `Arg0`s), which is why
the global consistency step never fully went away, and the authors showed that oracle
syntax would still lift the score — syntactic information still carried useful signal.

**Span-based, end-to-end.** The BIO tagger assumes the predicate is given and labels one
token at a time. He, Lee, Levy, and Zettlemoyer (ACL 2018) dropped both assumptions with
a **span-based** model: enumerate candidate spans, build a contextual representation for
each, and score every _predicate-span, argument-span_ pair for the relation (if any)
between them, jointly predicting predicates and arguments in one pass. Because the unit
of decision is a span rather than a tag, span-level features come back, and the model set
a new state of the art on PropBank **without gold predicates** — the realistic setting
the tagger sidesteps.[^span-srl]

$$
% caption: Span-based end-to-end SRL. Instead of tagging tokens, the model enumerates
% candidate argument spans and predicate spans, builds a representation of each, and
% scores every (predicate, argument) pair for a role label -- jointly finding predicates
% and arguments with no gold predicate and no parse tree.
\begin{tikzpicture}[>=stealth, font=\footnotesize,
  tok/.style={draw, minimum width=11mm, minimum height=6mm, align=center, font=\scriptsize},
  spn/.style={draw, fill=black!4, minimum width=20mm, minimum height=6mm, align=center, font=\scriptsize},
  psp/.style={draw=acc, text=acc, minimum width=16mm, minimum height=6mm, align=center, font=\scriptsize},
  sc/.style={draw=red, text=red, minimum width=20mm, minimum height=6mm, align=center, font=\scriptsize}]
  \definecolor{acc}{HTML}{2348F2}
  \definecolor{red}{HTML}{C0392B}
  % sentence
  \foreach \w/\x in {the/0, cats/1.2, love/2.4, warm/3.6, hats/4.8} {
    \node[tok] at (\x,0) {\w};
  }
  \node[anchor=north, font=\scriptsize, text=black] at (2.4,-0.55) {contextual encoder over all tokens};
  % predicate span on its own tier
  \node[psp] (pr) at (2.4,1.6) {"love" (pred)};
  \draw[->, black] (2.4,0.35) -- (pr.south);
  % candidate argument spans, higher tier, flanking
  \node[spn] (s1) at (0.2,3.2)  {"the cats"};
  \node[spn] (s2) at (4.6,3.2)  {"warm hats"};
  \draw[->, black] (0.6,0.35) to[out=90,in=-90] (s1.south);
  \draw[->, black] (4.2,0.35) to[out=90,in=-90] (s2.south);
  % pair scoring: predicate up to each argument span, wide bows clearing the boxes
  \draw[->, red, thick] (pr.west) to[out=150,in=-60] node[left, font=\scriptsize, pos=0.55] {score pair} (s1.south east);
  \draw[->, red, thick] (pr.east) to[out=30,in=240] (s2.south west);
  % role labels above the argument spans
  \node[sc] (r1) at (0.2,4.6) {Arg0};
  \node[sc] (r2) at (4.6,4.6) {Arg1};
  \draw[->, red] (s1) -- (r1);
  \draw[->, red] (s2) -- (r2);
\end{tikzpicture}
$$

**Syntax comes back, folded in.** He et al.'s open question — that better syntax would
still help — was answered by Strubell, Verga, Andor, Weiss, and McCallum (EMNLP 2018)
with **linguistically-informed self-attention** (LISA). Rather than pre-computing a
parse and reading off features, LISA trains **one attention head** to attend to each
token's syntactic parent, inside a single transformer that is multi-tasked over
dependency parsing, part-of-speech tagging, predicate detection, and SRL at once. Syntax
is injected as an attention bias learned from raw tokens, not a preprocessing stage. On
CoNLL-2005 with predicted predicates it set a new state of the art, about 2.5 F1 above
the prior best on newswire and more than 3.5 F1 on out-of-domain text — the largest gains
exactly where a parser's grammatical signal helps a shallow lexical model
generalize.[^lisa-srl] The progression ends where it began: the classical pipeline read features
off a parse; deep tagging discarded the parse and learned the same signal implicitly;
span models restored span-level structure; and LISA put syntax back, now as a trainable
component of the same network rather than a separate front end.

### Selectional restrictions

A predicate also constrains the _kind_ of thing that can fill a role. The theme of
_eat_ tends to be edible, which is how we know that in _I want to eat someplace
nearby_, _someplace nearby_ is a location adjunct rather than the object being
eaten.[^jm-select] Such a semantic-type constraint is a **selectional restriction**.
It attaches to a word sense, not the whole lexeme: the food-serving sense of _serve_
restricts its theme to food (_serves green-lipped mussels_), while the
transportation sense restricts it to a place (_Which airlines serve Denver?_).

> **Definition (Selectional restriction).** A semantic-type constraint a predicate
> imposes on the concepts allowed to fill one of its argument roles — e.g. the theme
> of `eat` must be edible. Restrictions vary in strictness: `diagonalize` demands a
> matrix, while `imagine` places almost no constraint on its theme.

Rigid restrictions are brittle — negated or hyperbolic sentences violate them
freely (_you can't eat gold_) — so modern systems treat them as soft **selectional
preferences**, scoring the association between a predicate and an argument class
rather than forbidding violations outright.

## Primitive decomposition of predicates

Thematic roles decompose an event into a finite list of participant labels. A more
radical program decomposes the **predicate itself** into a small set of semantic
**primitives** — a move called **primitive decomposition** or **componential
analysis**.[^jm-prim] Consider _kill_. From a truth-conditional standpoint,

> Jim killed his philodendron.

means the same as

> Jim did something to cause his philodendron to become not alive.

so we can write the meaning of _kill_ as a composition of primitives:

$$
\textsc{KILL}(x, y) \iff \textsc{CAUSE}\big(x,\ \textsc{BECOME}(\textsc{NOT}(\textsc{ALIVE}(y)))\big).
$$

The same primitives explain the diathesis alternations from
earlier in the lesson. A single state-like predicate for _open_ underlies three
surface forms, differing only in whether $\textsc{CAUSE}$ and $\textsc{BECOME}$ wrap
it:

$$
\begin{aligned}
\text{John opened the door.} \ &\Rightarrow\ \textsc{CAUSE}\big(\text{John},\ \textsc{BECOME}(\textsc{OPEN}(\text{door}))\big) \\
\text{The door opened.} \ &\Rightarrow\ \textsc{BECOME}(\textsc{OPEN}(\text{door})) \\
\text{The door is open.} \ &\Rightarrow\ \textsc{OPEN}(\text{door}).
\end{aligned}
$$

The causative, the inchoative, and the plain stative are one predicate under
different primitive scaffolding — decomposition turns the alternation into a fact
about structure.

$$
% caption: Primitive decomposition of "John opened the door" as a tree. The stative
% core OPEN(door) is wrapped by BECOME (a change of state) and then by CAUSE (an
% agent bringing it about); stripping the outer primitives recovers the inchoative
% and stative readings.
\begin{tikzpicture}[>=stealth, font=\footnotesize,
  prim/.style={draw=acc, text=acc, minimum width=20mm, minimum height=7mm, align=center, font=\scriptsize},
  core/.style={draw, minimum width=22mm, minimum height=7mm, align=center, font=\scriptsize}]
  \definecolor{acc}{HTML}{2348F2}
  \node[prim] (cause) at (0,3.0) {CAUSE};
  \node[core] (john)  at (-2.6,1.6) {John (agent)};
  \node[prim] (become) at (1.6,1.6) {BECOME};
  \node[core] (open)  at (1.6,0.2) {OPEN(door)};
  \draw[->, acc] (cause) -- (john);
  \draw[->, acc] (cause) -- (become);
  \draw[->, acc] (become) -- (open);
  % readings labels
  \node[anchor=west, font=\scriptsize, text=black] at (3.4,3.0) {causative};
  \node[anchor=west, font=\scriptsize, text=black] at (3.4,1.6) {inchoative};
  \node[anchor=west, font=\scriptsize, text=black] at (3.4,0.2) {stative};
\end{tikzpicture}
$$

A predicate like _open_ is itself still fairly specific, so more aggressive schemes
break even it down. The **conceptual dependency** system used ten primitive
predicates to cover all of verbal meaning — among them $\textsc{ATRANS}$ (abstract
transfer of possession or control), $\textsc{PTRANS}$ (physical transfer of an object
between locations), $\textsc{MTRANS}$ (transfer of mental concepts), $\textsc{PROPEL}$
(applying force to move something), and $\textsc{INGEST}$ (taking in a substance).
Under this scheme one verb often maps to several primitives at once. _The waiter
brought Mary the check_ decomposes into an $\textsc{ATRANS}$ (control of the check
passes to Mary) _and_ a $\textsc{PTRANS}$ (the check physically moves to her), each
carrying its own fixed thematic roles:

$$
\begin{aligned}
&\textsc{Atrans}(x) \wedge \text{Actor}(x, \text{waiter}) \wedge \text{Object}(x, \text{check}) \wedge \text{To}(x, \text{Mary}) \\
\wedge\ &\textsc{Ptrans}(y) \wedge \text{Actor}(y, \text{waiter}) \wedge \text{Object}(y, \text{check}) \wedge \text{To}(y, \text{Mary}).
\end{aligned}
$$

Decomposition is the most granular answer to "who did what to whom": not a role list
over a whole predicate, but a factoring of the predicate into causes, changes, and
transfers that any two verbs with the same event structure will share.

## Where this continues

Semantic roles are the answer to "who did what to whom" for a single event.
Thematic roles generalize over the diathesis alternations that let one verb realize
its arguments in different syntactic slots; PropBank numbers those arguments
per-predicate and FrameNet groups predicates into frames; semantic role labeling
tags each argument automatically from a parse or an end-to-end neural model; and
primitive decomposition factors the predicate itself into shared causes, changes,
and transfers.

That is one predicate at a time. Scaling the same question to a whole corpus — pulling
entity-relation-entity triples, placing events and times on a timeline, and filling
templates into a queryable database — continues in
[relations, events, and templates](/natural-language-processing/linguistic-structure/relations-events-and-templates).

[^jm-roles]: **Jurafsky & Martin**, _Speech and Language Processing_ (3rd ed.), §19.1 — semantic roles: deep (event-specific) roles versus thematic roles, the thematic-role inventory (Figs. 19.1–19.2), and the Breaker/Opener = agent generalization.
[^jm-diath]: **Jurafsky & Martin**, §19.2 — Diathesis Alternations: the multiple syntactic realizations of _break_'s arguments, the thematic grid / θ-grid / case frame, the causative/inchoative and dative alternations, the semantic verb classes that share the dative (Levin 1993), and VerbNet.
[^jm-propbank]: **Jurafsky & Martin**, §19.4 — the Proposition Bank: per-verb-sense numbered arguments `Arg0`…`Arg4`, `Arg0` as proto-agent and `Arg1` as proto-patient, `ArgM` modifiers, frame files, and NomBank for nominal predicates.
[^jm-framenet]: **Jurafsky & Martin**, §19.5 — FrameNet: frames as background-knowledge structures, frame elements as frame-specific roles, the `change_position_on_a_scale` frame, core vs. non-core roles, and inter-frame relations like the causative inheritance.
[^jm-srl]: **Jurafsky & Martin**, §19.6 — semantic role labeling as a task, the parse-then-classify pipeline, and the pruning / identification / classification decomposition with global consistency.
[^jm-features]: **Jurafsky & Martin**, §19.6.1 — the Gildea & Jurafsky feature templates: governing predicate, phrase type, headword and its POS, tree path, voice, position, and subcategorization.
[^jm-select]: **Jurafsky & Martin**, §19.7 — selectional restrictions as semantic-type constraints attached to word senses, and the shift from strict restrictions to soft selectional preferences.
[^jm-prim]: **Jurafsky & Martin**, §19.8 — Primitive Decomposition of Predicates: componential analysis; $\textsc{KILL} \iff \textsc{CAUSE}(\textsc{BECOME}(\textsc{NOT}(\textsc{ALIVE})))$; the $\textsc{CAUSE}$/$\textsc{BECOME}$/$\textsc{OPEN}$ decomposition of the causative/inchoative/stative alternation (Lakoff 1965, Dowty 1979); and Schank's conceptual dependency with its ten primitives ($\textsc{ATRANS}$, $\textsc{PTRANS}$, …) and the _brought_ = $\textsc{ATRANS}$ + $\textsc{PTRANS}$ example (Fig. 19.8).
[^jm-proto]: **Jurafsky & Martin**, §19.3 — problems with thematic roles: role fragmentation (two kinds of instrument), the difficulty of formally defining agent, and the proto-agent / proto-patient generalization.
[^jm-neural-srl]: **Jurafsky & Martin**, §19.6.2 — neural SRL as BIO sequence labeling: encoder outputs concatenated with a predicate indicator, feedforward + softmax per token, optional CRF for global decoding.
[^deep-srl]: **He, Lee, Lewis, and Zettlemoyer**, "Deep Semantic Role Labeling: What Works and What's Next," ACL 2017. A deep highway BiLSTM with constrained decoding over BIO tags, no parse tree or hand-built features; the 8-layer ensemble reaches 83.2 F1 on CoNLL-2005 (~10% relative error reduction over the syntax-dependent prior state of the art). The analysis shows deep models recover long-distance dependencies well but still make obvious structural errors, and that oracle syntax would further improve results.
[^span-srl]: **He, Lee, Levy, and Zettlemoyer**, "Jointly Predicting Predicates and Arguments in Neural Semantic Role Labeling," ACL 2018. A span-based, end-to-end model that scores every candidate (predicate-span, argument-span) pair for its relation, jointly predicting predicates and arguments with span-level features; sets a new state of the art on PropBank without gold predicates, the realistic setting the gold-predicate BIO tagger avoids.
[^lisa-srl]: **Strubell, Verga, Andor, Weiss, and McCallum**, "Linguistically-Informed Self-Attention for Semantic Role Labeling" (LISA), EMNLP 2018. A single transformer, multi-tasked over dependency parsing, POS tagging, predicate detection, and SRL, that trains one attention head to attend to each token's syntactic parent — injecting syntax as a learned attention bias from raw tokens rather than a preprocessing stage. New state of the art on CoNLL-2005 with predicted predicates: ~2.5 F1 over the prior best on newswire and >3.5 F1 out-of-domain.