Semantic Roles and Information Extraction

A sequence labeler tells you that United Airlines is an organization and Friday is a time. It does not tell you that United raised a fare, that the raise took effect Friday, or that American followed. Those are relations between the entities — the structured content buried in the sentence — and recovering them is the subject of these two lessons. We work outward in two linked steps. First, semantic roles — the subject here: a level of representation that says who played what part in a single event, abstracting away the syntax that expresses it. Then information extraction — the companion lesson, relations, events, and templates — running that idea across a corpus to fill a database of relations, events, and times that a downstream system can query.

The connecting thread is the same question the Roman rhetoricians already asked — who, what, where, when, with what, why, how — and the same output: free text in, predicate-argument and relational structure out.

Why syntax is not enough

Consider a single event, a purchase of stock by XYZ Corporation, and the many surface forms that describe it:¹

• XYZ Corp bought the stock.
• They sold the stock to XYZ Corp.
• The stock was bought by XYZ Corp.
• The purchase of the stock by XYZ Corp…

In every one, XYZ Corp is the buyer and the stock is the thing bought — the same two participants in the same roles. Yet XYZ Corp appears as a subject, an indirect object, and the object of a preposition; the event is named by a verb (bought, sold) and by a noun (purchase). A parse tree records these differences faithfully, and that is the problem: the differences are what we want to throw away. If a document says Company A acquired Company B, we want to answer Was Company B acquired? — and the parse offers no direct route from one to the other.

Thematic roles supply the missing generalization. The subject of both break and open is a volitional causer of the event; we call that role the agent. The direct object of both is the participant most directly affected; that is the theme. Roles like these are shared across verbs, so they let us state what two syntactically different sentences have in common.

The verb break takes (at least) an agent, a theme, and an instrument, and it can realize them in several syntactic slots — a flexibility the next section makes precise. What matters here is that the role stays fixed while the slot moves, which is why a level of representation above the parse tree is needed at all.

Here is the standard inventory of thematic roles used across the computational literature, with a defining example for each:¹

Diathesis alternations

The reason to bother with thematic roles at all is that they let us make inferences the surface string and even the parse tree cannot. If a document says Company A acquired Company B, we want to answer Was Company B acquired? — and the two sentences share no useful syntactic structure. What they share is the roles. The same role can surface in different syntactic positions, and a single verb often licenses several such mappings.

Take break. Its arguments — agent, theme, instrument — can be realized in many ways:²

• John broke the window. — agent/subject, theme/object
• John broke the window with a rock. — agent/subject, theme/object, instrument/PP
• The rock broke the window. — instrument/subject, theme/object
• The window broke. — theme/subject
• The window was broken by John. — theme/subject, agent/PP

So break has (at least) the arguments agent, theme, and instrument, and admits the realizations agent/subject with theme/object; instrument/subject with theme/object; and theme/subject alone. The set of thematic arguments a verb takes is its thematic grid (or θ-grid, or case frame), and the fact that a verb permits multiple mappings from that grid to syntax is a diathesis alternation.

Alternations come in named families tied to semantic verb classes. The one above is the causative/inchoative alternation — the window can be broken by an agent or can simply break. Another is the dative alternation of give: Doris gave the book to Cary (theme, then goal in a PP) versus Doris gave Cary the book (goal, then theme, both bare objects) realize the same theme and goal in opposite orders. The dative alternation is not arbitrary; it recurs across whole semantic classes — verbs of future having (advance, allocate, offer, owe), send verbs (forward, hand, mail), verbs of throwing (kick, pass, throw). Levin's catalog lists 3,100 English verbs sorted into classes by exactly which alternations they permit, a resource folded into VerbNet, which links each verb to WordNet and FrameNet. Because a verb's roles move around the sentence, a level of description above syntax is required.

The trouble with a universal list

A single fixed list of a dozen roles is attractive, but every attempt to draw one up runs into trouble. Roles fragment under scrutiny: there seem to be at least two kinds of instrument — an intermediary one that can be the subject (The new gadget opened the jar) and an enabling one that cannot (*The fork ate the banana). And no crisp definition captures agent: agents are typically animate, volitional, sentient, and causal, but any given noun phrase might have only some of these properties.³

Two responses have shaped the field's two major resources. One is to use generalized roles — proto-agent and proto-patient — defined not by necessary conditions but by a bundle of agent-like features (volitionally involved, causing a change of state, sentient, moving) and patient-like features (undergoing a change of state, causally affected, stationary). The more of the bundle an argument exhibits, the more it counts as that proto-role. The other route is to define roles that are specific to a verb or to a group of related predicates. PropBank uses both proto-roles and per-verb roles; FrameNet uses roles specific to a shared situation called a frame.

PropBank

The Proposition Bank (PropBank) annotates the sentences of the Penn Treebank with semantic roles. Because defining a universal role set is so hard, PropBank defines roles per verb sense, and names them only with numbers: Arg0, Arg1, Arg2, and so on. Two conventions carry across verbs: Arg0 is the proto-agent and Arg1 the proto-patient. The higher-numbered arguments are looser — Arg2 is often a benefactive, instrument, attribute, or end state — and their meaning is defined verb by verb in a frame file.⁴

A frame file for one sense of increase reads:

increase.01  "go up incrementally"  Arg0: causer of increase  Arg1: thing increasing  Arg2: amount increased by  Arg3: start point  Arg4: end point

The payoff is that three syntactically different sentences receive the same argument labels, so a system can infer they describe the same event structure.

PropBank also carries a set of non-numbered modifiers, the ArgM labels, stable across predicates: ArgM-TMP (when? yesterday), ArgM-LOC (where? at the museum), ArgM-DIR (where to/from? to Bangkok), ArgM-MNR (how? clearly), ArgM-CAU (why? because…). A companion project, NomBank, adds the same treatment to noun predicates, so Apple's agreement with IBM labels Apple as Arg0 and IBM as Arg2.

For example, take the sentence

The San Francisco Examiner issued a special edition around noon yesterday.

Its predicate is issued, whose frame file (issue.01, put out, publish) declares Arg0 = the publisher and Arg1 = the thing published. The two numbered arguments are filled by the subject and object; the two remaining phrases are modifiers — a temporal ArgM-TMP for yesterday and another temporal for around noon — so the complete label is:

The value of the numbering shows when the same event is rephrased. A special edition was issued by the Examiner around noon moves the publisher into a by-phrase and the thing published into subject position, yet both keep their labels — Arg0 on the Examiner, Arg1 on a special edition — because PropBank labels the role, not the syntactic slot. That is the same invariance the increase.01 figure showed, now with a modifier in play: the ArgM-TMP phrases travel with the event regardless of voice.

FrameNet

PropBank's roles are tied to individual verbs, so it still cannot relate increase, rise (a different verb), and rise (the noun). FrameNet raises the level of abstraction: it defines roles relative to a frame, a chunk of background knowledge that a whole family of words evokes.⁵ The words reservation, flight, travel, fare, meal, plane are not related by simple synonymy or hyponymy; they cohere because they all presuppose the same background — air travel. That background is a frame.

The change_position_on_a_scale frame covers the change of an Item's position on a scale (the Attribute) from an Initial_value to a Final_value. Its predicates include verbs (rise, fall, increase, climb, plummet, soar), nouns (increase, decline, hike), and adverbs (increasingly). Every one of them shares the same frame elements.

Labeled examples:

• ITEM Oil rose ATTRIBUTE in price DIFFERENCE by 2%.
• ITEM Microsoft shares fell FINAL_VALUE to 7 5/8.
• a steady increase INITIAL_VALUE from 9.5 FINAL_VALUE to 14.3 ITEM in dividends.

Because the noun increase and the verb rose evoke the same frame, a system can recognize that the price of bananas is the Item and 5% the Difference whether the sentence uses the verb or the noun. FrameNet also encodes relations between frames: a cause_change_position_on_a_scale frame inherits from the plain frame and adds an Agent role, so AGENT They raised ITEM the priceDIFFERENCE by 2% shares its event semantics with the non-causal versions.

The two resources trade off breadth against granularity:

Semantic role labeling

Semantic role labeling (SRL) is the task of automatically finding the semantic role of each argument of each predicate in a sentence.⁶ Modern systems are supervised, trained on PropBank or FrameNet. The classical pipeline begins with a syntactic parse and then, for each predicate, decides the role (if any) that each parse-tree constituent fills.

The generic procedure parses, then visits every predicate and every node:

Here Classify-Node is a 1-of-N classifier: it assigns one of the semantic roles, or a special NONE for a constituent that fills no role. Rather than one monolithic decision, most systems break the per-node task into three stages, which lets each stage use its own features and prunes the search early.

The feature-based classifier at the heart of the pipeline uses a small, now-standard template set, computed for each constituent relative to the predicate. For the subject The San Francisco Examiner in The San Francisco Examiner issued a special edition yesterday, the features include the governing predicate (issued), the constituent's phrase type (NP), its headword (Examiner) and headword POS (NNP), the path through the parse tree from the constituent to the predicate (NP↑S↓VP↓VBD), the voice of the clause (active), the position relative to the predicate (before), and the predicate's subcategorization (VP→VBD NP PP).⁷ The path feature is the most useful of the set: it compactly encodes the grammatical relation between the argument and its predicate.

Because the local classifier labels each argument independently, systems often add a fourth, global step: constituents must not overlap, and a verb cannot have two Arg0 arguments. A second Viterbi pass or an integer-linear-program over the per-constituent label distributions enforces these joint constraints.

Neural SRL as sequence labeling

The neural approach discards the parse and recasts SRL as sequence labeling, the same BIO scheme used for named-entity recognition. Given the predicate, each role gets a begin/inside pair (B-ARG0, I-ARG0, B-ARG1, I-ARG1, …) plus an outside tag O, and the model tags the whole sentence at once.⁸ The sentence is fed to a pretrained encoder; each token's contextual embedding is concatenated with an indicator of whether it is the predicate; and a feedforward network with a softmax emits a label distribution per token.

As with any tagger, the objective is the highest-probability tag sequence

over the input words $w$. A CRF layer can replace the per-token softmax to enforce valid label transitions globally, though in practice the gain is small once the encoder is strong. SRL is evaluated with precision, recall, and F-measure, scoring an argument correct only when its label spans exactly the right word sequence.

Deep, span-based, and syntax-aware SRL

The BIO tagger just described is the starting point of a research line that reshaped SRL between 2017 and 2018, and the three steps of that line answer a question Jurafsky & Martin leave open: does throwing away the parse tree actually help, and if so, what replaces the grammatical information the path feature used to carry?

Deep tagging without syntax. He, Lee, Lewis, and Zettlemoyer (ACL 2017) took the BIO formulation to its limit with an eight-layer highway BiLSTM and constrained decoding — no parse tree, no hand-built path or subcategorization features, just deep sequence labeling over word embeddings. Their ensemble reached 83.2 F1 on CoNLL-2005, about a 10% relative error reduction over the previous, syntax-dependent state of the art.⁹ Two findings accompanied the number. First, the deep stack recovers long-distance dependencies — an argument many words from its predicate — far better than shallow models, recovering the information the tree-path feature existed to supply, now learned implicitly. Second, the model still makes obvious structural errors (overlapping arguments, two Arg0s), which is why the global consistency step never fully went away, and the authors showed that oracle syntax would still lift the score — syntactic information still carried useful signal.

Span-based, end-to-end. The BIO tagger assumes the predicate is given and labels one token at a time. He, Lee, Levy, and Zettlemoyer (ACL 2018) dropped both assumptions with a span-based model: enumerate candidate spans, build a contextual representation for each, and score every predicate-span, argument-span pair for the relation (if any) between them, jointly predicting predicates and arguments in one pass. Because the unit of decision is a span rather than a tag, span-level features come back, and the model set a new state of the art on PropBank without gold predicates — the realistic setting the tagger sidesteps.¹⁰

Syntax comes back, folded in. He et al.'s open question — that better syntax would still help — was answered by Strubell, Verga, Andor, Weiss, and McCallum (EMNLP 2018) with linguistically-informed self-attention (LISA). Rather than pre-computing a parse and reading off features, LISA trains one attention head to attend to each token's syntactic parent, inside a single transformer that is multi-tasked over dependency parsing, part-of-speech tagging, predicate detection, and SRL at once. Syntax is injected as an attention bias learned from raw tokens, not a preprocessing stage. On CoNLL-2005 with predicted predicates it set a new state of the art, about 2.5 F1 above the prior best on newswire and more than 3.5 F1 on out-of-domain text — the largest gains exactly where a parser's grammatical signal helps a shallow lexical model generalize.¹¹ The progression ends where it began: the classical pipeline read features off a parse; deep tagging discarded the parse and learned the same signal implicitly; span models restored span-level structure; and LISA put syntax back, now as a trainable component of the same network rather than a separate front end.

Selectional restrictions

A predicate also constrains the kind of thing that can fill a role. The theme of eat tends to be edible, which is how we know that in I want to eat someplace nearby, someplace nearby is a location adjunct rather than the object being eaten.¹² Such a semantic-type constraint is a selectional restriction. It attaches to a word sense, not the whole lexeme: the food-serving sense of serve restricts its theme to food (serves green-lipped mussels), while the transportation sense restricts it to a place (Which airlines serve Denver?).

Rigid restrictions are brittle — negated or hyperbolic sentences violate them freely (you can't eat gold) — so modern systems treat them as soft selectional preferences, scoring the association between a predicate and an argument class rather than forbidding violations outright.

Primitive decomposition of predicates

Thematic roles decompose an event into a finite list of participant labels. A more radical program decomposes the predicate itself into a small set of semantic primitives — a move called primitive decomposition or componential analysis.¹³ Consider kill. From a truth-conditional standpoint,

Jim killed his philodendron.

means the same as

Jim did something to cause his philodendron to become not alive.

so we can write the meaning of kill as a composition of primitives:

The same primitives explain the diathesis alternations from earlier in the lesson. A single state-like predicate for open underlies three surface forms, differing only in whether $\text{CAUSE}$ and $\text{BECOME}$ wrap it:

The causative, the inchoative, and the plain stative are one predicate under different primitive scaffolding — decomposition turns the alternation into a fact about structure.

A predicate like open is itself still fairly specific, so more aggressive schemes break even it down. The conceptual dependency system used ten primitive predicates to cover all of verbal meaning — among them $\text{ATRANS}$ (abstract transfer of possession or control), $\text{PTRANS}$ (physical transfer of an object between locations), $\text{MTRANS}$ (transfer of mental concepts), $\text{PROPEL}$ (applying force to move something), and $\text{INGEST}$ (taking in a substance). Under this scheme one verb often maps to several primitives at once. The waiter brought Mary the check decomposes into an $\text{ATRANS}$ (control of the check passes to Mary) and a $\text{PTRANS}$ (the check physically moves to her), each carrying its own fixed thematic roles:

Decomposition is the most granular answer to who did what to whom: not a role list over a whole predicate, but a factoring of the predicate into causes, changes, and transfers that any two verbs with the same event structure will share.

Where this continues

Semantic roles are the answer to who did what to whom for a single event. Thematic roles generalize over the diathesis alternations that let one verb realize its arguments in different syntactic slots; PropBank numbers those arguments per-predicate and FrameNet groups predicates into frames; semantic role labeling tags each argument automatically from a parse or an end-to-end neural model; and primitive decomposition factors the predicate itself into shared causes, changes, and transfers.

That is one predicate at a time. Scaling the same question to a whole corpus — pulling entity-relation-entity triples, placing events and times on a timeline, and filling templates into a queryable database — continues in relations, events, and templates.