Extracting Times, Events, and Templates

This builds on information extraction: relation extraction, which turned tagged entities into relation triples. Relations are only part of what a database stores: facts also have to be placed in time and gathered into records. Here we detect and normalize temporal expressions, order events on a timeline, and fill templates for recurring situations — the stages that finish the pipeline from free text to a queryable database.

Extracting times

Times and dates are a special, important kind of entity for question answering and calendar assistants. To reason about them, extracted temporal expressions must be normalized — converted to a standard format.¹

Temporal expressions come in three flavors. Absolute expressions map directly to a calendar date or clock time (April 24, 1916; 10:15 AM). Relative expressions locate a time only through a reference point (yesterday; next semester; two weeks from yesterday). Durations denote a span at some granularity (four hours; three weeks; the last three quarters).

Temporal expressions have lexical triggers as their heads — nouns (morning, noon, winter), proper nouns (January, Monday, Ramadan), adjectives (recent, annual), and adverbs (hourly, daily) — projected into full noun, adjective, and adverbial phrases.

Recognition and TIMEX3

The TimeML annotation scheme wraps each temporal expression in a TIMEX3 XML tag. Recognition — finding the start and end of each such span — is done either by rule-based cascades of automata (POS-tag, then chunk by trigger-word patterns) or by sequence labeling with the same IOB scheme used for named entities:

A difficulty unique to this task is avoiding false positives from expressions that look temporal but are not — 1984 in 1984 tells the story of Winston Smith, or Sunday in U2's classic Sunday Bloody Sunday. Recognizers are scored with the usual precision, recall, and F-measure.

Temporal normalization

Temporal normalization maps a recognized expression to a point or a duration in the ISO 8601 standard. For example, take a document dated July 2, 2007 (its CREATION_TIME) containing A fare increase initiated last week ... over the weekend, marking the second successful fare increase in two weeks.

Several conventions are at work. ISO weeks run YYYY-Wnn, numbered 01-53 with week 01 the one holding the year's first Thursday; the document date falls in week 27, so last week normalizes to 2007-W26. Durations use the pattern PnU — $n$ an integer, $U$ a unit — so P3Y is three years, P2W two weeks, and the weekend is P1WE. When a duration is also anchored to a specific week, an anchorTimeID attribute records it.

Fully qualified expressions (a year, month, and day) are handled by compositional rules with temporal arithmetic: a pattern $\text{FQTE}\to \text{Month Date},\text{Year}$ computes its value from the values of its parts. Most news expressions are only implicitly anchored to the temporal anchor $A$ — the document date. Relative expressions resolve by offset arithmetic from $A$: today $\mapsto A$, tomorrow $\mapsto A+1\text{day}$, yesterday $\mapsto A-1\text{day}$ (modulo for month and clock rollover). Even the weekend is subtle: it normally refers to the weekend just past, but tense can flip it forward — Random security checks that began yesterday ... will continue at least through the weekend points at the coming weekend, signaled by continue. Most current normalizers are rule-based, pairing each matching pattern with a semantic procedure.

Extracting events and their times

Event extraction identifies mentions of events — any expression denoting an event or state that can be located at a point or interval in time. Marking up the airline text tags many events: citing, said, increased, matched, the move, took effect, applies, competes.² Most event mentions are verbs, but not all: events can be introduced by noun phrases (the move, the increase), and some verbs fail to introduce events — the phrasal verb took effect refers to when an event began, and light verbs like make, take, have push the event onto their nominal object (took a flight).

Each event mention $e$ is annotated with a class and attributes, $⟨\text{class},\text{tense},\text{aspect},\text{modality}⟩$; a said event is $⟨\text{REPORTING},\text{PAST},\text{PERFECTIVE},-⟩$. The TempEval classes group event predicates by function:

Event extraction is supervised: sequence models with IOB tagging detect events, and multi-class classifiers assign class and attributes, using features like part-of-speech, nominalization suffixes (-tion), light-verb flags, morphological stem, and WordNet hypernyms.

Temporal ordering and Allen relations

With events and times both detected, the next task is to fit the events onto a timeline. The full timeline is beyond current systems, but a useful weaker goal is a partial ordering — for instance, that American Airlines' fare increase came after United's. Determining such an ordering is a binary relation-classification task, deciding which of the standard set of Allen relations holds between two intervals.

An event is an interval $A=[A^{-},A^{+}]$ with $A^{-}<A^{+}$; each Allen relation is a constraint on the four endpoint orderings (e.g. $A\text{ before }B\leftrightarrow A^{+}<B^{-}$; $A\text{ meets }B\leftrightarrow A^{+}=B^{-}$). The 13 relations are the six asymmetric ones (before, meets, overlaps, starts, during, finishes), their inverses, and equals. These relations are classified with feature-based classifiers trained on the TimeBank corpus — 183 news articles annotated with events, times, and the temporal links between them, using TimeML. A TimeBank sentence like Delta Air Lines earnings soared 33% to a record in the fiscal first quarter, bucking the industry trend toward declining profits carries three events (soared, bucking, declining) and two times, plus links such as: soaring is included in the first quarter; soaring is before 1989-10-26; soaring is simultaneous with bucking; declining includes soaring. Classifier features include words/embeddings, parse paths, tense, and aspect.

Template filling

Many texts describe recurring, stereotyped situations — a script, in the sense of a prototypical sequence of sub-events with participants and roles. The strong expectations a script provides let a system classify entities, assign them to roles, and infer facts left unsaid. In their simplest form scripts become templates: a fixed set of slots $\{r_1,\dots ,r_m\}$, each slot $r_k$ constrained to fillers of a type ${\tau }_k$. Filling a template means selecting a text span (or normalized value) $v_k$ with $\text{type}(v_k)={\tau }_k$ for each slot. Template filling finds documents that invoke a template and fills its slots from the text.³

Our airline story is a stereotyped fare-raise situation. Reading it, we identify United as the lead airline that first raised fares, $6 as the amount, Thursday as the effective date, and American as a follower — a filled template:

The task is modeled with two supervised systems. Template recognition (also, confusingly, called event recognition) decides whether the template is present in a sentence — a text-classification task over the labeled spans. Role-filler extraction trains a separate classifier per role (LEAD-AIRLINE, AMOUNT, and so on), run over noun phrases or as a sequence model, to detect each filler. Both use the usual features: tokens, embeddings, word shapes, POS, chunk tags, and named entities. When several text segments carry the same slot label — United and United Airlines both label LEAD-AIRLINE — coreference resolution reconciles them.

Complex and hierarchical templates

The fare-raise template is flat. Consider instead a joint-venture report: Bridgestone Sports Co. said Friday it has set up a joint venture in Taiwan ... to produce golf clubs ... Bridgestone Sports Taiwan Co., capitalized at 20 million new Taiwan dollars, will start production in January 1990. Capturing all of this needs a hierarchical template, where one slot's filler is itself a template.

Early systems for such templates used cascades of finite-state transducers over hand-built rules. The FASTUS pipeline ran six stages, each extracting one kind of information and passing it up:

Modern learning-based systems handle flat templates well; the complex, hierarchical case remains partly the province of these earlier rule-based systems, and recent work induces templates automatically as sets of linked events.

From text to a database

The through-line is the pipeline that opened the first lesson: unstructured text becomes structured records. Relations among entities come from pattern-based methods, supervised classifiers when labels exist, semi-supervised bootstrapping and distant supervision when they are scarce, and unsupervised Open IE when the relations themselves are unknown. Temporal expressions are detected and normalized to ISO values so a system can compute with them. Events are detected and ordered on a timeline with Allen relations. Template filling assembles the pieces into slot-and-filler records for stereotyped situations.

Every stage starts from tagged entities, so IE sits directly downstream of sequence labeling: named-entity recognition supplies the arguments that relation, event, and template extraction connect. And it sits directly upstream of applications that need a knowledge base. Those triples and templates arrive already in the form of rows a knowledge base stores, which is why IE is the standard method for populating one — and a populated knowledge base is what lets a question-answering system look up who founded Apple or when did the fare increase take effect without re-reading the source text.