Learned and Neural Semantic Parsing

This builds on semantic parsing, which produced meaning representations by construction — a compositional walk of the syntax tree, then Combinatory Categorial Grammar with syntax and semantics in lockstep. Both routes need a hand-built grammar and a hand-built lambda term for every word. Here we replace that hand labor with learning: induce the parser from data, drop the grammar for a neural decoder, and take on the representations and executable targets that come with the learned approach.

Learning a semantic parser

Hand-writing a lexicon of lambda terms does not scale. A learned semantic parser induces the lexicon and a scoring model from data. Two supervision regimes matter, distinguished by how much the training signal reveals.¹

Supervision from logical forms. The training set is pairs $(x_i,y_i)$ of a sentence and its correct logical form. This is the easier signal: the target is fully specified, so learning is (roughly) structured prediction of $y$ from $x$. Datasets such as GeoQuery (questions about U.S. geography paired with logical queries) and ATIS (flight queries paired with SQL) provide exactly these pairs. The cost is annotation: someone must write a correct formal query for every training sentence, which demands expertise in the query language.

Supervision from denotations (weak supervision). Often we only have the answer, not the query — the sentence $x_i$ paired with the denotation $d_i$ obtained by executing the correct logical form against the knowledge base $\mathcal{K}$. The logical form $y$ is now a latent variable: the parser proposes candidates, executes each against $\mathcal{K}$, and is rewarded when a candidate's denotation matches the gold answer. Training marginalizes over forms that execute correctly, maximizing

where $[[y{]]}_{\mathcal{K}}$ denotes execution of $y$ against $\mathcal{K}$.

Weak supervision is far cheaper to collect — a non-expert can supply the answer to What states border Texas? without knowing any query language — but it is harder to learn from. The reward is sparse (most proposed forms execute to the wrong answer or crash), and spurious logical forms that happen to return the right answer for the wrong reason pollute the signal. This is the trade that recurs whenever the annotation is the bottleneck: cheaper labels, noisier learning.

Abstract Meaning Representation

First-order logic and lambda calculus are one way to write meaning; they are also verbose, and they entangle predicate-argument structure with quantifier scope, tense, and other logical scaffolding. Abstract Meaning Representation (AMR) strips that back to a single, simpler object: a rooted, directed, labelled graph over concepts and the relations between them.²

Nodes are concepts: word senses drawn from a frame inventory (PropBank-style predicates like $\text{have-01}$) or plain entities. Edges are relations, including the numbered semantic-role arguments $\text{arg0}$, $\text{arg1}$, and so on. The graph is rooted (one node is the focus), and a single node can be pointed at by several edges, so reentrancy (one entity filling two roles) is expressed directly, which a tree cannot do.

The same graph has an equivalent textual (PENMAN) form, which is what annotators actually write and parsers actually emit. Each node is (variable / concept) and each relation is a colon-prefixed label; reentrancy is a repeated variable:

(w / want-01   :arg0 (b / boy)   :arg1 (g / go-01            :arg0 b))

AMR is deliberately abstract: it drops tense, number, and articles, and it abstracts away from surface syntax so that the boy wants to go, the boy desires to go, and passive paraphrases all map to nearby graphs. That is the doctrine of canonical form — inputs that mean the same thing should get the same representation — pushed to the level of a whole-sentence graph. The cost is that AMR is not directly executable the way a database query is; it is a normalized meaning graph, not a program.

AMR parsing

AMR parsing maps a sentence to its graph. Because the output is a graph rather than a tree, it does not fit the compositional walk-the-syntax-tree recipe cleanly, and two broad strategies dominate.

• Graph-based / transition-based. Build the graph incrementally, in the spirit of transition-based dependency parsing: a sequence of actions adds concept nodes and draws labelled relation edges, often seeded by an alignment between words and concepts. Reentrancy is handled by allowing an action to point a new edge at an already-created node.
• Sequence-based (seq2seq). Linearize the PENMAN form into a token string and treat parsing as translation — encode the sentence, decode the linearized graph. This reuses the neural machinery of the next section, at the price of having to guarantee the output string is a well-formed, connected graph.

Because AMR abstracts away function words and inflection, alignment between input tokens and graph nodes is looser than in syntactic parsing, which is much of what makes AMR parsing hard. Evaluation uses Smatch: over all variable alignments $M$ between predicted graph $G$ and gold $G^{\ast }$, maximize the shared (relation, node, node) triples, then report $F_1$ on that best match,

Neural sequence-to-sequence parsing

The compositional and CCG parsers guarantee a well-formed logical form because they build it by grammar rules. The neural approach gives up that guarantee for generality: treat the logical form as just a sequence of tokens, and learn to generate it with the same encoder-decoder architecture used for machine translation. Semantic parsing becomes translation from English into a formal language.³

The encoder (a transformer or, in earlier systems, a biLSTM, frequently with a pretrained BERT front end) reads the question tokens. The decoder emits the logical form token by token: predicates, variables, parentheses, and keywords, each conditioned on the encoder and on the tokens produced so far, exactly as a translation decoder emits target words.

The freedom to emit any token sequence is also the failure mode: nothing stops the decoder from producing a syntactically illegal form — unbalanced parentheses, an undefined predicate, an argument count that no relation accepts. Constrained decoding addresses this: restrict the decoder, at each step, to tokens that keep the output a valid prefix of a well-formed logical form. In practice this means tracking the grammar of the target language while decoding and masking out any continuation that violates it — closing every bracket that was opened, only emitting predicate names that exist, respecting arities. Constrained decoding folds the guarantee of the grammar-based parsers back into the neural one: the model still learns from data, but it can only output forms the query engine can run.

One recurring difficulty is that a logical form is full of tokens copied straight from the question. What states border Texas? becomes a form containing the literal $\text{texas}$, and a fixed output vocabulary either has to enumerate every entity in the knowledge base or fails on any name it never saw in training. The standard solution is a copy mechanism: at each decoding step $t$ a learned gate $p_{\text{gen}}\in [0,1]$ mixes a generation distribution over the fixed vocabulary with a copy distribution over the input positions,

where $a_{t,i}$ is the decoder's attention on input token $x_i$. Rare entity names flow through untouched; copying also handles SQL literals — table values, string constants — that could never fit a closed vocabulary.

The trade against compositional parsing is stark. The neural parser needs no hand-built grammar or lexicon and degrades gracefully on constructions it has never seen, but it offers no compositional account of why a form was produced and can hallucinate structure absent constrained decoding. The two approaches are increasingly combined: neural scoring inside a grammar-constrained search.

Executable parsing: text-to-SQL and QA

The most consequential setting for semantic parsing is question answering over a structured database — knowledge-based QA. Here the logical form is an outright executable program: predicate calculus, a query language like SQL or SPARQL, or a small program over knowledge-base relations. Parse the question, execute the query, return the rows.⁴

Predicate calculus can be mechanically converted to SQL, so the choice of target language is partly cosmetic and partly a matter of what the backend runs. The two supervision regimes from earlier map straight onto QA datasets: fully supervised parsers learn from questions paired with hand-written queries (GeoQuery, ATIS with SQL, WebQuestionsSP with SPARQL), while weakly supervised parsers learn from questions paired only with the answer, treating the query as latent. A standard, strong baseline for text-to-SQL is precisely the encoder-decoder of the previous section: BERT-encode the question, decode the SQL string, optionally constrained to the database schema so the query references only real tables and columns.

This closes the loop opened by the first figure. What states border Texas? enters as text; a learned encoder-decoder, constrained by the target grammar, emits a query; the query executes against the database; the rows are the answer. Every earlier idea is a way of making that middle arrow reliable — compositionality for a guaranteed-valid form, CCG for a syntax-and-semantics-in-one lexicon, weak supervision for cheap labels, constrained decoding for validity in a neural model.

Neural semantic parsing in practice

The encoder-decoder view opened a decade of work on making learned semantic parsers accurate and general, past what the textbook sketches. Four threads stand out.

Sequence-to-tree decoding. A logical form is a tree, and forcing a plain left-to-right decoder to spell out its brackets wastes structure it already has. Dong and Lapata (2016), Language to Logical Form with Neural Attention (ACL), introduced Seq2Tree: a decoder that generates the logical form top-down as a tree, emitting a nonterminal and then recursively expanding its children, so bracket structure is guaranteed by construction rather than learned as a string convention. On GeoQuery and ATIS their attention-based model matched or beat the hand-engineered parsers that preceded it, with no grammar written by hand.⁵

Data recombination for compositional generalization. Neural parsers are data-hungry and generalize poorly to novel combinations of familiar pieces. Jia and Liang (2016), Data Recombination for Neural Semantic Parsing (ACL), induced a synchronous grammar from the training pairs and sampled new (sentence, logical form) pairs from it — recombining fragments the model had seen into constructions it had not — then trained a sequence-to-sequence model with attention and copying on the augmented data. The recombined data alone raised accuracy on GeoQuery and ATIS, a cheap way to inject compositional bias into a model that has none by default.⁶

Text-to-SQL at scale. Early text-to-SQL datasets (ATIS, GeoQuery) each targeted a single database, so a parser could quietly memorize its schema. Zhong et al. (2017) released WikiSQL with Seq2SQL, framing the task over thousands of Wikipedia tables and using the executed answer as a reinforcement-learning reward so that queries with different text but the same result are treated as equally correct.⁷ Yu et al. (2018), Spider (EMNLP), pushed harder: a large, cross-database benchmark of complex multi-table queries (joins, nesting, aggregation) with a strict split so that the databases at test time are ones the parser has never seen. Spider made schema encoding — reading the table and column names alongside the question and grounding the query in them — the central problem, and it remains the standard yardstick for text-to-SQL.⁸

Pretrained encoder-decoders as parsers. The same large pretrained models that carry the rest of modern NLP fold semantic parsing back into plain text-to-text generation: fine-tune a model like T5 or BART to emit the logical form as a string, and it inherits broad linguistic competence for free. Combined with constrained or grammar-guided decoding, these models now hold the top of the Spider and GeoQuery leaderboards — a neural model that writes the query, constrained by the grammar of the target language.

Evaluation

How a semantic parser is scored depends on what supervision is available, and the choice shapes what correct means.

Exact-match on the logical form is the most stringent: the predicted query must match the reference string (up to normalization). It is unforgiving, because two different-looking queries can be semantically equivalent — the same reason exact-match undersells a dependency parser.

Denotation accuracy sidesteps that by running both forms and comparing answers: any query that returns the correct set counts as correct, regardless of its syntax. This is the natural metric under weak supervision, where gold logical forms do not even exist, but it credits spurious forms that get the right answer by accident.

Smatch is the graph analogue for AMR: after aligning the two graphs' variables to maximize overlap, it computes precision, recall, and $F_1$ over shared (relation, node, node) triples, awarding partial credit for a mostly-correct graph rather than an all-or-nothing verdict.

The through-line across both semantic-parsing lessons is a single arrow: from the words of a sentence to a structured, executable meaning that a machine can run against the world. Compositional parsing and CCG build that meaning from the parts by grammar rules; here, learned parsers induce the lexicon and scoring model from data, AMR captures meaning as a normalized graph, and neural seq2seq learns to write the logical form out directly, constrained to well-formed outputs. All four routes end at the same target: a sentence turned into something with a denotation.