---
title: Dialogue and Chatbots
module: Applications
moduleNumber: 7
lessonNumber: 5
order: 705
summary: >
Conversation is the most natural interface to a machine and one of the
hardest to build. We set up what makes human dialogue work — turns, speech
acts, grounding, and the local structure of adjacency pairs — then trace the
two traditions that answer it: chatbots built to chat (ELIZA's pattern-matching,
corpus retrieval, and seq2seq generation with its blandness problem) and
task-oriented systems built to get something done (the GUS frame-and-slot
architecture and the modern NLU / state-tracker / policy / NLG pipeline that
accumulates a frame across turns).
topics: [Applications]
sources:
- book: Jurafsky & Martin
ref: "Ch. 24 — Chatbots & Dialogue Systems; §24.1 Properties of Human Conversation"
- book: Jurafsky & Martin
ref: "§24.2 Chatbots; §24.3 GUS — Simple Frame-based Dialogue Systems"
- book: Jurafsky & Martin
ref: "§24.4 The Dialogue-State Architecture; §24.5 Evaluating Dialogue Systems; §24.6 Dialogue System Design"
---
Conversation is the first kind of language a child learns and the kind adults use
most: ordering lunch, booking a flight, arguing about the weather. For a machine it
is one of the hardest interfaces to build. A **dialogue
system** — a **conversational agent** — communicates with a person in natural
language, and the field splits its programs into two kinds by what they are trying
to do.[^jm-open] **Task-oriented** agents use conversation to complete something:
the assistants in Siri, Alexa, and Google Assistant that set alarms, find
restaurants, and book flights. **Chatbots** are built for the open-ended talk
itself — mimicking the unstructured "chat" of informal human interaction, mostly
for entertainment but also to make task agents feel less mechanical. The two goals
require different machinery; the modern LLM assistant collapses the split.
Building either kind of system requires understanding what is being imitated.
Human conversation is a joint activity with more structure than it
appears to have, and every property of it is a design constraint on the machine.
## Properties of human conversation
Consider a fragment of an actual phone call between a human travel agent (A) and a
human client (C), the running example throughout Jurafsky and Martin's
treatment:[^jm-props]
> C₁: … I need to travel in May.
> A₂: And, what day in May did you want to travel?
> C₃: OK, uh, I need to be there for a meeting that's from the 12th to the 15th.
> A₄: And you're flying into what city?
> C₅: Seattle.
> A₆: And what time would you like to leave Pittsburgh?
Every feature of this ordinary exchange is a problem for a machine. Four
properties matter most.
### Turns and turn-taking
A dialogue is a sequence of **turns** — C₁, A₂, C₃, and so on — each a single
contribution from one speaker. A turn can be a word ("Seattle") or several
sentences. The _timing_ is precise: speakers hand the floor back and
forth with almost no gap and almost no overlap, because a listener can predict
when the current speaker is about to finish.
For a spoken system this predictive handoff becomes two hard problems. The machine
must know when to _stop_ talking — the user may interrupt, or begin a correction
mid-sentence — and it must know when to _start_, detecting that the user has
finished so it can process and reply. That second problem, **endpointing** or
endpoint detection, is genuinely difficult: people pause in the middle of turns,
and background noise blurs the boundary between a pause and an ending.[^jm-props]
### Speech acts
The central observation, due originally to Wittgenstein and worked out by Austin,
is that each utterance in a dialogue is an **action** performed by the speaker — a
**speech act** (or **dialogue act**). One common taxonomy has four classes:
| Speech act | What the speaker does | Verbs | In the example |
| --- | --- | --- | --- |
| **Constative** | commits to something being the case | answering, claiming, confirming, stating | C₁ "I need to travel in May" |
| **Directive** | attempts to get the hearer to do something | asking, requesting, ordering, inviting | A₂ "what day did you want to travel?" |
| **Commissive** | commits the speaker to a future course of action | promising, planning, betting, vowing | "I'll take the first one" |
| **Acknowledgment** | expresses an attitude toward the hearer | greeting, thanking, apologizing | "OK", "thank you" |
A user asking a machine to do something ("Turn up the music") issues a
**directive**; a question that expects an answer (A₂) is also a directive, a very
polite command to answer. Stating a constraint (C₁) is a **constative**. The
speech act names the _intention_ behind the words, and recovering it — not just
the surface string — is what a dialogue system must do.
### Grounding
A dialogue is not a stack of independent speech acts but a collective act, and
partners in a collective act must keep track of what they mutually agree on, the
**common ground**. They build it by **grounding**: acknowledging that the hearer
has understood the speaker, the conversational equivalent of an ACK confirming
receipt on a data channel.[^jm-props]
Humans ground constantly. A₆ ("And what time would you like to leave
_Pittsburgh_?") repeats "Pittsburgh" to show it was heard; the "And" that opens
so many of the agent's turns signals that the new question is _in addition_ to
the last, implying the previous answer was understood. A dialogue system that never
grounds — that answers questions without any signal of understanding — feels
broken even when it is technically correct, which is why the choice between
_explicit_ and _implicit_ confirmation, later in this lesson, matters.
### Dialogue structure: adjacency pairs
Conversation has local structure. Certain speech acts set up an expectation for a
particular next act: a QUESTION sets up an expectation for an ANSWER, a PROPOSAL
for an ACCEPTANCE or REJECTION, a COMPLIMENT for a DOWNPLAYER. These two-part
templates are **adjacency pairs**, a **first pair part** raising the expectation
and a **second pair part** satisfying it.[^jm-adj] The expectations help a system
decide what to do: after it asks a question, the next user turn is _supposed_ to
be an answer.
The two halves need not be adjacent. A **side sequence** — a **subdialogue** — can
open between them; the common case is a **clarification question** when the machine
mis-hears part of the user's turn.
$$
% caption: A short travel dialogue with each turn's dialogue act labeled. A
% question sets up an expectation for an answer (an adjacency pair); a
% clarification subdialogue can open between the two pair parts.
\begin{tikzpicture}[>=stealth, font=\footnotesize,
turnU/.style={draw, minimum width=52mm, minimum height=8mm, align=left, fill=black!3},
turnA/.style={draw, minimum width=52mm, minimum height=8mm, align=left, draw=acc, fill=acc!5},
act/.style={font=\scriptsize, text=black}]
\definecolor{acc}{HTML}{2348F2}
\node[turnU] (u1) at (0,0) {C: I need to travel in May.};
\node[act, anchor=west] at (2.9,0) {CONSTATIVE};
\node[turnA] (a1) at (0,-1.15) {A: What day in May?};
\node[act, anchor=west] at (2.9,-1.15) {DIRECTIVE (question)};
\node[turnU] (u2) at (0,-2.3) {C: A meeting from the 12th to 15th.};
\node[act, anchor=west] at (2.9,-2.3) {CONSTATIVE (answer)};
\node[turnA] (a2) at (0,-3.45) {A: Flying into what city?};
\node[act, anchor=west] at (2.9,-3.45) {DIRECTIVE (question)};
\node[turnU] (u3) at (0,-4.6) {C: Seattle.};
\node[act, anchor=west] at (2.9,-4.6) {CONSTATIVE (answer)};
% adjacency-pair bracket linking a1 (question) to u2 (answer)
\draw[acc, thick] (-2.95,-0.75) -- (-3.25,-0.75) -- (-3.25,-2.7) -- (-2.95,-2.7);
\node[acc, font=\scriptsize, rotate=90, anchor=south] at (-3.5,-1.7) {adjacency pair};
\end{tikzpicture}
$$
These properties — turns, speech acts, grounding, dialogue structure, and the
inferences a speaker expects the hearer to draw — are what make natural
conversation hard to build, and most of them are still open research problems. The
two traditions below each attack a different subset.
## Chatbots
The simplest dialogue systems are **chatbots**: systems that carry on extended
conversations meant to mimic the unstructured chat of informal human interaction.
Their goals are entertainment and engagement rather than task completion, and their
architectures fall into two classes — **rule-based** systems, the early influential
ELIZA and PARRY, and **corpus-based** systems, which mine large datasets of
human-human conversation.[^jm-chat]
### The rule-based tradition: ELIZA
ELIZA, written by Joseph Weizenbaum in 1966, is the most important chatbot in the
history of the field, and it works by nothing more than pattern matching.[^jm-eliza]
It simulates a Rogerian psychologist — a style of therapy in which the therapist
reflects the patient's statements back as questions, and can, as Weizenbaum noted,
"assume the pose of knowing almost nothing of the real world." That pose lets a
machine with no world model sustain a conversation.
ELIZA is built from pattern/transform rules, the same
[regular-expression](/natural-language-processing/foundations/regex-and-text-normalization)
machinery introduced at the start of the course. A rule pairs a pattern with a
transform:
```
(0 YOU 0 ME) [pattern]
->
(WHAT MAKES YOU THINK I 3 YOU) [transform]
```
Here `0` is a Kleene star (match anything) and the digits in the transform index
the matched constituents, so `3` refers to the material captured by the second `0`.
The rule turns _You hate me_ into _WHAT MAKES YOU THINK I HATE YOU_. Each rule is
attached to a **keyword** that might appear in the user's sentence, and keywords
carry a **rank**: specific words (_everybody_) outrank general ones (_I_), because
someone using a universal like "everybody" is usually referring to something quite
specific worth probing. The algorithm finds the highest-ranked keyword in the
input and fires its best-matching rule; if nothing matches, it falls back to a
non-committal "PLEASE GO ON" or "THAT'S VERY INTERESTING". One trick gives
the illusion of memory: whenever "my" is the top keyword, ELIZA stores a transform
of the sentence on a queue and, later, if nothing else matches, returns the oldest
stored entry — so it seems to circle back to something the user said turns ago.
```algorithm
caption: $\textsc{Eliza-Generator}(sentence)$ — pattern/transform response, per Weizenbaum
input: user $sentence$, keyword-ranked rule set, a memory queue
find the word $w$ in $sentence$ with the highest keyword rank
if $w$ exists then
choose the highest-ranked rule $r$ for $w$ that matches $sentence$
$response \gets$ apply the transform in $r$ to $sentence$
if $w$ is "my" then
$future \gets$ apply a memory-list transform to $sentence$
push $future$ onto the memory queue
else
if the memory queue is non-empty then
$response \gets$ pop the oldest entry from the memory queue
else
$response \gets$ apply the NONE-keyword transform to $sentence$
return $response$
```
ELIZA understands nothing, yet it fooled people — the enduring lesson.[^jm-eliza]
Users confided in it, Weizenbaum's own secretary asked him to leave
the room while she "talked" to it, and some kept believing it understood them after
being shown the rules. A few years later PARRY added a model of mental state —
affect variables for fear and anger — and became the first program to pass a
Turing test, when psychiatrists could not distinguish its transcripts from those of
real patients. ELIZA's pattern/action framework survives today in tools like ALICE,
and it is the baseline for everything that follows.
### Corpus-based chatbots
Instead of hand-built rules, **corpus-based** chatbots mine conversations — hundreds
of millions or billions of words of them.[^jm-corpus] Sources include transcribed
telephone speech (the Switchboard corpus), movie dialogue, and crowdsourced
conversation sets, supplemented by pseudo-conversations scraped from Twitter,
Reddit, and Weibo. Given the user's turn in context, these systems produce a
response in one of two ways.
**Response by retrieval** treats the user's turn as a query $q$ and returns the
most appropriate turn $r$ from a corpus $C$ of conversations, scoring each candidate
by similarity and picking the best. The classic form uses tf-idf cosine; the neural
form is a **bi-encoder**, two separately trained encoders — one for the query, one
for the candidate — whose dot product is the score, the same
[dense-retrieval](/natural-language-processing/applications/question-answering)
idea question answering uses:
$$
\response(q, C) = \argmax_{r \in C}\; h_q \cdot h_r,
\qquad h_q = \mathrm{BERT}_Q(q),\;\; h_r = \mathrm{BERT}_R(r).
$$
**Response by generation** instead casts response production as an encoder-decoder
task — transducing the user's turn (plus context) into the system's turn, a machine
learning version of ELIZA. It is the same [sequence-to-sequence
transduction](/natural-language-processing/applications/machine-translation) used
for translation, generating each response token $r_t$ by conditioning on the
encoded query $q$ and the response so far:
$$
\hat r_t = \argmax_{w \in V}\; P\!\left(w \mid q,\; r_1 \ldots r_{t-1}\right).
$$
$$
% caption: Two ways a neural chatbot produces a response. Retrieval (left) scores
% each corpus turn by the dot product of query and candidate encodings and returns
% the best; generation (right) runs an encoder-decoder over the whole conversation.
\begin{tikzpicture}[>=stealth, font=\footnotesize,
enc/.style={draw, minimum width=24mm, minimum height=7mm, align=center, fill=acc!6},
op/.style={draw, minimum width=17mm, minimum height=6mm, align=center, fill=black!4}]
\definecolor{acc}{HTML}{2348F2}
% --- retrieval (left) ---
\node[enc] (eq) at (0,0) {ENCODER (query)};
\node[enc] (er) at (3.1,0) {ENCODER (cand.)};
\node[op, draw=acc] (dp) at (1.55,1.4) {dot-product};
\draw[->, acc] (eq.north) -- (dp.south west);
\draw[->, acc] (er.north) -- (dp.south east);
\node[font=\scriptsize, anchor=north] at (0,-0.55) {user turn q};
\node[font=\scriptsize, anchor=north] at (3.1,-0.55) {candidate r};
\node[font=\footnotesize\bfseries, anchor=north] at (1.55,-1.15) {(a) retrieval};
% --- generation (right) ---
\begin{scope}[xshift=7.4cm]
\node[enc] (ge) at (0,0) {ENCODER};
\node[enc, draw=acc] (gd) at (2.6,1.4) {DECODER};
\draw[->, acc] (ge.north) -- (gd.south west);
\node[font=\scriptsize, anchor=north] at (0,-0.55) {conversation so far};
\node[font=\scriptsize, anchor=south] at (2.6,2.15) {response r1 r2 ...};
\node[font=\footnotesize\bfseries, anchor=north] at (1.3,-1.15) {(b) generation};
\end{scope}
\end{tikzpicture}
$$
Generation has a characteristic failure. A basic encoder-decoder, trained to
maximize response likelihood, learns that safe, generic replies fit almost any
context, so it converges on **bland**, repetitive turns — "I'm OK", "I don't know" —
that are fluent but shut the exchange down.[^jm-bland] The dullness is a
consequence of the objective, not the data: the most probable response is the
least informative one. Remedies include diversity-promoting beam search and
training objectives, minimum-length constraints, and, because encoder-decoders
optimize a single turn at a time and ignore the long-term shape of the
conversation, reinforcement learning and adversarial training that reward turns
making the _whole_ dialogue more natural.
## Task-oriented dialogue: the GUS architecture
The other tradition helps a user get something done — reserve a flight, book a
hotel — and almost all of it, from the 1977 GUS travel planner to today's
commercial assistants, is built around a **frame**.[^jm-gus] A frame is a knowledge
structure representing a kind of intention the system can extract from user
sentences. It is a set of **slots**, each with a semantic **type** and a set of
possible **values**, and a question template used to fill it.
> **Definition (Frame).** A collection of **slots** the system needs filled to
> perform a task, each slot constrained to values of a semantic type (a `city`
> slot takes values like _San Francisco_; a `date` slot is itself a frame with
> `month`, `day`, `weekday` sub-slots). The set of frames for a domain is a
> **domain ontology**.
$$
% caption: A frame for the air-travel domain. Each slot has a semantic type and a
% question template the system uses to fill it; the goal of the dialogue is to fill
% every slot, then perform the action (query the flight database).
\begin{tikzpicture}[>=stealth, font=\footnotesize,
hd/.style={font=\footnotesize\bfseries},
cell/.style={anchor=west, font=\footnotesize}]
\definecolor{acc}{HTML}{2348F2}
\draw[draw=acc] (-0.2,0.55) rectangle (11.6,-3.85);
\draw[acc] (-0.2,0.05) -- (11.6,0.05);
\node[hd] at (1.4,0.3) {Slot};
\node[hd] at (4.3,0.3) {Type};
\node[hd, anchor=west] at (5.6,0.3) {Question template};
\foreach \y/\slot/\typ/\q in {
-0.4/ORIGIN-CITY/city/{From what city are you leaving?},
-1.0/DEST-CITY/city/{Where are you going?},
-1.6/DEPART-TIME/time/{When would you like to leave?},
-2.2/DEPART-DATE/date/{What day would you like to leave?},
-2.8/ARRIVE-TIME/time/{When do you want to arrive?},
-3.4/ARRIVE-DATE/date/{What day would you like to arrive?}} {
\node[cell] at (0.2,\y) {\texttt{\slot}};
\node[cell] at (4.0,\y) {\texttt{\typ}};
\node[cell] at (5.6,\y) {\q};
}
\end{tikzpicture}
$$
The control architecture is **frame-filling**. The system asks the question
attached to each empty slot, records whatever the user supplies, and skips
questions for slots already filled. Because one user utterance can fill several
slots at once —
> I want a flight from San Francisco to Denver one way leaving after five p.m. on
> Tuesday.
— the system fills all of them and then asks only for what remains. Condition-action
rules attached to slots handle defaults (fill the hotel-booking city from the flight
destination), and because different inputs must switch control among multiple frames
(a fares frame, a route-information frame), GUS is at bottom a **production-rule**
system: different inputs fire different productions that fill different frames.
Filling a slot from a sentence means extracting three things: the **domain** (is
this about airlines or alarm clocks?), the user's **intent** (`SHOW-FLIGHTS`,
`SET-ALARM`), and the **slot fillers** themselves. From _Show me morning flights
from Boston to San Francisco on Tuesday_ a system builds a representation like
```
DOMAIN: AIR-TRAVEL
INTENT: SHOW-FLIGHTS
ORIGIN-CITY: Boston
DEST-CITY: San Francisco
DEPART-DATE: Tuesday
DEPART-TIME: morning
```
The original GUS did this extraction with handwritten rules — a **semantic grammar**
whose non-terminals are slot names — and rule-based slot-filling, prized for high
precision, is still common in industry. It has the usual weaknesses of hand-built
systems: the grammars are expensive to write and suffer from recall problems,
missing the phrasings the designer did not anticipate.
## The dialogue-state architecture
Modern research systems replace the rules with a more sophisticated,
machine-learned version of the same frame idea: the **dialogue-state** (or
belief-state) architecture.[^jm-ds] It keeps the frame but wraps it in four
learned components, a pipeline that runs on every user turn.
$$
% caption: The dialogue-state pipeline. NLU extracts dialogue acts and slot
% fillers from the user's turn; the state tracker accumulates them into the full
% frame; the policy chooses the next system act; NLG realizes it as text (and, for
% spoken systems, ASR and TTS wrap the whole loop).
\begin{tikzpicture}[>=stealth, font=\footnotesize,
comp/.style={draw, minimum width=27mm, minimum height=11mm, align=center},
core/.style={draw, minimum width=27mm, minimum height=11mm, align=center, draw=acc, fill=acc!6}]
\definecolor{acc}{HTML}{2348F2}
\node[core] (nlu) at (0,0) {NLU\\(slot f\/illing +\\dialogue acts)};
\node[core] (dst) at (4.2,0) {dialogue-state\\tracker};
\node[core] (pol) at (8.4,0) {dialogue\\policy};
\node[core] (nlg) at (8.4,-2.6) {NLG\\(response text)};
\node[comp] (asr) at (0,2.4) {ASR\\(spoken input)};
\node[comp] (tts) at (0,-2.6) {TTS\\(spoken output)};
\draw[->, acc, thick] (nlu) -- (dst);
\draw[->, acc, thick] (dst) -- (pol);
\draw[->, acc, thick] (pol) -- (nlg);
\draw[->, thick] (asr) -- (nlu);
\draw[->, thick] (nlg) -- (tts);
\draw[->, thick] (nlg.west) .. controls (3.0,-2.6) and (2.2,-1.6) .. (nlu.south);
\node[font=\scriptsize, text=black, anchor=south] at (4.5,-2.3) {next turn};
\end{tikzpicture}
$$
**Natural language understanding (NLU)** extracts the slot fillers and the user's
**dialogue act** from the turn. Dialogue acts fuse the speech-act idea with
grounding: a tag like `inform`, `request`, `confirm`, or `bye`, carried together
with its slot-value content. So a user might `inform(food=Italian, near=museum)`,
and the system later `confirm(pricerange=moderate)`.[^jm-acts] Slot-filling itself
is [sequence labeling](/natural-language-processing/sequences/sequence-labeling):
pass the words through a contextual encoder like BERT and tag each with a **BIO**
label ($2n+1$ tags for $n$ slots — a `B-` and `I-` per slot plus a single `O`).
The **dialogue-state tracker** determines the current state of the frame and the
user's most recent act; its main job is accumulation. The state is not
just the slots named in the latest sentence; it is the _entire_ frame so far,
summarizing every constraint the user has expressed across all turns:
> User: I'm looking for a cheaper restaurant
> `inform(price = cheap)`
> User: Thai food, somewhere downtown
> `inform(price = cheap, food = Thai, area = centre)`
Notice the tracker carries `price = cheap` forward into the second turn even though
the user did not repeat it. A subtle special case is the **user correction act** —
when a user re-states a mis-heard value ("I said BAL-ti-more, not Boston"), often
with exaggerated **hyperarticulation**. Corrections are _harder_ to
recognize than ordinary turns, and detecting them is its own classification problem.
To see the accumulation and a correction in one place, trace the frame across four
turns of the air-travel task. The NLU tags each turn with a dialogue act and its
slot-value content; the tracker folds that into the running frame — adding new slots,
overwriting on a correction, and carrying the rest forward untouched.
| turn | user utterance | NLU act | frame after tracker update |
| --- | --- | --- | --- |
| 1 | _I want to fly from Boston_ | `inform(orig=Boston)` | `orig=Boston` |
| 2 | _to Denver_ | `inform(dest=Denver)` | `orig=Boston, dest=Denver` |
| 3 | _no, I said Austin_ | `correct(orig=Austin)` | `orig=Austin, dest=Denver` |
| 4 | _on Tuesday morning_ | `inform(date=Tue, time=morning)` | `orig=Austin, dest=Denver, date=Tue, time=morning` |
Turn 2 adds `dest` without disturbing `orig`. Turn 3 is the correction case: the act
is `correct`, not `inform`, so the tracker _overwrites_ `orig` from `Boston` to
`Austin` rather than opening a second origin slot — the recognized `correct` act is
what tells it to replace rather than append. Turn 4 fills the last two slots,
and the frame is now complete enough for the policy to execute the flight query. Every
slot the user did not mention on a given turn is inherited unchanged; that inheritance
is the whole job of the tracker.
$$
% caption: A dialogue-state tracker accumulating a frame over four turns. Each turn's
% NLU act updates the running frame: new slots are added (dest at turn 2), a
% correction act overwrites an existing slot (orig Boston to Austin at turn 3, in
% red), and unmentioned slots carry forward. The frame is complete after turn 4.
\begin{tikzpicture}[>=stealth, font=\footnotesize,
frm/.style={draw, minimum width=40mm, minimum height=13mm, align=left, font=\scriptsize},
lbl/.style={font=\scriptsize, text=black, anchor=west}]
\definecolor{acc}{HTML}{2348F2}
\definecolor{red}{HTML}{C0392B}
\node[frm] (f1) at (0,0) {orig = Boston};
\node[frm] (f2) at (0,-1.6) {orig = Boston\\dest = Denver};
\node[frm] (f3) at (0,-3.5) {orig = Austin\\dest = Denver};
\node[frm] (f4) at (0,-5.7) {orig = Austin\\dest = Denver\\date = Tue, time = morning};
\node[lbl] at (2.6,0) {turn 1: inform(orig)};
\node[lbl] at (2.6,-1.6) {turn 2: add dest};
\node[lbl, text=red] at (2.6,-3.5) {turn 3: correct(orig)};
\node[lbl] at (2.6,-5.7) {turn 4: f\/ill date + time (complete)};
\draw[->, acc] (f1) -- (f2);
\draw[->, red, thick] (f2) -- (f3);
\draw[->, acc] (f3) -- (f4);
\end{tikzpicture}
$$
The **dialogue policy** decides what the system should do next — which dialogue act
to generate. Formally, given the state at turn $i$, it predicts the best next act
$\hat A_i$; a practical policy conditions on the current frame and the last exchange:
$$
\hat A_i = \argmax_{A_i \in \mathcal{A}}\; P\!\left(A_i \mid \mathrm{Frame}_{i-1},\, A_{i-1},\, U_{i-1}\right).
$$
The GUS policy was trivial — ask questions until the frame is full, then query the
database. A learned policy is richer: it can decide when to answer, when to ask a
clarification question, when to make a suggestion. The most consequential decisions
concern **confirmation and rejection**, where grounding becomes an engineering
choice. Under **explicit confirmation** the system asks a direct yes-no
question ("Do you want to leave from Baltimore?"), which is easy for a user to
correct but stilted and long-winded. Under **implicit confirmation** it grounds by
repeating understanding inside the next question ("When do you want to travel to
Berlin?"), which is far more natural. Sophisticated policies can be trained with
[reinforcement learning](/deep-learning/reinforcement-learning/rl-from-human-feedback):
a small action space (execute, confirm, elicit) earns a large reward for finishing
with the correct slots, a large penalty for wrong ones, and a small penalty per
confirmation to stop the system re-confirming everything.
Finally, **natural language generation (NLG)** turns the chosen act into text. GUS
used fixed **templates** ("Hello, how can I help you?") with variables filled in;
the dialogue-state version models generation as **content planning** (what to say,
handled by the policy) then **sentence realization** (how to say it), often trained
with **delexicalization** — replacing specific slot values with placeholders so the
model generalizes across restaurants and cities, then relexicalizing the real values
back in.
This lesson covered two traditions: chatbots that chat and frame systems that
complete tasks. Until recently a system had to choose between them, and each had to
be evaluated and designed on its own terms. A large language model removes the choice.
This continues in [Dialogue Systems: LLM Assistants, Evaluation, and Design](/natural-language-processing/applications/dialogue-systems-and-assistants),
which folds both traditions into one aligned model and takes up evaluation, design,
and ethics.
[^jm-open]: **Jurafsky & Martin**, _Speech and Language Processing_ (3rd ed.), Ch. 24 opening — dialogue systems / conversational agents, split into task-oriented agents (digital assistants that complete tasks) and chatbots (extended conversation for entertainment or to make task agents more natural).
[^jm-props]: **Jurafsky & Martin**, §24.1 — Properties of Human Conversation: the travel-agent/client example; turns and turn-taking with the endpointing problem; grounding, common ground, and the ACK analogy.
[^jm-adj]: **Jurafsky & Martin**, §24.1 — Subdialogues and Dialogue Structure: conversational analysis, adjacency pairs (first and second pair parts), and side sequences / subdialogues including the clarification question.
[^jm-chat]: **Jurafsky & Martin**, §24.2 — Chatbots: systems mimicking unstructured human chat, with architectures split into rule-based (ELIZA, PARRY) and corpus-based (retrieval and generation) classes.
[^jm-eliza]: **Jurafsky & Martin**, §24.2.1 — Rule-based chatbots ELIZA and PARRY: the Rogerian-psychologist design, pattern/transform rules keyed to ranked keywords, the memory-queue trick, and the simplified generator algorithm (Weizenbaum, 1966).
[^jm-corpus]: **Jurafsky & Martin**, §24.2.2 — Corpus-based chatbots: conversational corpora and social-media pseudo-conversations; response by retrieval (tf-idf and bi-encoder scoring) and response by generation (encoder-decoder transduction of the conversation-so-far).
[^jm-bland]: **Jurafsky & Martin**, §24.2.2 — encoder-decoder response generators tend to produce predictable, repetitive, dull responses ("I'm OK", "I don't know") that shut down the conversation; addressed with diversity-promoting decoding/training, minimum-length constraints, and reinforcement/adversarial methods for multi-turn coherence.
[^jm-gus]: **Jurafsky & Martin**, §24.3 — GUS: Simple Frame-based Dialogue Systems: frames, slots, types, and question templates; the frame-filling control architecture as a production-rule system; domain, intent, and slot-filler extraction with semantic grammars.
[^jm-ds]: **Jurafsky & Martin**, §24.4 — The Dialogue-State Architecture: the NLU / dialogue-state-tracker / dialogue-policy / NLG components augmenting the GUS frame model, wrapped by ASR and TTS for spoken systems.
[^jm-acts]: **Jurafsky & Martin**, §24.4.1–§24.4.3 — dialogue acts as speech-act-plus-grounding tags with slot-value content; slot filling as BIO sequence labeling ($2n+1$ tags); the dialogue-state tracker accumulating the full frame across turns, including user correction acts and hyperarticulation.