--- title: Intelligent Agents module: Foundations moduleNumber: 1 lessonNumber: 3 order: 103 summary: > An agent perceives an environment through sensors and acts on it through actuators; its behavior is an agent function mapping percept sequences to actions. A rational agent chooses, for each percept sequence, the action that maximizes its expected performance measure given its knowledge. We build the first half of the vocabulary the whole course rests on — the agent function, rationality, PEAS task specifications, and the six axes along which task environments vary. topics: [Foundations] sources: - book: AIMA ref: "Ch. 2 — Intelligent Agents; §2.1 Agents and Environments; §2.2 Good Behavior: The Concept of Rationality" - book: AIMA ref: "§2.3 The Nature of Environments" --- An **agent** is anything that perceives its **environment** through **sensors** and acts on that environment through **actuators**. A human agent has eyes and ears for sensors, hands and legs for actuators; a robot has cameras and range finders for sensors and motors for actuators; a software agent receives keystrokes and network packets and acts by writing files and sending packets. The definition is deliberately thin, because it is meant to be a lens for analyzing systems, not a line that divides the world into agents and non-agents. A pocket calculator, viewed this way, is an agent that outputs `4` on the percept `2 + 2 =` — an unhelpful reading, but not a wrong one. AI reserves its attention for the interesting end of that spectrum: agents with real computational resources whose task environment demands nontrivial decisions.[^aima-agents] This lesson works through three ideas in order. What an agent _is_ (the percept-to-action loop). What it means for an agent to be _good_ (rationality). And how to _specify_ the problem an agent faces (PEAS and the properties of task environments). The companion lesson, [agent architectures](/artificial-intelligence/foundations/agent-architectures), then shows how to _build_ one. Every later chapter — [search](/artificial-intelligence/search/uninformed-search), [logic](/artificial-intelligence/logic-and-planning/propositional-logic), [probability](/artificial-intelligence/uncertainty/probability-and-bayes), [learning](/artificial-intelligence/learning/learning-from-examples) — is a way of computing the action a rational agent should take in some class of environment. ## Agents and the agent function At any instant an agent has a **percept**: whatever its sensors report. Its **percept sequence** is the complete history of everything it has perceived. The central premise: _an agent's action at any instant can depend on the entire percept sequence to date, but on nothing it has not perceived_. So an agent's behavior is captured in full by an **agent function** mapping every percept sequence to an action.[^aima-func] Equivalently: given the full sensing history, the next action is determined. This map is the **agent function** $$ f : \mathcal{P}^\ast \to \mathcal{A}, $$ where $\mathcal{P}$ is the set of percepts, $\mathcal{P}^\ast$ the set of finite percept sequences, and $\mathcal{A}$ the set of actions. A percept sequence is $p_{1:t} = \langle p_1, \dots, p_t \rangle$, and the action at step $t$ is $a_t = f(p_{1:t})$. The function $f$ is an _external_, mathematical description — a (usually infinite) table pairing each percept history with a response. Internally, $f$ is realized by an **agent program** running on a physical device. Keep the two distinct: $f$ is the abstract specification, the program is the concrete implementation. The distinction returns when we ask how much _memory_ a program needs to reproduce a given $f$. $$ % caption: The agent as a loop. The environment delivers a percept through the % agent's sensors; the agent function $f$ maps the percept sequence so far to an % action; the actuators apply that action back to the environment. The cycle % repeats indefinitely. \begin{tikzpicture}[>=stealth, font=\small, ag/.style={draw, thick, minimum width=36mm, minimum height=34mm, align=center}, env/.style={draw, minimum width=22mm, minimum height=34mm, align=center}, fbox/.style={draw, minimum width=22mm, minimum height=12mm, align=center, font=\footnotesize}] \definecolor{acc}{HTML}{2348F2} \node[ag] (agent) at (0,0) {}; \node[anchor=north west, font=\footnotesize] at (agent.north west) {Agent}; \node[fbox, draw=acc, text=acc] (f) at (0,-0.1) {agent\\function}; \node[env, fill=black!5] (env) at (6.0,0) {Environment}; % sensors: env -> agent (top), enters function from above \draw[->, acc, thick] (env.north west) ++(0,-0.5) -- ++(-4.0,0) |- (f.north); \node[font=\scriptsize, anchor=south] at (3.0,1.55) {sensors}; \node[font=\scriptsize, anchor=north] at (4.9,1.08) {percept}; % actuators: agent -> env (bottom) \draw[->, acc, thick] (f.south) -- ++(0,-1.1) -| (env.south west); \node[font=\scriptsize, anchor=south] at (3.0,-1.66) {actuators}; \node[font=\scriptsize, anchor=south] at (4.9,-1.08) {action}; \end{tikzpicture} $$ Take the **vacuum-cleaner world**. Two squares $A$ and $B$, each either clean or dirty; the agent perceives its square and whether it is dirty, and acts in $\mathcal{A} = \{\texttt{Left}, \texttt{Right}, \texttt{Suck}, \texttt{NoOp}\}$. A simple agent function is: _if the current square is dirty, suck; otherwise move to the other square._ The design question of the whole field, in miniature, is: _what is the right way to fill out that table?_ That is what rationality answers. Writing out the tabulation shows $f$ is a finite object here even though it is infinite in general. Because this function depends only on the _current_ percept, $f(p_{1:t}) = f(p_t)$, so only the four one-step percepts matter and $f$ is four rows. | Percept sequence (last percept) | Action | | --- | --- | | $[A, \text{Clean}]$ | `Right` | | $[A, \text{Dirty}]$ | `Suck` | | $[B, \text{Clean}]$ | `Left` | | $[B, \text{Dirty}]$ | `Suck` | $$ % caption: The vacuum world and its agent trace. Starting in a dirty square A, the % agent sucks, moves right, and sucks again, cleaning both squares. Each transition % is one row of the agent function applied to the current percept. \begin{tikzpicture}[>=stealth, font=\footnotesize, sq/.style={draw, minimum width=15mm, minimum height=16mm, align=center}, ag/.style={draw, circle, minimum size=5mm, inner sep=0pt, fill=acc!22}] \definecolor{acc}{HTML}{2348F2} % three snapshots of the two-square world; agent dot sits in the lower corner \foreach \i/\x/\la/\lb/\dax/\dbx in {0/0/dirty/dirty/1/0, 1/4.6/clean/dirty/1/0, 2/9.2/clean/dirty/0/1} { \node[sq] (l\i) at (\x,0) {A\\\la}; \node[sq] (r\i) at (\x+1.5,0) {B\\\lb}; \ifnum\dax=1 \node[ag] at (\x-0.42,-0.45) {}; \fi \ifnum\dbx=1 \node[ag] at (\x+1.08,-0.45) {}; \fi } \draw[->, acc, thick] (1.9,-1.3) -- node[below, font=\scriptsize, text=black] {Suck} (3.4,-1.3); \draw[->, acc, thick] (6.5,-1.3) -- node[below, font=\scriptsize, text=black] {Right} (8.0,-1.3); \node[font=\scriptsize, anchor=north] at (0.75,-1.05) {t=0}; \node[font=\scriptsize, anchor=north] at (5.35,-1.05) {t=1}; \node[font=\scriptsize, anchor=north] at (9.95,-1.05) {t=2 (a further Suck cleans B)}; \end{tikzpicture} $$ ## Rationality A **rational agent** does the right thing — it fills out every entry of the agent function correctly. But "right" cannot mean "whatever the agent believes is right," or an agent could be rational by simply deluding itself. Rightness is judged from outside, by a **performance measure** that scores _sequences of environment states_ (not agent states). For the vacuum agent, a good measure might award one point per clean square at each time step over a lifetime of 1000 steps, perhaps with a penalty for wasted motion. As a design rule, it is better to specify performance in terms of what one actually wants in the environment than in terms of how one imagines the agent should behave — otherwise the agent will game the letter of the measure. (Reward "amount of dirt cleaned," and a rational agent will dump dirt back on the floor so it can clean it again.)[^aima-perf] What is rational at a given moment depends on four things: the performance measure, the agent's prior knowledge of the environment, the actions available to it, and its percept sequence to date. Assembling them gives the definition the rest of the course rests on. > **Definition (Rational agent).** For each possible percept sequence, a rational > agent selects an action that is expected to **maximize its performance measure**, > given the evidence provided by the percept sequence and whatever built-in > knowledge the agent has. Written as a choice rule: with performance measure $U$ over environment-state sequences, prior knowledge $K$, and percepts $p_{1:t}$, the rational action is $$ a_t \;=\; \arg\max_{a \in \mathcal{A}} \; \mathbb{E}\!\left[\, U \mid p_{1:t},\, K,\, a \,\right], $$ the action whose expected performance is greatest given everything known so far. Two words in that definition do heavy lifting. **Expected**: rationality maximizes _expected_ performance $\mathbb{E}[U]$, not _actual_ performance $U$. This is the difference between rationality and **omniscience**. An omniscient agent knows the actual outcome of its actions; no real agent does. Crossing an empty street is rational even if, freakishly, a cargo door falls from a passing airliner and flattens you — the percept sequence gave no reason to expect it, so the choice was correct even though the outcome was catastrophic. Rationality is about making the best decision given what you could know, not about being right after the fact. To see what "maximize the performance measure" means, score the simple vacuum agent above. Fix $U = \sum_{t=0}^{T-1} (\text{clean squares at } t)$ with $T = 1000$, the geography and initial dirt known, the start square random. Trace the worst start ($A$ dirty, $B$ dirty, agent in $A$): $t=0$ suck ($A$ clean), $t=1$ move right, $t=2$ suck ($B$ clean), both clean thereafter. Score: $1$ at $t=0$, $1$ at $t=1$, $2$ for each remaining step — $1 + 1 + 2 \times 998 = 1998$ of a possible $2000$. Against _this_ measure, in _this_ fully observable environment, the agent is rational: no $f$ does better, since two squares cannot both be clean before the agent has visited each, so the two lost points are unavoidable. Change the measure — penalize each move by one point to model energy — and the same agent is no longer rational; the rational agent now halts once both squares are clean rather than oscillate. Rationality is relative to the four inputs; changing any one can change which $f$ is rational. **Given the percept sequence**: rationality is relative to the information available. If an agent does not look before crossing a busy road, its percept sequence will not warn it of the approaching truck — but that does not license the crossing. A rational agent should first take the _looking_ action, because looking maximizes expected performance. Acting to improve future percepts is called **information gathering**, and it is a genuine part of rational behavior. A rational agent should also **learn**: use its percepts to correct and extend whatever prior knowledge it started with. An agent that relies wholly on its designer's prior knowledge, rather than its own percepts, lacks **autonomy**; a truly rational agent is autonomous, learning enough to compensate for partial or incorrect priors.[^aima-auto] ## Specifying the task: PEAS Before designing an agent, specify the problem it must solve. Rationality is defined relative to a performance measure, an environment, and the agent's sensors and actuators — group these four under the heading **task environment**, and remember them by the acronym **PEAS**: **P**erformance, **E**nvironment, **A**ctuators, **S**ensors. The first step in building any agent is to specify the PEAS as fully as possible.[^aima-peas] Take an automated taxi driver — a task still somewhat beyond current technology, and open-ended enough to exercise every part of the framework. | PEAS | Automated taxi | | --- | --- | | **Performance** | safe, fast, legal, comfortable trip; maximize profits | | **Environment** | roads, other traffic, pedestrians, customers, weather | | **Actuators** | steering, accelerator, brake, signal, horn, display | | **Sensors** | cameras, sonar, speedometer, GPS, odometer, accelerometer, engine sensors, keyboard | The performance measure already exposes real design tension: safety, speed, and profit conflict, so the agent needs a way to trade them off — a hint that goals alone will not be enough. The same template describes agents in purely software environments, where "sensors" are keyboard input and "actuators" are screen output; what matters is not whether the environment is physical but how complex the coupling between behavior, percepts, and performance is. | Agent | Performance | Environment | Actuators | Sensors | | --- | --- | --- | --- | --- | | Medical diagnosis | healthy patient, reduced costs | patient, hospital, staff | display of questions, tests, diagnoses | keyboard entry of symptoms, findings | | Part-picking robot | fraction of parts in correct bins | conveyor belt with parts | jointed arm and hand | camera, joint-angle sensors | | Refinery controller | purity, yield, safety | refinery, operators | valves, pumps, heaters, displays | temperature, pressure, chemical sensors | | Interactive English tutor | student's score on a test | set of students, testing agency | display of exercises, corrections | keyboard entry | Work one all the way through to see how the PEAS specification drives the design. Take a **spam filter**, an agent almost everyone runs. Its PEAS is compact but every entry carries a design consequence. | PEAS | Spam filter | | --- | --- | | **Performance** | fraction correctly classified; heavier penalty for a lost real message than for a spam that slips through | | **Environment** | the stream of incoming email, the sender population, the user who marks mistakes | | **Actuators** | file to inbox, file to spam, flag as uncertain | | **Sensors** | the raw message: headers, body text, links, attachments | Two design facts fall straight out of the P and the E. The asymmetric penalty in _Performance_ tells the agent to bias toward the inbox when unsure — a false "spam" is worse than a false "inbox" — which is a threshold choice, not a new mechanism. And the _Environment_ contains an adversary: spammers change their messages precisely to defeat the filter, so the sender population is non-stationary and the agent must keep learning. That single observation is why spam filtering is a machine-learning problem and not a fixed rulebook, and it is visible in the PEAS before any algorithm is chosen. Specifying PEAS well is most of the analysis. ## Properties of task environments Task environments vary along a handful of dimensions, and those dimensions — more than anything else — determine which agent design and which family of techniques is appropriate. The definitions here are informal; later chapters sharpen each one.[^aima-props] - **Fully vs. partially observable.** Write the current percept as $p_t$ and the environment state as $s_t$. The environment is _fully observable_ if $p_t$ determines the action-relevant part of $s_t$ at each instant; then the agent needs no internal state. It is _partially observable_ if sensors are noisy, inaccurate, or miss part of $s_t$ — a vacuum agent with only a local dirt sensor cannot tell whether the other square is dirty. An environment with no sensors is _unobservable_. - **Single-agent vs. multi-agent.** Solving a crossword is single-agent; chess is two-agent. What makes an entity $B$ count as an agent (rather than as stochastic scenery) is whether $B$'s behavior is best described as maximizing a performance measure that _depends on_ agent $A$'s behavior. Chess is **competitive** (the opponent's gain is your loss); taxi driving is partly **cooperative** (everyone wants to avoid collisions) and partly competitive (only one car fits the parking space). - **Deterministic vs. stochastic.** _Deterministic_ if the next state is a function of the current state and action, $s_{t+1} = T(s_t, a_t)$; _stochastic_ if instead governed by a distribution $P(s_{t+1} \mid s_t, a_t)$. Taxi driving is stochastic — traffic is never exactly predictable, tires blow out. An environment that is not fully observable or not deterministic is **uncertain**. **Nondeterministic** is subtly different: outcomes are listed as _possible_ values with no probabilities attached. - **Episodic vs. sequential.** In an _episodic_ environment, experience divides into atomic episodes — one percept, one action — and the next episode does not depend on actions taken in previous ones. An assembly-line defect spotter is episodic. In a _sequential_ environment, a current decision can affect all future ones; chess and taxi driving are sequential, and demand looking ahead. - **Static vs. dynamic.** If the environment can change while the agent deliberates, it is _dynamic_; otherwise _static_. Taxi driving is dynamic — the world does not wait while the agent decides. Crossword puzzles are static. If the environment itself holds still but the agent's performance score keeps ticking (chess with a clock), the environment is **semidynamic**. - **Discrete vs. continuous.** This applies to the state, to time, and to the agent's percepts and actions. Chess has a finite set of states and a discrete set of moves; taxi driving sweeps continuously through speeds, positions, and steering angles. $$ % caption: The six environment axes as a spectrum from the easy pole (left) to the % hard pole (right). Each step right removes an assumption a simpler agent relied % on. The real world sits at the right end of every axis. \begin{tikzpicture}[>=stealth, font=\footnotesize] \definecolor{acc}{HTML}{2348F2} \foreach \y/\lft/\rgt in { 0/{fully observable}/{partially observable}, -0.8/{single-agent}/{multi-agent}, -1.6/{deterministic}/{stochastic}, -2.4/{episodic}/{sequential}, -3.2/{static}/{dynamic}, -4.0/{discrete}/{continuous}} { \node[anchor=east, font=\scriptsize] at (3.0,\y) {\lft}; \draw[->, black] (3.2,\y) -- (5.4,\y); \node[anchor=west, font=\scriptsize, text=acc] at (5.6,\y) {\rgt}; } \node[font=\scriptsize, text=black] at (1.5,0.7) {easier}; \node[font=\scriptsize, text=acc] at (6.8,0.7) {harder (the real world)}; \end{tikzpicture} $$ A seventh axis, **known vs. unknown**, refers not to the environment itself but to the agent's knowledge of its "laws of physics" — whether it is given the outcomes (or outcome probabilities) of its actions. This is not the same as observability: solitaire is known but partially observable (you know the rules, but not the face-down cards), while a new video game can be fully observable yet unknown (the screen shows everything, but you don't yet know what the buttons do). | Task environment | Observable | Agents | Deterministic | Episodic | Static | Discrete | | --- | --- | --- | --- | --- | --- | --- | | Crossword puzzle | Fully | Single | Deterministic | Sequential | Static | Discrete | | Chess with a clock | Fully | Multi | Deterministic | Sequential | Semi | Discrete | | Poker | Partially | Multi | Stochastic | Sequential | Static | Discrete | | Backgammon | Fully | Multi | Stochastic | Sequential | Static | Discrete | | Taxi driving | Partially | Multi | Stochastic | Sequential | Dynamic | Continuous | | Medical diagnosis | Partially | Single | Stochastic | Sequential | Dynamic | Continuous | | Image analysis | Fully | Single | Deterministic | Episodic | Semi | Continuous | | Part-picking robot | Partially | Single | Stochastic | Episodic | Dynamic | Continuous | | Refinery controller | Partially | Single | Stochastic | Sequential | Dynamic | Continuous | | Interactive English tutor | Partially | Multi | Stochastic | Sequential | Dynamic | Discrete | The hardest case is _partially observable, multi-agent, stochastic, sequential, dynamic, continuous_, and _unknown_ — which is roughly the real world, and roughly what taxi driving is. Almost every technique in the course is a way of coping with one or more of these axes. [^aima-agents]: **Russell & Norvig**, _AIMA_ 3rd ed., §2.1 — Agents and Environments: the sensor/actuator definition of an agent, and the point that "agent" is an analytical stance rather than an absolute category (the hand-held calculator as a degenerate agent). [^aima-func]: **Russell & Norvig**, _AIMA_ 3rd ed., §2.1 — the percept sequence and the agent function mapping percept sequences to actions, and the distinction between the agent function (external, mathematical) and the agent program (internal, concrete). [^aima-perf]: **Russell & Norvig**, _AIMA_ 3rd ed., §2.2 — Good Behavior: The Concept of Rationality; the performance measure evaluates sequences of environment states, and should be designed around what one wants in the environment rather than how one thinks the agent should behave. [^aima-auto]: **Russell & Norvig**, _AIMA_ 3rd ed., §2.2.2 — Omniscience, Learning, and Autonomy: rationality maximizes expected (not actual) performance, is relative to the percept sequence, and requires information gathering, learning, and autonomy. [^aima-peas]: **Russell & Norvig**, _AIMA_ 3rd ed., §2.3.1 — Specifying the Task Environment: the PEAS description (Performance, Environment, Actuators, Sensors) and the automated-taxi worked example. [^aima-props]: **Russell & Norvig**, _AIMA_ 3rd ed., §2.3.2 — Properties of Task Environments: fully vs. partially observable, single- vs. multi-agent, deterministic vs. stochastic, episodic vs. sequential, static vs. dynamic, discrete vs. continuous, and known vs. unknown.