The Foundations of AI

This builds on What Is Artificial Intelligence?, which settled on a single definition: AI is the study and design of rational agents. A definition is only half the answer; where the idea came from and how it has fared are the other half. This lesson traces the disciplines AI draws on, the cycles of boom and winter it has passed through, and the deep-learning turn that now dominates its public face, and shows that none of it disturbs the rational-agent frame fixed in the previous lesson.

The disciplines AI draws on

AI did not arise in isolation. It inherited ideas, and sometimes whole formalisms, from eight disciplines that had each spent centuries on a piece of the problem.¹

A short history

The field advances in cycles: optimism, a wall, retrenchment, a new idea that clears the wall. Six eras cover the arc.²

Gestation (1943–1955). McCulloch and Pitts modelled an artificial neuron as an on/off switch and showed that networks of them could compute any logical function; Hebb gave a rule for updating connection strengths. Turing, in Computing Machinery and Intelligence (1950), already sketched machine learning, genetic algorithms, and reinforcement learning, and proposed teaching a machine as one teaches a child.

Birth (1956). John McCarthy convened a two-month workshop at Dartmouth, coining the name artificial intelligence. The proposal's conjecture set the field's agenda: that every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it. Newell and Simon arrived with the Logic Theorist, a running program that proved theorems, and for the next twenty years the Dartmouth attendees and their students dominated the field.

Early enthusiasm (1952–1969). A run of successes — Newell and Simon's General Problem Solver, Samuel's self-improving checkers player, McCarthy's invention of Lisp and of time sharing, Minsky's microworlds — fed extravagant predictions. Simon claimed in 1957 that machines that think, learn and create already existed and would soon match the human mind across the board.

A dose of reality (1966–1973). The predictions collapsed on contact with scale. Machine translation foundered because translation needs world knowledge, not just syntax (the spirit is willing but the flesh is weak came back as the vodka is good but the meat is rotten). Early programs solved microworlds by trying all combinations of steps, a strategy that dies on the combinatorial explosion of real problems. Minsky and Papert's Perceptrons exposed the representational limits of simple neural nets, and funding evaporated — the first AI winter, formalized in Britain by the Lighthill report (1973).

Knowledge-based systems (1969–1979). The lesson drawn was that general-purpose search — the weak methods — does not scale, and that power comes from domain-specific knowledge. DENDRAL inferred molecular structure from mass spectra; MYCIN diagnosed blood infections with about 450 rules and outperformed junior doctors. Expert systems cleanly separated a knowledge base from the reasoning engine, and briefly turned AI into an industry — before overpromising brought a second AI winter in the late 1980s.

Statistics, agents, and the scientific method (1986–present). Three shifts converged. Back-propagation was reinvented in the mid-1980s, reviving neural nets. AI adopted rigorous experimental methodology, building on existing theory rather than replacing it: hidden Markov models took over speech recognition, Bayesian networks gave a principled formalism for uncertain reasoning, and machine learning reintegrated with statistics and information theory. In parallel, researchers returned to the whole agent and its environment, and the recent availability of very large data sets shifted emphasis from the cleverness of the algorithm to the scale of the data.

The state of the art

AI stopped being a promise and became infrastructure. A short sample of what already worked when AIMA was written suggests the range:³

None of it is science fiction; it is the ordinary output of engineering, mathematics, and statistics. As one summary of the period put it, thousands of AI applications are now embedded in the infrastructure of every industry.

The deep-learning era

The third edition of AIMA was published in 2009, a year before the shift that now defines the field's public face. That shift was not a new definition of AI — the rational-agent frame absorbs it without strain — but a change in which technique dominates the state of the art. It is worth tracing, both because it is the AI most people now meet and because it is the concrete subject of the sibling deep-learning course.

The 2012 turn. The catalyst was image classification. Krizhevsky, Sutskever, and Hinton entered a deep convolutional network, later called AlexNet, in the ImageNet Large Scale Visual Recognition Challenge and cut the top-5 error from about $26\%$ to $16\%$ — a margin no incremental method had approached (Krizhevsky et al., NeurIPS 2012).⁴ Three ingredients that AIMA names individually as loose threads came together at once: back-propagation (the algorithm the statistical turn revived), very large labeled data sets (ImageNet's ~1.2 million images), and enough parallel compute (two GPUs) to train a large network in days. Nothing in the underlying learning rule was new; the scale was.

The 2017 architecture. For sequence data — language above all — the decisive step was the Transformer, which replaced recurrence with an attention mechanism that lets every position attend directly to every other (Vaswani et al., Attention Is All You Need, NeurIPS 2017).⁵ Attention parallelizes cleanly across a sequence, so Transformers scale to far larger models and corpora than the recurrent networks before them. This is the architecture underneath essentially every large language model, and it is developed in full in the NLP course.

Scaling and the language-model agents. Trained on very large text corpora, Transformers improved smoothly as data, parameters, and compute grew together — loss falling as a power law $L(N)\propto N^{-\alpha }$ in each resource (Kaplan et al., 2020, the scaling laws). GPT-3 (Brown et al., Language Models are Few-Shot Learners, NeurIPS 2020) showed that at scale a single model performs many tasks it was never explicitly trained for, specified only by a natural-language prompt.⁶ The connection back to this lesson is direct: such a model, wrapped in a loop that calls tools and acts on an environment, is an agent in the exact sense of the next lesson — it perceives (prompt and tool outputs) and acts (the text and tool calls it emits). The rational-agent frame needed no revision; the model is a new way of computing the agent function $f:{\mathcal{P}}^{\ast }\to \mathcal{A}$, not a new kind of thing.

Learning to act, not just to classify. In parallel, reinforcement learning scaled the same way. AlphaGo combined deep networks with Monte-Carlo tree search and defeated a top human Go professional (Silver et al., Mastering the Game of Go with Deep Neural Networks and Tree Search, Nature 2016) — a result long thought a decade or more away, because Go's branching factor defeats the brute search that cracked chess. Its successor AlphaGo Zero reached superhuman play from self-play alone, given only the rules (Silver et al., Nature 2017).⁷ These are squarely the utility-maximizing agents this course builds toward, developed in the reinforcement-learning subject.

Two honest qualifications keep this in proportion. First, these systems inherit the field's oldest problem in a new form: they are trained to imitate or to score well on a proxy measure, and a rational agent maximizes the letter of its measure, so misspecified objectives produce confident, fluent errors — the alignment concern the course revisits in decision theory. Second, the deep-learning turn did not retire the rest of the field. Search still plans, logic still represents structured knowledge, probability still quantifies belief; the strongest current systems (AlphaGo is the clean example) combine learned components with search and probabilistic reasoning. The chapters that follow are not superseded by the deep-learning era — they are the other half of it.

The frame for everything that follows

Strip away the history and the disciplines and one commitment remains: AI is the study and design of rational agents. An agent perceives an environment and acts on it; a rational agent chooses, for each percept sequence, the action expected to maximize an external performance measure. Every remaining topic in this course is a tool for making that choice well under some new difficulty.

• When the environment is known and the agent must find a sequence of actions to a goal, the tool is search — uninformed, informed, local, and, against an opponent, adversarial.
• When the agent must represent what it knows and derive new facts, the tools are propositional and first-order logic, inference, and planning.
• When the environment is uncertain, the tools are probability, Bayesian networks, reasoning over time, and decision theory.
• When the agent must improve from experience, the tools are learning from examples, probabilistic learning, and reinforcement learning.

The next lesson, intelligent agents, makes the agent, its environment, and the performance measure precise, and shows how the structure of the environment dictates the structure of a rational agent. Everything after that is a way to fill in the one line at the center of the frame: given the percepts, choose the best action.