Exceptional Control Flow

A program's control flow, as built in the processor-design module, is a smooth sequence: fetch the instruction at %rip, run it, advance — with branches, calls, and returns letting the program redirect itself. But the system must also react to things the running program did not ask for and cannot see coming: a packet arrives, a timer expires, a divide-by-zero happens, a memory reference touches a page that is on disk. These abrupt, event-driven changes of control are exceptional control flow (ECF), and they are the mechanism behind system calls, page faults, the I/O system, and process switching. This lesson sorts ECF into its four classes, then opens up the machinery: how the hardware finds the handler, what state it saves and where, what the important x86-64 exceptions actually do, and how the same mechanism carries every system call your programs make.

What an exception is

The defining contrast is with ordinary flow. A branch or call is visible in the program text: the instruction itself names where control goes. An exception is not: it is triggered by an event (a device signal, or a side effect of executing an instruction), and control vectors to a handler the program never named. The processor enters kernel mode, runs the handler, and, for most exceptions, returns to the interrupted program as if nothing happened.

The four classes

Exceptions divide along two axes: whether the event is synchronous (a direct result of executing an instruction) or asynchronous (from outside the processor core), and whether control returns afterward. The combination gives four classes.

Each class has a sharp definition worth stating in prose.

• Interrupts are asynchronous: they are raised by I/O devices outside the processor — a network card, a disk, the interval timer — by asserting an interrupt request line. They are not caused by any particular instruction, so the handler returns control to the next instruction, the one that would have run anyway. Interrupts are how the machine notices the outside world; the next lesson is devoted to them.
• Traps are intentional, synchronous exceptions: the running program deliberately triggers one with an instruction (int on x86, syscall on x86-64) to request a service from the kernel: open a file, allocate memory, write to a socket. A trap always returns, to the instruction after the trapping one, carrying the service's result.
• Faults are synchronous errors that may be recoverable. The handler either fixes the condition and re-runs the very instruction that faulted, or, if it cannot, escalates to an abort. The archetype is the page fault: the page was on disk, the handler loads it, and the instruction retries and succeeds.
• Aborts are synchronous and unrecoverable: a hardware error such as a parity failure in memory. There is nothing to retry; the handler does not return to the program but terminates it (or halts the system).

The return-behavior column deserves a second look, because it encodes the whole semantics of each class. Interrupts and traps resume at the next instruction: the current one either completed normally (interrupt) or was the request (trap), so there is nothing to redo. A fault resumes at the same instruction: the instruction did not complete; it was stopped partway by a condition the handler may be able to repair. If the repair succeeds, the only correct continuation is to try it again. An abort resumes nowhere. A program that dies with a Floating point exception or a Segmentation fault was killed by a fault whose handler decided it could not repair the condition and converted it into a process-terminating signal instead.

Exception numbers and the exception table

The CPU cannot know what code handles each kind of event; that is the OS's business. The link between the two is a number. Every distinct exception type is assigned an exception number (also called a vector): some are fixed by the processor architecture, the rest are given out by the operating system. On x86-64, numbers 0 through 31 belong to the architecture (divide error is always 0, page fault is always 14), while numbers 32 through 255 are the OS's to assign to device interrupts and software conventions.

At boot, the kernel builds the exception table (on x86-64, the interrupt descriptor table, IDT): an array whose entry $k$ holds the address of the handler for exception $k$, along with a few control bits. It then executes a privileged instruction that loads the table's base address into a dedicated register. From that point on, dispatch is pure hardware: when exception $k$ fires, the processor reads entry $k$ — base plus $k$ times the entry size, a scaled index exactly like a jump-table or page-table access — and jumps to the address it finds there.

The table being in kernel memory, and the base register being loadable only in kernel mode, is what keeps the mechanism trustworthy: a user program can neither move the table nor rewrite an entry to point dispatch at its own code.

What the hardware does

When an exception fires, the transition into the handler is performed by hardware, because the running program cannot be trusted to do it (and may be the cause of the problem). The sequence is fixed.

Concretely, the hardware (1) saves the processor state, at minimum the faulting/next %rip and condition flags, onto the kernel stack, so the program can be resumed; (2) switches to kernel mode, granting the privileges the handler needs; and (3) jumps through the exception table to the handler. The handler runs, does its work, and finishes with a return-from-interrupt instruction (iret), which restores the saved state and switches back to user mode, resuming the interrupted program. The one detail that distinguishes the classes is which instruction it resumes at: the next one for interrupts and traps, the same one for faults that recovered — and for an abort, it does not return at all.

Not a procedure call

The description above sounds like a call with extra ceremony, and the analogy is useful: control transfers to a routine, state is pushed, a matching return instruction comes back. But three differences separate an exception from a procedure call, and each exists for a reason.

More state is pushed. A call pushes exactly one thing: the 8-byte return address. An exception must reconstruct the processor as it was, so the hardware pushes the return %rip, the flags register (a fault handler will clobber the condition codes, but the interrupted program may be one instruction away from a jle that depends on them), the stack pointer, and the segment state that encodes the privilege level. Some exceptions also push an error code describing the cause.

The state goes on the kernel stack. If control is transferring from a user program into the kernel, everything is pushed onto a kernel stack, not the user's. The user's %rsp cannot be trusted: it may point at an unmapped page, or be deliberately aimed at kernel data as an attack. Each process has a small dedicated kernel stack, known-good, on which its exceptions and syscalls run; the hardware switches %rsp to it as part of taking the exception.

The privilege level changes. The handler starts executing in kernel mode regardless of what mode the machine was in, and iret is what drops back. No sequence of user-mode instructions can produce this transition with control of where it lands: the only way into kernel mode is through an entry the kernel itself installed in the exception table.

The pushed record carries what iret needs: it pops the return address, the flags, and the old stack pointer, and the privilege level comes back with the segment state. Nothing about the interrupted computation escapes, which is why a program can take thousands of interrupts per second without observing any of them.

Three exceptions on x86-64, concretely

Three x86-64 exceptions illustrate the taxonomy.

Divide error (#DE, number 0) fires when a divide instruction divides by zero, or when the quotient overflows the destination: idivq with the most negative 64-bit value divided by $-1$ produces a quotient that does not fit, and faults identically. It is classified a fault, but no handler can produce a correct quotient, so Unix systems do not attempt a repair: the kernel converts it into a SIGFPE signal for the process, and the shell reports Floating point exception (a historical misnomer; the exception is an integer one).

General protection (#GP, number 13) is the catch-all protection violation: executing a privileged instruction in user mode (hlt, cli, a write to a control register), or referencing a non-canonical address. There is nothing to repair: the program attempted something the protection model forbids. Linux delivers SIGSEGV and the default outcome is the familiar Segmentation fault.

Page fault (#PF, number 14) is the one fault that routinely recovers. When an instruction references a virtual page whose PTE says not-present, the hardware pushes an error code describing the access (read or write, user or kernel), stores the faulting address in a control register where the handler can read it, and — the essential part — saves the address of the faulting instruction itself, not the next one. The handler examines the address: if it belongs to a valid region whose page is on disk, it reads the page into a free frame, updates the PTE, and executes iret — which resumes at the same instruction. The load or store executes a second time, finds the page present, and completes as if nothing had happened. If instead the address is outside every valid region, the fault was a bug, and the handler delivers SIGSEGV.

Re-execution is why a page fault is invisible to the program. It is also why fault handling must be idempotent-friendly: the faulting instruction ran zero times as far as architectural state is concerned (the hardware guarantees no partial effects survive), so running it again is really running it once.

The system call, end to end

Traps deserve their own walkthrough, because the trap is the interface through which every program uses the operating system: reading a file (read), creating a process (fork), loading a program (execve), exiting (exit): each is a kernel service with a number.

The convention on x86-64 Linux mirrors the procedure-call convention with one substitution: the syscall number goes in %rax, arguments go in %rdi, %rsi, %rdx, %r10, %r8, %r9, and the result comes back in %rax, where a value in the range $-\mathrm{4095..}-1$ encodes a negated error number. The fourth argument register is %r10 rather than the usual %rcx, and the reason is the mechanism itself: the syscall instruction clobbers %rcx to save the return address, as we are about to see.

Here is write(1, "hello", 5) with no library in the way:

.section .rodata
msg:
.ascii "hello"

.text
.globl _start
_start:
movq $1, %rax          # syscall number 1 = write
movq $1, %rdi          # arg 1: fd 1 (stdout)
leaq msg(%rip), %rsi   # arg 2: buffer address
movq $5, %rdx          # arg 3: byte count
syscall                # trap into the kernel
                       # back here with %rax = 5 (bytes written)
movq $60, %rax         # syscall number 60 = exit
xorq %rdi, %rdi        # arg 1: status 0
syscall                # does not return

What syscall itself does is small and fast: it copies the address of the next instruction into %rcx and the flags into %r11 (this is the clobber), switches to kernel mode, and jumps to a kernel entry point whose address the kernel installed in a machine register at boot. The kernel's entry code switches to the process's kernel stack, saves the user registers, checks that %rax holds a valid number, and indexes the system-call table (the same number-to-handler-array pattern as the exception table) to reach sys_write. That routine validates the arguments (is fd 1 open? are those 5 bytes readable in the user's address space?), performs the write, and leaves the result in the saved %rax slot. The exit path restores the user registers and executes sysret, which jumps back to the address in %rcx, restores the flags from %r11, and drops to user mode. The program continues at the instruction after syscall with %rax holding 5.

From the program's perspective a system call looks like a function call: set up arguments, one instruction, result in %rax. The differences are everything this lesson is about: it runs in kernel mode, on the kernel stack, and its entry point is chosen by the kernel, not the caller. A user program cannot jump to an arbitrary kernel address; it can only present a number, and the kernel decides what that number means.

Processes: the abstraction ECF builds

Step back from the mechanism and look at what it enables. The most consequential application of ECF is the process: the illusion, granted to every running program, that it has the processor to itself. A program sees its own instructions executing one after another — a private, uninterrupted logical control flow — while in physical fact the CPU is multiplexed among many programs, tens of milliseconds at a time.

The mechanism that sustains the illusion is a pairing of two things from this lesson. The kernel programs an interval timer to raise an interrupt every few milliseconds; each timer interrupt hands control to the kernel, through the exception table with the running process's state saved, whether that process likes it or not. The kernel may then perform a context switch: save the rest of process A's state (its general-purpose registers, %rip, flags, stack pointer, and its page-table base register), restore the previously saved state of process B, and iret into B instead of A. Everything that made the CPU be process A is on A's kernel-side record; when A's turn comes again, the same restore replays it, and A resumes mid-computation with no way to tell it was ever paused. The next lesson walks the context switch in detail, and the multicore module builds on the process abstraction from there.

Without asynchronous ECF there is no preemption. A cooperative system can share the CPU only if every program volunteers to yield. The timer interrupt is what lets the kernel take the CPU back, and it is the difference between an operating system that hosts programs and one that is hostage to them.

Signals: ECF surfaced to user code

One rung further up, the same pattern — an event diverts control to a handler — is offered to programs themselves as signals. A signal is a small message the kernel delivers to a process; delivery interrupts the process's normal flow and runs a user-registered handler function, after which control returns to wherever the process was. It is exception handling rebuilt in software, one privilege level up.

The two layers connect directly. When the page-fault handler finds a genuinely invalid address, its abort is not a machine halt: it is the delivery of SIGSEGV to the offending process. The divide error becomes SIGFPE. Typing control-C makes the terminal driver send SIGINT. In each case a hardware-level exception (or interrupt) enters the kernel, and the kernel re-expresses it as a user-level event that the process may catch with a handler, ignore, or die from (SIGSEGV's default). A program that installs a SIGSEGV handler is registering an entry in its own private exception table, with the kernel playing the role the hardware plays one level down.

The software echoes of hardware ECF

CS:APP develops exceptional control flow as a hardware and kernel mechanism. The same shape — divert control to a handler, then resume — reappears at every layer above the kernel, and recognizing it makes a range of language features look like one idea. The vocabulary is shared, but the enforcement gets weaker at each level up.

At the bottom, a hardware exception is enforced by the CPU and is non-optional: the divide-by-zero happens whether the program consents or not. One level up, a signal is the kernel re-offering that event to the process; the process may catch it, but the delivery is still the kernel's decision. Higher still, purely in user space, a language runtime builds control-flow diversions with no hardware help at all. C's setjmp/longjmp saves a snapshot of the registers and stack pointer and later restores it, jumping non-locally back up the call stack — a hand-rolled version of the state save-and-restore the hardware does for an exception.¹ C++ and Rust build exception handling and panic/unwind on top of that same stack-unwinding capability, walking the call frames and running destructors as they go; Go's panic/recover and every language's try/catch are the same pattern. The through-line is exact: an event interrupts the normal sequence, control transfers to a handler chosen not by the interrupted code but by a surrounding registration, and (sometimes) control resumes.

One practical wrinkle the hardware view predicts: mixing the layers is delicate. Only a small set of functions are async-signal-safe — safe to call from inside a signal handler — because a signal can arrive between any two instructions, including halfway through malloc's update of its internal data structures. A handler that calls a non-reentrant function can corrupt state the interrupted code was midway through, the user-space version of exactly why the kernel switches to a known-good stack and masks further interrupts on entry. The mechanism at every level has to reckon with the same fact: an asynchronous event can strike at an inconvenient moment, and safe handling means assuming it will.

The most important asynchronous case — the interrupt — is also the gateway to the I/O system and to switching between processes. The next lesson takes up interrupt controllers, the user/kernel boundary, DMA, and how a disk read actually reaches memory.