Interrupts and the Kernel

The previous lesson placed interrupts among the four exception classes: asynchronous events raised by I/O devices. This lesson develops them in full. Interrupts are how a processor that executes billions of instructions per second coexists with devices that take milliseconds to respond, without sitting idle waiting on them. The same mechanism, applied to a timer, is how the operating system takes the CPU back from one process and gives it to another. Along the way we fix the user/kernel privilege boundary that every interrupt crosses, meet the interrupt controller that funnels many devices into one CPU pin, walk the context switch, and work out the actual costs: what polling wastes, what an interrupt costs, and how DMA moves bulk data while the CPU does something better.

User mode and kernel mode

A processor runs in one of two privilege levels, recorded in a control register.

The boundary is enforced by hardware and is the foundation of protection: it is what stops a buggy program from halting the CPU, reprogramming the MMU, or reading another process's memory. The supervisor bit in each PTE is part of the same scheme. What counts as privileged is the set of instructions that could subvert the system from below: hlt (stop the processor), cli/sti (disable and enable interrupts), loads of the page-table base or the exception-table base, and direct device access through in/out. A user-mode program that executes any of them takes a general-protection fault, and the kernel's #GP handler turns the attempt into a SIGSEGV.

User code reaches kernel services only by raising an exception, which switches the mode for it: a trap (syscall) for a deliberate request, or an interrupt for an asynchronous device event. Both vector through the exception table into kernel mode, and iret drops back to user mode on the way out. There is no third door. The mode bit cannot be written directly from user mode, and every kernel-mode entry point is an address the kernel itself installed.

The interrupt controller

A machine has one CPU interrupt input (per core) and dozens of interrupt sources: disks, network cards, keyboards, timers, other cores. The component in between is the interrupt controller, and it has three jobs: collect request lines from every device, decide which pending request the CPU should see first (prioritization), and tell the CPU which device it was by supplying the vector number that indexes the exception table.

The classic part is the 8259A PIC (programmable interrupt controller): eight input lines, cascadable to fifteen, each line mapped to a consecutive vector. Modern x86 systems replace it with the APIC architecture: a local APIC attached to each core, plus an I/O APIC that routes device lines to whichever core the OS chooses, with an arbitrary vector per line; newer devices skip wires entirely and signal by a special memory write (message-signaled interrupts). The architecture changed to scale across cores, but the contract did not: the controller presents one prioritized request at a time and hands the CPU a vector number.

Two controls matter to software. Masking: the kernel can block interrupts, either globally at the CPU (the interrupt-enable flag, cleared by cli and set by sti, both privileged) or per line at the controller. Handlers run critical code that must not be re-entered carelessly, so the CPU automatically masks interrupts on entry to a handler and unmasks on iret. Nesting: a kernel may re-enable interrupts inside a long handler so that a higher-priority request (say the timer) can preempt a lower one (a slow device); handlers then nest on the kernel stack exactly as calls do. The practical discipline that falls out is to keep handlers short: acknowledge the device, record what must be done, defer the rest to ordinary kernel code running with interrupts enabled.

An interrupt, end to end

A device — say a disk that has finished reading a block — does not get to run code on the CPU; it gets to raise a signal. It asserts a line into the interrupt controller, which prioritizes pending requests and, if interrupts are enabled, raises the processor's interrupt input. The CPU finishes its current instruction, then takes the interrupt: it saves state, switches to kernel mode, and jumps through the exception table to that device's interrupt handler. The handler services the device — copies or acknowledges the data, tells the controller the interrupt is handled so the next one can be delivered, wakes any process waiting on the I/O — and executes iret, resuming whatever the CPU had been running.

The timing has one subtlety worth naming: the CPU checks for pending interrupts between instructions. An asynchronous signal never tears an instruction in half; the interrupted instruction either completed or (for a fault) counts as not-run. That is what makes the saved %rip a clean resumption point, and it is the property pipelined implementations work hard to preserve: whatever the pipeline was doing, the exception must appear to strike at an instruction boundary.

The timer interrupt and preemptive scheduling

The interrupt mechanism does double duty. Besides devices reporting completions, the kernel programs the interval timer to interrupt on a fixed period, typically every 1 to 10 milliseconds. Each tick hands control to the kernel no matter what is running. The timer handler updates time accounting, and the scheduler decides whether the current process has had enough CPU. If it has, the kernel does not iret back into it. It performs a context switch and returns into a different process instead.

The switch is mechanical but total: every piece of state that makes the CPU be process A must be put aside and replaced with process B's. Switching the page-table base changes the virtual-to-physical mapping, so the same virtual addresses now name B's memory; the TLB must therefore be flushed (or its entries tagged by process) so stale translations from A are not reused. The flush is the expensive part. The register save and restore are a few dozen memory operations, but the switched-to process starts with a cold TLB and a cache full of someone else's data, so the true cost of a context switch is paid in the misses that follow it. This is why time slices are milliseconds and not microseconds: the switch overhead must stay small relative to the slice.

There is a second trigger besides the timer. When a process makes a syscall that cannot complete immediately — a read from a disk that will take milliseconds — the kernel does not spin waiting inside the syscall. It marks the process blocked, context-switches to a runnable one, and lets the eventual completion interrupt mark the blocked process runnable again. Preemption and I/O overlap are the same machinery viewed from two sides, and the disk-read walkthrough at the end of this lesson uses both.

Polling versus interrupts, with numbers

How does software learn an I/O operation is done? There are two strategies.

• Polling has the CPU repeatedly read a device's status register in a loop until it reports ready. It is simple, and it notices readiness almost instantly. But every loop iteration is a cycle the CPU could have spent on something else.
• Interrupt-driven I/O lets the CPU issue the request and go do other work; the device raises an interrupt when finished, exactly as above.

The trade is worth quantifying, because it flips depending on the device. Take a 3 GHz processor and a disk read that completes in 4 ms. Polling spins for $0.004\times 3\times 10^9=12,000,000$ cycles — twelve million cycles of status reads, enough to execute tens of millions of instructions of useful work. The interrupt path instead costs one handler round trip: saving and restoring state, running the handler, and the cache and TLB disturbance it leaves behind — call it on the order of $10,000$ cycles. For the disk, interrupts win by three orders of magnitude.

Now replace the disk with a fast NVMe device that completes a read in 10 microseconds: the wait is $0.00001\times 3\times 10^9=30,000$ cycles, the same order as the interrupt overhead itself. Sleeping and being woken can take longer than the I/O. This is why high-performance storage and network stacks have swung back to polling for their fastest devices: when the wait is shorter than the cost of being interrupted, spinning is the efficient choice. The rule is not interrupts good, polling bad but a comparison of two costs: expected wait time against handler round trip.

Device registers: how the CPU talks to hardware

Interrupts are the device-to-CPU direction. The CPU-to-device direction goes through device registers: small storage locations inside the device controller that the processor reads and writes like memory. A typical controller exposes at least three:

• a status register the CPU reads (ready? busy? error?),
• a command register the CPU writes to start an operation, and
• data/parameter registers for operands: a block number, a byte count, a memory address.

Two addressing schemes give the CPU access to them. Port I/O gives devices a separate, small address space (64 K ports on x86), reached only by the dedicated in and out instructions, which are privileged. Memory-mapped I/O is the modern default: a region of the physical address space is reserved for a device, and its registers appear at those addresses, so ordinary mov instructions read and write them. x86-64 keeps both, port I/O surviving for legacy devices; most controllers today are memory-mapped.

A device register is not memory, and the difference shows through the memory hierarchy: reading a status register has a side effect in the device, and two reads may return different values. Register regions are therefore marked uncacheable in the page tables; a cached copy of status: busy would never be updated. This is also why device access is kernel-only: the pages holding register regions are mapped with the supervisor bit, so user code cannot poke hardware directly.

As a concrete miniature (following CS:APP's example), suppose a simple disk controller is memory-mapped at address 0xa0. The kernel initiates a read of one logical block with three stores: first a command word saying read, and interrupt me when done, then the logical block number, then the destination address in main memory. Command issued, the kernel moves on; everything after that is the device's problem until the interrupt arrives. What fills the gap between here is a destination address and the data is there is DMA.

DMA: moving data without the CPU

Interrupts remove the waiting cost of I/O. One cost remains: if the CPU itself must copy every byte between the device and memory, a large transfer still consumes the processor word by word — a 1 MB read at 8 bytes per load/store pair is on the order of a quarter million instructions of pure copying. Direct memory access removes that too.

The walkthrough has three steps:

1. Setup. The CPU (kernel code) writes the transfer parameters into the controller's registers: which blocks to read, how many bytes, and the physical address of the destination buffer. The buffer's pages must be pinned — the VM system must not evict or move them mid-transfer, because the controller uses physical addresses and takes no page faults.
2. Transfer. The controller becomes a bus master: it drives read and write transactions on the memory bus itself, moving data device-to-DRAM (or the reverse) at device speed. The CPU is not merely idle during this, it is elsewhere, running other processes; it competes with the DMA engine only for memory-bus bandwidth.
3. Completion. The controller raises one interrupt. The handler checks status, unpins the buffer, and wakes whoever was waiting for the data.

The cache hazard. DMA writes go to DRAM, but the CPU reads through its caches — and the cache may still hold lines from the buffer's previous contents. Read those addresses after the transfer and the cache serves stale data, with no way to observe that DRAM changed underneath it. The mirror hazard exists on output: a dirty line sitting in the cache means DRAM does not yet hold what the CPU wrote, and a DMA read from DRAM ships the old bytes. Systems solve this in one of two ways: hardware makes DMA coherent (the DMA engine's bus traffic snoops the caches, invalidating or fetching lines as needed, as most x86 systems do), or software must explicitly flush dirty lines before an outbound transfer and invalidate the buffer's lines after an inbound one. Either way, the lesson generalizes: once more than one agent can touch memory, the memory the CPU sees through its cache and actual DRAM are different things that must be actively kept in agreement — the same problem that returns at full scale with multiple cores.

A disk read, end to end

All the pieces are now in place; here is the whole machine working. A process calls read(fd, buf, 8192) for a file whose blocks are not cached in memory.

1. Trap. syscall enters the kernel; the syscall handler finds the file's blocks and a free kernel buffer.
2. Command. The kernel writes the disk controller's registers: read these blocks, this many bytes, DMA them to this physical address, interrupt when done.
3. Block and switch. The read cannot complete for milliseconds, so the kernel marks the process blocked and context-switches to another runnable process. The CPU now runs unrelated work.
4. DMA. The disk reads the blocks and its controller masters the bus, depositing the data in the kernel buffer. No CPU involvement.
5. Interrupt. The controller raises its completion interrupt. Whatever process is running is briefly suspended; the handler checks status, marks the blocked process runnable, acknowledges the controller, and returns.
6. Resume. When the scheduler next picks the original process, the kernel copies the data from its buffer into the user's buf, sets %rax to the byte count, and sysrets. The program sees read return, milliseconds older and none the wiser.

Count what the mechanisms bought. During a 4 ms disk read, a 3 GHz CPU executes roughly 12 million cycles of other processes' work instead of spinning (the interrupt bought that), and not one of those cycles is spent copying bytes (DMA bought that). The process that asked for the data paid two context switches and a buffer copy. That is the shape of all modern I/O: the CPU orchestrates and is notified; the data moves on its own.

When interrupts cost too much

The polling-versus-interrupts analysis earlier in this lesson assumed a slow device, where an interrupt's round-trip cost is negligible against a multi-millisecond wait. Modern high-speed devices broke that assumption, and the system's software has spent the last two decades adapting — a live example of the same two costs, expected-wait versus handler-overhead, tipping the other way.

The pressure came first from networking. A 10-gigabit link receiving small packets can deliver well over a million packets per second, and one interrupt per packet would spend the entire CPU just entering and leaving handlers — an interrupt storm that starves the very work the packets are for. Two fixes are now standard. Interrupt coalescing lets the device hold its interrupt until several packets have arrived or a short timer expires, amortizing one handler round trip over many packets. And Linux's NAPI switches a busy interface out of interrupt mode entirely: on the first packet it disables that device's interrupts and has the kernel poll the device in a loop until the burst drains, then re-enables interrupts.¹ The device that this lesson taught to raise interrupts is told, under load, to stop.

Storage followed the same path. When flash and then NVMe drives cut latency from milliseconds to microseconds, the interrupt's round trip — the very cost the disk-read walkthrough treated as free — became comparable to the wait itself, the crossover this lesson's NVMe example flagged. Two responses now ship in production kernels. Hybrid polling lets a thread issuing a fast I/O spin briefly instead of sleeping, betting the device finishes before a context switch would have paid for itself. And user-space I/O frameworks — DPDK for networking, SPDK and io_uring's polled mode for storage — bypass the interrupt path altogether for the hottest devices, dedicating a core to poll rings of completions.² The unifying rule is the one this lesson stated and these systems rediscovered under load: interrupts win when waiting is expensive relative to being interrupted, and polling wins when it is not — and on today's fastest hardware, it often is not.

The kernel's three doors

Everything in this module enters the kernel one of three ways, and it is worth seeing them side by side.

One table, three rows, and every kernel entry in the system is one of them. The exception table serves the first two; the third has its own registered entry point but the same shape: save state, run kernel code, restore, drop privilege.

This closes the loop opened back in processor design: the fetch-decode-execute cycle is the normal flow, exceptions and interrupts are the diversions, and together with virtual memory they are what turns a raw datapath into a machine that can safely run many programs at once.