Adversarial Defenses

This builds on Adversarial Robustness, which set up the threat model, the fast gradient sign method, projected gradient descent, and adversarial training as a min-max problem. Those tools tell an attacker how to break a network and give the defender one honest response: train on the worst case. This lesson is about everything on the defense side that the first pass left open — how to prove robustness rather than merely measure it, why attacks crafted on one model fool another, and the long history of defenses that looked strong and were not.

Why a small ball matters

Start from the geometry, because it explains what a defense must actually accomplish. A clean point can sit a vanishing distance from the decision boundary while still being classified correctly. Standard training only requires that $x$ land on the right side; it does not enforce any margin,¹ so the boundary is free to pass arbitrarily close. Robustness requires more: every point in the $ϵ$-ball around $x$ must stay on the correct side, which forces the boundary out beyond radius $ϵ$.

This is the same brittleness flagged for deep nets in what is deep learning, made quantitative: the failure is not that the model is wrong on $x$, but that the correct region does not even contain a small ball around $x$. Adversarial training, from the previous lesson, attacks this by pushing the boundary out; the defenses below either prove the boundary is far enough, or fail by only appearing to move it.

Certified defenses: randomized smoothing

Adversarial training gives an empirical number: robustness against the strongest attack tried so far, with no guarantee a cleverer attack will not appear. A certified defense instead proves a radius $R$ such that no perturbation with $\Vert \delta {\Vert }_2\le R$ can change the prediction. The most scalable construction is randomized smoothing, which turns any base classifier $f$ into a smoothed classifier $\hat{f}$ that reports the class $f$ predicts most often under Gaussian input noise:

Averaging over noise smooths the decision function, and a smooth function cannot change its majority vote quickly. Concretely, let $p_A$ be the probability the base classifier returns the top class $c_A$ under the noise and $p_B$ the probability of the runner-up. Then $\hat{f}(x+\delta )=c_A$ for every $\delta$ with

where ${\Phi }^{-1}$ is the inverse standard-normal CDF. The certified radius grows with the noise level $\sigma$ and with the confidence gap between the top two classes: a model that is barely sure ($p_A$ just above $p_B$) certifies nothing, while a model that is almost always right under noise ($p_A\to 1$, so ${\Phi }^{-1}(p_A)\to +\infty$) certifies a large ball. The cost is that a larger $\sigma$ blurs the input and lowers clean accuracy — the same robustness-accuracy tension, now controlled by the noise scale. In practice $p_A,p_B$ are unknown and estimated from $n$ noise samples with a confidence interval, so the certificate is probabilistic: it holds with probability $1-{\eta }_{\text{err}}$ over the sampling.

For example, suppose a smoothed classifier at noise scale $\sigma =0.5$ returns the top class on $p_A=0.99$ of the noise draws and the runner-up on $p_B=0.01$. The standard-normal quantiles are ${\Phi }^{-1}(0.99)\approx 2.33$ and ${\Phi }^{-1}(0.01)\approx -2.33$, so the certified radius is

No $L_2$ perturbation of norm below $1.16$ can flip the smoothed prediction — a hard guarantee, not an empirical one. Halve the confidence gap (say $p_A=0.9$, ${\Phi }^{-1}(0.9)\approx 1.28$, $p_B=0.1$, ${\Phi }^{-1}(0.1)\approx -1.28$) and the radius collapses to $0.25\times 2.56\approx 0.64$: the certificate is only as wide as the model is confident under noise. Raising $\sigma$ widens $R$ for a fixed confidence gap, but a larger $\sigma$ also lowers $p_A$ by blurring the signal, so the two effects trade off; $\sigma$ is tuned, not maximized.

Randomized smoothing is not the only certification route, but it is the one that scales to large models, because it treats the base network as a black box and only needs its outputs under noise. The alternatives propagate a guaranteed range through the network layer by layer; they are tighter but limited to small models.

Transferability and black-box attacks

The linearity hypothesis from the previous lesson has a corollary the attacker exploits when it cannot see $\theta$: adversarial examples transfer. Two models trained on the same data tend to learn similar near-linear functions, so their input-gradients point in similar directions, and a $\delta$ that raises the loss on one raises it on the other. An attacker with no gradient access trains its own surrogate model, runs white-box FGSM or PGD against the surrogate, and ships the resulting $x^{\prime }$ at the target.

The success rate falls off with how different the two models are, but for undefended targets it is high enough to make black-box robustness no easier to earn than white-box. Transfer is also the diagnostic tool of last resort: because it needs no gradient of the target at all, a transfer attack that beats a white-box attack is proof the white-box gradient was being masked.

Defenses and their failure modes

Beyond adversarial training and certified bounds, the persistent failure mode is gradient masking: a defense that lowers the apparent attack success by obfuscating or zeroing the gradient the attacker differentiates, without actually flattening the loss surface. It collapses the moment the attacker estimates the gradient another way or transfers an example from a surrogate model.

FGSM and PGD fix the budget $ϵ$ and maximize the loss inside it. The Carlini–Wagner (CW) attack inverts the question: fix misclassification as a hard requirement and minimize the perturbation needed to achieve it. It solves

where $Z_i$ are the pre-softmax logits, $t$ is the target class, $c$ trades the two terms, and $\kappa \ge 0$ sets a confidence margin. The surrogate loss $h$ is zero exactly when the target logit exceeds every other logit by at least $\kappa$, so the optimizer pushes just past the boundary and then shrinks $\delta$. Because CW optimizes the size of the perturbation rather than climbing a fixed-budget loss, it finds smaller perturbations than PGD and defeats defenses tuned to a specific $ϵ$.

Obfuscated gradients: a false sense of security

For several years after FGSM, a steady stream of defenses reported high robust accuracy against PGD, and almost all of them were later broken.² The common fault was shared: the defense lowered the measured attack success by corrupting the gradient the attacker reads, not by moving the decision boundary. PGD climbs the loss using ${\nabla }_x\ell$; if that gradient is shattered, randomized, or saturated, PGD stalls and reports the model as robust while the boundary still sits a perturbation away. This is obfuscated gradients, the most common form of gradient masking. It postdates Goodfellow's text, which already names the honest baseline.³

Obfuscated gradients come in three forms, distinguished by how they corrupt the gradient the attacker differentiates through.

Three symptoms are diagnostic. A defense is masking the gradient when (i) a one-step attack like FGSM beats iterative PGD, the reverse of the true ordering; (ii) black-box or transfer attacks beat white-box ones, though white-box has strictly more information; or (iii) robust accuracy stays high at large $ϵ$ where an unbounded adversary should win outright. Any of these signals that the gradient, not the model, is doing the defending.

Adaptive attacks that defeat it

To address this, evaluation uses the adaptive attack: an attack rebuilt with full knowledge of the defense, so the gradient it follows is the gradient of the defended model, not of a raw surrogate. Two constructions recover a usable gradient from the two ways defenses break it.

BPDA (backward-pass differentiable approximation) handles a non-differentiable component $g(\cdot )$. Run the true $g$ on the forward pass to get the real prediction, but on the backward pass replace $g$ by a differentiable surrogate $\hat{g}\approx g$ whose Jacobian is computable. The common surrogate is the identity, $\hat{g}(x)=x$, valid whenever $g$ is an input purifier with $g(x)\approx x$:

EOT (expectation over transformation) handles a stochastic defense that draws a transform $t\sim \mathcal{T}$ each query. A single sample of ${\nabla }_x\ell (t(x))$ is noisy, but its expectation recovers the gradient of the quantity the defense reports (the expected loss), and gradients commute with the expectation:

Averaging $m$ samples drives down the variance and recovers a descent direction the randomization cannot hide. When neither applies cleanly, a transfer attack from a separately trained, fully differentiable surrogate sidesteps the masked gradient entirely, which is why transfer beating white-box is a red flag.

The state of the robustness arms race

Goodfellow's §7.13 predates almost the entire defense literature, so the modern picture rests on public papers. Two results anchor it. First, the obfuscated gradients audit (Athalye, Carlini & Wagner, 2018) took seven defenses accepted at a single conference and broke six of them with BPDA and EOT within weeks — the study that made adaptive evaluation a requirement. Second, the RobustBench benchmark (Croce et al., 2021) standardized robustness measurement around AutoAttack, an ensemble of four parameter-free attacks (two PGD variants, the FAB minimum-norm attack, and the gradient-free Square attack) whose diversity makes gradient masking hard to hide behind. Under that yardstick, PGD-based adversarial training (Madry et al., 2018) and its refinement TRADES (Zhang et al., 2019), which splits the loss into a clean-accuracy term and an explicit robustness regularizer to tune the tradeoff directly, remain the defenses that survive.

The honest summary is that no scalable defense yet closes the gap on large-$ϵ$$L_{\infty }$ threat models: certified radii from smoothing stay small, and empirical robust accuracy on hard benchmarks sits well below clean accuracy. The lasting lesson is methodological — robustness is a claim that must survive the strongest adaptive attack, and any number reported against a fixed non-adaptive attack means nothing about a defense engineered to mask its gradient.

Takeaways

• Robustness is a margin requirement: the correct region must contain the whole $ϵ$-ball around $x$, so a defense must push the boundary out, not merely classify $x$ correctly.
• Certified defenses prove a radius rather than measure one. Randomized smoothing classifies under Gaussian noise and certifies an $L_2$ ball of radius $R=\frac{\sigma }{2}({\Phi }^{-1}(p_A)-{\Phi }^{-1}(p_B))$ — probabilistic, $L_2$, and paid for in clean accuracy through the blur.
• Adversarial examples transfer across models with similar near-linear functions, so an attacker with no gradient access attacks a surrogate and ships the result; transfer beating white-box is a diagnostic for a masked gradient.
• Beware gradient masking / obfuscated gradients (shattered, stochastic, vanishing): they break the attacker's gradient, not the boundary, and collapse under adaptive attacks — BPDA for non-differentiable ops, EOT for randomization, transfer as a baseline.
• Every robustness claim needs adaptive evaluation; PGD-based adversarial training and TRADES are the defenses that survive standardized benchmarks, and no scalable method yet closes the large-$ϵ$ gap.