- 01Adversarial evasion attacks require no model access in the transfer attack setting. Perturbations crafted against a public surrogate model often transfer to the target production model.
- 02Certified defenses provide formal guarantees that no perturbation within a radius can change the output. They are the only class of defense that does not degrade against adaptive adversaries.
- 03Ensemble diversity reduces transfer attack success because an adversary optimizing against one model in the ensemble is less likely to fool all members simultaneously.
- 04Input purification before scoring adds a layer that many transfer attacks do not survive, at a modest latency cost that is acceptable in most production paths.
Adversarial evasion was, for several years, primarily a concern of academic machine learning researchers. That era has ended. Documented attacks against real production systems now span fraud detection, biometric access control, content moderation, and autonomous vehicle perception. The adversary does not need academic expertise. They need access to a surrogate model and a few days of compute.
The transfer attack setting is what makes evasion a practical production threat. An adversary who cannot query your production model can optimize a perturbation against a public model in the same model family, then apply that perturbation to inputs submitted to your system. Transfer rates between models of the same architecture family are high enough to make this economically viable.
The Threat Model in Plain Language
For a CISO audience, the relevant threat model is this. An adversary wants to pass an input through your AI system and receive an output that serves their goal. Without adversarial techniques, their input would be correctly classified or rejected. With adversarial perturbation, the input is subtly modified in a way that is imperceptible or irrelevant to a human reviewer but causes the model to produce the desired output.
The practical examples are a fraudulent document that passes an AI document verification check, a deepfake that passes a biometric liveness check, malware that passes an AI malware classifier, and content that bypasses an AI content moderation filter.
If your AI system gates access or enforces policy, evasion attacks are in scope.
Any AI system that makes a consequential binary decision based on model output is a potential target for evasion. The adversary only needs to change the output, not understand the model. Start with your highest consequence decision points and work backward to the model inputs.
Certified and Empirical Defenses
Defenses against adversarial evasion divide into certified and empirical. Certified defenses provide a formal guarantee that no perturbation within a specified radius can change the model output. The most widely deployed certified defense is randomized smoothing, which adds Gaussian noise to inputs and uses majority voting across many noisy copies to produce a certified prediction.
Empirical defenses, including adversarial training, input purification, and ensemble methods, do not provide formal guarantees but are often more practical at production scale. Adversarial training adds adversarial examples to the training set, improving robustness to the attack types seen during training. Input purification pipelines the input through a denoising step before scoring. Ensemble diversity means no single adaptive adversary strategy works against all members simultaneously.
/Defense Comparison for Production Deployment
| Defense | Guarantee type | Latency impact | Best suited for |
|---|---|---|---|
| Randomized smoothing | Certified radius | High, requires N forward passes | High security, lower throughput |
| Adversarial training | Empirical robustness | None at inference time | High throughput, known attack families |
| Input purification | Empirical, transfer resilient | Low to medium | Content moderation, document AI |
| Ensemble with diversity | Empirical, transfer resilient | Proportional to ensemble size | Fraud detection, access control |
Operational Workflow for Evasion Defense
Standing up an evasion defense program requires more than choosing a technical defense. It requires an operational workflow that keeps the defenses calibrated as the production model and the adversary landscape both evolve.
The workflow has four steps. First, threat model each production AI decision point by consequence level, identifying which systems are plausible evasion targets. Second, for each high consequence system, select a defense appropriate to the latency and throughput budget. Third, run adversarial robustness evaluations on each major model version before promotion to production. Fourth, monitor production inputs for perturbation signatures and feed confirmed adversarial examples back into the adversarial training dataset.
Starting the robustness evaluation practice.
The first evaluation does not need to be comprehensive. Pick the single highest consequence AI decision point in your environment. Run a standard evaluation suite against the current production model using publicly available attack implementations. The results will tell you your current certified radius and your robustness against the most common transfer attack families. That baseline is more valuable than any amount of planning without data.
- ▸Select one high consequence AI decision point for the first evaluation
- ▸Run standard robustness evaluation suite against production model
- ▸Record certified radius and transfer attack success rate as baseline metrics
- ▸Schedule the next evaluation for the next model version promotion
Metrics and Closing Actions
The metrics for an evasion defense program are certified robustness radius per production model, transfer attack success rate from a defined surrogate model family, adversarial example detection rate from production input monitoring, and time to retrain after a new adversarial example family is documented.
The closing action is to add adversarial robustness evaluation to your model promotion checklist. A model that has not been evaluated for robustness should not be promoted to a high consequence decision point. This is a process control, not a technical one, and it costs almost nothing to implement once the evaluation tooling exists.