Invisible Stickers on X-rays: Adversarial Examples as a Threat Model for Diagnostic Imaging AI

A Brief Mechanics of Fooling a Network

An adversarial example is an input x + δ, with δ small under some norm, such that the model’s label changes. The fast gradient sign method uses the loss gradient to pick a direction that increases error (Goodfellow et al.). Iterative methods and expectation-over-transformation produce perturbations that survive printing, lighting, or JPEG compression. In imaging AI, the pixel grid is already a discretization of a physical acquisition; that gives attackers—and nature—more room than a standardized ImageNet thumbnail.

Finlayson et al. demonstrated that standard medical CNNs could be flipped by tiny perturbations and discussed insurance-fraud and sabotage scenarios (Finlayson et al.). Ma et al. surveyed medical adversarial work and noted that many papers still use digital-only noise rather than scanner-realistic threats. The gap matters: a perturbation that exists only as a NumPy array is a different risk than a sticker on a calibration phantom.

Clinic-Specific Attack Surfaces

Hospitals are not open internet APIs, which lowers some risks and raises others. An insider could alter images in a PACS. A vendor model might be queried enough times to approximate gradients. More mundanely, changing reconstruction kernels, kVp, or compression settings can shift a model that was trained on one protocol—functionally an attack without malice (DeGrave et al. documented shortcut learning on hospital tokens and metal tokens).

Defenses include adversarial training, input purification, and ensembling. Adversarial training often hurts clean accuracy and still fails against a slightly different attack (Athalye et al.). Detection methods that look for “weird high-frequency noise” fail when perturbations are optimized to be low-frequency. Regulatory language about “locked” algorithms does not automatically include robustness test suites. For a student project that claims 97% sensitivity, the missing figure is performance under protocol shift and under a simple FGSM budget.

What Would Count as Evidence of Safety

A mid-size paper cannot solve certified robustness. It can insist on a threat model: who can modify pixels, at what amplitude, before or after compression, with or without access to weights. Without that, “the model is robust” is unfalsifiable.

The constructive niche for student researchers is evaluation, not a new attack name. Report clean metrics and a small, pre-registered perturbation budget; test on a second hospital’s technique factors if possible; refuse to treat a Kaggle leaderboard as a clinical claim. Adversarial examples are a lens on brittleness. In medicine, brittleness is a patient-safety problem, not a party trick.

Threat Models That Matter in a Hospital

Academic attacks often assume an adversary who can add a crafted overlay to a PNG. A hospital threat model is different: a compromised PACS, a malicious insider, a supply-chain tweak to a preprocessing library, or a model update that silently changes operating points (Finlayson et al.). Physical-world stickers on a scanner gantry are less likely than a software path that already has write access to pixels.

That does not make digital adversarial examples irrelevant. They prove that the decision surface is brittle. If a 1% intensity perturbation can flip “malignant” to “benign” in a published network, the network is correlating textures that radiologists do not use as primary signs. Brittleness is a clinical quality problem even when no attacker appears.

Evaluation beyond AUC

Receiver-operating curves on i.i.d. test splits hide distribution shift: new scanners, pediatric versus adult cohorts, and under-representation of rare diseases. Adversarial accuracy is one stress test among others, including site-shift and label-noise audits (Oakden-Rayner et al.). A model that is both shift-fragile and attack-fragile should not be described as “superhuman” because it beat a small reader study.

Regulation is catching up in language more than in metrics. Transparency about training sites, intended population, and known failure modes is more useful than a single robustness number. Ensemble disagreement and uncertainty maps may help readers, but they also fail when all members share the same spurious cue.

Conclusion

Adversarial examples in medical imaging are a niche diagnostic of a broader failure: high AUC without a causal or anatomical guarantee. Defense research should be scored on whether radiologists still catch the miss, not only on whether a particular attack’s perturbation norm rose.

For student builders, the assignment is to report a shift test and a simple attack budget beside accuracy. For student clinicians, it is to treat an imaging AI as a second reader with unknown blind spots, not as a sensor. Complexity here is sociotechnical: pixels, incentives, and liability share the same DICOM file.

Works Cited