How Adaptive are�Adaptive Test-Time Defenses?

Francesco Croce*, Sven Gowal*, Thomas Brunner*, Evan Shelhamer* Matthias Hein, Taylan Cemgil

Categorization, Evaluation, and Recommendations

🗺�Recommendations

How can we reliably measure robustness of adaptive defenses?

We need such measures, because�adaptive defenses continue to be proposed.

🔍�Case Study

We categorize adaptive defenses�and explain their pluses & minuses.

We evaluate nine recent defenses,�and we find that their robustness falls short of their claims.

❓�Adaptive Test-Time Defense

What is a test-time defense?�What is an adaptive defense?

Adversarial Attacks & Defenses

=

88% tabby cat

99% guacamole

in

Deep nets are vulnerable to adversarial attacks: small input (x) perturbations (𝜹) that result in errors

• Train-Time Defenses: devise models, losses, …, and optimization algorithms for training classifier, e.g. adversarial training (Madry et al., 2018). They are static and do not change during testing�
• Test-Time Defenses: alter inference during testing by altering the input or model, e.g. test-time data augmentation. While these alter inference, they do not adapt, as their alterations do not depend on the input

illustration credit: Athalye and Carlini

Adaptive Test-Time Defenses

Adaptive test-time defenses

• alter inference and condition their changes on the input by optimization�
• iteratively update (during inference) the input x or parameters 𝜃 of the network�to improve robustness to adversarial attack

​

Potential benefits

• preserving clean accuracy, by adapting differently to natural inputs and attacked inputs,�unlike train-time defenses
• improving robust accuracy, by changing the defense as a function of the attack,�unlike non-adaptive test-time defenses

​

Note: Test-time adaptation has shown improvement for natural shifts, so why not adversarial shifts?�(Sun et al. ICLR'20, Schneider et al. NeurIPS'20, Wang et al. ICLR'21, …)

Principles of Adaptive Defenses

input adaptation or "purification" optimizes the input then passes it�to the model.�

model adaptation or "test-time training"�optimizes the parameters or latents of the model.

Both principles update by iterative optimization during testing

Building Blocks of Adaptive Defenses

iteration algorithm (IA) iteratively updating test-time optimization problems by e.g. gradient descent

auxiliary networks (AN) equipping the model with another network to define the loss or update for optimization

randomization (R) randomizing explicitly, by adding noise, or implicitly, by sampling data

external data (ED) exploiting additional data for prediction, so that it does not depend solely on test inputs

Building Blocks of Adaptive Defenses: Iteration

iteration algorithm (IA) iteratively updating test-time optimization problems by e.g. gradient descent

auxiliary networks (AN) equipping the model with another network to define the loss or update for optimization

randomization (R) randomizing explicitly, by adding noise, or implicitly, by sampling data

external data (ED) exploiting additional data for prediction, so that it does not depend solely on test inputs

gradient iteration

model iteration

Building Blocks of Adaptive Defenses: Auxiliary Networks

iteration algorithm (IA) iteratively updating test-time optimization problems by e.g. gradient descent

auxiliary networks (AN) equipping the model with another network to define the loss or update for optimization

randomization (R) randomizing explicitly, by adding noise, or implicitly, by sampling data

external data (ED) exploiting additional data for prediction, so that it does not depend solely on test inputs

classifier

auxiliary network

Building Blocks of Adaptive Defenses: Randomization

iteration algorithm (IA) iteratively updating test-time optimization problems by e.g. gradient descent

auxiliary networks (AN) equipping the model with another network to define the loss or update for optimization

randomization (R) randomizing explicitly, by adding noise, or implicitly, by sampling data

external data (ED) exploiting additional data for prediction, so that it does not depend solely on test inputs

explicit: adding noise

Building Blocks of Adaptive Defenses: External Data

iteration algorithm (IA) iteratively updating test-time optimization problems by e.g. gradient descent

auxiliary networks (AN) equipping the model with another network to define the loss or update for optimization

randomization (R) randomizing explicitly, by adding noise, or implicitly, by sampling data

external data (ED) exploiting additional data for prediction, so that it does not depend solely on test inputs

88% tabby cat

model

Case Study: Categorization

We study nine recent adaptive test-time defenses from 2020-2022 that span the principles and building blocks.�

All are iterative, most apply input purification, and many include auxilary networks.

Case Study: Evaluation

Our evaluation measures defenses on the CIFAR-10 dataset, with the norm, and bound ε=8/255.

The best attack varies, but a strong gradient attack (APGD), approximation of gradients (BPDA), and expectations (EoT) are common.

Case Study: Evaluation

Our evaluation measures defenses on the CIFAR-10 dataset, with the norm, and bound ε=8/255.

💥 actually hurts!

* CLC uses 2/255

*

💥

💥

💥

💥

The best attack varies, but a strong gradient attack (APGD), approximation of gradients (BPDA), and expectations (EoT) are common.�No adaptive defense improves over the static defense of adversarial training.

Case Study: Evaluation

Our evaluation measures defenses on the CIFAR-10 dataset, with the norm, and bound ε=8/255.

💥 actually hurts!

🏅 highest gain

* CLC uses 2/255

*

💥

💥

💥

💥

🏅

The best attack varies, but a strong gradient attack (APGD), approximation of gradients (BPDA), and expectations (EoT) are common.�No adaptive defense improves over the static defense of adversarial training.

Case Study: Evaluation

Our evaluation measures defenses on the CIFAR-10 dataset, with the norm, and bound ε=8/255.

💥 actually hurts!

🏅 highest gain

🐌

🐌

🐌

🐌

🐌

🐌

The best attack varies, but a strong gradient attack (APGD), approximation of gradients (BPDA), and expectations (EoT) are common.�No adaptive defense improves over the static defense of adversarial training. All take more computation.

🐌 >10x slower

* CLC uses 2/255

💥

💥

💥

💥

🏅

*

Case Study: Summary

• Complex defenses are often more difficult to evaluate (Tramèr, arXiv'20)�
• There is not a single attack which works across all defenses, but we can�combine existing techniques, depending on the building blocks of each�
• Adaptive test-time defenses that make use of a robust static model�do not improve its robustness, and can even degrade it(!)�
• Adaptive test-time defenses using a nominal, non-robust static model�might yield some robustness, but less than the degree obtained by�adversarial training�
• Test-time adaptation increases inference time by multiplicative factors,�or even orders of magnitude(!)

Come to the Poster for Recommendations and the Latest Defenses

We have recommendations and code for evaluating adaptive defenses,�and we will need such a toolkit because adaptive defenses keep appearing!

​

​

​

​

​

​

​

�

​

Thank you!�

LINAC�Lossy Implicit Network Activation Coding�ICML'22

DiffPure

Diffusion for Adversarial Purification�ICML'22

​

​