1 of 37

DeformRS: Certifying Input Deformations

with Randomized Smoothing

�

* indicates equal contribution

Motasem Alfarra*

(KAUST)

Adel Bibi*

(Oxford)

Naeemullah Khan

(Oxford)

Philip H. S. Torr

(Oxford)

Bernard Ghanem

(KAUST)

2 of 37

Adversarial Attacks

3 of 37

Adversarial Attacks

The classical threat model

4 of 37

Adversarial Attacks

The classical threat model

Additive attack

5 of 37

Adversarial Attacks

The classical threat model

Additive attack

Attack is norm

bounded

6 of 37

Defenses Against Adversaries

does not scale to large networks

provides guarantees

no guarantees

scales to large networks

TRADES

MART

HYDRA

AWP

Wong et al. ICML18.

CROWN

PROVEN

Empirical Defenses

Certified Defenses

7 of 37

Randomized Smoothing

Defenses Against Adversaries

Empirical Defenses

Certified Defenses

does not scale to large networks

provides guarantees

no guarantees

scales to large networks

TRADES

MART

HYDRA

AWP

Wong et al. ICML18.

CROWN

PROVEN

Provides probabilistic guarantees

that scales to large networks

8 of 37

Goal of this Paper

Build models that are certifiably robust

Randomized Smoothing

9 of 37

Goal of this Paper

Randomized Smoothing

Build models that are certifiably robust

Robustness beyond additive adversaries: geometric deformations

10 of 37

Goal of this Paper

Randomized Smoothing

Build models that are certifiably robust

Robustness beyond additive adversaries: geometric deformations

11 of 37

Goal of this Paper

Randomized Smoothing

Robustness beyond additive adversaries: geometric deformations

Build models that are certifiably robust

12 of 37

Motivation

Attacking parameterized simulators

SADA: Semantic Adversarial Diagnostic Attacks for Autonomous Applications, Hamdi et al. AAAI20.

13 of 37

Examples to Geometric Deformations

Random Gaussian Perturbations to Pixel locations

14 of 37

Examples to Geometric Deformations

Rotation, Translations, Scaling, and Sheering of the Pixel Grid

15 of 37

Examples to Geometric Deformations

General Affine Deformations of the Pixel Grid

16 of 37

Examples to Geometric Deformations

Deformations of the Pixel Grid Sampled from a DCT Basis

17 of 37

Deformation Smoothed Classifier

Consider the geometrically interpolated image:

18 of 37

Deformation Smoothed Classifier

Interpolation function

vector field

deformation

Consider the geometrically interpolated image:

19 of 37

Deformation Smoothed Classifier

Then, the deformation smoothed classifier is given as follows:

Interpolation function

average predictions

20 of 37

Deformation Smoothed Classifier

Interpolation function

average predictions

21 of 37

Deformation Smoothed Classifier

Interpolation function

average predictions

What about parameterized vector fields?

e.g. Flows representing rotations

22 of 37

Parametric Smoothed Classifier

Interpolation function

Then, the deformation smoothed classifier is given as follows:

average predictions

23 of 37

Interpolation function

average predictions

Parametric Smoothed Classifier

24 of 37

Certifying Deformations

We certify both the deformation smoothed classifier and the

parametric deformation smoothed classifier

The base classifier is trained with Gaussian and uniform

noise augmentations

25 of 37

Experiments

DeformRS can certify individual parameterized deformations

26 of 37

DeformRS can certify individual parameterized deformations

Experiments

27 of 37

DeformRS can certify individual parameterized deformations
DeformRS can certify affine deformations