1 of 46

Security vs Privacy in Machine Learning Models

P. Venkata Sai Charan (19111270)

Ph.D. Scholar, Department of Computer Science and Engineering

Indian Institute of Technology Kanpur

2 of 46

Agenda for Today’s Talk

Introduction to problem statement
Membership Inference attacks

Against State of the Art Robust Machine learning Models

Song, Liwei, Reza Shokri, and Prateek Mittal. "Privacy risks of securing machine learning models against adversarial examples." Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security. 2019.

Against Generative Models

Hayes, Jamie, et al. "LOGAN: Membership inference attacks against generative models." Proceedings on Privacy Enhancing Technologies 2019.1 (2019): 133-1

Defense Mechanisms
Discussion and conclusion

Important Research question addressed
Practicality
Novelty
Papers Relevance in todays security domain

3 of 46

Introduction

Machine Learning models originally designed without considering potential adversarial threats [4,5].

Adversarial Attacks

Evasion Attacks
Poisoning Attacks

In defense Security community came

up with ”Robust” Machine Learning Models[1]

Fig.1. Adversarial Attack on Machine Learning Models [3]

4 of 46

Introduction (contd..)

Robust Models Aim : Enhance Robustness of target models prediction unchanged for small area around each input.
Expectation: make ML model robust against any input
Reality: optimized only for Training set
Result : generalization error + sensitive towards training data

Membership Inference Attacks

Fig.2. Input perturbation[2]

5 of 46

Membership Inference Attack

Aims to infer whether a data point is part of the target’s model training set or not ?

Result : Leakage of users sensitive private information.

Fig.3. Membership Inference Attack [6]

6 of 46

At a glance

ML Models

Robust Models

Adversarial Attacks

Security Research

Community

Privacy

Research Community

MI Attack

Leakage of sensitive info used for training

Fig.4. Birds eye view of Problem statement

7 of 46

Adversarial Examples and Robust Defenses

8 of 46

State of the art Robust Defenses

9 of 46

1. PGD-Based Adv Training (Empirical)

Fig.5. Projected Gradient Descent [7]

10 of 46

2. Distributional Adv Training

Instead of using projection step, solve Lagrangian relxation to generate adversarial examples

Basic Idea of Lagrangian is to convert constrained problem into non constrained problem

Fig.6. Constrained optimization using Legrangian [7]

11 of 46

3. Diff-Based Adv Training

12 of 46

Problems with Empirical based methods

Fig.7. Fooling PGD-based Robust Model [12]

13 of 46

Abstract Interpretation Based Verification

Idea : Here we use Abstract interpretation to find maximum verified range

Fig.8. Abstract Interpretation Based verification [10]

14 of 46

Interval Bound Based verification

Fig.9. Interval Bound Based Verification [8,12]

15 of 46

Duality based verification

solving non-convex problems is hard and can have multiple local optima like the neural network.
whereas the convex problems have single global optima so have unique solution.

Idea: sometimes it might be hard to solve the actual optimization problem but it's dual problem might be easy to solve and figure out upper bound during verification process [11]

Fig.10. Dual problem solving [11]

16 of 46

Membership Inference Attack working…

Robust training algorithms aim to ensure model predictions unchanged for a small area around any data point.
Highly sensitive towards training data + Large Gap between train and test Accuracy leads to increase membership inference attack accuracy.

Fig.11. Cross Entropy loss distribution between Robust and Natural Models [1]

17 of 46

Inference strategies

In practice an adversary use shadow training technique [3].
Idea: Similar models trained on relatively similar data using same service behave in similar way
K-disjoint datasets distributed similarly on target training dataset.

Fig.12. Shadow Training in Practice [3]

18 of 46

Results (Empirically Robust Models)

Yale

Fashion

MNIST

CIFAR10

Compared to Natural Models, Empirical robust models increase membership inference 3.2X, 2X, 3.5X

19 of 46

Results (Verifiable Robust Models)

Yale

Fashion

MNIST

Compared to Natural Models, IBP robust model leads to increase membership inference > 75 %

20 of 46

Part-2: MIA on Generative Models

Facebook’s Director of AI Research, mentions GANs as ‘the most interesting idea in the last ten years in machine learning’.

Idea : Given Access to an Generative model & individual data record, can an attacker tell if that record was a part of training set ?

Fig.13. Simple GAN Architecture[2]

21 of 46

Threat Model

Membership Inference Attack on Generative Models

White-Box Attack Black-Box Attack

Access to internal parameters No Access to internal parameters

Assumptions : In both the settings attacker know data size of the but not its original points + some side info

22 of 46

White-Box Attack

In the white-box attacks, the adversary only needs access to the discriminator of GAN model.

Fig.14. White-Box Attacks on Generative Models[2]

23 of 46

Black-Box Attack

In Black-Box setting attacker has no info about :

Target Model Parameters
Target Model Architecture
Dataset used to train the model

Back-Box Attack

With No Auxiliary Knowledge With Limited Auxiliary Knowledge

24 of 46

Black-Box Attack (contd..)

Fig.15. Black box with No auxiliary knowledge[2]

Fig.16. Black box with Limited auxiliary knowledge[2]

25 of 46

Experimental Setup

One complaint about GAN is that its very hard to train them.
Training is not very stable; Involves lot of issues :

Mode Collapse (Generator Collapse into narrow distribution)
Discriminator learns very fast and Generator lags behind

So here they chosen 3 stable GAN for conducting Experiments :

DCGAN (Deep Convolution GAN )
DCGAN+ VAE (Deep Convolution GAN + Variational Auto Encoders)
BEGAN ( Boundary Equilibrium GAN)

Datasets : LFW(labeled faces in Wild), CIFAR-10, Diabetic Retinopathy

26 of 46

Results

Fig.17. Attack Accuracy plots on LFW, CIFAR1- and DR datasets[2]

Table.1 Overall Accuracy results for the attack

27 of 46

Potential Defenses

Dropout
Differential Privacy [13,14] (Quite popular Research Area – IOS, Google Chrome)
Using this Attack as defense

28 of 46

Importance of Research Problem

Importance:

Direct Privacy Breach
Establishing Wrong doing
Raised a fundamental question about balance b/w Security and Privacy in Machine Learning Community

Traditional ML

Adversarial ML

Empirical Robust ML

Verifiable Robust ML

Privacy Preserving Robust ML

29 of 46

Practicality

Cost to Perform Attacks : roughly $2352 for performing black box ( pricing as per Google MLaaS)
Attacks leads to : Property Inference Attacks + User Profiling ( what crowd say about you ?)

My Question : I doubt the practicality of MIA in today’s realtime ML models

Data is mostly private in Multinational companies; Then feasibility ?
This approach is fine for static Models, will this same approach useful in online learning ?

Ex : Amazon daily user prediction Models

30 of 46

Novelty and Relevance to Security

Novel Contributions :

Shadow Training : similar models trained on relatively similar data records using the same service behave in a similar way.

Bridged the gap between Privacy and Security fields in Machine learning Area.
Although these 2 papers pointed out some serious issues related to privacy; This problem reminded me of ”privacy Preserving Techniques in Databases” (k-Anonymus, L-Diverse)[15]

Table2. Privacy Preserving techniques in practice

31 of 46

References

Song, Liwei, Reza Shokri, and Prateek Mittal. "Privacy risks of securing machine learning models against adversarial examples." Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security. 2019.
Hayes, Jamie, et al. "LOGAN: Membership inference attacks against generative models." Proceedings on Privacy Enhancing Technologies 2019.1 (2019): 133-152.
Shokri, Reza, et al. "Membership inference attacks against machine learning models." 2017 IEEE Symposium on Security and Privacy (SP). IEEE, 2017.
Adversarial Machine learning, https://towardsdatascience.com/adversarial-machine-learning-mitigation-adversarial-learning-9ae04133c137
Panda, P., Chakraborty, I., & Roy, K. (2019). Discretization based solutions for secure machine learning against adversarial attacks. IEEE Access, 7, 70157-70168.
Membership inference Attacks in detail, https://bella.cc/blog/membership_inference/
Introduction to Machine Learning by Piyush Rai : https://www.cse.iitk.ac.in/users/piyush/courses/ml_autumn18/index.html
Gowal, Sven, et al. "On the effectiveness of interval bound propagation for training verifiably robust models." arXiv preprint arXiv:1810.12715 (2018).

32 of 46

References

Temperature Scaling, https://github.com/gpleiss/temperature_scaling
Abstract Interpretation for designing robust Machine learning models by SAFE AI : http://safeai.ethz.ch/
Dual decomposition, https://jonathan-hui.medium.com/machine-learning-lagrange-multiplier-dual-decomposition-4afe66158c9
Huang, Po-Sen, et al. "Achieving verified robustness to symbol substitutions via interval bound propagation." arXiv preprint arXiv:1909.01492 (2019).
Abadi, Martin, et al. "Deep learning with differential privacy." Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security. 2016.
Tang, Jun, et al. "Privacy loss in apple's implementation of differential privacy on macos 10.12." arXiv preprint arXiv:1709.02753 (2017).
Li, Ninghui, Tiancheng Li, and Suresh Venkatasubramanian. "t-closeness: Privacy beyond k-anonymity and l-diversity." 2007 IEEE 23rd International Conference on Data Engineering. IEEE, 2007.

34 of 46

a) DCGAN[2]

b) DCGAN + VAE[2]

c) BEGAN[2]

35 of 46

Sensitivity vs MIA attack (Additional)

compared to the natural model, the robust model is indeed more sensitive to the training data, thus leaking more membership information.

Fig: Confidence difference vs Excluded training points plot

36 of 46

Privacy risk with robustness generalization

As more training points are used for computing the robust loss, the membership inference accuracy increases.
Mainly due to the larger gap between adv-train accuracy and adv-test accuracy.

Table: Mixed PGD-based adversarial training experiments

37 of 46

Privacy vs model capacity

As the model capacity increases, the model has a higher membership inference accuracy

Fig: Membership Inference Attacks against models with difference capacities

38 of 46

Potential Defenses(contd..)

Temperature Scaling :

Fig.12. Visualization of Temperature Scaling [9]

39 of 46

Differential Privacy

Fig: Differential Privacy Algorithm

Reference : https://towardsdatascience.com/understanding-differential-privacy-85ce191e198a

40 of 46

Variational Auto Encoders

Reference : https://www.jeremyjordan.me/variational-autoencoders/

41 of 46

Advanced persistent and stealthy malware detection in�sensitive corporate networks