Part 2: Differential Privacy in the Context of Information Sharing
Privacy-Aware Sequential Learning.
Motivation: Social Sequential Learning
People make decisions after observing others:
→ Public actions transmit information�→ This creates information cascades & herd behavior
Motivation: Privacy Leakage
The public report can reveal sensitive information:
→ Public observability = privacy leakage
4
Key Questions and Insights
Surprisingly! Privacy considerations can, in some cases, lead to a faster learning process.
5
Sequential learning model
…
6
Binary Signal and Information Cascade
Information Cascade: Individuals ignore their private information and imitate prior, public actions (Bikhchandani et al., 1992)
-1
+1
7
Gaussian Signal and Learning Efficiency
Asymptotic learning is inefficient for Gaussian private signals!
📢
Metric Differential Privacy
Continuous/Gaussian signals
No information flow
No privacy
Best Response Under Privacy Constraints
10
Privacy Impact on Learning
Information traps get less sticky when you introduce “noise”. Sequential learning under privacy can be faster and more efficient!
11
The Binary Model
12
The Binary Model
Before Cascade
After Cascade
+1
-1
-1
-1
-1
…
13
The Binary Model: Information Traps
14
The Binary Model：Nonmonotonic Pattern
15
The Binary Model
Information cascade Threshold
Signal Accuracy
16
The Gaussian Model
17
Randomized Response and Global Sensitivity
It flips the action with a fixed probability, independent of what the signal is
18
Global Sensitivity
Randomized Response does not depend on the private signal!
+1
Signal Distribution
Decision
Threshold
+1
19
Smooth Randomized Response
Smooth Randomized Response
Decision
Threshold
Decision
Threshold
20
The Gaussian Model
21
Learning Rate
22
Time to First Correct Action
Finite!
23
Total Number of Incorrect Action
Finite!
24
The Gaussian Model
25
Mechanism Behind Acceleration
26
Order-Optimal Asymptotic Learning with Heterogeneous Privacy Budgets
27
Order-Optimal Asymptotic Learning with Heterogeneous Privacy Budgets
28
Learning Rate Bound
29
Learning Rate Bound
30
Learning Efficiency
31
Part 2: Differential Privacy in the Context of Information Sharing
Differentially Private Distributed Estimation and Inference.
A recent challenge in power grids
Average consumption?
Can we efficiently learn the average consumption while preserving the privacy of the households?
Learning about the effectiveness of a treatment
Patient data
Patient data
Patient data
Patient data
Privacy regulations regarding patient data
Can we efficiently learn if the new treatment is effective while preserving privacy of the patients?
Privacy is important in both contexts!
Average consumption?
1st part of the talk: Distributed estimation and learning of the sufficient statistics of exponential family variables
Papachristou, Marios, and M. Amin Rahimian. "Differentially Private Distributed Estimation and Learning." IISE Transactions, 2024
2nd part of the talk: Non-Bayesian social learning under privacy constraints
Papachristou, Marios, and M. Amin Rahimian. “Differentially Private Distributed Inference” Under review, 2024
Patient data
Patient data
Patient data
Patient data
Distributed Estimation
(Non-Bayesian) Social Learning
Some related works in Distributed Estimation
Some related works in Social Learning
Social learning relies on information flow. It offers an interesting context to study privacy.
Section 0: Preliminaries
Graph structure, learning task
Graph structure
DP Protections
Signal DP
Network DP
Section 1: Private Distributed Estimation and Learning
Learning the sufficient statistics of exponential family variables
Minimum Variance Unbiased Estimation
Online Learning
Minimum Variance Unbiased Estimation
Algorithm
Minimum Variance Unbiased Estimation
Algorithm
Online Learning
Algorithm (Signal DP)
Online Learning
Algorithm (Network DP)
Section 2: Private Non-Bayesian Distributed Social Learning
Distributed maximum likelihood estimation and online learning
Distributed Maximum Likelihood Estimation
Online Learning
Non-private Distributed MLE Benchmark
Non-private Online Learning Benchmark
DP Protections
Multiplicative Noise
Caveat
Algorithms become non-deterministic, i.e., they can return the wrong answer with non-zero probability!
Private Distributed MLE – Updates within rounds
Aggregations
AM/GM Aggregation
Double Thresholding
AM/GM Aggregation
Theorem (Informal) [Papachristou, R, 2024]
Global sensitivity of log-likelihood
Max log-likelihood value
Min abs value of identifiability condition
Double Threshold Aggregation
Theorem (Informal) [Papachristou, R, 2024]
Global sensitivity of log-likelihood
Max log-likelihood value
Min abs value of identifiability condition
Private Online Learning Algorithm
Online Learning
Theorem (Informal) [Papachristou, R, 2024]
Max sum of variances of KL divergence
Global sensitivity of log-likelihood
Sum of variances of the number of signals
Min abs value of identifiability condition
Differentially Private Distributed Inference