1 of 12

Wasserstein Dependency Measure�for Representation Learning

Sherjil Ozair�Corey Lynch�Yoshua Bengio�Aäron van den Oord�Sergey Levine�Pierre Sermanet

2 of 12

Contrastive Predictive Coding Representation Learning

3 of 12

InfoNCE

4 of 12

On Variational Bounds of Mutual Information

Poole et al. 2018

5 of 12

Outlier Risk Lemma

McAllester and Stratos 2018

6 of 12

Contrastive Predictive Coding (InfoNCE)

7 of 12

Wasserstein Dependency Measure (WDM)

8 of 12

Wasserstein Predictive Coding

9 of 12

MultiOmniglot Dataset

  • 12M unique samples

10 of 12

Varying Number of Characters (Mutual Information)

11 of 12

Varying Dataset Size

12 of 12

Recent Developments