1 of 14

Probabilistic Foundations of Machine Learning

I am: Yaniv / Professor Yacoby

I use: he/they

Instructions:

  • Sit next to someone you don’t know.
  • Write question(s) you had about the readings on the board.

2 of 14

Announcements

  • Regular office hours are listed on the website
  • Solutions to HW1: available only at office hours
    • Can’t take them with you
    • Please look at them: they will show you effective, simple uses of Jax
  • Add to calendars: talk by Dr. Safiya Noble
    • Title: Algorithms of Oppression
    • Monday Nov 11, 5-6:30pm @ Tishman
    • Extra credit!
  • Tips on writing code:
    • Draw pictures / write pseudo code first
    • Don’t write the whole thing at once!
    • Check each Jax call does what you want it to do
  • When to use vectorization?
    • Model / model-fitting: yes
    • Plotting: if you really want

3 of 14

Today:

  • Review: Probability (discrete)
  • Conditional Probability (Discrete)
    • Terminology and Notation
    • Beginning NumPyro

​

​

​

4 of 14

Probability: Summary of Notation

Together: Suppose we wanted to compute the probability that a patient comes into the IHH ER with intoxication.

5 of 14

Exercise (pairs): gaining comfort with commonly-used discrete distributions

6 of 14

Exercise (pairs): matching the distribution to the data

7 of 14

Motivation for Conditional Probability (Discrete)

  • Example: support we want to compute chance of intoxication
    • Name?
    • Distribution?
  • How to estimate from data?

Question: How accurate will this prediction be?

8 of 14

Looking at variables individually can be misleading

9 of 14

Notation and Terminology

Goal:

  • Programming language = interface to communicate with computer
  • Probability = interface for machine learning
  • Later: probability → NumPyro code

Terms:

  • Conditional Random Variable
  • Sample
  • Sample Space / Support
  • Probability Mass Function (PMF)
  • Parameter
  • Independent, Identically Distributed (i.i.d)

10 of 14

Summary of Terminology and Notation

​

​

​

​

Questions?

11 of 14

In Pairs: Parts 1+2 of HW2, “Conditional Probability”

12 of 14

Live Coding: Distributions in NumPyro

  • Marginal distribution of intoxication
    • Sampling
    • Evaluating (log probability)
  • Random Generator Keys
  • Conditional distribution: intoxication given day

13 of 14

In Pairs: Parts 3+4 of HW2, “Conditional Probability”

14 of 14

That’s all for today!

​

​

​

​

Questions?

  • Tell your partner: what did you appreciate about working with them today!
  • By Wednesday: finish all problems on (non-conditional) probability
  • By Thursday: finish all problems on conditional probability
  • So that: on Thursday, we can focus on joint probability!