To the extent possible under law, Somik Raha has waived all copyright and related or neighboring rights to the slide deck “The Magic of Probability.” This work is published from: United States and India.

The Magic of Probability

Workshop by Dr. Somik Raha

Make a copy of this deck.�

​

You can now edit it

This deck has Speaker Notes.

​

Please review the speaker notes to prepare

Make sure they are turned on in your view

Somik Raha

The Magic of Probability

Hello!

Poll Question:

Are you joining from your computer or mobile device?

Which year are you?

Which college are you from?

Which department are you in?

What is your level of comfort with probability?

ವಿಜ್ಞಾನದಲ್ಲಿ ನಾವು ಹೇಳುವ ಎಲ್ಲಾ ವಿಷಯಗಳು, ಎಲ್ಲಾ ತೀರ್ಮಾನಗಳು ಅನಿಶ್ಚಿತವಾಗಿವೆ, ಏಕೆಂದರೆ ಅವುಗಳು ಕೇವಲ ತೀರ್ಮಾನಗಳಾಗಿವೆ. ಏನಾಗಲಿದೆ ಎಂಬುದಕ್ಕೆ ಅವು ಊಹೆಗಳಾಗಿವೆ ಮತ್ತು ಏನಾಗುತ್ತದೆ ಎಂದು ನಿಮಗೆ ತಿಳಿದಿಲ್ಲ, ಏಕೆಂದರೆ ನೀವು ಸಂಪೂರ್ಣ ಪ್ರಯೋಗಗಳನ್ನು ಮಾಡಿಲ್ಲ.

ಆದ್ದರಿಂದ, ವಿಜ್ಞಾನಿಗಳು ಅನುಮಾನ ಮತ್ತು ಅನಿಶ್ಚಿತತೆಯನ್ನು ಎದುರಿಸಲು ಬಳಸಲಾಗುತ್ತದೆ. ಎಲ್ಲಾ ವೈಜ್ಞಾನಿಕ ಜ್ಞಾನವು ಅನಿಶ್ಚಿತವಾಗಿದೆ. ಅನುಮಾನ ಮತ್ತು ಅನಿಶ್ಚಿತತೆಯೊಂದಿಗಿನ ಈ ಅನುಭವವು ಮುಖ್ಯವಾಗಿದೆ. ಇದು ಹೆಚ್ಚಿನ ಮೌಲ್ಯವನ್ನು ಹೊಂದಿದೆ ಮತ್ತು ವಿಜ್ಞಾನವನ್ನು ಮೀರಿ ವಿಸ್ತರಿಸಿದೆ ಎಂದು ನಾನು ನಂಬುತ್ತೇನೆ. ಹಿಂದೆಂದೂ ಪರಿಹರಿಸದ ಯಾವುದೇ ಸಮಸ್ಯೆಯನ್ನು ಪರಿಹರಿಸಲು, ನೀವು ಅಜ್ಞಾತಕ್ಕೆ ಬಾಗಿಲು ತೆರೆಯಬೇಕು ಎಂದು ನಾನು ನಂಬುತ್ತೇನೆ. ನೀವು ಅದನ್ನು ನಿಖರವಾಗಿ ಹೊಂದಿಲ್ಲದಿರುವ ಸಾಧ್ಯತೆಯನ್ನು ನೀವು ಅನುಮತಿಸಬೇಕು. ಇಲ್ಲದಿದ್ದರೆ, ನೀವು ಈಗಾಗಲೇ ನಿಮ್ಮ ಮನಸ್ಸನ್ನು ಹೊಂದಿದ್ದರೆ, ನೀವು ಅದನ್ನು ಪರಿಹರಿಸದಿರಬಹುದು.

ವಿಜ್ಞಾನಿ ನಿಮಗೆ ಉತ್ತರ ತಿಳಿದಿಲ್ಲ ಎಂದು ಹೇಳಿದಾಗ, ಅವನು ಅಜ್ಞಾನಿ. ಅದು ಹೇಗೆ ಕೆಲಸ ಮಾಡುತ್ತದೆ ಎಂಬುದರ ಕುರಿತು ತನಗೆ ಹಂಬಲವಿದೆ ಎಂದು ಅವನು ನಿಮಗೆ ಹೇಳಿದಾಗ, ಅವನು ಅದರ ಬಗ್ಗೆ ಅನಿಶ್ಚಿತನಾಗಿರುತ್ತಾನೆ. ಅದು ಹೇಗೆ ಕೆಲಸ ಮಾಡುತ್ತದೆ ಎಂದು ಅವನು ಖಚಿತವಾಗಿ ಹೇಳಿದಾಗ ಮತ್ತು "ಇದು ಕೆಲಸ ಮಾಡುವ ಮಾರ್ಗವಾಗಿದೆ, ನಾನು ಬಾಜಿ ಕಟ್ಟುತ್ತೇನೆ" ಎಂದು ಅವನು ನಿಮಗೆ ಹೇಳಿದಾಗ, ಅವನು ಇನ್ನೂ ಕೆಲವು ಅನುಮಾನಗಳಲ್ಲಿರುತ್ತಾನೆ. ಮತ್ತು ಪ್ರಗತಿಯನ್ನು ಸಾಧಿಸಲು, ಈ ಅಜ್ಞಾನ ಮತ್ತು ಈ ಅನುಮಾನವನ್ನು ನಾವು ಗುರುತಿಸುವುದು ಅತ್ಯುನ್ನತ ಪ್ರಾಮುಖ್ಯತೆಯಾಗಿದೆ. ನಮಗೆ ಸಂದೇಹವಿರುವುದರಿಂದ, ಹೊಸ ಆಲೋಚನೆಗಳಿಗಾಗಿ ಹೊಸ ದಿಕ್ಕುಗಳಲ್ಲಿ ನೋಡುವುದನ್ನು ನಾವು ಪ್ರಸ್ತಾಪಿಸುತ್ತೇವೆ. ವಿಜ್ಞಾನದ ಬೆಳವಣಿಗೆಯ ದರವು ನೀವು ಕೇವಲ ಅವಲೋಕನಗಳನ್ನು ಮಾಡುವ ದರವಲ್ಲ ಆದರೆ, ಹೆಚ್ಚು ಮುಖ್ಯವಾಗಿ, ನೀವು ಪರೀಕ್ಷಿಸಲು ಹೊಸ ವಿಷಯಗಳನ್ನು ರಚಿಸುವ ದರ.

01. WHERE DOES PROBABILITY COME FROM?

Let’s call a coin toss

ನಾಣ್ಯ ಟಾಸ್ ಅನ್ನು ಮುನ್ಸೂಚಿಸೋಣ

HEADS

TAILS

Let’s call a coin toss

ನಾಣ್ಯ ಟಾಸ್ ಅನ್ನು ಮುನ್ಸೂಚಿಸೋಣ

<NAME1>, What’s your probability of heads?

<NAME2>, What’s your probability of heads?

50%

50%

Let’s call a coin toss

ನಾಣ್ಯ ಟಾಸ್ ಅನ್ನು ಮುನ್ಸೂಚಿಸೋಣ

Psst… <NAME2>, take a look.

<NAME1>, What’s your probability of heads?

<NAME2>, What’s your probability of heads?

NAME2, What’s your probability of heads?

<NAME1>, What’s your probability of heads?

50%

50%

100%

50%

How can two people have different probabilities for the same event?

ಒಂದೇ ಈವೆಂಟ್‌ಗೆ ಇಬ್ಬರು ವ್ಯಕ್ತಿಗಳು ಹೇಗೆ ವಿಭಿನ್ನ ಸಂಭವನೀಯತೆಯನ್ನು ಹೊಂದಿರುತ್ತಾರೆ?

• Dialog

Probability = Your State of Information

ಸಂಭವನೀಯತೆ = ನಿಮ್ಮ ಮಾಹಿತಿಯ ಸ್ಥಿತಿ

THERE IS NO OBJECTIVE PROBABILITY

ವಸ್ತುನಿಷ್ಠ ಸಂಭವನೀಯತೆಯಂತಹ ಯಾವುದೇ ವಿಷಯವಿಲ್ಲ

If it was objective, it would be data, not probability!�ಅದು ವಸ್ತುನಿಷ್ಠವಾಗಿದ್ದರೆ, ಅದು ಡೇಟಾ, ಸಂಭವನೀಯತೆ ಅಲ್ಲ!

Probability = Your State of Information

ಸಂಭವನೀಯತೆ = ನಿಮ್ಮ ಮಾಹಿತಿಯ ಸ್ಥಿತಿ

THERE IS NO OBJECTIVE PROBABILITY

ವಸ್ತುನಿಷ್ಠ ಸಂಭವನೀಯತೆಯಂತಹ ಯಾವುದೇ ವಿಷಯವಿಲ್ಲ

Inappropriate Phrases

ಸೂಕ್ತವಲ್ಲದ ನುಡಿಗಟ್ಟುಗಳು

The probability

Appropriate Phrases

ಸೂಕ್ತವಾದ ನುಡಿಗಟ್ಟುಗಳು

My probability

Your probability

Prabha’s probability

Sojan’s probability

Our group’s probability

You can only place a probability on clear distinctions

ನೀವು ಸ್ಪಷ್ಟ ಪರಿಕಲ್ಪನೆಗಳ ಮೇಲೆ ಮಾತ್ರ ಸಂಭವನೀಯತೆಯನ್ನು ಇರಿಸಬಹುದು

The Clarity Test�ಸ್ಪಷ್ಟತೆ ಪರೀಕ್ಷೆ

Can a fact-based clairvoyant tell you how the distinction has resolved?

What is your probability of rain in Bengaluru tomorrow?

Let’s first define rain: At least 10 cm of water from the sky

Let’s define tomorrow: From 12:01 AM to 11:59 PM

Quick poll

Poll Question:

What is your probability of rain in Bengaluru tomorrow?

When should you place a 100% or 0% probability?

ನೀವು ಯಾವಾಗ 100% ಅಥವಾ 0% ಸಂಭವನೀಯತೆಯನ್ನು ಇರಿಸಬೇಕು?

I did not die yesterday

I will definitely die someday

The Epistemology of Probability from a Decision Analysis Perspective

ನಿರ್ಧಾರ ವಿಶ್ಲೇಷಣೆಯ ದೃಷ್ಟಿಕೋನದಿಂದ ಸಂಭವನೀಯತೆಯ ಜ್ಞಾನಶಾಸ್ತ್ರ

Epistemology.�ಜ್ಞಾನಶಾಸ್ತ್ರ�

The theory of knowledge, especially with regard to its methods, validity, and scope, and the distinction between justified belief and opinion�ಜ್ಞಾನದ ಸಿದ್ಧಾಂತ, ವಿಶೇಷವಾಗಿ ಅದರ ವಿಧಾನಗಳು, ಸಿಂಧುತ್ವ ಮತ್ತು ವ್ಯಾಪ್ತಿ ಮತ್ತು ಸಮರ್ಥನೀಯ ನಂಬಿಕೆ ಮತ್ತು ಅಭಿಪ್ರಾಯದ ನಡುವಿನ ವ್ಯತ್ಯಾಸಕ್ಕೆ ಸಂಬಂಧಿಸಿದಂತೆ

dictionary.com

​

It is the branch of philosophy concerned with knowledge�ಇದು ಜ್ಞಾನಕ್ಕೆ ಸಂಬಂಧಿಸಿದ ತತ್ವಶಾಸ್ತ್ರದ ಶಾಖೆಯಾಗಿದೆ

wikipedia.com

Poll Question:

How many probability theories are out there?

A tale of friendship

Rev Thomas Bayes�Died: 1761

1763

An Essay towards solving a Problem in the Doctrine of Chances

Read out by his friend Richard Price to the Royal Society

Richard Price

Worked out Conditional Probability and the Beta Distribution!

Poll Question:

The doctors tell you: “Above 90% of those who have haemophilia are male.” What is your probability of a person having hemophilia, given that person is a male?

Poll Question:

Lung cancer doctors tell you: “95% of my lung cancer patients are smokers.” Given this information, what is your probability of an individual having lung cancer, given that individual is a smoker?

Conditional Probability: Haemophilia in US

Male

Female

Male

Male

Female

Female

Haemophiliac

Not Haemophiliac

Haemophiliac

Not Haemophiliac

Haemophiliac

Not Haemophiliac

​

Conditional Probability: Haemophilia in US

0.02%

99.98%

Male

Female

Male

Male

Female

Female

Haemophiliac

Not Haemophiliac

Haemophiliac

Not Haemophiliac

Haemophiliac

Not Haemophiliac

​

Conditional Probability Quiz 2

If someone has lung cancer, what is your probability for this person being a smoker? ��If someone smokes, what is your probability for this person getting lung cancer?

Your turn:

https://tinyurl.com/smoker-lungcancer

Conditional Probability: Lung Cancer in India

Lung Cancer

No Lung Cancer

Lung Cancer

Lung Cancer

No Lung Cancer

No Lung Cancer

Smoker

Non-smoker

Smoker

Non-smoker

Smoker

Non-Smoker

​

Poll Question:

In the early CDC guidelines for serologic antibody tests, you will find them trying to educate you with an example where the sensitivity of the test is 90%, the specificity is 95%, and the prevalence of Covid-19 in the population is 5%. Under these circumstances, if an individual gets a positive serologic result, what probability should you assign to that individual being Covid +ve?

Conditional Probability Quiz 3

If someone has Covid, what is your probability for this person testing positive for antibodies? ��If someone tests positive for Covid antibodies, what is your probability for this person having Covid?

Your turn:

https://tinyurl.com/covid-inference

Conditional Probability: Covid Serological (Antibody) Testing

Had Covid

No Covid

Had Covid

Had Covid

No Covid

No Covid

Antibodies detected

Antibodies not detected

Antibodies detected

Antibodies not detected

Antibodies detected

Antibodies not detected

​

Let’s look at Shruti

Lived in New York City at the height of the pandemic. She thought she had Covid and took the antibody test several months later.

​

She just got a negative antibody test result

Her doctor says: “There is no chance you got Covid.” �Why is he saying that?

Conditional Probability: Covid Serological (Antibody) Testing

Had Covid

No Covid

Had Covid

Had Covid

No Covid

No Covid

Antibodies detected

Antibodies not detected

Antibodies detected

Antibodies not detected

Antibodies detected

Antibodies not detected

​

Let’s look at Shruti

Let’s add more context

Shruti presents with all the serious symptoms (according to WHO ) of COVID:

​

• Difficulty breathing or shortness of breath
• Chest pain or pressure
• Loss of speech or movement

​

She just got a negative antibody test result

What prior probability should the doctor have placed?

What is the conditional probability of her having Covid with this prior?

Poll Question:

Given Shruti presents with the severe Covid symptoms, what should you do with your prior probability of her having Covid?

Poll Question:

If you changed your prior probability for Shruti having Covid, what probability would you place on her having Covid given she got a negative serologic test result?

Conditional Probability: Covid Serological (Antibody) Testing

Had Covid

No Covid

Had Covid

Had Covid

No Covid

No Covid

Antibodies detected

Antibodies not detected

Antibodies detected

Antibodies not detected

Antibodies detected

Antibodies not detected

​

Where else can this apply?

Can you come up with your own examples

Judicial Sentencing

Decaying as Society

What about Machine Learning?

Let’s go back to the Rain example

What is your probability of rain in Bengaluru tomorrow?

What is your probability of rain in Bengaluru tomorrow?

A smart engineer has built a rain detector using ML on weather models

What would you like to know from the evaluation results of the model in a lab setting?

ML Example: Rain Detector

What metrics do ML classification model designers report?

How can we combine my P(Rain) to evaluate model?

Can you easily do inference with these metrics?

If yes, why?

If not, why not?

PRECISION

RECALL

TPR & FPR allow human evaluation of model

Instead, ask for True Positive Rate (Recall / Sensitivity) and �False Positive Rate (1-Specificity)

TPR

SENSITIVITY

RECALL

FPR

1-SPECIFICITY

Poll Question:

If you saw seven heads in a row in a coin toss, what probability would you place on the next coin landing heads?

Poll Question:

If you saw seven heads in a row in a coin toss, what probability would you place on the next coin landing heads?

​

Maybe it’s time to question whether it is a “normal” coin.

Introduce a little doubt about the nature of the coin

If you see a tails, you can be sure �it’s not a 1-headed coin

If you see a heads, your probability of �this being a normal coin goes down�from 90% to 82%

Questions?

Continuous Probability Quick Review

ನಿರಂತರ ಸಂಭವನೀಯತೆ ತ್ವರಿತ ವಿಮರ್ಶೆ

Let’s play a game ಒಂದು ಆಟ ಆಡೋಣ

Roll two dice and bet on the sum of the roll�ಎರಡು ದಾಳಗಳನ್ನು ಉರುಳಿಸಿ ಮತ್ತು ರೋಲ್‌ನ ಮೊತ್ತದ ಮೇಲೆ ಬಾಜಿ ಕಟ್ಟಿಕೊಳ್ಳಿ

Poll: If you could bet on only one number for the sum of two die rolls (2-12), what would you bet on?

Let’s play a game ಒಂದು ಆಟ ಆಡೋಣ

Roll two dice and bet on the sum of the roll�ಎರಡು ದಾಳಗಳನ್ನು ಉರುಳಿಸಿ ಮತ್ತು ರೋಲ್‌ನ ಮೊತ್ತದ ಮೇಲೆ ಬಾಜಿ ಕಟ್ಟಿಕೊಳ್ಳಿ

Poll: If you could bet on three numbers for the sum of two die rolls (2-12), what would you bet on?

Let’s play a game ಒಂದು ಆಟ ಆಡೋಣ

Roll two dice and bet on the sum of the roll�ಎರಡು ದಾಳಗಳನ್ನು ಉರುಳಿಸಿ ಮತ್ತು ರೋಲ್‌ನ ಮೊತ್ತದ ಮೇಲೆ ಬಾಜಿ ಕಟ್ಟಿಕೊಳ್ಳಿ

Let’s try it: Open Workbook

Discrete vs Continuous Measures

Discrete:

Sum of two die rolls�2,3,4,5,6,7,8,9,10,11,12

Continuous:

Revenue in Crores�Test Positivity Rate (0%-100%)

?

Poll Question:

What is the following curve?

Poll Question:

How would you read off the Probability of making revenue less than X crores from this curve?

Poll Question:

What is the following curve?

Poll Question:

How would you read off the Probability of making revenue less than X crores from this curve?

CMF and PMF become CDF and PDF when dealing with “continuous variables”

Let’s go back to Bayes

Bayes also worked out the Beta Distribution

It’s conjugate is also a Beta!

Binomial trial

Exactly two (“bi”) outcomes, “success or failure”

Each trial has same chance of success

Beta Distribution

Ref: Wikipedia

The Uniform Prior

B(alpha, beta) = B(1,1)

Bayes: Suggested “with a great deal of doubt” as the prior probability�distribution to express ignorance about the correct prior distribution.

Why is this statement problematic? How would you edit it?

Hint: Remember the clairvoyant?

Worked out Probability Theory

“Probability as an instrument for repairing defects in knowledge”

Pierre Simon Laplace

1771

Mémoire sur la probabilité des causes par les événements

Memoir on the Probability of the Causes of Events

Independently worked out what Bayes had worked out, and �then some.

What is the probability of the sun rising tomorrow?

Laplace had no hesitation in using the Uniform prior to express total ignorance

What is the probability of the sun rising tomorrow?

Laplace’s succession rule, illustrated with an unfortunate example

Prior Knowledge:�both success and failure are possible

�Assumes�we observed one success and �one failure for sure before we started��Not correct if s=0, or s=n

“But this number [the probability of the sun coming up tomorrow] is far greater for him who, seeing in the totality of phenomena the principle regulating the days and seasons, realizes that nothing at present moment can arrest the course of it.”

Probability theory as an extension of logic

Edwin Thompson Jaynes

1950s

Probability Theory: The Logic of Science

The “Robot”, the ancestor of the “Clairvoyant”

“In order to direct attention to constructive things and away from controversial irrelevancies, we shall invent an imaginary being. Its brain is to be designed by us, so that it reasons according to certain definite rules. These rules will be deduced from simple desiderata which, it appears to us, would be desirable in human brains; i.e. we think that a rational person, on discovering that they were violating one of these desiderata, would wish to revise their thinking.”

Excerpt From: E. T. Jaynes & G. Larry Bretthorst. “Probability Theory.” Apple Books. https://books.apple.com/us/book/probability-theory/id811951223

Probability theory as an extension of logic

Edwin Thompson Jaynes

1950s

Probability Theory: The Logic of Science

The “Robot”, the ancestor of the “Clairvoyant”

“Our robot is going to reason about propositions. As already indicated above, we shall denote various propositions by italicized capital letters, {A, B, C, etc.}, and for the time being we must require that any proposition used must have, to the robot, an unambiguous meaning and must be of the simple, definite logical type that must be either true or false.”

Excerpt From: E. T. Jaynes & G. Larry Bretthorst. “Probability Theory.” Apple Books. https://books.apple.com/us/book/probability-theory/id811951223

Probability theory as an extension of logic

Edwin Thompson Jaynes

1950s

Probability Theory: The Logic of Science

Let me make what, I fear, will seem to some a radical, shocking suggestion: The merits of any statistical method are not determined by the ideology which led to it. For, many different, violently opposed ideologies may all lead to the same final “working equations” for dealing with real problems. Apparently, this phenomenon is something new in statistics; but it is so commonplace in physics that we have long since learned how to live with it. Today, when a physicist says, “Theory A is better than theory B,” he does not have in mind any ideological considerations; he means simply, “There is at least one specific application where theory A leads to a better result than theory B.” I suggest that we apply the same criterion in statistics: The merits of any statistical method are determined by the results it gives when applied to specific problems. The Court of Last Resort in statistics is simply our commonsense judgment of those results.

Foundations of Probability Theory, Statistical Inference, and Statistical Theories of Science

Decision Analytic lens

Ronald A. Howard

1966

Coined the term “Decision Analysis”

Cofounder of the field

Brought the notion of “decisions”

First published version of “Encoding of priors” in �Decision Analysis: Applied Decision Theory

Prior development as “psychoanalytic process”

Watch interview

Conditional Probability: Covid Serological (Antibody) Testing

Had Covid

No Covid

Had Covid

Had Covid

No Covid

No Covid

Antibodies detected

Antibodies not detected

Antibodies detected

Antibodies not detected

Antibodies detected

Antibodies not detected

​

What about Shruti?

​

She presents with all the serious symptoms (according to WHO ) of COVID:

​

• Difficulty breathing or shortness of breath
• Chest pain or pressure
• Loss of speech or movement

Conditional Probability: Covid Serological (Antibody) Testing

Had Covid

No Covid

Had Covid

Had Covid

No Covid

No Covid

Antibodies detected

Antibodies not detected

Antibodies detected

Antibodies not detected

Antibodies detected

Antibodies not detected

​

Distinctions that pass clarity test

Meaningless to put probability on unclear distinctions

We know nothing at all about the distinction

-AND-

Believe that every value is equally likely

The Metalog Distribution

Tom Keelin

2016

Published “The Metalog Distributions”

A significant advance in statistics

A distribution to match data/assessments

Prior development and updating �a fully “psychoanalytic” process

Entirely compatible with data, �huge implications for ML

Homework

Beta Distribution

​

Play with the Beta.DIST Excel function to create:

1. Uniform Prior
2. Updating Beta Distributions

Decision Analytic AB Testing

​

Read Article�Try spreadsheet and Web model

Metalog Distributions

​

Watch Video 1, Video 2��Download Metalog Workboooks in Excel

​

Encode your beliefs with Metalog Distributions

Appendix

Updating the beta is a simple addition operation

Add the number of tosses to update N and the number of successes to update S

Decision Analytic AB tests