Probability Refresher
Probability and events
Random variables
PROBABILITY AND EVENTS
2
Key Take Aways (Events)
3
Events
E ∩ F = ∅ and E ∪ F = Ω
4
Events & Probabilities – (1)
5
Events & Probabilities – (2)
P{E∩F|G} = P{E|G}×P{F|G}
6
Events & Probabilities – (3)
7
Bayes Law
(1) P{E|F} = P{E∩F}/P{F} ⇒ P{E∩F} = P{E|F}×P{F}
(2) P{F|E} = P{E∩F}/P{E} ⇒ P{E∩F} = P{F|E}×P{E}
P{F|E} = P{E|F}×P{F}/P{E}
8
Example – Anti-virus s/w Test
Note: In general, the two may not be equal, and defining accuracy is more challenging
P{positive} = P{positive | infected}×P{infected}+P{positive | not infected} ×P{not infected}
P{positive} = 0.95×0.001+0.05×0.999 = 0.0509
If we test 1,000 computers ~50 will return a positive test. Of those 50, one is likely to be really infected (based on P{infected}=0.001), but the other 49 are not ⇒ True positives = 1/50 or ~2%
9
Example – Anti-virus s/w Test
Note: In general, the two may not be equal, and defining accuracy is more challenging
P{positive} = P{positive | infected}×P{infected}+P{positive | not infected} ×P{not infected}
P{positive} = 0.95×0.001+0.05×0.999 = 0.0509
If we test 1,000 computers ~50 will return a positive test. Of those 50, one is likely to be really infected (based on P{infected}=0.001), but the other 49 are not ⇒ True positives = 1/50 or ~2%
10
Example – Anti-virus s/w Test
Note: In general, the two may not be equal, and defining accuracy is more challenging
P{positive} = P{positive | infected}×P{infected}+P{positive | not infected} ×P{not infected}
P{positive} = 0.95×0.001+0.05×0.999 = 0.0509
If we test 1,000 computers ~50 will return a positive test. Of those 50, one is likely to be really infected (based on P{infected}=0.001), but the other 49 are not ⇒ True positives = 1/50 or ~2%
11
Example – Anti-virus s/w Test
Note: In general, the two may not be equal, and defining accuracy is more challenging
P{positive} = P{positive | infected}×P{infected}+P{positive | not infected}×P{not infected}
P{positive} = 0.95×0.001+0.05×0.999 = 0.0509
If we test 1,000 computers ~50 will return a positive test. Of those 50, one is likely to be really infected (based on P{infected}=0.001), but the other 49 are not ⇒ True positives = 1/50 or ~2%
12
Some “Intuition”
13
RANDOM VARIABLES
14
Key Take Aways (Random Variables - 1)
15
Key Take Aways (Random Variables - 2)
16
Random Variables
17
Joint Probability
18
Conditional Probabilities & Expectations
19
Conditional Probabilities & Expectations
20
Conditional Probabilities & Expectations
21
Conditional Probabilities & Expectations
22
Conditional Probabilities & Expectations
23
Conditional Probabilities & Expectations
24
Conditional Probabilities & Expectations
25
Conditional Probabilities & Expectations
26
More on Expectation
27
Random Sum of Random Variables
28
Mean of Random Sum of Random Variables
29
Variance of Random Sum of Random Variables
30
Sample Paths, Convergence, and Averages
Key Take Aways (Convergence & Averages)
Probability of the limit vs. limit of the probability
32
Convergence
∀ε > 0, ∃n0(ε), such that ∀n > n0(ε), |an – b| < ε
33
Why Do We Care About Convergence?
34
Convergence Almost Surely�(with Probability 1)
∀ε > 0, P{limn→∞ |Yn – μ| > ε} = 0
Consider the average of n coin flips. Most sample paths will converge to a value of ½. However, some sample paths wont, e.g., the sample path 1,1,1,1,1,1,… We have convergence almost surely, because even though there are an uncountable number of “bad” paths, that number becomes vanishingly small (of measure 0) as n goes to infinity
35
Convergence in Probability
∀ ε > 0, limn→∞P{|Yn – μ| > ε} = 0
(the limit applies to the probability, and not the random variables)
36
Strong & Weak Laws of Large Numbers�Convergence and Averages
Then Yn converges in probability to E[X]
Then Yn converges almost surely to E[X]
Strong Law is harder to prove than Weak Law but implies it
37
Time & Ensemble Averages
where pi = limt→∞P{N(t) = i}
38
Computing Averages in Practice
39
Ergodicity
40
Non-Ergodic Systems
41