Basics of Probability�

Dr. Suresh Chandra Raikwar

• Probability is the study of randomness and uncertainty.
• A random experiment is a process whose outcome is uncertain.
• Examples:
• Tossing a coin once or several times
• Tossing a die
• Tossing a coin until one gets Heads
• ...

Events and Sample Spaces

3

Sample Space

The sample space is the set of all possible outcomes.

Simple Events

The individual outcomes are called simple events.

Event

An event is any collection of one or more simple events

Sample Space

Events

• Events are subsets of the sample space
• A= {the outcome that the die is even} ={2,4,6}
• B = {exactly two tosses come out tails}=(htt, tht, tth}
• C = {at least two heads} = {hhh, hht, hth, thh}

Probability

Properties of Probability

• For any event A, P(A^c) = 1 - P(A).
• If A ⊂ B, then P(A) ≤ P(B).
• For any two events A and B,
• P(A ∪ B) = P(A) + P(B) - P(A ∩ B).
• For three events, A, B, and C,
• P(A∪B∪C) =
• P(A) + P(B) + P(C)
• - P(A∩B) - P(A∩C) - P(B∩C)
• + P(A∩B ∩C)

​

Intuitive Development (agrees with axioms)

• Intuitively, the probability of an event a could be defined as:

8

Where N(a) is the number that event a happens in n trials

Random Variable

Discrete Random Variables

• Random variables (RVs) which may take on only a countable number of distinct values
• e.g., the sum of the value of two dies

​

• X is a RV with arity k if it can take on exactly one value out of k values,
• e.g., the possible values that X can take on are �2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12

Probability of Discrete RV

• Probability mass function (pmf):
• Simple facts about pmf
• if
• if

Common Distributions

Joint Distribution

• Given two discrete RVs X and Y, their joint distribution is the distribution of X and Y together
• e.g.�you and your friend each toss a coin 10 times �P(You get 5 heads AND you friend get 7 heads)

​

Conditional Probability

Example

Let two honest coins, marked 1 and 2, be tossed together. The four possible outcomes are T₁T₂, T₁H₂, H₁T₂, H₁H₂. (T₁ indicates toss of coin 1 resulting in tails; similarly T₂ etc.) We shall treat that all these outcomes are equally likely; that is the probability of occurrence of any of these four outcomes is 1/4. (Treating each of these outcomes as an event, we find that these events are mutually exclusive and exhaustive). Let the event A be 'not H₁H₂' and B be the event 'match'. (Match comprises the two outcomes T₁T₂, H₁H₂). Find P(B|A). Are A and B independent?

Solution

We know that

P(B|A)= P(AB)/P(A)

AB is the event 'not H₁H₂' and 'match'; i.e., it represents the outcome T₁T₂. Hence, P(AB) = 1/4.

The event A comprises of the outcomes ‘T₁T₂, T₁H₂ and H₁T₂’; therefore, P(A) =3/4

P(B/A)= (1/4) / (3/ 4) =1/3

Priya (P1) and Prasanna (P2), after seeing each other for some time (and after a few tiffs) decide to get married, much against the wishes of the parents on both the sides. They agree to meet at the office of registrar of marriages at 11:30 a.m. on the ensuing Friday (looks like they are not aware of Rahu Kalam or they don’t care about it). �� However, both are somewhat lacking in punctuality and their arrival times are equally likely to be anywhere in the interval 11 to 12 hrs on that day. Also arrival of one person is independent of the other. Unfortunately, both are also very short tempered and will wait only 10 min. before leaving in a huff never to meet again.���a) Picture the sample space �b) Let the event A stand for “P1 and P2 meet”. Mark this event on the sample space. �c) Find the probability that the lovebirds will get married and (hopefully) will live happily ever after. � �

Solution:

​

Bayes Rule

• X and Y are discrete RVs…

Independent RVs

More on Independence

​

​

​

• E.g. no matter how many heads you get, your friend will not be affected, and vice versa

Conditionally Independent RVs

• Intuition: X and Y are conditionally independent given Z means that once Z is known, the value of X does not add any additional information about Y
• Definition: X and Y are conditionally independent given Z iff

More on Conditional Independence

Continuous Random Variables

PDF

• Properties of pdf

​

​

​

​

• Actual probability can be obtained by taking the integral of pdf
• E.g. the probability of X being between 0 and 1 is

Cumulative Distribution Function

• Discrete RVs

​

• Continuous RVs

​

Common Distributions

Multivariate Normal

• Generalization to higher dimensions of the one-dimensional normal

​

Covariance Matrix

Mean

Mean and Variance

• Mean (Expectation):
• Discrete RVs:

​

• Continuous RVs:

​

• Variance:
• Discrete RVs:

​

• Continuous RVs:

Mean Estimation from Samples

• Given a set of N samples from a distribution, we can estimate the mean of the distribution by:

30

Variance Estimation from Samples

• Given a set of N samples from a distribution, we can estimate the variance of the distribution by:

31

Thank You