Presentation on ��Bayes Theorem ��Presented by �Mr. A. R. Karande�Department of Statistics �Raje Ramrao Mahavidyalaya, Jath

Theorem of total probability

• Let B₁, B₂, …, B_N be mutually exclusive events whose union equals the sample space S. We refer to these sets as a partition of S.

​

​

​

​

​

• An event A can be represented as:

​

​

​

• Since B₁, B₂, …, B_N are mutually exclusive, then

P(A) = P(A ∧ B₁) + P(A ∧ B₂) + … + P(A ∧ B_N)

• And therefore

P(A) = P(A|B₁)*P(B₁) + P(A|B₂)*P(B₂) + … + P(A|B_N)*P(B_N)

= Σ_i P(A | B_i) * P(B_i)

Exhaustive conditionalization

Marginalization

Bayes theorem

• P(A ∧ B) = P(B) * P(A | B) = P(A) * P(B | A)

​

​

​

​

A

P

B

P

A

B

P

)

(

)

(

)

|

(

=

=>

Posterior probability

Prior of A (Normalizing constant)

B

A

P

)

|

(

Prior of B

Conditional probability

(likelihood)

This is known as Bayes Theorem or Bayes Rule, and is (one of) the most useful relations in probability and statistics

Bayes Theorem is definitely the fundamental relation in Statistical Pattern Recognition

Bayes theorem (cont’d)

• Given B₁, B₂, …, B_N, a partition of the sample space S. Suppose that event A occurs; what is the probability of event B_j?

​

​

• P(B_j | A) = P(A | B_j) * P(B_j) / P(A)

= P(A | B_j) * P(B_j) / Σ_jP(A | B_j)*P(B_j)

B_j: different models / hypotheses

In the observation of A, should you choose a model that maximizes P(B_j | A) or P(A | B_j)? Depending on how much you know about B_j !

Posterior probability

Likelihood

Prior of B_j

Normalizing constant

(theorem of total probabilities)

Another example

• We’ve talked about the boxes of casinos: 99% fair, 1% loaded (50% at six)
• We said if we randomly pick a die and roll, we have 17% of chance to get a six
• If we get 3 six in a row, what’s the chance that the die is loaded?
• How about 5 six in a row?

THANK YOU