1 of 46

Week 3 & 4

Probability As Used In Simulation

2 of 46

Basic Probability Concepts

What do we mean if we say, probability of occurrence of an event is 0.5?

What is an event?

Let us consider a simple experiment of tossing a coin. Can one predict, whether head or tail will come.

No, it is not possible to predict.

Modeling and Simulation

Department of Computer Science

3 of 46

Basic Probability Concepts

In order to know the answer, we have to toss the coin at least hundred times.

Suppose we get 49 times head and 51 times tail.

Then probability of getting head in hundred trials is defined as:

“total number of heads/total number of toss”

Thus, getting head in tossing a coin is an event.

Modeling and Simulation

Department of Computer Science

4 of 46

Basic Probability Concepts

This can be defined in technical language as:

Probability of occurrence of an event =

total favourable events/total number of experiments

Modeling and Simulation

Department of Computer Science

5 of 46

Basic Probability Concepts

In this case probability is 0.49, if all the events are equally likely.
This is classical definition of probability.

Here equally likely is the condition. In case of coin, it has to be unbiased coin.

Now if we toss the same coin more number of times say 1000 times, we will come to know that probability of getting head or tail in case of coin is closer to 0.5.

Modeling and Simulation

Department of Computer Science

6 of 46

Basic Probability Concepts

Higher is the number of trials, accurate will be the probability of success.

In probability, all favourable and unfavourable events are called sample points.

In above example of a coin, if we toss a coin thousand times, then thousand outcomes are sample points.

Modeling and Simulation

Department of Computer Science

7 of 46

Basic Probability Concepts

Sample Point

Each possible outcome of an experiment is called a sample point.

Sample Space

The sample space is the set of all possible sample points of an experiment.

Modeling and Simulation

Department of Computer Science

8 of 46

Basic Probability Concepts

let us take an example of a dice.
If we throw this dice, any outcome (number on the top side of a dice) of it will be a sample point.

And the sample space will be a set of all the outcomes taken together i.e., S {1, 2, 3, 4, 5, 6} will be a sample space.

where symbol { } defines a set of sample points 1, 2, 3, 4, 5, 6.

Modeling and Simulation

Department of Computer Science

9 of 46

Basic Probability Concepts

If sample space S has finite number of points, it is called finite sample space.

Now let us consider a set of all the natural numbers.

Can you count these numbers?

Of course one can count but up to what number.

There are infinite natural numbers.

You surely can not count up to infinity, but yet you can count up to your capacity.

Modeling and Simulation

Department of Computer Science

10 of 46

Basic Probability Concepts

Thus, a sample space consisting of all the natural numbers 1, 2, 3, 4, 5,… is called a countable infinite sample space.

It is infinite yet countable.

Now let us consider a different case i.e., number of points on a number line.

If sample space S has as many points as there are numbers in some interval (0,1) on line, such as 0 ≤ x ≤ 1, it is called a non-countable infinite sample space.

Modeling and Simulation

Department of Computer Science

11 of 46

Basic Probability Concepts

There are infinite points on a line of unit length, but can you count all these points.

Of course, not.

A sample space that is finite or countable infinite is called a discrete sample space, while one that is non-countable infinite is called non-discrete (continuous) sample space.

Modeling and Simulation

Department of Computer Science

12 of 46

Basic Probability Concepts

Event: An event is a subset of a sample space S, i.e., it is a set of possible outcomes.

If an outcome of an experiment is an element of subset A, we say the event A has occurred.

An event consisting of single point of S is often called a simple event.

What is a set? A set can be defined as a collection of similar types of items.

For example, outcome of throw of a pair of dice is a set. This set is nothing but numbers from 2 to 12.

Modeling and Simulation

Department of Computer Science

13 of 46

Basic Probability Concepts

If we express the outcomes of an experiment in terms of a numerically valued variable X , which can assume only a finite or denumerable number of values, each with a certain probability , then such a variable is called a random variable (also called stochastic variable or variate)

Here onward a random variable will be denoted by a capital letters while the values of random variables will be denoted by small letters.

Modeling and Simulation

Department of Computer Science

14 of 46

Basic Probability Concepts

For example, if we roll a pair of dice, sum X of two numbers which turn up, must be an integer between 2 and 12.

But it is not possible to predict which value of X will occur in the next trial.

If we can predict the chance of a particular number to come in the next trial then such an outcome of trial is called probability of occurrence of that number which will be the value of the event, called random variable.

Therefore, we can say that, if X depends on chance and a probability can be attached to it then it is a random variable.

Or it can be predicted that next time when a dice is rolled, what will be the outcome.

Modeling and Simulation

Department of Computer Science

15 of 46

Basic Probability Concepts

Universal Set:

In general, a sample space is said to be discrete if it has finitely many or countable infinite elements.

A sample space, which is called universal set, can have number of subsets.

For example, in the example of throw of a pair of dice, set of all even outcome can be called one subset and that of odd outcomes can be called second subset.

Modeling and Simulation

Department of Computer Science

16 of 46

Basic Probability Concepts

Set Operations:

By using set operations on events in S, we can obtain few other events in S. For example if A and B are events, then:

Modeling and Simulation

Department of Computer Science

17 of 46

Basic Probability Concepts

Set Operations:

Modeling and Simulation

Department of Computer Science

18 of 46

Basic Probability Concepts

Set Operations:

Modeling and Simulation

Department of Computer Science

19 of 46

Basic Probability Concepts

Statistical Independence:
We now give definition of statistically independent random events.

Two random events A and B are statistically independent if and only if P (A ∩B ) = P (A )P (B )

Thus, if A and B are independent, then their joint probability can be expressed as a simple product of their individual probabilities.

Equivalently, for two independent events A and B,� P(A|B) = P(A) and P(B|A) = P(B)

Modeling and Simulation

Department of Computer Science

20 of 46

Basic Probability Concepts

Statistical Independence:

Symbol P (A|B ) here means, probability of occurrence of A if B has already occurred.

In other words, if A and B are independent, then the conditional probability of A, given B is simply the individual probability of A alone; likewise, the probability of B given A is simply the probability of B alone.

This result is after Swine and is called SWINE’S THEOREM.

Modeling and Simulation

Department of Computer Science

21 of 46

Basic Probability Concepts

Independent Events Examples :

The probability of two independent events, A and B, is equal to the probability of event A times the probability of event B.

Modeling and Simulation

Department of Computer Science

22 of 46

Basic Probability Concepts

Independent Events Examples :

Example: Suppose you spin each of these two spinners. What is the probability of spinning an even number and a vowel?

P(even) =

(3 evens out of 6 outcomes)

(1 vowel out of 5 outcomes)

P(vowel) =

P(even, vowel) =

Modeling and Simulation

Department of Computer Science

23 of 46

Basic Probability Concepts

Independent Events Examples :

P(jack, factor of 12)

1
5

5
8

5
40

1
8

Modeling and Simulation

Department of Computer Science

24 of 46

Basic Probability Concepts

Independent Events Examples :

P(6, not 5)

1
6

5
6

5
36

Modeling and Simulation

Department of Computer Science

25 of 46

Basic Probability Concepts

Independent Events Examples :

Modeling and Simulation

Department of Computer Science

26 of 46

Basic Probability Concepts

Independent Events Examples :

Modeling and Simulation

Department of Computer Science

27 of 46

Basic Probability Concepts

Dependent Events :

When two events are said to be dependent, the probability of one event occurring influences the likelihood of the other event.

For example, if you were to draw a two cards from a deck of 52 cards.

If on your first draw you had an ace and you put that aside, the probability of drawing an ace on the second draw is greatly changed because you drew an ace the first time.

Let's calculate these different probabilities to see what's going on.

Modeling and Simulation

Department of Computer Science

28 of 46

Basic Probability Concepts

Dependent Events :

Modeling and Simulation

Department of Computer Science

29 of 46

Basic Probability Concepts

Dependent Events :

Modeling and Simulation

Department of Computer Science

30 of 46

Basic Probability Concepts

Dependent Events Examples:

There are 6 black pens and 8 blue pens in a jar. If you take a pen without looking and then take another pen without replacing the first, what is the probability that you will get 2 black pens?

P(black second) =

(There are 13 pens left and 5 are black)

P(black first) =

P(black, black) =

Modeling and Simulation

Department of Computer Science

31 of 46

Basic Probability Concepts

Dependent Events Examples:

P(Q, Q)
All the letters of the alphabet are in the bag 1 time
Do not replace the letter

1
26

0
25

0
650

Modeling and Simulation

Department of Computer Science

32 of 46

Basic Probability Concepts

Conditional Probability:

We have already defined dependent and independent events and seen how probability of one event relates to the probability of the other event.

Having those concepts in mind, we can now look at conditional probability.

Conditional probability deals with further defining dependence of events by looking at probability of an event given that some other event first occurs.

Conditional probability is denoted by the following:

Modeling and Simulation

Department of Computer Science

33 of 46

Basic Probability Concepts

Conditional Probability:

The above is read as the probability that B occurs given that A has already occurred.

The above is mathematically defined as:

Modeling and Simulation

Department of Computer Science

34 of 46

Basic Probability Concepts

Mutual Exclusivity:

Certain special pairs of events have a unique relationship referred to as mutual exclusivity.

Two events are said to be mutually exclusive if they can't occur at the same time.

For a given sample space, its either one or the other but not both.

As a consequence, mutually exclusive events have their probability defined as follows:

Modeling and Simulation

Department of Computer Science

35 of 46

Basic Probability Concepts

Mutual Exclusivity:

An example of mutually exclusive events are the outcomes of �a fair coin flip.

When you flip a fair coin, you either get a head or a tail but not both, we can prove that these events are mutually exclusive by adding their probabilities:

For any given pair of events, if the sum of their probabilities is equal to one, then those two events are mutually exclusive.

Modeling and Simulation

Department of Computer Science

36 of 46

Basic Probability Concepts

Rules of Probability for Mutually Exclusive Events:

Multiplication Rule (Intersection): From the definition of mutually exclusive events, we should quickly conclude the following:

Addition Rule: As we defined above, the addition rule applies to mutually exclusive events as follows:

Subtraction Rule: From the addition rule above, we can conclude that the subtraction rule for mutually exclusive events takes the form:

Modeling and Simulation

Department of Computer Science

37 of 46

Basic Probability Concepts

Rules of Probability for Mutually Exclusive Events:

Conditional Probability for Mutually Exclusive Events:

We have defined conditional probability with the following equation:

We can redefine the above using the multiplication rule

hence:

This is a venn diagram of a set containing �two mutually exclusive events A and B.

Modeling and Simulation

Department of Computer Science

38 of 46

Basic Probability Concepts

Rules of Probability for Mutually Exclusive Events:

Conditional Probability for Mutually Exclusive Events:

We have defined conditional probability with the following equation:

We can redefine the above using the multiplication rule

hence:

This is a venn diagram of a set containing �two mutually exclusive events A and B.

Modeling and Simulation

Department of Computer Science

39 of 46

Random Variables

Random Variable (RV): A numeric outcome that results from an experiment

For each element of an experiment’s sample space, the random variable can take on exactly one value

Discrete Random Variable: An RV that can take on only a finite or countably infinite set of outcomes

Continuous Random Variable: An RV that can take on any value along a continuum (but may be reported “discretely”)

Modeling and Simulation

Department of Computer Science

40 of 46

Probability Distributions

Probability Distribution: Table, Graph, or Formula that describes values a random variable can take on, and its corresponding probability (discrete RV) or density (continuous RV)
Discrete Probability Distribution: Assigns probabilities (masses) to the individual outcomes
Continuous Probability Distribution: Assigns density at individual points, probability of ranges can be obtained by integrating density function
Discrete Probabilities denoted by: p(y) = P(Y=y)
Continuous Densities denoted by: f(y)
Cumulative Distribution Function: F(y) = P(Y≤y)

Modeling and Simulation

Department of Computer Science

41 of 46

Discrete Probability Distributions

Modeling and Simulation

Department of Computer Science

42 of 46

Thank You

Modeling and Simulation

Department of Computer Science

43 of 46

Exercise:

Modeling and Simulation

Department of Computer Science

44 of 46

Q8. There are 6 black pens and 8 blue pens in a jar. If you take a pen without looking and then take another pen without replacing the first, what is the probability that you will get 2 black pens?

Q9. Explain probability distribution.

Q10. Higher is the number of trials, accurate will be the probability of ?

Unsuccessful
Success
Even
Uneven

Q11. In probability, all favourable and unfavourable events are called ?

Sample point
Sample space
Sample collection
None of the above

Modeling and Simulation

Department of Computer Science

45 of 46

Q12. What is called the set of all possible sample points of an experiment?

Sample point
Sample space
Sample collection
None of the above

Q13. If sample space S has finite number of points, it is called.

Even sample space
Infinite sample space
Finite sample space
None of the above

Q14. A sample space that is finite or countable infinite is called a.

Discrete sample space
Continuous sample space
Even sample space
None of the above

Modeling and Simulation

Department of Computer Science

46 of 46

Q15. Certain special pairs of events have a unique relationship referred to as.

Mutual exclusivity
Manual exclusivity
Mathematical exclusivity
None of the above

Q16. For each element of an experiment’s sample space, the random variable can take on exactly.

One value
Two value
Three value
Four value

Modeling and Simulation

Department of Computer Science