Probability

Table of Contents

1. Introduction
2. The classical approach
3. Relative frequency approach
4. Axiomatic approach
5. Key terms of Probability
6. Random variable
7. Discrete probability distribution
8. Continuous probability distribution
9. Mathematical Expectation
10. Conditional probability
11. Bayes' theorem
12. Exercise

Introduction

Probability is a measure of likelihood that an event will occur in a random experiment.It is quantified by a number between 0 and 1, where, 0 indicates impossibility and 1 indicates certainty. For example, if a coin is tossed, the probability of getting "head" is $\frac{1}{2}$, and it represents that there is 50% chance of getting head.
We can define probability in three different ways/approaches as below.

1. Classical approach
2. Relative frequency approach
3. Axiomatic approach

---

---

---

The classical approach

Classical definition of probability is for equally likely outcomes. If an experiment can produced $n$ mutually exclusive and equally likely outcomes out of which $k$ outcomes are favorable to the occurrence of event $A$ , then the probability of $A$ is denoted by $P(A)$ and is defined by
$P(A)=\frac{\text{Number of outcomes favourable to A}}{\text{Number of possible outcomes}}=\frac{k}{n}$
For example, if a bag contains 3 red and 5 white marbles, then the probability of getting red marble is
$P(Red)=\frac{\text{Number of outcomes favourable to Red marbles}}{\text{Number of marbles}}=\frac{3}{8}$

Limitations of Classical definition
The classical definition of probability has always been criticized for the following reasons:

1. This definition assumes that the outcomes are equally likely.
2. The definition is not applicable when the numbers of outcomes are not equally likely.
3. The definition is also not applicable when the total number of outcomes is infinite or it is difficult to count the total outcomes. For example, it is difficult to count the fish in the ocean.

Example 1

A fair die is thrown. Find the probabilities that the face on the die is (a) Maximum (b) Prime (c) Multiple of 3 (d) Multiple of 7

Solution
There are 6 possible outcomes when a die is tossed. We assumed that all the 6 faces are equally likely. The classical definition of probability is to be applied here
The sample space is $S=\{1,2,3,4,5,6\}$ and $n(S)=6$

1. (a) Let $A$ be the event that the face is maximum, thus
   $A=\{6\},n(A)=1$
   Therefore, $P(A)=\frac{\text{Number of outcomes favorable to A}}{\text{Number of possible outcomes in S}}=\frac{(n(A)}{(n(S)}=\frac{1}{6}$
2. (b) Let $B$ be the event that is prime, thus
   $B=\{2,3,5\},n(B)=3$
   Therefore, $P(B)=\frac{\text{Number of outcomes favorable to B}}{\text{Number of possible outcomes in S}}=\frac{(n(B)}{(n(S)}=\frac{3}{6}$
3. (c) Let $C$ be the event that is multiples of 3, thus
   $C=\{3,6\},n(C)=2$
   Therefore, $P(C)=\frac{\text{Number of outcomes favorable to C}}{\text{Number of possible outcomes in S}}=\frac{(n(C)}{(n(S)}=\frac{2}{6}$
4. (d) Let $D$ be the event that is multiples of 7, Thus
   $D=\varphi ,n(D)=0$
   Therefore, $P(D)=\frac{\text{Number of outcomes favorable to D}}{\text{Number of possible outcomes in S}}=\frac{(n(D)}{(n(S)}=0$

---

---

---

Relative frequency approach

Let us take an example. When a meteorologist states that the chance of rain is 90\%, the meteorologist is in terms of relative frequency. The relative frequency is the ratio of observed frequency to the total frequency of a random experiment. Suppose a random experiment is repeated n times and outcomes A is observed k times, then $\frac{k}{n}$ is the relative ratio of outcome A. In this case, probability is defined as limiting value of the ratio. It is given by
$P(A)={\displaystyle \underset{n\to \mathrm{\infty }}{lim}\frac{k}{n}}$
For example, probability of having rain today? Probability of selecting a motorbike of YAMAHA brand in KTM valley? Both of these answer are sample dependent. We need a sample to describe these probabilities. Therefore, this can be answered based on relative frequency approach of probability.

---

---

---

Axiomatic approach

An axiomatic approach to probability refers to the probability of an event based on additional evidence. This approach is based on three axioms. According to this approach, probability is defined as follows.
Let S is a sample space, then probability is a function that assign a real number to every event E of S. In this case, the probability of en even event is P(E), and it satisfy following three axioms:

1. Non negativity: $0\le P(E)\le 1$ for any event E
2. Additive: $P(E_1\cup E_2\cup ..)=P(E_1)+P(E_2)+\dots$ for mutually exclusive events
3. Certainty: P(S) =1, for sample space S.

---

---

---

Key terms of Probability

1. Sample Space and Event
   The set of all possible outcomes in a random experiment is called sample space. It is denoted by S. A subset E of sample space S is called an event.
   For example, if a dice is rolled, then sample space is
   $S=\{1,2,3,4,5,6\}$
   And, if we are interested in even number, then
   $E=\{2,4,6\}$ is an event.
2. Mutually exclusive events
   In probability, two events A and B are called mutually exclusive events if both events A and B cannot occur together. For example,
   1. in a dice, 1 and 2 are mutually exclusive events because, either 1 will come or 2 will come but not both at a time.
   2. in a coin, H and T are mutually exclusive events because, either H will come or T will come but not both at a time.
   3. in a dice, even and prime are not-mutually exclusive events because, if 2 is the output, then it is even as well as prime, thus even AND prime both can come together at a time.
3. Independent events
   In probability, two events A and B are called independent events if occurrence of A has no effect in the occurrence of B. For example, in a dice 1 and in a coin H are independent events because, occurrence of either will have no effect on the other.
   NOTE: Independent events are applied for series (more than one) of experiment.
   1. If a bag contains 3 red and 5 white marbles, and two marbles are selected one after another with replacement then the events in both experiment are independent.
   2. If a bag contains 3 red and 5 white marbles, and two marbles are selected one after another without replacement then the events in both experiment are not independent.

---

---

---

Random variable

A headmaster bought one dozen of computer monitor, of which, unknown to him three are broken. He checks four of the monitors at random to see how many are broken in his sample of 4. The answer which he label X, is one of the numbers 0,1,2,3,4. Since the value of X is random, X is called random variable. The mathematical definition is as follows.
Random variable is a function that assign a numerical value to sample space of a random experiment. A random variable is also called chance variable and it is abbreviated by $r.v.$ and denoted by capital letter $X,Y,...,$ of English alphabets. For example, if we toss a coin and define sample space $S=\{H,T\}$. Then we can define X= number of heads, as a random variable. In this case, X=1 represents occurrence of 1 head. The table for the distribution of the values of X is given below.

S (Sample Space )	T	H
X=x (Number of heads)	0 (no head)	1(1 head)

The value of the random variable is denoted by small letters of English alphabets, like $X=x$. A random variable can be of any form: one-variate, bi-variate or multi-variate.

---

---

---

Types of random variable (Discrete and Continuous)

Random variable are of two types. They ate discrete and continuous. A random variable $X$ is called discrete if it can assume finite number of values, and the values it has taken can be separated by gaps. Some example of discrete $r.v.$ are number of students in a class, no of books in students bag, number of family member of student. If $X=$ number of books in students bag, then $X$ can take 0, 1 or 2 as value. But $X$ cannot take its value as 0.1 or 1.5 etc. Here, $X$ can take only the specific values which are 0, 1 and 2 and so on. Therefore, in this case $X$ is discrete random variable.
A random variable $X$ is called continuous if it can take all values in a possible range, and there are no gaps between its values. For example, the weight of a student could be any real number between certain extreme limits. Suppose, $X$ = weight of students in a certain class which lies been between 40kg to 70kg . Then $X$ can take any value between the ranges 100 to 150. The weight of a student may be 45kg or 45.2kg or it may take any value between 45 and 45.2 . Here, value of $X$ can be any real number between 40 to 70. So, $X$ is continuous random variable.

---

---

---

Probability distribution

Probability distribution is a table or a function that give a numerical value for each outcome of a random experiment. With this probability distribution, one can model behavior of a random variable. In this essence, probability distribution is called function of random variable.

---

---

---

Discrete probability distribution

Let $X$ be a discrete random variable then probability distribution of $X$ is denoted by $f(x)$ and defined by $f(x)=P(X=x)$ satisfying (a) $f(x)\ge 0$ and (b) $\sum f(x)=1$.

S	TT	HT, TH	HH
$X=x$	$0$	$1$	$2$
$f(X=x)$	$\frac{1}{4}$	$\frac{1}{2}$	$\frac{1}{4}$

For example, if we toss a pair of coins, and X= number of heads obtained, then the probability distribution of X is given in the table above, and the corresponding histogram is given below.

0,0

0.25

0.50

0.25

X=0

X=1

X=2

Example 2

The function of a random variables$X$ is $f(x)=\frac{x}{6}$ for $x=1,2,3$. Fin the probability at X=2.
Solution
Given the probability function $f(x)=\frac{x}{6}$ for $x=1,2,3$, the probability at X=2 is
$f(x)=\frac{x}{6}=\frac{2}{6}=\frac{1}{3}$

Example 2

The probability function of a random variables$X$ is given below in a diagram.

0,0

0.30

0.20

0.50

X=50

X=200

X=350

Based on the graph above, answer the followng question.

f(50)=?		F(50)=?
f(200)=?		F(200)=?
f(350)=?		F(350)=?

---

---

---

Continuous probability distribution

Let $X$ be a continuous random variable then probability distribution of $X$ is denoted by $f(x)$ and defined by $f(x)=P(a\le X\le b)$ satisfying (a) $f(x)\ge 0$ and (b) $\underset{-\mathrm{\infty }}{\overset{\mathrm{\infty }}{\int }}f(x)dx=1$. It is also called probability density of X or probability function of X.

Example 3

The probability function of a continuous random variables X is
$f(x)=\frac{2}{3}(x+1)$ for 0<x <1
find the probability at 0<x<$\frac{1}{2}$

0,0

–o+←↓↑→

∫ = 0.4167

Solution
Given the probability function
$f(x)=\frac{2}{3}(x+1)$ for 0<x<1
the probability at 0<x<1/2 is
${\int }_0^{1/2}f(x)dx={\int }_0^{1/2}\frac{2}{3}(x+1)dx$
$=\frac{2}{3}{\int }_0^{1/2}(x+1)dx$
$=\frac{2}{3}{[\frac{x^2}{2}+x]}_0^{1/2}$
$=\frac{2}{3}[\frac{1}{8}+\frac{1}{2}]$
$=\frac{5}{12}$

---

---

---

Mathematical Expectation

Mathematical expectation is one of the most important the digital characteristic of the random variable. It ha a good application in life, including insurance, investment decision, profit on sales, customer service period of service, medical laboratory.

Example 4

A headmaster bought one dozen of computer monitor, of which, unknown to him three are broken. He checks four of the monitors at random to see how many are broken in his sample of 4. Now, what is the expected number of broken monitors that the headmaster finds? To answer this question, we need mathematical expectation.

Definition

In statistics, expected value, mathematical expectation and mean are same. They all represents arithmetic mean as in ordinary calculation.

For example,
suppose, we toss a pair of coin and read
X= number of heads,
Then sample space and value of random variable are given below.

S	TT	HT,TH	HH
X=x	0	1	2

Now, using random variable, mean of X is
$\bar{X}=\frac{\sum x}{n}=\frac{0+1+1+2}{4}=1$ (A)
In this example, probability distribution of X is

S	TT	HT,TH	HH
X=x	0	1	2
f(X=x)	$\frac{1}{4}$	$\frac{1}{2}$	$\frac{1}{4}$

Also, using the probability function, mean of X is

$\bar{X}=\sum x.f(x)=0.\frac{1}{4}+1.\frac{2}{4}+2.\frac{1}{4}=1$ (B)
So, both (A) and (B) are same.

Let X be a random variable with probability function f(x), then expected value of X is denoted by E(X) or $\bar{X}$ or ${\mu }_X$ and defined as
$E(X)=\sum x.f(x)$ if X is discrete
$E(X)=\int x.f(x)dx$ if X is continuous

Properties of expected value

1. If a is a constant then $E(a)=a$
2. If X is a random variable and a is a constant then $E(aX)=aE(X)$
3. If X is a random variable, also a and b are constants then $E(aX+b)=aE(X)+b$

Example 5

The probability of X is $f(x)=\frac{x}{6}$ for x=1,2,3, then find E(X).

Solution

Given, the probability of X the expected value of X is
$E[X]={\displaystyle \sum _{X}x.f(x)}$
or $E[X]={\displaystyle \sum _{X=1}^{3}x.\frac{x}{6}}$
or $E[X]={\displaystyle \sum _{X=1}^3\frac{x^2}{6}}$
or $E[X]=\frac{1^2}{6}+\frac{2^2}{6}+\frac{3^2}{6}$
or $E[X]=\frac{7}{3}$

Example 6

Based on the distribution given below, find the Expected Value?

X	0	1	2	3
f(x)	14/55	28/55	1/55	12/55

Solution

The expected value of X is
$E[X]={\displaystyle \sum _{X}x.f(x)}$
or $E[X]=0\times \frac{14}{55}+1\times \frac{28}{55}+2\times \frac{12}{55}+3\times \frac{1}{55}$
or $E[X]=1$

Example 7

The probability function of X is $f(x)=\frac{6}{5}x(x+1)$ for 0<x <1

Solution

Given, joint probability of X and Y, the expected value of X is
$E(X)={\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}x.f(x)dx={\int }_0^{1}x.\frac{3}{5}x(x+1)dx$
or $E(X)=\frac{3}{5}{\int }_0^1(x^3+x^2)dx$
or $E(X)=\frac{3}{5}{[\frac{x^4}{4}+\frac{x^3}{3}]}_0^1$
or $E(X)=\frac{7}{10}$

---

---

---

Conditional probability

We start with an example.
Amar and Binod are going out to dinner. They toss a fair coin ‘best of three’ to decide who pays: if there are more heads than tails in the three tosses, Amar pays, otherwise Binod pays.Clearly each has a 50% chance of paying.

The sample space corresponding to this experiment is
S = {HHH,HHT,HTH,HTT,THH,THT,TTH,TTT}
and the favorable events for ‘Amar pays’ and ‘Binod pays’ are respectively
$E_1$ = {HHH,HHT,HTH,THH}
$E_2$ = {HTT,THT,TTH,TTT}
It shows that, the probabilities that Amar and Binod pay for dinner are $\frac{1}{2}$ for each of them.

They toss the coin once and the result is head; call this event A. Then
A = {HHH,HHT,HTH,HTT}
i.e given the information that result of the first toss is head, then A becomes the sample space of the experiment.

For this case, the outcomes ‘Amar pays’ and ‘Binod pays’ are
$A\cap E_1$ = {HHH,HHT,HTH}
$A\cap E_2$ = {HTT}
Thus the new probabilities that Amar and Binod pay for dinner are $\frac{3}{4}$ and $\frac{1}{4}$ respectively.

Let us see another example too

Let a sample space S is given by
S={1,2,3,4,5}
Within this sample space, let us define two even $E_1$ and $E_2$ as below
$E_1$=even prime number ={2}
$E_2$=even number={2,4}
We know that
$P(E_1)=\frac{1}{6}$
$P(E_2)=\frac{2}{6}$
Now
$P(E_1|E_2)=$ we search $E_1$ inside $E_2$, which is $\frac{1}{2}$

Definition

Let $A$ be an event with non-zero probability, and $E_1$ be any event. Then the conditional probability of $E_1$ given $A$ is defined as
$P(E_1|A)=\frac{P(A\cap E_1)}{P(A)}$
The vertical bar in the notation is $P(E_1|A)$, not $P(E_1/A)$ or $P(E_1\mathrm{\setminus }A)$.
The definition only applies in the case where P(A) is not equal to zero, since we have to divide by it, and this would make no sense if P(A) = 0. Thus,
$P(E_1|A)=\frac{P(A\cap E_1)}{P(A)};P(A)\ne 0$

Example 8

Amar and Binod are going out to dinner. They toss a fair coin ‘best of three’ to decide who pays: if there are more heads than tails in the three tosses, Amar pays, otherwise Binod pays.They toss the coin once and the result is head, now find the probability that Amar will pay.

Solution

If $E_1$ is the event ‘Amar pay’ and A be the event ‘first result is Head’, then we are given that
$P(E_1)=\frac{4}{8}$
or $P(E_1\cap A)=\frac{3}{8}$
So we have
$P(E_1|A)=\frac{P(E_1\cap A)}{P(A)}=\frac{3}{4}$

Example 9

A random car is chosen among all those passing through Singhadarbar Gate on a certain day. The probability that the car is yellow is 0.03, the probability that the driver is female is 0.2, and the probability that the car is yellow and the driver is female is 0.02. Find the conditional probability that the driver is female given that the car is yellow.

Solution

If A is the event ‘the car is yellow’ and $E_1$ be the event ‘the driver is female’, then we are given that
$P(A)=0.03$
or $P(E_1\cap A)$ =0.02
So we have
$P(E_1|A)=\frac{P(E_1\cap A)}{P(A)}=\frac{0.02}{0.03}=0.667$

---

---

---

Bayes' theorem

Bayes' theorem is a formula that describes how to update the probabilities of hypotheses when given an evidence. It follows simply from the axioms of conditional probability.
More explicitely, the Bayes theorem describes the probability of an event based on the prior knowledge of the given conditions. If we know the conditional probability P(A|B), we can use the bayes theorem to find out the reverse probabilities P(B|A).

Theorem

Baye's Theorem: Let A and B be two events and let P(A|B) be the conditional probability of A given that B has occurred, then
$P(B|A)=\frac{P(A|B)P(B)}{P(A)}$

0,0 A B C D A B

0,0 A B C D A B

Solution

Given that, A and B be two events.
Now suppose that event B is occured (Fig1), then the conditional probability of A given that B has occurred is
$P(A|B)=\frac{P(A\cap B)}{P(B)}$
or $P(A\cap B)=P(A|B)\times P(B)$ (1)
Again, assume that event A is occured (Fig 2), then the conditional probability of B given that A has occurred is
$P(B|A)=\frac{P(A\cap B)}{P(A)}$
or $P(A\cap B)=P(B|A)\times P(A)$ (2)
Using (1) and (2), we get
$P(A\cap B)=P(A\cap B)$
or $P(A|B)\times P(B)=P(B|A)\times P(A)$
or $P(B|A)=\frac{P(A|B)P(B)}{P(A)}$

Example 10

Amar and Binod are going out to dinner. They toss a fair coin ‘best of three’ to decide who pays: if there are more heads than tails in the three tosses, Amar pays, otherwise Binod pays.They toss the coin once and the result is head, now find the probability that Amar will pay.

Solution

If $E_1$ is the event ‘Amar pay’ and A be the event ‘first result is Head’, then we are given that
$P(E_1)=\frac{1}{2}$
$P(A)=\frac{1}{2}$
$P(A|E_1)=\frac{3}{4}$
So we have
$P(E_1|A)=\frac{P(A|E_1)\times P(E_1)}{P(A)}=\frac{\frac{3}{4}\frac{1}{2}}{\frac{1}{2}}=\frac{3}{4}$

Law of total Probability

0,0

A

B

C

D

E₁

E₂

E₃

E_i

E_m

A

Suppose a sample space S can be partitioned into a set of disjoint events $E_i$ such that
$S=E_1\cup E_2\cup E_3\cup E_4\cup \cdots \dots E_{𝑚}$
Then the probability of an arbitrary event A in S can be written as
$P(A)=P(E_1\cap 𝐴)+P(E_2\cap 𝐴)+P(E_3\cap 𝐴)+\dots +P(E_{𝑚}\cap 𝐴)$
or$P(A)=P(𝐴|E_1)P(E_1)+P(𝐴|E_2)P(E_2)+P(𝐴|E_3)P(E_3)+\dots +P(𝐴|E_{𝑚})P(E_{𝑚})$
or $P(A)=\sum P(𝐴|E_{𝑖})P(E_{𝑖})$

Bayes Theorem

Let $E_1,E_2,\dots ,E_n$ be a set of events associated with a sample space S, where all the events $E_1,E_2,\dots ,E_n$ have nonzero probability of occurrence and they form a partition of S. Let A be any event associated with S, then
$P(E_i|A)=\frac{P(E_i)P(A|E_i)}{P(E_1)P(A|E_1)+P(E_2)P(A|E_2)+...}$

Example 11

Amar and Binod are going out to dinner. They toss a fair coin ‘best of three’ to decide who pays: if there are more heads than tails in the three tosses, Amar pays, otherwise Binod pays.They toss the coin once and the result is head, now find the probability that Amar will pay.

Solution

If $E_1$ is the event ‘Amar pay’ , $E_2$ is the event ‘Amar NOT pay’ and A be the event ‘first result is Head’, then we are given that
$P(E_1)=\frac{1}{2}$
$P(E_2)=\frac{1}{2}$
$P(A)=\frac{1}{2}$
$P(A|E_1)=\frac{3}{4}$
$P(A|E_2)=\frac{1}{4}$
So we have
$P(E_1|A)=\frac{P(A|E_1)\times P(E_1)}{P(A|E_1)\times P(E_1)+P(A|E_2)\times P(E_2)}=\frac{\frac{3}{4}\frac{1}{2}}{\frac{3}{4}\frac{1}{2}+\frac{1}{4}\frac{1}{2}}=\frac{3}{4}$

Example 12

A man is known to speak truth 2 out of 3 times. He throws a die and reports that number obtained is a four. Find the probability that the number obtained is actually a four.

Solution

Let us suppose that, A be the event that the "man reports that number four is obtained". Also let, $E_1$ be the event that "four is obtained" and $E_2$ be its complementary event "four is NOT obtained". Then,
$P(E_1)$ = Probability that four occurs = $\frac{1}{6}$
$P(E_2)$= Probability that four does not occurs = $1-P(E_1)=1-\frac{1}{6}=\frac{5}{6}$
Now,
$P(A|E_1)$= Probability that man reports four and it is actually a four = $\frac{2}{3}$
$P(A|E_2)$ = Probability that man reports four and it is not a four = $\frac{1}{3}$
By using Bayes’ theorem, probability that number obtained is actually a four is
$P(E_1|A)=\frac{P(E_1)P(A|E_1)}{P(E_1)P(A|E_1)+P(E_2)P(A|E_2)}=\frac{\frac{1}{6}\frac{2}{3}}{\frac{1}{6}\frac{2}{3}+\frac{5}{6}\frac{1}{3}}\frac{2}{7}$

---

---

---

Exercise

1. Suppose 36% of students read computer, 30% of students read math, and 22% of the students read computer also read math. A student is chosen at random and found reading math. What is the probability they also read computer?
2. Suppose the test for Covid-19 is 99% accurate in both directions and 0.3% of the population is Covid-19 positive. If someone tests positive, what is the probability they actually are Covid-19 positive.[23%]
3. Suppose 30% of the girl in a class received an A on the test and 25% of the boy received an A. The class is 60% girl. Given that a student chosen at random received an A, what is the probability this student is a girl?
4. A fair coin is flipped 12 times. What is the probability that exactly 10 heads appear given that at least two heads appeared?
5. In a class, 60% of the students like Mathematics and 35% of students like Physics and 25% like both the subjects. One student select at random, find the probability that he likes Physics if it is known that he likes Mathematics.