1

BASICS OF PROBABILITY
Probability and statistics are concerned with events which occur by chance. Examples include
occurrence of accidents, errors of measurements, production of defective and non-defective items from
a production line, and various games of chance, such as drawing a card from a well-mixed deck,
flipping a coin, or throwing a symmetrical six sided die. In each case we may have some knowledge of
the likelihood of various possible results, but we cannot predict with any certainty the outcome of any
particular trial. Probability and statistics are used throughout engineering. In electrical engineering,
signals and noise are analyzed by means of probability theory, Civil, mechanical, and industrial
engineers use statistics and probability to test and account for variations in materials and goods.
Chemical engineers use probability and statistics to assess experimental data and control and improve
chemical processes, it is essential for todays engineer to master these tools.
Random Experiment:
Let a coin be tossed. Nobody knows what we will get whether a Head or tail. But it is certain
that either a head or tail will occur. In a Similar way, if a dice is thrown, we may get any of the faces
1, 2,3,4,5, and 6. But nobody knows which one will occur. Experiments of this type where the
outcome cannot be predicted are called random experiments. The word probability or chance is used
commonly in day to-day life. For example the chances of India and South Africa winning the world
cup cricket, before the start of the game are equal (i.e., 50:50). We often say that it will rain
tomorrow. Probably I will not come to function today. All these terms chance, probable, etc., convey
the same meaning i.e., that event is not certain to take place. In other words, there is an uncertainty
about the happening of the event. The term probability refers to the randomness and uncertainty.
An experiment is said to be random experiment if
(i) It can be repeated any number of times under similar condition
(ii) It should have more than one out come
(iii) The outcome cannot be predicted
Sample point:
Each simple outcome of a random experiment is called a sample point. Example: tossing a
coin. Head and tail are sample points
Sample space:
The set of all possible cases of an experiment is called the Sample space and is denoted by S
Trail and event
Any subset of a sample space is called events. Consider an experiment of throwing a coin.
When tossing a coin, we may get a head (H) or tail (T). Here tossing of a coin is a trail and Getting a
head or tail is an event. From a pack of cards, drawing any three cards is trail and getting a king or a
queen or a jack are events. Throwing of a dice is a trail and getting 1 or 2 or 3 or 4 or 5 or 6 is an
event.
2

Classical Definition for Probability
Probability of happening an event,
E =



Example: A fair die is thrown what is a probability of getting a number less than 4
Sample Space, S = {1, 2, 3, 4, 5, 6}
Favorable case = {1, 2, 3} = 3
Total case = 6
P (A) = 3/6 =
Random Variable:
A random variable, usually written X, is a variable whose possible values are numerical
outcomes of a random phenomenon. That is a function x which transforms events of a random
experiment into real numbers is called random variable. It is denoted as X: SR where S is sample
space of random experiment and R is set of real numbers. The outcome of an experiment need not be a
number, for example, the outcome when a coin is tossed can be 'heads' or 'tails'. However, we often
want to represent outcomes as numbers. A random variable is a function that associates a unique
numerical value with every outcome of an experiment. The value of the random variable will vary
from trial to trial as the experiment is repeated. A random variable has either an associated probability
distribution (discrete random variable) or probability density function (continuous random variable).
Example:
1. Two coins are tossed at a time Sample space is S = {HH, HT, TH, TT}
If we take X is the number of heads appearing then HH becomes 2, HT and TH becomes 1 and
TT becomes 0
X (number of heads) is a random variable
2. A coin is tossed ten times. The random variable X is the number of tails that are noted. X can
only take the values 0, 1... 10, so X is a discrete random variable.
3. A light bulb is burned until it burns out. The random variable Y is its lifetime in hours. Y can
take any positive real value, so Y is a continuous random variable.
TYPES OF RANDOM VARIBLES
There are two types of random variables known as
(i) Discrete random variable
(ii) Continuous random variable
(i) Discrete random variable
If a random variable takes only a finite or a countable number of values, it is called a discrete
random variable. For example, when two coins are tossed the number of heads obtained is the
random variable X. X assumes the values 0, 1, 2 which is a countable set. Such a variable is called
discrete random variable.
3

Probability Mass Function:
The probability distribution of a discrete random variable is a list of probabilities associated
with each of its possible values. It is also sometimes called the probability function or the probability
mass function. Let X be a discrete random variable with values x
1
,x
2
,x
3
,.x
n
. Let p (xi) be a
number associated with each x
i
Then the function p is called the probability function of X if it
satisfies the conditions:
(i) p (x
i
) 0for i=1,2,3,....n
(ii) p (x
i
) = 1
The set of ordered pairs (x
i
, p(x
i
)) is called the probability distribution of X. The probability
function is also known as Probability Mass Function of X.
(ii) Continuous Random Variable:
A continuous random variable is one which takes an infinite number of possible values. Continuous
random variables are usually measurements.. A random variable X is said to be continuous if it can
take all possible values between certain limits.
Examples: 1. Life time of electric bulb in hours
2. Height, weight, temperature, etc.
Probability density function:
A function f is said to be probability density function (pdf) of the continuous random variable
X if it satisfies the following conditions:
1. f(x) 0 for all x R;
2.

Distribution Function (Cumulative Distribution Function):
All random variables (discrete and continuous) have a cumulative distribution function. It is a
function giving the probability that the random variable X is less than or equal to x, for every value x.
For a discrete random variable, the cumulative distribution function is found by summing up the
probabilities. The function F(x) is said to be the distribution function of the random variable X if
F(x) = P ( X x ); x . The distribution function F is also called Cumulative distribution
function.
Note:
1. If X is a discrete random variable then from the definition it follows that F(x) = p (x
i
) where the
summation is overall X
i
such that xi x.
2. If X is a continuous random variable, then from the definition it follows that
F(x) =

. x .
where f(t) is the value of the probability density function of X at t.

4

Probability Functions
A probability function is a function which assigns probabilities to the values of a random variable.
(i) All the probabilities must be between 0 and 1 inclusive
(ii) The sum of the probabilities of the outcomes must be 1.
If these two conditions aren't met, then the function isn't a probability function. There is no
requirement that the values of the random variable only be between 0 and 1, only that the probabilities
be between 0 and 1.

Probability Distribution
The probability distribution of a discrete random variable is a list of probabilities associated
with each of its possible values. It is also sometimes called the probability function or the probability
mass function. More formally, the probability distribution of a discrete random variable X is a function
which gives the probability p(xi) that the random variable equals xi, for each value xi: p(x
i
) =
P(X=x
i
)
It satisfies the following conditions:
(i) 0 p(x
i
) 1
(ii) p (x
i
) = 1
A listing of all the values the random variable can assume with their corresponding probabilities
make a probability distribution. A note about random variables. A random variable does not mean that
the values can be anything (a random number). Random variables have a well-defined set of outcomes
and well defined probabilities for the occurrence of each outcome. The random refers to the fact that
the outcomes happen by chance -- that is, you don't know which outcome will occur next. Here's an
example probability distribution that results from the rolling of a single fair die.

x 1 2 3 4 5 6 sum
p(x) 1/6 1/6 1/6 1/6 1/6 1/6 6/6=1

MATHEMATICAL EXPECTATION OF DISCRETE VARIABLE
If X denotes a discrete random variable which can assume the value x
1
,x
2
,..,x
n
with
respective probabilities p
1
,p
2
,,p
n
then the mathematical expectation of X, denoted by E(X) is
defined by
E(X) = p
1
x
1
+p
2
x
2
+.+p
n
x
n

= p
i
x
i
where p
i
= 1
Thus E(X) is the weighted arithmetic mean of the values x
i
with the weight to p (x
i
)
mean X = E(X) Hence the mathematical expectation E(X) of a random variable is simply the
arithmetic mean.
5

The expected value (or population mean) of a random variable indicates its average or central
value. It is a useful summary value (a number) of the variable's distribution. Stating the expected value
gives a general impression of the behaviour of some random variable without giving full details of its
probability distribution (if it is discrete) or its probability density function (if it is continuous). Two
random variables with the same expected value can have very different distributions. There are other
useful descriptive measures which affect the shape of the distribution, for example variance. The
expected value of a random variable X is symbolised by E(X) or . If X is a discrete random variable
with possible values x
1
, x
2
, x
3
, ..., x
n
, and p(x
i
) denotes P(X = x
i
), then the expected value of X is
defined by:
= E (X) = x
i
p(x
i
)
Where the elements are summed over all values of the random variable X.
If X is a continuous random variable with probability density function f(x), then the expected value of
X is defined by:
= E (X) =
Result: If (x) is a function of a random variable X, then E [(x)] = P(X=x) (x).
Properties of mathematical expectation:
(1) E (c) = c where c is a constant
(2) E (c X) = c E (X) where c is a constant.
(3) E (a X +b) = a E (X) + b where a & b are constants.
(4) Variance of X = var (X)
= E{X -E(X)}
2

(5) Var (X) = E (X
2
) - [E(X)]
2

(6) Var (X c) = Var (X) where c is a constant.
(7) Var (a X) = a
2
Var (X)
(8) Var (a X +b) = a
2
Var(X)
(9) Var (c) = 0 where c is a constant.

Example:
Discrete case: When a die is thrown, each of the possible faces 1, 2, 3, 4, 5, 6 (the xi's) has a
probability of 1/6 (the p(xi)'s) of showing. The expected value of the face showing is therefore:
= E(X) = (1 x 1/6) + (2 x 1/6) + (3 x 1/6) + (4 x 1/6) + (5 x 1/6) + (6 x 1/6) = 3.5
Notice that, in this case, E(X) is 3.5, which is not a possible value of X.







6

BINOMIAL DISTRIBUTION
Binomial Experiment
A binomial experiment is an experiment which satisfies these four conditions
A fixed number of trials
Each trial is independent of the others
There are only two outcomes
The probability of each outcome remains constant from trial to trial.
These can be summarized as: An experiment with a fixed number of independent trials, each of
which can only have two possible outcomes. The fact that each trial is independent actually means
that the probabilities remain constant.
Examples of binomial experiments
(i) Tossing a coin 20 times to see how many tails occur.
(ii) Asking 200 people if they watch ABC news.
(iii) Rolling a die to see if a 5 appears.
Examples which aren't binomial experiments
(i) Rolling a die until a 6 appears (not a fixed number of trials)
(ii) Asking 20 people how old they are (not two outcomes)
(iii) Drawing 5 cards from a deck for a poker hand (done without replacement, so not
independent)
Introduction:
Binomial distribution was discovered by James Bernoulli (1654 - 1705) in the year 1700 and was first
published in 1713. An experiment which has two mutually disjoint outcomes is called a Bernoulli trail.
The two outcomes are usually called success and failure. An experiment consisting of repeated
number of Bernoulli trails is called Binomial experiment. A Binomial distribution can be used under
the following conditions:
i. The number of trials is finite.
ii. The trials are independent of each other.
iii. The probability of success is constant for each trial.
Probability Function of Binomial Distribution
Let X denotes the number of success in n trial satisfying binomial distribution conditions. X is
a random variable which can take the values 0,1,2,..,n, since we may get no success, one
success,.or all n success.
The general expression for the probability of x success is given by
P(X=x) =
n
C
x
p
x
q
n-x
,x=0,1,2,3,.,n. Where p = probability of success in each trial, q=1-p
Definition: A random variable X is said to follow binomial distribution, if is probability mass function
if given by P(X = x) =
n
C
x
p
x
q
n-x
,x=0,1,2,3,.,n.
= 0 ; Otherwise Where n, p are called parameters of the distribution.
7

Constants of the binomial distribution:
Mean = np
Variance = npq
Standard Deviation =
Note:
(i) 0 p 1, 0 q 1 and p + q = 1
(ii) In binomial distribution mean is always greater then variance.
(iii) To denote the random variable X which follows binomial distribution with
parameters n and p is X~B(n,p).
We define a Bernoulli trial. as a random event that can take on only one of two possible outcomes,
with the outcomes arbitrarily denoted as either a success or failure. For example, flipping a coin is
an example of a Bernoulli trial since the outcome is either a head or a tail. (It doesnt matter which of
these outcomes we call a success, as long as we are consistent throughout the course of the problem.)
The total number of successes (X) observed in a series of n independent Bernoulli trials is a binomial
random variable, and the listing of probabilities for all possible outcomes is a binomial distribution.
For example, the number of heads occurring when tossing a fair coin twice is a binomial random
variable with the following distribution:
P (0 heads) = 0.25 P (1 heads) = 0.50 P (2 heads) = 0.25
Although the number of heads in tossing a fair coin twice is a trivial example of a binomial
distribution
Calculation of Binomial Probabilities
To calculate binomial probabilities, we must first learn about the choose function. The
choose function quantifies the number of different ways to choose i objects of n. Let
n
C
i
denote the
number of different ways to choose i items out of n without repetition. Then, where "!" represents the
factorial function. The factorial function is the product of the series of integers from n to 1.
Thus, n! = (n)(n - 1) (n - 2)(n - 3) . . . (1). For example, 2! = (2)(1) = 2, 3! = (3)(2)(1) = 6, and 4! =
(4)(3)(2)(1) =
24. by definition, 0! = 1
Cumulative Probability Functions
In addition to wanting to know the probability of a given number of successes, we often want
to know the probability of observing less than or equal to a given number of successes. This is
referred to as the cumulative probability of the event.
Cumulative Binomial I llustrative Distribution. The cumulative probability function for illustrative
distribution 1 (the number of head out of 2 tosses) is:
Probability of at least 0 heads = P (X 0) = P (X = 0) = .25
Probability of at least 1 head = P (X 1) = P (X 0) + P (X = 1) = .25 + .50 = .75
Probability of at least 2 heads = P (X 2) = P (X1) + P (X = 2) = .75 + .25 = 1.00

8

GEOMETRIC DISTRIBUTION
Geometric distributions model (some) discrete random variables. Typically, a Geometric random
variable is the number of trials required to obtain the first failure, for example, the number of tosses of
a coin until the first 'tail' is obtained, or a process where components from a production line are tested,
in turn, until the first defective item is found.
A discrete random variable X is said to follow a Geometric distribution with parameter p, written X ~
Ge(p), if it has probability distribution
P(X=x) = p
x-1
(1-p)
x

where x = 1, 2, 3, ... p = success probability; 0 < p < 1
The trials must meet the following requirements:
a) The total number of trials is potentially infinite;
b) There are just two outcomes of each trial; success and failure;
c) The outcomes of all the trials are statistically independent;
d) All the trials have the same probability of success.
The Geometric distribution has expected value E(X)= 1/(1-p) and variance V(X)=p/{(1-p)
2
}.
The Geometric distribution is related to the Binomial distribution in that both are based on
independent trials in which the probability of success is constant and equal to p. However, a
Geometric random variable is the number of trials until the first failure, whereas a Binomial random
variable is the number of successes in n trials.
UNIFORM DISTRIBUTION
Uniform distributions model (some) continuous random variables and (some) discrete random
variables. The values of a uniform random variable are uniformly distributed over an interval. For
example, if buses arrive at a given bus stop every 15 minutes, and you arrive at the bus stop at a
random time, the time you wait for the next bus to arrive could be described by a uniform distribution
over the interval from 0 to 15.
A discrete random variable X is said to follow a Uniform distribution with parameters a and b, written
X ~ Un(a,b), if it has probability distribution
P(X=x) = 1/(b-a) where x = 1, 2, 3, ......., n.
A discrete uniform distribution has equal probability at each of its n values. A continuous random
variable X is said to follow a Uniform distribution with parameters a and b, written X ~ Un(a,b), if its
probability density function is constant within a finite interval [a,b], and zero outside this interval
(with a less than or equal to b). The Uniform distribution has expected value E(X)=(a+b)/2 and
variance {(b-a)
2
}/12.




9

Chi-Square
Chi-square distribution
A distribution obtained from the multiplying the ratio of sample variance to population
variance by the degrees of freedom when random samples are selected from a normally distributed
population
Contingency Table: Data arranged in table form for the chi-square independence test
Expected Frequency: The frequencies obtained by calculation.
Goodness-of-fit Test: A test to see if a sample comes from a population with the given distribution.
I ndependence Test :A test to see if the row and column variables are independent.
Observed Frequency :The frequencies obtained by observation. These are the sample frequencies.
HYPOTHESIS TESTING
Null Hypothesis ( H
0
): Statement of zero or no change. If the original claim includes equality (<=, =,
or >=), it is the null hypothesis. If the original claim does not include equality (<, not equal, >) then the
null hypothesis is the complement of the original claim. The null hypothesis always includes the equal
sign. The decision is based on the null hypothesis.
Alternative Hypothesis ( H
1
or H
a
): Statement which is true if the null hypothesis is false. The type of
test (left, right, or two-tail) is based on the alternative hypothesis.
Type I error: Rejecting the null hypothesis when it is true (saying false when true). Usually the more
serious error.
Type I I error: Failing to reject the null hypothesis when it is false (saying true when false).
Alpha: Probability of committing a Type I error.
Beta: Probability of committing a Type II error.
Test statistic: Sample statistic used to decide whether to reject or fail to reject the null hypothesis.
Critical region: Set of all values which would cause us to reject H0
Critical value(s): The value(s) which separate the critical region from the non-critical region. The
critical values are determined independently of the sample statistics.
Significance level (alpha): The probability of rejecting the null hypothesis when it is true. Alpha =
0.05 and alpha = 0.01 are common. If no level of significance is given, use alpha = 0.05. The level of
significance is the complement of the level of confidence in estimation.
Decision: A statement based upon the null hypothesis. It is either "rejects the null hypothesis" or "fail
to reject the null hypothesis". We will never accept the null hypothesis.
Conclusion: A statement which indicates the level of evidence (sufficient or insufficient), at what
level of significance, and whether the original claim is rejected (null) or supported (alternative).