Why do we need to study Probability and Random Processes

Probability theory provides a power tools to

Explain Model Analyze Design the technologies which are developed.

 PROBABILITY THEORY and PROBABILITY MODELs provide a tool that enables the designer to successfully design systems than must operate in RANDOM ENVIORNMENT, but nevertheless are efficient, reliable and cost effective.

Examples
Signal Processing Optical Communication Wireless Communications Variability in electronic circuits. Computer network traffic.

Telecommunication
Analog signal : Deterministic signal Modulated Signal : Deterministic signal Noise signal : Predictable Signal. Transmitted signal as well as the noise in the physical transmitted medium: RANDAOM / STOCASTIC. These phenomena cannot be predicted in advance but they have certain predictable characteristics . These predictable characteristics are summarized and called as : RANDOM PROCESS MODEL.

Radar Communication

Signal hidden in Noise. Noise is a random signal Noise can be characterized by its probabilitistic nature.

Optical Communication System

Photo detector acts as the interface between optical and electronic subsystem. The number of photoelectrons produced by the photo detector depends on the intensity of light and is modeled by POISSON RANDON VARIABLE. In deciding whether the transmitted bit is zero or one, the receiver counts the number of photoelectrons and compares it with threshold. System performance is determined by computing the probability that the threshold is exceeded.

Wireless Communication System

In order to increase the weak signals we use Amplifiers. These amplifiers generate thermal noise, which is added to the desired signal. Noise has a Guassian distribution (Underlying Physics)

Variability in electronics circuits

Although circuit manufacturing process attempt to ensure that all items have nominal parameters values, there is always some amount of variations among items. Using probability theory we can
Estimate the average values in a batch of items without testing all of them. Check how good is our estimate

Reliability of Systems
Two basic Configurationss
Series System
If one component fails the entire system fails.

Parallel System
If one component fails still the system functions

It is not possible to predict exactly when a component will fail However probability theory allows us to evaluate the MEASURES OF RELIABILITY.
Average time to failure. Probability that a component is still functioning after certain time has elapsed.

Markov Chain / Markov Process

All digital communication sytems are modeled as markov process/ markov chain. Digital circuits with internal memory such as flipflops, RAM, registers, Finite state machines are modelled by markov process / markov chain.

Queing Theory
The question that usually arises in a communication system is queing problem. Que : buffer which stores messages. Assume server can process one message at a time. So if more than one message is being processes then since the buffer has limited space the other messages have to wait for service.

Performance measures to be found

Average number of customers (messages) in the system (L) Average number of customers waiting in que. (Lq) Average number of customers in service (Ls) L =Lq+ Ls Average time customer spends in que (Wq) Average time a customer spends in service (Ws) Average time a customer spends in system (W). W= Wq + Ws

Experiments
Deterministic Randon

Deterministic Experiment
An experiment whose outcome or result can be predicted with CERTAINITY is called deterministic experiment. For example: If E is the potential difference between the two ends of a resistor (R) , then the current flowing through the resistor is uniquely determined by Ohms law: I= E/R This is a Deterministic experiment.

Random Experiment
Any experiment in which ALL the outcomes are known in ADVANCE but the outcome of a PARTICULAR PERFORMANCE of the experiment cannot be predicted with certainty is called as Random experiment.

Example of Random Experiment

Tossing a coin:
Two possible outcomes (Head or Tail) But outcome of a particular toss cannot be predicted.

A fair 6 face cubic dice is rolled.

It is known that the outcome will be any of the 6 possible values But it cannot be predicted what exactly the outcome will be, when the dice is rolled at a point of time.

Example of Random Experiment

Although the number of telephone calls received in a board in a 5 min interval is a non negative interger we cannot predict exactly the number of calls in the next 5 mins. In such cases we talk of CHANCE or the PROBABILITY (RELATIVE FREQUENCY) of occurrence of a particular outcome

Sample space
We perform an experiment which can have a number of different outcomes. The sample space is the set of all possible outcomes of the experiment. We usually call it S.

Sample Space / Set

A set of all possible outcomes ( which are assumed to be equally likely) of a random experiment is called SAMPLE SPACE or UNIVERSEL SET . An element of Set is called as sample point. Definition:
A set is a collection of objects / outcomes called elements. The elements of a set can also be called as set Set are represented by uppercase letter A Elements of set are represented by lower case letter a.

Sample Space / SET

Set A contains elements a1, a2,an. Element ak which is an element of A can be represented as: Element ak which is not an element of A can be represented as:

Example
It is important to be able to list the outcomes clearly. For example:
if I plant ten bean seeds and count the number that germinate, the sample space is

S = {0,1,2,3,4,5,6,7,8,9,10}.
If I toss a coin three times and record the result, the sample space is

S = {HHH,HHT,HTH,HTT,THH,THT,TTH,TTT},

Examples of Sample Space

Example of Sample Space

Sample space for variability in electronic circuits

Sample space for Computer network traffic

 Noise voltage is modelled by a sample space consisting of all real numbers, i.e., = (,). Outcomes: the individual numbers such as 1.5, 8, and are outcomes. Events: Subsets such as the interval [0,5] = {v : 0 v 5} Another event would be {2,4,7.13}

Event
Elements or points in the sample space are called as OUTCOMES. Collection of outcomes is called : Event Event is a subset of sample space S. Event is a collection of certain sample points of S. Example: If I toss a coin three times and record the result, the sample space is
S = {HHH,HHT,HTH,HTT,THH,THT,TTH,TTT},

let A be the event more heads than tails and B the event heads on last throw. Then
A = {HHH,HHT,HTH,THH}, B = {HHH,HTH,THH,TTH}.

Singleton Set
Singleton sets, that is sets
consisting of a single point, are also events; e.g., {1.5}, {8}, {}

Be sure you understand

the difference between the outcome 8 and the event {8}, which is the set consisting of the single outcome 8.

EVENT
SAMPLE SPACE / SET which consists of all possible outcomes of the experiment. If the sample space is the set of all triples (b1,b2,b3), where the bi are 0 or 1,
then any particular triplet, say (0,0,0) or (1,0,1) would be an outcome.

An event would be a subset of the sample space (universal set).

such as the set of all triples with exactly one 1; i.e., {(0,0,1), (0,1,0), (1,0,0)}.

An example of a singleton event would be {(1,0,1)}.

EVENT In modeling the resistance and capacitance of the RC filter

above, we suggested the sample space = {(r,c) : 95 r 105 and 300 c 340}. OUTCOME: If a particular circuit has R=101 ohms and C =327 F, this would correspond to the outcome (101,327), which is indicated by the dot in Figure. EVENT If we observed a particular circuit with R97 ohms andC 313 F, this would correspond to the event {(r,c) : 95 r 97 and 313 c 340}, which is the shaded region in Figure The dot is the outcome (101,327). The shaded region is the event {(r,c) : 95 r 97 and 313 c 340}.

EVENTS
Exhaustive events Favorable events Mutually exclusive events Equally likely events Independent events

Venn Diagram
Sets can be represented graphically by means of Venn diagram. We first define a universal set S is the universal set S={x: x is all positive integers} A subset (EVENT) A can be represented by means of venn diagram as shown in fig. A={1,2,3,4,5,6} Then its venn diagram is as follows:

Venn diagram
S={x: x is all positive integers} A={1,2,3,4,5,6}

Inclusion
Another kind of relationship is that
one set may be a subset of another set. If we have two sets:

S = {1,3, 5, 7,9} and T = {3, 7} then T is a proper subset of S because every element of T is also an element of S. Using the set inclusion symbols (is contained in) and (includes), we may write either T S or S T. If the two sets contain exactly the same elements, then they are both subsets of each other i.e. S S and S S.

Venn Diagram
Set B is a subset of A That is every element of B is an element of A. The subset B of A can be written as B is a subset of A but if every element of A is not in B then it is written as:

Equality of Sets
Two sets A and B. They are said to be equal if That is every element of B is contained in A and every element of A is contained in B. That is sets A and B contain exactly same elements.

Example
S={x: x is all positive integers} Set A={1,2,3,4,5,6} Subset set B = {1,3,5} B is a subset of A. Whereas set C = {x:x is a positive integer 6} and set D = {2,4,6} C and D are same since both contain the same elements

Venn diagram of above example

Empty Set
A empty set is a set that contains no elements. It is denoted by NOTW: The set A={0} is not an empty set since it contains the element 0.

Set Operations / Mathematical Operations

We can build new events from old events by using set operators.
Union Intersection Complement

Union
Union of two sets A and B is a third set C which includes all of the elements that are in either A or B or both.

Union

Intersection
The intersection of two sets, A and B, is the set of all elements that are in both A and B."

Intersection

Complement of an event

Summary

Disjoint or Mutually Exclusive Events

Mutually exclusive set / event

In throwing a dice, the outcomes are mutually exclusive as occurrence of one face excludes the occurrence of remaining 5 faces.

Exhaustive Event
All possible outcomes of an experiment is called exhaustive event. Example:
In tossing a coin either {head} or {tail} turns up. There is no other possibility Therefore these two outcomes are exhaustive events.

Favorable Event
The number of outcomes which results to the desired event are called favorable event. Example:
In throwing of 2 dice , the number of cases favorable to get a sum of 6 is: (1,5); (2,4); (3,3); (4,2); (5,1) i.e there are 5 favorable cases.

Equally likely event

Two or more events are said to be equally likely if the chances of their happening is equal. That is there is no preference of occurance of one event over the other. Example:
In throwing a coin , getting a {head} or a {tail} is equally likely . In throwing a dice getting events as {1}, {2}, {3}, {4}, {5}, {6} are equally likely Such events are called as equally likely events.