Probability and Random

Processes
Lecture no. (15)
Dr Sherif Rabia
Department of Engineering Mathematics and Physics,
Faculty of Engineering, Alexandria University
sherif.rabia@ gmail.com
sherif.rabia@ alex.edu.eg

Part 4: Stochastic processes

4.1 Introduction
4.2 Markov chains
4.3 Expected value and correlation

Math 9 - 2014/2015 (second term)

4.1 Introduction
From RVs to stochastic processes
Definition and classification
Examples

Math 9 - 2014/2015 (second term)

From RVs to stochastic processes

Example 15.1
W

A random variable

Your waiting time

W 1, W 2

Your and his waiting time

W 1, W 2, . . .

A pair of random variables

Waiting time of each customer

Discrete-parameter
stochastic process

A sequence of random variables

Example 15.2
Q

Queue length upon your arrival

A random variable

Queue length initially and

at the end

A pair of random variables

Q0, QT

Qt, t [0, T]

Queue length at any time t

A collection of random variables

4

Math 9 - 2014/2015 (second term)

Continuous-parameter
stochastic process

Definition and classification

A stochastic process is a collection of random
variables defined on the same sample space

Stochastic process

Discreteparameter(time)

Continuousparameter(time)

Stochastic process

Discrete-value
Example 15.2

Example 15.1
5

Example 15.2
Math 9 - 2014/2015 (second term)

Continuous-value
Example 15.1

Time index (parameter) of a stochastic

process

The collection of random variables is usually

obtained by observing the same phenomenon at
various time epochs.
If we observe at discrete points, we have a
discrete-time (discrete-parameter) process:
{W n, n = 1, 2, 3, . . .}.
If the observation is over a certain time
period,
we
have
a
continuous-time
(continuous-parameter) stochastic process:
{Qt, t [0, T]}.
Math 9 - 2014/2015 (second term)

State space of a stochastic process

The random variables in the collection have a

common set of possible values RX.
RX is called the state space of the process.
At any time, the process is in a certain state
which is an element of RX.
If RX is a discrete set, the process is a
discrete-state process (a collection of discrete
RVs)
If RX is a continuous set, the process is a
continuous-state process (a collection of
continuous RVs)
Math 9 - 2014/2015 (second term)

Classification
Based on the notions of time index and state
space, stochastic processes can be
classified into four categories.
State space

Time index

Discrete

Continuous

Discrete

DD

DC

Continuous

CD

CC

The following examples illustrate this classification.

8

Math 9 - 2014/2015 (second term)

Example 15.3
A

(1) Let Xi be the trip time from city A to city B in the

ith day.
Then {Xi, i = 1, 2, 3, . . .} is a DC stochastic
process.

(2) Let Yi be the number of passengers in the train

from city A to city B in the ith day.
Then {Yi, i = 1, 2, 3, . . .} is a DD stochastic
process.
9

Math 9 - 2014/2015 (second term)

Example 15.4
Hall
Teller
Queue

(1) Let Qt be the queue length at time t.

Then {Qt, t [0, T]} is a CD stochastic
process.
(2) Let Pt be the hall temperature at time t.
Then {Pt, t [0, T]} is a CC stochastic process.
10

Math 9 - 2014/2015 (second term)