Tutorial on Particle Filters

assembled and extended by Longin Jan Latecki

Temple University, latecki@temple.edu

using slides from
Keith Copsey, Pattern and Information Processing Group, DERA
Malvern;
D. Fox, J. Hightower, L. Liao, D. Schulz, and G. Borriello,
Univ. of Washington, Seattle
Honggang Zhang, Univ. of Maryland, College Park
Miodrag Bolic, University of Ottawa, Canada
Michael Pfeiffer, TU Gratz, Austria

Outline
Introduction to particle filters
Recursive Bayesian estimation
Bayesian Importance sampling
Sequential Importance sampling (SIS)
Sampling Importance resampling (SIR)
Improvements to SIR
On-line Markov chain Monte Carlo
Basic Particle Filter algorithm
Example for robot localization
Conclusions
Particle Filters
Sequential Monte Carlo methods for on-line learning within
a Bayesian framework.

Known as
Particle filters
Sequential sampling-importance resampling (SIR)
Bootstrap filters
Condensation trackers
Interacting particle approximations
Survival of the fittest
History
First attempts simulations of growing polymers
M. N. Rosenbluth and A.W. Rosenbluth, Monte Carlo calculation of the average extension of molecular chains, Journal of
Chemical Physics, vol. 23, no. 2, pp. 356359, 1956.
First application in signal processing - 1993
N. J. Gordon, D. J. Salmond, and A. F. M. Smith, Novel approach to nonlinear/non-Gaussian Bayesian state estimation,
IEE Proceedings-F, vol. 140, no. 2, pp. 107113, 1993.
Books
A. Doucet, N. de Freitas, and N. Gordon, Eds., Sequential Monte Carlo Methods in Practice, Springer, 2001.
B. Ristic, S. Arulampalam, N. Gordon, Beyond the Kalman Filter: Particle Filters for Tracking Applications, Artech House
Publishers, 2004.
Tutorials
M. S. Arulampalam, S. Maskell, N. Gordon, and T. Clapp, A tutorial on particle filters for online nonlinear/non-gaussian
Bayesian tracking, IEEE Transactions on Signal Processing, vol. 50, no. 2, pp. 174188, 2002.

Problem Statement


Tracking the state of a system as it evolves over time

Sequentially arriving (noisy or ambiguous)
observations

We want to know: Best possible estimate of the
hidden variables


Solution: Sequential Update

Storing and processing all incoming measurements is
inconvenient and may be impossible
Recursive filtering:
Predict next state pdf from current estimate
Update the prediction using sequentially arriving
new measurements

Optimal Bayesian solution: recursively calculating
exact posterior density
Particle filtering ideas
Particle filter is a technique for implementing recursive Bayesian filter by Monte Carlo
sampling
The idea: represent the posterior density by a set of random particles with associated
weights.
Compute estimates based on these samples and weights
Sample space
Posterior density
Global Localization of Robot with Sonar
http://www.cs.washington.edu/ai/Mobile_Robotics/mcl/animations/global-floor.gif
Tools needed
1 1 1
( ) ( | ) ( )
t t t t t
p x p x x p x dx

=
}
( | ) ( )
( | )
( )
t t t
t t
t
p z x p x
p x z
p z
=
Recall law of total probability (or marginalization)
and Bayes rule
Recursive Bayesian estimation (I)
Recursive filter:
System model:

Measurement model:

Information available:
) | ( ) , (
1 1
=
k k k k k k
x x p x f x e
) | ( ) , (
k k k k k k
x y p x h y = u
) , , (
1 k k
y y D =
) (
0
x p
Recursive Bayesian estimation (II)
Seek:
i = 0: filtering.
i > 0: prediction.
i<0: smoothing.

Prediction:

since:
) | (
k i k
D x p
+
}
=
1 1 1 1
) | , ( ) | (
k k k k k k
dx D x x p D x p
}
=
1 1 1 1 1
) | ( ) | ( ) | (
k k k k k k k
dx D x p x x p D x p
) | ( ) | ( ) | ( ) , | ( ) | , (
1 1 1 1 1 1 1 1 1
= =
k k k k k k k k k k k k
D x p x x p D x p D x x p D x x p
Recursive Bayesian estimation (III)
Update:



where:


since:

}
=
k k k k k k
dx D x y p D y p ) | , ( ) | (
1 1
}
=
k k k k k k k
dx D x p x y p D y p ) | ( ) | ( ) | (
1 1
) | (
) | ( ) | (
) | (
1
1

=
k k
k k k k
k k
D y p
D x p x y p
D x p
) | ( ) | ( ) | ( ) , | ( ) | , (
1 1 1 1
= =
k k k k k k k k k k k k
D x p x y p D x p D x y p D x y p
Bayes Filters (second pass)
1
( , )
( , )
t t t t
t t t t
x f x w
z g x v

=
=
System state dynamics
Observation dynamics
1
( ) ( | , , )
t t t
Bel x p x z z =
We are interested in: Belief or posterior density
Estimating system state from noisy observations
1:( 1) 1 1
where , ,
t t
z z z

1:( 1) 1, 1:( 1) 1 1:( 1) 1
( | ) ( | ) ( | )
t t t t t t t t
p x z p x x z p x z dx

=
}
From above, constructing two steps of Bayes Filters
1:( 1)
1:( 1) 1:( 1)
1:( 1)
( | , )
( | , ) ( | )
( | )
t t t
t t t t t
t t
p z x z
p x z z p x z
p z z

=
Predict:
Update:
1:( 1) 1, 1:( 1) 1 1:( 1) 1
( | ) ( | ) ( | )
t t t t t t t t
p x z p x x z p x z dx

=
}
1:( 1)
replace ( | , ) with ( | )
t t t t t
p z x z p z x

Predict:
Update:
Assumptions: Markov Process
1 1: 1 1
replace ( | , ) with ( | )
t t t t t
p x x z p x x

1:( 1)
1:( 1) 1:( 1)
1:( 1)
( | , )
( | , ) ( | )
( | )
t t t
t t t t t
t t
p z x z
p x z z p x z
p z z

=
1:( 1) 1:( 1)
( | , ) ( | ) ( | )
t t t t t t t t
p x z z p z x p x z o

=
Bayes Filter
1:( 1) 1 1 1:( 1) 1
( | ) ( | ) ( | )
t t t t t t t
p x z p x x p x z dx

=
}
1
( | )
( | )
t t
t t
p x x
p z x

How to use it? What else to know?

Motion Model
Perceptual Model
Start from:
0 0
0 0 0
0
( | )
( | ) ( )
( )
p z x
p x z p x
p z
=
Example 1
1
0 0
( ) or ( ) Bel x p x

Step 0: initialization
0 0 0
0 0 0 0
( ) or ( | )
( | ) ( )
Bel x p x z
p z x p x o =
Step 1: updating
Example 1 (continue)
1 1 1
1 1 1 0 0
( ) or ( | )
( | ) ( | )
Bel x p x z
p z x p x z o =
Step 3: updating
1
2 2 1
2 1 1 1 1
( ) or ( | )
( | ) ( | )
Bel x p x z
p x x p x z dx

=
}
Step 4: predicting
1
1 1 0
1 0 0 0 0
( ) or ( | )
( | ) ( | )
Bel x p x z
p x x p x z dx

=
}
Step 2: predicting
1
Classical approximations
Analytical methods:
Extended Kalman filter,
Gaussian sums (Alspach et al. 1971)
Perform poorly in numerous cases of interest

Numerical methods:
point masses approximations,
splines. (Bucy 1971, de Figueiro 1974)
Very complex to implement, not flexible.
Perfect Monte Carlo simulation
Recall that


Random samples are drawn from the posterior distribution.

Represent posterior distribution using a set of samples or particles.



Easy to approximate expectations of the form:



by:
) , , (
0 : 0 k k
x x x =
}
=
k k k k k
dx D x p x g x g E
: 0 : 0 : 0 : 0
) | ( ) ( )) ( (

=
=
N
i
i
k k
x g
N
x g E
1
: 0 : 0
) (
1
)) ( (
i
k
x
: 0

=
=
N
i
i
k k k k
x x
N
D x p
1
: 0 : 0 : 0
) (
1
) | ( o

Random samples and the pdf (I)

Take p(x)=Gamma(4,1)
Generate some random samples
Plot histogram and basic approximation to pdf
0 2 4 6 8 10 12 14 16 18 20
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0 20 40 60 80 100 120 140 160 180 200
0
2
4
6
8
10
12
200 samples
Random samples and the pdf (II)
0 2 4 6 8 10 12 14 16 18 20
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0 2 4 6 8 10 12 14 16 18 20
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
500 samples 1000 samples
Random samples and the pdf (III)
0 5 10 15 20 25
0
0.05
0.1
0.15
0.2
0.25
0 5 10 15 20 25
0
0.05
0.1
0.15
0.2
0.25
200000 samples
5000 samples
Importance Sampling
Unfortunately it is often not possible to sample directly from the posterior
distribution, but we can use importance sampling.

Let p(x) be a pdf from which it is difficult to draw samples.

Let x
i
~ q(x), i=1, , N, be samples that are easily generated from a proposal pdf q,
which is called an importance density.
Then approximation to the density p is given by


) (
) (
i
i
i
x q
x p
w =
) ( ) (
1
i
n
i
i
x x w x p ~

=
o
where
Bayesian Importance Sampling


By drawing samples from a known easy to sample proposal
distribution we obtain:

=
=
N
i
i
k k
i
k k k
x x w D x p
1
: 0 : 0 : 0
) ( ) | ( o

) | (
: 0 k k
D x q
i
k
x
: 0
) | (
) | (
: 0
: 0
k
i
k
k
i
k
i
k
D x q
D x p
w =
where
are normalized weights.
Sequential Importance Sampling (I)
Factorizing the proposal distribution:



and remembering that the state evolution is modeled as a Markov process

we obtain a recursive estimate of the importance weights:




Factorizing is obtained by recursively applying
[
=

=
k
j
j j j k k
D x x q x q D x q
1
1 : 0 0 : 0
) , | ( ) ( ) | (
) , | (
) | ( ) | (
1 : 0
1
1
k k k
k k k k
k k
D x x q
x x p x y p
w w

=
) | ( ) , | ( ) | (
1 1 : 0 1 : 0 : 0
=
k k k k k k k
D x q D x x q D x q
Sequential Importance Sampling (SIS) Particle Filter
SIS Particle Filter Algorithm
] , } , [{ ] } , [{
1 1 1 1 k
N
i
i
k
i
k
N
i
i
k
i
k
z w x SIS w x
= =
=
for i=1:N

Draw a particle

Assign a weight


end
) , | ( ~
1 k
i
k
i
k
i
k
z x x q x

) , | (
) | ( ) | (
1 : 0
1
1
k
i
k
i
k
i
k
i
k
i
k k
i
k
i
k
D x x q
x x p x z p
w w

=
(k is index over time and i is the particle index)
Derivation of SIS weights (I)
The main idea is Factorizing :

[
=

=
k
j
j j k
x x p x p x p
1
1 0 : 0
) | ( ) ( ) (
and
[
=
=
k
j
j j k k
x y p x D p
1
: 0
) | ( ) | (
) | (
) | (
: 0
: 0
k
i
k
k
i
k
i
k
D x q
D x p
w =
) | ( ) , | ( ) | (
1 1 : 0 1 : 0 : 0
=
k k k k k k k
D x q D x x q D x q
Our goal is to expand p and q in time t
Derivation of SIS weights (II)
) ( ) | (
) ( ) | ( ) | ( ) , | (
) (
) ( ) | (
) | (
1 1
1 : 0 1 : 0 : 0 1 : 0 1 : 0 : 0
: 0


= =
k k k
k k k k k k k k
k
k k k
k k
D p D z p
x p x x p x D p x D z p
D p
x p x D p
D x p
) | (
) | ( ) | ( ) , | (
1
1 1 : 0 1 : 0 : 0 1

=
k k
k k k k k k k
D z p
D x p x x p x D z p
) ( ) | (
) ( ) | ( ) | ( ) , | (
1 1
1 : 0 1 : 0 1 1 : 0 : 0 1


=
k k k
k k k k k k k k
D p D z p
x p x D p x x p x D z p
) | ( ) | ( ) , | (
1 1 : 0 1 : 0 : 0 1

k k k k k k k
D x p x x p x D z p
) | (
) | (
) , | (
) | ( ) , | (
) | (
) | (
1 1 : 0
1 1 : 0
1 : 0
1 : 0 : 0 1
: 0
: 0

= =
k
i
k
k
i
k
k
i
k
i
k
i
k
i
k
i
k k k
k
i
k
k
i
k
i
k
D x q
D x p
D x x q
x x p x D z p
D x q
D x p
w
Derivation of SIS weights (II)
) , | (
) | ( ) , | (
1 : 0
1 : 0 : 0 1
1
k
i
k
i
k
i
k
i
k
i
k k k
i
k
D x x q
x x p x D z p
w

=
) , | (
) | ( ) | (
1
1
1
k
i
k
i
k
i
k
i
k
i
k k
i
k
i
k
z x x q
x x p x z p
w w

=
and under Markov assumptions
SIS Particle Filter Foundation

At each time step k
Random samples are drawn from the proposal distribution for i=1, , N



They represent posterior distribution using a set of samples or particles



Since the weights are given by


and q factorizes as

=
=
N
i
i
k k
i
k k k
x x w D x p
1
) ( ) | ( o

i
k
x
) , | (
1 : 0 k k k
D x x q

) | (
) | (
: 0
: 0
k
i
k
k
i
k
i
k
D x q
D x p
w =
[
=

=
k
j
j j j k k
D x x q x q D x q
1
1 : 0 0 : 0
) , | ( ) ( ) | (
Sequential Importance Sampling (II)
Choice of the proposal distribution:


Choose proposal function to minimize variance of (Doucet et al. 1999):


Although common choice is the prior distribution:



We obtain then

) , | (
1 : 0 k k k
D x x q

k
w
) , | ( ) , | (
1 : 0 1 : 0 k k k k k k
D x x p D x x q

=
) | ( ) , | (
1 1 : 0
=
k k k k k
x x p D x x q
) | (
) , | (
) | ( ) | (
1
1 : 0
1
1
i
k k
i
k
k
i
k
i
k
i
k
i
k
i
k k
i
k
i
k
x z p w
D x x q
x x p x z p
w w

= =
Illustration of SIS:









Degeneracy problems:
variance of importance ratios
increases stochastically over time (Kong et al. 1994; Doucet
et al. 1999).
In most cases then after a few iterations, all but one particle
will have negligible weight
Sequential Importance Sampling (III)
w
Time 19
w
Time 10
w
Time 1
) | ( / ) | (
: 0 : 0 k k k k
D x q D x p
Sequential Importance Sampling (IV)
Illustration of degeneracy:
w
Time 19
w
Time 10
w
Time 1
SIS - why variance increase
Suppose we want to sample from the posterior
choose a proposal density to be very close to the posterior
density
Then

and


So we expect the variance to be close to 0 to obtain
reasonable estimates
thus a variance increase has a harmful effect on accuracy
1
) | (
) | (
: 0
: 0
=
|
|
.
|

\
|
k k
k k
q
D x q
D x p
E
0 1
) | (
) | (
) | (
) | (
var
2
: 0
: 0
: 0
: 0
=
|
|
.
|

\
|
|
|
.
|

\
|
=
|
|
.
|

\
|
k k
k k
q
k k
k k
q
D x q
D x p
E
D x q
D x p
Sampling-Importance Resampling
SIS suffers from degeneracy problems so we dont want to
do that!
Introduce a selection (resampling) step to eliminate
samples with low importance ratios and multiply samples
with high importance ratios.
Resampling maps the weighted random measure
on to the equally weighted random measure
by sampling uniformly with replacement from
with probabilities

Scheme generates children such that and
satisfies:
)} (
~
, {
: 0 : 0
i
k k
i
k
x w x
} {
1 -
: 0
N x
j
k
} , , 1 ; {
: 0
N i x
i
k
=
} , , 1 ;
~
{ N i w
i
k
=
N N
N
i
i
=

=1
i
N
i
k i
w N N E
~
) ( =
)
~
1 (
~
) var(
i
k
i
k i
w w N N =
Basic SIR Particle Filter - Schematic
Initialisation
Importance
sampling step
Resampling
step
0 = k
1 + k k
)} (
~
, {
: 0 : 0
i
k k
i
k
x w x
} , {
1
: 0

N x
i
k
measurement
k
y
Extract estimate,
k
x
: 0

Basic SIR Particle Filter algorithm (I)

Initialisation

For sample
and set
0 = k
N i , , 1 = ) ( ~
0 0
x p x
i
1 = k
Importance Sampling step

For sample


For compute the importance weights w
i
k

Normalise the importance weights,

=
=
N
j
j
k
i
k
i
k
w w w
1
/
~
N i , , 1 =
) | ( ~
~
1
i
k k
i
k
x x q x

) , (
~
1 : 0 : 0
i
k
i
k
i
k
x x x

= and set
N i , , 1 =
Basic SIR Particle Filter algorithm (II)
Resampling step
Resample with replacement particles:

from the set:

according to the normalised importance weights,

Set
proceed to the Importance Sampling step, as the next
measurement arrives.





N
) , , 1 ; (
: 0
N i x
i
k
=
) , , 1 ;
~
(
: 0
N i x
i
k
=
i
k
w
~
1 + k k
Resampling
M
m
m
k
M
x
1
) (
1
1
,
=

)
`

x
{ }
M
m
m
k
m
k
w x
1
) ( ) (
,
=
M
m
m
k
M
x
1
) (
~
1
,
=
)
`

M
m
m
k
M
x
1
) (
1
1
,
=
+
)
`

{ }
M
m
m
k
m
k
w x
1
) (
1
) (
1
,
= + +
M
m
m
k
M
x
1
) (
1
~
1
,
=
+
)
`

M
m
m
k
M
x
1
) (
2
1
,
=
+
)
`

Generic SIR Particle Filter algorithm

M. S. Arulampalam, S. Maskell,
N. Gordon, and T. Clapp,
A tutorial on particle filters ,
IEEE Trans. on Signal Processing,
50( 2), 2002.
Improvements to SIR (I)
Variety of resampling schemes with varying performance in
terms of the variance of the particles :
Residual sampling (Liu & Chen, 1998).
Systematic sampling (Carpenter et al., 1999).
Mixture of SIS and SIR, only resample when necessary (Liu &
Chen, 1995; Doucet et al., 1999).

Degeneracy may still be a problem:
During resampling a sample with high importance weight may
be duplicated many times.
Samples may eventually collapse to a single point.
) var(
i
N
Improvements to SIR (II)
To alleviate numerical degeneracy problems, sample
smoothing methods may be adopted.
Roughening (Gordon et al., 1993).
Adds an independent jitter to the resampled
particles
Prior boosting (Gordon et al., 1993).
Increase the number of samples from the
proposal distribution to M>N,
but in the resampling stage only draw N
particles.
Improvements to SIR (III)
Local Monte Carlo methods for alleviating degeneracy:
Local linearisation - using an EKF (Doucet, 1999; Pitt &
Shephard, 1999) or UKF (Doucet et al, 2000) to estimate the
importance distribution.
Rejection methods (Mller, 1991; Doucet, 1999; Pitt & Shephard,
1999).
Auxiliary particle filters (Pitt & Shephard, 1999)
Kernel smoothing (Gordon, 1994; Hrzeler & Knsch, 1998; Liu &
West, 2000; Musso et al., 2000).
MCMC methods (Mller, 1992; Gordon & Whitby, 1995; Berzuini et
al., 1997; Gilks & Berzuini, 1998; Andrieu et al., 1999).
Improvements to SIR (IV)
Illustration of SIR with sample smoothing:
w
Time 19
w
Time 10
w
Time 1
Ingredients for SMC
Importance sampling function
Gordon et al
Optimal
UKF pdf from UKF at

Redistribution scheme
Gordon et al SIR
Liu & Chen Residual
Carpenter et al Systematic
Liu & Chen, Doucet et al Resample when necessary

Careful initialisation procedure (for efficiency)

) | (
1
i
k k
x x p

) , | (
1 : 0 k
i
k k
D x x p

i
k
x
1
Particle filters
Also known as Sequential Monte Carlo Methods
Representing belief by sets of samples or
particles



are nonnegative weights called importance
factors
Updating procedure is sequential importance
sampling with re-sampling
( ) ~ { , | 1,..., }
i i
t t t t
Bel x S x w i n = < > =
i
t
w
Example 2: Particle Filter
Step 0: initialization
Each particle has the
same weight
Step 1: updating
weights. Weights are
proportional to p(z|x)
Example 2: Particle Filter
Particles are more concentrated in
the region where the person is more
likely to be
Step 3: updating
weights. Weights are
proportional to p(z|x)
Step 4: predicting.
Predict the new
locations of particles.
Step 2: predicting.
Predict the new
locations of particles.
Compare Particle Filter
with Bayes Filter with Known Distribution
Example 1
Example 2
Example 1
Example 2
Predicting
Updating
Particle Filters
) | (
) (
) ( ) | (
) ( ) | ( ) (
x z p
x Bel
x Bel x z p
w
x Bel x z p x Bel
o
o
o
=

Sensor Information: Importance Sampling

}

' d ) ' ( ) ' | ( ) ( , x x Bel x u x p x Bel

Robot Motion
) | (
) (
) ( ) | (
) ( ) | ( ) (
x z p
x Bel
x Bel x z p
w
x Bel x z p x Bel
o
o
o
=

Sensor Information: Importance Sampling

Robot Motion
}

' d ) ' ( ) ' | ( ) ( , x x Bel x u x p x Bel

Tracking in 1D: the blue trajectory is the target.
The best of10 particles is in red.
Matlab code: truex is a vector of 100 positions to be tracked.
Application Examples
Robot localization
Robot mapping
Visual Tracking
e.g. human motion (body parts)
Prediction of (financial) time series
e.g. mapping gold price to stock price

Target recognition from single or multiple images
Guidance of missiles
Contour grouping

Nice video demos:
http://www.cs.washington.edu/ai/Mobile_Robotics/mcl/

2nd Book Advert
Statistical Pattern Recognition
Andrew Webb, DERA
ISBN 0340741643,
Paperback: 1999: 29.99
Butterworth Heinemann


Contents:
Introduction to SPR, Estimation, Density estimation, Linear
discriminant analysis, Nonlinear discriminant analysis - neural
networks, Nonlinear discriminant analysis - statistical methods,
Classification trees, Feature selction and extraction, Clustering,
Additional topics, Measures of dissimilarity, Parameter estimation,
Linear algebra, Data, Probability theory.
Homework
Implement all three particle filter algorithms

SIS Particle Filter Algorithm (p. 27)
Basic SIR Particle Filter algorithm (p. 39,40)
Generic SIR Particle Filter algorithm (p. 42)

and evaluate their performance on a problem of your
choice.
Groups of two are allowed.
Submit a report and a ready to run Matlab code (with a
script and the data).
Present a report to the class.