Tracking Motion: Stanford CS223B Computer Vision, Winter 2007

Stanford CS223B Computer Vision, Winter 2007
Lecture 12
Tracking Motion
Professors Sebastian Thrun and Jana Koeck
CAs: Vaibhav Vaish and David Stavens

Sebastian Thrun and Jana Koeck CS223B Computer Vision, Winter 2007
Overview
The Tracking Problem
Bayes Filters
Particle Filters
Kalman Filters
Using Kalman Filters

Can we estimate the position of the object?
Can we estimate its velocity?
Can we predict future positions?
Image 4 Image 1 Image 2 Image 3
Given Sequence of Images
Find center of moving object
Camera might be moving or stationary

We assume: We can find object in individual
images.
The Problem: Track across multiple images.

Is a fundamental problem in computer vision
Methods
Bayes Filter
Particle Filter
Uncented
Kalman Filter
Kalman Filter
Extended
Kalman Filter
Further Reading
Example: Moving Object
Kalman Filter Tracking
Particle Filter Tracking
Mixture of KF / PF (Unscented PF)
Overview
Bayes Filters
Particle Filters
Kalman Filters

Slow
Down!
Example of Bayesian Inference
?
Environment prior
p(staircase) = 0.1
Bayesian inference
p(staircase | image)
p(image | staircasse) p(staircase)
p(im | stair) p(stair) + p(im | no stair) p(no stair)
= 0.7 - 0.1 / (0.7 - 0.1 + 0.2 - 0.9) = 0.28
Sensor model
p(image | staircase) = 0.7
p(image | no staircase) = 0.2
p(staircase)
= 0.28
Cost model
cost(fast walk | staircase) = $1,000
cost(fast walk | no staircase) = $0
cost(slow+sense) = $1
Decision Theory
E[cost(fast walk)] = $1,000 - 0.28 = $280
E[cost(slow+sense)] = $1
=
Bayes Filter Definition
Environment state x
t
Measurement z
t

Can we calculate p(x
t
| z
1
, z
2
, , z
t
) ?
Bayes Filters Illustrated
Bayes Filters: Essential Steps
Belief: Bel(x
t
)

Measurement update: Bel(x
t
) Bel(x
t
) p(z
t
|x
t
)
Time update: Bel(x
t+1
) Bel(x
t
) p(x
t+1
|u
t
,x
t
)

x = state
t = time
z = observation
u = action
q = constant
Bayes Filters
) , | ( ) (
0 0 t t t t
u z x p x Bel

=
) , , , | ( ) , , , , | (
0 1 1 0 1 1
z z u x p z z u x z p
t t t t t t t

= q
) , , , | ( ) | (
0 1 1
z z u x p x z p
t t t t t

= q
1 0 1 1 0 1 1
) , , | ( ) , , , | ( ) | (

}
=
t t t t t t t t
dx z u x p z u x x p x z p q
1 1 1 1
) ( ) , | ( ) | (

}
=
t t t t t t t
dx x Bel u x x p x z p q
) , , , , | (
0 1 1
u z u z x p
t t t t

=
Markov
Bayes
Markov
1 0 2 1 1 1 1
) , , | ( ) , | ( ) | (

}
=
t t t t t t t t t
dx u u z x p u x x p x z p q
Bayes Filters
x = state
t = time
z = observation
u = action
1 1 1 1
) ( ) , | ( ) | ( ) (

}
=
t t t t t t t t
dx x Bel u x x p x z p x Bel q
Bayes Filters Illustrated
Bayes Filters
Initial Estimate of State
Iterate
Receive measurement, update your belief (uncertainty
shrinks)
Predict, update your belief (uncertainty grows)

Methods
Bayes Filter
Particle Filter
Uncented
Kalman Filter
Kalman Filter
Extended
Kalman Filter
Overview
Bayes Filters
Particle Filters
Kalman Filters

Particle Filters: Basic Idea
) (x p
t
x
) | ( ) (
... 1 t t t t
z x p X x p ~ e
(equality for ) | n
set of n particles X
t

Particle Filter Explained
1 1 ... 1 1 ... 1 1 1 ... 1 ... 1
) , | ( ) , | ( ) | ( ) , | (

}
=
t t t t t t t t t t t t
dx u z x p x u x p x z p u z x p q
Basic Particle Filter Algorithm
) , | ( ) (
... 1 ... 1 t t t t t
u z x p X x p ~ e
Initialization:
X
0
n particles x
0
[i]
~ p(x
0
)
particleFilters(X
t 1
){
for i=1 to n
x
t
[i]
~ p(x
t

| x
t 1
[i]
) (prediction)
w
t
[i]
= p(z
t

| x
t
[i]
) (importance weights)
endfor
for i=1 to n
include x
t
[i]
in X
t
with probability w
t
[i]
(resampling)

}
Weight samples: w = f / g
Importance Sampling
Particle Filter
By Frank Dellaert
Case Study:
Track moving objects from Helicopter
1. Harris Corners
2. Optical Flow (with clustering)
3. Motion likelihood function
4. Particle Filter
5. Centroid Extraction
David Stavens, Andrew Lookingbill, David Lieb, CS223b Winter 2004
1. Harris Corner Extraction
2. Optical Flow + Motion Detection
3. Motion Likelihood Function
4. Particle Filters
5. Extract Centroid
More Particle Filter Tracking
Some Robotics Examples
Tracking Hands, People
Mobile Robot localization
People localization
Car localization
Mapping
Examples Particle Filter
Siu Chi Chan McGill University

Another Example
Mike Isard and Andrew Blake

Tracking Fast moving Objects

K. Toyama, A.Blake
Particle Filters: Illustration
With: Wolfram Burgard, Dieter Fox, Frank Dellaert

) , | ( ) (
... 1 ... 1 t t t t t
u z x p X x p ~ e
Particle Filters (1)
Particle Filters (2)

Particles = Robustness
Tracking People from Moving Platform

- robot location (particles)
- people location (particles)
- laser measurements (wall)

With Michael Montemerlo
Tracking People from Moving Platform
With Michael Montemerlo

- robot location (particles)
- people location (particles)
- laser measurements (wall)

Particle Filters for Tracking Cars
With Anya Petrovskaya
Mapping Environments (SFM)
With Dirk Haehnel
Overview
Bayes Filters
Particle Filters
Kalman Filters

Tracking with KFs: Gaussians!
update initial estimte
x
y
x
y
prediction
x
y
measurement
x
y
Kalman Filters
) , ( ~ ) ( E N x p
)
`
E

) ( ) (
2
1
exp ) (
1
x x x p
T
Kalman Filters
prior
Measurement
evidence
posterior
) ( ) | ( ) | ( x p x z p z x p
dx x p x x p x p ) ( ) | ' ( ) ' (
}
=
A Quiz
prior
Measurement
evidence
) ( ) | ( ) | ( x p x z p z x p
posterior?
Gaussians
2
2
) (
2
1
2
2
1
) (
: ) , ( ~ ) (
o
o t
o
=
x
e x p
N x p
-o o

Univariate
) ( ) (
2
1
2 / 1
2 /
1
) 2 (
1
) (
: ) ( ~ ) (
x x
x
x

=
t
e p
, p
d
t

Multivariate
) , ( ~
) , ( ~
2 2
2
o
o
a b a N Y
b aX Y
N X
+
)
`
+ =
Properties of Univariate Gaussians
|
|
.
|
\
|
+ +
+
+

)
`
2
2
2
1
2
2
2
2
1
2
1
1
2
2
2
1
2
2
2 1
2
2 2 2
2
1 1 1
1
, ~ ) ( ) (
) , ( ~
) , ( ~
o o
o o
o
o o
o
o
o
N X p X p
N X
N X
Measurement Update Derived
( ) ( )
)
`
=
)
`
2
2
2
2
2
1
2
1
2 1
2
2 2 2
2
1 1 1
2
1
2
1
exp const. ) ( ) (
) , ( ~
) , ( ~
o
o
o x x
X p X p
N X
N X
( ) ( )
) new (for 0
2
1
2
1
2
2
2
2
1
1
2
2
2
2
2
1
2
1
o

o

o
=
)
`
c
c x x x x
x
( ) ( ) 0
2
1 2
2
2 1
= + o o
2
1 2
2
2 1
2
2
2
1
) ( o o o o + = +
2
2
2
1
2
1 2
2
2 1
o o
o o
+
+
=
( ) ( )
2
2
2
1
2
2
2
2
2
1
2
1
2
2
2
1
2
1

+ =
)
`
c
c
o o
o
o
x x
x
2
2
2
1
1

+
=
o o
o

We stay in the Gaussian world as long as we start with Gaussians
and perform only linear transformations.
) , ( ~
) , ( ~
T
A A B A N Y
B AX Y
N X
E +
)
`
+ =
E
Properties Multivariate Gaussians

Essentially the same as in the 1-D case, but with more general notation
( )
1 1
2
1
1 2 1
1
2 1 1 2
1
2 1 2 1
2 2 2
1 1 1
) ( , ) ( ) ( ~ ) ( ) (
) , ( ~
) , ( ~

E + E E E + E + E E + E
)
`
E
E

N X p X p
N X
N X
Linear Kalman Filter
t t t t t t
u B x A x c + + =
1
t t t t
x C z o + =
Estimates the state x of a discrete-time
controlled process that is governed by the
linear stochastic difference equation
with a measurement
Components of a Kalman Filter
t
c
Matrix (n n) that describes how the state
evolves from t to t+1 without controls or
noise.
t
A
Matrix (n i) that describes how the control
u
t
changes the state from t to t+1.
t
B
Matrix (k n) that describes how to map the
state x
t
to an observation z
t
.
t
C
t
o
Random variables representing the process
and measurement noise that are assumed to
be independent and normally distributed
with covariance R
t
and Q
t
respectively.
Kalman Filter Algorithm
1. Algorithm Kalman_filter(
t-1
,

E
t-1
, u
t
, z
t
):

2. Prediction:
3.
4.

5. Correction:
6.
7.
8.
9. Return
t
,

E
t

t t t t t
u B A + =
1

t
T
t t t
t R A A + E = E
1
t
t t t
C K I E = E ) (
1
) (

+ E E =
t
T
t
t
t
T
t
t
t
Q C C C K
) (
t t t t t t
C z K + =
Kalman Filter Updates in 1D
old belief
measurement
belief
new belief
belief
1
) ( with
) (
) (
) (

+ E E =
E = E
+ =
=
t
T
t
t
t
T
t
t
t
t
t t t
t t t t t t
t
Q C C C K
C K I
C z K
x bel

2
,
2
2
2 2
with
) 1 (
) (
) (
t obs t
t
t
t t t
t t t t t
t
K
K
z K
x bel
o o
o
o o

+
=
=
+ =
=
old belief
new belief
+ E = E
+ =
=
t
T
t t t
t
t t t t t
t
R A A
u B A
x bel
1
1
) (

+ =
+ =
=

2
,
2 2 2
1
) (
t act t t t
t t t t t
t
a
u b a
x bel
o o o

old belief
new belief new belief
newest belief
Kalman Filter Updates
belief
latest belief
belief
measurement
belief
The Prediction-Correction-Cycle
+ E = E
+ =
=
t
T
t t t
t
t t t t t
t
R A A
u B A
x bel
1
1
) (

+ =
+ =
=

2
,
2 2 2
1
) (
t act t t t
t t t t t
t
a
u b a
x bel
o o o

Prediction
1
) ( ,
) (
) (
) (

+ E E =
E = E
+ =
=
t
T
t
t
t
T
t
t
t
t
t t t
t t t t t t
t
Q C C C K
C K I
C z K
x bel

2
,
2
2
2 2
,
) 1 (
) (
) (
t obs t
t
t
t t t
t t t t t
t
K
K
z K
x bel
o o
o
o o

+
=
=
+ =
=
Correction
1
) ( ,
) (
) (
) (

+ E E =
E = E
+ =
=
t
T
t
t
t
T
t
t
t
t
t t t
t t t t t t
t
Q C C C K
C K I
C z K
x bel

2
,
2
2
2 2
,
) 1 (
) (
) (
t obs t
t
t
t t t
t t t t t
t
K
K
z K
x bel
o o
o
o o

+
=
=
+ =
=
+ E = E
+ =
=
t
T
t t t
t
t t t t t
t
R A A
u B A
x bel
1
1
) (

+ =
+ =
=

2
,
2 2 2
1
) (
t act t t t
t t t t t
t
a
u b a
x bel
o o o

Correction
Prediction
Kalman Filter Summary
Highly efficient: Polynomial in measurement
dimensionality k and state dimensionality n:
O(k
2.376
+ n
2
)

Optimal for linear Gaussian systems!

Most robotics systems are nonlinear!
Overview
Bayes Filters
Particle Filters
Kalman Filters

Lets Apply KFs to Tracking Problem
Image 4 Image 1 Image 2 Image 3
Kalman Filter with 2-Dim Linear Model
Linear Change (Motion)

Linear Measurement Model
) , 0 ( Q N D Cx z + + =
|
|
.
|
\
|
=
2
2
0
0
o
o
Q
|
|
.
|
\
|
=
0
0
D
|
|
.
|
\
|
=
1 0
0 1
C
|
|
.
|
\
|
=
0
0
B
|
|
.
|
\
|
=
1 0
0 1
A
) , 0 ( ' R N B Ax x + + =
|
|
.
|
\
|
=
2
2
0
0
R
Kalman Filter Algorithm
1. Algorithm Kalman_filter(
t-1
,

E
t-1
, u
t
, z
t
):

2. Prediction:
3.
4.

5. Correction:
6.
7.
8.
9. Return
t
,

E
t

t t t t t
u B A + =
1

t
T
t t t
t R A A + E = E
1
1
) (

+ E E =
t
T
t
t
t
T
t
t
t
Q C C C K
) ( D C z K
t t t t t t
+ =
t
t t t
C K I E = E ) (
|
|
.
|
\
|
=
2
2
0
0
o
o
Q
|
|
.
|
\
|
=
0
0
D
|
|
.
|
\
|
=
1 0
0 1
C
|
|
.
|
\
|
=
0
0
B
|
|
.
|
\
|
=
1 0
0 1
A
|
|
.
|
\
|
=
2
2
0
0
R
Kalman Filter in Detail
Measurements
Change
Prediction
Measurement Update

) , 0 ( R N x z + =
) , 0 ( ' Q N x x + =
( )
) ' ( ' ' ' ' '
' ' '
1
1
1 1
E + =
+ E = E

z R
R
= + E = E ' ' Q
update initial position
x
y
x
y
prediction
x
y
measurement
x
y
Can We Do Better?
Kalman, Better!
initial position prediction measurement
x
x
x
x
x
x
next prediciton
x
x
update
x
x
We Can Estimate Velocity!
past measurements
prediction
Kalman Filter For 2D Tracking
Linear Measurement model (now with 4 state variables)

Linear Change

|
|
|
|
|
.
|
\
|
=
2
2
0 0 0
0 0 0
0 0 0 0
0 0 0 0
q
q
Q
|
|
.
|
\
|
=
2
2
0
0
r
r
R ) , 0 (
0 0 1 0
0 0 0 1
R N
y
x
y
x
y
x
obs
obs
+
|
|
|
|
|
.
|
\
|
|
|
.
|
\
|
=
|
|
.
|
\
|
) , 0 (
1 0 0 0
0 1 0 0
0 1 0
0 0 1
'
'
'
'
Q N
y
x
y
x
t
t
y
x
y
x
+
|
|
|
|
|
.
|
\
|
|
|
|
|
|
.
|
\
|
A
A
=
|
|
|
|
|
.
|
\
|
Putting It Together Again
Measurements

Change

Prediction

Measurement Update

) , 0 (
0 0 1 0
0 0 0 1
R N x z +
|
|
.
|
\
|
=

) , 0 (
1 0 0 0
0 1 0 0
0 1 0
0 0 1
' Q N x
t
t
x +
|
|
|
|
|
.
|
\
|
A
A
=

( )
) ( ' '
'
1
1
1 1
A z R A
A R A
T
T
E + =
+ E = E

C
CQC
T
=
+ E = E
'
'
Summary Kalman Filter
Estimates state of a system
Position
Velocity
Many other continuous state variables possible
KF maintains
Mean vector for the state
Covariance matrix of state uncertainty
Implements
Time update = prediction
Measurement update
Standard Kalman filter is linear-Gaussian
Linear system dynamics, linear sensor model
Additive Gaussian noise (independent)
Nonlinear extensions: extended KF, unscented KF: linearize
More info:
CS226
Probabilistic Robotics (Thrun/Burgard/Fox, MIT Press)
Summary Kalman Filter
Estimates state of a system
Position
Velocity
Many other continuous state variables possible
KF maintains
Mean vector for the state
Covariance matrix of state uncertainty
Implements
Time update = prediction
Measurement update
Standard Kalman filter is linear-Gaussian
Linear system dynamics, linear sensor model
Additive Gaussian noise (independent)
Nonlinear extensions: extended KF, unscented KF: linearize
More info:
CS226
Probabilistic Robotics (Thrun/Burgard/Fox, MIT Press)
Particle
and discrete
set of particles
(example states)
= predictive sampling
= resampling, importance weights
fully nonlinear
Summary
Bayes Filters
Particle Filters
Kalman Filters

Tracking Motion: Stanford CS223B Computer Vision, Winter 2007

Diunggah oleh

Informasi Dokumen

Judul Asli

Hak Cipta

Format Tersedia

Bagikan dokumen Ini

Bagikan atau Tanam Dokumen

Opsi Berbagi

Apakah menurut Anda dokumen ini bermanfaat?

Apakah konten ini tidak pantas?

Hak Cipta:

Format Tersedia

Tracking Motion: Stanford CS223B Computer Vision, Winter 2007

Diunggah oleh

Hak Cipta:

Format Tersedia

Stanford CS223B Computer Vision, Winter 2007

Properties Multivariate Gaussians

Anda mungkin juga menyukai