3852
10, OCTOBER 2013
Monocular Human Motion Tracking by Using
DE-MC Particle Filter
Ming Du, Xiaoming Nan, and Ling Guan, Fellow, IEEE
Abstract— Tracking human motion from monocular video
sequences has attracted significantly increased interests in recent
years. A key to accomplishing this task is to efficiently explore
a high-dimensional state space. However, the traditional particle
filter method and many of its variants have not been able to meet
expectations as they lack a strategy to do efficiently sampling
or stochastic search. We present a novel approach, namely
differential evolution - Markov chain (DE-MC) particle filtering.
By taking the advantage of the DE-MC algorithm’s ability to
approximate complicated distributions, substantial improvement
can be made to the traditional structure of the particle filter. As
a result, an efficient stochastic search can be performed to locate
the modes of likelihoods. Furthermore, we apply the proposed
algorithm to solve the 3D articulated model-based human motion
tracking problem. A reliable image likelihood function is built
for visual tracker design. Based on the proposed DE-MC particle
filter and the image likelihood function, we perform a variety of
monocular human motion tracking experiments. Experimental
results, including the comparison with the performance of other
particle filtering methods demonstrate the reliable tracking
performance of the proposed approach.
Index Terms— Articulated human motion tracking, importance
sampling, particle filtering, DE-MC.
I. I NTRODUCTION
H UMAN motion tracking plays a critical role in human
movement understanding and analysis in that it quanti-
fies human’s motion and is of interest for many applications
including security and surveillance system, video indexing
and retrieval, sports training, movie special effect production
etc. For human being, the vision system can locate a person
and track body parts easily, accurately and quickly, but the
same task remains arduous for computers. Many challenges
have been impeding the progress of the research in this field.
First, a large number of degrees of freedom (DOF) will be
introduced by the articulated human body model in practical
human motion tracking, and thus the tracking task need to
search for the optimal position in the high dimensional state
space. Second, the crucial factors to a tracking problem are
how to allocate the samples and how to guide their movement
Manuscript received March 6, 2012; revised July 24, 2012, November
19, 2012, and April 14, 2013; accepted May 1, 2013. Date of publication
May 14, 2013; date of current version August 28, 2013. The associate editor
coordinating the review of this manuscript and approving it for publication
was Prof. Jenq-Neng Hwang.
M. Du is with the Department of Electrical and Computer Engineer-
ing, University of Maryland, College Park, MD 20742 USA (e-mail:
mingdu@umd.edu).
X. Nan and L. Guan are with the Department of Electrical and Computer
Engineering, Ryerson University, Toronto, ON M5B 1Z2, Canada (e-mail:
xnan@ee.ryerson.ca; lguan@ee.ryerson.ca).
Color versions of one or more of the figures in this paper are available
online at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/TIP.2013.2263146
1057-7149 © 2013 IEEE
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 22, NO.
in the state space. In the sense of importance sampling, if the
proposal distribution is biased and no corrections are made, the
tracker will fail quickly. In addition, the relationship between
human motion and image appearance is too complex to be
formulated mathematically. Due to the motion blur, depth
ambiguity and self occlusion, the extracted low-level visual
features cannot describe the human motion effectively and
robustly. Although a variety of approaches have been proposed
in recent years, the high dimensional human motion tracking
problem is still inherently difficult.
In this paper, we propose a novel tracking algorithm, namely
the Differential Evolution - Markov Chain (DE-MC) particle
filter. It is based on the particle filter framework but makes
substantial changes to the core structure of the CONDENSA-
TION algorithm. Guided by the DE-MC algorithm, samples
can make progressive move to the important regions of the
conditional distribution as defined by the image likelihood.
Furthermore, we apply the DE-MC particle filter to the
3D articulated model-based human motion tracking problem.
A 3D articulated human body model is proposed in this paper,
and we also design a robust multi-cue based measurement
function which describes the resemblance between hypothesis
and image observations. Based on the proposed algorithm
and multi measurements, we perform a variety of monocular
human motion tracking experiments. The experimental results
demonstrate the reliable tracking performance of the proposed
tracking algorithm.
Our major contributions in this paper are listed as follows:
1) Since the traditional CONDENSATION algorithm ignores
the most recent observation and thus produces unreliable human
motion tracking results, we introduce the DE-MC algorithm from
statistics and optimization theory to address this problem. The
proposed DE-MC particle filter incorporates both the advantage
of the Differential Evolution algorithm in global optimization
and the ability of the Markov Chain Monte Carlo in reasonably
sampling a high dimensional state space. 2) We apply the
proposed DE-MC particle filter to address the 3D articulated
model-based human motion tracking problem. Specifically, we
present our 3D articulated human body model and fuse silhouette,
color and boundary information to build a robust multi-cue
measurement function. Among them, the boundary information
represented by the Fourier Descriptors (FD) sets up a novel and
effective connection between the estimated model parameters
and the image likelihoods. Based on these novelties, our human
motion tracking system achieves great improvement over the
traditional particle filtering methods.
The remainder of this paper is organized as follows.
Section II reviews the related work in the field of human
DU et al.: MONOCULAR HUMAN MOTION TRACKING BY USING DE-MC PARTICLE FILTER
motion tracking. Section III firstly introduces the background
of Markov Chain Monte Carlo (MCMC), Differential Evolu-
tion (DE) algorithm and DE-MC algorithm, and then proposes
DE-MC particle filter algorithm. The 3D articulated human
body model and image likelihoods are built in Section IV.
Experimental results are shown in Section V and the conclu-
sions are drawn in Section VI.
II. R ELATED W ORK
The core of the tracking algorithm is its mechanism of
searching for configurations which interpret observations best.
For this reason, a rich body of technical literature was devoted
to designing an efficient sampling and search strategy. Basi-
cally the two tasks are towards the same goal and are closely
linked: a good sampling strategy will substantially increase
the efficiency of searching, and a proper search strategy
will increase the possibility of finding extrema. Most human
motion trackers are based on particle filters. Application of
particle filters (or Sequential Monte Carlo Sampling) in the
computer vision society can be traced back to the CONDEN-
SATION algorithm [1]. Though paving a path to solving many
visual tracking problems, the CONDENSATION algorithm
and most of its variants turn out to be inadequate when
the dimensionality of the state space increases by a certain
degree. In such a space, the samples are distributed extremely
sparsely. Unless enough of them fall into the neighboring
region of a solution, there is no guarantee of reaching this
solution. Unfortunately, the simplification which differentiates
the CONDENSATION (or generic particle filter) from the
Sequential Monte Carlo sampling (SMC) makes a nontrivial
move away from the objective.
A possible compensation is to reduce the dimensionality of
the searching space and hence make the samples relatively
more dense in their distribution. The well-known dimen-
sionality reduction algorithms such as Principle Component
Analysis (PCA) and Independent Component Analysis (ICA)
sometimes are very useful. In [2], for example, PCA is used
to explore the 3D head motion and pose estimation problem.
However, the linear nature of the PCA and ICA usually
make them inadequate to handle the complicated relations
between joint angles. There are many non-linear dimensional
reduction techniques available, but some of them, such as
Isomap, Laplacian Eigenmaps, Locally Linear Embedding, are
non-invertible. The shortcoming prevents them from being a
qualified candidate in visual tracking applications since we
must return to the state space to retrieve the tracking result.
In addition, non-linear dimensionality reduction methods with
inverse mapping ability such as the Locally Linear Coor-
dination (LLC) [3]–[5] algorithm and the Gaussian Process
Latent Variable Model (GPLVM) [6]–[10] have been used
for tracking human motion. In [5], the LLC preserves the
clustering behavior of similar high-dimensional data points and
separate the different clusters in the global coordinate system,
and the models learned from the LLC is then used for tracking.
Authors in [10] employ GPLVM for pose synthesis given
kinematics constraints. Some researchers also try to partition
the search space by looking for the conditional dependencies
3853
between different state variables, as has been done through
using the Rao-Blackwellised Particle Filter in [11]. Generally
speaking, most of these work rely on training sets and are
successful when the types of motion to be tracked are close
to those in the training set.
From a different perspective, many researchers focus their
attentions on refining dynamical models. Given the posterior
history, they try to accurately predict the region that covers the
solution at current time step by deriving process noise from
an uncertainty description matrix [12], [13], by building an
adaptive velocity model [14]–[16], and especially, by learning
motion templates. Authors in [17]–[19] present a real-time
full body tracker. In their work, the parameter space is first
partitioned into Gaussian clusters each representing an elemen-
tary motion. A prior dynamics model is then learnt from this
low dimensional representation by using an unsupervised EM
clustering algorithm. The temporal dependencies of high-level
behaviors are captured by a variable length Markov model
(VLMM). By using the learnt dynamics model, the propaga-
tion of candidate poses is biased in the low dimensional space.
Authors in [17]–[19] apply their 3D human motion tracker
to multi-view videos. They propose a hierarchical algorithm
to merge silhouette extraction and volumetric reconstruction
together, which can efficiently evaluate candidate poses against
evidences captured from multiple views. The hierarchical
volumetric reconstruction algorithm can effectively fuse infor-
mation from different views and resolve spatial ambiguities.
A rich body of literature [20]–[23] has discussed how to utilize
motion templates to guide the tracking even though it can only
be applied to limited motion types. All of these efforts rely
heavily on the knowledge about dynamical prior. They usually
make too rigid assumptions to be applied to general tracking
scenes.
Because of the aforementioned limitations of those schemes,
we realize that the improvement should be made on the
core of particle filtering, for instance, the sampling strat-
egy. Importance sampling is first introduced in the form
of ICONDENSATION [24], [25] in which the samples are
not only drawn according to the dynamical priors but also
an auxiliary importance function. Poon and Fleet’s Hybrid
Monte Carlo Filtering (HMCF) [26] bears some similarity
to our work in that it uses MCMC to generate samples.
But unlike our work, the HMCF follows the gradient of the
posterior distribution to find promising samples. It requires
analytical form of image likelihood gradient, which is not
available for most image cues. Therefore it cannot maximize
the use of abundant visual information contained in a video
for tracking. Sminchisescu and Triggs [27] use kinematic jump
sampling to handle the forwards/backwards ambiguities in the
estimation of limb’s pose. While this sampling strategy targets
for a specific problem structure, our work is concerned with
more general case of sampling. In [28], Sminchisescu and
Triggs develop a proposal density based on uncertainty of
local parameter estimation. Along the eigen-directions which
account for the most uncertain parameters (usually the ones
that describe the movement in depth), sampling covariances
are inflated by a large scale. So the generated samples can
focus on the modes that cause ambiguities. The unscented
3854 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 22, NO. 10, OCTOBER 2013

particle filter (UPF) [29]–[31] draws samples from a proposal state at each time step according to a posterior distribution
distribution which is determined by the calculation result of p(X 0:k |Y1:k ), where X 0:k = {X 0 , X 1 , . . . , X k } are the state
the unscented Kalman filter (UKF). Alongside approaches that vectors up to and including time k and Y1:k = {Y1 , . . . , Yk }
try to “generate good samples”, there are also research projects are the observations in the same period of time. By Bayesian
devoted to “turning bad samples into good ones”, i.e. driving Inference [1]:
the existing biased samples to a global or quasi-global extrema.
p(X k |Y1:k ) = λk P(Yk |X k ) p(X k |Y1:k−1 )
The Kernel Particle Filter (KPF) [32]–[34] is based on kernel
density estimation and moves the samples towards the gradient = λkP(Yk |X k ) ·
ascending direction of the posterior distribution density. Wu p(X k |X 0:k−1 , Y1:k−1 )
et al. [35] model the 2D tracking problem using a dynamic
Markov Random Field, in which the node potential at each × p(X 0:k−1 |Y1:k−1 )d X 0:k−1

time instance is determined by dynamical model and edge
= λk P(Yk |X k ) p(X k |X k−1 )
potentials are location constraints for body part pairs. They use
the mean field algorithm for inference. In [25], [36], [37], the × p(X k−1 |Y1:k−1 )d X k−1 , (1)
simulating annealing algorithm and the genetic algorithm (GA)
together constitute the foundation of the annealed particle filter where λk is a normalization constant that is independent with
(APF). In this case samples are pushed gradually toward the Xk.
global maximum of a weighting function by progressively By applying the Monte Carlo principle, the continuous
adjusting the sensitivity of the weighting function. In [38], distribution can be represented by discrete samples through
the articulated human body motion is decomposed into limb particle filtering. And since the direct sampling is difficult for
motions. The coupling between limb motions are captured by the complicated posterior distribution, importance sampling
Sequential Ancestral Simulation (SAS) of Bayes Networks principle is adopted. We summarize a typical particle filter
and by Markov Random Field model in the sample weights. step as follows:
(i)
Although this work relies on cyclic motion prior and thus not At time step k, starting with a sample set: {X k−1 ,
(i)
applicable to general motions, an iterative sampling is found ω(X k−1 )}i=1N
:
ˆ N
to be able to improve the quality of sampling, which resembles 1) Selection: Select a new set of samples { X k(i) }i=1 from
the idea of the APF. (i) N (i)
{X k−1 }i=1 according to ω(X k−1 ). The samples with a
The preceding works have demonstrated that the decisive
larger weight should be selected with a higher probabil-
factors to a high-dimensional visual tracking problem are how
ity. The detailed selection strategy can be found in [1]
to allocate the samples and how to guide their movement in
and [39].
the state space. Motivated by this perspective, we present a
2) Prediction: Sample from the proposal function,
new algorithm which makes substantial changes to the core
ˆ
structure of the CONDENSATION algorithm. An information {X k(i) } ∼ g(X k(i) | X k(i) , Yk ), i = 1, 2, . . . , N. (2)
exchange scheme is built between the sampling and weighting
steps to guide the moves of samples in the state space. This 3) Measurement: Evaluate the weight for each sample,
adjustment also has the effect of stochastic global optimiza- ˆ
tion. In addition, the proposed algorithm does not require p(Yk |X k(i) ) p(X k(i) | X k(i) )
ω(X k(i) ) = , i = 1, 2, . . . , N,
any training for dynamical priors. As our work is based on (i) ˆ(i)
g(X k | X k , Yk )
the DE-MC, a method originally developed for approximating (3)
target density functions in statistics, we name the proposed where the p(Yk |X k(i) ) is the image likelihood and
algorithm the DE-MC particle filter. We will demonstrate its ˆ
p(X k(i) | X k(i) ) is the dynamical model. Then normalize
effectiveness with application in monocular 3D human motion N (i)
tracking. The generality of the algorithm however, allows it to the weight so that i=1 ω(X k ) = 1.
be readily used in other visual tracking contexts. 4) Representation: Estimate the state at time step k as
X̃ k = argmax X (i) ω(X k(i) ), i = 1, . . . , N, (4)
III. DE-MC PARTICLE F ILTER k

or
In this section, we will first give a brief introduction to 
N
(i) (i)
the particle filter, the Markov Chain Monte Carlo (MCMC) X̃ k = E[X k ] = ω(X k )X k . (5)
algorithm, the Differential Evolution (DE) Algorithm and the i=1
DE-MC algorithm. Then we propose the DE-MC particle filter. Particle filters often suffer from the degeneracy problem,
referring to the cases in which all but a few particles have
A. Particle Filter negligible weights after some iterations. An indicator of the
In an articulated model-based human motion tracking prob- degree of degeneracy is the effective sample size, or survival
lem, joint angles together with global translation and rotation diagnostic [40]:
parameters constitute a state vector. This vector gives a com- 1
Neff =  .
N 
(6)
plete description for the pose of human. Therefore, the visual (i) 2
tracking problem can be formulated as recursively estimating i=1 ω(X k )
DU et al.: MONOCULAR HUMAN MOTION TRACKING BY USING DE-MC PARTICLE FILTER
A small effective sample size indicates severe degeneracy. In
this paper, we try to use a better proposal distribution to tackle
the degeneracy problem. The optimal proposal distribution is
given by
ˆ ˆ
g(X k(i) | X k(i) , Yk ) = p(X k(i) | X k(i) , Yk ). (7)
However, in practice it is very difficult to either obtain the
analytical form of the optimal proposal distribution or draw
samples from it. Generic sequential importance resampling
(SIR) particle filters use the dynamical prior as substitute for
the proposal distribution,
(i) ˆ(i) (i) ˆ(i)
g(X k | X k , Yk ) = p(X k | X k ). (8)
In this case, the sample weight in Equation (3) is nothing but
the image likelihood,
ω(X k(i) ) = p(Yk |X k(i) ). (9)
However, a state space of 20 or so dimensions appears to be so
vast that the dynamical model alone cannot project the samples
into the most probable locations of the solution. Indeed, the
sparse samples thus generated are almost certain to miss many
local or global modes. Therefore, to find a new strategy that
we can perform efficiently stochastic search in such a high-
dimensional space has become an urgent issue. This is our
motivation in developing the DE-MC particle filter.
B. Markov Chain Monte Carlo
A Markov Chain (MC) can be described by a transition
matrix T in which
Tmn = p(X k = Sn |X k−1 = Sm ), (10)
where Sn and Sm are two of the probable states. Regardless
of which initial state it starts, the Markov chain will always
reach a steady state distribution p(X) if the transition matrix T
possesses irreducibility and aperiodicity properties [41]. These
two properties guarantee a finite path from each state to every
other state with non-zero transition probability, which is the
so-called ergodicity.
The Markov Chain Monte Carlo (MCMC) algorithm takes
aim at constructing a MC which has the given target distri-
bution as its invariant distribution [41]. Normally we ensure
the stationarity by making the chain satisfy the reversibility
property,
p(X k )T (X k−1 |X k ) = p(X k−1 )T (X k |X k−1 ). (11)
The most frequently adopted MCMC method is the
Metropolis-Hasting (MH) algorithm [41]. According to this
algorithm the transition probability is given by
T (X k |X k−1 ) = α(X k−1 , X k )g(X k |X k−1 ), (12)
where g(X k |X k−1 ) is the proposal distribution we can directly
sample from and
 
p(X k )g(X k−1 |X k )
α(X k−1 , X k ) = min 1, (13)
p(X k−1 )g(X k |X k−1 )
is the acceptance rate.
3855
Now we verify that Equation (13) has the prop-
erty of reversibility. Firstly, we verify the case when
( p(X k )g(X k−1 |X k ))/( p(X k−1 )g(X k |X k−1 )) > 1. Given this
inequality condition, we have:
α(X k−1 , X k ) = 1
and
p(X k−1 )g(X k |X k−1 )
α(X k , X k−1 ) = ,
p(X k )g(X k−1 |X k )
so:
p(X k−1 )T (X k |X k−1 ) = p(X k−1 )g(X k |X k−1 )
p(X k−1 )g(X k |X k−1 )
= p(X k )
p(X k )g(X k−1 |X k )
×g(X k−1 |X k )
= p(X k )T (X k−1 |X k ).
The cases of ( p(X k )g(X k−1 |X k ))/( p(X k−1 )g(X k |X k−1 )) < 1
and ( p(X k )g(X k−1 |X k ))/( p(X k−1 )g(X k |X k−1 )) = 1 can be
proved the same way.
One condition that ensures the quality of convergence is
that g(X)/ p(X) > 0 everywhere, so g(X) is usually chosen
such that it is similar in shape to p(X), the target distribution.
g(X k |X k−1 ) determines how the state space is exploited. This
is especially important to a high-dimensional problem such as
human tracking. A popular choice is to generate new samples
from a symmetric random walker sampler, as the Metropolis
algorithm does. It means the sampling proposal is determined
only by the samples’ separation from X k−1 :
g(X k |X k−1 ) = g(|X k − X k−1 |). (14)
Thus the acceptance rate reduces to
 
p(X k )
α(X k−1 , X k ) = min 1, . (15)
p(X k−1 )
The calculations of the MH algorithm and the Metropolis algo-
rithm are especially convenient because they can be applied
to situations in which we cannot directly draw samples from
the target distribution, but know how to roughly evaluate their
values everywhere. This is precisely the case we encounter in
human motion tracking.
C. Differential Evolution Algorithm
The Differential Evolution algorithm (DE) is an algorithm
dealing with the parallel search for a global maximum through
high dimensional state space [42]. Similar to other evolution-
ary program methods such as the Genetic algorithm, it is also
based on evolution theory and the competition mechanism:
stronger members can more easily survive to the next gen-
eration so as to guarantee that the new generation is better
than the previous one as a whole. Compared with the Genetic
algorithm, the Differential Evolution algorithm is defined in
real parameter spaces instead of binary code parameter spaces.
So it is much simpler to implement. The DE algorithm can
explore non-isotropic structures such as ridges in the target
function because the vector differences are usually aligned
with the direction of the ridges.
3856
Assume that a complicated function f (E) is defined over a
D-dimensional state space ε. Assume also that we do not know
the analytical form of this function (or the analytical form is
too complicated for a gradient-based method to apply), but can
evaluate the value indirectly. We can use the DE algorithm
to search the global maximum with an initial population
E n,0 , n = 0, 1, . . . , N − 1. N is the number of populations.
The simplest version of the DE algorithm generates a new
generation of population in time step k + 1 according to
∗
E n,k+1 = E n,k + λ(Er1,k − Er2,k ), (16)
where r 1, r 2 are random integers drawn from [0, 1, . . . , N −1]
and mutually different, and λ > 0 controls the amplification
of differential variation. Its typical value is in the range of
[0.5, 1]. Whether a newly generated state vector will be
accepted is solely dependent on the value evaluated by the
∗
target function. In a global maximum search, if E n,k+1 yields
a larger function value than E n,k , the state will be updated;
otherwise it will be kept unchanged. Several variants of the
DE algorithm are available and the use of crossover operator
is optional [42].
D. Differential Evolution Markov Chain (DE-MC) Algorithm
By examining the characteristics of the MCMC and the DE
algorithm we find that they aid each other in searching for
an optimal solution. The acceptance rule in the DE part is
controlled by the MCMC acceptance mechanism, whilst the
step size and orientation of the random walk of the MCMC
part is produced by the DE algorithm. By constructing multiple
MCMCs in parallel, the state space is more efficiently explored
since the state vectors can be more reasonably distributed than
with a single chain. All these Markov chains can interact
with each other, sharing information with the aid of the
DE algorithm. Under the guidance of the DE algorithm, the
MCMCs will gradually concentrate on the important regions
of the target distribution without being trapped in local basins.
The DE-MC algorithm [43] is summarized as follows.
DE-MC Algorithm
1) Start with a target function f (E) and an initial pop-
ulation (E 0,0 , E 1,0 , . . . , E N−1,0 ), whose members are
D-dimensional vectors.
2) In the kth iteration, for each member of the population
E n,k−1 , n = 0, 1, . . . , N − 1, randomly choose two
integers r 1 and r 2 so that r 1 = r 2 = n.
3) Create a new member E n,k ∗ by
∗ bivariate distribution with respect to random variables X and Y
E n,k = E n,k−1 + λ(Er1,k−1 − Er2,k−1 ) + g. (17)
λ is the scalar parameter as used in Equation (16).
It can control the tradeoff between the jumping pace of
the samples and the degree of diffusion of the samples.
It is suggested that the value should be set around:
√ by 200 samples with the DE-MC algorithm after only 30
λ = 2.38/ 2D (18)
and g is drawn from a symmetric distribution such as
Gaussian or uniform distribution with small variance
compared to that of E.
∗ )/ f (E
4) Compute the ratio R = f (E n,k n,k−1 ).
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 22, NO. 10, OCTOBER 2013
Fig. 1. The DEMC algorithm simulation result: bimodal distribution (200
samples, 20 iterations).
5) Choose a number h from U (0, 1). If R > h, E n,k =
∗ ; otherwise E
E n,k n,k = E n,k−1 .
6) Repeat steps 2–5 for iteration k + 1 until a convergence
or a preset end point is reached.
Note that to make the DE-MC algorithm match our work,
here we are searching for a maximum instead of a minimum.
∗
Equation (17) satisfies the detailed balance condition if E n,k
is accepted:
E n,k−1 = E n,k + λ(Er2,k−1 − Er1,k−1 ) − g (19)
and if it is rejected, the value remains unchanged.
We run simulations to verify the performance of the DE-
MC algorithm. As it is difficult to visualize a space of
more than 3 dimensions, we only choose the target density
functions with one or two variables, which require 2D and
3D plot to display respectively. The first one is a bimodal
distribution p(x) ∝ 0.3e−0.2x + 0.7e−0.2(x−10) . The number
2 2
of samples is 200 and the value of λ is set to 1.19 according
to Equation (18). We start with a population of samples drawn
from uniform distribution on [−10, 20]. Fig. 1 shows the result
after 20 simulation iterations. The approximated distribution
is represented in the form of histogram and the ground-truth
density curve is overlayed. The second example is the Muller
potential surface as appeared in [44], [45], with the form

4
Ai eai (X −xi )
2 +b
i (X −x i )(Y −yi )+ci (Y −yi )
2
V (X, Y ) = . (20)
i=1
We set A = (−200, −100, −170, 15), a = (−1, −1, −6.5,
0.7), b = (0, 0, 11, 0.6), c = (−10, −10, −6.5, 0.7), x = (1,
0, −0.5, −1), y = (0, 0.5, 1.5, 1) and initially put all the
samples at the minimum (−0.5, 1.5). We can view it as a
and dependent on parameters A, a, b, c, x and y. It has been
shown in [44] that with the standard dynamic sampling the
samples are still trapped in the starting cost basin even after
6000 iterations, while from Fig. 2(b) we can see that the
important regions of target distribution are well approximated
iterations. Most samples finally occur around the three minima.
Here we do not compare our result with that obtained by the
hyperdynamicas importance sampling because that algorithm
attempts to attract the samples to saddle points instead of
extrema.
DU et al.: MONOCULAR HUMAN MOTION TRACKING BY USING DE-MC PARTICLE FILTER
Fig. 2. The DEMC algorithm simulation result: Muller potential surface
(200 samples, 30 iterations).
E. DE-MC Particle Filter
Based on the DE-MC algorithm, we propose a novel
sequential Monte Carlo sampling approach, namely the DE-
MC particle filter. The DE-MC particle filtering iteration at
time step k can be summarized as follows.
DE-MC Particle Filter Algorithm
Starting from the set of particles which are the filtering
(i) (i)
result of time step k − 1, {X k−1 , ω(X k−1 )}i=1
N .
ˆ(i) N
1) Selection: select a new set of samples { X k }i=1 from
(i) N (i)
{X k−1 }i=1 with the probability proportional to ω(X k−1 ).
2) Prediction and Measurement: Apply a constant veloc-
ity dynamical model to the samples:
ˆ result, samples will be concentrated around the modes of the
X k(i)− = X k(i) + Vk−1 , (21)
where Vk−1 is the velocity vector computed in time
(i)− N
step k − 1. The particle set {X k }i=1 then acts as
the initial population for a T -iteration DE-MC process-
ing. The processing follows the DE-MC algorithm we
listed in Section III-D. The fitness function is the
image likelihoods in the case of visual tracking. For
Equation (17), we choose g ∼ U (−cσ, cσ ) wherein
σ = [σ0 , σ1 , . . . , σ D−1 ]T is a vector with the elements
equal to standard deviations for the elements in X.
Normal distribution can be used here instead of uniform
distribution. c is a small number which can be flexibly
chosen. Also in the same equation, the value of λ is
determined by
2.38
λ = (1 − c) × √ . (22)
2D
At the end of this step, we take the output population as
(i) (i) N
the particle set of current time step: {X k , ω(X k )}i=1 .
3) Representation and Velocity Updating: Estimate the
state at time step k as
X k = argmax X (i) ω(X k(i) ), i = 1, . . . , N, (23)
k
and calculate the velocity vector of current time step
Vk = X k − X k−1 . (24)
We adopt a strategy inspired by [36] to help the filter adapt
to the changes of situations: to calculate the value of σ in
step 2 as proportional to the survival diagnostic as defined in
3857
Fig. 3. The structure comparison of the DE-MC particle filter and the generic
particle filter.
Equation (6). The step size of random jumping for current DE-
MC iteration is reduced if the survival rate of the last DE-MC
iteration is high and is inflated otherwise.
As previously mentioned, current observations are invisible
to a generic SIR particle filter prior to the measurement stage.
As such, there is a high risk that these sparse samples are not
placed near the modes of the conditional distribution as defined
by the likelihood. The DE-MC particle filter, on the other
hand, can iteratively use the current observation to adjust the
distribution of particles (See Fig. 3). The success of the DE-
MC particle filter relies on two facts. First, as an importance
sampling algorithm, MCMC guarantees that the samples are
in accordance with the target conditional distribution. As a
observation likelihood. Second, as a stochastic optimization
algorithm, the DE algorithm reduces the chance that the
samples are trapped in some of the local modes.
IV. 3D H UMAN B ODY M ODEL AND I MAGE L IKELIHOODS
In Section III, we present the proposed DE-MC particle
filter to perform an efficient stochastic search to locate the
modes of likelihoods. In this section, we will discuss some
details and application level issues of the proposed DE-MC
particle filter when it is used for tracking monocular human
motion video sequences.
A. 3D Articulated Human Body Model
Articulated human body models are consistent with the
natural mechanism of human motion. Therefore, we are able
to directly apply our knowledge about human motion to it.
The model usually has a hierarchical structure, so the motion
of a parent node will constrain that of its child or grandchild
nodes. This relationship is reflected by the rigid geometric
transformations between the local coordinate systems of the
body parts:
P  = R P + T, (25)
where the same point is represented in two different body part
coordinate systems by P = [x, y, z]T and P  = [x  , y  , z  ]T .
R is a 3 × 3 rotation matrix and T = [Tx , Ty , Tz ]T is the
translation vector.
The model we built has 14 body segments, and 21 DOFs
(Degrees of Freedom) are associated with them. The segments
3858
Fig. 4. The proposed 3D articulated human body model and its hierarchical
structure. Number of DOFs assigned to each joint is presented in the diagram.
LUA means Left-Upper-Arm, RLL means Right-Lower-Leg, LF means Left-
Foot etc.)
are represented by truncated cones or spheres, which provide
acceptable approximation to the real body parts and save much
computational cost at the same time. Geometric parameters
such as the radius and height of the truncated cones correspond
to anthropometric data of humans, so they are supposed to
vary with subjects but remain constant for the same video
sequence. We do not model the hands separately. Instead, we
regard them as a part of lower arms, because they do not
make a big difference in the observation model and modeling
them will introduce additional DOFs. For the same reason we
do not assign DOFs to the ankle joint. Joint limit constraints
are imposed to make the model kinematics consistent with
common sense. The articulated model and its hierarchical
structure are shown in Fig. 4.
In our human motion tracking application, the state vector
is defined as X = [T0 , θ0 , θ1 , . . . , θ13 ], where T0 and θ0 are
the global translation and rotation parameters for the root body
part - torso, and θi are the joint angle vectors of the remaining
(i)
body parts as shown in Fig. 4. Correspondingly, X k is a
hypothesis state vector or a sample generated by the DE-
MC particle filter. In the following, we are going to model
the conditional probability p(Yk |X k(i) ), which will be used to
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 22, NO. 10, OCTOBER 2013
Apparently, if the hypothesis silhouette and the ground-truth
silhouette totally fit, p1 (Yk |X k(i) ) = 1. Otherwise if they do
not overlap each other at all, p1 (Yk |X k(i) ) = 0.
2) Color: 2D image pixels are generated by points in the
real world through perspective projection. We make an inverse
projection from the image to the 3D model:
−1 −1
Pm = Mlm (X 0 , Mmw (Q −1 (Pi , E))), (27)
where Pm and Pi are the local coordinates of a human body
surface point and the image coordinates of its projection,
respectively; Q is the perspective projection; E is the extrinsic
camera parameter solved from calibration; Mmw and Mlm are
the rigid transformation which relates the model coordinates
to the world coordinates and relates the body segment local
coordinates to the model’s global coordinates, respectively;
and X 0 is the initial pose learned from initialization.
After the correspondences between model points and image
pixels have been built through Equation (27), we construct
a reference appearance model for visible image patches of
individual body segments in the form of 10 × 10 × 10 bins
normalized color histogram. This is done after initialization.
For the invisible body parts or the invisible side of a part,
we make justified assumptions such as left-right symmetry of
human appearance and uniform appearance distribution for the
surface of a body segment. At each time step k, the difference
of the reference histogram Hr and the hypothesis histogram
Hk(i) for hypothesis pose vector X k(i) is measured. A popular
way of doing so is to sum the Bhattacharyya distance of all
the individual body segment histograms [46], [47]:
(i)

14 
3
(i)
D(Hr , Hk ) = D(Hr , Hk,( j,m) ), (28)
m=1 j =1
where
f 
(i)

calculate the sample weights ω(X k ) in the DE-MC particle (i) 
D(Hr , Hk,( j,m)) = 1 − (i)
Hr,(n, j,m) Hk,(n, j,m) , (29)
filter. n=1

n is the bin index, j is the RGB channel index, m is the

B. Image Likelihoods body segment index and f is the total number of bins.
In this work we choose silhouette, color and boundary as Considering different contributions each body segment makes
image cues to define the image likelihood function. Consid- to the observations, we propose the assignment of different
ering the task scenario, an improved scheme for histogram weight factors to their histograms, which turns Equation (28)
distance calculation is proposed. We also characterize the into
boundary of human bodies with the Fourier Descriptor (FD). (i)
14 3
(i)
It provides a reliable measurement for image likelihood and D(Hr , Hk ) = αm D(Hr , Hk,( j,m)), (30)
is especially suitable for the current task. m=1 j =1

1) Silhouette: Assuming a static camera and video back-
ground, a ground-truth silhouette S is extracted through back-
ground subtraction. We compare it with S  , which is generated
by a hypothesis pose X k(i) . The pixels are categorized into 2
groups, R1 and R2 with R1 = S ∩ S  and R2 = S ∪ S  .
Let the number of pixels in R1 and R2 be N1 and N2 ,
respectively, and then the silhouette area measurement density
can be represented by
(i) N1
P1 (Yk |X k ) = .
N2
where αm denotes the weights. They are proportional to the
area of the image patches projected by different body parts.
We will show in Section V how this change improves the per-
formance of the tracker. The color measurement distribution
can then be formulated as
2 (H ,H (i) )
p2 (Yk |X k(i) ) = e−β D r k , (31)
where β is a scalar that helps the calculation result more
reasonably distributed in the range of (0, 1). To make the ref-
(26)
erence appearance model adaptive to the variation of lighting
DU et al.: MONOCULAR HUMAN MOTION TRACKING BY USING DE-MC PARTICLE FILTER 3859

Fig. 5. Fourier descriptors as image cue. (a) Boundary extraction result

containing more than 2300 pixels (left) and the boundary reconstructed from
the first 100 (middle) and the first 50 (right) FD coefficients. (b) Euclidean
distance between Fourier Descriptors extracted from ground-truth image and Fig. 6. Image likelihood function surface with respect to 2 DOFs (the
that extracted from hypothesis image. translational movement of left knee joint and right hip joint).

conditions throughout the whole sequence, an update process
can be applied:
+ − + have to worry about in many shape analysis scenarios, such as
Hr,k = λHr,k + (1 − λ)Hr,k−1 ,
where the sign + and − distinguish the reference appearance
model after and before the update has occurred.
3) Boundary: Boundaries are often confused with edges and
contours. Here we define the boundary as the outer border of
an object that does not enclose any holes. Consequently, we
can not use typical edge and contour extraction method for
boundary extraction. Instead, a morphology operator is applied
to the silhouettes S:
B=S−S
M (33)
where M is a 3 × 3 uniform structuring element and
otherwise. As we expected, the blocks along the diagonal axis
signifies erosion. We choose Fourier Descriptor as the feature
to represent a boundary B:
1 
N−1
n image likelihood density function
B( f ) = b(n)e−2π f N ,
N
n=0
where
b(n) = x(n) + j y(n),
and [x(n), y(n)](n = 0, 1, . . . , N − 1) are the image coordi-
nates of the pixels on B. N is the total number of pixels on B.
The boundary information-based measurement density is then
formulated as
(i)
p3 (Yk |X k(i) ) = e−ρ D(B( f )k ,B( f )k ) ,
where ρ has a similar function as the β in Equation (31)
(i)
and D(B( f )k , B( f )k ) is the Euclidean distance between the
FD of the ground-truth boundary and that of the boundary
(i)
generated by hypothesis pose X k .
The first 50 or 100 coefficients in B( f ) are usually sufficient
to reconstruct a boundary of thousands pixels without losing
much fidelity, as we can see from Fig. 5(a). Using only low
frequency components of the FD allows a strong emphasis to
be laid on the gross essence of boundary since high-frequency
components of the FD correspond to noise or trivial details.
Moreover, we can utilize the Fast Fourier Transform (FFT) to
accelerate the computation to a large extent. The FD can also
be directly integrated into the human tracking framework. We
do not need to do any modifications to make the FD insensitive
to translation, rotation and scaling, which we would normally
(32)
image retrieval. On the contrary, the FD needs to be sensitive
to those transformations to reflect the human motion. However,
we do wish to avoid the FD’s sensitivity to the starting point,
so we set a fixed corner of the boundary as the starting point.
Fig. 5(b) demonstrates the effectiveness of FD as a measure-
ment feature for tracking. The images on the left and the top
of the checkerboard are the ground-truth observations and the
hypothesis ones, respectively. The gray-level values of those
blocks are proportional to the Euclidean distance between the
FDs of the ground-truths and that of the hypotheses. A dark
block indicates a strong resemblance and a bright one indicates
are the darkest among the row and the column they are located
in.
4) Fusion: We fuse the three image cues for an overall
(34)
p(Yk |X k(i) ) = p1 (Yk |X k(i) ) · p2 (Yk |X k(i) ) · p3 (Yk |X k(i) ). (37)
In our experiments, we assume three cues are of the same
(35)
importance. Fig. 6 shows the surfaces of the proposed image
likelihood function for two different frames. In order to be
visualized, the likelihood function is shown with regard only
to 2 DOFs of the human body model and all the other DOFs
are set as constant. The significant peak suggests the validity
of our image likelihood design scheme.
(36)
V. E XPERIMENTAL R ESULTS
We performed experiments with the proposed DE-MC par-
ticle filter and the measurement function. We used seven
monocular human motion video sequences in the exper-
iments. Sequences 1 to 4 are walking, hopping, run-
ning and jumping sequences, respectively in a regular set-
ting. Sequence 5 is a public test sequence downloaded
from www.csc.kth.se/hedvig/movies.html, which is a walking
sequence in a circular trace with a complex background.
Sequences 6 and 7 are public test sequences from HumanEva
Database [48], with the subject performing boxing actions
in Sequence 6, and a combination of walking and jogging
actions in an elliptical path in Sequence 7. The lengths of
3860
(a) Tracking results for the walking sequence by the DE-MC
particle filter.
(c) Tracking results for the running sequence by the DE-MC
particle filter.
Fig. 7. Snapshots of the DE-MC particle filters tracking results for: (a) walking, (b) hopping, (c) running, and (d) jumping sequences, respectively.
video sequences vary from 1.2 seconds to 5.5 seconds and the
frame size is 720 × 480 for Sequences 1 to 4, 320 × 240
for Sequence 5, 640 × 480 for Sequences 6 and 7. For
Sequences 1 to 4 we do not ask the subjects to wear shorts and
T-shirts or other clothes that can label the body parts by using
different colors or patterns as clues. The human subject wears
loose fit clothing with uniform color and pattern. It will bring
even more difficulties to the task. Although Sequences 1–4
are shot in side view, the hopping and jumping videos also
contain many limb motions in depth direction. Sequence 5
is more challenging in that not only its background is more
complicated but also there are considerably more motions
vertical to image plane in it. Due to serious self-occlusions
and depth ambiguity, monocular image sequences offer a
huge challenge for any human motion tracking algorithm.
Moreover, there are both cyclic and non-cyclic motions in
those test sequences. To test the performance of the pro-
posed algorithm, we carry out different kinds of experiments.
In these experiments, model parameters are manually initial-
ized for each test video. However, this stage can be replaced
with an automatic initialization module which is driven by
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 22, NO. 10, OCTOBER 2013
(b) Tracking results for the hopping sequence by the DE-MC
particle filter.
(d) Tracking results for the jumping sequence by the DE-MC
particle filter.
any state-of-art bottom-up detectors. Please refer to Yang and
Ramanan’s “Mixture of Deformable Parts” model [49] and
Viola and Jones’ object detection work [50] for examples of
such detectors.
A. Full-Body Human Motion Tracking Experiments
In our work, the captured human motion video sequence is
the ground-truth observation, while an image generated by the
articulated human model is called a hypothesis observation.
Since the 3D joint angle can only be measured by asking the
subject to wear motion capture devices and there is always an
unsolvable ambiguity in the depth direction based on only 2D
observation, we currently consider the overlapping as the way
to make qualitative analysis. This method is adopted by many
related works [9], [28], and [36].
Figs. 7 and 8 show part of the tracking results for the
test sequences. For the walking sequence, a 7-layer DE-MC
particle filter is used (here we use “layer” to represent a DE-
MC iteration). The number of particles is 500, which can
lead to a satisfactory balance between the reliability and the
DU et al.: MONOCULAR HUMAN MOTION TRACKING BY USING DE-MC PARTICLE FILTER
Fig. 8. Tracking results for the walking in circle sequence by the DE-MC particle filter.
computational cost of the tracker. For the hopping, running
and jumping sequences, a 9-layer DE-MC particle filter with
600 particles is used, in order to handle faster motions and
more depth ambiguities than those presented in Sequence 1.
In Fig. 8, a 12-layer, 500-sample DE-MC particle filter is
used for the walking in circle sequence. As we observed
from the shown results, motion blurs do affect performance
of the tracker, and so do self-occlusions. Misplacement of
limbs is inevitable in these cases. However, when blurs or self-
occlusions disappear, the tracker is able to correct the errors
in time.
Fig. 9 shows tracking results for several representative
frames of Sequence 6 and Sequence 7 from the HumanEva
Database [48]. In the first sequence, the subject performs
boxing actions, and, in the second, the subject moves in circle
and performs a combination of walking and jogging actions,
following by certain free limb motions. Both sequences con-
tain complex limb motions. We used a DE-MC particle filter
with 5-layer and 500 samples/layer for the first sequence, and a
filter with 12-layer and 500 samples/layer for the second. Even
though more samples were used for the second sequence to
offset the difficulty caused by view variations, some tracking
3861
errors were still observed in this case, especially when the
limbs of the subject were bending and projected only to small
areas on the image plane. However, overall, our algorithm
produces satisfactory results on both sequences.
B. Comparison Experiment 1
In Fig. 10, we compare the tracking results obtained by
implementing a 7-layer DE-MC particle filter with those
obtained by implementing other popular particle filtering-
based algorithms. In terms of fair comparison, the experi-
ment is based on almost the same number of measurement
function evaluations because it is the most time-consuming
part for particle filtering. The experiment settings are: CON-
DENSATION – 5000 samples; annealed particle filter – 10
layers, 500 samples per layer; DE-MC particle filter – 7
layers, 500 samples per layer. The other factors, such as
the initialization result, initial standard deviation of the state
vector, constant velocity model, adaptive strategy and mea-
surement function are all given the same settings for each
algorithm. Despite consuming only 70% of the computations
spent by the other two algorithms, DE-MC particle filter still
3862 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 22, NO. 10, OCTOBER 2013

Fig. 9. Tracking results for Sequences 6 and 7 from the HumanEva Database [48]. The first two rows are the 50th, 100th, 150th, and 200th frames of the
Box-1(C1) video and the rest are the 500th, 600th, 700th, 800th, 900th, 1600th, 1800th, and 1900th frames of the Combo-2(C3) video.

outperforms them. It estimates every human body part with
relatively high accuracy. The annealed particle filter, which
is based on the Simulated Annealing algorithm, is shown
to successfully track human walking from multi-view video
sequences in [36]. But in our monocular experiment, it just
roughly captures the global location of the human subject
but makes invalid estimates to the position of limbs in many
cases. The classic CONDENSATION algorithm cannot even
find the human body accurately after 20 or so frames. This
experiment remarkably demonstrate the performance of the
proposed approach. Quantitative analysis of this experiment
is conducted by measuring the average of sample weights
ω(X k(i) ) at each frame. The average sample weights ω̄(X k(i) )
N
can be formulated as ω̄(X k(i) ) = (i)
i=1 ω(X k )/N at each
time step k. They are regarded as a “score” of a tracker’s
efficiency since they can reflect, to a certain degree, how many
of the samples are considered “valid”. The results are plotted
in Fig. 11. As we have pointed out, DE-MC particle filter’s
DU et al.: MONOCULAR HUMAN MOTION TRACKING BY USING DE-MC PARTICLE FILTER 3863

Fig. 12. Comparison of the performance of the DE-MC particle filters with
two layers, five layers, and seven layers (from left to right, respectively). The
original image is the 48th frame of Sequence 1.

Fig. 10. Comparison of the tracking performance of the DE-MC particle

filters with CONDENSATION algorithm and the annealed particle filter
algorithm.

Fig. 13. The quantitative analysis of comparison experiment 2 in terms of

the average sample weights.

Fig. 11. The quantitative analysis of comparison experiment 1 in terms of

the average sample weights.

success in tracking is largely due to the efficient usage of

particles, which can be reflected by its stable score in Fig. 11.
On the contrary, tracking failure causes accumulated errors for
the annealed particle filter and CONDENSATION trackers and
leads to the consecutive dropping in scores. We can observe
consecutive drop of the score over the first several frames.This
Fig. 14. Comparison of the tracking performance based on different image
is however, because the impact of a precise initialization likelihood functions.
gradually decreases when the tracking proceeds.

C. Comparison Experiment 2 D. Comparison Experiment 3

In this experiment we compare the efficacy of different
In this experiment we compare the tracking results for
image cues. We compare the tracking results for Sequence
Sequence 1 obtained by using 2-layer, 5-layer and 7-layer
1 based on 4 different image likelihood functions and list
DE-MC particle filters. The numbers of samples per layer for
some of the results in Fig. 14. This experiment exactly
them are 1750, 700 and 500, respectively. The goal of this
demonstrates the necessity of fusing multiple image features
experiment is to verify that in terms of equal number of image
for tracking. We also test our weighting scheme of calculating
likelihood function evaluations more DE-MC iterations can
Bhattacharyya distance between two histograms. Fig. 15 shows
boost the tracker’s performance. This speculation is supported
the results of tracking Sequence 1 with a DE-MC particle
by the results shown in Fig. 12 and the quantitative analysis
filter before and after adopting the proposed color cue scheme
in terms of average sample weights shown in Fig. 13. Note
as formulated in Equation (30). The performance gain by
that the results shown here should not be confused with the
adopting this simple change is significant and evident.
intermediate output of a 7-layer DE-MC particle filter obtained
after the computation in the second, fifth and seventh layer. It
also differs from the DE-MC particle filtering demonstration E. Computation Time Analysis
experiment in that the result is an accumulative one and that The computation time varies with a number of factors with
the three trackers have different particle number settings. the two most important being the number of particles and
3864
Fig. 15. Tracking results before (top) and after (bottom) adopting weighting
scheme for color-cue calculation.
the number of layers. On a workstation with quad-core Xeon
2.4 GHz CPU, it takes 0.5 ∼ 1.4 seconds for a 500-particle
DE-MC particle filter to finish the computation of one layer, of
which the overwhelming majority is spent on the calculation
of image likelihoods and feature extraction. The ratio of
computation time for the three cues (silhouette: boundary:
color) is approximately 1 : 5.6 : 31.3. This is as expected
considering their respective computational complexities.
VI. C ONCLUSION
In this paper, we propose a novel approach for human
motion tracking, the DE-MC particle filter, based on the
Differential Evolution algorithm, the Markov Chain Monte
Carlo theory and the particle filtering. This method can
improve the traditional particle filter based approach in that
it bridges the gap between sampling and measurement. This
is realized by incorporating a stochastic optimization method,
the DE algorithm and a statistical sampling method, the
MCMC algorithm into the particle filtering frame work. In
terms of searching for global optimal pose configuration
in a high-dimensional state vector space, the DE-MC par-
ticle filter achieves very good balance between exploration
and exploitation. In addition, we apply the proposed DE-
MC particle filter to track monocular human motion video
sequences. An articulated 3D human body model is built.
Our robust image likelihood model contributes a lot to the
satisfactory performance of the proposed algorithm. The multi-
cue fusion scheme is shown to help shape a peaky observation
density function. We evaluate the proposed DE-MC particle
filter in a variety of experiments which include comparison
with other popular trackers. Experimental results have shown
that a boosted tracking performance is achieved compared to
traditional particle filter based algorithms.
R EFERENCES
[1] M. Isard and A. Blake, “Condensation conditional density propagation
for visual tracking,” Int. J. Comput. Vis., vol. 29, no. 1, pp. 5–28, 1998.
[2] K. An and M. Chung, “3D head tracking and pose-robust 2D texture
map-based face recognition using a simple ellipsoid model,” in Proc.
IEEE/RSJ Int. Conf. Intell. Robots Syst., Sep. 2008, pp. 307–312.
[3] S. Roweis, L. Saul, and G. Hinton, “Global coordination of local linear
models,” in Advances in Neural Information Processing Systems, vol. 2.
Cambridge, MA, USA: MIT Press, 2002, pp. 889–896.
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 22, NO. 10, OCTOBER 2013
[4] H. Chang and D. Yeung, “Robust locally linear embedding,” Pattern
Recognit., vol. 39, no. 6, pp. 1053–1065, 2006.
[5] R. Li, M. Yang, S. Sclaroff, and T. Tian, “Monocular tracking of 3D
human motion with a coordinated mixture of factor analyzers,” in Proc.
9th Eur. Conf. Comput. Vis., 2006, pp. 137–150.
[6] N. Lawrence, “Gaussian process latent variable models for visualisation
of high dimensional data,” in Advances in Neural Information Processing
Systems, vol. 16. Cambridge, MA, USA: MIT Press, 2004, pp. 329–336.
[7] N. Lawrence and A. Moore, “Hierarchical Gaussian process latent
variable models,” in Proc. 24th Int. Conf. Mach. Learn., 2007,
pp. 481–488.
[8] C. Ek, P. Torr, and N. Lawrence, “Gaussian process latent variable
models for human pose estimation,” in Proc. 4th Int. Conf. Mach. Learn.
Multimodal Interact., 2007, pp. 132–143.
[9] R. Urtasun, D. Fleet, and P. Fua, “3D people tracking with Gaussian
process dynamical models,” in Proc. IEEE Comput. Soc. Conf. Comput.
Vis. Pattern Recognit., vol. 1. Jun. 2006, pp. 238–245.
[10] K. Grochow, S. Martin, A. Hertzmann, and Z. Popović, “Style-based
inverse kinematics,” ACM Trans. Graph., vol. 23, no. 3, pp. 522–531,
2004.
[11] X. Xu and B. Li, “Adaptive Rao–Blackwellized particle filter and its
evaluation for tracking in surveillance,” IEEE Trans. Image Process.,
vol. 16, no. 3, pp. 838–849, Mar. 2007.
[12] S. Wachter and H. Nagel, “Tracking persons in monocular image
sequences,” Comput. Vis. Image Understand., vol. 74, no. 3,
pp. 174–192, 1999.
[13] C. Stauffer and W. Grimson, “Learning patterns of activity using real-
time tracking,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 8,
pp. 747–757, Aug. 2000.
[14] S. Zhou, R. Chellappa, and B. Moghaddam, “Visual tracking and
recognition using appearance-adaptive models in particle filters,” IEEE
Trans. Image Process., vol. 13, no. 11, pp. 1491–1506, Nov. 2004.
[15] E. Kim, “Output feedback tracking control of robot manipulators with
model uncertainty via adaptive fuzzy logic,” IEEE Trans. Fuzzy Syst.,
vol. 12, no. 3, pp. 368–378, Jun. 2004.
[16] S. Zhou, R. Chellappa, and B. Moghaddam, “Appearance tracking using
adaptive models in a particle filter,” in Proc. 6th Asian Conf. Comput.
Vis., 2004, pp. 1–9.
[17] F. Caillette, A. Galata, and T. Howard, “Real-time 3-D human body
tracking using learnt models of behaviour,” Comput. Vis. Image Under-
stand., vol. 109, no. 2, pp. 112–125, 2008.
[18] S. Hou, A. Galata, F. Caillette, N. Thacker, and P. Bromiley, “Real-time
body tracking using a Gaussian process latent variable model,” in Proc.
IEEE 11th Int. Conf. Comput. Vis., Oct. 2007, pp. 1–8.
[19] F. Caillette, A. Galata, and T. Howard, “Real-time 3-D human body
tracking using variable length Markov models,” in Proc. Brit. Mach.
Vis. Conf., vol. 1. 2005, pp. 469–478.
[20] Y. Ouyang, Y. Ling, and J. Xing, “Based-APF human motion tracking
from monocular videos,” in Proc. IEEE Int. Conf. Image Anal. Signal
Process., Apr. 2009, pp. 179–183.
[21] I. Patras and E. Hancock, “Template trackingwith observation rele-
vance determination,” in Proc. IEEE Int. Conf. Image Process., vol. 1.
Sep.–Oct. 2007, pp. 501–504.
[22] I. Patras and E. Hancock, “Regression-based template tracking in pres-
ence of occlusions,” in Proc. 8th Int. Workshop Image Anal. Multimedia
Interact. Services, Jun. 2007, pp. 15–19.
[23] P. Perez and B. Gidas, “Motion detection and tracking using deformable
templates,” in Proc. IEEE Int. Conf. Image Process., vol. 2. Nov. 1994,
pp. 272–276.
[24] M. Isard and A. Blake, “Icondensation: Unifying low-level and high-
level tracking in a stochastic framework,” in Proc. 5th Eur. Conf.
Comput. Vis., 1998, pp. 893–901.
[25] M. Fontmarty, F. Lerasle, and P. Danes, “Data fusion within a mod-
ified annealed particle filter dedicated to human motion capture,”
in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst., Oct.–Nov. 2007,
pp. 3391–3396.
[26] E. Poon and D. Fleet, “Hybrid Monte Carlo filtering: Edge-based
people tracking,” in Proc. Workshop Motion Video Comput., Dec. 2002,
pp. 151–158.
[27] C. Sminchisescu and B. Triggs, “Kinematic jump processes for monoc-
ular 3D human tracking,” in Proc. IEEE Comput. Soc. Conf. Comput.
Vis. Pattern Recognit., vol. 1. Jun. 2003, pp. 69–76.
[28] C. Sminchisescu and B. Triggs, “Covariance scaled sampling for monoc-
ular 3D body tracking,” in Proc. IEEE Comput. Soc. Conf. Comput. Vis.
Pattern Recognit., vol. 1. Dec. 2001, pp. 447–452.
DU et al.: MONOCULAR HUMAN MOTION TRACKING BY USING DE-MC PARTICLE FILTER
[29] R. Van Der Merwe, A. Doucet, N. De Freitas, and E. Wan, “The
unscented particle filter,” in Advances in Neural Information Processing
Systems. Cambridge, MA, USA: MIT Press, 2001, pp. 584–590.
[30] K. Okuma, A. Taleghani, N. Freitas, J. Little, and D. Lowe, “A boosted
particle filter: Multitarget detection and tracking,” in Proc. 8th Eur. Conf.
Comput. Vis., 2004, pp. 28–39.
[31] X. Wang, S. Wang, and J. Ma, “An improved particle filter for target
tracking in sensor systems,” J. Sensors, vol. 7, no. 1, pp. 144–156, 2007.
[32] C. Chang and R. Ansari, “Kernel particle filter for visual tracking,” IEEE
Signal Process. Lett., vol. 12, no. 3, pp. 242–245, Mar. 2005.
[33] C. Chang, R. Ansari, and A. Khokhar, “Multiple object tracking with
kernel particle filter,” in Proc. IEEE Comput. Soc. Conf. Comput. Vis.
Pattern Recognit., vol. 1. Jun. 2005, pp. 566–573.
[34] J. Schmidt, J. Fritsch, and B. Kwolek, “Kernel particle filter for real-
time 3D body tracking in monocular color images,” in Proc. 7th Int.
Conf. Autom. Face Gesture Recognit., Apr. 2006, pp. 567–572.
[35] Y. Wu, G. Hua, and T. Yu, “Tracking articulated body by dynamic
Markov network,” in Proc. 9th IEEE Int. Conf. Comput. Vis., Oct. 2003,
pp. 1094–1101.
[36] J. Deutscher, A. Blake, and I. Reid, “Articulated body motion capture
by annealed particle filtering,” in Proc. IEEE Conf. Comput. Vis. Pattern
Recognit., vol. 2. Jun. 2000, pp. 126–133.
[37] G. Klein and D. Murray, “Full-3D edge tracking with a particle filter,”
in Proc. 17th Brit. Mach. Vis. Conf., 2006, pp. 256–260.
[38] C. Chang, R. Ansari, and A. Khokhar, “Cyclic articulated human motion
tracking by sequential ancestral simulation,” in Proc. IEEE Comput. Soc.
Conf. Comput. Vis. Pattern Recognit., vol. 2. Jun.–Jul. 2004, pp. 45–52.
[39] M. Arulampalam, S. Maskell, N. Gordon, and T. Clapp, “A tutorial
on particle filters for online nonlinear/non-Gaussian Bayesian tracking,”
IEEE Trans. Signal Process., vol. 50, no. 2, pp. 174–188, Feb. 2002.
[40] J. MacCormick and M. Isard, “Partitioned sampling, articulated objects,
and interface-quality hand tracking,” in Proc. 6th Eur. Conf. Comput.
Vis., 2000, pp. 3–19.
[41] C. Andrieu, N. De Freitas, A. Doucet, and M. Jordan, “An introduction
to MCMC for machine learning,” Mach. Learn., vol. 50, no. 1, pp. 5–43,
2003.
[42] R. Storn, “On the usage of differential evolution for function optimiza-
tion,” in Proc. Biennial Conf. North Amer. Fuzzy Inf. Process. Soc.,
Jun. 1996, pp. 519–523.
[43] C. Ter Braak, “Genetic algorithms and Markov chain Monte Carlo:
Differential evolution Markov chain makes Bayesian computing easy,”
Biometris, Wageningen UR, Wageningen, The Netherlands, Tech.
Rep. 010404, 2004.
[44] C. Sminchisescu and B. Triggs, “Hyperdynamics importance sampling,”
in Proc. 7th Eur. Conf. Comput. Vis., 2002, pp. 769–783.
[45] C. Sminchisescu and B. Triggs, “Building roadmaps of minima and
transitions in visual models,” Int. J. Comput. Vis., vol. 61, no. 1,
pp. 81–101, 2005.
[46] P. Perez, J. Vermaak, and A. Blake, “Data fusion for visual tracking
with particles,” Proc. IEEE, vol. 92, no. 3, pp. 495–513, Mar. 2004.
[47] T. Roberts, S. McKenna, and I. Ricketts, “Online appearance learning for
3D articulated human tracking,” in Proc. 16th IEEE Int. Conf. Pattern
Recognit., vol. 1. Aug. 2002, pp. 425–428.
[48] L. Sigal, A. O. Balan, and M. J. Black, “Humaneva: Synchronized
video and motion capture dataset and baseline algorithm for evaluation
of articulated human motion,” Int. J. Comput. Vis., vol. 87, nos. 1–2,
pp. 4–27, 2010.
[49] Y. Yang and D. Ramanan, “Articulated pose estimation with flexible
mixtures-of-parts,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit.,
Jun. 2011, pp. 1385–1392.
[50] P. Viola and M. Jones, “Robust real-time object detection,” Int. J.
Comput. Vis., vol. 57, no. 2, pp. 137–154, 2001.
3865
Ming Du received the B.S. degree in electrical engi-
neering from the Beijing Institute of Technology,
Beijing, China, in 2002, and the M.S. degree from
the Department of Electrical and Computer Engi-
neering, Ryerson University, Toronto, ON, Canada,
in 2005. He is currently pursuing the Ph.D. degree
with the Department of Electrical and Computer
Engineering, University of Maryland, College Park,
MD, USA. His current research interests include
computer vision and statistical pattern recognition,
especially video-based face detection, and tracking
and recognition. He is working on video analysis based on machine learning
algorithms.
Xiaoming Nan received the M.S. degree in telecom-
munication engineering from the Beijing University
of Posts & Telecommunications, Beijing, China, in
2010. He is currently pursuing the Ph.D. degree with
the Ryerson Multimedia Research Laboratory, Ryer-
son University, Toronto, ON, Canada. His current
research interests include multimedia cloud comput-
ing, content based video retrieval, immersive com-
munication, and human body tracking. He received
the Meritorious Prize in Mathematical Contest in
Modeling in 2006.
Ling Guan (S’88–M’90–SM’96–F’08) received the
Ph.D. degree in electrical engineering from the
University of British Columbia, Vancouver, BC,
Canada, in 1989. He is currently a Professor and a
Tier I Canada Research Chair with the Department
of Electrical and Computer Engineering, Ryerson
University, Toronto, ON, Canada. He held visiting
positions with British Telecom, London, U.K., in
1994, Tokyo Institute of Technology, Tokyo, Japan
in 1999, Princeton University, Princeton, NJ, USA,
in 2000, National ICT Australia, Sydney, Australia,
in 2007, Hong Kong Polytechnic University, Hong Kong, from 2008 to
2009, and Microsoft Research Asia, Cambridge, U.K., in 2002 and 2009.
He has published extensively in multimedia processing and communications,
human-centered computing, machine learning, and adaptive image and signal
processing. He is a fellow of the Engineering Institute of Canada and an
Elected member of the Canadian Academy of Engineering. He is the IEEE
Circuits and System Society Distinguished Lecturer from 2010 to 2011 and he
is a recipient of the 2005 IEEE T RANSACTIONS ON C IRCUITS AND S YSTEMS
Best Paper Award.