IEEE TRANSAMOXS

ON AUTOXATIC COWTROL, VOL.

ac-15, NO. 2, APRIL 1970

175

On the Identification of Variances and Adaptive

Kalman Filtering

Abstract-A Kalman filter requires an exactknowledge of the

noise
processnoise covariance matrix Q andthemeasurement
covariance matrix R. Here we consider the case in which the true
values of Q and R are unknown. The system is assumed to be
constant, and the random inputs are stationary. First, a correlation
test is given which checks whether a particular Kalman filter is
working optimally or not. If the filter is suboptimal, a technique is
given to obtain asymptotically normal, unbiased,and consistent
estimates of Q and R. This technique works only for the case in
which the form of Q is known and the numberof unknown elements
in Q is less than n X r where n is the dimension of the state vector
and r is the dimension of the measurement vector. For other cases,
the optimal steady-state gain KO,is obtained directly by an iterative procedurewithout identifying Q. As a corollary, it is shown
thatthesteady-state
optimal Kalman filter gain KO, depends
only on R X r linear functionals of Q. The results are &st derived
for discrete systems. They are thenextended to continuous systems.
A numerical example is given to show the usefulnessof the approach.

the number of unknown elements in Q must be less than

n X r where n is the dimension of t,he state vector and T

is t.he dimension of the measurement vector. It, is shown

that in spite of this limitation, the optimal steady-state
filter ga.in can be obtained by an it,erative procedure.As a
corolhry, it is shown that the Ka1ma.n filter gain depends
only on n X r linear relationships between the elements
of Q.
A numerical example is included to il1ustrat.e the application of the results derived in the paper. The extension
of the results to t,he continuous case is st.raightfonvard
and is given in the last sect.ion.
OF THE PROBLEM
11. STATEMEKT

XYStt???t

I. IKTRODCCTIOX

Consider a ndtivariable linear discrete system

+
+

= cpzi
rui
(1)
HE OPTIMUM filtering result,s of Kalman a.nd B U C ~
[l], [ a ] for linear dynamic systems require an exact
zi = HZ; ~i
(2)
knowledge of the process noise covariance matrix Q and
the measurement noise covariance ma,trix R. I n a number where x, is 7% X 1 state vector, 9 is n X n nonsingular
of practical situations, Q and R are either unknown or are transition matrix, r is n X q const.ant input matrix, zi
is T X 1 measurement vector, and H is r X 77. constant
knownonlyapproximately.Heffes
[3] andKishimura
[4] have considered the effect of errors in Q and R on the output matrix.
The sequences ui(q X 1) and vi(r X 1) are uncorreperformance of the optimal filter. Several other investilated
Gaussian white noise sequences n-ith means and cogators [5+[9] have proposed on-line schemes to identify
variances
as follows :
Q a.nd R. Most of these schemes do well in identifying R
but run into diaculties in ident.ifying Q. Moreover, their
E ( u ; ) = 0; E(uiujT]= Q S ,
extension to continuous cases is not clear. A different
E { v ~=) 0; E ( v , v , ~ =
] R&j
approach ha.s been taken in this paper. It is assumed that
the system under consideration is time invariant, comE (u,$'} = 0, for all i,j
pletely controllable, and observable [a]. Both the system
andthe filter (optimal or suboptinlal)areassumed
to where E { denotes the expectation, and 6 i j denotes the
have reached steady-state conditions. First, a correla.tion Kronecker delta function.
Q and R are bounded positive definite matrices (Q > 0,
test is performed on the filt.er to check whether it is workR
> 0). Initial state zo is normally distributed with zero
ing optimally or not. The t.est is based on the innovation
property of an optimal filter [lo]. If the filter is subopti- mean and covariance Po.
The system is assumed to be completely observable and
mal, the auto-correlation function
of the innovationprocess
controllable,
i.e.,
is used to obtain asymptotically unbiased and consistent
estimates of Q and R. The met,hod hasthe linut,a.tionthat
rank CHT,( H @ T,
)
(Han-')'1 = 72

a }

--

Manuscript received August 2, 1968; revised May 16, 1969.

This paper was presented a tt h e 1969 Joint AutomaticControl
Conference, Boulder, Colo.
The author WBS mjth theAnalytic Sciences Corporation, Reading,
Mass. 01867. He is now withSystemsControl,
Inc., Pdo Alto,
Calif. 94306.

e,

rank Lr,@r,-.,cpn-lr]

= n.

Filter
Let

(Qo

Qo

and Ro be the initialestimates

> 0, Ro > 0).

estimates,
these
Using

of

steadythe let

and R

176

IEEE TRAZTSACTIOKS ON AUTOMATIC CONTROL, APRIL

state Kdman filter gain be KO (n X

KO

For i > j , vi is independent of ej and v j :

matrix)'

+ Bo)-'

&IoHT(HIWd2T

1970

E(vivjTj

(3)

&lo = @
- A ~p
J P (l
Ho
M ~ H T+ R ~ ) - ~ H M ~ +
] WrQorT.
(4)
M o may be recognized as the steady-state solution to the
covariance equations of Kalman [ l ] .

The filtering equations a.re

+ vj)'l

E[Hei(He,

E [ Hei (zj - H?j+l)

T}.

The orthogonality principle states that ei is orthogonal

to (zIs,k< i}. Since i j , j - l depends only on (z~s,k< j ) , we
conclude that
E(vivjT} = 0, f o r i > j.

Similarly, E(vivjT} = 0, for i < j.

For i = j , E{viviT}= H M H T R. Further, since vi i s
% q i = 2ili-1
Ko(zi - HkIi-1)
(6) a linearsum of Gaussianrandom variables, it is also
Gaussian. Hence vi is a Gaussian n-hite noise sequence.
n-here Pi+lli is t.he estimate of xi+l based on all the measureHeuristically, the innovation vi represents the new informents up to i, i.e., (zo,
mation brought by zi. Iiailath [ l o ] shon-s tha.t v i and zi
I n a n optimal Ka1ma.n filter (i.e., n-hen Qo = Q and contain the same st.atist.icalinforma.tion and are equinlent
Ro = R ) , df0 is the covariance of the error in estimating as far as 1inea.r opera,tions are concerned. Schweppe [12]
the stat.e. But in a subopt,imal case, the covariance of the shon-s tha.t,vi can be obtained fromzi by a Gra.nlSchmidt
error ( H l ) is given by the follo-iving equation [3]:
orthogonalization (or a whitening) procedure.
In this paper, we use the innovation sequence to check
3 1 1 = @[,M1 - K J M 1 - MIHTKoT
the optimality of a Kalman filter and t,o estimat,e Q and
K o ( H M I H T R)KoT]CpT I ' Q P ( 7 ) R. With this in mind, we investigate the effect. of suboptima1it.y on the innovat.ion sequence.
n-here M 1 = E { (xi - Oili-1) ( x i - &+I) T}.
117. INKOVATION
SEQUEIWE
FOR A
Problem
SUBOPTIXAL
FILTER
The true values of Q and R a.re unlmown. It is required
Let K denote the steady-state filter gain. m e will show
that under steady state, the innovation sequence vi is a
check whether the Ka1ma.n filter construct,ed using stationary Gaussian sequence;
?i+l/i

(5)

= Cp&

--

s , ~ } .

some estimates of Q a.nd R is close to optimal or

not (hypothesis testing),
obta.in unbiaseda.nd consistent estimates of Q and R
(statistical estimation), and
adapt the Kalman filter a t regular intervals using
all the previous information (adaptive filtering).

To solve these problems, we ma.ke use of the innovat.ion

property of an optimal filter [ l o ] .

E [ v s i _ k T }= HE(eiei-t}HT

+ HE(eivi-;,T), f o r k > 0.

A recursive relationship can be obtained for e; by using

(11, (21, (5), and (6) :
ei

@ ( I- KH)ei-l

+Kuidl

(9)

a OPTIMALF'ILTER~

ei

noise sequence.
proof; A direct proof is obta.ined using the orthogonality
principle of linear estimation C10].3 Let ei = xi - &-I
denote the error in estimating the state.
Then
V ; = Hei
vi

(8)

1 The conditions of completecontrollability

and observability
together with t.he positive definiteness of Qo and Ro ensure the asymptotic global stability of theKalmanflter.
See Deyst and
Price 1111.
2 For a detailed discussion, see Kaj1at.h [lo].
3 An alternate proof will be given in Section I
T'.

[ @ ( I- KH)]j-l@Kvi.-J

[ @ ( I- KH)]Iseei-~:
i-1

Statenwnt
For an optimal fdter, the sequence vi = (zi - H&-d,
k0n.n as the innovation sequence, is a Gaussian white

+ vi) (Hej + vj)'}.

+ vi

Hei

Carrying (9) k steps back,

111. THE INNOVATION

PROPERTY
OF

E{vivjT}= E [ (Hei

- H2ili-1

Is

[ @ ( I- K H ) ] i - l I h - (j 1. 0 )
i=l

Postmultiplying (10) by eiAT and taking expecta.tions,

E { e i e i A T }= [ @ ( I- K H ) Y N
where iil is the steady-stat,e error covariance ma.tri.. An
expression for $1 is obtaineddirectlyfrom
(9) orfrom
(7) :
iif = $ ( I

- K H ) M ( I - K H ) T @ T+ @KRKTQT+ rQP.
(11)

Postmultiplying ( 1 0 ) by v+kT and t.aking expectations,

E(eai-kT} = -[+(I - K H ) I b l @ K R .

177

M3HR4: I D E N T I F I C A T I O X O F 1-ARIANCES A S D ADAPTIVE IC.4LM.4X FILTERING

Therefore,

E(vpi--kT) = H[@((IT- K H ) Y -

- @[,.MHT - K ( H X H T + R )1,

> 0.

When k = 0, E ( v i v i T j = H M H T R.
It. is seen that the autocorrelation funct,ion of v i does
not dependon i. Therefore, vi is ast.ati0na.n.Gaussian
random sequence (Gaussian because of linearity) and we
can define

Ck = E { V i V i A * } .
Then

C/: = H N H T
=

+ R,

h:

H[@CI - KH j]k-l@[MHT- KC0]? IC

Furthermore,

c-, =

> 0.

(12)
( 1 3 ) Fig.

1. Kormalized autocorrelation function of innovation process.

(a) Srtboptimal filter. (b) Optimal filter. (Arro.crs indicate points
for which 95 percent confidence limits do not, enclose zero,)

CkT.

Kotice thatthe
opt,imal choice of K , viz. K =
3 f H T ( H 3 f H T I?)- makes Cc vanish for all k # 0 (the
innovation property).

It is seen from ( 1 3 ) that CI;+ 0 for large k. It can be

shown4 that the infinite series in ( 1 6 ) has a finite sum so
that the covariance of e k is proportional to l/N. Thus,
the estimates
are
asynzptotically unbiased and conIT.
A TESTO F OPTIhL4LITY FOR -4I<-kL&1L41V FILTER
sist.ent. Moreover, since all the eigenvalues of @((IT- K H )
From the discmsion of the preceding two sections, it is lie imide the unit. circle, v i belongs to t,he class of linear
clear tha.t a necessary and sufficient condition for the processes [I41 for which Parzen [lrj] has shou-n that, t k
opt,irna.lity of aKalman
filter is thatthe innovation a.re asympt.otically norma,l.
sequence vi be white. This condit,ion can be t,ested statisFor t.he white noise case, ( 1 6 ) is greatly simplified by
tically by a number of different methods [13], [l6]-[19]. putt,ing C k = 0, for all k # 0:
Here lve consider a particular method given in Jenkins
cov ( [ t k ] i j , [ t l l P P )= 0, k # 2
and Wat.ts [13].
In this method, we obtain an estimate of Cr, denoted
= ( 1 / N )[Co]ip[Co]jqq k = > 0
a.s
by using the ergodic propertg of a st,ationary ran= (1/N) [C~lipCC~ljq[Coliq[Coljp,
dom sequence

el:,

N
t k =

( l / N ) 2 YjVi-kT

(14)

i=k

pe

( 1 - J;/NJc~.

0. (17)

Estimates of t,he normalized autocorrelation coefficients

a.re obta,inedbydividing the elements of 6~.
by the
appropriate elements of
e.g.,

where N is the number of sample points.

The estinlat,es C k are biased for finite sample sizes:

E{C,I

eo,

r.r.

(15)

I n case an unbiased estimat>e is desired, we divide by

( X - k ) instead of N in ( 1 4 ) . However, it is shown in

[13] that the estimate of ( 1 4 ) is preferable since it gives

less mean-square error than the corresponding unbia.sed
estimate.
An expression for the covariance of e k can be derived
by straight,forward manipulation, but the general results
are rat,her involved. We qu0t.e here approximate result.s
for large N given in Bartlett [14] :

fik

Of particular interest here are the diagonal elements of

for the case of whit.e noise. Using (17), we can show that
var [p&

l/N

+ 0 ( 1 / N ?()1. 9 )

Further, [p& like [ t k ] ; ; a.re asymptot.icallynormal

[15]. Therefore, the 95 percent confidence limits for [pklii,
k > 0 are =t( 1.96/N1/2),or equivalently the 93 percent
confidence limits for [ e ~ . ] ia.re
i f( 1.96!AT19 [&]ii.

cov ([l+t:]ij,[?l]pq)e

([~tlip[~t+l-.ljq

Test
Look at a set of values for [&Iij, k > 0 and check the
[ C t + ~ l d C t ~ l j p )( 1 6 ) number of times they lie outside the band f( 1.96/N12).
where [(?kIij denotes the element in the ith row and t,he If this number is less than 5 percent of the total, the
j t h column of the matrix tkand cov (a.,b) denotes the sequence v i is white. (Examples of a nonlvhite and a white
covariance of a and b ; viz.
&
- m

COY

(a$)

E ( [ u- E ( a ) ] [ b- E ( b ) ] } .

The proof is essentially similar to the

st.ability of a K a h n mter 1111.
f

one for proving the

178

IEEE TRkYSACTIONS ON AUTOMATIC CONTROL, APRIL

sequence are shown in Fig. 1. See the example in Section

E.)
This test is based on the assumption of large N . If N
is small, othertests proposed by Anderson [17] and
Hannan [lS], etc., may be used. Jenkins and Watts [13]
also give a frequency domain test which is useful if there
are slow periodic components in the time series.
VI. ESTIMATION
OF Q AND R

If the test of Section Tr revea.ls that the filter is suboptimal, the next step will be to obtain better estimates
of Q and R. This can be done using Ck computed ea.rlier.
The method proceeds in t.hree steps.
1) Obtain an estimate of MHT using (13). Rewriting
(13) explicitly,
C1

Therefore

H@'MHT- H@KCo

rcl

+ H@KCo
H@KC1+ HWKCo

LC,

+ H@KC,-1+

M H T = B" Cp

I n numerical computation, it has been found preferable

to use ( 2 2 ) since matrix A is better conditioned than
matrix B. (This is a n experimental observation.)
2 ) Obta.in an estimate of R using (12) :

+ H@,"KCo]

I-@.

where

= @X@T

= @[

rank ( B ) = n

and

B + = (BTB)-lBT.

+ n + rQrT

of MHT a.ndusing (20),

(24)

Ill on the right-hand

side

= @w(@)T

of (X),

+ Q + @rQrTQT
+ rQF.
T

(25)
Repeating the sa.me procedure n times and separating the
terms involving Q on the left-hand side of the equation,
we obtain
k-1

k-1

@jrQrT(@j)T
=M -

@M(@k)T

i=O

@jn(@j)T,

j=O

for k

1
.

(23)

-K H M - &IHTKT + KC&']@'.

Substituting back
for

Notice that B is the product of the observability matrix

and the nonsingular transition matrix @.Therefore

rel + H @ K ~ ,

80- H ( & f i T ) .

myhere
f2

LH@~-~J

Denoting5 bytheestimate
we can write

3) Obtain an estimate of Q using (11).

This step gets complicated due to the fact that only
the estimate of MHT instead of M is anilable. Consequently, only n X T linear relationships between the unknown elements of Q are available. If the number of
unknowns in Q is n X T or less, a solution can beobtained
But if the number of unknowns in Q is greater than n X T.
a uniquesolutioncannot
be obtained. Hoxever, it d,
be shown in the next section that a unique solution for
the optimal gain K , ca.n still be obtained.
Restricting ourselves to the case in which the number
of unlmonm in Q is n X T or less, we can solve for the
(20)
unknown elements of Q by rewriting (1 1) as follows :

where B* is the pseudo-inverse of matrix B [l] defined as

B =@ ;

1970

l,...,n . (26)

Premultiplying both sides of (26) by H and postmulti(21) plying by ( Wk)'HT, we obtain

k- 1

H @ T Q r T( W
An alternate form for MHT can be obta.ined directly
from (13):

k ) THT =

Hill ( W k ) ' H T

- H@klt;HT

i
d

(22)

k
A

The s ~ m b o -23
l always implies AB

FS B

single symbol.

l,-..,n. (27)

The right-ha.nd side of (27) is completely determined

from &lHT and Co. Substituting their estimated values,

179

NEHFA: IDENTIFICATION OF VABMNCES A N D AD-4PTIVE K.ALMAK FILTERIWG

Subtracting (30) from (31) and

(kf2

Mi) =

@( I -

K1H) (

M 2

simplifying
-

AI,) ( I - K1H)T@T

- @(K1- KO)(HM1HT

+ R )(K1-

Ko)T@T.
(32)

The solution to (32) can be written as a.n infinite sum.

Then, using observability and controllability conditions,
it can be shown that6
P f l ' S l C A L ATSTEII
iAL:.!h\

t :LTER

lW2

-x

<0

or

<MI.

M z

Similarly, define K2 = il!12HT(H1MzHT R)-' and M a as

the correspondingerrorcovariance
matrixThen,bya
similar argument,

Fig. 2. Identificat,ion scheme.

we obtain

M 3

< .LIZ2 < lM,.

B-1

- H@:,$IBT

H@frQrT(@.j--k)THT
= fi$l(@-k)THT
i=O

The above sequence of monotonically decreasing matrices mustconverge

since it is boundedfrom
below
( A I > 0). Hence, the sequence Ko,K1,K2, must converge to Kop.
Based on thepreceding property of K , we now construct
the following scheme for est,imating KO,.
1) Obtain an estimate of K1, denoted as 2
1 from (22) :

--

IC-1

H@jfj(@i-k)

THT,

j=O

1,"',12

(28)

where

fj = @[ - KfiXf - & f B T K T KC'&T]@T. (29)

The set of equations (28) is not linearly independent.
In any pa.rticular case, one has to choose a linearly independent subset of these equations. The procedure will
be illustrated by anexample in Section IX.
The preceding identification scheme is shown schematically in Fig. 2.
VII. DIRECTESTIMATION
OF THE OPTIMAL
GAIN

Also, obtain estimates of M I H T and R from ( 2 2 ) and

(23).
2) Define 6 M 1 = AIz - 311.Obtain &@1, an estimate of
6 M 1 , using (32) :

8&T1 = @ ( I- klH)8.&1(1- I Z I H ) ~ @ ~
If the number of unknowns in Q is more than n X r,
- a(& - Ko)(?o(kl- Ko.!T@T.(34)
or thestructure of Q isunknown, themethod of the
previous section for estimating Q does not work. However,
it is still possible to estimate the opt.ima1 gain KO,by a.n 8.&, can be calcula.ted recursively in the same manner as
JIo is calculated for a Kalnlan filter. For convergence, it
iterative procedure.
Following the notation of Section 11, let KOdenote the is sufficient tha,t @ ( I- k 1 H ) be stable, i.e., all eigenvalues
be inside the unit circle.
initialgain of theIialman filter. L,et M 1 be theerror
3) Obtain &f2fiT a.nd k2 as follows:
covariance mat,rix corresponding to KO.Then 311 sat,isfies
the following equation [cf., (7)]:
&2AT = &lAT
&f41HT
(35)

AI1

@[A41- KJIM1 - MIHTKoT

+ Ko(HMIHT+ R ) K J I P + r&rT. (30)

Define

K1 3 AIIHT(HXIHT R)-'.
Let the error covariance matrix corresponding to gain

Kl be called &I2.Then

+ K1(H$lzHT + R ) K I ~ ]+@rQrT.
~
(31)

2 2 =

&2BT(HM2BT

(36)

4) Repeat steps2) and 3 ) until 118.!@< I I or I I l?i - I?-1 ]I

become small compared to 11 &Ti 11 or 11 Ki 11 where 11 -11
denot,es a suitable matrix norm,
An a.lternat,ive way to
get Kz would be to filter data z again using K1 and then
use (33).
This procedure for obtaining KO,reveals a.n interesting
relationship between KO,and Q. It is seen tha.t. theequa6 The proof is similar t.o the one by Kalman [l]for showing the
positive definiteness of M in (11).

180

tion for ( 3 1 2 - M I ) does not involve Q. We need Q only

to calculate M I H T . This leads us to the following corollary.
corollmy: It, is sufficient to Icnon- n_xr linear functions
of
order
Q in
to obtain the optimal
gain
Kalman
of a
iilter.
Proof: Consider (30) which can be written a.s
M1

1970

IEEE TRANGACTIOXS OXAPRIL

AUTOMATIC COSTROL,

For
of

N >> n, the bias in

is negligible. The covariance

cov

(:YfiP) w K var

+ -4::cov

A?aTfor large N is

( ~ O ) K T

(* [?j)
cn

Writing the solution as an infinite series,

m

+ rQrTj

,)I:[(

Expressions for

[ @ ( I- KoH)]j(@K&KoT@T
]=o

C,

+ K cov eo,

@ ( I- KOH)Ml(I - K o H ) T @ T

M1HT =

etc.

cov

M I H T depends on n X r linear functions of Q ; x&.,

5 [@(I

([q ) P

Cn

can be obta.ined from ( l G ) . It can be seen that cov

decreases as l / N for h g e sa,mple sizes. Similarly,

(I@@)

K a ) ] j r Q r T [ ( I- KJI)TPI~HT.

i=O

If these linear functionsare given, we do not need to know

Q itself to obtain M I H T . Furthermore, since the equation
for ( M 2 - M ~ does
) not, involve Q explicitly, the optimal
gain KO,can be obtained by knowing the preceding n x r
par (I?, = var (to)H cov ( & A T j H T
linear functions of Q only.
Notice that a complete knowledge of Q is required t,o
- cov ( eo,Kf@)HT - H cov (
obt,ain the covariance matrix Ji of a Ihlman filter. If one
is interested only in KO,, the preceding corollary shows
(43)
that a complete knowledge of Q is not essential. Since our
The expressions for EL$] a.nd var ([QJj) can be obiterative scheme tries to identify KO,by mhit.ening t.he
tained
similarly.
residuals vi, it fails to identify the complete Q matrix if
The
usefulness
of t,he preceding expressions is limited by
the unknowns in Q are more t.han n X r.
the fact. that, theydepend on t.he a.ctual values of Q and R
which are unknon-n. If the values of Q and R are known
VIII. STATISTICAL
PROPERTIES
OF THE ESTIMATES
to lie n-it.hin a certain range, one might use these expres*
It -ms shorn-n inSection T: that t,he esthat.es C k are sions to plot curves of var (8)and var ([(jlij) versus Ar
asymptotica,lly normal! unbiased, a.nd consistent. Since fordifferent va.lues of Q and R. The dependence on *Q
and R ma,!* be removed by considering t.he covaria,nce of K :
1i1QT, 8,and 6 are linearly relat.ed to e k , it is easy t o
shon- t,hat they are also asymptotically normal, unbiased,
and consistent..
The general expressions for the mean and covariance of
the estimates arerat.her involved. We, therefore, specialize
to the ca.se of a scalar measurement.
It. canbe shown [14], [15] that Jl; are asymptot,ically
Csing ( Z j , (23), and (%),
normal with mean p k a,nd covariance

+-

JffiT,e0).

00

cov

($k,$d

(l/lV)

pjpj+a-a.
m
-,

A satisfactory estimate of cov (pk,$J is provided by

A-1

or

r 1

(1/2h7)

BjJj+l-.s

j=-(X-l)

m-hich can be used in (43) to calculate cov (kj.For the

special case of an optimal filter, (43) reduces to

cov (I?)

( l / N ) A * A * T= (l/N) (ATA)-l. (44)

181

NEHRA: IDEXTIFICATIOK O F YARIXTCEB ASD AD.4FTITE KALbL4S FILTERING

Equat,ion(44) gives usa simple expression for the

minimumvalianceinestimating
K. It can be used in
deciding upon the minimum sa.mple size N .
We now consider the asymptotic convergence ( N 1a.rge)
of theiterative scheme of Section VII.Equation(34)
shows that E[8JI1] dependson the second- and higher
ordermoments of k1 which for a. normal process are
finite and
tend
to
zero asymptotically.
Therefore,
E [ ~ $ T J= 6 . ~ 1 .
Similarly, the comriance of 8 A f l asymptothlly tends
to zero. Thus, 8L@l tends to 6 M 1 withprobability one.
Extending the same argument, K 2 -+ K?, K 3 -+ Ka, * -,
k,, -+ KO,with probabilit>yone.

I 1,

The startingvalues of Q a.nd R are taken as

0.25 0

QO

0.5 0

RO=

lo

0.4 0 ;
o.6] .

0.75

Using these \dues, the innomtion

sequence

vi = (2, -

H&,iLl) is generated from (3) to ( 6 ) . Theestimates

eo,el,- -

.,6k
of the autocorrelation are calculated using
(14). For a typical sample of 950 points, Fig. 1(a) ahom
a plot of the first diagonal element, of fik for k = 0>40.
The 95 percent confidence limits are +0.0636 and four
points lie outside this band (i.e., 10 percent of the tota.1).
Therefore, we reject the hypot.hesis t,hat v i is n-hite. The
E. ,4NUMERICAL
EXAMPLE
FROM
same conclusion is reached by looking at the second diagINERTIAL
KAVIGATION
onal element of PI;.
The results of Sections T and VI are
applied toa da.mped
We now proceed to the ident,ification of Q and R. Since
Schuler loop forced by an exponentiallycorrelated sta- t,he number of unknowns in Q is less than n X r = 10,
tionary random input. Two measurements a.re made on we can identify Q completely. The set of equat,ions (28)
thesystem, bot,h of which arecorruptedby
exponen- gives us alargenumber
of linear equa.tions for G1, &,
tially correlated as well a.s white noise t,ype errors. The and G3. However, the most important of these occur along
state of the syst,em is augment,ed to include all the cor- the diagonal for k = 1 and k = 5.
relat,ed random input,sso that the augmented &ate vector
For k = 1 t.he left-hand side of (28) is
z is 5 x 1, the ra.ndom input vector u is 3 X 1, a.nd the
measurement noise vector v is 3 X 1. The sgstem is
discretized using a timestep of 0.1 andthe result.ant
system mat.rices are

p.73

- 1.74

0.09

0.91

0.95

0.55

0.905

+=

-0.3

-0.0016 0

-0.15

5 the left,-ha.ndside of ( 2 8 ) is

-- 0.008

R = r
0

For k

1.
T2

The actual values of ql, q2, qa, rl, and r2 are unity, but
t.hey are assumed unknown. It is required to identify these
values using n1easurement.s { z i , i = l,Ar].

The diagonal elements of the first. equation are used to

calculate i j 3 and i2.The first diagonal elementof the second
equation is then used to calcula.te 61.
It is possible to use a. few other equations a.nd to make
a lea,st-squares fit for 41, &, and 4 3 . This, however, does
not. alter the resultssignificantly in the present example.
The results obtained by using the identification scheme
repeatedly onthe same bat,chof data areshown in TableI.
It is seen that most of t.he identification is done during
the first iteration. Further it.erations do not increase the
likelihood function7 much, even though the changes in Q
and R are significant. ,4check case using true values of
Q and R is also sholm in TableI. It is seen that the value
of t.he likelihood function in the check case is very close to
that in thefirst iteration. This indicatesthat theestimates
obtained are quite close to the maximum likelihood estimates. It was further noticed that even if different starting
valuesare used for Q and R, the identificationscheme
converges to the same va.lues.
The ljkelihood function L(Q,R) has beengiven

[12]:

L(Q,R)

s
- ( l / S ) i=l
Z:

viT(HXHT

+ R)

by SchwePPe

- 111 I H M H T

+ R I.

182

1970

OR AUTOMATIC CONTROL, APBIL

IEXE TMSACTIONS

TABLE I
ESTIUTESO F Q -am R BASEDOK A SETOF 950 POINTS
Percentage of
Points Lying
Outside the 95 Percent
Con6dence Limits

Number of
Iterations

?1

0 0.5

!73

71

72

0.25
0.75
0.7310.i761.31
1.444 0.867

1
2
3

0.87

fi

1.40
1.41
1.0

0.91
0.92
1.O

4
Check case

* Estimate of

0.776
0.i7
1.0

0.4
1.565
1.5i3
1.o

Second
h.1easul.e-

First
Likelihood
MeasureFunc$on
Square
Square ment ment
L (Q,R)
(percent)
(percent)

0.6

-5.17
10
-4.67B

10
2.5

0.765
0.7646
1.o

-4.67%
-4.671
-4.669

2.5
2.5
2.5

5 2390
2725
5
5
5

Estimat,e
of -4ctual
h*.lean-

Calculat.ed
h,lean-

Error*

Errori

2915
2755
2720
2714
2712
2720

902
2814

x
4
4
0
2900

mean-square error is
K

(l/lv) z

i-1

(Xi

- .;i i-l)T(Zi - G , L I )

tainedwhere zi is
by actual simulation.
h
t Calculated mean-square error is t r (AT,) There X, is obtained from the variance equation using Q and R [cf. (4)].
h

1.5

Similarly,

q;

J.

........L.....:..................-.
=-.

-. ......;i-""t 0

h : z

0.5

Rb+l

T7

EP + C l l ( k + 1 ) ] ( 2 M l , k

0.d

X. CONTINUOUS
SYSTEM

1.5

..... ........

......9

The results of the Drevious sections can

tended t o continuous systems. We simplystate theresults
below.8

c. 01

0.d

System

......

1.0

*=Fx+Gu

-..-

0.5

z =
2

Hx+v.

(
4
8
)

Filter

%tch h'.xzber
:

(47)

Fig. 3. On-line ident,ification of Q and R.

FP

+ Ko(z - HP)

(49)

where
We now check the optimality of the filter after i d e n a KO = PdY'R0-l
(50)
cation. Fig. 1(b) shows a plot of the &st diagonal element and
of p k , for k = 0140. It is seen that only one point lies
FPo PoF'
GQoG' - P&'Ro-'HPo = 0. (51)
outside the band of 95 percent confidence limits (2.5 percent of the total). This supports the hypothesis that
vi The error covariance P1 is given as
is white.
GQG'
K8KoT = 0.
The a,symptotic convergence of Q and R towards their (8' - K J I ) Pi+ PI( F - KdY)
actual values is s h o r n in Fig. 3. The estimates of Q and
(52)
R are updated after every batch of N points ( N = 950).
In the absence of any knowledge about the variances of Innovation Process
v=z-HP
the estimates, a simple averagingof all the previous values
is performed. This is equiva.lent to the following stochastic
=He+v
(53)
approximation scheme[5] :
where e = ( x - 2 ) . For an optimal filter, v is white with
the same covariance as 4) [lo]. For a suboptimal flter,

'+

where k denotes the batch number,

Qk
QP+~,L
Qk+l

d=

(F - Ka)e

+ Gu - Kov

theestimate of Q after k batches

the estimate of Q based on the ( k 1)th batch
* These result.s havenot been applied to apractical
the estimate of Q after ( k 1) batches.
so fa.r.

(54)
problem

MEHRA: IDENTIFICATIOH OF TARIANCES A X 3 ADAPTIVE EALMAN

and

and the autocorrelation functionC ( T ) of v is given as

c(T)= E{V(t)VT(t-

183

FILTERlNG

K 2
T)

HE{e(t)eT(t
- T ) )HT

+R ~ ( T ) , > 0
HePtr[PIHT- K&] + R ~ ( T
, ) F f F - KJl.
+ HE{e(t)vT(t-

7))

+ (HP1- RKoT)
- (-iw - FIT)--IHT + R. (56)

S ( W )= H ( ~ w
- Ff)-'(PIHT - K&)

Test of Optimality and the Estimation of Q and R

We may use either the estimates of C ( T ) or of S ( w )
to t.est the optimality of the Kalman filter and to identify
Q and R . These estimates are obt.ained by using methods
given in [13].
PIHTand R may be obtained from the set of equations
(55) or (56) by using methods very similar to the diserebe case. If the number of unknowns in Q is n X r or
less, Q can be obtained using (52). We obtain expressions
for
k-1

(- l)jHFjGQGTFk-jHT, for

0,1,

j=O

[the set of equations analogous to (28)].

If the number of unknowns in Q is more than n X r,
KO,is obtained directly without identifying Q. The procedure is as follon-s. D e h e

XI. SUMMARY
AND COXCLUSIOKS
The problem of optimal filtering for a linear timeinvariant system with unknown Q (process noise covaria.nce ma.trix) and R (measurement noise covariance mat.rix) is c.onsidered. Ba.sed on the innovation property of
an optima.1filter, a statistical testis given t.0 check whether
a part.icula.r filter is working optimally or not. I n case the
filter is suboptimal, a.n identification scheme is given to
obtain asymptotically unbiased and consistent est.imates
of Q and R. For the case in which the form of Q is unknown ort,he number of unknowns in Q is more than n X P
(nis the dimension of the state vector andr is the dimension of t'he mea.surement vector), the preceding scheme
fails and an a1ternat.e scheme is given to obtain an estimatmeof the optimal ga.in directly without identifying Q.
A numerical example is given t,o illustrate the result,s and
to show the usefulness of the approach. The results a.re
first derived for a discrete system. They are then extended
to continuous systems.
XOMENCLATURE

Matrices
@,I',H,F,G,B,A
K1 = PIHTR-l.
(57)
Q,R,N,P,M1,Pl
Qo,Ro,Jfo,Po
Let PP be the error cova.riance corresponding to K1.
Then it canbe shown that
K,K@l$op, * *

Q,&w,K,.

+ (P2 - P I )( F - K I H )

- ( K I - Ko)R(Kl-

0.

(58)

Therefore

P,

(F - k1H)

(KI - KO)&(&- K o )=~ 0.

(59)

+ 8$1HT

(60)

Then

F,BT = fill?'

System matrices
covariance matrices
initial values of Q,R,M,P
Kalman filter gains
estimated values of Q,R,M,K (caret
over any quantity denotes an
estimate)
autocorrelation function
normalized
autocorrelation
function
power spectral density
increment matrices.5

Vectors

ui,vi

--

8@1+

8~1,8~,8.ii?1,8P
xi,ii+~

< P2 < PI.

I n this way, P is decreased at each step and the sequence

Ko,Kl,Kz- converges to Kop.
Equation (58) is now used to obtain kOp,
an estimate
of K,. After obtaining
= FlBT&l, me substitute it
in (58) to get an estimate of 8P1 = P2 - P1:

( F - &H)

Ck,&,C(T)

S(W)

Similarly, define K2 = P2HTR--Iand let P3 be the error

covariance for K2. Then

P3

..

Pkrh

< P1.

(61)

and so on unt,il the relative changes in I? become sma,ll.

R e omit the proof of the asymptotic convergence of
these estimates since t,hey are essentially similar to the
discrete case. All the estimates obtained are asymptotically
unbiased and consistent.

(55)
Let, S ( w ) denote the Fourier t.ransform of C ( T )

( F - K1H) (Pz - P I )

= I;,BT&l

zi
Vi

ei

Scalars
n,q,r
N

Actual and estimated states

white noise sequences
measurements
innovations
error in state estimation.

Dimension variables
sample size
[Ck]ij
element in the ith row and the jth column of
the matrix Ck
6 ..
i d (. r.) Kroneckerdeltaand the delta function
L(&,R) likelihood function.

184

IEEE TRAWSACTIOXS O X .4UTQJUTIC COXTROL, APRIL

Operations

E(.)
Expected
value
operator
cov ( ) covarisnce operator
variance operator
var ( - )
( jT
transpose of a matrix
( )
pseudo-inverse ofmatrix
a
a ,

>-

II-II

inverse of a matrix
norm of a mat.rix.

ACKNOTTZEDGMENT
The a,uthor wishes to thank all his colleagues a t The
Anal\-tic Sciences Corporation for their help during the
course of this
research,
and C. L. Bradleyinparticular
.
~.
many stimulating discussions and ideas.

1970

1111 J. J. Deyst., Jr., and C. F. Price, Conditions for asgmptot.ic

stability of the discrete minimum variance linear estimator,
IEEE Trans. dutomatic Control (Short Papers), vol. AC13,
pp. iO2-iO5, December 1968.
[la] F. C. Bchaeppe, Evaluation of likelihood functions for Gaussian signals, IEEE Trans. InformationTheory, vol. IT-11,
pp. 61-70, January 1965.
[13] G. N . Jenkjrsand U. G. Wat.ts, Spectral -4nalysis a-nd 2 s
Applications. San Francisco: Holden Day Publ., 1968.
1141 A i . S.Bartlet.t.,An Introdudim lo Stochastic frocesses. London:
Cambridge Universit,y Press, 1962.
[15] E. Parxen, An approach to t,ime series analysis, dnn. Math..
Statist., vol. 32, no. 4,December 1961.
[16] E. J. Ilannan, Time Series Analysis. K e a Tork: Wiley, 1960.
1171 R. L. {Fderson, Distribution of t,he serial correlabion coefficient, -4nn. Math.. Statist., vol. la, 19U.
[18] G. S. \Vat-on and J. Durbin, Exact tests of serial correlation
using noncircular statistics, Ann. d l a h Statist.. vol. 22,
pp. 446451, 1951.
1191 C. W. J. Granaer. -4 auick test for serial correlation suitable
for use n i t h n&-&atioiary time serie3, A m . Statist. Assoc. J.,
pp. 728-730, September 1963.

REFERENCES
R.E. Kalman, Kern met.hods and results in linear prediction
and filtering theory, PTOC.Symp. on Engineering Applications
of Random Function Theory
and
Probability. New York:
IViley, 1961.
R. E. Kalmsn and R. S. Bucy, K e a results in linear filtering
and prediction theory, Trans. A S X E , J . BasicEngrg., ser.
I), vol. 8 3 , pp. 95-108, March 1961.
El. Heffes, The effects of erroneous models on the Ihlman
filter response, IEEE Trans. Autmm~icControl (Short Papers),
V O ~ .AC-11, pp. 541-543, July 1966.
T. Sishimura,Error bounds of continuousKalnlan filters
and t.he application to orbit det.erminat,ion problems, IEEE
Trans. Automatic Control, vol. -kc-12, pp. 268-5, June 19ti7.
R. C. K. Lee, 0pt.imal estimation, identificat,ion and cont.lol,
Massachusetts
1nst.itute
of Technology, Cambridge, l h .
?rlono. 28, 1961.
D. T. McGill, 0pt.imal adaptive estimation of sampled
stochast.ic process, St,anford Electronics Labs., Stanford,
Calif., Tech. Rept. SEL-63-143(TR 6 3 0 7 3 ) : December 19ti3.
J. S. Shellenberger, A multivarianceIesrninutechnique
for
improved dynamic system performance, 15& Proc. _\-EC,
vol. 23, pp. 146-151.
G. L. Smit.h, Sequentialestimation
of observation
error
variances in a trajectory est.imat.ionproblem, SldA J., vol.
5, pp. 1964-1970, yovember 19Gi.
R. L. Kashyap, hIaximum likelihood identification of stochastic linearsystems, Purdue University: Lafsyette,Ind.,
Tech. Rept.. TR-EE 68-28, August 1968.
T. Kailath,An innovationsapproach to least-squares estimat.ion, pt. I : linear filtering in additive white noise,! ZEEE
Trans. rlufornatic Coonfrol, vol. AC-13, pp. 646-655,1)ec.ember
1968.

Raman E;.Mehra (S6$-hI68) was

born in Lahore, Pakistan, on
February 10, 1943. He received
the B.S. degree in electrical engineering from Punjab Engineering
College, Chandigarh, India, in 1964,
1,I.P. and P11.D. degrees
andthe
in engineering from Harvard Univenity, Cambridge, AIass., in 1965
and 1968, respectively.
He worked at B~11 Telephone
Laboratories, Inc., Andover, N a s . ,
during the summer of 1965. From
1966t.o 196i, he was a Research
Assist.ant atHarvard
University.
From 1967 to 1969, he was employed by the Analytic Sciences Corp., Reading, Mass., where he
applied modern estimation and controltheory
to problems in
inert.ia1 navigation. He is presentlywith System Control,Inc.,
Palo Alto, Calif. His interests lie in the areas of trajectory optimization; linear, nonlinear, and adaptive filtering; smoothing and
System identificat.ion; stochasticcontrol; and pattern recognition.