Contents

Introduction to Adaptive Filters

1.1 Motivation for Adaptive Filtering . . . . . . . . .
1.2 Overview of Adaptive Filtering Principles . . . . .
1.2.1 Advantages and Features of Adaptive Filters
1.2.2 Adaptive Filter Structures . . . . . . . . .
1.2.3 Adaptation Approaches . . . . . . . . . .
1.3 Applications of Adaptive Filters . . . . . . . . . .
1.3.1 Modelling or System Identification . . . .
1.3.2 Inverse Modelling . . . . . . . . . . . . .
1.3.3 Linear Prediction . . . . . . . . . . . . .
1.3.4 Interference Cancellation . . . . . . . . .
1.4 Concluding Remarks . . . . . . . . . . . . . . .
1.5 Learning Objectives of this Module . . . . . . . .
1.6 Conventions and Notations . . . . . . . . . . . .

2
4
7
8
11
17
19
19
21
22
23
25
26
28

Chapter 1
Introduction to Adaptive Filters
In this chapter, a brief and introductory overview on the topic
of adaptive filtering is given. In the context of signal processing,
filter is a device that is designed to process the signals at its
input in a specified manner and generate output signals that
meet certain well defined objectives. For example, a filter may
be designed
to extract information about specific quantities of interest
from noisy data (e.g. enhancing a desired signal buried
in noise, recovering the transmitted data bits from received
noisy signal etc),
to aid in tracking dynamic physical processes (e.g. missile
guidance, jamming the radar of an enemy vehicle etc),
to aid in medical diagnosis by analyzing the measured biomedical signals,
to find efficient representation of signals (e.g. compression
of speech and image data), etc.
2

EE5301 Chap -1: Introduction to Adaptive Filters; S. Puthusserypady

In general, adaptive filtering refers to the process by which

a filter tries to adjust its parameters so as to respond, with some
specified objectives, to the changes that are taking place in its
surroundings. The changes and/or state of the surroundings may
or may not be directly measurable. That is why an adaptive
filter is employed to learn these changes. Therefore, the study of
adaptive filters involves several aspects as listed below.
Filter Structure: The type of filters used (e.g. non-recursive
filters, recursive filters etc.)
Task of the Filter: The function to be fulfilled by the filter
(e.g. channel equalization, channel identification, intereference cancellation etc.)
Adaptation Algorithm:
adaptation

The steps specified for doing the

Performance Measures: The measures that can be used to

assess the quality of the adaptation process (e.g. speed of
convergence, accuracy etc.)
Analysis: Theoretical analysis of the adaptation process and
performance so as to aid in the selection as well as design of
the adaptive algorithm.
In this module on adaptive signal processing, we will touch upon
all the above aspects and the application of adaptive filters to
real-world problems.

EE5301 Chap -1: Introduction to Adaptive Filters; S. Puthusserypady

v(n)

s (n)

Channel

H (z )

Threshold
detector

Equalizer

x(n)

W (z )

s(n )

y(n)

Figure 1.1: A baseband transmission and reception system with fixed

equalizer.

1.1

Motivation for Adaptive Filtering

Consider the baseband data transmission and reception system

shown in Figure 1.1. Here, the input s(n) denotes the transmitted data symbols, H(z) denotes the channel transfer function and v(n) denotes the channel noise. A simple reception
scheme, consisting of an equalizer W (z) and a threshold detector, is considered. As the transmitted symbols pass through the
channel, they are subjected to several distortions. The received
signal x(n) is a noisy version of the distorted symbols. The role
of the equalizer is to compensate for the distortions introduced
by the channel and noise on the data symbols, in such a way that
the threshold detector should be able to satisfactorily recover the
data symbols.
To design an equalizer to achieve the above objective, we
could resort to a one-time effort by setting up the mean squared
error (MSE) at the equalizer output as a cost function and solving
for the optimum equalizer that minimizes the MSE. However, as
we will see later, the equation for the optimum equalizer involves

EE5301 Chap -1: Introduction to Adaptive Filters; S. Puthusserypady

certain statistics (e.g. auto-correlation and cross-correlation) of

the underlying signals. To compute these statistics, we need to
know the channel characteristics. If these quantities are not available, then we will need to use time-averaging on the measured
signals (i.e. channel input, equalizer input, desired signal) to
estimate the required statistics. This can be very costly in terms
of computations, memory and time. Furthermore, if the channel
characteristics are time-varying, then we will need to repeat the
above computations very often so as to respond to the changes
in the channel characteristics. On the other hand, the adaptive
equalization approach offers a very attractive alternative way for
getting the (near) optimum filter settings.
v(n)
s (n )

Channel

H (z )

Equalizer

x(n)

W (z )

e(n )
Training
sequence

y(n)

Threshold
detector

s(n )

d (n ) = s (n )

Figure 1.2: A baseband transmission and reception system with adaptive

equalizer.

The adaptive equalization set-up is shown in Figure 1.2. A

training sequence, which is also known to the receiver, is usually transmitted at the start of transmission. Ideally, the equalizer output is expected to be equal to the transmitted symbols.

EE5301 Chap -1: Introduction to Adaptive Filters; S. Puthusserypady

Because the training sequence is known to the receiver, we can

generate an error signal that indicates the amount of discrepancy
between the actual output of the equalizer and the desired output. This error signal can in turn be used to adjust (or, adapt)
the equalizer parameters to reduce the error in some specified
sense (e.g. MSE). At the end of the training mode, the equalizer parameters would have converged close to their optimal values. At this time, the detected symbols (or, decisions) at the
threshold detector output can be considered very reliable. From
then onwards, the decisions can be used to continue adapting
the equalizer. This is called the decision directed mode. This is
helpful to fine-tune the equalizer and to track slow changes in
the channel characteristics.
In the above process of adapting the equalizer parameters,
what is equivalently happening is that we are indirectly acquiring
some knowledge about the channel characteristics. This is the
main idea behind adaptive filtering, i.e. to learn about a system
by observing the changes taking place in the response of the
system to known (partially or fully) inputs and then to reflect
this knowledge in adjusting the parameters of a filter to achieve
certain desired objectives.

EE5301 Chap -1: Introduction to Adaptive Filters; S. Puthusserypady

desired signal

d(n)

x (n )
input signal

Adaptive
filter

y(n)
output signal

error signal

e(n)

Figure 1.3: Schematic diagram showing the main components of an adaptive filtering set-up.

1.2

Overview of Adaptive Filtering Principles

The main principles of adaptive filtering can be described by considering the schematic shown in Figure 1.3. In a nut-shell, the
purpose of the adaptive filter is to produce an output signal,
based on a given input signal, that very closely matches a specified desired signal. The match between the desired signal and
filter output is quantified by a performance function (also called,
cost function) defined using the error signal. An adaptation
algorithm, which adjusts the filter coefficients, is developed to
optimize the given performance function in an iterative manner,
e.g. minimize the mean square value of the error. The mean
square error (MSE) is one of the most widely used performance
functions.

EE5301 Chap -1: Introduction to Adaptive Filters; S. Puthusserypady

1.2.1

Advantages and Features of Adaptive Filters

The example of adaptive equalization discussed in Section 1.1

answers the following questions:
i) Why choose the adaptive approach ?
ii) Why do not we solve for the optimum filter coefficients by a
one-time optimization of the selected cost function ?
In an attempt to give a more complete picture of adaptive filters,
we list below some of their advantages and features.
Advantages of Adaptive Filters:
1. Memory and Computational Requirements: Estimation of
the required statistics for direct computation of the optimum
filter parameters requires accumulation of large amounts of
signal samples and doing averaging on these large data sequences. Further, this approach also results in large delay in
the filter output.
On the other hand, adaptive approaches do not need to
accumulate signal samples, thereby resulting in large savings
in memory and computational costs. Furthermore, they do
not introduce any significant delay in the filter output.
2. Self-Optimization: Adaptive filters are self-optimizing systems. That is, they can adapt themselves to an existing
environment to achieve near-optimum performance.
3. Time-Varying Environment: Adaptive filters can adapt their

EE5301 Chap -1: Introduction to Adaptive Filters; S. Puthusserypady

parameters to provide near-optimum performance even when

the environment parameters are (slowly) time-varying.
4. Graceful Degradation: Adaptive filters can repair themselves
to certain extent. That is, if some parts of an adaptive
filter are damaged, the remaining parts can (to some extent)
automatically compensate for the damaged parts. In other
words, they exhibit graceful degradation in performance.
On ther other hand, non-adaptive systems almost collapse if
some parts are damaged.
5. Ease of Design: Adaptive filters help to relieve the designer
from having to collect accurate knowledge of the environment (e.g. channel characteristics).
6. Implementation: Adaptive algorithms are usually much simpler to code in software and DSP (digital signal processing)
processors, or to implement in hardware, compared to nonadaptive approaches.
General Features of Adaptive Filters:
1. Adaptive filters require an initial training effort. However,
there are also blind adaptive algorithms that relax this requirement to varying degrees1.
In digital communication systems, the study of blind adaptation
approaches is receiving more attention currently since blind schemes
result in efficient use of the available bandwidth for transmitting the
user data (rather than training+user data).
1

EE5301 Chap -1: Introduction to Adaptive Filters; S. Puthusserypady

10

2. Strictly speaking, adaptive filters are non-linear because the

filter parameters are data-dependent. As a result, an adaptive filter does not obey the principle of superposition even
if the filter structure is that of a linear filter.
3. Outputs of adaptive filters are non-stationary in nature since
the filter parameters are changing with time.
Selection of Adaptive Algorithms: In the literature, there exists a wide variety of adaptive algorithms. For any particular
application, we need to wisely choose the adaptive algorithm to
use. In general, examining the following factors help to decide
on the choice of one algorithm above another.
1. Convergence speed: It is the number of iterations required
for the algorithm to converge close enough to the optimum
filter parameters.
2. Mis-adjustment: It is a quantitative measure of the amount
by which the final value of the MSE deviates from the minimum MSE.
3. Tracking: It is the ability of the algorithm to track variations in the environment. Tracking performance involves the
trade-off between convergence speed and accuracy.
4. Robustness: It refers to the ability of the algorithm to operate satisfactorily in the presence of small disturbances (external or internal).

EE5301 Chap -1: Introduction to Adaptive Filters; S. Puthusserypady

11

5. Complexity: It refers to the number of computations and

memory elements required for completing each iteration of
the adaptive algorithm.
6. Structure: This refers to the structure of information flow in
the algorithm. This has great impact on efficient hardware
implementation and achievable speed of adaptive algorithms.
7. Numerical Properties: This refers to the behaviour of the
algorithm under finite-precision implementation (i.e. quantization erors) and ill-conditioned input data. We prefer to
have a numerically robust adaptive algorithm.
1.2.2

Adaptive Filter Structures

In signal processing applications, the most commonly used structures for implementing digital (discrete-time) adaptive filtering
are transversal filter, linear combiner, lattice predictor, and recursive filter.
Transversal Filter: A transversal filter with N coefficients
(taps or weights) is shown in Figure 1.4. The input x(n) passes
through a series of identical delay elements whose delay is matched
to the arrival rate of the input samples. The filter output y(n) is
a linear combination of the outputs of the delay elements, and is

EE5301 Chap -1: Introduction to Adaptive Filters; S. Puthusserypady

x (n )

w0

z 1

w1

x(n1)

z 1

x(n2)

w2

z 1

12

x(n N +1)

w N 1

y(n)
Figure 1.4: A transversal filter (or, tapped-delay line filter) with N taps.

given by2
y(n) =

NX
1
i=0

wix(n i).

For this reason, it is also known as a tapped-delay line filter.

The transfer function of this filter is given by
1
Y (z) NX
H(z) =
=
wiz i
X(z)
i=0

where Y (z) and X(z) are the z-transforms of y(n) and

x(n), respectively.
Linear Combiner: In some applications (e.g. beamforming),
the tap inputs may not be delayed samples of a single input. In
such cases, the linear combiner structure shown in Figure 1.5 is
In adaptive filters, the filter coefficients should be denoted as
w0 (n), w1 (n), , wN 1 (n) to express the fact that the coefficients are
changing with time due to adaptation. For the sake of convenience, we
do not explicitly show this in the filter structures shown in Figures 1.4
to 1.7.
2

EE5301 Chap -1: Introduction to Adaptive Filters; S. Puthusserypady

x 0 (n )

w0

x1(n )
w1

x 2 (n )

w2

13

xN 1(n)

w N 1

y(n)
Figure 1.5: A linear combiner with N taps.

used. Its output y(n) is a linear combination of the different

input signals, and is given by
y(n) =

NX
1
i=0

wixi(n).

Note that the linear combiner becomes a transversal filter if we

choose xi(n) = x(n i), i = 0, 1, . . . , N 1. The computational complexities of linear combiner and transversal filter are
equal and is given by N multiplications and N 1 additions.
x (n)

f0(n)
b0(n)

f1(n)
b1(n)

f2 (n )
b2 (n )

fN 1(n )

bN 1(n) N

fN (n )

bN (n)

Figure 1.6: A lattice predictor of order N .

Lattice Predictor: The lattice predictor is known for its modular structure. A lattice predictor of order N consists of
N stages as shown in Figure 1.6. Each of the stages has two

EE5301 Chap -1: Introduction to Adaptive Filters; S. Puthusserypady

14

inputs and two outputs. The mth stage is described by a pair

of input-output relations given by
fm(n) = fm1(n) mbm1(n 1)
bm(n) = bm1(n 1) mfm1(n)
for m = 1, 2, . . . , N , with f0(n) = b0(n) = x(n). Its computational complexity is 2N multiplications and 2N additions.
b0

x (n)

z 1

x(n1)
z 1

x(n2)

x(n Nb )

a1

b1

b2

z 1

a2

z 1

y(n1)
z 1

y(n2)

aNa

bN b

y(n)

z 1

y(n Na )

Figure 1.7: The structure of an IIR filter with Nb zeros and Na poles.

Recursive Filter: The structures in Figures 1.4, 1.5 and 1.6

correspond to non-recursive filters, i.e. computation of the filter
output does not involve any feedback or recursive mechanism.
Therefore, they are also called finite impulse response (FIR) filters since their impulse responses are of finite duration in time.

EE5301 Chap -1: Introduction to Adaptive Filters; S. Puthusserypady

15

Further, the number of coefficients in these filters specify the

effective length of the impulse response of these filters. Consequently, it is impossible to use these filters to implement impulse
responses that are very long, due to complexity.
On the other hand, impulse responses of infinite duration
can be implemented using filters with finite number of parameters by incorporating feedback in the computation of the filter
output. The structure of the resulting filter, called infinite impulse response (IIR) filter, is shown in Figure 1.7. Its output is
given by the recursive equation
y(n) =

Nb
X
i=0

bix(n i) +

N
a
X
i=1

aiy(n i)

where a1, a2, , aNa and b0, b1, , bNb are coefficients of
the feedback and feedforward paths, respectively. The transfer
function of this filter is given by
Nb
X

H(z) =

Y (z)
=
X(z)

biz i

i=0
N
a
X

i=1

.
ai z

Thus, H(z) has Nb zeros (i.e. roots of the numerator) and

Na poles (i.e. roots of the denominator). One of the main
advantages of IIR filters is the computational simplicity in implementing infinite (very long) impulse responses. Its computational complexity is Na +Nb +1 multiplications and Na +Nb ad-

EE5301 Chap -1: Introduction to Adaptive Filters; S. Puthusserypady

16

ditions. The IIR structure becomes a FIR structure if Na = 0.

Unlike FIR filters, stability is an important concern in IIR
filters. This makes adaptation of IIR filters more difficult compared to FIR filters. To gurantee stability, all the poles of the
filter must lie within the unit circle |z| = 1. Another difficulty
associated with adaptive IIR filters is that the MSE performance
function associated with these filters may have local optima due
to their recursive nature.
Systolic Array: The systolic array structure is developed for
parallel computing applications, and is suitable for implementing linear algebraic computations such as matrix multiplication,
triangularization, matrix transformation etc. The use of systolic
arrays help to achieve high throughput rates, which are required
in several real-time applications of advanced signal processing.
Non-linear Filters: The filter structures considered above are
linear in nature. In situations where the environment or system to
be modeled behaves non-linearly, we need to use non-linear filter
structures. Volterra filters and neural networks are examples of
non-linear filter structures.
Real and Complex Filters: In some applications, the filter
input and desired signal are complex-valued rather than realvalued. For example, the baseband signal corresponding to a

EE5301 Chap -1: Introduction to Adaptive Filters; S. Puthusserypady

17

QAM (quadrature amplitude modulation) modulated signal is

complex-valued. In processing such signals, we need to use filters
with complex-valued coefficients.
1.2.3

Adaptation Approaches

The various adaptation approaches can be broadly classified to

belong to either stochastic or deterministic frameworks depending upon if the underlying cost function is a statistical average
or time-average. The most popularly used cost function in the
stochastic framework is mean square error (MSE) and the resulting filters are called Wiener filters. The most popularly used
cost function in the deterministic framework is the weighted sum
of squared errors and the resulting approaches are called least
squares approaches (LS).
Wiener Filter Approach: The optimum coefficients of the
Wiener filter are obtained by minimizing the MSE cost function.
Clearly, construction and minimization of this cost function requires certain statistics which need to be obtained by ensemble
averaging. The adaptive approach to Wiener filters is to replace
these statistics by easily obtainable estimates. For example, the
well known LMS (least mean square) adaptive algorithm is obtained when the MSE E[e2(n)] (E[] denotes statistical expectation operator) is replaced by the instantaneous squared error
e2(n).

EE5301 Chap -1: Introduction to Adaptive Filters; S. Puthusserypady

18

Transform domain adaptive filtering approaches help to

achieve faster convergence speed. The adaptation is done in
transform domain, which is so selected as to result in faster
convergence of the algorithm in this domain compared to
regular time domain implementation.
Block adaptation approaches help to achieve reduction in
computational requirements when the number of filter coefficients is very large. The adaptation is done only once every
block. By making use of FFT (fast Fourier transform), this
can be done efficiently.
Least Squares Approach: In this case, the optimum filter coefficients are obtained by minimizing the sum of weighted squared
errors for a given data. This deterministic approach results in algorithms that may, in general, converge faster than MSE-based
algorithms. However, these algorithms are generally more computationally demanding and numerically sensitive compared to
the MSE-based algorithms. There are three classes of recursive
least squares (RLS) adaptive algorithms.
Standard RLS Algorithm: Its complexity is proportional to
N 2 where N is the number of filter coefficients.
QR-Decomposition based RLS Algorithm: This version is
more robust to numerical errors. Its complexity is also proportional to N 2. It can be implemented using systolic
arrays which are suitable for parallel computing.

EE5301 Chap -1: Introduction to Adaptive Filters; S. Puthusserypady

19

Fast RLS Algorithms: The complexity of this family of algorithms is proportional to N .

1.3

Applications of Adaptive Filters

Adaptive filtering has been successfully applied in several diverse

fields such as communications, control, speech processing, biomedical engineering, data storage etc. In all these applications,
the role of the adaptive filter is to filter a given input signal to
match a specified desired signal. An essential difference between
the various applications is the manner in which the desired signal
is obtained. The functions of the four basic classes of adaptive filtering applications are modelling or system identification,
inverse modelling, linear prediction and interference cancellation.
1.3.1

Modelling or System Identification

In modelling applications, the task of the adaptive filter is to

provide a model that represents the best fit to an unknown plant.
This is illustrated in Figure 1.8. Usually, the plant output is a
noisy signal. The task of the adaptive filter is to learn the transfer
function of the plant.
Typical applications of modelling are self-tuning regulators in
control systems and adaptive channel identification in communication or storage systems. Figure 1.9 illustrates the self-tuning
regulator. The adaptive filter assumes the role of Plant Model
in this figure. The identified model parameters are used to mod-

EE5301 Chap -1: Introduction to Adaptive Filters; S. Puthusserypady

20

d (n )

x(n)

Plant

e(n)

Adaptive
Filter

y(n)

Figure 1.8: Adaptive system modelling.

Model Parameters

Plant
Model

Design

d (n)

Regulator
Parameters

u(n)
Regulator

x(n)

Plant

Figure 1.9: Applications of adaptive system modelling: Self-tuning regulator.

ify the control signal for the plant. Figure 1.10 illustrates the
adaptive channel identification system. The extracted channel
parameters can be used to design the equalizer and detector. As
mentioned earlier, after the initial training mode, the adaptation
can continue by replacing the training sequence with the symbol
decisions from the output of the detector.

EE5301 Chap -1: Introduction to Adaptive Filters; S. Puthusserypady

21

v(n)
s (n)

Training
sequence

Channel

G (z )

e(n )

W (z )
Channel model

Equalizer
&
Detector

d(n)

s(n )

y(n)

Channel
Parameters

Figure 1.10: Applications of adaptive system modelling: Adaptive channel

identification.

1.3.2

Inverse Modelling

In inverse modelling applications, the task of the adaptive filter

is to provide an inverse model of an unknown plant. This is
illustrated in Figure 1.11.
s(n)

Plant

Delay

x(n)

d (n )

Adaptive
Filter

e(n)

y(n)

Figure 1.11: Adaptive inverse system modelling.

A typical application of inverse modelling is adaptive channel equalization in communication or storage systems. We discussed this in Section 1.1 and is illustrated in Figure 1.2. The
adaptive filter models inverse of the channel transfer function
while minimizing the channel noise. Again, after the initial train-

EE5301 Chap -1: Introduction to Adaptive Filters; S. Puthusserypady

22

ing mode, the adaptation can continue by replacing the training

sequence with the symbol decisions from the detector output.
u(n)

Delay

x(n)

Adaptive
Filter

e(n)

y(n)

d (n )
Figure 1.12: Adaptive linear prediction.

1.3.3

Linear Prediction

In linear prediction applications, the adaptive filter is meant to

provide the best prediction of the current sample of a random
signal using its past samples. This is illustrated in Figure 1.12.
The adaptive filter output is the prediction estimate of the signal u(n). Consequently, the error signal e(n) is the unpredictable part of u(n). This, coupled with the concept of autoregressive modelling, can be used for finding parametric representations of correlated random processes.
Typical applications of linear prediction are in speech processing, signal detection and spectral estimation. For the sake of
illustration, we consider the adaptive line enhancement (ALE)
problem which is an example of adaptive signal detection. The
objective of ALE is to extract a narrowband signal from a wideband interference or vice-versa. An example of ALE is to clean up

EE5301 Chap -1: Introduction to Adaptive Filters; S. Puthusserypady

23

the measured bio-medical signal from the 50/60 Hz power-line

interference. The required system set-up will look exactly similar
to Figure 1.12 with u(n) being the bio-medical signal mixed with
power-line interference. Because narrowband signals are highly
correlated processes compared to wideband signals, output of the
adaptive filter (or, adaptive predictor) will be a good estimation
of the narrowband component in u(n). Consequently, the error
signal e(n) will represent the cleaned-up bio-medical signal.
d (n )

Primary input
desired
signal
Interference

x(n)
Reference
input

Adaptive
Filter

y(n)

e(n)

Figure 1.13: Adaptive interference cancellation.

1.3.4

Interference Cancellation

In interference cancellation applications, the adaptive filter is to

cancel unknown interferences contained in a primary signal. The
primary signal is a mixture of the desired signal and the interferences. A reference signal is applied to the adaptive filter input
and this signal is supposed to be very weakly correlated with the
desired signal component in the primary signal and highly correlated with the interference component. This is illustrated in
Figure 1.13. Thus, the output of the adaptive filter becomes a
good estimate of the interference. Subtracting this estimated in-

EE5301 Chap -1: Introduction to Adaptive Filters; S. Puthusserypady

24

terference y(n) from the primary input, we get the error signal
e(n) as the cleaned-up desired signal.
Conference room

d (n )

receive signal

Adaptive
Filter

Loudspeaker

y (n )

e(n)
transmit signal

Person A
noise
Microphone

reflections

Figure 1.14: Application of adaptive interference cancellation for acoustic

echo cancellation in teleconferencing systems.

Typical applications of interference cancellation are in echo

cancellation in telephone networks, active noise control and
beamforming in communications. The application of acoutic
echo cancellation in teleconferencing systems is illustrated in Figure 1.14. The adaptive filter plays the role of creating a replica
of the echo of picked up by the microphone. It does this by using the speech signal from the far-end speaker as the reference
signal.

EE5301 Chap -1: Introduction to Adaptive Filters; S. Puthusserypady

1.4

25

Concluding Remarks

Throughout this module, we will be using non-recursive filter

structures for our study of adaptive algorithms.
Our studies will be mainly focused on real-valued systems.
For the sake of generality and completion, we will give extensions to complex-valued systems.
We will not consider finite-precision effects and details of
hardware implementation aspects in this module.
Our study will be mainly centered around stationary signal
scenarios. Non-stationary signal scenarios and tracking issues will not be discussed in detail.

EE5301 Chap -1: Introduction to Adaptive Filters; S. Puthusserypady

1.5

26

Learning Objectives of this Module

Formulation, design and development, and analysis of

adaptive approaches for practical problems.
Principles of iterative/adaptive approaches for optimization
of nonlinear functions:
Various algorithms (e.g. steepest descent, LMS, Newton, RLS)
Stochastic and deterministic approaches to optimum estimation
Simplified approaches to adaptive filtering
Adaptive filtering in the presence of constraints.
Analysis of iterative/adaptive algorithms:
convergence behaviour (speed, stability)
accuracy
complexity
Quantification of the goodness of adaptive algorithms.
Influence of signal statistics on the performance of the algorithm.
Influence of applications (e.g. modelling, inverse modelling
etc.) on the performance of the algorithm.
Approaches to improve convergence of adaptive algorithms
(e.g. transform domain adaptive filtering).

EE5301 Chap -1: Introduction to Adaptive Filters; S. Puthusserypady

27

Properties of transversal and lattice filter structures.

Parametric modelling of random processes.
Mathematical tools to help in algorithm development
and analysis.
Strengthen the fundamentals of signals and systems.
Acquire deeper insights on digital signal processing methodologies, interpretation of signals etc.
Translation of mathematical results into physical
interpretation in the study of signals and systems.

EE5301 Chap -1: Introduction to Adaptive Filters; S. Puthusserypady

1.6

28

Conventions and Notations

Boldface uppercase letters denote matrices and boldface lowercase letters denote vectors. Non-boldface lowercase letters
denote scalar quantities.
Superscripts H, T and denote Hermitian (conjugate)
transposition, ordinary transposition, and complex conjugation, respectively.
Unless otherwise mentioned, the signals and systems are assumed to be real-valued.
E[x] denotes expectation of x and |x| denotes magnitude
of x. The operator z 1 denotes one sample delay.
IM denotes the M M identity matrix. Boldface 0 denotes
a vector or matrix of zeros, with appropriate dimensions.
Let A be a M M matrix with its (i, j)th element
given by Ai,j for i, j = 0, 1, . . . , M 1. Then i) tr(A)
stands for trace of A and is equal to

MX
1
i=0

Ai,i, ii) diag(A)

stands for diagonal of A and it results in a diagonal matrix

whose main diagonal is same as the main diagonal of A, and
iii) det(A) stands for determinant of A. If x is a M 1
vector, then diag(x) results in a M M diagonal matrix
whose main diagonal is equal to x.
All the signals and systems considered here are discrete-time
in nature. The signal samples are assumed to be at the

EE5301 Chap -1: Introduction to Adaptive Filters; S. Puthusserypady

29

normalized sampling period of 1 second, i.e. the sampling

frequency is 1Hz.
The integers k and n are used to denote iteration indices
and time indices, respectively. The time dependence of a
variable x is represented by x(n), meaning the value of
x at nth instant.
Impulse responses of filters are represented using subscripted
variables. For example, hk denotes the value of the k th coefficient (or weight, tap) in the discrete-time impulse response of a filter. These coefficients are also assumed to be
at the spacing of 1 second.
The autocorrelation of a random process {x(n)} is defined
as xx(m) = E [x(n)x(n m)] for m = 0, 1, 2, . . ..
All the random processes are assumed to be zero-mean and
wide-sense stationary, unless otherwise stated explicitly.
Similarly, cross-correlation between random processes {x(n)}
and {y(n)} is defined as xy (m) = E [x(n)y (n m)]
for m = 0, 1, 2, . . ..
The random processes {x(n)} and {d(n)} (i.e. the tap
inputs and desired signal, respectively) are assumed to be
individually and jointly wide-sense stationary.
The power spectral density (PSD) of {x(n)} is defined as

X
xx ej =
xx(m)ejm, where is the normalized
m=

EE5301 Chap -1: Introduction to Adaptive Filters; S. Puthusserypady

30

frequency variable in radians (i.e. = 2F T = 2f and

f = F T where F is the unnormalized frequency in Hz,
T is the sampling period in Seconds, and f is the normalized frequency).
Similarly, the cross-spectral density between {x(n)} and {y(n)}

X
j
is defined as xy e
=
xy (m)ejm.
m=

Note: Bandwidth for real and complex signals.

Output of a filter with impulse response hi, i = 0, 1, . . . , N
1, and input x(n) is computed as
y(n) = h0x(n) + h1x(n 1) + + hN 1x(n N + 1).
In vector notation, this can be written as y(n) = hT x(n)
where the N 1 vectors h and x(n) are given by
h = [h0, h1, , hN 1]T
and x(n) = [x(n), x(n 1), , x(n N + 1)]T ,
respectively.
Let a = bH c where a is a complex scalar and b and c

T
H
2
are
complex
vectors.
Then,
a
=
b
c
=
c
b
and
|a|
=

H 2
b c = bH c cH b = bH ccH b.

Let R denote the N N correlation matrix of the signal

{x(n)}. Further, let q0, q1, , qN 1 be the N orthonormal eigenvectors of R corresponding to the eigenvalues
0 1 N 1. Then, its eigen (or, spectral)

EE5301 Chap -1: Introduction to Adaptive Filters; S. Puthusserypady

31

decomposition is given by
T

R = QQ =

NX
1
i=0

iqiqTi

(1.1)

where Q = [q0, q1, , qN 1] is a unitary matrix and =

diag [0, 1, , N 1] is a diagonal matrix.
The trace of R is given by the sum of its eigenvalues:
tr (R) = N xx(0) =
h

NX
1
i=0

i,

(1.2)

where xx(0) = E x (n) is the diagonal element of R.

The determinant of R is given by the product of its eigenvalues:
NY
1
det (R) =
i.
(1.3)
i=0

For any N N matrices A and B, we have

det (AB) = det(A) det(B).

(1.4)

For any M N matrix A and N M matrix B, we

have
tr (AB) = tr(BA)

(1.5)

(AB)T = BT AT

(1.6)

(AB)H = BH AH

(1.7)

(AB) = AB.

(1.8)

For any N N invertible matrices A and B, we have

(AB)1 = B1A1.

(1.9)