MUS421/EE367B Lecture 6
FIR Digital Filter Design In a previous lecture, we looked at many windows, and
examined their properties. Today, we are going to see
Julius O. Smith III (jos@ccrma.stanford.edu) how these windows can be used to design Finite Impulse
Center for Computer Research in Music and Acoustics (CCRMA) Response (FIR) digital filters.
Department of Music, Stanford University
Stanford, California 94305 FFT processors implement long FIR filters more
efficiently than any other method (using Overlap-Add).
March 28, 2005
We need flexible ways to design all kinds of FIR filters for
use in FFT processors.
Outline
1 2
Amplitude Response:
∆
hideal(n) = DTFT−1
n Hideal
Z ωc
∆ 1
1
= ejωndω
2π −ωc
sin(ωcn)
Gain
=
πn
0
= 2fcsinc(2fcn), n∈Z
0 Frequency fc fs/2
Problems:
• Infinitely long
• Gain = 1 for f < fc • Non-causal
• Gain = 0 for f > fc • Cannot be shifted to make it causal
• Cannot be implemented in practice
Conclusion:
• We must accept some compromise(s) in the design of
any practical lowpass filter.
• Managing such trade-offs is the topic of digital filter
design.
3 4
Optimal (but Poor) Least-Squares Length L = 30 Least-Squares LPF Design
Impulse Response Design
Length 30 FIR Amplitude Response
Let 1.5
Magnitude (Linear)
• h(n) = ideal filter impulse response. 1
∞
X 2 XL−1 2
∆
J2(ĥ) = h(n) − ĥ(n) = h(n) − ĥ(n) + c2 −10
0
n=−∞
Magnitude (dB)
n=0 −20
∆ P0 2 P∞ 2 −30
where c2 = n=−∞ |h(n)| + n=L |h(n)| does not −40
Result: The error is minimized (in the least-squares Normalized Frequency (cycles/sample)
1
0
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
Normalized Frequency (cycles/sample)
0.5
0
−10
0
−20 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
Magnitude (dB)
−70
−30
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 −40
Normalized Frequency (cycles/sample)
−50
−60
1 + δp
Ideal lowpass
δp
1 filter response
1 − δp
Pass-band Stop-band
TW
δs
0
0 Frequency ωp Transition ωs
band
9 10
Example Ripple Calculations in Matlab So, that’s about 60 dB stop-band rejection, which is not
too bad. Now let’s set the stopband ripple to 1/100
Let’s first consider the passband ripple spec, ±0.1 dB. times the passband ripple:
Converting that to linear ripple amplitude gives, in
Matlab, >> ds=dp/100;
>> 20*log10(ds)
format long; ans =
dp=10^(0.1/20)-1 -78.72623816882052
dp =
0.01157945425990 So now we’re close to 80dB SBA, which is getting into
the ”high fidelity” zone.
Let’s check it:
>> 20*log10(1+dp) Ideal Lowpass Filter
ans =
0.10000000000000 The ideal lowpass filter is defined by the following
>> 20*log10(1-dp) specifications:
ans =
-0.10116471483635 ∆
• ωs = ωp = ωc = 2πfc ⇒ TW = 0
Ok, close enough. Now let’s set the stopband ripple to • δp = δs = 0 ⇒ SBA = ∞
1/10 times the passband ripple and see where we are:
>> ds=dp/10;
>> 20*log10(ds)
ans =
-58.72623816882052
11 12
The Window Method for FIR Filter Example
Design
A typical windowed ideal lowpass filter response is
• In practical FFT processors, we are limited to a finite depicted in the following diagram:
duration impulse response.
Length 101 Lowpass FIR Filter - Hamming Window
• An “obvious” method for FIR filter design is to simply 1
∆ 0.4
ĥw (n) = w(n) · hideal(n)
Amplitude
0.2
13 14
M −1
Ĥcausal(ω) = e−jω
0.2 0.2
2 Ĥ(ω)
Amplitude
Amplitude
0.1 0.1
0 0
-0.1 -0.1
0 10 20 0 10 20
time (samples) time (samples)
Hamming Kaiser
0.3 0.3
0.2 0.2
Amplitude
Amplitude
0.1 0.1
0 0
-0.1 -0.1
0 10 20 0 10 20
time (samples) time (samples)
15 16
Amplitude Response of Examples Effect of Changing Filter Length, M = 11, 15, 41
Hamming Window
Rectangular Hanning
0 0 0
Amplitude - dB
Amplitude - dB
-20 -20 −10
-40 -40
−20
-60 -60
Amplitude − dB
−30
-2 0 2 -2 0 2
Frequency (radians/sample) Frequency (radians/sample)
Hamming Kaiser −40
0 0
Amplitude - dB
Amplitude - dB
−50
-20 -20
−60
-40 -40
17 18
This is equivalent to a modulation by e−jπn in the time A bandpass can be derived by designing the appropriate
domain. Hence, lowpass filter (with cutoff frequency equal to half the
width of the bandpass), and then modulating it to the
desired center frequency ωc. In the frequency domain,
hhp(n) = hlp(n)e−jπn this means shifting two copies of the lowpass prototype,
one to the left by −ωc, and the other to the right by ωc.
= hlp(n)(−1)n
The two shifted copies are added together to give a
bandpass filter having a real impulse response.
19 20
|H|
Summary of the Window Method
21 22
replacements ω
π
ω 0
π |H|
0 2π
|H|
−π
Example:
2π Chebyshev window and its transform: −2π DFT of Chebyshev Window: M = 11, Ripple = -40 dB
0
−π ω
−2π 1
Chebyshev Window: M = 101, Ripple = -40 dB -10
0.8 π
dB Magnitude
-30
|H| 0.7 ωc
-40
π 0.6 ω0
Amplitude
ωc 0.5 ωu -50
ω0 0.4
ωl -60
ωu 0.3
−π -70
ωl 0.2
−ωc -3 -2 -1 0 1 2 3
−π 0.1
−ω0 freq
−ωc -100
0 -50 0 50 100
Time (samples)
−ω0
27 28
Lp norms • L∞-norm
N −1
! p1
∆
X
The Lp norm of an N -dimensional vector x is defined as k x k∞ = lim |xi|p
p→∞
i=0
! p1
N
X −1 In the limit as p → ∞, the Lp norm of x is
∆
k x kp = |xi|p dominated by the maximum element of x.
i=0
Special cases
• L1 norm
N
X −1
∆
k x k1 = |xi|
i=0
• Sum of the absolute values of the elements
• “City block” distance
• L2 norm v
uN −1
uX
∆ t
k x k2 = |xi|2
i=0
• “Euclidean” distance
• Minimized by “Least Squares” techniques
29 30
35 36
2π
−π
−2π
Chebyshev Optimal Linear Phase filter Design Hence we are minimizing a linear objective, subject to a
ω linear constraints. This is known as a linear
set of
0
programming problem (linprog in Matlab).
Consider the L∞-norm minimization problem: |H|
π
min k Ax − b k∞ ωc
x
ω0
We said earlier that the ∞ norm of a vector is the ωu
maximum element of that vector, hence minimizing the ωl
∞ norm is minimizing the maximum element of a vector: −π
−ωc
min max |aTk x − bk |
x k −ω0 Linear
Constraints
where aTk denotes the kth row of the matrix A.
We can then write this as: Linear
Objective
min t
s.t. |aTk x − bk | < t
∆ x
If we introduce a new variable x̃ = , then
t
t = f T x̃ = [ 0 . . . 0 1 ]x̃, and our optimization problem
becomes:
min f T x̃
x̃
s.t. [ aTk 0 ] · x̃ − bk < x̃f T x̃
37 38
So far we have looked at the design of linear phase FIR In linear-phase filter design, we assumed symmetry of our
filters. In this case, A, x and b are all real. filter coefficients [h(n) = h(−n)] ⇒
Sometimes we may want to design filters and specify both • The filter frequency response became a sum of
the magnitude and the phase of the frequency response. cosines (“zero phase”)
Examples: • The matrix A was real
• Minimum phase filters • The desired magnitude response b was real
• Inverse filters • The final zero-phase filter x̂ could be right-shifted
L/2 samples to get a corresponding causal
• Fractional delay filters
linear-phase FIR filter
• Interpolation polyphase subfilters
Now we would like to specify a complex frequency
response. This means that:
• b is complex
• A is complex
• We still want x (our filter coefficients) to be real
39 40
Summarizing our problem: Optimal FIR Filters — Arbitrary Magnitude and
Phase Specificiations
min k Ax − b k2
x
cremez (Matlab Signal Processing Toolbox) performs
where, A ∈ CN xM , b ∈ CN x1, and x ∈ RM x1 complex L∞ FIR filter design:
Hence we have,
• Documented online at The Mathworks
(search for cremez)
min k[R(A) + jI(A)] x − [R(b) + jI(b)]k22
x • Convergence theoretically guaranteed for arbitrary
which can be written as: magnitude and phase specifications versus frequency.
min k R(A)x − R(b) + j [I(A)x + I(b)] k22 • Reduces to Parks-McClellan algorithm (Remez second
x
or
2 algorithm) as a special case.
R(A) R(b)
min
x−
• Written by Karam and McClellan. See “Design of
x I(A) I(b)
2
Optimal Digital FIR Filters with Arbitrary Magnitude
which is written in terms of only real variables. and Phase Responses,” by Lina J. Karam and James
H. McClellan, ISCAS-96. The paper may be
Hence, we can use the standard least squares solvers in
downloaded at
Matlab and end up with a real solution.
http://www.eas.asu.edu/~karam/papers/iscas96 km.html
Related paper
• Solved by efficient primal-dual interior-point methods • Optimal Chebyshev using Remez Exchange
• Number of iterations required to solve a problem
grows at most as the square root of the problem size
• Typical number of iterations ranges between 5 and
50, almost independent of the problem size
43 44
Problem Statement Hilbert Transform
∆
y(t) = (hi ∗ x)(t) (Hilbert transform of x)
• Design a “negative-frequency filter” which converts a
real signal into its complex “analytic signal” ∆ 1
hi(t) = πt (Hilbert transform “kernel”)
counterpart by filtering out negative frequencies.
• Equivalently, design a “single sideband” (SSB) filter −j, ω > 0
∆
which outputs a single-sideband signal in response to Hi(ω) = j, ω < 0 (Hilbert frequency response)
a complex-conjugate pair of sidebands (a Hermitian
1, ω = 0
spectrum). ∆
xa(t) = x(t) + jy(t) (Analytic signal from x)
• We could also call it a “positive-frequency-pass” filter. Z ∞
• The imaginary part of such a filter is called a = π1 X(ω)ejωtdω (Note: Lower limit usually −∞)
Hilbert transform filter. 0
Proof:
• The ideal Hilbert transform is an allpass filter that
delays positive-frequency components by 90 degrees, Xa(ω) = X(ω) + jY (ω) (By linearity of Fourier transform)
and advances negative-frequency components by 90
degrees. = X+ + X− (Apply frequency response)
+j[−jX+ + jX−]
= 2X+(ω)
45 46
PSfrag replacements
ω
π
0
|H|
Kaiser Window 2π 0 Kaiser Window Transform, FFT size = 2048, Window length = 257
−π
−20
Kaiser window in causal, linear-phase form: −2π
ω −40
π −80
(N = FFT size = 2048): ωc
ω0 −100
W = fft(wzp); −ωc
−ω0
−160
−0.5 −0.4 −0.3 −0.2 −0.1 0 0.1 0.2
Normalized Frequency (cycles/sample)
0.3 0.4 0.5
47 48
eplacements
ω
π
Kaiser window transform, close-up on main lobe Oversimplified Window Method
0
|H| Sample the ideal Hilbert-transform kernel
Close Up on Kaiser LF Response, FFT size = 2048, Window length = 257
2π 0
h(t) = 1/πt
−π
−2π −20 to get
ω ∆ 1
ĥi(n) = (Sampled Hilbert transform kernel)
−40 πnT
|H| ∆
ĥw (n) = w(n)ĥi(n) (Windowed ideal impulse response)
Gain (dB)
π −60
ωu Design Parameters:
ωl −100 fs = 22050; % sampling rate (Hz)
−π T = 1/fs; % sampling period (sec)
−ωc M = 257; % FIR filter length = window length
−ω0
−120
−0.05 −0.04 −0.03 −0.02 −0.01 0 0.01 0.02
Normalized Frequency (cycles/sample)
0.03 0.04 0.05
N = 8*(M-1); % for interpolated spectral displays
beta = 8; % beta for Kaiser window
Filter Design:
n = [-N/2+0.5:N/2-0.5]; % Time axis (avoid t=0)
hi = T ./ (pi*n*T); % Sampled Hilbert kernel
hr = sin(pi*n) ./ (pi*n); % 1/2 sample delay filter
h = (hr + j*hi)/2; % Sampled ideal final filter
plot(f,fftshift(max(-100,20*log10(abs(fft(h)))))); grid;
...
49 50
eplacements
ω
π
0
FFT of Truncated(2048) Ideal Impulse Response Hwp = [Hw(N/2+2:N), Hw(1:N/2+1)]; % Neg. freqs on left
|H| Hwpn = abs(Hwp); Hwpn = Hwpn/max(Hwpn);
2π plot(f,20*log10(Hwpn)); grid;
PSfrag replacements
...
−π 20
FFT of Truncated Analytically Specified "Ideal" Impulse Response
−2π ω
ω 0 FIR
π Frequency Response by the Window Method
0
|H| -20 |H|
Length 257 Analytic−Derived FIR Frequency Response, FFT size = 2048
π 2π 0
Gain (dB)
ωc
-40 −π
ω0 −2π −20
-60
ωu ω
−40
ωl -80
|H| −60
−π
Gain (dB)
−ωc -100
-0.6 -0.4 -0.2 0 0.2 0.4 0.6
π
−ω0 Normalized Frequency (cycles/sample) ω c −80
ω0
h = [h(N/2+1:N),h(1:N/2)]; % zero-phase form (almost) ωu −100
hw = wzp .* h; % Apply window to ideal impulse response ωl
−120
Hw = fft(hw); % Frequency response we really get −π
−ωc −140
% Compute total stopband attenuation:
−ω0 −0.5 −0.4 −0.3 −0.2Normalized −0.1 0 0.1 0.2 0.3 0.4 0.5
Frequency (cycles/sample)
ierr = norm(Hw(N/2+2:N))/norm(Hw)
= 0.0378 = 3.8 percent
π −60 ∆ 1, ω ≥ 0
ωc u(ω) =
0, ω < 0
ω0 −80
PSfrag replacements
ω
need a transition band. We can work this out analytically, π Desired Frequency Response
or we can simply “draw” it in the frequency domain, take 0
a (large) inverse DFT, and window that. Since the latter |H|
Desired Frequency Response, FFT size = 2048
approach is more general, we’ll do that. 2π 0
−π
Window Method applied to Bandlimited Ideal −2π
Impulse Response ω
−50
π
f1 = 530; % lower passband limit (Hz) ωc
N = 2^(nextpow2(8*fs/f1)) % FFT size from xition width ω0 −100
if N<8*M, N = 8*(M-1); end; % spectral interpolation ωu
ωl
Bandlimited “Ideal” Hilbert transformer frequency
response (FR), including transition bands ∆ matched to −π
FFT size N such that N fs/∆: −ωc
−ω0
−150
−0.5 −0.4 −0.3 −0.2 −0.1 0 0.1 0.2
Normalized Frequency (cycles/sample)
0.3 0.4 0.5
H = [ ([0:k1-2]/(k1-1)).^8,ones(1,k2-k1+1), ...
([k1-2:-1:0]/(k1-1)).^8,zeros(1,N/2-1)];
Hp = [H(N/2+2:N), H(1:N/2+1)];
f = [-0.5 + 1/N : 1/N : 0.5]; normalized freq axis
plot(f,Hp); grid; ...
55 56
PSfrag replacements
ω
Desired Impulse Response π Real Part of Desired Impulse Response
0
|H|
Ideal SSB Filter Impulse Response, Real Part, FFT size = 2048
h = ifft(H); % zero-phase form 2π 0.6
−π
% measure of round-off error:
ierr = norm(imag(h(1:2:N)))/norm(h) % zero?
−2π 0.5
ω 0.4
ierr =
|H| 0.3
Amplitude
4.1958e-15 π
ωc 0.2
aerr =
ωl
0
−π
4.8300e-04 −ωc
−ω0
−0.1
−150 −100 −50 0
Time (samples)
50 100 150
57 58
eplacements
PSfrag replacements
ω
π ω
Imaginary Part of Desired Impulse Response π Final FIR Filter Frequency Response
0
|H| 0
Ideal SSB Filter Impulse Response, Imaginary Part, FFT size = 2048 |H|
2π 0.4 Length 257 FIR filter Frequency Response, FFT size = 2048
−π 2π 0
−2π
0.3 −π −20
−2π
ω 0.2
ω −40
|H|
Amplitude
π 0 −80
Gain (dB)
ωc π
ω0 −0.1 ωc −100
ωu ω0 −120
−0.2
ωl ωu
−140
ωl
−π −0.3
−ωc −π −160
−ω0
−0.4
−150 −100 −50 0 50 100 150 −ωc
Time (samples)
−ω0
−180
−0.5 −0.4 −0.3 −0.2 −0.1 0 0.1 0.2
Normalized Frequency (cycles/sample)
0.3 0.4 0.5
59 60
eplacements
Zoom-in
ω on Transition Region of Final Frequency Remez Multiple Exchange Method
π Response
0
|H| • Remez exchange algorithm for optimal Chebyshev
2π 20
Low−Frequency Transition Band, f1/fs=0.023926 (marked by "+") (equiripple) FIR filter design.
−π • Original reference: J. H. McClellan, T. W. Parks, and
−2π 0
L. R. Rabiner, “A Computer Program for Designing
ω −20 Optimum FIR Linear Phase Digital Filters,” IEEE
Trans. Audio Electro., vol AU-21, Dec. 1973,
|H| −40
pp. 506-526.
Gain (dB)
π −60
• Text reference: L. R. Rabiner and B. Gold, Theory
ωc
and Application of Digital Signal Processing,
ω0 −80
Prentice-Hall, 1975.
ωu −100
PSfrag replacements
passband ripple can be 0.1 dB or more. However, ω Optimal Chebyshev FIR(257) Frequency
consider an FM signal in which a sinusoid is sweeping π Response
back and forth in the passband. In that case, the 0
passband ripple generates AM sidebands, so a spec more |H|
Length 257 Optimal Chebyshev FIR Single−Sideband−Filter Freq. Response
like that in the stopband may be called for. Here, we 2π 50
circle by 90 degrees: π
ωc −100
hr = hrm .* j .^ [0:M-1];
ω0
ωu
ωl −150
−π
−ωc
−ω0
−200
−0.5 −0.4 −0.3 −0.2 −0.1 0 0.1 0.2
Normalized Frequency (cycles/sample)
0.3 0.4 0.5
63 64
eplacements
PSfrag replacements
ω
ω
π Zoom-In on Transition Band Passband Ripple
π
0
|H| 0
Low−Frequency Transition Band, f1/fs=0.023926 (marked by "+")
|H|
2π 50 x 10
−4 Passband Ripple
2π 4
−π
−π
−2π 3
0 −2π
ω
ω 2
|H| −50 1
|H|
Gain (dB)
Gain (dB)
π 0
ωc
−100 ωc
ω0 −1
ω0
ωu
ωu
ωl −150
−2
ωl
−π
−π −3
−ωc −200
−0.05 −0.04 −0.03 −0.02 −0.01 0 0.01 0.02 0.03 0.04 0.05 −ωc
−ω0 Normalized Frequency (cycles/sample)
−4
0.18 0.2 0.22 0.24 0.26 0.28 0.3 0.32
−ω0 Normalized Frequency (cycles/sample)
65 66
eplacements
ω
π
Initial Impulse Response Summary of Trade-Offs
0
|H| −4 First 50 impulse−response coefficients Window-method filter:
x 10
8
2π
−π 6 • Stopband ”droops” which ”wastes” filter taps if the
−2π worst-case stopband attenuation is the only stopband
ω 4
spec. However, such ”roll-off” is natural in the
2
frequency domain and corresponds to a ”smoother
|H| pulse” in the time domain.
Amplitude
π 0
• Passband is degraded by early roll-off. Edge is off.
ωc
ω0
−2
• Filter length can be thousands of taps as needed for
ωu −4
full audio band.
ωl
−π −6 Optimal Remez-Exchange filter:
−ωc
−ω0
−8
0 5 10 15 20 25 30 35 40 45 50
• Stopband is ideal, equiripple.
Index
67 68
• Fails to converge for filters much longer than 256 Matlab hilbert() function
taps. (Need to increase working precision to get
longer filters.) The Matlab Hilbert function returns an analytic signal
given its real part.
PSfrag replacements
Nh
ω = M-2; % This looks best
delta = [1,zeros(1,Nh)]; % zero-phase impulse
π
hh = hilbert(delta); % zeros negative-freq FFT bins
0 = fft(hh); % FIR frequency response
Hh
|H|
Frequency Response of Length 256 hilbert() using Matlab
2π 0
−π
−2π −50
ω
−100
|H| −150
Gain (dB)
π
ωc −200
ω0
ωu −250
ωl
−300
−π
−ωc
−ω0
−350
−0.5 −0.4 −0.3 −0.2 −0.1 0 0.1 0.2
Normalized Frequency (cycles/sample)
0.3 0.4 0.5
PSfrag replacements
ω
eplacements π
0
ω |H|
π = [hh(Lh/2+1:Lh), zeros(1,N-Lh), hh(1:Lh/2)];
hhzp Low−Frequency Transition Band, f1/fs=0.023926 (marked by "+")
2π 50
0 = fft(hhzp); % Frequency response
Hhzp
−π
|H| 0
Interpolated Frequency Response of Length 256 hilbert() using Matlab −2π
2π 0
ω −50
−π
−2π −50
|H| −100
ω
Gain (dB)
−100 π −150
ωc
|H| −150 ω0 −200
Gain (dB)
π ωu
ωc −200
−250
ωl
ω0
−π −300
ωu −250
−ωc
ωl −350
−0.05 −0.04 −0.03 −0.02 −0.01 0 0.01 0.02 0.03 0.04 0.05
−300 −ω0 Normalized Frequency (cycles/sample)
−π
−ωc −350
−0.5 −0.4 −0.3 −0.2 −0.1 0 0.1 0.2 0.3 0.4 0.5 Problems:
−ω0 Normalized Frequency (cycles/sample)
71 72
More generally, any real signal given to hilbert() must be
interpreted as precisely one period of a periodic signal.
73