A New Bit-Level Algorithm for General Purpose Median Filtering

K Benkrid, D Crookes School of Computer Science, The Queens University of Belfast, Belfast BT7 1NN, UK

ABSTRACT

Median filtering has proved an effective way to remove impulse noise while preserving rapid signal changes. A classic general purpose Median filter is based on a Bubble Sort approach. In this paper, we present a novel bit-level algorithm which is readily scalable and very compact. The basis of the algorithm is similar to that of Quicksort, and is based on a Bit Voter (BV) block. The paper also extends the basic BV algorithm to include Weighted Median and Ranked Order Median filtering. The median finding unit, whose inputs are the N pixel values to sort, has a O(N) hardware complexity compared to a O(N2) complexity for an equivalent unit using Bubble sort. The novel algorithm has been implemented on a Xilinx Spartan XCS30XL-4 FPGA chip. For comparison purposes, we also present an FPGA implementation of an existing Triple Input Sorter based algorithm (TIS) which is an optimised version of the Bubble sort algorithm for the special case of a 3x3 window size. The BVs median finding unit occupies 15 Configurable Logic Blocks (CLBs) only whereas TIS occupies 60 CLBs. For PAL video (720x576 images of 8-bit/pixel), comparative timings show that BV can operate at 26 frames per second (fps), while TIS can achieve 33 fps. Thus, both algorithms can achieve real time performance, and BV is the more compact.

1 INTRODUCTION

Median filtering has proved an effective way to satisfy the dual requirements of removing impulse noise while preserving rapid signal changes [1,2]. A median filter is a non linear digital filter which, in the two dimensional case, is based upon moving a PxQ window over an image (as in, for instance, convolution) and computing the output pixel as the median value of the pixel values within the input window. The median value replaces the pixel value at the centre of the window. If P*Q is odd and the samples sequence at a particular position of the window is S = (S1, S2, , SP.Q), then the median M is a function F of this sample sequence, M = F(S) = F(S1, S2, , SP.Q) and is equal to the (P*Q+1)/2 element in the list of ordered samples values. Sometimes, image processing application developers would like to vary the likelihood of a particular pixel position being selected by giving a weight to the corresponding window position. This leads to a more general form of median filtering called weighted median filtering. Suppose the weight sequence W = (W1, W2, , WP*Q), where Wi is the weight associated to sample Si. The weighted median WM is given by: WM = F( S1 ,4243 , S2 ,4243, ........., Si ,4243 , ......, SP*Q , SP*Q ,....., SP*Q ) S ,....., S1 S2 ,....., S2 S ,....., Si 1 1 14 4 1 i 144 2444 4 3
W1 W2 Wi WP*Q

where F is the median function defined above. Note that the median filter is a special case of this more general weighted median filter where W = (1, 1, 1, .1). The classic median filter is also a subset of a more general set of filters, called rank order filters, which are themselves a subset of stack filters [1]. As in median filtering, a 2-D rank order filter consists in passing a PxQ window over an image and computing a result pixel from the sample pixel values S = (S1, S2, , SP*Q) within the window. In the case of a rank order filter of rank R, however, the output pixel RM = FR(S1, S2, , SP*Q) is equal to the Rth element in the list of ordered samples values. For instance if R =1, the rank order filter

function FR reduces to a minimum function whereas if R=P*Q it reduces to a maximum function. Note that if R = P*Q/2 (i.e. middle of the ordered sample sequence), the rank order filter function FR is equivalent to the classic median filter function F defined above. Classic hardware implementations for general purpose Median filtering are based on a Bubble Sort approach [3,4], which uses a bubble sorter over the entire window elements to find the median value. In this paper, we present a new bit level median algorithm (called Bit Voter or BV) which is highly scalable and very compact. In this following, we firstly outline the general structure of a 2D median filter architecture, irrespective of which median finding algorithm is used. Then we present an FPGA implementation of an existing Triple Input Sorter based algorithm (TIS) which is an optimised version of the Bubble sort algorithm for the special case of a 3x3 window size [5,6]. After that, we present our new bit level median BV algorithm. The latter is based on a Bit Voter block, which outputs the most common bit among a set of input bit candidates. The basic idea of this algorithm was first published in 1999 [7]. This paper gives an improved hardware implementation of the BV algorithm on the low cost Xilinx Spartan FPGA family, and presents two extensions of the basic BV algorithm: one for weighted median filtering and another for ranked median filtering. Comparative timings and area measurements for the FPGA implementation of the two algorithms (TIS and BV) are also given.

2 BASIC ARCHITECTURE OUTLINE FOR A 2D MEDIAN FILTER

For the two dimensional (2D) case, a median filter consists of sliding a PxQ window over an image, and computing the output pixel as the median value of all the pixel values (brightness) within the input window. In a hardware implementation, we would like to supply each pixel only once to the chip and get a result pixel every clock cycle. This is indeed how data is

received from a video camera. A pipelined architecture is hence required, with line buffers needed to hold complete image columns (or rows depending on the image scan direction). The idea is that these internal line buffers are used to synchronise the supply of input values to the median filter ensuring that all the pixel values involved in a particular window position are processed at the same instance [8]. Figure 1 shows the outline structure of a 2D median filter for a PxQ window size. A result pixel is generated every clock cycle regardless of the input data length and value.

D
Line BufferQ

Median

Finding D
Line Buffer1

D Unit

Serial/Parallel converter Output Image

D
Parallel/Serial converter

D
P delay elements

Input Image

Figure 1 Outline structure of a bit-level 2D median filter for a PxQ window It is the design of the Median Finding Unit, whose inputs are the PxQ pixel values and whose output is the median value of these elements, which will differentiate the two following algorithms.

BUBBLE SORT MEDIAN FILTER AN IMPLEMENTATION FOR A 3X3 WINDOW

Figure 2 presents a systolic array architecture of the Median Finding Unit of 5 inputs based on the Bubble sort algorithm [4]. In the latter, the input array goes through several stages. In

each stage, successive pairs of elements are compared (see the Dual Input Sorters (DIS) elements in Figure 2). The maximum element bubbles to the top whereas the minimum one bubbles to the bottom. Hence, the maximum input within the whole input array bubbles through the stages to the top of the last stage whereas the minimum bubbles down to the bottom. The last stage presents the input array data sorted in an increasing order from bottom to top. In general, for a total of (2N+1) inputs, the architecture uses (2N+1) pipelined stages each requiring N dual input sorters, hence a hardware complexity (i.e. area) O(N2)

Figure 2 Architecture of a Bubble sorter Note that if we are only interested in the Median value, then there is no need for sorting the whole window. An optimised version of the Bubble sorter is presented in Figure 3, for the special case of a 3x3 window. The architecture is based on a Triple Input Sorter [5,6] and consists of the following stages: Stage 1: Stage 2: Stage 3: Sorts the elements of each window column. Sorts the elements of each window row. The median is finally obtained by sorting the cross diagonal and picking up the middle value. Only three results are guaranteed to be placed in the right place: the maximum, the minimum and the median; this saves a great deal of logic. The main component in this design is a Triple Input Sorter (which is based on a dual input sorter), as described in Figure 3.

max

max

max

DIS

min

DIS

min

Triple
med

Line Buffer

Input
D D

max

DIS

min

min

Sorter Triple Input Triple Input Sorter

Triple Input Sorter

Triple
Line Buffer

Input Sorter
D D

Serial/Parallel converter

Sorter Triple Input

Parallel/Serial converter

Sorter

Figure 3 Block diagram of a 3x3 Triple Input Sorter based median filter design Note that because the same operation (Triple Input Sort) is applied to the elements of each column, it is possible to use only one column sorter. A 3x3 bit-level TIS-Based median filter has been implemented for 720x576 input images of 8-bit/pixel (PAL video) on a Xilinx Spartan XCS30XL-4 FPGA chip (24x24 CLBs). A serial Triple Input Sorter block occupies 12 CLBs; the Median Finding Unit occupies 60 CLBs, and the whole circuit takes 401 CLBs. The corresponding FPGA floorplan is shown in Figure 4. The circuit can operate at 110 MHz. This would enable a frame rate processing of 33 frames per second.

Figure 4 Physical configuration of the TIS median filter on XCS30XL-4

4 THE BV ALGORITHM

We now present the novel BV algorithm for a PxQ 2D median filter. We will first describe the general principle of BV, and then discuss its implementation. The description assumes that pixels are unsigned. Observe that if the candidate values (9 for a 3x3 window) are compared with a pivot value and split into two groups (< and >), then the median value must be in the larger group. The smaller group can then be effectively eliminated from the set of possible candidates. If we process all the candidate values together in bit serial, Most Significant Bit (MSB) first, fashion, then this partitioning can be achieved by choosing just one pivot bit at each stage. To implement this approach, the pixels are therefore fed Most Significant Bit First to a Voter Block. This block will examine the candidate bits, and finds which is the larger group by voting i.e. by finding the most common bit among them (by counting the number of 1s). Assuming an odd number of candidates, clearly the median pixel will be among the group that contains more elements than the other. Hence, the output bit at any stage is the most common bit, which has just been found. The other group will be automatically eliminated from the race by latching each of the remaining candidates bits either to 0 (if it was in a 0s group) or to 1 in the opposite case. This latching method of elimination does not affect subsequent vote results, since all that it does is replace the weak candidates (lower pixel values) by even weaker ones, and strong candidates (higher pixel values) by even stronger ones. An example trace of the algorithm for finding the median of five values is shown in Figure 5. This shows candidate pixels progressively being eliminated, and the eliminated bits being substituted with bit values depending on why they were eliminated.

13
LSB 1 0 1 1 0

9
1 0 0 1 0 0

6
0 1 1 0 0 0

4
0 0 1 0 0 0

14
0 1 1 1 0 0

13
1 0 1 1 1

9
1 0 0 1 1

6
0 1 1 0 0

4
0 0 1 0 0

14
0 1 1 1 1

13

14

MSB

1 0 1 1

1 0 0 0

0 1 1 0

0 0 1 0

0 1 1 1

Bit 0 candidates

Bit Voter (Count(1's) > 2) 0

Bit Voter (Count(1's) > 2) 01

Bit Voter (Count(1's) > 2) 010

13

14

13

14 Eliminated candidate Elimination bit

1 0 1

1 0 0

0 1 0

0 0 0

0 1 1

1 1

1 1

0 0

0 0

0 1

Bit Voter (Count(1's) > 2)

Bit Voter (Count(1's) > 2) 0 1 0 0 1 Median of {4, 6, 9, 13, 14} = 010012 = 9

010 0

Figure 5 Illustration of BV median filtering algorithm A hardware architecture based on the BV algorithm is given in Figure 6.
PE PE PE

D
Line BufferQ

PE

B I T V O T E R

PE PE PE

D
Line Buffer1

PE PE PE PE

Serial/Parallel converter Output Image

D
Parallel/Serial converter

D
P delay elements

PE

FDBK

Input Image

Figure 6 Block diagram of the novel algorithm design The hardware implementation of the novel algorithm consists of two stages:

1.

An array of pixel Processing Elements (PEs): Each PE takes the next bit of its input pixel. It will first determine the bit value to be passed to the voting stage: this will be either its input pixel bit value, or the latched elimination bit if the pixel has been eliminated. When the next result bit is subsequently fed back (from the Bit Voter stage), each PE decides whether or not its future candidate bits are to be eliminated or not. If the input pixel is to be eliminated, the PEs output is latched to the elimination bit value. The structure of the Processing Element is given by Figure 7.
Bit_in

FDBK
Comparator

S 1 Bit Hold Value

Elimination_bit

WHILE (FDBK = Bit_out) DO { // Switch S to position 1. Bit_out <= Bit_in ; } // ELSE switch S to position 2. Bit_out <= Elimination_bit;

Bit_out

Figure 7 BVs Processing Element structure 2. Bit Voter stage (BV): This block receives as its input the entire set of processed candidate bits, and outputs the most common bit among them (1 or 0) as the next bit of the result. This is done simply by counting the number of 1s and comparing the result with a threshold value which, in the case of a 3x3 window, will be five. In the general case of a PxQ window, the threshold value will be P*Q/2. Note that the proposed BV algorithm works just as well with even-length median filters. In this case, the median value will be the candidate of rank N/2-1 in the sorted list of input candidates (assuming an even N-length median filter). Counting the number of 1s in an array can be performed by counting the number of 1s in small groups of m elements and using a tree of adders to sum the partial count results as shown in Figure 8 for m = 4. This value is suitable for FPGAs that are based on 4-input Look-Up-Tables (LUTs) such as Xilinx XC4000, Spartan and Virtex chips. In general,

10

the hardware complexity of the Bit Voter unit is O(N) where N is the number of input candidates.
4
Count 1s
Adder

PxQ Input bit candidates

Adder

> P*Q/2

4
Count 1s

3
Adder

Median bit

4
Count 1s

Figure 8 Architecture of Bit Voter unit (m = 4) In the case of 3x3 window, two 4 3 LUTs are needed to count the numbers of 1s in two groups of 4 inputs. The two partial results are then added using a 3 bits adder with the 9th input used as initial carry-in. The final count result is represented in 4 bits and can then be thresholded (compared to 4) using just one 4 1 LUT (see Figure 9).
3 4 9 Input bit candidates Count 1s 4 4 3 Count 1s Adder
Cin

Median bit >4

Figure 9 Implementation of a 9 input Bit Voter unit Since one Processing Element is associated with every input pixel, increasing the window size will increase the number of PEs proportionally. Increasing the window size also implies some additional hardware in the Bit Voter unit. On Xilinx Spartan FPGAs, the PE block consumes just one CLB; the Bit Voter block takes 6 CLBs for a 3x3 window size. The scalability and the compactness of this algorithm make it an ideal candidate for a general purpose median filter. A 3x3 BV-based median filter has been implemented for 720x576 input images of 8bit/pixel (PAL video) on a Xilinx Spartan XCS30XL-4 FPGA chip (24x24 CLBs). The

11

Median Finding Unit occupies just 15 CLBs. Compared to 60 CLBs for the equivalent TISbased unit, it can be seen that the Bit Voter Median Finding Unit consumes 75% less hardware than TIS. Moreover, the hardware complexity of BV is O(N) compared to O(N2) for Bubble sort. The whole BV-based median circuit occupies 356 CLBs. The corresponding FPGA floorplan is shown in Figure 10. The circuit can operate at 87 MHz. This slight decrease in the speed performance, compared to TIS, is due to the feedback loop. However, this speed still allows real time median filtering (26 frames per second). The above implementation results are summarised in Table 1. Total Area (CLBs) Triple Input Sorter Algorithm (TIS) Bit Voter Algorithm (BV) 401 356 Median Finding Unit area (CLBs) 60 15 Speed (MHz) 110 87 Frame rate (Frames/Sec) 33 26

Table 1 Area and performance comparison of the Triple Input Sorter and the Bit Voter median filter implementations

Figure 10 Physical configuration of the BV median filter on XCS30XL-4

12

5 EXTENDING THE DESIGN FOR WEIGHTED AND RANKED ORDER MEDIAN FILTERING

In this section, we present two extensions of the basic BV algorithm: one for weighted median filtering and another for ranked median filtering.

5.1

Weighted Median

As explained in the introduction, in Weighted Median filtering the user can vary the likelihood of a particular pixel position being selected by giving weights to each window position. This is illustrated by the window shown in Figure 11 and the corresponding 3x3 neighbourhood from the image being filtered (sample sequence).
1 2 1 2 3 2 1 2 1 a d g b e h c f i

Figure 11 Window with weights and corresponding 3x3 image neighbourhood The Weighted Median filter will therefore find the median of 15 numbers in this case: { a, b, b, c, d, d, e, e, e, f, f, g, h, h, i } This suggests a simple alteration to our BV architecture above to implement a Weighted Median Filter, in which the input bit pixel candidates are first replicated (by a factor given by the weight in the corresponding window position) before being input to a larger Bit Voter block. This will have the effect of increasing the likelihood of each pixel being selected by the corresponding weighting factor. The number of inputs to the Bit Voter block will be the sum of the weights in the window. The architecture corresponding to the weighted window in Figure 11 is shown in Figure 12. The standard median filter is equivalent to the simple case in which all the weights are 1.

13

a
PE

b
PE

c
PE

d
PE

e
PE

f
PE

g
PE

h
PE

i
PE

Count (1s) > 7

Figure 12

Modified Architecture for Weighted Median Filter

Although the architectural extension is straightforward, and requires only a scalable design for the Bit Voter block, there are performance implications when using weights larger than 1 because of the feedback loop necessary for eliminating candidate pixels. Since we use a binary tree structure for the bit voter, the clock period will depend on log2( Wi ) (where Wi are the weights associated to the pixel samples), without allowing for routing delays.

5.2

Ranked Order Median

The standard median filter conceptually ranks all the values in a neighbourhood, and selects the middle value (e.g. the 5th value for a 3x3 window). A generalisation of this is to select the Rth ranked value (where R = P*Q/2 finds the standard median, R = 1 finds the minimum, and R = P*Q finds the maximum). This is called the Ranked Median filter. A simple modification to our BV architecture is sufficient to implement this more general form of the median filter. For standard median, the BV block counts the number of 1 bits and compares this to P*Q/2. This is just a special case of the threshold value necessary for the Rth ranked value. We generalise the BV block so that it counts the number of 1 bits and compares this count with P*Q-R. By supplying any appropriate value of R, a user can obtain any required ranking. The architecture is outlined in Figure 13. There is no extra hardware or performance penalty for generalising to Ranked Order median filtering.

14

PE

PE

PE

PE

PE PE

Count (1s) > P*Q - R

Figure 13 Modified architecture for Rank R Median filter Figure 14 illustrates the ranked-median algorithm for 5 input pixels and a rank R = 2.
8
LSB 0 0 0 1 0

27
1 1 0 1 1 1

9
1 0 0 1 0 0

10 21
0 1 0 1 0 0 1 0 1 0 1 1

8
0 0 0 1 1

27
1 1 0 1 1

9
1 0 0 1 1

10 21
0 1 0 1 1 1 0 1 0 1

27

10 21

MSB

0 0 0 0

1 1 0 1

1 0 0 0

0 1 0 0

1 0 1 1

Bit 0 candidates

Bit Voter (Count(1's) >3 0

Bit Voter (Count(1's) >3) 01

Bit Voter (Count(1's) >3) 010

27

10 21

27

10 21

0 0 0

1 1 1

1 0 0

0 1 1

1 0 1

0 0

1 1

1 1

0 1

1 1

Bit Voter (Count(1's) >3)

Bit Voter (Count(1's) >3) 01001 Rank 2 Median of {8, 9, 10, 21, 27} = 010012 = 9

0100

Figure 14 Illustration of ranked-median filtering algorithm for P*Q = 5 and R=2 The above extensions to the algorithm can be combined, and used to give a very general median filter which does both Weighted Median and Ranked Median filtering.

6 CONCLUSION

This paper has presented a novel algorithm for the generalised median filter operation based on a Bit Voter. It has compared an implementation of the new algorithm with an existing

15

Triple Input Sorter based approach for the special case of a 3x3 window. Both algorithms were implemented on a Spartan XCS30XL-4 FPGA chip. The major advantage of the new algorithm is that it is very compact. The hardware complexity of a median finding unit based on BV algorithm is O(N) compared to a O(N2) complexity for an equivalent unit using the classic Bubble sort approach. Another major advantage is that the new algorithm in its general form also provides a Weighted Median filter (with some performance cost, which depends on the sum of the weights), and a Ranked Order Median filter (with no performance overhead). These extensions retain the properties of scalability and flexibility. Disadvantages of BV include the fact that it is slower than the existing Bubble sort algorithm because of the feedback loop. Nonetheless, a real time implementation of the BV algorithm for a 3x3 window has been achieved for PAL video (720x576 images of 8-bit pixels) on a Spartan XCS30XL-4 FPGA chip.

REFERENCES

[1] K. R. Castleman, Digital Image processing, Prentice Hall, 1995, ISBN: 0132114674. [2] Al Bovik, Handbook of image and video processing, Academic Press Series in Communications, Networking and Multimedia, ISBN: 0121197905. [3] L.A. Christopher, W.T. Mayweather and S.S. Perlman, VLSI median filter for impulse noise elimination in composite or component TV signals, IEEE Trans. On consumer Electronics, Vol. 34, no. 1, pp. 263-267, 1988. [4] K. Olafzar, Design and implementation of a single chip 1-D median filter, IEEE Trans. Acoustic Speech and Signal Processing, ASSP-31, pp. 1164-1168, Oct. 1983. [5] Bates_GL, Nooshabadi_S, FPGA implementation of a median filter, IEEE Region 10 Annual International Conference, Proceedings/TENCON, 1997, Vol.2, pp.437-440.

16

[6] Xilinx Application Notes, Implementing Median Filters in XC4000E FPGAs, Q4 1996. [7] K. Benkrid and D. Crookes, Design and FPGA implementation of a novel general purpose median filter, Proceedings of the Irish Machine Vision and Image Processing Conference IMVIP99, Dublin, September 1999. pp.280-287. [8] W. Kamp, H. Kunemund, H. Soldner and H. Hofer, Programmable 2D linear filter for video applications, IEEE journal of Solid State Circuits, pp 735-740, 1990.

17