fusion image

© All Rights Reserved

0 tayangan

Final Version

fusion image

© All Rights Reserved

- Physics
- Image Fusion and Guided Filtering
- physics module F4 (light)
- 9 ,10 Icse Physics
- SBES Final Report
- Phyiscs SPM revision questions
- ch1 stp
- Convex Lens
- New Microsoft Word Document (2)
- 12_physics_lyp_2016_allindia_set2
- Guia Do ProfessorFQ8
- Refraction of Light at Plane Surfaces
- 47591139 Yearly Lesson Plan Science Form 4 2011
- QU Balloon 2008 Report
- PHY131 Slide 8 Optics
- Phy Test Series Syllbs
- 7 Geometrical Optics - Lenses
- 12_physics_lyp_2016_allindia_set2.pdf
- light - reflection and refraction 10th phy.pdf
- Biswas Science 1

Anda di halaman 1dari 14

Pattern Recognition

journal homepage: www.elsevier.com/locate/pr

Zhaobin Wang a,b, Yide Ma a,, Jason Gu b

a

School of Information Science and Engineering, Lanzhou University, Lanzhou 730000, China

b

Department of Electrical and Computer Engineering, Dalhousie University, Halifax, Nova Scotia, Canada B3J 2X4

a r t i c l e in fo abstract

Article history: This paper proposes a new method for multi-focus image fusion based on dual-channel pulse coupled

Received 13 May 2009 neural networks (dual-channel PCNN). Compared with previous methods, our method does not

Received in revised form decompose the input source images and need not employ more PCNNs or other algorithms such as

24 November 2009

DWT. This method employs the dual-channel PCNN to implement multi-focus image fusion. Two

Accepted 17 January 2010

parallel source images are directly input into PCNN. Meanwhile focus measure is carried out for source

images. According to results of focus measure, weighted coefﬁcients are automatically adjusted. The

Keywords: rule of auto-adjusting depends on the speciﬁc transformation. Input images are combined in the dual-

PCNN channel PCNN. Four group experiments are designed to testify the performance of the proposed

Image fusion

method. Several existing methods are compared with our method. Experimental results show our

Focus measure

presented method outperforms existing methods, in both visual effect and objective evaluation criteria.

Finally, some practical applications are given further.

& 2010 Elsevier Ltd. All rights reserved.

1. Introduction based on the selection of image blocks from source images [14].

The idea of this scheme is to select the better image blocks from

Optics of lenses with a high degree of magniﬁcation suffers source images to construct the fused image. Clearly, the latter is

from the problem of a limited depth of ﬁeld. The larger the focal simpler than the former. However, the latter has some disadvan-

length and magniﬁcation of the lens, the smaller the depth of ﬁeld tages. For instance, the effect of this scheme depends on focus

becomes. As a result, fewer objects in the image are in focus. measurement to a great extent. Some errors in the fused image

However, sometimes people like to get the image in which more are obtained by this scheme. Therefore, the third scheme is

objects are in focus. This is to say that people want to see employed in the paper.

more and clearer objects in one image. The perfect state is that PCNN is a biologically inspired neural network based on the

the whole image is clear or in focus. This directly leads to the work by Eckhorn et al. [4]. Pioneering work in the implementation

appearance of multi-focus image fusion technology. of these algorithms was done by Johnson and his colleagues

Multi-focus image fusion is the process in which different [11,10,9,21]. It has been proven that PCNN is widely applicable in

images with different focus settings are fused to produce a new the ﬁeld of image processing [18,19] such as image segmentation,

image with extended depth of ﬁeld. Its purpose is to attempt to image enhancement, pattern recognition, etc. PCNN also plays an

increase the apparent depth of ﬁeld through the fusion of object important role in image fusion. In fact, Broussard et al. [2] applied

within several different ﬁelds of focus. Hence it plays important PCNN into image fusion for object detection as early as 1999. In

roles in many different ﬁelds such as biomedical imaging and the same year Johnson and Padgett [9] pointed out that there was

computer vision. Multi-focus image fusion is one of the main the great potential for PCNN in the ﬁeld of image and data fusion.

research ﬁelds of image fusion. Many multi-focus image fusion algorithms based on PCNN

Nowadays, there are two common schemes to be used in the [7,8,13,20,2,16] have been published in different journals or

ﬁeld of multi-focus image fusion. The one is to use multi- proceedings so far. Although different authors have different

resolution approaches which usually employ the discrete wavelet schemes, most of them employ the same characteristic of PCNN,

transform (DWT) [15,22] or various pyramid algorithms such as namely, the mechanism of synchronous pulse bursts.

contrast pyramid [25], FSD pyramid [1], gradient pyramid [3], For multi-focus image fusion, different from other methods,

morphological pyramid [24], etc. However, this scheme is PCNN has its own advantages. Firstly, PCNN model derives from

complicated and time-consuming to implement. The other is the research on cat visual cortex. Its mode of information

processing is much closer to the mode of human visual

processing. And then PCNN has the ﬂexible structure which can

Corresponding author. be changed according to different tasks. Additionally, the existing

E-mail addresses: zhaobin_wang@hotmail.com (Z. Wang), PCNN methods also show PCNN has the higher performance.

ydma@lzu.edu.cn (Y. Ma). However, the problem is that the existing PCNN methods are still

0031-3203/$ - see front matter & 2010 Elsevier Ltd. All rights reserved.

doi:10.1016/j.patcog.2010.01.011

ARTICLE IN PRESS

2004 Z. Wang et al. / Pattern Recognition 43 (2010) 2003–2016

complicated and time-consuming. After careful analysis, we ﬁnd: feeding). The linking receives local stimulus from the output of

(1) almost all the methods with PCNN employ more PCNNs surrounding neurons, while the feeding, besides local stimulus,

[7,8,13,20] or combine with other algorithms such as DWT [2,16]; still receives external stimulus.

(2) almost authors adopt the simpliﬁed PCNN model and all In the following expressions, the indexes i and j refer to the

models have one stimulus. Generally an intricate algorithm pixel location in the image, k and l refer to the dislocation in a

usually costs much time to compute and operate intermediate symmetric neighborhood around one pixel, and n denotes the

variables so as to make the whole system inefﬁcient. From (1) we current iteration (discrete time step). Here n varies from 1 to

know that their schemes are very complicated, it is why these N (N is the total number of iterations)

existing methods are not efﬁcient. From (2) we believe that only X

Fij ½n ¼ eaF Fij ½n1 þ VF wijkl Yij ½n1 þSij ð1Þ

having one stimulus is the root cause of complication and k;l

inefﬁciency of PCNN methods. In other words, to some extent,

X

the standard PCNN structurally limits its application in image Lij ½n ¼ eaL Lij ½n1 þVL mijkl Yij ½n1 ð2Þ

fusion. In order to make PCNN more suitable for image fusion, we k;l

improve the standard PCNN and propose the dual-channel PCNN,

which can solve the problem of complication and inefﬁciency of Uij ½n ¼ Fij ½nð1 þ bLij ½nÞ ð3Þ

PCNN methods very well. (

Compared with previous methods, this method with the dual- 1; Uij ½n 4 Tij ½n

Yij ½n ¼ ð4Þ

channel PCNN does not decompose the input source images and 0; otherwise

need not employ more PCNNs or combine with other algorithms

such as DWT. So its scheme of image fusion is very simple and Tij ½n ¼ eaT Tij ½n1 þ VT Yij ½n ð5Þ

experimental results also show the proposed method is feasible

The dendritic tree is given by Eqs. (1)–(2). The two main

and efﬁcient.

components F and L are called feeding and linking, respectively.

The rest of the paper is organized as follows. In Section 2, the

wijkl and mijkl are the synaptic weight coefﬁcients and S is the

standard PCNN is brieﬂy reviewed, and then the dual-channel

external stimulus. VF and VL are normalizing constants. aF and aL

PCNN is introduced in details. Section 3 concretely describes our

are the time constants; generally, aF o aL. The linking modulation

proposed multi-focus image fusion algorithm based on the dual-

is given in Eq. (3), where Uij[n] is the internal state of the neuron. b

channel PCNN. It mainly includes focus measurement introduc-

is the linking parameter and the pulse generator determines the

tion, principle and implementation of algorithm, and other

ﬁring events in the model in Eq. (4). Yij[n] depends on the internal

relative contents. Section 4 gives experimental results and

state and threshold. The dynamic threshold of the neuron is

performance evaluation, and then introduces some practical

Eq. (5), where VT and aT are normalized constant and time

applications. Conclusions are summarized in the end.

constant, respectively. Here is a brief review of the standard PCNN.

The detailed description of the implementation of the standard

PCNN model on digital computers can be found in literature [9].

2. PCNN model

More details about PCNN will be found in the literatures [18,19].

2.2. Dual-channel PCNN model

PCNN model, ﬁrstly PCNN model is brieﬂy introduced. After

analyzing its standard model, we will improve the model

Analysis of the PCNN exposes a defect preventing one PCNN

according to practical demands of multi-focus image fusion.

from fusing multi-focus images. In this case, we modify the

standard model and propose a new improved model which

2.1. Standard PCNN model overcomes some limits of the standard model in multi-focus

image fusion.

In the standard PCNN model, the PCNN neuron consists of The dual-channel neuron model (see Fig. 2) consists of three

three parts: the dendritic tree, the linking modulation, and the parts: the dentritic tree, information fusion pool and the pulse

pulse generator, as shown in Fig. 1. generator. The function of dentritic tree is to receive the stimulus

The role of the dendritic tree is to receive the inputs from two including external inputs and surrounding neuron stimuli;

kinds of receptive ﬁelds. Depending on the type of the receptive information fusion pool is the place where all data are fused;

ﬁeld, it is subdivided into two channels (the linking and the the pulse generator is to generate the output pulse.

surrounding

neurons

Si j

exp( −α F ) Threshold

VF exp( −α T ) VT

W Ti j

Fi j

Feeding

1 Ui j Yi j

Linking

Li j β

M

VL

exp(−α L )

ARTICLE IN PRESS

Z. Wang et al. / Pattern Recognition 43 (2010) 2003–2016 2005

surrounding

neurons SijA β ijA

1 Threshold

A

H ij exp( −α T ) VT

W σ Ti j

Ui j Yi j

M

H ijB

1

S B

ij β B

ij

In the improved model there are two input channels. So both (5) If all neurons have ﬁred, go to the next step, else go back to

stimuli can be input into the model at the same time. The step (4).

following expressions describe its mathematical model: (6) O is the output of the dual-channel PCNN.

X

HijA ½n ¼ SAij þ wijkl Ykl ½n1 ð6Þ

Note, ‘‘a neuron ﬁres’’ means a PCNN’s neuron generates a pulse.

k;l

Here we would like to explain the principle of data fusion

X

HijB ½n ¼ SBij þ mijkl Ykl ½n1Þ ð7Þ using the dual-channel PCNN. Eqs. (6)–(8) mathematically show

k;l the way of data fusion. For one neuron, the ﬁnal output value is

A B not completely determined by weighting coefﬁcients (bA and bB).

Uij ½n ¼ ð1þ bij HijA ½nÞð1 þ bij HijB ½nÞ þ s ð8Þ

The output from surrounding neurons also plays a role via the

( mechanism of synchronous pulse bursts and this output is

Uij ½nSurij ½n; Uij ½n 4 Tij ½n1

Yij ½n ¼ ð9Þ unexpected and even random for one neuron. The mechanism

0; otherwise of synchronous pulse bursts, implemented by time-domain

( iterative processing, can make neurons with similar state

eaT Tij ½n1; Yij ½n ¼ 0 generate synchronous pulses; hence, the way of PCNN fusing

Tij ½n ¼ ð10Þ

VT ; otherwise data is not linear but nonlinear.

For the dual-channel PCNN model, it inherits good features

Compared with the standard PCNN, the dual-channel PCNN

from the standard PCNN. For example, the dual-channel model

has few parameters. In the dual-channel model, instead of feed

remains the mechanism of synchronous pulse bursts. The

channel (F) and link channel (L), HA and HB stand for two

exponential attenuation characteristic of the threshold is also

symmetrical input channels. bA and bB are the weighting coefﬁ-

kept. It is believed that this characteristic is coincident with

cients of two symmetrical channels, s the level factor to adjust the

human visual characteristic. These remaining features are

average level of internal activity. When s = 1, UZ0. Parameters

propitious to image processing.

(U, T, Y, VT, wijkl, mijkl, and aT) have the same meanings as these in

the standard model. Sur denotes the input from surrounding

P

neurons. Generally, kijkl ¼ wijkl ¼ mijkl ; Surij ¼ k;l kijkl Ykl ½n1. 3. Image fusion algorithm

Now we describe the data fusion process of the dual-channel

PCNN. Firstly, two channels of neuron receive external stimuli and In the section, multi-focus image fusion scheme used in the

output of surrounding neurons. Furthermore, the data from these paper is introduced. Image sharpness measure is so important for

channels are weighted and mixed in the information fusion pool multi-focus image fusion that some common evaluation methods

according to the weighting coefﬁcients. Finally, the mixed data of image sharpness are given in order to make the paper

are released by neuron as its output with the attenuation of the understandable and logical.

threshold. The implementation of the dual-channel PCNN in our

experiments is described as follows: 3.1. Image focus measure

(1) Initialize parameters and matrices. U=O = Y= 0, T= 1. The For multi-focus image fusion, we usually select one or more

Initialization of W and M are different from those of other focus measure methods to estimate image sharpness. So, it is

matrices. Here K= W=M. Its values are determined manually. crucial to choose a good focus measure method.

(2) If SA =SB, then O = SA or SB; and go to step (6) The focus measure deﬁned in literatures [12,17] is a maximum

(3) Normalize external stimuli to lie within [0, 1]. for the best focused image and it generally decreases as the

(4) Sur= YK; defocus increases. That is, the focused image should produce

HA = SA +Surij; maximum focus measures, while the defocused should produce

HB = SB + Sur; minimum focus measures.

U= (1 + bA HA) (1+ bB HB)+ s; Recently, there are many focus measure techniques. Here are

If Uij 4Tij then Yij = Uij Surij, else Yij = 0; some focus measure methods: variance, energy of image gradient

If S Aij ¼ S Bij or bAij ¼ bBij , then Oij ¼ S Aij or S Bij ; else Oij = Yij; (EOG), energy of Laplacian of the image (EOL), sum-modiﬁed-

If Yij = 0 then Tij =exp( aT) Tij, else Tij = VT; Laplacian (SML), and spatial frequency (SF). Huang and Jing [7,8]

ARTICLE IN PRESS

2006 Z. Wang et al. / Pattern Recognition 43 (2010) 2003–2016

assessed these methods according to some objective standards. weighting coefﬁcients [5]. This method is explained in details in

Experiment results show that SML and EOL can provide better the following paragraphs.

performance than other focus measures. However, SML spends Suppose source images IA(i, j) and IB(i, j) turn into measured

more implementation time under the same conditions. Therefore, images MA(i, j) and MB(i, j) after focus measure. The difference

EOL is used in the paper to measure image focus. Because between MA(i, j) and MB(i, j) is deﬁned by: D(i, j)=MA(i, j)–MB(i, j).

evaluation of different focus measure methods is not our topic of Generally speaking, if D(i, j)40, it indicates the pixel value at

this paper, more details can be found in the Refs. [7,8]. For EOL, its location (i, j) in image A should be chosen. Otherwise, its counterpart

corresponding mathematic expressions are also given as follows. from B is selected. However, this measure alone in practice is not

Let f(i, j) denote the gray level intensity of pixel (i, j). sufﬁcient to pick out the better focused image on a pixel-by-pixel

XX basis, since using single pixel information to make decision is

EOL ¼ ½f ði1; j1Þ4f ði1; jÞf ði1; j þ 1Þ vulnerable to wide ﬂuctuations caused by outer environment such

i j

as various noises. So, it is necessary to maintain robustness of the

4f ði; j1Þ þ 20f ði; jÞ4f ði; j þ 1Þf ði þ1; j1Þ algorithm through more information from neighboring pixels.

4f ði þ 1; jÞf ði þ1; j þ1Þ2 ð11Þ In order to make full use of surrounding information, it is

necessary to require summing the D(i, j) s over a (r+ 1) (r + 1)

This method carries out focus measure by analyzing high region surrounding each decision-point

spatial frequencies associated with image border sharpness, and it r=2

X r=2

X

is implemented through Laplacian operator. Dði; jÞ ¼ Dði þ m; j þ nÞ ð12Þ

m ¼ r=2 n ¼ r=2

3.2. Principle of multi-focus mage fusion

1

bAij ¼ ð13Þ

Now we explain how to fuse multi-focus image using the dual- 1þ eZDði;jÞ

channel PCNN. Suppose IAand IB are two multi-focus images with the and

same size. IA(i, j) and IB(i, j) denote the pixels of IAand IB in the same

position, respectively. The goal of multi-focus image fusion is to get 1

bBij ¼ ð14Þ

clear one from IA(i, j) and IB(i, j). Actually, from Section 2.2, we know 1þ eZDði;jÞ

the dual-channel PCNN can carry on data fusion. Because the data where Z is a constant. Constant Z has an important inﬂuence on

from two channels are weighted according to the weighting weighting coefﬁcients bA and bB (shown in Fig. 3). Larger Z

coefﬁcients, as long as clear pixel get large weighting coefﬁcient markedly increases the difference between bA and bB while

while blurred pixel get small one, the dual-channel PCNN can reducing Z will shrink this difference. Typically, when Z = 0, bA = bB.

achieve the purpose of multi-focus image fusion. Hence, adjusting Z will change the trend of bA and bB and their

Therefore, the core of this method is how to make sure that the difference. Because weighting coefﬁcients play a crucial role in

change of the weighting coefﬁcients depends on the clarity of the multi-focus image fusion, constant Z is usually set via several

input stimuli. Here a new method is employed to implement the experiments to meet practical demands.

transformation from the importance of input stimuli to the

1

0.8 tation of our proposed multi-focus image fusion algorithm based on

η = 0.05 βA the dual-channel PCNN in this section. The dual-channel PCNN used

0.6 in our experiments is a single layer two-dimensional array of

β ij

laterally linked neurons and all neurons are identical. The number of

0.4

η = 0.01

neurons in the network is equal to the number of pixels in the input

βB

0.2

Table 1

Parameter setting in the experiment.

0

-300 -200 -100 0 100 200 300

Parameters s aT VT r Z

D (i, j )

Their values 1 0.12 1000 14 0.01

Fig. 3. The relation between constant Z and weighting coefﬁcients.

image A Measure transformation

IA SA

Dual-channel Fused

PCNN image

IB SB

image B IB Measure MB transformation βB

ARTICLE IN PRESS

Z. Wang et al. / Pattern Recognition 43 (2010) 2003–2016 2007

Parameters of Huang’s method. between the image pixels (IA(i, j) and IB(i, j)) and the neuron

(N(i, j)). In other words, the external stimuli of N(i, j) are IA(i, j) and

Parameters aL aT VL VT r

IB(i, j).

Their values 1.0 5.0 0.2 20.0 13.0 Now we describe the implementation of algorithm. The

procedure of our proposed multi-focus image fusion is shown in

Fig. 5. Test images which consist of four groups: LETTER, BADGE, TEAPOT, and PHOTO. Each group include two source images (labeled by A and B). (a) LETTER A, (b) LETTER

B, (c) BADGE A, (d) BADGE B, (e) TEAPOT A, (f) TEAPOT B, (g) PHOTO A, and (h) PHOTO B.

ARTICLE IN PRESS

2008 Z. Wang et al. / Pattern Recognition 43 (2010) 2003–2016

Fig. 4. Note that our algorithm is introduced on the assumption 4.1. Parameter setting

that input multi-focus images have been registered.

The proposed multi-focus image fusion method consists of the In the dual-channel PCNN, parameter settings are as follows:

following steps: large K will make computer simulation cost much time, so

synaptic weight matrices K =[1, 0.5, 1; 0.5, 0, 0.5; 1, 0.5, 1]; level

(1) Carry out focus measure with Eq. (11) for two multi-focus images factor, time constant and normalized offset parameter see Table 1.

(IA, IB). Denote measured images by MA and MB, respectively. In addition, parameters in Eqs. (12)–(14) are also in Table 1. Note

(2) Compute the weighting coefﬁcients (bA and bB) via MA and MB that these parameters are manually set by experiments.

according to Eqs. (12)–(14). For comparison, several existing methods are used: contrast

(3) Input IA and IB, taken as two stimuli, into the dual-channel pyramid (CP), FSD pyramid (FSD), gradient pyramid (GP),

PCNN, and then start PCNN. morphological pyramid (MP), SIDWT with Harr (SIDWT), and an

(4) Fuse the multi-focus images via PCNN. existing PCNN method (Huang’s method). In order to make

(5) Obtain the fused image after ﬁnishing the process of PCNN. comparison reliable and repeatable, we use the image fusion

toolbox for MATLAB developed by Rockinger [23]. The toolbox

includes all the above methods except Huang’s method and can be

4. Simulations and results downloaded on the web (http://www.metapix.de/toolbox.htm).

Their parameters in toolbox are set by: pyramid level= 4, selection

This section consists of three parts: parameter setting, rules: highpass= select max, lowpass= average. Note, with this

performance evaluation, and practical applications. Different toolbox, our experiment can be reproduced exactly according to

parameters of algorithm can produce different effects. So this parameters provided above. If you want to obtain more details,

section begins with parameter setting. In order to testify the please read related references [6]. For the scheme and PCNN

capability of the proposed method, performance evaluation is model of Huang’s method, Huang and Jing [7,8] give a detailed

done in succession with visual effect and objective evaluation explanation in their paper. Its iteration time is 300 and other

criteria. Finally, we illustrate some applications in practical work. parameters see Table 2.

Fig. 6. Test results (LETTER). (a) Our method, (b) CP method, (c) Huang’s method, (d) FSD method, (e) GP method, (f) MP method, (g) SIDWT method, and (h) reference

image.

ARTICLE IN PRESS

Z. Wang et al. / Pattern Recognition 43 (2010) 2003–2016 2009

To evaluate objectively those methods mentioned above, we

To evaluate the performance of our proposed fusion method, choose two methods: the root mean squared error (RMSE) and the

extensive experiments with multi-focus image fusion and structural similarity (SSIM) index. Here R and F denote reference

different sensor image fusion have been performed. Here, we image and fused image respectively. RMSE is used to evaluate the

give four groups (LETTER, BADGE, TEAPOT, and PHOTO) experi- performance of the focus measures. RMSE is deﬁned as

ments. All test images used in the experiment are shown in Fig. 5. vﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ

u

u 1 X M X N

Note, images in LETTER, BADGE and TEAPOT are synthetic while RMSE ¼ t ½Fði; jÞRði; jÞ2 : ð15Þ

images in PHOTO are real with synthetic blur. The content of M Ni¼1j¼1

LETTER is simple; there is more information in BADGE and

Usually, less RMSE signiﬁes better performance for the

TEAPOT; PHOTO is the real picture acquired by camera. Hence, the

algorithm of fusion. The results of RMSE assessment are shown

content of image becomes more and more complicated from

in Figs. 7, 10, 13 and 16.

LETTER to PHOTO.

The structural similarity (SSIM) index is proposed by Wang

Here we give an explanation for ﬁgures: images in Figs. 6, 9, 12

et al [26]. Because it is a better approach to image quality

and 15 are experimental results obtained by several methods. In

measure, we use it to assess the performance of different

each ﬁgure, there are nine images: one reference image and

methods, too. The mathematical expression of SSIM index is

eight images fused by eight corresponding algorithms. Figs. 7, 8,

10, 11, 13, 14, 16 and 17 show the performance of different ð2mR mF C1 Þð2sRF þ C2 Þ

SSIM ¼ ; ð16Þ

algorithms. ðm2R þ m2F þ C1 Þðs2R þ s2F þ C2 Þ

2.5

1.5

0.5

0

Our Huang's

CP FSD GP MP SIDWT

method method

RMSE 0.0216 0.5914 1.6782 2.8283 2.8288 0.6319 1.2887

0.9

0.8

0.7

0.6

0.5

0.4

0.3

0.2

0.1

0

Huang's

Our method CP FSD GP MP SIDWT

method

SSIM index 0.9999 0.9999 0.9308 0.8291 0.829 0.9372 0.9863

ARTICLE IN PRESS

2010 Z. Wang et al. / Pattern Recognition 43 (2010) 2003–2016

where C1 =(K1L)2, C2 = (K2L)2. Where L is the dynamic range of the Larger value shows both inputs more similar. Experimental

pixel values (255 for 8-bit grayscale images); K1, K2 51 are small results of SSIM index are shown in Figs. 8, 11, 14 and 17.

constants. More descriptions are found in reference. Note that As for objective evaluation, our proposed method is not inferior

SSIM index MATLAB program is downloaded on the website to most methods from Figs. 7, 8, 10, 11, 13 and 14. According to the

(http://www.ece.uwaterloo.ca/ z70wang/research/ssim/). The evaluation rule of RMSE and SSIM index, the better method of multi-

values of SSIM index in the paper are computed with default focus image fusion should have smaller RMSE and larger SSIM index.

parameters. SSIM index describes the similarity of two inputs. In fact, experimental results demonstrate that our method has the

Fig. 9. Test results (BADGE). (a) Our method, (b) CP method, (c) Huang’s method, (d) FSD method, (e) GP method, (f) MP method, (g) SIDWT method, and (h) reference

image.

0

Huang's

Our method CP FSD GP MP SIDWT

method

RMSE 0.5792 1.5996 2.3938 4.6158 4.5491 6.4707 2.7038

ARTICLE IN PRESS

Z. Wang et al. / Pattern Recognition 43 (2010) 2003–2016 2011

smallest RMSE (in Figs. 7, 10, 13 and 16) and the largest SSIM index 4.2.2. Subjective evaluation

(in Figs. 8, 11 and 17) among several algorithms. However, Fig. 14 As we know, performance evaluation usually includes two

shows SIDWT have the largest SSIM index. At the same time, the sides: subjective (or qualitative) evaluation and objective (or

suggested method, CP, SIDWT, and Huang’s methods have similar quantitative) evaluation. For multi-focus image fusion subjective

performances in Figs 13, 14, 16 and 17. Hence it is essential to carry evaluation means the evaluation of visual effect. Here we use

on subjective evaluation in this case. visual evaluation to evaluate every method.

0.99

0.98

0.97

0.96

0.95

0.94

Huang’s

Our method CP FSD GP MP SIDWT

method

SSIM index 0.9996 0.9981 0.9934 0.9818 0.9821 0.9625 0.9953

Fig. 12. Test results (TEAPOT). (a) Our method, (b) CP method, (c) Huang’s method, (d) FSD method, (e) GP method, (f) MP method, (g) SIDWT method, and (h) reference

image.

ARTICLE IN PRESS

2012 Z. Wang et al. / Pattern Recognition 43 (2010) 2003–2016

From Figs. 6, 9, 12, and 15, it is clear that these two images (3) Obtain error images. E0= XOR(R0,Bs); E1= XOR(R1,Bs); E2=

fused by FSD method and GP method have lower lightness than XOR(R2,Bs);E3= XOR(R3,Bs). Here XOR indicates the exclusive

the others. It shows FSD method and GP method make the fused OR operation. The results see Fig. 18.

images have low contrast. This is not what we want. MP method

has a good contrast but the process of edges in images is bad,

For TEAPOT:

typically such as Figs. 6f, 9f, and 12f.

Although the remaining four methods (the suggested method,

CP, SIDWT, and Huang’s methods) have the similar visual effect, (1) Let R0, R1, R2, and R3 denote the fused image of the suggested

the difference among them still exists. In fact, our method has the method, CP, SIDWT, and Huang’s methods, respectively.

better performance. Here we use one method to make this (2) Make the difference between Rs and R0, R1, R2, R3, respectively.

performance easily seen. We take LETTER (in Fig. 6) and TEAPOT Namely, D0=jRs R0j; D1=jRsR1j; D2=jRs R2j; D3=

for example; our schemes of magnifying the difference among jRs R3j. Here, Rs denotes reference image, and || indicates

these four methods are as follow: computing the absolute value of the difference.

For LETTER: (3) Obtain error images. E0 =D0 40; E1= D140; E2 =D2 40;

E3= D340. The results see Fig. 19.

(1) Let R0, R1, R2, and R3 denote the fused image of the suggested For Figs. 18 and 19, all the images are binary images and light

method, CP, SIDWT, and Huang’s methods, respectively. point means error. The error image of our method in Fig. 18 is

Rs denotes reference image. completely dark, which shows the fused image using our method

(2) Get the binary images. B0= R040; B1 =R140; B2 = R24 0;B3 is more similar to reference image than others. Obviously, CP

= R340; Bs =Rs 40. method and Huang’s method have very bad results in Fig. 19.

0

Huang's

Our method CP FSD GP MP SIDWT

method

RMSE 3.5905 4.5103 3.6549 5.1081 5.12 6.0499 3.7865

0.99

0.98

0.97

0.96

0.95

0.94

0.93

0.92

Huang's

Our method CP FSD GP MP SIDWT

method

SSIM index 0.9839 0.9884 0.9886 0.9608 0.9619 0.9456 0.9903

ARTICLE IN PRESS

Z. Wang et al. / Pattern Recognition 43 (2010) 2003–2016 2013

Comparing Fig. 19(1) with Fig. 19(4), you will ﬁnd that the big By the way, we would like to compare our method with

teapot have the same light while two little teapots in Fig. 19(1) Huang’s method from the references [7,8]; after all, they have

are clearly darker than the ones in Fig. 19(4). It is indicated that similar performance in both objective and subjective standards.

the proposed method is better than SIDWT method. Hence, our Compared with Huang’s method, our method at least has

method is better than others. two advantages. One is that our method has less complexity

Fig. 15. Test results (PHOTO). (a) Our method, (b) CP method, (c) Huang’s method, (d) FSD method, (e) GP method, (f) MP method, (g) SIDWT method, and (h) reference

image.

0

Huang's

Our method CP FSD GP MP SIDWT

method

RMSE 2.3865 2.3951 2.402 6.0212 6.0169 4.0758 2.4105

ARTICLE IN PRESS

2014 Z. Wang et al. / Pattern Recognition 43 (2010) 2003–2016

0.99

0.98

0.97

0.96

0.95

0.94

0.93

Huang's

Our method CP FSD GP MP SIDWT

method

SSIM index 0.9954 0.9934 0.9905 0.9876 0.9877 0.9583 0.9936

Fig. 18. Error test of LETTER. (a) Our method, (b) CP method, (c) Huang’s method, and (d) SIDWT.

Fig. 19. Error test of TEAPOT. (a) Our method, (b) CP method, (c) Huang’s method, and (d) SIDWT.

because it does not need to divide the input images into many Table 3

image blocks and it also need not compare and choose the best Comparison of time cost between two methods (time unit: second).

image block. Obviously, the other is that our method has less time

Methods LETTER BADGE TEAPOT PHOTO

cost (shown in Table 3), for it has fewer operates than the latter.

Additionally, a large amount of time, spent in the process of Our method 0.3340 0.4131 0.6866 0.6469

determining the adjustable parameter b, greatly decreases the Huang’s method 5.5749 10.1060 18.5210 20.6628

latter’s efﬁciency. Here, what we need to explain is that data in

Table 3 are obtained in such experimental platform that the

computer is IBM graphics workstation (Intellistation Z Pro) and all

codes have been run in Matlab 7.0. are microscope images required by Motic digital microscope

with focus change (see Figs. 20 and 21). The latter two groups

are a body of photographs (see Figs. 22 and 23) which are

4.3. Practical application obtained from the internet (http://www.imgfsr.com/ifsr_ifs.html;

http://www2.mta.ac.il/ tal/Fusion/). Image A and image B are

Here we take four groups of images as example to demonstrate source images with different focuses. Image C is the fused image

our method’s applications in practice. The ﬁrst two groups obtained by our proposed method.

ARTICLE IN PRESS

Z. Wang et al. / Pattern Recognition 43 (2010) 2003–2016 2015

Fig. 20. Microscope images (1). (a) image A, (b) image B, and (c) image C.

Fig. 21. Microscope images (2). (a) image A, (b) image B, and (c) image C.

Fig. 22. Example of photography (1). (a) image A, (b) image B, and (c) image C.

Fig. 23. Example of photography (2). (a) image A, (b) image B, and (c) image C.

5. Conclusion excels the existing methods in both visual effect and objective

evaluation criteria. In practical applications, it is proved that our

This paper presents a novel multi-focus image fusion algorithm method is feasible. Because the method is simple and easy to

based on the dual-channel PCNN. This method improves the implement, it is suitable to work in real-time system platform, too.

standard PCNN model and simpliﬁes the process of image fusion

using PCNN in comparison with previous methods. Previous

methods usually employ more PCNNs or combination with other Acknowledgements

algorithms such as DWT, while the proposed method in this paper

just uses one dual-channel PCNN to implement multi-focus image We thank the associate editor and the reviewers for their

fusion. Experimental results show that our presented method helpful and constructive suggestions. The authors also thank Ying

ARTICLE IN PRESS

2016 Z. Wang et al. / Pattern Recognition 43 (2010) 2003–2016

Zhu for her support and help. This paper is jointly supported by [12] E. Krotkov, Focusing, International Journal of Computer Vision 1 (1987)

National Natural Science Foundation of China (No.60872109), 223.

[13] M. Li, W. Cai, Z. Tan, A region-based multi-sensor image fusion scheme using

Program for New Century Excellent Talents in University (NCET- pulse-coupled neural network, Pattern Recognition Letter 27 (2006)

06-0900), China Scholarship, and the Fundamental Research 1948–1956.

Funds for the Central Universities of Lanzhou University in China [14] S. Li, J.T. Kwok, Y. Wang, Multifocus image fusion using artiﬁcial neural

networks, Pattern Recognition Letter 23 (2002) 985–997.

(lzujbky-2009-129). [15] H. Li, B.S. Manjunath, S.K. Mitra, Multisensor image fusion using the

wavelet transform, Graphical Models and Image Processing 57 (1995)

References 235–245.

[16] W.Li, X.F. Zhu, A new image fusion algorithm based on wavelet packet

analysis and PCNN, in: Proceedings of the Fourth International Conference on

[1] C.H. Anderson, A ﬁlter-subtract-decimate hierarchical pyramid signal Machine Learning and Cybernetics. Guangzhou. 2005, pp.5297–5301.

analyzing and synthesizing technique. US Patent 718104. 1987. [17] G. Ligthart, F. Groen, A comparison of different autofocus algorithms, in:

[2] R.P. Broussard, S.K. Rogers, M.E. Oxley, G.L. Tarr, Physiologically motivated Proceedings of International Conference on Pattern Recognition, Munich.

image fusion for object detection using a pulse coupled neural network, IEEE 1982, pp. 597–600.

Transaction Neural Networks 10 (1999) 554–563. [18] T. Lindblad, J.M. Kinser, Image processing using pulse-coupled neural

[3] P.J. Burt, A gradient pyramid basis for pattern-selective image fusion, Society networks (second edition), Springer press, New York, 2005.

for Information Displays (SID) International Symposium Digest of Technical [19] Y.D. Ma, L. Li, Y.F. Wang, R.L. Dai, Principle of Pulse Coupled Neural Network

Papers 23 (1992) 467–470. and Its Applications, Science Press, Beijing, 2006.

[4] R. Eckhorn, H.J. Reitboeck, M. Arndt, P.W. Dicke, Feature linking via [20] Q. Miao, B. Wang, A novel adaptive multi-focus image fusion algorithm based

synchronization among distributed assemblies: Simulation of results from on PCNN and sharpness, in: Proceedings of SPIE-the International Society for

cat cortex, Neural Computation 2 (1990) 293–307. Optical Engineering, Orlando, 2005, pp. 704–712.

[5] H.A. Eltoukhy, S. Kavusi, A computationally efﬁcient algorithm for multi-focus [21] H.S. Ranganath, G. Kuntimad, J.L. Johnson, Pulse coupled neural networks for

image reconstruction, in: Proceedings of SPIE-The International Society for image processing, in: Proceedings of Southeast conference on ‘Visualize the

Optical Engineering, Santa Clara. 2003, pp.332–341. Future’. Raleigh. 1995, pp. 37–43.

[6] R.C. Gonzalez, P. Wintz, Digital image processing, Perason Education, NJ, [22] O. Rockinger, Image sequence fusion using a shift-invariant wavelet trans-

1978. form. In: Proceedings of the IEEE International Conference on Image

[7] W. Huang, Z.L. Jing, Multi-focus image fusion using pulse coupled neural Processing. Santa Barbara. 1997, pp. 288–291.

network, Pattern Recognition Letter 28 (2007) 1123–1132. [23] O. Rockinger, 1999. Image fusion toolbox for Matlab. Technical report,

[8] W. Huang, Z.L. Jing, Evaluation of focus measures in multi-focus image fusion, Metapix. /http://www.metapix.de/toolbox.htmS.

Pattern Recognition Letter 28 (2007) 493–500. [24] A. Toet, A morphological pyramidal image decomposition, Pattern Recogni-

[9] J.L. Johnson, M.L. Padgett, PCNN models and applications, IEEE Transaction tion Letter 9 (1989) 255–261.

Neural Networks 10 (1999) 480–498. [25] A. Toet, L.J. van Ruyven, J.M. Valeton, Merging thermal and visual images by a

[10] J.L. Johnson, H.S. Ranganath, G. Kuntimad, H.J. Caulﬁeld, Pulse coupled neural contrast pyramid, Optical Engineering 28 (1989) 789–792.

networks, Neural Networks and Pattern Recognition (1998) 1–56. [26] Z. Wang, A.C. Bovik, H.R. Sheikh, E.P. Simoncelli, Image quality assessment:

[11] J.L. Johnson, D. Ritter, Observation of periodic waves in a pulse-coupled from error visibility to structural similarity, IEEE Transactions Image

neural network, Optical Letter 18 (1993) 1253–1255. Processing 13 (2004) 600–612.

About the Author—ZHAOBIN WANG received the B.A. degree in electronic information science and technology from Yantai University, Yantai, China, in 2004. He is

currently pursuing the Ph.D. degree in radio physics at Lanzhou University, Lanzhou, China. Since 2004, he has been working on image processing techniques, most in the

biological images. In particular, he has developed a special interest in biomedical image segmentation and measurement, pattern recognition and artiﬁcial neural network

especially PCNN.

About the Author—YIDE MA received his B.S. degree and M.S. degree in Electronic Engineering from University of Electronic Science and Technology of China, Chengdou

City, China. But his Ph.D. degree was received from Lanzhou University. He is now a professor in School of Information Science and Engineering, Lanzhou University. His

research focuses on image processing and embed-system’s application. He has authored and coauthored more than 100 publications in journals, books and international

conference proceedings.

About the Author—JASON GU received a B. Sc. degree in 1992 from the Department of Electrical Engineering and Information Science (Special Class for the Young Gifted

1987–1990), University of Science and Technology of China, M. Sc. degree in 1995 from Biomedical & Instrumentation Engineering, Jiaotong University (Shanghai), China

and Ph.D. degree from the Department of Electrical and Computer Engineering, University of Alberta, Edmonton, Alberta, Canada. Since September 2000, he has been with

the Department of Electrical and Computer Engineering at Dalhousie University and is presently an associate professor of Electrical Engineering. His research focuses on

Robotics, Biomedical Engineering, Intelligent Systems, etc.

- PhysicsDiunggah olehAnand Raj
- Image Fusion and Guided FilteringDiunggah olehsubashreeseenivasan
- physics module F4 (light)Diunggah olehped
- 9 ,10 Icse PhysicsDiunggah olehAnimesh Dikshit
- Phyiscs SPM revision questionsDiunggah olehSelinaTanShuLing
- SBES Final ReportDiunggah olehjclv.youknow8373
- ch1 stpDiunggah olehrumpyhw
- Convex LensDiunggah olehIsaac Wong
- New Microsoft Word Document (2)Diunggah olehParas Sundriyal
- 12_physics_lyp_2016_allindia_set2Diunggah olehAasheesh Sahu
- Guia Do ProfessorFQ8Diunggah olehAntónio Durão
- Refraction of Light at Plane SurfacesDiunggah olehShaik Hidayathulla
- 47591139 Yearly Lesson Plan Science Form 4 2011Diunggah olehAndros Chiew Ting Lim
- QU Balloon 2008 ReportDiunggah olehegrandis1
- PHY131 Slide 8 OpticsDiunggah olehmiya ramziah
- Phy Test Series SyllbsDiunggah olehAshish Kumar
- 7 Geometrical Optics - LensesDiunggah olehpradeep707rediffmail
- 12_physics_lyp_2016_allindia_set2.pdfDiunggah olehAasheesh Sahu
- light - reflection and refraction 10th phy.pdfDiunggah olehManeeshAkhandJyoyi
- Biswas Science 1Diunggah olehKevin Eapen
- HKPO06_Sol.pdfDiunggah olehlagostinha
- Auto Par Am EstDiunggah olehAnonymous 3klrkD
- JEE MainDiunggah olehAsharani Mahapatra
- 3Prep FirstTerm Unit 2 LensesDiunggah olehKamal Naser
- Ppc 1Diunggah oleha_roy003
- Understanding CCTV3 LensesDiunggah olehthouck
- Plant Layout 2Diunggah olehtadiwos
- Enegineering Qp NewDiunggah olehBhanu Myneni
- HMS - Care and Cleaning of Your MicroscopeDiunggah olehJussi Rautiainen
- As Cie Ict Prac 2017Diunggah olehRAMAN KUMAR

- Yin 2016Diunggah olehIness Ben
- color fusionDiunggah olehIness Ben
- Liu 2017Diunggah olehIness Ben
- Gravina 2017Diunggah olehIness Ben
- Bin 2017Diunggah olehIness Ben
- Liu 2015Diunggah olehIness Ben
- Xiao 2016Diunggah olehIness Ben
- Quadtree Based Image Fusion-Information Fusion 2015Diunggah olehIness Ben
- li2013Diunggah olehIness Ben
- fu2015Diunggah olehIness Ben
- Object Basedchangedetectionfromsatelliteimagerybysegmentationoptimizationandmulti FeaturesfusionDiunggah olehIness Ben
- 08000301Diunggah olehIness Ben
- Final VersionDiunggah olehIness Ben
- FINAL_VERSION.pdfDiunggah olehIness Ben
- zhu2017.pdfDiunggah olehIness Ben
- fusion block 2.pdfDiunggah olehIness Ben
- Fusion Region 2Diunggah olehIness Ben
- Fusion PixelDiunggah olehIness Ben
- 10.1.1.76.8737Diunggah olehDivya Reddy
- Ali Doost 2015Diunggah olehIness Ben
- Multi-modal Image Fusion With KNN MattingDiunggah olehIness Ben
- Researchpaper Multimodal Image Fusion a SurveyDiunggah olehIness Ben
- AlgorithmDiunggah olehIness Ben
- Algorithm (1)Diunggah olehWafa Benzaoui

- Chapter 24 MiscellaneousSystemsDiunggah olehMark Evan Salutin
- ophs58en.pdfDiunggah olehRosana Melone
- jk_saraswatDiunggah olehjksaraswat
- 908 OSH Offshore Oil & Gas Safety 1.pdfDiunggah olehテレブリコ ジェファーソン
- Effective DatingDiunggah olehPeter Leonov
- 262126453 Enzymes Mechanism of Enzyme ActionDiunggah olehkl42c4300
- AC 21-99 Amdt 1Diunggah olehHarry Nuryanto
- Curtain WallDiunggah olehZameer Ahmed
- Chemicals Based on MethaneDiunggah olehriloadd
- svnbook-1.6-en-r3453Diunggah olehlyax1365
- Discussion of strength of materialDiunggah olehTak They
- Fabrication of Automatic Sheet Metal Cutting MachineDiunggah olehElakkiya Karthic
- Manual ComapDiunggah olehJaimeCoello
- The Effects of Reservoir PropertiesDiunggah olehJessica King
- Simbolos Elétricos - EPLANDiunggah olehTúlio Luna
- CompleteAmpManual.pdfDiunggah olehLeonardo Leon
- 9170 manualDiunggah olehapi-170472102
- 5 Vasanth ResumeDiunggah olehgimel223600
- wj1106-264Diunggah olehpnbsvp1987
- A Guide to Software Package Evaluation and SelectionDiunggah olehJason Guo
- Gasperini Et Al (2010) - BSSA PaperDiunggah olehStacey Martin
- MOSFET as a SwitchDiunggah olehFaiz Ullah Faizi
- Time Table Y3 Sem2Diunggah olehEvonne Lee
- Biometrics and StandardsDiunggah olehITU-T Technology Watch
- Finite Element Methods and Modelling i Mid Term 2010Diunggah olehnveman
- card catalogue databasesDiunggah olehapi-223744460
- AR Setup ChecklistDiunggah olehDavid Sparks
- j.ijpvp.2013.04.017Diunggah olehBheemannaChandur
- Foxconn n15235 ManualDiunggah olehAlejandro E. Amaya Reyes
- SyllabusMSC(CS) I,III-2017.pdfDiunggah olehHitesh Adhao