ICCC2009 • IEEE 7th International Conference on Computational Cybernetics • November 26-29, 2009 • Palma de Mallorca, Spain

Algorithm for Pulse Coupled Neural Network

Parameters Estimation
R. Forgáþ, I. Mokriš*
* Institute of Informatics Slovak Academy of Sciences, Bratislava, Slovak Republic
radoslav.forgac@savba.sk, igor.mokris@savba.sk

Abstract— The paper introduces an approach for linking
and decay coefficient estimation in the Optimized Pulse
Coupled Neural Network with Modified feeding input (OM-
PCNN). These two parameters have significant importance
to improve the image recognition precision. The standard
PCNN has ten parameters, so it is very difficult to find
optimal closed system of all its parameters. By introducing
the OM-PCNN it is needed to set up four parameters only –
linking coefficient, decay coefficient of threshold potential,
linking radius and PCNN kernel. The rest two parameters
of OM-PCNN are adjusted by rule of computational
complexity minimization.
I. INTRODUCTION
The application spectrum of PCNN [1] usage is
extended to all areas of image processing nowadays. The
PCNN was used mainly in image segmentation, noise
filtering, feature generation for image recognition, edge
and object detection in the image and also in image
fusion process. There are also known cases of PCNN
application in the speech recognition area. The present
research is oriented mainly to optimization of PCNN
mathematical model and its parameters.
PCNN have several powerful properties such as high
degree of space dimension reduction, minimal set of used
etalons or fixed structure of input object matrices. With
these properties the minimal number of generated
features by PCNN is related, too. Next powerful property
is invariance of generated features against rotation,
dilatation and translation of images. PCNN does not
require the learning process.
On the other hand, the PCNN have several
disadvantages. The main disadvantage of PCNN is
difficult mathematical model with high number of
parameters and related problem with their optimal
estimation for feature generation. Next disadvantage is
high number of iteration steps in the feature generation
process and feature selection problem with the highest
information value. These problems were very often solved
through an experiment. The much better solution of this
problem is to modify the PCNN model [4 - 8].
Present research by [4 - 8] of the standard PCNN is
focused on the mathematical model optimization and
parameters estimation with aim to reduce the number of
generated features to reach high image recognition
performance. The optimized version OM-PCNN has all
assumptions for the dimension reduction of image
classification space and for image recognition.
The paper is divided into three sections. In the first
section the general structure of OM-PCNN, mathematical
model of OM-PCNN and process of feature generation by
OM-PCNN will be introduced. The second section
describes algorithm of linking and decay coefficient
estimation. The final third section shows experiments and
results of feature generation by using estimated
parameters.
II. PULSE COUPLED NEURAL NETWORK
A. Structure of OM-PCNN
The structure of OM-PCNN version will be the same as
structure of standard PCNN (Fig. 1), i.e. each modified
version come out from standard PCNN neuron model
(Fig. 2).
Figure 1. Structure of OM-PCNN
Every neuron (i, j) is associated just with only one
image pixel (i, j) of image matrix S. The neuron receives
input signals from its own neighborhood O, which consist
of feeding input OF and linking input OL. In general are OF
and OL circle areas of the same radius r0. Weights of
connections between neuron (i, j) and its neighborhood OF
are described by matrix of weight coefficients M and
weights of connections between neuron (i, j) and its
neighborhood OL are described by matrix of weight
coefficients W. Output quantity Yij of neuron (i, j) have
impact on activation potential Ukl through its neighbor
neurons (k,l) which belong to the neighborhood OL.
The structure of pulse coupled neuron in the OM-
PCNN is shown in the Fig. 2. The OM-PCNN neuron
consists of input part, linking part and pulse generator. In
the input part linking potential Lij(n) is controlled by
linking coefficient ȕ and is sequentially modulated with
feeding potential Fij(n) in the linking part. Pulse generator
consists of step function generator and sigmoid function
generator.

978-1-4244-5311-5/09/$26.00 ©2009IEEE 147

R. Forgác and I. Mokriš • Algorithm for Pulse Coupled Neural Network Parameters Estimation
Figure 2. Structure of OM-PCNN neuron
B. Mathematical model of OM-PCNN
OM-PCNN is based on PCNN with modified feeding
input (M-PCNN) [10]. It has neither exponential nor
convolution part in equation for feeding potential of
neuron Fij(n). The OM-PCNN mathematical model can be
described by following equations [4 - 8]:
Fij (n) = Sij
Lij (n) = Kij (n)
U ij (n) = Fij (n) ⋅ (1 + β o ⋅ Lij (n))
1 (4)
X ij (n) = Tij ( n −1) −U ij ( n )
1+ e
1 if X ij (n)>0,5
Yij (n) = {
0 otherwise
Tij (n) = { VT if Yij (n) =1
α o ⋅Tij (n −1) if Yij (n)=0
where Fij(n) is input feeding potential and Lij(n) input
linking potential of neuron. Sij is an intensity of pixel i,j in
the input image matrix S, which represents the intensity of
given image element. Xij is activation quantity of neuron
that is defined by sigmoid transfer function, Yij is
activation quantity of neuron that is defined by step
function. Uij is the activation quantity of neuron, Tij is
threshold potential of neuron, n is iteration step. Parameter
αo is threshold decay coefficient, ȕo is linking coefficient
and parameter VT is coefficient of the threshold potential.
Matrix elements K(n) are calculated by convolution
K ij (n) = (W ∗ Y (n − 1))ij = ¦ wijkl .Ykl (n − 1)
where * is convolution operator, W is the matrix of
weight coefficients and wijkl is the element of constituent
matrix W for the feeding and linking input, Ykl(n-1) is the
output quantity of the neuron (k, l) that belongs to the
neighbor OL of neuron (i, j) in the previous iteration step
n-1. The values of weight coefficients wijkl depend on the
value of linking radius r0 and on implemented PCNN
kernel for neighbor OF and OL.
The feature generation principle of image recognition
by PCNN is based on pulse generation of individual
neurons. The multilevel input image, which is represented
by two-dimensional matrix, is transformed through PCNN
into a sequence of temporary binary images. Each of these
binary images is a matrix with the same dimension as
input matrix and it is generated by group of pixels with
similar intensity.
The feature generation by (8) is invariant against non
standard cases presented in [6]:
¦ (X
ij
ij (n) ⋅ Yij ( n) )
(8)
G ( n) =
¦S ij
ij
where a numerator in fraction represents the value before
standardization and denominator represents the feature
standardization. The activation quantity Xij based on (4)
belongs to the interval (0, 1) and activation quantity Yij
based on (5) may obtain only two values, namely 0 a 1.
The OM-PCNN algorithm has following form:
1. Calculation of convolution matrix K for every iteration
n by (7).
(1)
2. Calculation of internal activation quantity Uij by (3).
3. Calculation of output quantity Xij by (4) and Yij by (5).
(2)
4. Calculation of threshold potential Τij by (6).
OM-PCNN was used as background for design of
PCNN parameters estimation for the better feature
(3)
generation purposes.
III. OM-PCNN PARAMETERS ESTIMATION
A. Estimation assumptions
(5) It holds in general that optimal feature generation of
image recognition by PCNN depends on PCNN model
selection and PCNN parameters settings [9].
(6) The reason of OM-PCNN optimized model comes out
of analysis of M-PCNN [4] image recognition precision
results. The idea of optimization is based on assumption
that change of period ǻPij of pulse generation of particular
neurons causes the incremented variance of generated
feature values. On the base of analysis of period Pij
evolution in particular pulses of G(n) function it was
shown that the minimal variance of values reach the
features in the first pulse of G(n) function.
In the standard PCNN it is needed to set up to 10
parameters. In the modified M-PCNN the number of
parameters was decreased to 8, but it is still high number
for estimation purposes. In the proposed OM-PCNN
model it is needed to set only 6 parameters - ro, N, αo, VT,
βo a OM-PCNN kernel. Only two parameters from it (i.e.
(7) αo, βo) are needed to estimate. The process of continual
reduction and particular parameters calculation can be
described in next five steps:
1. By simplifying of linking potential Lij of neuron
calculation by (2) there was eliminated parameters ĮL
and VL. The number of OM-PCNN parameters
decreased from 8 (M-PCNN) to 6.
2. By establishing the condition [4]
VT ≥ 6 (9)
148
 ICCC2009 • IEEE 7th International Conference on Computational Cybernetics • November 26-29, 2009 • Palma de Mallorca, Spain
parameter VT does not have influence on recognition
precision in the first pulse of function G(n). It remains
to set 5 parameters yet: N, Įo, ȕo, r0 and OM-PCNN
kernel.
3. The number of iteration steps N it is possible to set
deterministically. By decreasing of feature generation
scope on the first pulse of G(n) function it is achieved
effect of undesirable influence minimization of
neighborhood on each activated neuron. Thereby,
number of generated features N will be restricted to
scope of local maximum area of the first pulse of G(n)
function or whole first pulse of G(n) function. The
neighborhood of local maximum of the first pulse of
G(n) function is determined by three features. The
feature generation is finished after achievement of
iteration nmax+1, where nmax is iteration by which local
maximum of the first pulse of G(n) function was
found. For the classification purposes only three
feature values nmax – 1, nmax and nmax+ 1 were used. In
the case of first pulse G(n) scope it goes on widening
of local maximum neighborhood for the whole first
pulse of G(n) function. In this case the feature
generation is finished, if the zero feature value was
reached or feature amplitude in the given iteration step
decreases under established threshold value of local
maximum amplitude of the first pulse. There was
established threshold value to 0.01 % from first local
maximum value of G(n) function on the base of
functional G(n) flows of image etalons.
4. It can be assumed that with increasing linking area OL
it is possible to achieve better features for recognition.
For each type of PCNN kernel there is boundary by
which further linking radius increasing has minimal
impact on the G(n) function [2, 3], because values of
weight coefficient nearing to zero. Contrary, the
feature generation time is increasing because it is
needed to recalculate the convolution matrix K in
every iteration step. From the reason of computation
complexity minimization of feature generation by
OM-PCNN the linking radius was set to value r0 = 1.
5. The weight coefficients wijkl are calculated by
distribution 1/r [4]. In this case the matrix W has the
dimension 3 x 3 and the values of particular weight
coefficients are given in the following form
ª0 1 0º
W = ««1 1 1»»
«¬0 1 0»¼
The using of another PCNN kernel as for example
kernel with Gaussian distribution or setting of greater
value r0 has not significant influence on the recognition
precision [2].
On the base of given criteria of the computational
complexity of feature generation it is needed to propose
the algorithm to estimation only of pair of parameters Įo
and ȕo.
B. Algorithm for parameter estimation
The reason for estimation algorithm proposal for Įo and
ȕo parameters is based on finding of maximal distance ED
between two nearest images from different classes in
Euclidean space.
The algorithm requires creation of two groups of
images for which the following conditions hold:
1. The first group of images represents etalons from each
image class. The mutual distance between particular
etalons in Euclidean distance must be greater than zero.
2. The second group of images is represented by sample
of k tested images where distance of etalon from
particular class is the greatest. From the analysis of
image recognition results by M-PCNN follows that to
this group images with greatest rotation and at the same
time dilatation influence will be chosen [4]. In the case
of not choosing images with greatest distance from
particular etalons, the parameters Įo and ȕo will be
optimized only for scope of rotated and at the same
time dilated images ED of which corresponds to chosen
images. The sample of tested images must contain at
least one image from each class.
The scopes of intervals in which suboptimal values Įo
and ȕo will be searched, are defined by following criteria:
1. Linking coefficient ȕo will be tested on interval (0 : 1>.
If the parameter value ȕo was equal 0, then the
modulation member ȕoKij is eliminated by (3). This
modulation member contributes to synchronization of
PCNN neural pulse generation. By ȕo > 1 PCNN works
in the multipulse mode [10] which is not suitable for
feature generation because the linking input is saturated
and generated G(n) function converges to constant
value [3].
2. The value of decay coefficient Įo is in (0 : 1). From
equation (6) is obvious that if parameter Įo is set to 1,
neurons in the PCNN can not generate pulses because
threshold potential Tij of each neuron is by initialization
greater than 1, on the other side the other quantities, i.e.
Lij, Uij, Yij equal zero. It is possible to increase bottom
border of parameter value Įo to Įo ≈ 0.3 because by
lower values has G(n) function redented flow (Fig. 3).
If G(n) = 1, than Uij(n) = max and if G(n) = 0, than
Tij(n) = max.
(10)
Figure 3. Typical flow of function G(n) for lower values Įo<0.3
The parameter estimation algorithm of Įo and ȕo has
iterative nature. The parameter ȕo is iteratively
incremented with step Δȕo in the main loop and in the
inner loop is parameter Įo incremented with step ΔĮo:
1. for ȕo = (0 : 1> step Δȕo
2. for Įo = (0 : 1) step ΔĮo
3. Calculation of Euclidean distances eD between
etalon and tested images of the same class for
149
R. Forgác and I. Mokriš • Algorithm for Pulse Coupled Neural Network Parameters Estimation

all image classes. The values will be stored in step ΔĮo = 0.05 in the inner loop. In regard to large scope
vector ED1(i, k), where i is the index of of processed data there will presented only one part of
corresponding class and k is index of tested results focused on progress monitoring of suboptimal
image. values Įo and ȕo where the values of vector ED are the
highest.
4. Selection of the greatest value eD1(i) from The influence of parameter ȕo on finding the higher
vector ED1(i, k) for each image class i. Euclidean distance Emax of the two nearest textures from
various classification classes is shown on Fig. 4. The
5. Calculation of Euclidean distances eD between functional dependence ED(ȕo) of the nearest images has
etalon and tested images that belong to descending tendency with ascending value ȕo. The highest
different classes for all image classes. ED Emax value was achieved by ȕo = 0.1, i.e. this value is
values will be stored in vector ED2(i, k). considered as suboptimal.
6. Selection of the lowest value eD2(i) from vector
ED2(i, k) for each image class i.
7. Calculation of diference ǻeD(i) = eD2(i) - eD1(i)
for each image class i
¾ if ǻeD(i) > 0, then both etalon and tested
image belong to the same class.
¾ if ǻeD(i) < 0, then etalon and tested image
are from different classes.
8. Calculation of average value ǻe*D from all
ǻeD(i) calculated in the step 7 for set values Įo
and ȕo. The averages values ǻe*D for each pair
of parameters Įo and ȕo will be stored in vector Figure 4. Influence of parameter ȕo on finding the higher
E D. Euclidean distance Emax
9. end
10. end
11. Selection of the greatest value Emax of vector ED. The finding of optimal parameter Įo with the highest
Parameter values Įo and ȕo with corresponding value of Euclidean distance Emax of the two nearest
maximal value Emax are considered as suboptimal textures from various classification classes is shown on
values. Fig. 5.

In consideration of unreality to test the couple of

parameters Įo a ȕo in the whole continual interval for Įo =
(0 : 1) and ȕo = (0 : 1> it is possible to find only their
suboptimal values by using the proposed algorithm.
The algorithm calculation complexity depends on
number of image classes, on number of testing image
sample set in the second group and on setting of
incremental step of estimated parameters Įo a ȕo.
IV. RESULTS OF EXPERIMENTS
The testing image set was created from combination of
particular image geometrical transformations. In the case
of test image set enlarging on next image class, it is
sufficient to add to training set new etalon that is Figure 5. Influence of parameter ao on finding the higher
Euclidean distance Emax
represented by corresponding feature vector G(n).
The etalon set and the testing set are created from
texture images. The training set consists of 13 textures in On basis on results of algorithm for suboptimal
the basic form. There are images of bark, bricks, bubbles, parameters Įo and ȕo finding in the OM-PCNN was for the
grass, leather, pigskin, wave, wood, raffia, sand, straw, testing set of textures found the pair Įo = 0.6 and ȕo = 0.1
water and wool. The original database [11] with 13 at the constant setting of parameters VT = 20 and r0 = 1.
images in the basic form was processed into matrix of 450
x 450 image pixels. Each image class was enlarged on set The theoretical assumptions described in this paper of
of rotated, dilated and combination of both rotated and OM-PCNN with algorithm for parameter Įo and ȕo
dilated images. The testing set contains 4784 images estimation was confirmed by experiment. The recognition
together. precision comparison of OM-PCNN with only 3 generated
features and M-PCNN with 3, 10 and 50 features is shown
The estimation of parameters Įo and ȕo was realized by on Fig. 6. The best recognition precision at once and the
algorithm described in previous section of this paper. In shortest feature generation time was achieved by OM-
the algorithm was incremented parameter ȕo step by step PCNN.
with step Δȕo = 0.1 in the main loop and parameter Įo with

150
 ICCC2009 • IEEE 7th International Conference on Computational Cybernetics • November 26-29, 2009 • Palma de Mallorca, Spain

OM-PCNN, 3 features M-PCNN, 3 features

Theory and Applications, IOS Press, Amsterdam, 2002, ISBN 1-
M-PCNN, 10 features M-PCNN, 50 features
58603-256-9, pp. 304 – 308.
[4] R. Forgáþ, Dimension Reduction of Image Classification Space by
1
0,9
Pulse Coupled Neural Networks. [PhD thesis]. FEI TU Košice,
2005, 108 p. (in Slovak).
Recognition precision

0,8
0,7 [5] R. Forgáþ and I. Mokriš, “Influence of Pulse Coupled Neural
0,6 Network Initialization on Classification Tasks”, in Proc. of
0,5
0,4
Multimedia in Bussiness, 2006, Kielce, ISBN 83-7251-673-1, pp.
0,3 335-342.
0,2 [6] R. Forgáþ and I. Mokriš, “Feature Generation Improving by
0,1 Optimized PCNN”, in SAMI 2008 - 6th International Symposium
0
on Applied Machine Intelligence and Informatics, HerĐany,
nd Slovakia, January 21-22, 2008, IEEE Catalog Number CFP0808E-
f ia
r

w
s

l
n

e
rk

d
Bu s

ss

er

oo
he
k

le

ki

av
oo
ra
Ba

at
af
ic

ra

Sa
gs
bb

W
at

W
St

W
Br

W
CDR, ISBN 978-1-4244-2106-0, pp. 203-207
R
G

Le

Pi

Textures
[7] R. Forgáþ and I. Mokriš, “Threshold Potential Optimization in the
Pulse Coupled Neural Network”, submitted in SISY’08 - 6th
Figure 6. Recognition precision comparison of OM-PCNN and International Symposium on Intelligent Systems and Informatics,
M-PCNN Sept. 26-27, 2008, Subotica, IEEE Catalog Number CFP0884C-
CDR, ISBN 978-1-4244-2407-8.
[8] R. Forgáþ and I. Mokriš, “Linking and Activation Potential
Optimization in the Pulse Coupled Neural Network”, in ICCC
V. CONCLUSION 2008 - IEEE 6th International Conference on Computational
The aim of optimized OM-PCNN proposal with Cybernetics, 2008, ISBN 978-1-4244-2875-5, pp. 85-88.
algorithm for parameter Įo and ȕo estimation was [9] J. L. Johnson and M. L. Padgett, “PCNN Models and
achievement of better reduction of number of generated Applications”, IEEE Transactions on Neural Networks, Vol. 2,
No. 3, 1999, pp. 480-498.
features by preservation of comparable recognition
precision than in the case of M-PCNN. It was also [10] G. Kuntimad and H. S. Ranganath, “Perfect Image Segmentation
Using Pulse Coupled Neural Networks”, IEEE Transactions on
experimentally confirmed. There was achieved a lower Neural Networks, Vol. 2, No. 3, 1999, pp. 591-598.
dispersion of generated features in the scope of the first [11] USC-SIPI Image Database, Signal and Image Processing Institute,
pulse of G(n) function. University of Southern California, Los Angeles
http://sipi.usc.edu/database
ACKNOWLEDGMENT
This work was supported by Slovak Science Agency
VEGA No. 2/7098/27 and VEGA No. 2/0211/09.
REFERENCES
[1] R. Eckhorn et al, “Feature Linking via Synchronization among
Distributed Assemblies: Simulations of Results from Cat Visual
Cortex”, Neural Computation, Vol. 2, 1990, pp. 293-307.
[2] R. Forgáþ and I. Mokriš, “Parameter Influence of Pulse Coupled
Neural Network for Image Recognition”, Journal of Applied
Computer Science, Vol. 9, No. 2, 2001, pp. 31-44.
[3] R. Forgáþ and I. Mokriš, “New Approach to Estimation of Pulse
Coupled Neural Network Parameters in Image Recognition
Process”, in Sinþák, P. et al. (Eds.): Intelligent Technologies –

151