2016 24th Iranian Conference on Electrical Engineering (ICEE)

A Harmonic Emotional Neural Network

for Non-linear System Identification
Hengameh Mirhajianmoghadam Mohammad-R. Akbarzadeh-T Ehsan Lotfi
Center of Excellence on Soft Center of Excellence on Soft Department of Computer
Computing and Intelligent Computing and Intelligent Engineering
Information Processing Information Processing Torbat-e-Jam Branch, Islamic Azad
Departments of Electrical Departments of Electrical and Torbat-e-Jam, Iran
Engineering Computer Engineering esilotf@gmail.com
Ferdowsi University Ferdowsi University elotfi@bitools.ir
Mashhad, Iran Mashhad, Iran
hengameh.mirhajian@stu.um.ac.ir akbarzadeh@ieee.org
hengameh_mirhajian@yahoo.com

Abstract— Emotional neural networks (ENNs) are a learnable
structure that have been inspired from the physiological features
of human's emotional brain. In this paper, a single layered ENN
is modified in a way to be differentiable for nonlinear system
identification problems. In the proposed ENN, thalamus-
amygdala expansion link is modeled by a sine and cosine basis
function. That is why is named the harmonic emotional neural
network (HENN). Due to this proposed expansion, presented
method is well-suited for the model based identification
methodologies. Simulation results indicate the validity of this
claim besides that the supremacy of HENN in terms of accuracy,
easy learning and simplicity of model in system identification is
shown.
Keywords- Emotional neural network; nonlinear system
identification; computation cost reduction;modeling; BEL
I. INTRODUCTION
System identification techniques use statistical methods to
build a mathematical models of dynamical systems[1] from
measured informative data. A resulting mathematical model
indicates the behavior of a system or process in either the time
or frequency domain. The application of system identification
contains industrial process, control system, economic data,
biology, social system and many others[2]. The common black-
box nonlinear models are Nonlinear Auto Regressive with
eXternal input (NARX) polynomial models, fuzzy models,
volterra models and artificial neural networks[3].
Among the mentioned methods, neural networks have been
applied extensively. In particular, neural models are
recommended in the literature. It is because they are entirely
black-box and have excellent abilities to approximate nonlinear
behavior[3]. Multi layered perceptron (MLP) and radial basis
function (RBF) are the most popular ones. Nevertheless, the
main purpose in system identification is to derive the best
balance between the approximation accuracy, the model
complexity and the computational cost. To solve this issue
many worked tried to achieve such balance. Commonly,
evolutionary methods is used to extract the optimum structure
of neural network. In the latest study, it is tried to change the
neural network structure, so that the number of neurons in the
hidden layer is constant (equal to 2) for every models[2].
This paper tries to attain a new approach to fulfill the
suitable trade-off and addresses a new kind of artificial neural
networks (ANNs) named emotional neural networks (ENNs).
The significant reason for this choice is the low computational
complexity of single layered ENN [4]. To the best of our
knowledge, ENNs with brain emotional learning method
(BEL) have been successfully applied for most prediction
applications[4],[5],[6] and also have been used as a
controller[7] (named BELBIC [8]). This paper aims to employ
a modified single layered ENN with an evolutionary learning
algorithm in identification and modeling of nonlinear complex
processes and systems.
The rest of this paper is organized as follows. Section II
describes the structure of ENN. Modified expansion link and
network learning method are explained in section III. What
motivates employing emotional modeling in this context is
presented in section III, whereas the comparison of the
proposed method with neural network is discussed. The results
of the simulations Section IV is devoted to indicate the results
of the identification. Finally, the subsequently conclusion are
given in section V.
II. EMOTIONAL NEURAL NETWORK
A. Biological model
The structural model of ENN is inspired from the limbic
system of mammalian brain [4]. This part is responsible for
processing emotional and generally plays a main role in
human’s emotional life. Employing the resulting neural
network in various application [5],[7],[8] stems from some
biological features of emotional brain. The short expanding

978-1-4673-8789-7/16/$31.00 ©2016 IEEE

1260
2016 24th Iranian Conference on Electrical Engineering (ICEE)

projections and inhibiting connections can be mentioned. These

properties lead to rapid response of the inputs [6].
The design of emotional neural network is based on main
components of limbic system named amygdala, orbitofrontal
cortex, thalamus and sensory cortex [9]. The output of ENN is
lied on amygdala [6]. As shown in fig.1, there are two paths to
reach emotional response. One of them is quick but imprecise
and directly comes from thalamus, the other one is slow but
precise and passes through orbitofrontal cortex and sensory
cortex [10].

B. Applied model
Single layered ENN with BEL algorithm was first
introduced in 2014, which is elaborated in [4]. A single layered
ENN model is illustrated in fig.2. The solid lines are indicative
of the data flow and the dashed lines are learning lines. In this Figure 2. A single layered ENN with modified expansion link.
model genetic algorithm is carried out the learning to find the
optimum weights for ENN. The proposed model is comprised In (2), s n 1 is a functional expansion link between thalamus
of two main section named orbitofrontal cortex and amygdala.
and amygdala that is calculated as follows:
In each section, there are some nodes which are equal to the
dimensional of inputs except amygdala which consists of one
more node for output of thalamus. The final output is
s n 1 max j 1}n s 
j (4)
calculated by the following relations: In the traditional single layered ENN, thalamus-amygdala
connection is modeled by max operator that is not a suitable
 E Ea  E 0 
Where Ea and E0 are the outputs of Amygdala and
Orbitofrontal respectively. They simply defined as follows:
n 1
 expansion for system identification. The max operator makes
the ENN as a non-differentiable model that is not suitable for
system identification based applications such as model based
control methodologies. Here we modified this link in order to
apply the model in system identification problems.
III. PROPOSED MODIFICATION FOR SYSTEM
 Ea ¦ v j u s j  
j 1
 IDENTIFICATION
a. Feedforward computation
In HENN, thalamus-amygdala expansion link is modeled
n by the following
 E0 ¦ w j us j  b   n n
j 1 s n 1 sin[S¦ s j ].cos[S¦ s j ] (5)
j 1 j 1
Where v and w are the related weights, b is the optional bias Thus,
and p is the vector of input [4]-[6].
n n n
Ea ¦
j 1
(v j u s j )  v th u sin[S¦ s j ].cos[S¦ s j ] 
j 1 j 1


And

n n n
E ¦
j 1
Figure 1.The routes of signal propagation in emotional modeling [6].
(v j u s j )  v th u sin[S¦ s j ].cos[S¦ s j ]  E 0 
j 1 j 1
Where E 0 is calculated by (3) and v th is related amygdala-
thalamus weights. Actually in the proposed bias function,
maximum s increase the frequency of sins and cosines in (5)
that their multiplication determines input value of amygdala.
b. Learning
The basic goal in learning is to find the optimized weights
to find the best solution for a problem. In [6], the genetic
algorithm (GA) is applied so that the adjusting of parameters
1261
2016 24th Iranian Conference on Electrical Engineering (ICEE)

will no longer be required and the optimal mode.is achieved § ªn ·

N1
º
directly. The reason is GA is more robust compare to others yi ¦ f ¨w ij «¦f w jk x k  b1 »  b2 ¸  
methods. Moreover, it doesn't break easily even if the inputs j 1 © ¬k 1 ¼ ¹
changed slightly or in the presence of reasonable noise [11]. In
a genetic algorithm, a population of candidate solutions is
evolved toward better solutions. A chromosome is involved the   i 1,}, N 2  
weights of neural network and is formed as follows:
An analysis of (11) in terms of numbers of parameters,
Choromi [v 1,v 2 ,},v n ,v n 1 ,w 1 ,w 2 ,},w n , b ]  
As shown, the number of genes in each chromosome is
2n  2 and n is the number of inputs. Therefore, the fitness
function (9) must be minimized to find the optimized network
crisp weights. Where Y k is the output of model for the kth
pattern, S k is the kth input pattern, T k is the related target and
m is number of pattern-targets [6].
 complexity and structure shows the computational complexity
of MLP is O(n×m). Where n is number of inputs and m is
number of nodes in hidden layers. The inputs include the past
and current inputs of plant in addition to its past outputs.
In contrast, the computational complexity of the model is
achieved from the harmonic emotional neural network (HENN)
is O(n+1) because the hidden layer is absent. Equation (12)
indicates the validity of this claim.
n 1 n

¦f v s j  b1   ¦f w ik s k  b2  
0.5
1§m ·  Ei 
 
2
fitnesstrain Chorom  ¨¦ Y
ij

k
T k ¸    j 1 k 1
m ©k 1 ¹
Where s n+1 is calculated by proposed expansion link of (5).
The presented comparison leads to the following conclusions.
 Y k

E P k ;Choromi  
The goal was to represent a proposed BEL-based ENN to
reach the optimum neural network with low complexity for
system identification which was fulfilled finally.
IV. MODELING
Most systems in real world are complex and nonlinear
Hence, finding out the appropriate mathematical models for
these kinds of systems are difficult [2]. In addition, most of the
controllers, specially the model based ones like MPC, need the
appropriate model of the process [12]. In the other words, the
more accurate model will lead to the better control
performance. Nevertheless, the computational burden might be
considered because the more accurate performance is often
achieved at the expense of increased complexity [13].
Therefore a trade-off between the accurate
execution of controller speed is desired. Commonly,
models have been used for simplicity, while the
models behave more like the actual plants thus, it
validity.
Plant can be analyzed with regard to its inputs and outputs,
without any knowledge of its internal working. Artificial
Neural Networks has the ability to provide an ideal model of
the complicated nonlinear systems by using this property [14].
This feature differentiate them from the conventional
identification approaches. Multi layered perceptron (MLP) is
the popular kind of neural network that are commonly used for
system identification. Most kinds of neural networks have a
hidden layer.
The structure of the resulting model can be presented as
followed for each node of outputs:
 
Firstly, HENN has the ability to learn sophisticated nonlinear
relationships and can offer an exciting alternative for modelling
complex nonlinear systems. Furthermore, in [15] is proved this
neural network can be a universal approximator. Secondly, the
simple structure of presented approach and its low
computational complexity improve the computing efficiency,
which affects the control performance. It is highly functional
for the model of system is used iteratively in control algorithm
and the sampling time is small.
Indeed, from control point of view, the computational
burden plays a main role to achieve the best control
performance. This issue is highlighted in the model based
control strategy for fast system. As a result, the presented
approach can solve this difficulty.
V. SIMULATION RESULTS
To evaluate the performance of the proposed method, the
and the Continuous Stirred Tank Reactor (CSTR) was considered. It is
the linear a common chemical system and the non-adiabatic type [16] is
nonlinear studied. Multi-Input Multi-Output (MIMO), highly
has more nonlinearity and the interaction between the parameters are
importance features of system. Afterwards, applying the model
based control is difficult to this process. Hence we claim that
HENN is the best solution for this issue. Here the identification
of CSTR is carried out to demonstrate the superiority of the
proposed approach in terms of accuracy and complexity. As
mentioned before, neural network is chosen for this
comparison due to its capability and more usage. CSTR has
three inputs and two outputs. They are illustrated in fig.3 and
fig.4, respectively.
In neural network design, the number of inputs are 8 that
included the current and previous values of inputs variables and
the previous values of outputs. The number of input and output
1262
2016 24th Iranian Conference on Electrical Engineering (ICEE)

are the same in MLP and HENN. The performance measure for As it can be seen in table 1 and 2, on average, the error of
evaluating the validation accuracy of these two neural network MLP for training data is less than the error of HENN.
is mean square error (MSE). Tables 1 and 2 show the mean and Nevertheless in test data, HENN have less error. As a result,
standard deviation of the accuracy of output1 and output2 the proposed approach have a better performance in
respectively. The results are obtained after 10 times repeat. It identification of nonlinear complex systems. Moreover, with
should be noted that the defined MLP is trained with the standard deviation assessment, we figure out HENN acts more
Levenberg–Marquardt (LM) algorithm and designed in stable especially in test data compared with MLP.
“nftool”. For better comparison the MATLAB neural network
toolbox is used to proof the accuracy of claims. If the structure of these neural networks is evaluated, we
found out the number of parameters obtained from HENN is
The optimum structure of neural network for each model is less than MLP. The structure of proposed approach is O(8+1)
8:3:1 (the number of hidden nodes is 3). In this study, a and for MLP is O(8×3). As it can be seen, the model is
“purline” transfer function is found suitable for each obtained from HENN reaches the high accuracy with 9
architecture. The regression of the training and test data of parameters while reach it with 24 parameters.
output2 belonged to HENN and MLP are shown in fig.5 and Table 1. MSE comparison between HENN and MLP for output1
fig.6 respectively.
HENN (GA learing) MLP (LM learning)
Input #1: Concentration of A in inlet feed stream
Train Test Train Test
kgmol/m3

10.04
10.02
10
9.98 0.8194 0.7731 0.8118 0.7027
9.96
0 5 10 15 20
0.8050 0.8081 0.7812 0.8179
Input #2: Inlet feed stream temperature
299
0.8121 0.7823 0.8993 0.728
K

298
0.8090 0.8139 0.7928 0.7578
0 5 10 15 20
Input #3: Jacket coolant temperature 0.8117 0.7783 0.7851 0.7844
320

300 0.8251 0.7298 0.7505 0.951

K

280
0.8445 0.6403 0.8225 0.8132
0 5 10 15 20
Time (hours) 0.7766 0.9389
0.8237 0.7209
Figure 3. The three inputs of CSTR
0.8202 0.7466 0.8402 0.5614
Output #1: A Concentration
10
0.8201 0.7484 0.7662 0.8833
kgmol/m3

5 Mean
0
0 5 10 15 20 0.81975 0.76075 0.78895 0.7988
Output #2: Reactor temp.
380
Standard deviation
360

0.011102 0.050515 0.043232 0.116789

K

340

320

0 5 10 15 20
Time (hours)

Figure 4. The two outputs of CSTR Table 2. MSE comparison between HENN and MLP for output2

Training: R=0.99853 Test: R=0.99971 HENN (GA learing) MLP (LM learning)
390
Data 360 Data Train Test Train Test
380
Output ~= 1*Target + -5.1

Fit Fit
1.8067 0.9854
Output ~= 1*Target + 1

370 Y=T 350 Y=T 1.8596 0.9196

360
340 1.8491 0.9657 1.6996 1.4268
350
330 1.8734 0.8778 1.5852 1.8983
340

330 320 1.8679 0.9034 1.5904 1.8721

320
310 1.8730 0.8669 1.7504 1.2172
310
320 340 360 380 310 320 330 340 350 360
1.8710 0.8925 1.6978 1.4771
Target Target
Figure 5. Training and test output1 vs. target in HENN structure 1.8750 0.8825 1.6706 1.5468
Training: R=0.99865 Test: R=0.99849 1.8644 0.9192 1.7538 1.8384
390 390

380
Data
380
Data 1.8460 0.9782 1.8157 0.9665
Output ~= 1*Target + 0.91

Output ~= 1*Target + 1.3

Fit Fit
370 Y=T 370 Y=T
1.8744 0.8867 1.761 1.1945
360 360

350 350
Mean
340 340 1.86945 0.89795 1.725 1.45195
330 330

320 320
Standard deviation
310 310 0.010589 0.037221 0.080461 0.351054
320 340 360 380 320 340 360 380
Target Target

Figure 6. Training and test output1 vs. target in MLP structure

1263
2016 24th Iranian Conference on Electrical Engineering (ICEE)

Table 3. MSE comarison between HENN and MLP for 3_PSP robot
400
system output#2
390 MLP output
ENN output Motor1 Motor2 Motor3
380

370 train test train test train test

360
Mean
350

340
MLP 0.1807 0.2010 0.162 0.1838 0.1244 0.1319
330

320 HENN 0.1332 0.3736 0.0934 0.4571 0.0802 0.3042

310
Standard deviation
300
0 50 100 150 200 250
Figure 7. Validation of the outputs of ENN and MLP for output#2 MLP 0.0046 0.1015 0.0082 0.2201 0.0043 0.0457
6
error MLP HENN 0.0001 0.0003 8e-05 0.0003 0.0003 0.0015
5 error ENN

3
Table 4. Performance comarison between HENN and MLP
2

1
Output MLP HENN
0

-1
Concentration 0.79092 0.80795
CSTR
-2 Reactor temp. 1.67039 1.67515
-3
0 50 100 150 200 250
For each model 24 9
Figure 8. Compare the identification error of ENN and MLP for output#2 structure
Total 48 18
Another chosen complex system for evaluating the Motor1 0.18476 0.18128
proposed method is a 3-PSP (Prismatic–Spherical–Prismatic)
parallel robot. The 3-PSP robot has a symmetric structure with 3-PSP Motor2 0.16636 0.16614
three analogous PSP legs [17]. Identification and modeling of
this type of robots is so difficult due to their complexity of Motor3 0.12595 0.12501
structure, dynamics and kinematics compared with serial robots For each model 9 4
[18]. Accordingly, the model based control methodology of the structure
parallel robot is though. Total 27 12

In table.3, mean and standard deviation of the accuracy of

each robot's motor is indicated, after 10 times repeat. As
As a result, these two methods can properly identify the
shown, in this example, the error of HENN is less than MLP in black box systems with almost similar errors. The novelty of
training data. In fact, HENN have a better performance in this method is that the system can be modeled with less
training data contrary to previous example. Nonetheless, with parameters. This feature result in the reduction of
mean and the number of parameters assessment, we found out computational cost. It means the desired balance between the
the superiority of HENN in most cases compare with MLP. approximation accuracy and model complexity is achieved.
Table.4 completely demonstrates the general performance of
two neural network in all data for two systems as well as the
structure. VI. CONCLUSION
60
The purpose of this paper is to utilize the new approach for
ENN identification and modeling nonlinear complex system,
50
MLP
Target
especially the black box plants. In such systems, structure is
not available and just the measured informative input and
40
output data are accessible. Moreover, control of these system
due to the lack of reference models is difficult or impossible. It
30
is noteworthy that the structure and the number of parameters
20
of model significantly affect the computational cost. Neural
network has the best performance in this field owing to
10 possessing the train phase. Additionally, artificial neural
networks present the best approximation accuracy compared
0
0 100 200 300 400 500 600 700 800 900 1000
with the conventional techniques. Given the existence of

Figure 9. Validation of the outputs of HENN and MLP for motor1

1264
2016 24th Iranian Conference on Electrical Engineering (ICEE)
hidden layer, the complexity is incremented. Furthermore, the
number of neurons are growing with increasing complexity.
The computational complexity of ANN is O(n×m).
For resolving this issue, low order model with less
parameters is needed while the approximation accuracy is high.
In this paper, we indicated the single layered HENN is the best
solution due to the absence of hidden layer and having a simple
structure like O(n+1).Therefore with applying the proposed
modification, a single layered HENN becomes differentiable
hence it is well-suited for identification and modeling the
nonlinear behavior of complex systems. Plus, the simulations
demonstrate that HENN is more stable than other neural
networks. Therefore the desired trade-off is achieved.
Meanwhile, with applying the proposed method in some model
based control methodologies, the improvement of controller
speed is observed.
REFERENCES
[1] A. Juditsky, H. Hjalmarsson, A. Benveniste, B. Delyon, L. Ljung, J.
Sjöberg, et al., "Nonlinear black-box models in system identification:
Mathematical foundations," Automatica, vol. 31, pp. 1725-1750, 1995.
[2] H. M. R. Ugalde, J.-C. Carmona, J. Reyes-Reyes, V. M. Alvarado, and J.
Mantilla, "Computational cost improvement of neural network models in
black box nonlinear system identification," Neurocomputing, 2015.
[3] M. ławryńczuk, "Computationally efficient nonlinear predictive control
based on neural Wiener models," Neurocomputing, vol. 74, pp. 401-417,
2010.
[4] E. Lotfi and M.-R. Akbarzadeh-T, "Supervised brain emotional
learning," in Neural Networks (IJCNN), The 2012 International Joint
Conference on, 2012, pp. 1-6.
[5] E. Lotfi and M. R. Akbarzadeh-T, "Emotional brain-inspired adaptive
fuzzy decayed learning for online prediction problems," in Fuzzy
Systems (FUZZ), 2013 IEEE International Conference on, 2013, pp. 1-7.
[6] E. Lotfi, A. Khosravi, M.-R. Akbarzadeh-T, and S. Nahavandi, "Wind
power forecasting using emotional neural networks," in Systems, Man
and Cybernetics (SMC), 2014 IEEE International Conference on, 2014,
pp. 311-316.
[7] M. A. Rahman, R. M. Milasi, C. Lucas, B. N. Araabi, and T. S. Radwan,
"Implementation of emotional controller for interior permanent-magnet
synchronous motor drive," Industry Applications, IEEE Transactions on,
vol. 44, pp. 1466-1476, 2008.
[8] C. Lucas, D. Shahmirzadi, and N. Sheikholeslami, "Introducing
BELBIC: brain emotional learning based intelligent controller,"
Intelligent Automation & Soft Computing, vol. 10, pp. 11-21, 2004.
[9] J. Moren and C. Balkenius, "A computational model of emotional
learning in the amygdala," From animals to animats, vol. 6, pp. 115-
124, 2000.
[10] E. A. Phelps and J. E. LeDoux, "Contributions of the amygdala to
emotion processing: from animal models to human behavior," Neuron,
vol. 48, pp. 175-187, 2005.
[11] G. Winter, J. Periaux, M. Galan, and P. Cuesta, Genetic algorithms in
engineering and computer science: John Wiley & Sons, Inc., 1996.
[12] P. Kittisupakorn, P. Thitiyasook, M. Hussain, and W. Daosud, "Neural
network based model predictive control for a steel pickling process,"
Journal of Process Control, vol. 19, pp. 579-590, 2009.
[13] J. Prakash and K. Srinivasan, "Design of nonlinear PID controller and
nonlinear model predictive controller for a continuous stirred tank
reactor," ISA transactions, vol. 48, pp. 273-282, 2009.
[14] S. Chen and S. Billings, "Neural networks for nonlinear dynamic system
modelling and identification," International journal of control, vol. 56,
pp. 319-346, 1992.
[15] E. Lotfi and M.-R. Akbarzadeh-T, "A novel single neuron perceptron
with universal approximation and XOR computation properties,"
Computational intelligence and neuroscience, vol. 2014, p. 5, 2014.
[16] B. W. Bequette and W. B. Bequette, Process dynamics: modeling,
analysis, and simulation: Prentice Hall PTR Upper Saddle River, NJ,
1998.
[17] A. Rezaei, A. Akbarzadeh, P. M. Nia, and M.-R. Akbarzadeh-T,
"Position, Jacobian and workspace analysis of a 3-PSP spatial parallel
manipulator," Robotics and Computer-Integrated Manufacturing, vol.
29, pp. 158-173, 2013.
[18] S. F. Toloue, M.-R. Akbarzadeh, A. Akbarzadeh, and M. Jalaeian-F,
"Position tracking of a 3-PSP parallel robot using dynamic growing
interval type-2 fuzzy neural control," Applied Soft Computing, vol. 37,
pp. 1-14, 2015.
1265