(IJCNS) International Journal of Computer and Network Security, 47
Vol. 1, No. 2, November 2009
Selection of Proper Activation Functions in Back-
propagation neural networks algorithm for
Transformer Internal Fault Locations
A. Ngaopitakkul and A. Kunakorn
Faculty of Engineering
King Mongkut’s Institute of Technology Ladkrabang, Bangkok 10520, Thailand
knatthap@ kmitl.ac.th
Abstract: This paper presents an analysis on the selection of
an appropriate activation used in neural networks for locating
the internal fault locations of a two-winding three-phase
transformer. A decision algorithm based on a combination of
Discrete Wavelet Transforms and neural networks is developed.
Fault conditions of the transformer are simulated using
ATP/EMTP in order to obtain current signals. The training
process for the neural network and fault diagnosis decision are
implemented using toolboxes on MATLAB/Simulink. Various
activation functions in hidden layers and output layers are
compared in order to find and to select the best activation
function for locating the position of internal faults of the
winding transformer for the winding to ground faults. It is
found that the use of Hyperbolic tangent-function for the hidden
layers, and Linear activation function for the output layer gives
the most satisfactory accuracy in these particular case studies.
Keywords: Internal faults, Discrete Wavelet Transforms,
Back-propagation neural network, Transformer windings
1. Introduction
During the course of recent years, the development of
fault diagnosis techniques for the power transformer has
been progressed with the applications of wavelet transform
and artificial neural networks [1,2,3,4,5]. Many research
reports have paid consideration in effects of the magnetizing
inrush current as well as the discrimination between
magnetizing inrush current and internal faults [2,3,5]. It is
very useful for electrical engineers if the fault positions
along transformer windings can be detected. Therefore, a
decision algorithm used to locate the fault position along the
winding in order to decrease complexity and duration of
maintenance time is required. Neural networks have been
employed in the development of such an algorithm, and
proved to be a powerful tool in fault detection as well as
classification [2,3].
The activation function is a key factor in the artificial
neural network structure. Back-propagation neural networks
support a wide range of activation functions such as sigmoid
function and linear function etc. The choice of activation
function can change the behavior of the back-propagation
neural network considerably. There is no theoretical reason
for selecting a proper activation function. Hence, the
objective of this paper is to consider studies of an
appropriate activation function for the algorithm used in the
detection of internal fault locations along transformer
windings. The activation functions in each hidden layers
and output layer are varied, and the results obtained from
the decision algorithm are investigated.
The decision algorithm is a part of a transformer
protective scheme proposed in this paper. The structure of
the protective scheme is shown in Figure 1. The
simulations, analysis and diagnosis are performed using
ATP/EMTP and MATLAB on a PC Pentium IV 2.4 GHz
512 MB. It is noted that the discrete wavelet transform is
employed in extracting the high frequency component
contained in the internal fault currents of a transformer. The
construction of the decision algorithm is detailed and
implemented with various case studies based on Thailand
electricity transmission and distribution systems.
CT Y Y CT
Analogue Modal Calculating WT Detail
input module mixing unit Differential current Filter (scale 1-5)
Fault
position
Decision
Logic unit
Calculating by
Weight & bias of BP
Start
Detection
Decision Comparison
Trip signal
making unit Coefficient
Figure 1. The transformer protective scheme
2. Simulation
2.1 Transformer winding models
For a computer model of a two-winding three-phase
transformer having primary and secondary windings in each
phase, BCTRAN is a well-known subroutine on ATP/EMTP.
To study internal faults of the transformer, Bastard et al
proposed modification of the BCTRAN subroutine. Normally,
the BCTRAN uses a matrix of inductances with a size of 6x6
to represent a transformer, but with the internal fault
conditions the matrix is adjusted to be a size of 7x7 for
winding to ground faults and of 8x8 for interturn faults [6]. In
the research work of Bastard et al [6], the model was proved to
be validate and accurate due to a comparison with
measurement results. However, the effects of high frequency
components which may occur during the faults are not
included in such a model. Islam and Ledwich [7] described
the characteristics of high frequency responses of a
transformer due to various faults. It has been shown that the
fault types and fault locations have an influence on the
frequency responses of the transformer [7]. In addition, it has
48 (IJCNS) International Journal of Computer and Network Security,
Vol. 1, No. 2, November 2009

been proved that transient based protections using high The process for simulating winding to ground faults based
frequency components in fault currents can be applicable in on the BCTRAN routine of EMTP, can be summarized as
locating and classifying faults on transmission lines [8-9]. It follows:
is, therefore, useful to investigate the high frequency 1st step: Compute matrices [R] and [L] of the power
components superimposed on the fault current signals for a transformer from manufacture test data [11] without
development of a transient based protection for a transformer. considering the winding to ground faults [6].
As a result, in this paper the combination of the transformer
models proposed by Bastard et al [6] as shown in Figure 2, R 1 L 0 
with the high frequency model including capacitances of the [R ] =  M O M  (1)
transformer recommended by IEEE working group [10] as
 0 L R 6 
shown in Figure 3., are used for simulations of internal faults
a t t h e t r a n s f o r m e r w i n d i n g s .
 L 1 L 12 L L 16 
 
L L 26 
[L ] =  21 2
L L
a (2)
M M O M 
 
b
Phase A L 61 L 62 L L 6 
2
2nd step: Modify Equations 5 and 6 to obtain the new
internal winding fault matrices [R ]* and [L ]* as illustrated
in Equations 3-4 [6].
Phase B
 Ra 0 0 0 0 0 0
3 4 0
 Rb 0 0 0 0 0 
0 0 R2 0 0 0 0
[R] =  0
∗
0 0 R3 0 0 0
 (3)

Phase C 0 0 0 0 R4 0 0
 
0 0 0 0 0 R5 0
5 6 0
 0 0 0 0 0 R6 
Primary Secondary
 La M ab M a2 M a3 M a4 M a5 M a6 
M Lb M b2 M b3 M b4 M b5 M b6 
 ba
Figure 2. The modification on ATP/EMTP model for a  M 2a M 2b L2 M 23 M 24 M 25 M 26 
(4)
three-phase transformer with internal faults. [L]∗ =  M 3a M 3b M 32 L3 M 34 M 35 M 36 


Chl  M 4a M 4b M 42 M 43 L4 M 45 M 46 
 
 M 5a M 5b M 52 M 53 M 54 L5 M 56 
M M 6b M 62 M 63 M 64 M 65 L6 
 6a

Chg Clg
3rd step: The inter-winding capacitances and earth
capacitances of the HV and LV windings can be simulated
by adding lumped capacitances connected to the terminals of
the transformer.

2.2 Power System simulation using EMTP

Primary 115/23 kV Secondary
50 MVA A 50 MVA, 115/23 kV two-winding three-phase
transformer was employed in simulations with all
Figure 3. A two-winding transformer with the effects of parameters and configuration provided by a manufacturer
stray capacitances. [11]. The scheme under investigations is a part of Thailand
electricity transmission and distribution system as depicted
The capacitances shown in Figure 3 are as follows: in Figure 4. It can be seen that the transformer as a step
Chg = stray capacitance between the high voltage winding down transformer is connected between two sub-
and ground transmission sections. To implement the transformer model,
Clg = stray capacitance between the low voltage winding simulations were performed with various changes in system
and ground parameters as follows:
Chl = stray capacitance between the high voltage winding - The angles on phase A voltage waveform for the
and the low voltage winding. instants of fault inception were 0o-330o (each step is 30°).
- Internal faults type at the transformer windings (both
primary and secondary) which is winding to ground faults
was investigated.
 (IJCNS) International Journal of Computer and Network Security, 49
- The fault position were designated on any phases of the
transformer windings (both primary and secondary), was
varied at the length of 10%, 20%, 30%, 40%, 50%, 60%,
70%, 80% and 90% measured from the line end of the
windings.
- Fault resistance was 5 Ω.
Figure 4. The system used in simulations studies [12].
The primary and secondary current waveforms, then, can
be simulated using ATP/EMTP, and these waveforms are
interfaced to MATLAB/Simulink for a construction of fault
diagnosis process. Figure 5. illustrates an example of phase
A to ground fault at 10% in length of the high voltage
winding while phase A to ground fault occurred at 10% in
length of the low voltage winding as shown in Figure 6.
Figure 5. Primary and secondary currents for a case of
phase A to ground fault at 10% in length of the high voltage
winding.
Figure 6. Primary and secondary currents for a case of
phase A to ground fault at 10% in length of the low voltage
winding.
With fault signals obtained from the simulations, the
differential currents, which are a deduction between the
Vol. 1, No. 2, November 2009
primary current and the secondary current in all three
phases as well as the zero sequence, are calculated, and the
resultant current signals are extracted using the Wavelet
transform. The coefficients of the signals obtained from the
Wavelet transform are squared for a more explicit
comparison. Figure 7. illustrates an example of an
extraction using Wavelet transform for the differential
currents and zero sequence current from scale1 to scale 5 for
a case of phase A to ground fault at 10% in length of the
high voltage winding while case of phase A to ground fault
at 10% in length of the low voltage winding as shown in
Figure 8.
Figure 7. Wavelet transform of differential currents
(Winding to ground fault at 10% in length of the high voltage winding)
Figure 8. Wavelet transform of differential currents
(Winding to ground fault at 10% in length of the low voltage winding)
3. Fault Detection Algorithm
After applying the Wavelet transform to the differential
currents, the comparison of the coefficients from each scale
is considered. Wavelet transform is applied to the quarter
cycle of current waveforms after the fault inception. With
several trial and error processes, the decision algorithm on
the basis of computer programming technique is
constructed. The most appropriate algorithm for the decision
with all results from the case studies of the system under the
investigations can be concluded as Figure 9.
where,
scale = indicator scale of DWT which considered for
detecting fault
50 (IJCNS) International Journal of Computer and Network Security,
Vol. 1, No. 2, November 2009

(t + 5 µs ) = coefficient from Wavelet transform for the

X diff By performing many simulations, it has been found that
when applying the previously detailed algorithm for
differential current detected from phase X at the time of
detecting internal faults at the transformer winding, the
t+5µs coefficient in scale 1 (50-100 kHz) from DWT seems
max(0 → t) = coefficient from Wavelet transform for the
X diff enough to indicate the internal fault inception of the
differential current detected from phase X at the time from t transformer. As a result, it is unnecessary to use other
=0 to t = t+5µs coefficients from higher scales in this algorithm, and the
coefficients in scale 1 from DWT are used in training
chk = comparison indicator for a change in coefficient
X diff
processes for the neural networks later.
diff diff diff
from Wavelet transform ( A check , B check , C check ), used for
separation between normal conditions and faults 4. Neural Network Decision Algorithm
t1 = 5 µsec (depending on the sampling time used in From the simulated signals, DWT is applied to the
ATP/EMTP) quarter cycle of differential current waveforms after the fault
inception. The coefficients of scale 1 obtained using the
Start
wavelet transforms are used for training and test processes
of the BPNN. In this paper, a structure of a back
Differential Current Signal propagation neural network consists of three layers which
1. All three phases
2. Zero sequence are an input layer, two hidden layers and an output layer as
shown in Figure 10. Each layer is connected with weights
and bias. In addition, the activation function is a key factor
Wavelet scale 1-5
in the artificial neural network structure. The choice of
activation function can change the behavior of the back-
Differential current in each
propagation neural network considerably. Hence, the
phases activation functions in each hidden layers and output layer
are varied as illustrate in Table 1 in order to select the best
activation function for locating the positions along the
For scale = 1
transformer windings due to winding to ground faults of a
two-winding transformer.

diff
X chk =0 TABLE 1: Activation functions in all hidden layers and
output layers for training neural networks
For t = 5 us : 100 ms
Activation function in
diff
diff
X chk = 0 and find X max(0 → t) )
first hidden layer second hidden layer output layer

t = t + 5us
Linear function
Logistic
No Logistic sigmoid
Hyperbolic sigmoid function
t = 100 ms
(t + 5us ) ≥ 5 * X max(0→ t) )
(X diff diff Hyperbolic Tangent sigmoid
No Tangent sigmoid
function Linear function
Hyperbolic
Yes Yes Tangent sigmoid function Logistic sigmoid
Hyperbolic Tangent sigmoid
diff
X chk = 1 and record time (t+5us) Linear function
Logistic
sigmoid function Logistic sigmoid
Logistic Hyperbolic Tangent sigmoid
sigmoid function Linear function
Hyperbolic
Tangent sigmoid function Logistic sigmoid
No Hyperbolic Tangent sigmoid
X diff
chk =1 scale = scale+1

Yes
A training process was performed using neural network
toolboxes in MATLAB. It can be divided into three parts as
No
follows [13]:
diff
Sum of X chk ≥2 1 The feedforward input pattern, which has a propagation
of data from the input layer to the hidden layer and finally to
Yes the output layer for calculating responses from input
Normal
patterns illustrated in Equations 5 and 6.
Fault condition

(lw ( ) )
condition

Figure 9. Flowchart for detecting the phase with a fault a2 = f 2 2,1

* f 1 iw1,1 * p + b1 + b 2 , (5)
condition (
o / p ANN = f 3 lw3,2 * a 2 + b 3 . ) (6)
 (IJCNS) International Journal of Computer and Network Security, 51
Vol. 1, No. 2, November 2009

Input Layer 1st Hidden Layer 2 nd Hidden Layer Output Layer

2 ,1 3, 2
iw11,,11 n11 a11 lw1,1 n12 a12 lw1,1 n13 a13
P1
∑ f1 ∑ f 2
∑ f3

b11 b12 b13

1 1 1
P2
n 12 a 12 n22 a22 n23 a23
∑ f1 ∑ f 2
∑ f3
P3
• b22 • b23 •
• • b 12 •
• •
•
• •
•
• •
•
•
•
•
1 • • 1 • • 1 • •

n1S1 a 1S 1 nS22 a S2 2 nS3 3 a 3S 3

PR1 ∑ f1 ∑ f 2
∑ f 3
iw1S,1,R lwS2,21, S1 lwS3,32,S 2
b1S1 bS22 bS33
1 1 1

Figure 10. Back-propagation with two hidden layers

where, repeating the cycle of the training process. The training

p = input vector of BPNN procedure was stopped when reaching the final number of
iw1,1 = weights between input and the first hidden layer neurons for the first hidden layer or the MAPE of test sets
lw2,1 = weights between the first and the second hidden was less than 0.5%. The training process can be
layers summarized as Figure 11.
lw3,2 = weights between the second hidden layer and output
layers
b1, b2= bias in the first and the second hidden layers
respectively
b3 = bias in output layers
f1, f2 = activation function (Hyperbolic tangent sigmoid
function : tanh)
f3 = activation function (Linear function)

2 The back-propagation for the associated error between

outputs of neural networks and target outputs; The error is
fed to all neurons in the next lower layer, and also used as
an adjustment of weights and bias.
3 The adjustment of the weights and bias by Levenberg-
Marquardt (trainlm). This process is aimed at trying to
match between the calculated outputs and the target outputs.
Mean absolute percentage error (MAPE) as an index for
efficiency determination of the back-propagation neural
networks is computed in Equation 7.
1 n o / p ANNi − o / pTARGETi
MAPE = * ∑ * 100% (7)
n i =1 o / pTARGETi
where,
n = number of test sets

As a result, a structure of the back propagation neural

network consists of 4 neuron inputs, two hidden layers and 1
neuron output. The inputs pattern are the maximum
coefficients details (cD1) in scale 1 at ¼ cycle of phase A, B,
C and zero sequence for post-fault differential currents as
mentioned in the previous section. The output variables of
the neural networks are designated as values range 0.1 to
0.9 which corresponding to length of the winding that fault
occurs.
In this training process, a number of neurons in both
hidden layers were increased as well as varying the
activation functions in all hidden layers and the output layer
in order to select the best performance. Input data sets are
normalized and divided into 216 sets for training and 108
sets for tests. During the training process, the weight and Figure 11. Flowchart for the training process.
biases were adjusted, and there were 20,000 iterations in
order to compute the best value of MAPE. The number of
neurons in both hidden layers was increased before
52 (IJCNS) International Journal of Computer and Network Security,
Vol. 1, No. 2, November 2009

Figure 12. Comparison of average error for fault position at various activation functions between each transformer
windings.

Table 2 : Average error of test sets for locating of fault

Activation function in High voltage winding Low voltage winding
Maximum Minimum Average Maximum Minimum Average
the first hidden layer the second hidden layer the output layer
error error error error error error
Hyperbolic tangent Logistic sigmoid function Linear function 0.0414 0.0000 0.0099 0.1693 0.0000 0.0309
sigmoid function Hyperbolic tangent sigmoid function Linear function 0.0322 0.0001 0.0089 0.0621 0.0001 0.0211
Logistic sigmoid Logistic sigmoid function Linear function 0.0497 0.0000 0.0094 0.1759 0.0000 0.0307
function Hyperbolic tangent sigmoid function Linear function 0.0483 0.0001 0.0098 0.1709 0.0006 0.0377

After the training process, the algorithm was employed in
order to locate fault positions in the winding transformer.
Case studies were varied so that the algorithm capability can
be verified. Case studies were performed with various types
of fault at each position in the transformer. The total
number of the case studies was 216. The result obtained
from various activation functions of test set both high
voltage and low voltage winding as shown in Figure 12.
From Figure 12, it can be seen that there are four cases
activation functions with average error less than 5% as
follows:
- Hyperbolic tangent – Logistic – Linear.
- Hyperbolic tangent – Hyperbolic tangent – Linear.
- Logistic – Logistic – Linear.
- Logistic – Hyperbolic tangent – Linear.
When the training process was completed, the algorithm
was implemented to locate fault positions due to winding to
ground faults along the transformer windings. The results
obtained from the algorithm proposed in this paper are
shown in Table 2. It can be seen that the accuracy from
Hyperbolic tangent – Hyperbolic tangent – Linear activation
function case is highly satisfactory.
From Figure 13, the comparison of average error at
various lengths of the winding among four case activation
functions is shown. It can be seen that the average error of
fault locations from the high voltage winding is 2.5% while
the average error of low voltage winding is 6% at various
lengths of the transformer winding.
 (IJCNS) International Journal of Computer and Network Security, 53
Vol. 1, No. 2, November 2009

(a) High Voltage (b) Low Voltage

Figure 13. Comparison of average errors for fault positions at various lengths of windings among various activation
functions.

(a) High Voltage (b) Low Voltage

Figure 14. Comparison of average error for fault position at various lengths of the winding among phases that fault occur
with Hyperbolic tangent – Hyperbolic tangent – Linear are activation functions in each layers.
54 (IJCNS) International Journal of Computer and Network Security,
Vol. 1, No. 2, November 2009

From Fig. 14, it can be seen that Hyperbolic tangent –

Hyperbolic tangent – Linear as activation function in each 5. Conclusion
layers, are tested with various fault types on both high
voltage and low voltage windings of the three-phase In this paper, Studies of an appropriate activation
transformer, the accuracy of fault locations from the function for the decision algorithm used in the detection of
prediction of the decision algorithm is highly satisfactory as internal fault locations along transformer winding have been
well as Table 3-6. discussed. The maximum coefficient from the first scale at
¼ cycle of phase A, B, and C of post-fault differential
Table 3 : Results of phase A to ground fault at high voltage current signals and zero sequence current obtained by the
winding with various inception angles. wavelet transform have been used as an input for the
(Fault position at 10% in length of the winding)
training process of a neural network in a decision algorithm
with a use of the back propagation neural networks. The
Inception Prediction
Actual activation functions in each hidden layers and output layer
Fault Type angle
position (%) have been varied, and the results obtained from the decision
(Degree) Output Error
algorithm have been investigated with the variation of fault
90 0.1 0.1125 0.0125 inception angles, fault types and fault locations. The results
Phase A to have illustrated that the use of Hyperbolic tangent sigmoid
150 0.1 0.1243 0.0243
ground fault function in the first and the second layers with Linear
(HV) 240 0.1 0.1121 0.0121
function in the output layer is the most appropriate scheme
300 0.1 0.1169 0.0169
for the internal fault detection of the transformer windings
as summarized in Table II. This technique should be useful
in checking and repairing the transformer when winding to
Table 4 : Results of phase A to ground fault at high voltage ground faults occur. The further work will be the
winding with various lengths of the winding. improvement of the algorithm so that positions of interturn
(Inception angle of 240o) faults along the windings of the transformer can be
Inception Prediction identified.
Actual
Fault Type angle
position (%)
(Degree) Output Error References
240 0.2 0.2005 0.0005 [1] A.G. Phadke and J.S. Thorp, A new computer-based
Phase A to
240 0.4 0.4054 0.0054 flux restrained current-differential relay for power
ground fault
240 0.6 0.6042 0.0042 transformer protection, IEEE Trans. Power Appar. Syst.
(HV)
240 0.8 0.8025 0.0025
PAS-102 (5) (1983) 3624-3629.
[2] T.S. Sidhu and M.S. Sachdev, On-line identification of
magnetizing inrush current and internal faults in three-
phase transformers, IEEE Trans. Power Delivery 7 (4)
Table 5 : Results of phase A to ground fault at low voltage
(1992) 1885-1891.
winding with various inception angles.
[3] Y.Zhang, X.Ding, Y.Liu and P.J. Griffin, An artificial
(Fault position at 10% in length of the winding)
neural network approach to transformer fault diagnosis,
Inception Prediction IEEE Trans. Power Delivery 11 (4) (1996) 1836-1841.
Actual
Fault Type angle [4] M.G. Morante and D.W. Nocoletti, A wavelet-based
position (%)
(Degree) Output Error differential transformer protection, IEEE Trans. Power
60 0.1 0.0886 0.0114 Delivery 14 (4) (1999) 1352-1358.
Phase A to [5] O.A.S. Youssef, A wavelet-base technique for
120 0.1 0.136 0.036
ground fault discrimination between faults and magnetizing inrush
210 0.1 0.1386 0.0386
(LV) currents in transformers, IEEE Trans. Power Delivery
330 0.1 0.0568 0.0432 18 (1) (2003) 170-176.
[6] P. Bastard, P. Bertrand and M. Meunier, A transformer
model for winding fault studies, IEEE Trans. Power
Table 6 : Results of phase A to ground fault at low voltage Delivery 9 (2) (1994) 690-699.
winding with various lengths of the winding. [7] S. M. Islam and G. Ledwich, Locating transformer
(Inception angle of 210o) faults through sensitivity analysis of high frequency
Inception modeling using transfer function approach, IEEE
Actual Prediction
Fault Type angle International Symposium on Electrical Insulation,
position (%) (1996) 38-41.
(Degree) Output Error
[8] Z. Q. Bo, M. A. Redfern and G. C. Weller, Positional
210 0.2 0.1943 0.0057 Protection of Transmission Line Using Fault
Phase A to
210 0.4 0.4265 0.0265 Generated High Frequency Transient Signals, IEEE
ground fault
210 0.6 0.5871 0.0129 Trans. Power Delivery 15 (3) (2000) 888-894.
(LV)
210 0.8 0.7968 0.0032 [9] P.Makming, S. Bunjongjit, A.Kunakorn, S.
Jiriwibhakorn and M. Kando, Fault diagnosis in
 (IJCNS) International Journal of Computer and Network Security, 55
Vol. 1, No. 2, November 2009

transmission lines using wavelet transforms, in:
Proceedings of IEEE Transmission and Distribution
Conference, pp. 2246-2250, 2002.
[10] IEEE working group 15.08.09, Modeling and analysis
of system transients using digital programs, IEEE PES
special publication
[11] ABB Thailand, Test report no. 56039.
[12] “Switching and Transmission Line Diagram”,
Electricity Generation Authorisation Thailand (EGAT).
[13] A. Ngaopitakkul and A. Kunakorn, “Internal Fault
Classification in Transformer Windings using
Combination of Discrete Wavelet Transforms and
Back-propagation Neural Networks,” International
Journal of Control, Automation, and Systems (IJCAS),
pp. 365-371, June 2006.
Anantawat Kunakorn graduated with
B.Eng (Hons) in electrical engineering from
King Mongkut’s Institute of Technology
Ladkrabang, Bangkok, Thailand in 1992. He
received his M.Sc in electrical power
engineering from University of Manchester
Institute of Science and Technology, UK in
1996, and Ph.D. in electrical engineering
from Heriot-Watt University, Scotland, UK
in 2000. He is currently an associate professor at the department of
electrical engineering, King Mongkut’s Institute of Technology
Ladkrabang, Bangkok, Thailand. He is a member of IEEE and IEE.
His research interest is electromagnetic transients in power
systems.

Authors Profile
Atthapol Ngaopitakkul graduated with
B.Eng, M.Eng and D.Eng in electrical
engineering from King Mongkut’s Institute
of Technology Ladkrabang, Bangkok,
Thailand 2002, 2004 and 2007 respectively.
His research interests are on the
applications of wavelet transform and neural
networks in power system analysis.