1

CONTENT

Chapter No.
Description
Page no.
COVER PAGE AND TITLE PAGE i - ii
CERTIFICATE Iii
ACKNOWLEDGEMENT Iv
SYNOPSIS

V
1. INTRODUCTION TO ARTIFICIAL NEURAL
NETWORK
3-8
1.1 INTRODUCTION 1
1.2 HISTORICAL BACKGROUND 2
1.3 BIOLOGICAL INSPIRATION 2


2. NEURAL NETWORK IN PROCESS CONTROL

9
3. FEATURE OF ANN

10-11
5. NETWORK FOUNDATION


12
6. NEURAL NETWORK ARCTITECTURE 13-14
7. BASIC ELEMENT OF ANN 15-16
8. DIFFERENT MODELS OF ANN 17-19
9. TRANING OF ANN 20-28
9.1 LEARNING PROCESS
9.2 LEARNING TECHNIQUE
9.3 ANN BASED CONTROL CONFIGURATION 26



10. RECENT ADVANCEMENT IN ANN 29-30
11. ADVANTAGES AND DISADVANTAGES OF ANN 31
12. APPLICATION OF NEURAL NETWORK 32-33
13. CONCLUSION 34
14. REFRENCE 35





2


LIST OF FIGURE
S.no Description Page No.
1
Biological Neuron
7
2
Neural network
8
3
Two layer neural network
14
4
Artificial neuron
15
5
McCulloch-Pitts Model of Neuron
17
6
McCulloch-Pitts Model
19
7 Linear Threshold Function 19
8
Basic Model of DIC
24
9
Basic Model of DAC
25
10
Basic Model of IAC
26
11
Basic Model of MIC
27
12
Basic Model of MAC
28











3

INTRODUCTION OF ARTIFICIAL NEURAL NETWORK

1.1 INTRODUCTION

Definition:
An Artificial Neural Network (ANN) is an information-processing paradigm that
is inspired by the way biological nervous systems, such as the brain, process information. The
key element of this paradigm is the novel structure of the information processing system. It is
composed of a large number of highly interconnected processing elements (neurons) working in
unison to solve specific problems. ANNs, like people, learn by example. An ANN is configured
for a specific application, such as pattern recognition or data classification, through a learning
process. Learning in biological systems involves adjustments to the synaptic connections that
exist between the neurons.
Introduction:
ANNS are used as a solution to various problems; however, their success as an
intelligent pattern recognition methodology has been most prominently advertised. The most
important, and attractive, feature of ANNs is their capability of learning (generalizing) from
example (extracting knowledge from data). ANNs can do this without any pre-specified rules
that define intelligence or represent an experts knowledge. This feature makes ANNs a very
popular choice for gene expression analysis and sequencing. Due to their power and flexibility,
ANNS have even been used as tools for relevant variable selection, which can in turn greatly
increase the experts knowledge and understanding of the problem.
A neural network is characterized by its pattern of connections between the neurons referred to
as network architecture and its method of determining the weights on the connections called
training or learning algorithm. The weights are adjusted on the basis of data. In other words,
neural networks learn from examples and exhibit some capability for generalization beyond the
training data. This feature makes such computational models very appealing in application
domains where one has little or incomplete understanding of the problem to be solved, but where
training data is readily available. Neural networks normally have great potential for parallelism,
since the computations of the components are largely interdependent of each other.
Artificial neural networks are viable computational models for a wide variety of
problems. Already, useful applications have been designed, built, and commercialized for
various areas in engineering, business and biology. These include pattern classification, speech
synthesis and recognition, adaptive interfaces between human and complex physical systems,
4

function approximation, image compression, associative memory, clustering, forecasting and
prediction, combinatorial optimization, nonlinear system modeling, and control. Although they
may have been inspired by neuroscience, the majority of the networks have close relevance or
counterparts to traditional statistical methods such as non-parametric pattern classifiers,
clustering algorithm, nonlinear filters, and statistical regression models.
An input is presented to the neural network and a corresponding desired or target response set at
the output (when this is the case the training is called supervised). . The error information is fed
back to the system and adjusts the system parameters in a systematic fashion (the learning rule).
The process is repeated until the performance is acceptable. It is clear from this description that
the performance hinges heavily on the data. If one does not have data that cover a significant
portion of the operating conditions or if they are noisy, then neural network technology is
probably not the right solution.
On the other hand, if there is plenty of data and the problem is poorly understood to derive an
approximate model, then neural network technology is a good choice. This operating procedure
should be contrasted with the traditional engineering design, made of exhaustive subsystem
specifications and intercommunication protocols. In artificial neural networks, the designer
chooses the network topology, the performance function, the learning rule, and the criterion to
stop the training phase, but the system automatically adjusts the parameters. So, it is difficult to
bring a priori information into the design, and when the system does not work properly it is also
hard to incrementally refine the solution.
But ANN-based solutions are extremely efficient in terms of development time and resources,
and in many difficult problems artificial neural networks provide performance that is difficult to
match with other technologies. Denker 10 years ago said that "artificial neural networks are the
second best way to implement a solution" motivated by the simplicity of their design and
because of their universality, only shadowed by the traditional design obtained by studying the
physics of the problem. At present, artificial neural networks are emerging as the technology of
choice for many applications, such as pattern recognition, prediction, system identification, and
control.







5

1.2 Historical Background

Neural network simulations appear to be a recent development. However, this field was
established before the advent of computers, and has survived at least one major setback and
several eras.
Many important advances have been boosted by the use of inexpensive computer emulations.
Following an initial period of enthusiasm, the field survived a period of frustration and
disrepute. During this period when funding and professional support was minimal, relatively
few researchers made important advances. The first artificial neuron was produced in 1943 by
the neurophysiologist Warren McCulloch and the logician Walter Pits. But the technology
available at that time did not allow them to do too much.
1. First Attempts: There were some initial simulations using formal logic. McCulloch and
Pitts (1943) developed models of neural networks based on their understanding of
neurology. These models made several assumptions about how neurons worked. Their
networks were based on simple neurons, which were considered to be binary devices with
fixed thresholds. The results of their model were simple logic functions such as "a or b"
and "a and b". Another attempt was by using computer simulations. Two groups (Farley
and Clark, 1954; Rochester, Holland, Haibit and Duda, 1956). The first group (IBM
researchers) maintained closed contact with neuroscientists at McGill University. So
whenever their models did not work, they consulted the neuroscientists. This interaction
established a multidisciplinary trend, which continues to the present day.
2. Promising & Emerging Technology: Not only was neuroscience influential in the
development of neural networks, but psychologists and engineers also contributed to the
progress of neural network simulations. Rosenblatt (1958) stirred considerable interest
and activity in the field when he designed and developed the Perceptron. The Perceptron
had three layers with the middle layer known as the association layer. This system could
learn to connect or associate a given input to a random output unit.
Another system was the ADALINE (ADAptive LInear Element), which was developed in
1960 by Widrow and Hoff (of Stanford University). The ADALINE was an analogue
electronic device made from simple components. The method used for learning was
different to that of the Perceptron; it employed the Least-Mean-Squares (LMS) learning
rule.
3. Period of Frustration & Disrepute: In 1969 Minsky and Papert wrote a book in which
they generalized the limitations of single layer Perceptions to multilayered systems. In
the book they said: "...our intuitive judgment that the extension (to multilayer systems) is
sterile". The significant result of their book was to eliminate funding for research with
neural network simulations. The conclusions supported the disenchantment of researchers
in the field. As a result, considerable prejudice against this field was activated.
4. Innovation: Although public interest and available funding were minimal, several
researchers continued working to develop neuromorphically based computational
methods for problems such as pattern recognition.
6

During this period several paradigms were generated which modern work continues to
enhance. Grossberg's (Steve Grossberg and Gail Carpenter in 1988) influence founded a
school of thought, which explores resonating algorithms. They developed the ART
(Adaptive Resonance Theory) networks based on biologically plausible models.
Anderson and Kohonen developed associative techniques independent of each other.
Klopf (A. Henry Klopf) in 1972 developed a basis for learning in artificial neurons based
on a biological principle for neuronal learning called homeostasis.
The original network was published in 1975 and was called the Cognitron.
5. Re-Emergence: Progress during the late 1970s and early 1980s was important to the re-
emergence on interest in the neural network field. Several factors influenced this
movement. For example, comprehensive books and conferences provided a forum for
people in diverse fields with specialized technical languages, and the response to
conferences and publications was quite positive. The news media picked up on the
increased activity and tutorials helped disseminate the technology. Academic programs
appeared and courses were introduced at most major Universities (in US and Europe).
Attention is now focused on funding levels throughout Europe, Japan and the US and as
this funding becomes available, several new commercial with applications in industry and
financial institutions are emerging.
6. Today: Significant progress has been made in the field of neural networks-enough to
attract a great deal of attention and fund further research. Advancement beyond current
commercial applications appears to be possible, and research is advancing the field on
many fronts. Neutrally based chips are emerging and applications to complex problems
developing. Clearly, today is a period of transition for neural network technology











7

1.3 BIOLOGICAL INSPIRATION
Neural networks grew out of research in Artificial Intelligence; specifically, attempts to mimic
the fault-tolerance and capacity to learn of biological neural systems by modeling the low-level
structure of the brain (see Patterson, 1996).These are based upon a high-level model of reasoning
processes (specifically, the concept that our reasoning processes are built upon manipulation of
symbols). It became rapidly apparent that these systems, although very useful in some domains,
failed to capture certain key aspects of human intelligence. According to one line of speculation,
this was due to their failure to mimic the underlying structure of the brain. The axons of one cell
connect to the dendrites of another via a synapse. When a neuron is activated, it fires an
electrochemical signal along the axon. This signal crosses the synapses to other neurons, which
may in turn fire. A neuron fires only if the total signal received at the cell body from the
dendrites exceeds a certain level (the firing threshold).
The strength of the signal received by a neuron (and therefore its chances of firing)
critically depends on the efficacy of the synapses. Each synapse actually contains a
gap, with neurotransmitter chemicals poised to transmit a signal across the gap. One
of the most influential researchers into neurological systems (Donald Hebb)
postulated that learning consisted principally in altering the "strength" of synaptic
connections. For example, in the classic Pavlovian conditioning experiment, where a
bell is rung just before dinner is delivered to a dog, the dog rapidly learns to associate
the ringing of a bell with the eating of food.
The synaptic connections between the appropriate part of the auditory cortex and the
salivation glands are strengthened, so that when the auditory cortex is stimulated by
the sound of the bell the dog starts to salivate. Recent research in cognitive science, in
particular in the area of non-conscious information processing, have further
demonstrated the enormous capacity of the human mind to infer ("learn") simple
input-output co variations from extremely complex stimuli (e.g., see Lewicki, Hill,
and Czyzewska, 1992).
manages to perform extremely complex tasks. Of course, there is a great deal of
complexity in the brain which has not been discussed here, but it is interesting that
artificial neural networks can achieve some remarkable results using a model not
much more complex than this

Fig no. 1: Biological Neuron
8

NEURAL NETWORKS
Artificial neural networks have emerged from the studies of how brain performs. The human
brain consists of many millions of individual processing elements, called neurons that are highly
interconnected.
Information from the outputs of the neurons, in the form of electric pulses is received by the cells
at connections called synapses. The synapses connect to the cell inputs, or dendrites and the
single output pf the neuron appears at the axon. An electric pulse is sent down the axon when the
total input stimuli for all of the dendrites exceed a certain threshold.
Artificial neural networks are made up of simplified individual models of the biological neuron
that are connected together to form a network. Information is stored in the network in the form of
weights or different connections strengths associated with synapses in the artificial neuron
models.
Many different types of neural networks are available and multi layer neural networks are the
most popular which are extremely successful in pattern reorganization problems. An artificial
neuron model is shown below. Each neuron input is weighted by W. changing the weights of an
element will alter the behavior of the whole network. The output y is obtained by summing the
weighted inputs to the neuron and passing the result through a non-linear activation function, f ().


Fig no 2: Neural network

Multi layer networks consists of an input layer, a hidden layer are made up of no. of
nodes. Data flows through the network in one direction only, from input to output; hence
this type of network is called a feed-forwarded network. A two-layered network is shown
below.
9

NEURAL NETWORKS IN PROCESS CONTROL
Artificial neural networks are implemented as software packages in computers and being
used to incorporate of artificial intelligence in control system. ANN is basically
mathematical tools which are being designed to employ principles similar to neurons
networks of biological system. ANN is able to emulate the information processing
capabilities of biological neural system. ANN has overcome many of the difficulties that t
conventional adaptive control systems suffer while dealing with non linear behavior of
process.

PROCEDURES FOR ANN SYSTEM ENGINEERING
In realistic application the design of ANN system is complex, usually iterative and
interactive task. Although it is impossible to provide an all inclusive algorithmic
procedure, the following highly interrelated, skeletal steps reflect typical efforts and
concerns. The plethora of possible ANN design parameters include:
The interconnection strategy/network topology/network structure.
Unit characteristics (may vary within the network and within subdivisions within
the network such as layers).
Training procedures.
Training and test sets.
Input/output representation and pre- and post-processing.








10

FEATURES OF ANN
Their ability to represent nonlinear relations makes them well suited for non linear
modeling in control systems.
Adaptation and learning in uncertain system through off line and on line weight
adaptation
Parallel processing architecture allows fast processing for large-scale dynamic
system.
Neural network can handle large number of inputs and can have many outputs.

Neural network architecture have learning algorithm associated with them. The most
popular network architecture used for control purpose is multi layered neural network
[MLNN] with error propagation [EBP] algorithm.

BENEFITS OF NEURAL NETWORKS
It is apparent that a neural network derives its computational power through, first,
its massively parallel distributed structure and, second its ability to learn and therefore
generalize. Generalization refers to the neural network producing reasonable outputs for
inputs not encountered during training. These two information-processing capabilities
make it possible for neural networks to solve complex problems that are currently
intractable. A complex problem of interest is decomposed into a number of relatively
simple tasks, and neural networks are assigned a subset of tasks that match their inherent
capabilities. The use of neural networks offers the following useful properties and
capabilities.
Nonlinearity, An artificial neuron can be linear or nonlinear. A neural network
made up of an interconnection of nonlinear neurons, is itself nonlinear. Nonlinearity is a
highly important property, particularly if the underlying physical mechanism responsible
for generation of the input signal (e.g., speech signal) is inherently nonlinear.
Input-Output Mapping, A popular paradigm of learning called learning with a
teacher or supervised learning involves modification of the synaptic weights of a neural
network by applying a set of labeled training samples or task examples. Each example
consists of a unique input signal and a corresponding desired response. The network is
presented with an example picked at random from the set, and the synaptic weights of the
network are modified to minimize the difference between the desired response and the
11

actual response of the network produced by the input signal in accordance with an
appropriate statistical criterion. Thus the network learns from the examples by
constructing an input-output mapping for the problem at hand.
Adaptivity, Neural networks have a built-in capability to adapt their synaptic
weights to changes in the surrounding environment. In particular, a neural network
trained to operate in a specific environment can be easily restrained to deal with minor
changes in the operating environmental conditions. Moreover, when it is operating in a
non-stationary environment, a neural network can be designed to change its synaptic
weights in real time.
Evidential Response, In the context of pattern classification, a neural network can
be designed to provide information not only about which pattern to select, but also about
the confidence in the decision made. This latter information may be used to reject
ambiguous patterns, should they rise, and thereby improving the classification
performance of the network.
Contextual Information, Knowledge is represented by the very structure and
activation state of a neural network. Every neuron in the network is potentially affected
by the global activity of all other neurons in the network. Consequently, contextual
information is dealt with naturally by a neural network.
Fault Tolerance, A neural network implemented in hardware form, has the
potential to be inherently fault tolerant, or capable of robust computation, in the sense
that its performance degrades gracefully under adverse operating conditions. This can be
attributed to its massively distributed nature of information stored in the network. Thus,
in principle, a neural network exhibits a graceful degradation in performance rather than
catastrophic failure.
VLSI Implement ability, The massively parallel nature of a neural network makes
it potentially fast for the computation of certain tasks. This same feature makes a neural
network well suited for implementation using very-large-scale-integrated technology.
One particular beneficial virtue of VLSI is that it provides a means of capturing truly
complex behavior in a highly hierarchical fashion.
Neurobiological Analogy, The design of a neural network is motivated by analogy
with the brain, which is living proof that fault tolerant parallel processing is not only
physically possible but also fast and powerful.


12

NEURAL NETWORK FOUNDATIONS
Artificial Neural networks (ANNs) belong to the adaptive class of techniques in the
machine learning arena. ANNS are used as a solution to various problems; however, their
success as an intelligent pattern recognition methodology has been most prominently
advertised. Most models of ANNs are organized in the form of a number of processing units
called artificial neurons, or simply neurons, and a number of weighted connections referred to
as artificial synapses between the neurons. The process of building an ANN, similar to its
biological inspiration, involves a learning episode. During learning episode, the network
observes a sequence of recorded data, and adjusts the strength of its synapses according to a
learning algorithm and based on the observed data. The process of adjusting the synaptic
strengths in order to be able to accomplish a certain task, much like the brain, is called
learning. Learning algorithms are generally divided into two types, supervised and
unsupervised. The supervised algorithms require labeled training data. In other words, they
require more a priori knowledge about the training set.
MODELS OF A NEURON
A neuron is an information-processing unit that is fundamental to the operation of a
neural network. Fig.2.1a shows the model of a neuron, which forms the basis for designing
artificial neural networks. The three basic elements of the neuronal model are
1. A set of synapses or connecting links, each of which is characterized by a weight or
strength of its own. Specifically, a signal x
j
at which the input of synapse j connected
to neuron k is multiplied by the synaptic weight w
jk
.
2. An adder for summing the input signals, weighted by the respective synapses of the
neuron; the operations described here constitutes a linear combiner.
3. An activation function for limiting the amplitude of the output of a neuron. The
activation is also referred to as a squashing function as it squashes the permissible
amplitude range of the output signal to some finite value.


13

NETWORK ARCHITECTURES:
The manner in which the neurons of a neuron network are structured is intimately
linked with the learning algorithm used to train the network. Network structure can be broadly
divided into three classes of network architectures.

Single-Layer Feed forward Networks
In a layered neural network the neurons are organized in the form of layers. The
simplest form of a layered network consists of an input layer of source nodes those projects
onto an output layer of neurons, but not vice-versa. In other words, this network is strictly a
feed forward or acyclic type.

Such a network is called a single-layer network, with the designation single layer
referring to the output layer of computational nodes. The input layer of source nodes are not
counted as no computation is performed here.

Multilayer Feed forward Networks
The second class of a feed forward neural network distinguishes itself by the
presence of one or more hidden layers, whose computation nodes are correspondingly called
hidden neurons or hidden units. The function of hidden neurons is to intervene between the
external input and the network output in some useful manner. By adding one or more hidden
layers, the network is enabled to extract higher-order statistics.

In a rather loose sense the network acquires a global perspective despite its local
connections due to the extra set of synaptic connections and extra dimension of neural
14

interactions. The ability of hidden neurons to extract higher-order statistics is particularly
valuable when the size of the input is large.

Recurrent Networks
A recurrent network distinguishes itself from a forward neural network in that it has
at least one feedback loop. For example, a recurrent network may consist of a single layer of
neurons with each neuron feeding its output signal back to the inputs of all the other neurons,
as illustrated in the architectural graph. The presence of feedback loops has a profound impact
on the learning compatibility of the network and its performance. Moreover, the feedback
loops involve the use of particular branches composed of unit-delay elements (denoted by z
-1
),
which result in a nonlinear dynamical behavior, assuming that the neural network contains
nonlinear units.

Fig no 3. :Two layer neural network



15

Basic Element of ANN
Artificial neurons are information-processing units that are only approximations (usually very
crude ones) of the biological neuron. Three basic elements of the artificial neuron can be
identified as-:


Fig 4-: Artificial neuron

Input (xi)
Typically, these values are external stimuli from the environment or come from the outputs of
other artificial neurons. They can be discrete values from a set, such as {0,1}, or real-valued
numbers.

Weights (wi)
These are real-valued numbers that determine the contribution of each input to the neuron's
weighted sum and eventually its output. The goal of neural network training algorithms is to
determine the best possible set of weight values for the problem under consideration. Finding
16

the optimal set is often a trade-off between computation time and minimizing the network error.

Threshold (u)
The threshold is referred to as a bias value. In this case, the real number is added to the
weighted sum. For simplicity, the threshold can be regarded as another input / weight pair,
where w0 = u and x0 = -1.
Activation Function (f)
The activation function for the original McCulloch-Pitts neuron was the unit stepfunction.
However, the artificial neuron model has been expanded to include other functions such as the
sigmoid, piecewise linear, and Gaussian.








17

Different models of artificial neuron

1.Adaline model
2.Madaline model
3.Rosenballet model
4.Mcculloch pits model
5.Widrow hoff model
6.Kohonen model

1. The adaptive linear element (Adaline)
In a simple physical implementation

Fig no.5 :The McCulloch-Pitts Model of Neuron
18


this device consists of a set of controllable resistors connected to a circuit which can sum up
currents caused by the input voltage signals. Usually the central block, the summer, is also
followed by a quantiser which outputs either +1 of 1,depending on the polarity of the sum.
Although the adaptive process is here exemplified in a case when there is only one output, it
may be clear that a system with many parallel outputs is directly implementable by multiple units
of the above kind.
If the input conductances are denoted by wi, i = 0; 1; : : : ; n, and the input and output
signals by xi and y, respectively, then the output of the central block is defined to be:
where = w0.



2 .Mcculloch pitts model
The early model of an artificial neuron is introduced by Warren McCulloch and Walter Pitts in
1943. The McCulloch-Pitts neural model is also known as linear threshold gate. It is a neuron of
a set of inputs and one output The linear threshold gate simply classifies the
set of inputs into two different classes. Thus the output is binary. Such a function can be
described mathematically using these equations:


19


are weight values normalized in the range of either or and
associated with each input line, is the weighted sum, and is a threshold constant. The
function is a linear step function at threshold as shown in figure. The symbolic
representation of the linear threshold gate is shown in figure :


Fig 6-: The McCulloch-Pitts Model of Neuron




Fig 7: Linear Threshold Function
20

TRAINING OF NEURAL NETWORKS
Neural networks are models that may be used to approximate summaries, classify,
generalize or otherwise represent real situations. Before models can be used they have to be
trained or made to fit into the representative data. The model parameters, e.g., number of
layers, number of units in each layer and weights of the connections between them, must be
determined. In ordinary statistical terms this is called regression. There are two fundamental
types of training with neural networks: supervised and unsupervised learning. For supervised
training, as in regression, data used for the training consists of independent variables (also
referred to as feature variables or predictor variables) and dependent variables (target values).
The independent variables (input to the neural network) are used to predict the dependent
variables (output from the network). Unsupervised training does not have dependent (target)
values supplied: the network is supposed to cluster the data automatically into meaningful sets.

The fundamental idea behind training, for all neural networks, is in picking a set of
weights and then applying the inputs to the network and gauging the networks performance
with this set of weights. If the network does not perform well, then the weights are modified
by an algorithm specific to each architecture and the procedure is then repeated. This iterative
process continues until some pre-specified criterion has been achieved. A training pass
through all vectors of the input data is called an epoch. Iterative changes can be made to the
weights with each input vector, or changes can be made after all input vectors have been
processed. Typically, weights are iteratively modified by epochs.





21

Learning Processof ANN
An ANN learns from its experience. The usual process of learning involves three tasks:
1.Compute output(s).
2.Compare outputs with desired patterns and feed-back the error.
3.Adjust the weights and repeat the process
4.The learning process starts by setting the weights by some rules . The difference
between the actual output (y) and the desired output(z) is called error (delta).
5.The objective is to minimize delta (error)to zero. The reduction in error is done by
changing the weights
1.Supervised learning-: or Associative learning in which the network is trained by providing it
with input and matching output patterns. These input-output pairs can be provided by an
external teacher, or by the system which contains the neural network (self-supervised).

2.Unsupervised learning -:or Self-organisation in which an (output) unit is trained to respond
to clusters of pattern within the input. In this paradigm the system is supposed to discover
statistically salient features of the input population. Unlike the supervised learning paradigm,
there is no a priori set of categories into which the patterns are to be classified; rather the system
must develop its own representation of the input stimuli.
3.Reinforcement Learning -:This type of learning may be considered as an intermediate form
of the above two types of learning. Here the learning machine does some action on the
environment and gets a feedback response from the environment. The learning system grades its
action good (rewarding) or bad (punishable) based on the environmental response and
accordingly adjusts its parameters. Generally, parameter adjustment is continued until an
equilibrium state occurs, following which there will be no more changes in its parameters. The
selforganizing neural learning may be categorized under this type of learning.
22

LEARNING TECHNIQUES

Learning rules are algorithm for slowly alerting the connections weighs to achieve a desirable
goal such a minimization of an error function. The generalized step for any neural network
leaning algorithm is follows are the commonly used learning algorithm for neural networks.

Multi layer neural net (MLNN)
Error back propagation (EBB)
Radial basis functions (RBF
Reinforcement learning
Temporal deference learning
Adaptive resonance theory (ART)
Genetic algorithm

Selection of a particular learning algorithm depends on the network and network
topology. As MLNN with EBP is most extensively used and widely accepted network for
process application, namely for identification and control of the process.

MLNN IN SYSTEM IDENTIFICATION:
There has been an explosion of neural network application in the areas
of process control engineering in the last few years. Since it become very difficult to obtain the
model of complex non-linear system due its unknown dynamics and a noise environment. it
necessitates the requirement for a non classic technique which has the ability to model the
physical process accurately. since nonlinear governing relationships can be handled very
contendly by neutral network, these networks offer a cost effective solution to modeling of time
varying chemical process.
23

Using ANN carries out the modeling of the process by using ANN by any one of the following
two ways:
Forward modeling
Direct inverse modeling

FORWARD MODELING
The basic configuration used for non-linear system modeling and identification using
neural network. The number of input nodes specifies the dimensions of the network input. In
system identification context, the assignment of network input and output to network input
vector.

DIRECT INVERSE MODELING:
This approach employs a generalized model suggested by Psaltis et al.to learn the inverse
dynamic model of the plant as a feed forward controller. Here, during the training stage, the
control input are chosen randomly within there working range. And the corresponding plant
output values are stored, as a training of the controller cannot guarantee the inclusion of all
possible situations that may occur in future. Thus, the developed model has take of robustness
The design of the identification experiment used to guarantee data for training the neural
network models is crucial, particularly, in-linear problem. The training data must contain process
input-output information over the entire operating range. In such experiment, the types of
manipulated variables used are very important.
The traditional pseudo binary sequence (PRBS) is inadequate because the training data
set contains most of its steady state information at only two levels, allowing only to fit linear
model in over to overcome the problem with binary signal and to provide data points throughout
the range of manipulated variables. The PRBS must be a multilevel sequence. This kind of
modeling of the process play a vital role in ANN based direct inverse control configuration
24

ANN BASED CONTROL CONFIGURATION:

Direct inverse control
Direct adaptive control
Indirect adaptive control
Internal model control
Model reference adaptive control

DIRECT INVERSE CONTROL
This control configuration used the inverse planet model. Fro the direct inverse control.
The network is required to be trained offline to learn the inverse dynamics of the plant. The
networks are usually trained using the output errors of the networks and not that of the plant. The
output error of the networks is defined.

En=1/2(ud-on)
2
Where En is the networks output error ud is the actual controls signal required to get desired
process output and on is the networks output. When the networks is to be trained as a controller.
The output errors of the networks are unknown. Once the network is trained using direct inverse
modeling learns the inverse system model. It is directly placed in series with the plant to be
controlled and the configuration shown in figure. Since inverse model of the plant is in off line
trained model, it tacks robustness.

25


Fig no. 8: Block Model of DIC

DIRECT ADAPTIVE CONTROL (DAC)
In the direct adaptive control. The network is trained on line. And the weights of
connections are updated during each sampling interval. In this case the cost function is the plant
output error rather than the networks output error. The configuration of DAC is shown in figure.




Fig no.9 : Block Model of DAC

The limitation of using this configuration is that one must have the some knowledge about the
plant dynamics i.e. Jacobin matrix of the plant. To solve the problems; initially, Psaltis D.et al
proposed a technique for determining the partial derivatives of the plant at its operating point
Xianzhang et al and Yao Zhang et al presented a simple approach, in which by using the sign of
the plant Jacobin. The modifications of the weights are carried out.
26


INDIRECT ADAPTIVE CONTROL
Narendra K S et al. Presented an indirect adaptive control strategy.
In this approach, two neutral networks for controller purpose and another for plant modeling and
is called plant emulator decides the performance of the controller. The configuration of indirect
adaptive control scheme becomes as shown FIG.3.



Fig no. 10 : Block Model of IAC

In direct learning the neutral controller, it is well known that the partial derivatives of the
controlled plant output with respect to the plant input (plant Jacobin) is required. One method to
overcome this problem is the use NN to identify the plant, and to calculate its Jacobin. Since the
plant emilator learning converges before the neutral controllers learning begins, an effective
neutral control system is achieved.



27

INTERNAL MODEL CONTROL (IMC)
The IMC uses two neutral networks for implementation. I n this configrurations,one
neutral networks is placed in parallel with the plant and other neutral network in series the plant.
The structure of nonlinear IMC is shown in FIG.4.
The IMC provides a direct method for the design of nonlinear feedback controllers. If a
good model of the plant is savable, the close loop system gives an exact set point tracking despite
immeasurable disturbance acting on the plant.


Fig no. 11 :Block Model of IMC

For the development of NN based IMC, the following two steps are required:

Plant identification
Plant inverse model
Plant identification is carried out using the forward modeling techniques. Once the network is
trained, it represents the perfect dynamics of the plant the error signal used to adjust the networks
weights is the difference between the plant output and the model output.
28

The neutral networks used to represent the inverse of the plant (NCC) are trained using
the plant itself. The error signal used to train the plant inverse model is the difference between
the desired plant and model outputs.

DIRECT NEURAL NETWORK MODEL REFERENCE ADAPTIVE
CONTROL:
The neural network approximates a wide variety of nonlinear control laws by adjusting
the weights in training to achieve the desired approximate accuracy. One possible MRAC
structure based on neural network is shown
In this configuration, control systems attempted to make the plant output Y
P
(t) to follow
the reference model output asymptotically. The error signal
Used to train the neural network controller is the difference between the model and the plant
outputs, principally; this network works like the direct adaptive neural control system.


Fig no. 12 : Block Model of MAC

29

RECENT ADVANCES IN ANN FIELD

Integration of fuzzy logic into neural networks
1. Fuzzy logic is a type of logic that recognizes more than simple true and false values, hence
better simulating the real world. For example, the statement today is sunny might be 100%
true if there are no clouds, 80% true if there are a few clouds, 50% true if it's hazy, and 0%
true if rains all day. Hence, it takes into account concepts like -usually, somewhat, and
sometimes.
2. Fuzzy logic and neural networks have been integrated for uses as diverse as automotive
engineering, applicant screening for jobs, the control of a crane, and the monitoring of
glaucoma.

Pulsed neural networks :
1. Most practical applications of artificial neural networks are based on a computational model
involving the propagation of continuous variables from one processing unit to the next. In
recent years, data from neurobiological experiments have made it increasingly clear that
biological neural networks, which communicate through pulses, use the timing of the pulses
to transmit information and perform computation. This realization has stimulated significant
research on pulsed neural networks, including theoretical analyses and model development,
neurobiological modeling, and hardware implementation.
30


Hardware specialized for neural networks
1. Some networks have been hardcoded into chips or analog devices ? this technology will
become more useful as the networks we use become more complex.
2. The primary benefit of directly encoding neural networks onto chips or specialized analog
devices is SPEED!
3. NN hardware currently runs in a few niche areas, such as those areas where very high
performance is required (e.g. high energy physics) and in embedded applications of simple,
hardwired networks (e.g. voice recognition).
4. Many NNs today use less than 100 neurons and only need occasional training. In these
situations, software simulation is usually found sufficient
When NN algorithms develop to the point where useful things can be done with 1000's of
neurons and 10000's of synapses, high performance NN hardware will become essential for
practical operation.






31

Advantages & Disadvantages of ANN

Advantages
Robustness: fault tolerance due to network redundancy
.Autonomous learning and generalization
A neural network can perform tasks that a linear program can not.
When an element of the neural network fails, it can continue without
any problem by their parallel nature.
A neural network learns and does not need to be reprogrammed.
It can be implemented in any application.
It can be implemented without any problem

Disadvantages
Not exact
Large complexity of the network structure
The neural network needs training to operate.
The architecture of a neural network is different from the architecture of
microprocessors therefore needs to be emulated.
Requires high processing time for large neural networks

32

APPLICATIONS :
Aerospace
High performance aircraft autopilots, flight path simulations, aircraft control systems,
autopilot enhancements, aircraft component simulations, aircraft component fault detectors
Automotive
Automobile automatic guidance systems, warranty activity analyzers
Banking
Check and other document readers, credit application evaluators
Defense
Weapon steering, target tracking, object discrimination, facial recognition, new kinds of
sensors, sonar, radar and image signal processing including data compression, feature
extraction and noise suppression, signal/image identification
Electronics
Code sequence prediction, integrated circuit chip layout, process control, chip failure analysis,
machine vision, voice synthesis, nonlinear modeling



Financial
Real estate appraisal, loan advisor, mortgage screening, corporate bond rating, credit line use
analysis, portfolio trading program, corporate financial analysis, currency price prediction

33


Manufacturing
Manufacturing process control, product design and analysis, process and machine diagnosis,
real-time particle identification, visual quality inspection systems, beer testing, welding quality
analysis, paper quality prediction, computer chip quality analysis, analysis of grinding
operations, chemical product design analysis, machine maintenance analysis, project bidding,
planning and management, dynamic modeling of chemical process systems
Medical
Breast cancer cell analysis, EEG and ECG analysis, prosthesis design, optimization of
transplant times, hospital expense reduction, hospital quality improvement, emergency room
test advisement
Robotics
Trajectory control, forklift robot, manipulator controllers, vision systems

Speech
Speech recognition, speech compression, vowel classification, text to
speech synthesis
Securities
Market analysis, automatic bond rating, stock trading advisory systems
Telecommunications
Image and data compression, automated information services, real-time translation of spoken
language, customer payment processing systems
Transportation
Truck brake diagnosis systems, vehicle scheduling, routing systems.
34

CONCLUSION

All current NN technologies will most likely be vastly improved upon in the future.
Everything from handwriting and speech recognition to stock market prediction will become
more sophisticated as researchers develop better training methods and network architectures
Although neural networks do seem to be able to solve many problems, we must put our
exuberance in check sometimes ? they are not magic! Overconfidence in neural networks can
result in costly mistakes: see for a rather funny story about the government and neural
networks.
.
NNs might, in the future, allow:
a. Robots that can see, feel, and predict the world around them
b. Improved stock prediction
c. Common usage of self-driving cars
d. Composition of music
35


REFERENCE
1) Bradshaw, J.A. , Carden , K.J., Riordan, D., 1991. Ecological Applications Using a
Novel Expert System Shell. Comp. Appl. Biosci. 7, 7983.
2) Lippmann, R.P., 1987. An introduction to computing with neural nets. IEEE Accost.
Speech Signal Process. Mag., April: 4-22.
3) N. Murata, S. Yoshizawa, and S. Amari, Learning curves, model selection and
complexity of neural networks, in Advances in Neural Information Processing Systems
5, S. Jose Hanson, J. D. Cowan, and C. Lee Giles, ed. San Mateo, CA: Morgan
Kaufmann, 1993, pp. 607-614.

4) Abraham, A. (2004) Meta-Learning Evolutionary Artificial Neural Networks,
Neurocomputing Journal, Vol. 56c, Elsevier Science, Netherlands.

5) Carpenter, G. and Grossberg, S. (1998) in Adaptive Resonance Theory (ART), The
Handbook of Brain Theory and Neural Networks, (ed. M.A. Arbib), MIT Press,
Cambridge, MA.

6) Chen, S., Cowan, C.F.N. and Grant, P.M. (1991) Orthogonal Least Squares Learning
Algorithm for Radial Basis Function Networks. IEEE Transactions on Neural
Networks,302309.

7) Fausett, L. (1994) Fundamentals of Neural Networks, Prentice Hall, USA.

8) Grossberg, S. (1976) Adaptive Pattern Classification and Universal Recoding: Parallel
Development and Coding of Neural Feature Detectors. Biological Cybernetics, .

9) Hebb, D.O. (1949) The Organization of Behavior, John Wiley,New York.

10) Kohonen, T. (1988) Self-Organization and Associative Memory, Springer-Verlag, New
York.

11) Macready, W.G. and Wolpert, D.H. (1997) The No Free Lunch Theorems. IEEE
Transactions on Evolutionary Computing.

12) Mandic, D. and Chambers, J. (2001) Recurrent Neural Networks for Prediction: Learning
Algorithms, Architectures and Stability, John Wiley & Sons, New York.

13) McCulloch, W.S . and Pitts, W.H. (1943) A Logical Calculus of the Ideas Immanent in
Nervous Activity. Bulletin of Mathematical Biophysics.

36