Anda di halaman 1dari 10

Hopfield Network Example

We have a 5 node Hopfield network and we want it to recognize the pattern (0 1 1 0 1). Since there
are 5 nodes, we need a matrix of 5 x 5 weights, where the weights from a node back to itsef
are 0. !he weight matrix wi ook ike this"
0
W
12
W
13
W
14
W
15
W
21
0
W
23
W
24
W
25
W
31
W
32
0
W
34
W
35
W
41
W
42
W
43
0
W
45
W
51
W
52
W
53
W
54
0
Where W
ij
= W
ji

Since the weights are s#mmetric, we on# have to cac\$ate the \$pper diagona of weights, and then we
can cop# each weight to its inverse weight. %n this case, & is the vector (0 1 1 0 1), so &
1
' 0, &
(
' 1,
&
)
' 1, &
*
' 0, and &
5
' 1. !h\$s the comp\$tation of the weights is as foows"
W
12
= (2V
1
1!(2V
2
1! = (0 1!(2 1! = (1!(1! = 1
W
13
= (2V
1
1!(2V
3
1! = (0 1!(2 1! = (1!(1! = 1
W
14
= (2V
1
1!(2V
4
1! = (0 1!(0 1! = (1!(1! = 1
W
15
= (2V
1
1!(2V
5
1! = (0 1!(2 1! = (1!(1! = 1
W
23
= (2V
2
1!(2V
3
1! = (2 1!(2 1! = (1!(1! = 1
W
24
= (2V
2
1!(2V
4
1! = (2 1!(0 1! = (1!(1! = 1
W
25
= (2V
2
1!(2V
5
1! = (2 1!(2 1! = (1!(1! = 1
W
34
= (2V
3
1!(2V
4
1! = (2 1!(0 1! = (1!(1! = 1
W
35
= (2V
3
1!(2V
5
1! = (2 1!(2 1! = (1!(1! = 1
W
45
= (2V
4
1!(2V
5
1! = (0 1!(2 1! = (1!(1! = 1
1
So now o\$r weight matrix ooks ike this"
0 1 1 1 1
W
21
0 1 1 1
W
31
W
32
0 1 1
W
41
W
42
W
43
0 1
W
51
W
52
W
53
W
54
0
+# refecting abo\$t the diagona, we get the f\$ weight matrix"
0 1 1 1 1
1 0 1 1 1
1 1 0 1 1
1 1 1 0 1
1 1 1 1 0
,or competeness- sake, #o\$- remember that the origina form\$a was set \$p to aow #o\$ to have n
patterns. So et-s consider the case where we want o\$r 5 node .opfied net to store both the
pattern &
1
' (0 1 1 0 1) and another pattern &
(
' (1 0 1 0 1). /ne wa# #o\$ co\$d go abo\$t
cac\$ating the weights is on a weight b# weight basis. ,or exampe, W
1(
co\$d be cac\$ated
as"
W
12
=

(2V
"
1
1!(2V
"
2
1!
= (2V
1
1
1!(2V
1
2
1! # (2V
2
1
1!(2V
2
2
1!
= (2\$0 1!(2\$1 1! # (2\$1 1!(2\$0 1!
(
= (0 1!(2 1! # (2 1!(0 1!
= (1!(1! # (1!(1!
= 1 # 1
= 2
0o\$ co\$d go thro\$gh each weight ike this and cac\$ate the new weight matrix. !his is ike# the wa#
#o\$ wo\$d do it b# comp\$ter, b\$t % find if %-m cac\$ating them b# hand, it-s easier to foc\$s on
one pattern at a time and then s\$m the res\$ting matrices. !o do this, #o\$ wo\$d cac\$ate the
matrix for the first pattern (which we did above), then cac\$ate the va\$e for the second matrix
and fina# add the two matrices together. .ere-s the weight matrix for the pattern (1 0 1 0 1)"
0 1 1 1 1
1 0 1 1 1
1 1 0 1 1
1 1 1 0 1
1 1 1 1 0
1ow add this to the previo\$s weight matrix and we get"
0 2 0 0 0
2 0 0 0 0
0 0 0 2 2
0 0 2 0 2
0 0 2 2 0
)
How to %pdate a node in a Hopfield network
So now we have a weight matrix for a 5 node .opfied network that-s meant to recognize the patterns
(0 1 1 0 1) and (1 0 1 0 1). /ne thing #o\$ might notice is that these patterns on# differ b# (
bits. 2s #o\$ might imagine, patterns this cose together might be diffic\$t to te apart. +#
anaog#, #o\$ might have tro\$be discriminating a ower case 3c3 from 3e3 or an \$pper case 3/3
from 343 if the# were manged bad# eno\$gh. 5et-s start from the pattern (1 1 1 1 1), which
on# differs from each of these patterns b# ( bits, and see what happens.
6pdating a node in a .opfied network is ver# m\$ch ike \$pdating a perceptron. ,irst #o\$ do a
weighted s\$m of the inp\$ts from the other nodes, then if that va\$e is greater than or e7\$a to 0,
#o\$ o\$tp\$t 1. /therwise, #o\$ o\$tp\$t 0. %n form\$a form"
V
i
in =

W
ji
V
j

V
i
& 1 if V
i
in &= 0
el"e V
i
& 0
So ass\$ming we have the fina weight matrix from the previo\$s section, and we start from the state (1
1 1 1 1), for the )rd node, we have"
V
3
in =

W
j3
V
j

=
W
13
V
1
# W
23
V
2
# W
43
V
4
# W
53
V
5

= 0\$1 # 0\$1 # 2\$1 # 2\$1
= 0
"in'e 0 &= 0(
V
3
= 1
*
%n this case, the va\$e of &
)
doesn-t change. %t-s worth noticing that since the weight from node ) to
itsef is 0, we co\$d have 8\$st cac\$ated the dot prod\$ct of the )rd co\$mn o\$t of the weight
matrix and the c\$rrent state to cac\$ate the weighted s\$m"
V
3
in = (0 0 0 2 2! \$ (1 1 1 1 1! = 2 # 2 = 0
)e*%en'in+ of node %pdate" in a Hopfield network
0o\$ might have noticed b# now that se7\$encing the \$pdates of the nodes in a .opfied network is
somewhat trick#. .ow can #o\$ \$pdate a ne\$ron if the va\$es of its inp\$ts are changing9 We,
there are two approaches. !he first is s#nchono\$s \$pdating, which means a the nodes get
\$pdated at the same time, based on the existing state (i.e. not on the va\$es the nodes are
changing to). !o \$pdate the nodes in this method, #o\$ can 8\$st m\$tip# the weight matrix b#
the vector of the c\$rrent state.
!his isn-t ver# reaistic in a ne\$ra sense, as ne\$rons don-t a \$pdate at the same rate. !he# have
var#ing propagation dea#s, var#ing firing times, etc., so a more reaistic ass\$mption wo\$d be
to \$pdate them in random order. !his was the method described b# .opfied, in fact. 0o\$
random# seect a ne\$ron, and \$pdate it. !hen #o\$ random# seect another ne\$ron and \$pdate
it. 0o\$ keep doing this \$nti the s#stem is in a stabe state (which we- tak abo\$t ater).
%n practice, peope code .opfied nets in a semi:random order. !he# \$pdate a of the nodes in one
step, b\$t within that step the# are \$pdated in random order. So it might go ), (, 1, 5, *, (, ), 1,
5, *, etc. !his is 8\$st to avoid a bad pse\$do:random generator from favoring one of the nodes,
which co\$d happen if it was p\$re# random" ), (, 1, (, (, (, 5, 1, (, (, *, (, 1, etc.
5
How to tell when ,o% 'an "top %pdatin+ the network
;
!he main reason to want to c#ce thro\$gh a the nodes each step is that it-s the on# wa# #o\$ can te
when to stop. +asica#, if #o\$ go thro\$gh a the nodes and none of them changes, #o\$ can
stop. %f #o\$-re \$pdating them in a fixed se7\$ence (e.g. 1, (, ), *, 5, 1, (, ), *, 5, 1, (, ), etc.),
this means that if #o\$ go thro\$gh 5 consec\$tive ne\$rons witho\$t changing an# of their va\$es,
then #o\$-re at an attractor so #o\$ can stop.
<
-ini"hin+ %p the example
=
1ow et-s finish the exampe % started. %n other words, given the weight matrix for a 5 node network
with (0 1 1 0 1) and (1 0 1 0 1) as attractors, start at the state (1 1 1 1 1) and see where it goes.
!o keep it simpe, %-m going to \$pdate the nodes in the fixed order ), 1, 5, (, *, ), 1, 5, (, *, etc.
% aread# did the first \$pdate of node ), and it didn-t change, so contin\$ing"
%pdate node 3 did it( no 'han+e
%pdate node 1
V
1
in = (0 2 0 0 0! . (1 1 1 1 1! = 2
"in'e 2 / 0( V
1
= 0 (it changed)
%pdate node 5
V
5
in = (0 0 2 2 0! . (0 1 1 1 1! = 0
"in'e 0 &= 0( V
5
= 1 (it didn't change)
%pdate node 2
V
2
in = (2 0 0 0 0! . (0 1 1 1 1! = 0
"in'e 0 &= 0( V
2
= 1 (it didn't change)
%pdate node 4
V
4
in = (0 0 2 0 2! . (0 1 1 1 1! = 4
"in'e 4 / 0( V
4
= 0 (it changed)
%pdate node 3
V
3
in = (0 0 0 2 2! . (0 1 1 0 1! = 2
"in'e 2 &= 0( V
3
= 1 (it didn't change)
%pdate node 1
V
1
in = (0 2 0 0 0! . (0 1 1 0 1! = 2
"in'e 2 / 0( V
1
= 0 (it didn't change)
%pdate node 5
V
5
in = (0 0 2 2 0! . (0 1 1 0 1! = 2
"in'e 2 &= 0( V
5
= 1 (it didn't change)
%pdate node 2
V
2
in = (2 0 0 0 0! . (0 1 1 0 1! = 0
"in'e 0 &= 0( V
2
= 1 (it didn't change)
>
10