Digits recognition

Rodrigo Barbosa de Santis

May 21, 2019

1 Introduction
Rosenblatt’s perceptron was invented in 1950s for optical character recognition. In this work it is implemented
the algorithm for developing this same task, classifying digits from 0 to 9 according to a benchmark dataset.

2 Materials and methods

2.1 Perceptron
The perceptron is a learning machine that has multiple inputs fully conected to an output layer with multiple
processor elements (PE), as shown in Figure 1. The output yj are calculated from the input xj according to the
equation: (Principe et al., 1999)
 
∑
yi = f (neti ) = f  wij xj + bi  (1)
j

where bi is the bias, wij the local weights and f (net) the activation function – sigmoid.
The weight update using delta rule is given by:(Principe et al., 1999)

wij (n + 1) = wij (n) + ηδip xjp (2)

where wjp is the local weights, η the step size, y the net output and d the desired response, and the local error δip
is calculated by
∂J ′
δip = f (netip ) (3)
∂yip
One main advantage of this method, is that the weight update is local, therefore, no matter how many processor
elements how complex the net is, the training method can be executed.

Figure 1: The perceptron with D inputs and M outputs. (Principe et al., 1999)

1
2.2 Digits dataset
The digit dataset is composed by 1797 8x8 images, labeled from 0 to 9. Each figure is transformed into a feature
vector with length 64, which contains the color fill in a gray scale. A number 8-example is shown in Figure 2.
The digit dataset is available as a database example in Scikit-learn (http://scikit-learn.org/).

Figure 2: 1 of the 1797 records in dataset (label= 8)

3 Development
The algorithm is implemented using Python 2.7.8 (Van Rossum, 1998), including the following libraries:
1. NumPy (http://www.numpy.org/) – A large set of funcations that allows arrays manipulation;
2. PyLab (https://pypi.python.org/pypi/pylab/) – A scientific library, which provides a group of graphic and
chart functions;
3. Scikit-learn (http://scikit-learn.org/) – Machine learning library in Python.

4 Results
The accuracy for the classifier was equal to 0.896 + −0.042 after the training and cross validation. The matrix
confusion is shown in Table 1. The matrix diagonal presents the correct classifications. The major number of errors
were classifying the digit ’1’.

178 0 0 0 0 0 0 0 0 0
0 181 0 0 1 0 0 0 0 0
0 1 175 0 0 0 0 0 1 0
0 0 0 179 0 1 0 1 1 1
0 3 0 0 177 0 0 0 1 0
0 0 0 0 0 181 0 0 0 1
0 0 0 0 0 0 181 0 0 0
0 0 0 0 0 0 0 179 0 0
0 8 1 1 0 0 0 0 164 0
0 0 0 0 0 0 0 0 2 178

5 Conclusion
The perceptron algorithm presented a major advance for learning classifiers, due to it simplicity and capacity of
weigh learning locally. The algorithm until today is wide applied and had been used as basis for developing several
others in the area. In this work we applied the perceptron in a digit database, achieving a accuracy of nearly 90%.

2
References
Principe, J. C., Euliano, N. R., & Lefebvre, W. C. (1999). Neural and adaptive systems: fundamentals through
simulations with CD-ROM. John Wiley & Sons, Inc..

Van Rossum, G. (1998). Python: a computer language. Version 2.7.8. Amsterdam, Stichting Mathematisch Cen-
trum. (http://www.python.org).