Project:

The novel dynamic supervised forward propagation neural network for handwritten character recognition


Project Leader(s): Dr S. Dlay
Contact: s.s.dlay@ncl.ac.uk
Sponsors: EPSRC, IEE-Vodafone

A neural network technique for identifying handwritten characters in real-time has been developed. The neural network is based on the Counterpropagation Network and trains using a supervised learning algorithm and incremental training. In addition it allows unsupervised dynamic growth of the middle layer permitting unknown subclasses to be learnt. The network classifies input patterns using a competitive winner takes all hidden middle layer, in which only the neuron with the greatest activation gives a non-zero output. The activation of each neuron is given by the scalar product of the input data vector - passed on by the input layer - and the input-weight-vector of the neuron. The output is a binary ASCII code equal to the output-weight-vector of the winning neuron.

Introduction

Recognition of handwritten characters in real-time is an important prerequiste for many human-computer interfaces systems and has obvious applications in commercial devices (e.g. personal digital assistants, digital diaries etc). Neural network techniques are an attractive option for this kind of applications : offering fast recognition and the ability to train the system to an individual's personal handwriting style. In this project, a novel dynamic supervised foreward propagation neural network is proposed and tested against common character recognition issues. A schematic figure of the proposed network and the associated character recognition algorithm is shown below:

[ character recognition diagram ]

Results

The results tables below show the new network, DSFPN compared to the well-known CPN and BPN networks.. Table 1 shows that the DSFPN trains using 405 times fewer presentations than the BPN and gives a far better accuracy than the CPN. Table 2 shows the results of tests using a DSFPN to classify data presented as level 3, 4 and 5 wavelet descriptors. The error rate for level 5 data is 0.208% compared to 2.22% for the Fourier data.

Network

Middle layer size

Presentations

Training time (S)

Test (%)

DSFPN

20.09

16264

5.35

97.78

CPN

50

4100

2.03

91.33

BPN

10

6.58E6

4838

94.25


Table 1. Mean test results using Fourier descriptors

 

Res.level

Middle layer size

Presentations

Training time (S)

Test (%)

3

22.42

21270

52.6

99.6

4

22.32

23820

28.8

99.6

5

22.84

50874

33.2

99.792


Table 2. Mean test results using wavelet descriptors

 

The diagram below shows a schematic design for a competitive middle layer - constructed from competitive layer neurons

[ schematic ]

Conclusions

  • A new neural network, named the DSFPN, has been presented which performs the numeric character recognition task as well other popular learning networks
  • the method is simpler and trains using far less pattern presentations.The DSFPN not only produces the highest recognition performance but trains using 405 times times fewer presentations than the Back Propagation network.
  • This is a considerable improvement in training time over the BPN.
  • Fourier descriptors for three different networks show that, although the CPN trains faster, its accuracy is far lower than the other networks.
  • Data presented at resolution levels 3, 4 and 5 show that the wavelet data requires slightly more training presentations than the Fourier descriptors The level 5 descriptors give a recognition error rate of 0.208% for numerals and 4.09% for lowercase letters. This compares to respective error rates of 2.22% and 17.68% for Fourier descriptors, which is a significant improvement. Thus these results show that the wavelet descriptor allow considerably more accurate recognition

Staff

Professor Satnam Dlay
Professor of Signal Processing Analysis