Gradient-based learning applied to document recognition

Yann LeCun, Léon Bottou, Yoshua Bengio, Patrick Haffner

Research output: Contribution to journalArticlepeer-review


Multilayer neural networks trained with the back-propagation algorithm constitute the best example of a successful gradient-based learning technique. Given an appropriate network architecture, gradient-based learning algorithms can be used to synthesize a complex decision surface that can classify high-dimensional patterns, such as handwritten characters, with minimal preprocessing. This paper reviews various methods applied to handwritten character recognition and compares them on a standard handwritten digit recognition task. Convolutional neural networks, which are specifically designed to deal with the variability of two dimensional (2-D) shapes, are shown to outperform all other techniques. Real-life document recognition systems are composed of multiple modules including field extraction, segmentation, recognition, and language modeling. A new learning paradigm, called graph transformer networks (GTN's), allows such multimodule systems to be trained globally using gradient-based methods so as to minimize an overall performance measure. Two systems for online handwriting recognition are described. Experiments demonstrate the advantage of global training, and the flexibility of graph transformer networks. A graph transformer network for reading a bank check is also described. It uses convolutional neural network character recognizers combined with global training techniques to provide record accuracy on business and personal checks. It is deployed commercially and reads several million checks per day.

Original languageEnglish (US)
Pages (from-to)2278-2323
Number of pages46
JournalProceedings of the IEEE
Issue number11
StatePublished - 1998


  • Convolutional neural networks
  • Document recognition
  • Finite state transducers
  • Gradient-based learning
  • Graph transformer networks
  • Machine learning
  • Neural networks
  • Optical character recognition (OCR)

ASJC Scopus subject areas

  • General Computer Science
  • Electrical and Electronic Engineering


Dive into the research topics of 'Gradient-based learning applied to document recognition'. Together they form a unique fingerprint.

Cite this