Transformation invariance in pattern recognition - Tangent distance and tangent propagation

Patrice Y. Simard; Yann A. Lecun; John S. Denker; Bernard Victorri

doi:10.1007/978-3-642-35289-8_17

Transformation invariance in pattern recognition - Tangent distance and tangent propagation

Patrice Y. Simard, Yann A. Lecun, John S. Denker, Bernard Victorri

Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Chapter

Abstract

In pattern recognition, statistical modeling, or regression, the amount of data is a critical factor affecting the performance. If the amount of data and computational resources are unlimited, even trivial algorithms will converge to the optimal solution. However, in the practical case, given limited data and other resources, satisfactory performance requires sophisticated methods to regularize the problem by introducing a priori knowledge. Invariance of the output with respect to certain transformations of the input is a typical example of such a priori knowledge. In this chapter, we introduce the concept of tangent vectors, which compactly represent the essence of these transformation invariances, and two classes of algorithms, "tangent distance" and "tangent propagation", which make use of these invariances to improve performance.

Original language	English (US)
Title of host publication	Neural Networks
Subtitle of host publication	Tricks of the Trade
Publisher	Springer Verlag
Pages	235-269
Number of pages	35
ISBN (Print)	9783642352881
DOIs	https://doi.org/10.1007/978-3-642-35289-8_17
State	Published - 2012

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	7700 LECTURE NO
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

ASJC Scopus subject areas

Theoretical Computer Science
General Computer Science

Access to Document

10.1007/978-3-642-35289-8_17

Cite this

Simard, P. Y., Lecun, Y. A., Denker, J. S., & Victorri, B. (2012). Transformation invariance in pattern recognition - Tangent distance and tangent propagation. In Neural Networks: Tricks of the Trade (pp. 235-269). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 7700 LECTURE NO). Springer Verlag. https://doi.org/10.1007/978-3-642-35289-8_17

Transformation invariance in pattern recognition - Tangent distance and tangent propagation. / Simard, Patrice Y.; Lecun, Yann A.; Denker, John S. et al.
Neural Networks: Tricks of the Trade. Springer Verlag, 2012. p. 235-269 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 7700 LECTURE NO).

Research output: Chapter in Book/Report/Conference proceeding › Chapter

Simard, PY, Lecun, YA, Denker, JS & Victorri, B 2012, Transformation invariance in pattern recognition - Tangent distance and tangent propagation. in Neural Networks: Tricks of the Trade. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 7700 LECTURE NO, Springer Verlag, pp. 235-269. https://doi.org/10.1007/978-3-642-35289-8_17

Simard PY, Lecun YA, Denker JS, Victorri B. Transformation invariance in pattern recognition - Tangent distance and tangent propagation. In Neural Networks: Tricks of the Trade. Springer Verlag. 2012. p. 235-269. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-642-35289-8_17

@inbook{47d9539ba845406d95577251d0d0de84,

title = "Transformation invariance in pattern recognition - Tangent distance and tangent propagation",

abstract = "In pattern recognition, statistical modeling, or regression, the amount of data is a critical factor affecting the performance. If the amount of data and computational resources are unlimited, even trivial algorithms will converge to the optimal solution. However, in the practical case, given limited data and other resources, satisfactory performance requires sophisticated methods to regularize the problem by introducing a priori knowledge. Invariance of the output with respect to certain transformations of the input is a typical example of such a priori knowledge. In this chapter, we introduce the concept of tangent vectors, which compactly represent the essence of these transformation invariances, and two classes of algorithms, {"}tangent distance{"} and {"}tangent propagation{"}, which make use of these invariances to improve performance.",

author = "Simard, {Patrice Y.} and Lecun, {Yann A.} and Denker, {John S.} and Bernard Victorri",

year = "2012",

doi = "10.1007/978-3-642-35289-8_17",

language = "English (US)",

isbn = "9783642352881",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "235--269",

booktitle = "Neural Networks",

}

TY - CHAP

T1 - Transformation invariance in pattern recognition - Tangent distance and tangent propagation

AU - Simard, Patrice Y.

AU - Lecun, Yann A.

AU - Denker, John S.

AU - Victorri, Bernard

PY - 2012

Y1 - 2012

N2 - In pattern recognition, statistical modeling, or regression, the amount of data is a critical factor affecting the performance. If the amount of data and computational resources are unlimited, even trivial algorithms will converge to the optimal solution. However, in the practical case, given limited data and other resources, satisfactory performance requires sophisticated methods to regularize the problem by introducing a priori knowledge. Invariance of the output with respect to certain transformations of the input is a typical example of such a priori knowledge. In this chapter, we introduce the concept of tangent vectors, which compactly represent the essence of these transformation invariances, and two classes of algorithms, "tangent distance" and "tangent propagation", which make use of these invariances to improve performance.

AB - In pattern recognition, statistical modeling, or regression, the amount of data is a critical factor affecting the performance. If the amount of data and computational resources are unlimited, even trivial algorithms will converge to the optimal solution. However, in the practical case, given limited data and other resources, satisfactory performance requires sophisticated methods to regularize the problem by introducing a priori knowledge. Invariance of the output with respect to certain transformations of the input is a typical example of such a priori knowledge. In this chapter, we introduce the concept of tangent vectors, which compactly represent the essence of these transformation invariances, and two classes of algorithms, "tangent distance" and "tangent propagation", which make use of these invariances to improve performance.

UR - http://www.scopus.com/inward/record.url?scp=84872577241&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84872577241&partnerID=8YFLogxK

U2 - 10.1007/978-3-642-35289-8_17

DO - 10.1007/978-3-642-35289-8_17

M3 - Chapter

AN - SCOPUS:84872577241

SN - 9783642352881

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 235

EP - 269

BT - Neural Networks

PB - Springer Verlag

ER -

Transformation invariance in pattern recognition - Tangent distance and tangent propagation

Abstract

Publication series

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this