Loss functions for discriminative training of energy-based models

Yann Le Cun, Fu Jie Huang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Probabilistic graphical models associate a probability to each configuration of the relevant variables. Energy-based models (EBM) associate an energy to those configurations, eliminating the need for proper normalization of probability distributions. Making a decision (an inference) with an EBM consists in comparing the energies associated with various configurations of the variable to be predicted, and choosing the one with the smallest energy. Such systems must be trained discriminatively to associate low energies to the desired configurations and higher energies to un-desired configurations. A wide variety of loss function can be used for this purpose. We give sufficient conditions that a loss function should satisfy so that its minimization will cause the system to approach to desired behavior. We give many specific examples of suitable loss functions, and show an application to object recognition in images.

Original languageEnglish (US)
Title of host publicationAISTATS 2005 - Proceedings of the 10th International Workshop on Artificial Intelligence and Statistics
Pages206-213
Number of pages8
StatePublished - 2005
Event10th International Workshop on Artificial Intelligence and Statistics, AISTATS 2005 - Hastings, Christ Church, Barbados
Duration: Jan 6 2005Jan 8 2005

Publication series

NameAISTATS 2005 - Proceedings of the 10th International Workshop on Artificial Intelligence and Statistics

Other

Other10th International Workshop on Artificial Intelligence and Statistics, AISTATS 2005
CountryBarbados
CityHastings, Christ Church
Period1/6/051/8/05

ASJC Scopus subject areas

  • Artificial Intelligence
  • Statistics and Probability

Fingerprint Dive into the research topics of 'Loss functions for discriminative training of energy-based models'. Together they form a unique fingerprint.

  • Cite this

    Cun, Y. L., & Huang, F. J. (2005). Loss functions for discriminative training of energy-based models. In AISTATS 2005 - Proceedings of the 10th International Workshop on Artificial Intelligence and Statistics (pp. 206-213). (AISTATS 2005 - Proceedings of the 10th International Workshop on Artificial Intelligence and Statistics).