On-line learning for very large data sets

Léon Bottou, Yann Le Cun

Research output: Contribution to journalArticlepeer-review


The design of very large learning systems presents many unsolved challenges. Consider, for instance, a system that 'watches' television for a few weeks and learns to enumerate the objects present in these images. Most current learning algorithms do not scale well enough to handle such massive quantities of data. Experience suggests that the stochastic learning algorithms are best suited to such tasks. This is at first surprising because stochastic learning algorithms optimize the training error rather slowly. Our paper reconsiders the convergence speed in terms of how fast a learning algorithm optimizes the testing error. This reformulation shows the superiority of the well designed stochastic learning algorithm.

Original languageEnglish (US)
Pages (from-to)137-151
Number of pages15
JournalApplied Stochastic Models in Business and Industry
Issue number2
StatePublished - Mar 2005


  • Convergence speed
  • Learning
  • Online learning
  • Stochastic optimization

ASJC Scopus subject areas

  • Modeling and Simulation
  • General Business, Management and Accounting
  • Management Science and Operations Research


Dive into the research topics of 'On-line learning for very large data sets'. Together they form a unique fingerprint.

Cite this