Universum prescription: Regularization using unlabeled data

Xiang Zhang, Yann LeCun

Research output: Contribution to conferencePaper

Abstract

This paper shows that simply prescribing "none of the above" labels to unlabeled data has a beneficial regularization effect to supervised learning. We call it universum prescription by the fact that the prescribed labels cannot be one of the supervised labels. In spite of its simplicity, universum prescription obtained competitive results in training deep convolutional networks for CIFAR-10, CIFAR-100, STL-10 and ImageNet datasets. A qualitative justification of these approaches using Rademacher complexity is presented. The effect of a regularization parameter - probability of sampling from unlabeled data - is also studied empirically.

Original languageEnglish (US)
Pages2907-2913
Number of pages7
StatePublished - 2017
Event31st AAAI Conference on Artificial Intelligence, AAAI 2017 - San Francisco, United States
Duration: Feb 4 2017Feb 10 2017

Other

Other31st AAAI Conference on Artificial Intelligence, AAAI 2017
CountryUnited States
CitySan Francisco
Period2/4/172/10/17

ASJC Scopus subject areas

  • Artificial Intelligence

Fingerprint Dive into the research topics of 'Universum prescription: Regularization using unlabeled data'. Together they form a unique fingerprint.

  • Cite this

    Zhang, X., & LeCun, Y. (2017). Universum prescription: Regularization using unlabeled data. 2907-2913. Paper presented at 31st AAAI Conference on Artificial Intelligence, AAAI 2017, San Francisco, United States.