Conjugate conformal prediction for online binary classification

Mustafa A. Kocak, Elza Erkip, Dennis E. Shasha

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Binary classification (rain or shine, disease or not, increase or decrease) is a fundamental problem in machine learning. We present an algorithm that can take any standard online binary classification algorithm and provably improve its performance under very weak assumptions, given the right to refuse to make predictions in certain cases. The extent of improvement will depend on the data size, stability of the algorithm, and room for improvement in the algorithms performance. Our experiments on standard machine learning data sets and standard algorithms (k-nearest neighbors and random forests) show the effectiveness of our approach, even beyond what is possible using previous work on conformal predictors upon which our approach is based. Though we focus on binary classification, our theory could be extended to multiway classification. Our code and data are available upon request.

Original languageEnglish (US)
Title of host publication32nd Conference on Uncertainty in Artificial Intelligence 2016, UAI 2016
EditorsDominik Janzing, Alexander Ihler
PublisherAssociation For Uncertainty in Artificial Intelligence (AUAI)
Pages347-356
Number of pages10
ISBN (Electronic)9781510827806
StatePublished - 2016
Event32nd Conference on Uncertainty in Artificial Intelligence 2016, UAI 2016 - Jersey City, United States
Duration: Jun 25 2016Jun 29 2016

Publication series

Name32nd Conference on Uncertainty in Artificial Intelligence 2016, UAI 2016

Other

Other32nd Conference on Uncertainty in Artificial Intelligence 2016, UAI 2016
CountryUnited States
CityJersey City
Period6/25/166/29/16

ASJC Scopus subject areas

  • Artificial Intelligence

Fingerprint Dive into the research topics of 'Conjugate conformal prediction for online binary classification'. Together they form a unique fingerprint.

Cite this