Active Learning for Single Neuron Models with Lipschitz Non-Linearities

Aarshvi Gajjar, Chinmay Hegde, Christopher Musco

    Research output: Contribution to journalConference articlepeer-review


    We consider the problem of active learning for single neuron models, also sometimes called “ridge functions”, in the agnostic setting (under adversarial label noise). Such models have been shown to be broadly effective in modeling physical phenomena, and for constructing surrogate data-driven models for partial differential equations. Surprisingly, we show that for a single neuron model with any Lipschitz non-linearity (such as the ReLU, sigmoid, absolute value, low-degree polynomial, among others), strong provable approximation guarantees can be obtained using a well-known active learning strategy for fitting linear functions in the agnostic setting. Namely, we can collect samples via statistical leverage score sampling, which has been shown to be near-optimal in other active learning scenarios. We support our theoretical results with empirical simulations showing that our proposed active learning strategy based on leverage score sampling outperforms (ordinary) uniform sampling when fitting single neuron models.

    Original languageEnglish (US)
    Pages (from-to)4101-4113
    Number of pages13
    JournalProceedings of Machine Learning Research
    StatePublished - 2023
    Event26th International Conference on Artificial Intelligence and Statistics, AISTATS 2023 - Valencia, Spain
    Duration: Apr 25 2023Apr 27 2023

    ASJC Scopus subject areas

    • Artificial Intelligence
    • Software
    • Control and Systems Engineering
    • Statistics and Probability


    Dive into the research topics of 'Active Learning for Single Neuron Models with Lipschitz Non-Linearities'. Together they form a unique fingerprint.

    Cite this