Neural Network Acceptability Judgments

Alex Warstadt, Amanpreet Singh, Samuel R. Bowman

    Research output: Contribution to journalArticlepeer-review


    This paper investigates the ability of artificial neural networks to judge the grammatical acceptability of a sentence, with the goal of testing their linguistic competence. We introduce the Corpus of Linguistic Acceptability (CoLA), a set of 10,657 English sentences labeled as grammatical or ungrammatical from published linguistics literature. As baselines, we train several recurrent neural network models on acceptability classification, and find that our models outperform unsupervised models by Lau et al. (2016) on CoLA. Error-analysis on specific grammatical phenomena reveals that both Lau et al.’s models and ours learn systematic generalizations like subject-verbobject order. However, all models we test perform far below human level on a wide range of grammatical constructions.

    Original languageEnglish (US)
    Pages (from-to)625-641
    Number of pages17
    JournalTransactions of the Association for Computational Linguistics
    StatePublished - 2019

    ASJC Scopus subject areas

    • Communication
    • Human-Computer Interaction
    • Linguistics and Language
    • Computer Science Applications
    • Artificial Intelligence


    Dive into the research topics of 'Neural Network Acceptability Judgments'. Together they form a unique fingerprint.

    Cite this