Erratum: ‘‘blimp: The benchmark of linguistic minimal pairs for english’’

Alex Warstadt, Alicia Parrish, Haokun Liu, Anhad Mohananey, Wei Peng, Sheng Fu Wang, Samuel R. Bowman

    Research output: Contribution to journalArticlepeer-review


    We correct wrongly reported results on BLiMP. We wrongly reported some results on BLiMP. The error does not change any of our conclusions in the paper. We made the following changes: 1. We corrected 12 values reported in Table 3 of the paper, shown in Table 1 of this statement. 2. In Section 5.1 of the paper, we wrote, ‘‘Only GPT-2 performs well above chance, and it remains 20 points below humans.’’ The corrected version reads ‘‘Only GPT-2 performs well above chance, and it remains 14 points below humans.’’ 3. We updated Figure 2 in the paper to reflect the updated results. These errors resulted from the erroneous inclusion in our calculations of the final results of two paradigms that we had eliminated from the dataset. These paradigms had been eliminated due to low human agreement with the labels. They belonged to the FILLER-GAP and ISLAND EFFECTS phenomena, respectively. Thus, only results for these phenomena and the overall results are affected.

    Original languageEnglish (US)
    Pages (from-to)867-868
    Number of pages2
    JournalTransactions of the Association for Computational Linguistics
    StatePublished - 2020

    ASJC Scopus subject areas

    • Artificial Intelligence
    • Communication
    • Human-Computer Interaction
    • Computer Science Applications
    • Linguistics and Language


    Dive into the research topics of 'Erratum: ‘‘blimp: The benchmark of linguistic minimal pairs for english’’'. Together they form a unique fingerprint.

    Cite this