TY - JOUR
T1 - The defeat of the Winograd Schema Challenge
AU - Kocijan, Vid
AU - Davis, Ernest
AU - Lukasiewicz, Thomas
AU - Marcus, Gary
AU - Morgenstern, Leora
N1 - Publisher Copyright:
© 2023 Elsevier B.V.
PY - 2023/12
Y1 - 2023/12
N2 - The Winograd Schema Challenge—a set of twin sentences involving pronoun reference disambiguation that seem to require the use of commonsense knowledge—was proposed by Hector Levesque in 2011. By 2019, a number of AI systems, based on large pre-trained transformer-based language models and fine-tuned on these kinds of problems, achieved better than 90% accuracy. In this paper, we review the history of the Winograd Schema Challenge and discuss the lasting contributions of the flurry of research that has taken place on the WSC in the last decade. We discuss the significance of various datasets developed for WSC, and the research community's deeper understanding of the role of surrogate tasks in assessing the intelligence of an AI system.
AB - The Winograd Schema Challenge—a set of twin sentences involving pronoun reference disambiguation that seem to require the use of commonsense knowledge—was proposed by Hector Levesque in 2011. By 2019, a number of AI systems, based on large pre-trained transformer-based language models and fine-tuned on these kinds of problems, achieved better than 90% accuracy. In this paper, we review the history of the Winograd Schema Challenge and discuss the lasting contributions of the flurry of research that has taken place on the WSC in the last decade. We discuss the significance of various datasets developed for WSC, and the research community's deeper understanding of the role of surrogate tasks in assessing the intelligence of an AI system.
KW - Commonsense reasoning
KW - Winograd Schema Challenge
UR - http://www.scopus.com/inward/record.url?scp=85165339713&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85165339713&partnerID=8YFLogxK
U2 - 10.1016/j.artint.2023.103971
DO - 10.1016/j.artint.2023.103971
M3 - Review article
AN - SCOPUS:85165339713
SN - 0004-3702
VL - 325
JO - Artificial Intelligence
JF - Artificial Intelligence
M1 - 103971
ER -