Evaluating vector space models using human semantic priming results

Allyson Ettinger; Tal Linzen

doi:10.18653/v1/w16-2513

Evaluating vector space models using human semantic priming results

Allyson Ettinger, Tal Linzen

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Vector space models of word representation are often evaluated using human similarity ratings. Those ratings are elicited in explicit tasks and have well-known subjective biases. As an alternative, we propose evaluating vector spaces using implicit cognitive measures. We focus in particular on semantic priming, exploring the strengths and limitations of existing datasets, and propose ways in which those datasets can be improved.

Original language	English (US)
Title of host publication	Proceedings of the 1st Workshop on Evaluating Vector-Space Representations for NLP, RepEval 2016 at the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016
Publisher	Association for Computational Linguistics (ACL)
Pages	72-77
Number of pages	6
ISBN (Electronic)	9781945626142
DOIs	https://doi.org/10.18653/v1/w16-2513
State	Published - 2016
Event	1st Workshop on Evaluating Vector-Space Representations for NLP, RepEval 2016 at the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016 - Berlin, Germany Duration: Aug 7 2016 → …

Publication series

Name	Proceedings of the Annual Meeting of the Association for Computational Linguistics
ISSN (Print)	0736-587X

Conference

Conference	1st Workshop on Evaluating Vector-Space Representations for NLP, RepEval 2016 at the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016
Country/Territory	Germany
City	Berlin
Period	8/7/16 → …

ASJC Scopus subject areas

Computer Science Applications
Linguistics and Language
Language and Linguistics

Access to Document

10.18653/v1/w16-2513

Cite this

Ettinger, A., & Linzen, T. (2016). Evaluating vector space models using human semantic priming results. In Proceedings of the 1st Workshop on Evaluating Vector-Space Representations for NLP, RepEval 2016 at the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016 (pp. 72-77). (Proceedings of the Annual Meeting of the Association for Computational Linguistics). Association for Computational Linguistics (ACL). https://doi.org/10.18653/v1/w16-2513

Evaluating vector space models using human semantic priming results. / Ettinger, Allyson; Linzen, Tal.
Proceedings of the 1st Workshop on Evaluating Vector-Space Representations for NLP, RepEval 2016 at the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016. Association for Computational Linguistics (ACL), 2016. p. 72-77 (Proceedings of the Annual Meeting of the Association for Computational Linguistics).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Ettinger, A & Linzen, T 2016, Evaluating vector space models using human semantic priming results. in Proceedings of the 1st Workshop on Evaluating Vector-Space Representations for NLP, RepEval 2016 at the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016. Proceedings of the Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics (ACL), pp. 72-77, 1st Workshop on Evaluating Vector-Space Representations for NLP, RepEval 2016 at the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, Berlin, Germany, 8/7/16. https://doi.org/10.18653/v1/w16-2513

Ettinger A, Linzen T. Evaluating vector space models using human semantic priming results. In Proceedings of the 1st Workshop on Evaluating Vector-Space Representations for NLP, RepEval 2016 at the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016. Association for Computational Linguistics (ACL). 2016. p. 72-77. (Proceedings of the Annual Meeting of the Association for Computational Linguistics). doi: 10.18653/v1/w16-2513

Ettinger, Allyson ; Linzen, Tal. / Evaluating vector space models using human semantic priming results. Proceedings of the 1st Workshop on Evaluating Vector-Space Representations for NLP, RepEval 2016 at the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016. Association for Computational Linguistics (ACL), 2016. pp. 72-77 (Proceedings of the Annual Meeting of the Association for Computational Linguistics).

@inproceedings{335427138d884932860cb5e994f8705a,

title = "Evaluating vector space models using human semantic priming results",

abstract = "Vector space models of word representation are often evaluated using human similarity ratings. Those ratings are elicited in explicit tasks and have well-known subjective biases. As an alternative, we propose evaluating vector spaces using implicit cognitive measures. We focus in particular on semantic priming, exploring the strengths and limitations of existing datasets, and propose ways in which those datasets can be improved.",

author = "Allyson Ettinger and Tal Linzen",

note = "Publisher Copyright: {\textcopyright} 2016 Proceedings of the Annual Meeting of the Association for Computational Linguistics. All Rights Reserved.; 1st Workshop on Evaluating Vector-Space Representations for NLP, RepEval 2016 at the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016 ; Conference date: 07-08-2016",

year = "2016",

doi = "10.18653/v1/w16-2513",

language = "English (US)",

series = "Proceedings of the Annual Meeting of the Association for Computational Linguistics",

publisher = "Association for Computational Linguistics (ACL)",

pages = "72--77",

booktitle = "Proceedings of the 1st Workshop on Evaluating Vector-Space Representations for NLP, RepEval 2016 at the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016",

}

TY - GEN

T1 - Evaluating vector space models using human semantic priming results

AU - Ettinger, Allyson

AU - Linzen, Tal

PY - 2016

Y1 - 2016

N2 - Vector space models of word representation are often evaluated using human similarity ratings. Those ratings are elicited in explicit tasks and have well-known subjective biases. As an alternative, we propose evaluating vector spaces using implicit cognitive measures. We focus in particular on semantic priming, exploring the strengths and limitations of existing datasets, and propose ways in which those datasets can be improved.

AB - Vector space models of word representation are often evaluated using human similarity ratings. Those ratings are elicited in explicit tasks and have well-known subjective biases. As an alternative, we propose evaluating vector spaces using implicit cognitive measures. We focus in particular on semantic priming, exploring the strengths and limitations of existing datasets, and propose ways in which those datasets can be improved.

UR - http://www.scopus.com/inward/record.url?scp=85028695949&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85028695949&partnerID=8YFLogxK

U2 - 10.18653/v1/w16-2513

DO - 10.18653/v1/w16-2513

M3 - Conference contribution

AN - SCOPUS:85028695949

T3 - Proceedings of the Annual Meeting of the Association for Computational Linguistics

SP - 72

EP - 77

BT - Proceedings of the 1st Workshop on Evaluating Vector-Space Representations for NLP, RepEval 2016 at the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016

PB - Association for Computational Linguistics (ACL)

T2 - 1st Workshop on Evaluating Vector-Space Representations for NLP, RepEval 2016 at the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016

Y2 - 7 August 2016

ER -

Evaluating vector space models using human semantic priming results

Abstract

Publication series

Conference

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this