Dynamic meta-embeddings for improved sentence representations

Douwe Kiela; Changhan Wang; Kyunghyun Cho

Dynamic meta-embeddings for improved sentence representations

Douwe Kiela, Changhan Wang, Kyunghyun Cho

Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

While one of the first steps in many NLP systems is selecting what pre-trained word embeddings to use, we argue that such a step is better left for neural networks to figure out by themselves. To that end, we introduce dynamic meta-embeddings, a simple yet effective method for the supervised learning of embedding ensembles, which leads to state-of-the-art performance within the same model class on a variety of tasks. We subsequently show how the technique can be used to shed new light on the usage of word embeddings in NLP systems.

Original language	English (US)
Title of host publication	Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, EMNLP 2018
Editors	Ellen Riloff, David Chiang, Julia Hockenmaier, Jun'ichi Tsujii
Publisher	Association for Computational Linguistics
Pages	1466-1477
Number of pages	12
ISBN (Electronic)	9781948087841
State	Published - Jan 1 2020
Event	2018 Conference on Empirical Methods in Natural Language Processing, EMNLP 2018 - Brussels, Belgium Duration: Oct 31 2018 → Nov 4 2018

Publication series

Name	Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, EMNLP 2018

Conference

Conference	2018 Conference on Empirical Methods in Natural Language Processing, EMNLP 2018
Country	Belgium
City	Brussels
Period	10/31/18 → 11/4/18

ASJC Scopus subject areas

Computational Theory and Mathematics
Computer Science Applications
Information Systems

Access to Document

Fingerprint Dive into the research topics of 'Dynamic meta-embeddings for improved sentence representations'. Together they form a unique fingerprint.

Cite this

Kiela, D., Wang, C., & Cho, K. (2020). Dynamic meta-embeddings for improved sentence representations. In E. Riloff, D. Chiang, J. Hockenmaier, & J. Tsujii (Eds.), Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, EMNLP 2018 (pp. 1466-1477). (Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, EMNLP 2018). Association for Computational Linguistics.

Dynamic meta-embeddings for improved sentence representations. / Kiela, Douwe; Wang, Changhan; Cho, Kyunghyun.

Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, EMNLP 2018. ed. / Ellen Riloff; David Chiang; Julia Hockenmaier; Jun'ichi Tsujii. Association for Computational Linguistics, 2020. p. 1466-1477 (Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, EMNLP 2018).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Kiela, D, Wang, C & Cho, K 2020, Dynamic meta-embeddings for improved sentence representations. in E Riloff, D Chiang, J Hockenmaier & J Tsujii (eds), Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, EMNLP 2018. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, EMNLP 2018, Association for Computational Linguistics, pp. 1466-1477, 2018 Conference on Empirical Methods in Natural Language Processing, EMNLP 2018, Brussels, Belgium, 10/31/18.

Kiela D, Wang C, Cho K. Dynamic meta-embeddings for improved sentence representations. In Riloff E, Chiang D, Hockenmaier J, Tsujii J, editors, Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, EMNLP 2018. Association for Computational Linguistics. 2020. p. 1466-1477. (Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, EMNLP 2018).

Kiela, Douwe ; Wang, Changhan ; Cho, Kyunghyun. / Dynamic meta-embeddings for improved sentence representations. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, EMNLP 2018. editor / Ellen Riloff ; David Chiang ; Julia Hockenmaier ; Jun'ichi Tsujii. Association for Computational Linguistics, 2020. pp. 1466-1477 (Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, EMNLP 2018).

@inproceedings{b195661a51e047329585836738c93d1c,

title = "Dynamic meta-embeddings for improved sentence representations",

abstract = "While one of the first steps in many NLP systems is selecting what pre-trained word embeddings to use, we argue that such a step is better left for neural networks to figure out by themselves. To that end, we introduce dynamic meta-embeddings, a simple yet effective method for the supervised learning of embedding ensembles, which leads to state-of-the-art performance within the same model class on a variety of tasks. We subsequently show how the technique can be used to shed new light on the usage of word embeddings in NLP systems.",

author = "Douwe Kiela and Changhan Wang and Kyunghyun Cho",

year = "2020",

month = jan,

day = "1",

language = "English (US)",

series = "Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, EMNLP 2018",

publisher = "Association for Computational Linguistics",

pages = "1466--1477",

editor = "Ellen Riloff and David Chiang and Julia Hockenmaier and Jun'ichi Tsujii",

booktitle = "Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, EMNLP 2018",

note = "2018 Conference on Empirical Methods in Natural Language Processing, EMNLP 2018 ; Conference date: 31-10-2018 Through 04-11-2018",

}

TY - GEN

T1 - Dynamic meta-embeddings for improved sentence representations

AU - Kiela, Douwe

AU - Wang, Changhan

AU - Cho, Kyunghyun

PY - 2020/1/1

Y1 - 2020/1/1

N2 - While one of the first steps in many NLP systems is selecting what pre-trained word embeddings to use, we argue that such a step is better left for neural networks to figure out by themselves. To that end, we introduce dynamic meta-embeddings, a simple yet effective method for the supervised learning of embedding ensembles, which leads to state-of-the-art performance within the same model class on a variety of tasks. We subsequently show how the technique can be used to shed new light on the usage of word embeddings in NLP systems.

AB - While one of the first steps in many NLP systems is selecting what pre-trained word embeddings to use, we argue that such a step is better left for neural networks to figure out by themselves. To that end, we introduce dynamic meta-embeddings, a simple yet effective method for the supervised learning of embedding ensembles, which leads to state-of-the-art performance within the same model class on a variety of tasks. We subsequently show how the technique can be used to shed new light on the usage of word embeddings in NLP systems.

UR - http://www.scopus.com/inward/record.url?scp=85081732442&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85081732442&partnerID=8YFLogxK

M3 - Conference contribution

T3 - Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, EMNLP 2018

SP - 1466

EP - 1477

BT - Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, EMNLP 2018

A2 - Riloff, Ellen

A2 - Chiang, David

A2 - Hockenmaier, Julia

A2 - Tsujii, Jun'ichi

PB - Association for Computational Linguistics

T2 - 2018 Conference on Empirical Methods in Natural Language Processing, EMNLP 2018

Y2 - 31 October 2018 through 4 November 2018

ER -