CoNLL 2017 shared task: Multilingual parsing from raw text to universal dependencies

Daniel Zeman; Martin Popel; Milan Straka; Jan Hajič; Joakim Nivre; Filip Ginter; Juhani Luotolahti; Sampo Pyysalo; Slav Petrov; Martin Potthast; Francis Tyers; Elena Badmaeva; Memduh Gökırmak; Anna Nedoluzhko; Silvie Cinková; Jan Hajič; Jaroslava Hlaváčová; Václava Kettnerová; Zdeňka Urešová; Jenna Kanerva; Stina Ojala; Anna Missilä; Christopher Manning; Sebastian Schuster; Siva Reddy; Dima Taji; Nizar Habash; Herman Leung; Marie Catherine de Marneffe; Manuela Sanguinetti; Maria Simi; Hiroshi Kanayama; Valeria de Paiva; Kira Droganova; Héctor Martínez Alonso; Çağrı Çöltekin; Umut Sulubacak; Hans Uszkoreit; Vivien Macketanz; Aljoscha Burchardt; Kim Harris; Katrin Marheinecke; Georg Rehm; Tolga Kayadelen; Mohammed Attia; Ali Elkahky; Zhuoran Yu; Emily Pitler; Saran Lertpradit; Michael Mandl; Jesse Kirchner; Hector Fernandez Alcalde; Jana Strnadová; Esha Banerjee; Ruli Manurung; Antonio Stella; Atsuko Shimada; Sookyoung Kwak; Gustavo Mendonça; Tatiana Lando; Rattima Nitisaroj; Josie Li

doi:10.18653/v1/k17-3001

CoNLL 2017 shared task: Multilingual parsing from raw text to universal dependencies

Daniel Zeman, Martin Popel, Milan Straka, Jan Hajič, Joakim Nivre, Filip Ginter, Juhani Luotolahti, Sampo Pyysalo, Slav Petrov, Martin Potthast, Francis Tyers, Elena Badmaeva, Memduh Gökırmak, Anna Nedoluzhko, Silvie Cinková, Jan Hajič, Jaroslava Hlaváčová, Václava Kettnerová, Zdeňka Urešová, Jenna KanervaStina Ojala, Anna Missilä, Christopher Manning, Sebastian Schuster, Siva Reddy, Dima Taji, Nizar Habash, Herman Leung, Marie Catherine de Marneffe, Manuela Sanguinetti, Maria Simi, Hiroshi Kanayama, Valeria de Paiva, Kira Droganova, Héctor Martínez Alonso, Çağrı Çöltekin, Umut Sulubacak, Hans Uszkoreit, Vivien Macketanz, Aljoscha Burchardt, Kim Harris, Katrin Marheinecke, Georg Rehm, Tolga Kayadelen, Mohammed Attia, Ali Elkahky, Zhuoran Yu, Emily Pitler, Saran Lertpradit, Michael Mandl, Jesse Kirchner, Hector Fernandez Alcalde, Jana Strnadová, Esha Banerjee, Ruli Manurung, Antonio Stella, Atsuko Shimada, Sookyoung Kwak, Gustavo Mendonça, Tatiana Lando, Rattima Nitisaroj, Josie Li

Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

The Conference on Computational Natural Language Learning (CoNLL) features a shared task, in which participants train and test their learning systems on the same data sets. In 2017, one of two tasks was devoted to learning dependency parsers for a large number of languages, in a real-world setting without any gold-standard annotation on input. All test sets followed a unified annotation scheme, namely that of Universal Dependencies. In this paper, we define the task and evaluation methodology, describe data preparation, report and analyze the main results, and provide a brief categorization of the different approaches of the participating systems.

Original language	English (US)
Title of host publication	CoNLL 2017 - SIGNLL Conference on Computational Natural Language Learning, Proceedings of the CoNLL 2017 Shared Task
Subtitle of host publication	Multilingual Parsing from Raw Text to Universal Dependencies
Publisher	Association for Computational Linguistics (ACL)
Pages	1-19
Number of pages	19
ISBN (Electronic)	9781945626708
DOIs	https://doi.org/10.18653/v1/k17-3001
State	Published - Jan 1 2017
Event	2017 SIGNLL Conference on Computational Natural Language Learning- CoNLL Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, CoNLL 2017 - Vancouver, Canada Duration: Aug 3 2017 → Aug 4 2017

Publication series

Name	CoNLL 2017 - SIGNLL Conference on Computational Natural Language Learning, Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies

Conference

Conference	2017 SIGNLL Conference on Computational Natural Language Learning- CoNLL Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, CoNLL 2017
Country/Territory	Canada
City	Vancouver
Period	8/3/17 → 8/4/17

ASJC Scopus subject areas

Artificial Intelligence
Human-Computer Interaction
Linguistics and Language

Access to Document

10.18653/v1/k17-3001

Cite this

Zeman, D., Popel, M., Straka, M., Hajič, J., Nivre, J., Ginter, F., Luotolahti, J., Pyysalo, S., Petrov, S., Potthast, M., Tyers, F., Badmaeva, E., Gökırmak, M., Nedoluzhko, A., Cinková, S., Hajič, J., Hlaváčová, J., Kettnerová, V., Urešová, Z., ... Li, J. (2017). CoNLL 2017 shared task: Multilingual parsing from raw text to universal dependencies. In CoNLL 2017 - SIGNLL Conference on Computational Natural Language Learning, Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies (pp. 1-19). (CoNLL 2017 - SIGNLL Conference on Computational Natural Language Learning, Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies). Association for Computational Linguistics (ACL). https://doi.org/10.18653/v1/k17-3001

CoNLL 2017 shared task: Multilingual parsing from raw text to universal dependencies. / Zeman, Daniel; Popel, Martin; Straka, Milan et al.
CoNLL 2017 - SIGNLL Conference on Computational Natural Language Learning, Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies. Association for Computational Linguistics (ACL), 2017. p. 1-19 (CoNLL 2017 - SIGNLL Conference on Computational Natural Language Learning, Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Zeman, D, Popel, M, Straka, M, Hajič, J, Nivre, J, Ginter, F, Luotolahti, J, Pyysalo, S, Petrov, S, Potthast, M, Tyers, F, Badmaeva, E, Gökırmak, M, Nedoluzhko, A, Cinková, S, Hajič, J, Hlaváčová, J, Kettnerová, V, Urešová, Z, Kanerva, J, Ojala, S, Missilä, A, Manning, C, Schuster, S, Reddy, S, Taji, D, Habash, N, Leung, H, de Marneffe, MC, Sanguinetti, M, Simi, M, Kanayama, H, de Paiva, V, Droganova, K, Alonso, HM, Çöltekin, Ç, Sulubacak, U, Uszkoreit, H, Macketanz, V, Burchardt, A, Harris, K, Marheinecke, K, Rehm, G, Kayadelen, T, Attia, M, Elkahky, A, Yu, Z, Pitler, E, Lertpradit, S, Mandl, M, Kirchner, J, Alcalde, HF, Strnadová, J, Banerjee, E, Manurung, R, Stella, A, Shimada, A, Kwak, S, Mendonça, G, Lando, T, Nitisaroj, R & Li, J 2017, CoNLL 2017 shared task: Multilingual parsing from raw text to universal dependencies. in CoNLL 2017 - SIGNLL Conference on Computational Natural Language Learning, Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies. CoNLL 2017 - SIGNLL Conference on Computational Natural Language Learning, Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, Association for Computational Linguistics (ACL), pp. 1-19, 2017 SIGNLL Conference on Computational Natural Language Learning- CoNLL Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, CoNLL 2017, Vancouver, Canada, 8/3/17. https://doi.org/10.18653/v1/k17-3001

Zeman D, Popel M, Straka M, Hajič J, Nivre J, Ginter F et al. CoNLL 2017 shared task: Multilingual parsing from raw text to universal dependencies. In CoNLL 2017 - SIGNLL Conference on Computational Natural Language Learning, Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies. Association for Computational Linguistics (ACL). 2017. p. 1-19. (CoNLL 2017 - SIGNLL Conference on Computational Natural Language Learning, Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies). doi: 10.18653/v1/k17-3001

Zeman, Daniel ; Popel, Martin ; Straka, Milan et al. / CoNLL 2017 shared task : Multilingual parsing from raw text to universal dependencies. CoNLL 2017 - SIGNLL Conference on Computational Natural Language Learning, Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies. Association for Computational Linguistics (ACL), 2017. pp. 1-19 (CoNLL 2017 - SIGNLL Conference on Computational Natural Language Learning, Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies).

@inproceedings{aa2680219f71415a9e036575b99ea7e1,

title = "CoNLL 2017 shared task: Multilingual parsing from raw text to universal dependencies",

abstract = "The Conference on Computational Natural Language Learning (CoNLL) features a shared task, in which participants train and test their learning systems on the same data sets. In 2017, one of two tasks was devoted to learning dependency parsers for a large number of languages, in a real-world setting without any gold-standard annotation on input. All test sets followed a unified annotation scheme, namely that of Universal Dependencies. In this paper, we define the task and evaluation methodology, describe data preparation, report and analyze the main results, and provide a brief categorization of the different approaches of the participating systems.",

author = "Daniel Zeman and Martin Popel and Milan Straka and Jan Haji{\v c} and Joakim Nivre and Filip Ginter and Juhani Luotolahti and Sampo Pyysalo and Slav Petrov and Martin Potthast and Francis Tyers and Elena Badmaeva and Memduh G{\"o}kırmak and Anna Nedoluzhko and Silvie Cinkov{\'a} and Jan Haji{\v c} and Jaroslava Hlav{\'a}{\v c}ov{\'a} and V{\'a}clava Kettnerov{\'a} and Zde{\v n}ka Ure{\v s}ov{\'a} and Jenna Kanerva and Stina Ojala and Anna Missil{\"a} and Christopher Manning and Sebastian Schuster and Siva Reddy and Dima Taji and Nizar Habash and Herman Leung and {de Marneffe}, {Marie Catherine} and Manuela Sanguinetti and Maria Simi and Hiroshi Kanayama and {de Paiva}, Valeria and Kira Droganova and Alonso, {H{\'e}ctor Mart{\'i}nez} and {\c C}ağrı {\c C}{\"o}ltekin and Umut Sulubacak and Hans Uszkoreit and Vivien Macketanz and Aljoscha Burchardt and Kim Harris and Katrin Marheinecke and Georg Rehm and Tolga Kayadelen and Mohammed Attia and Ali Elkahky and Zhuoran Yu and Emily Pitler and Saran Lertpradit and Michael Mandl and Jesse Kirchner and Alcalde, {Hector Fernandez} and Jana Strnadov{\'a} and Esha Banerjee and Ruli Manurung and Antonio Stella and Atsuko Shimada and Sookyoung Kwak and Gustavo Mendon{\c c}a and Tatiana Lando and Rattima Nitisaroj and Josie Li",

note = "Publisher Copyright: {\textcopyright} 2017 Association for Computational Linguistics.; 2017 SIGNLL Conference on Computational Natural Language Learning- CoNLL Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, CoNLL 2017 ; Conference date: 03-08-2017 Through 04-08-2017",

year = "2017",

month = jan,

day = "1",

doi = "10.18653/v1/k17-3001",

language = "English (US)",

series = "CoNLL 2017 - SIGNLL Conference on Computational Natural Language Learning, Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies",

publisher = "Association for Computational Linguistics (ACL)",

pages = "1--19",

booktitle = "CoNLL 2017 - SIGNLL Conference on Computational Natural Language Learning, Proceedings of the CoNLL 2017 Shared Task",

}

TY - GEN

T1 - CoNLL 2017 shared task

T2 - 2017 SIGNLL Conference on Computational Natural Language Learning- CoNLL Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, CoNLL 2017

AU - Zeman, Daniel

AU - Popel, Martin

AU - Straka, Milan

AU - Hajič, Jan

AU - Nivre, Joakim

AU - Ginter, Filip

AU - Luotolahti, Juhani

AU - Pyysalo, Sampo

AU - Petrov, Slav

AU - Potthast, Martin

AU - Tyers, Francis

AU - Badmaeva, Elena

AU - Gökırmak, Memduh

AU - Nedoluzhko, Anna

AU - Cinková, Silvie

AU - Hajič, Jan

AU - Hlaváčová, Jaroslava

AU - Kettnerová, Václava

AU - Urešová, Zdeňka

AU - Kanerva, Jenna

AU - Ojala, Stina

AU - Missilä, Anna

AU - Manning, Christopher

AU - Schuster, Sebastian

AU - Reddy, Siva

AU - Taji, Dima

AU - Habash, Nizar

AU - Leung, Herman

AU - de Marneffe, Marie Catherine

AU - Sanguinetti, Manuela

AU - Simi, Maria

AU - Kanayama, Hiroshi

AU - de Paiva, Valeria

AU - Droganova, Kira

AU - Alonso, Héctor Martínez

AU - Çöltekin, Çağrı

AU - Sulubacak, Umut

AU - Uszkoreit, Hans

AU - Macketanz, Vivien

AU - Burchardt, Aljoscha

AU - Harris, Kim

AU - Marheinecke, Katrin

AU - Rehm, Georg

AU - Kayadelen, Tolga

AU - Attia, Mohammed

AU - Elkahky, Ali

AU - Yu, Zhuoran

AU - Pitler, Emily

AU - Lertpradit, Saran

AU - Mandl, Michael

AU - Kirchner, Jesse

AU - Alcalde, Hector Fernandez

AU - Strnadová, Jana

AU - Banerjee, Esha

AU - Manurung, Ruli

AU - Stella, Antonio

AU - Shimada, Atsuko

AU - Kwak, Sookyoung

AU - Mendonça, Gustavo

AU - Lando, Tatiana

AU - Nitisaroj, Rattima

AU - Li, Josie

PY - 2017/1/1

Y1 - 2017/1/1

N2 - The Conference on Computational Natural Language Learning (CoNLL) features a shared task, in which participants train and test their learning systems on the same data sets. In 2017, one of two tasks was devoted to learning dependency parsers for a large number of languages, in a real-world setting without any gold-standard annotation on input. All test sets followed a unified annotation scheme, namely that of Universal Dependencies. In this paper, we define the task and evaluation methodology, describe data preparation, report and analyze the main results, and provide a brief categorization of the different approaches of the participating systems.

AB - The Conference on Computational Natural Language Learning (CoNLL) features a shared task, in which participants train and test their learning systems on the same data sets. In 2017, one of two tasks was devoted to learning dependency parsers for a large number of languages, in a real-world setting without any gold-standard annotation on input. All test sets followed a unified annotation scheme, namely that of Universal Dependencies. In this paper, we define the task and evaluation methodology, describe data preparation, report and analyze the main results, and provide a brief categorization of the different approaches of the participating systems.

UR - http://www.scopus.com/inward/record.url?scp=85073123106&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85073123106&partnerID=8YFLogxK

U2 - 10.18653/v1/k17-3001

DO - 10.18653/v1/k17-3001

M3 - Conference contribution

AN - SCOPUS:85073123106

T3 - CoNLL 2017 - SIGNLL Conference on Computational Natural Language Learning, Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies

SP - 1

EP - 19

BT - CoNLL 2017 - SIGNLL Conference on Computational Natural Language Learning, Proceedings of the CoNLL 2017 Shared Task

PB - Association for Computational Linguistics (ACL)

Y2 - 3 August 2017 through 4 August 2017

ER -

CoNLL 2017 shared task: Multilingual parsing from raw text to universal dependencies

Abstract

Publication series

Conference

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this