TY - GEN
T1 - Neural unsupervised parsing beyond english
AU - Kann, Katharina
AU - Mohananey, Anhad
AU - Cho, Kyunghyun
AU - Bowman, Samuel R.
N1 - Funding Information:
This work has benefited from the support of Sam-sung Research under the project Improving Deep Learning using Latent Structure and from the donation of a Titan V GPU by NVIDIA Corporation.
Funding Information:
This work has benefited from the support of Samsung Research under the project Improving Deep Learning using Latent Structure and from the donation of a Titan V GPU by NVIDIA Corporation.
Publisher Copyright:
© 2019 Association for Computational Linguistics
PY - 2021
Y1 - 2021
N2 - Recently, neural network models which automatically infer syntactic structure from raw text have started to achieve promising results. However, earlier work on unsupervised parsing shows large performance differences between non-neural models trained on corpora in different languages, even for comparable amounts of data. With that in mind, we train instances of the PRPN architecture (Shen et al., 2018a)-one of these unsupervised neural network parsers-for Arabic, Chinese, English, and German. We find that (i) the model strongly outperforms trivial baselines and, thus, acquires at least some parsing ability for all languages; (ii) good hyperparameter values seem to be universal; (iii) how the model benefits from larger training set sizes depends on the corpus, with the model achieving the largest performance gains when increasing the number of sentences from 2,500 to 12,500 for English. In addition, we show that, by sharing parameters between the related languages German and English, we can improve the model's unsupervised parsing F1 score by up to 4% in the low-resource setting.
AB - Recently, neural network models which automatically infer syntactic structure from raw text have started to achieve promising results. However, earlier work on unsupervised parsing shows large performance differences between non-neural models trained on corpora in different languages, even for comparable amounts of data. With that in mind, we train instances of the PRPN architecture (Shen et al., 2018a)-one of these unsupervised neural network parsers-for Arabic, Chinese, English, and German. We find that (i) the model strongly outperforms trivial baselines and, thus, acquires at least some parsing ability for all languages; (ii) good hyperparameter values seem to be universal; (iii) how the model benefits from larger training set sizes depends on the corpus, with the model achieving the largest performance gains when increasing the number of sentences from 2,500 to 12,500 for English. In addition, we show that, by sharing parameters between the related languages German and English, we can improve the model's unsupervised parsing F1 score by up to 4% in the low-resource setting.
UR - http://www.scopus.com/inward/record.url?scp=85118981999&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85118981999&partnerID=8YFLogxK
M3 - Conference contribution
AN - SCOPUS:85118981999
T3 - DeepLo@EMNLP-IJCNLP 2019 - Proceedings of the 2nd Workshop on Deep Learning Approaches for Low-Resource Natural Language Processing - Proceedings
SP - 209
EP - 218
BT - DeepLo@EMNLP-IJCNLP 2019 - Proceedings of the 2nd Workshop on Deep Learning Approaches for Low-Resource Natural Language Processing - Proceedings
PB - Association for Computational Linguistics (ACL)
T2 - 2nd Workshop on Deep Learning Approaches for Low-Resource Natural Language Processing, DeepLo@EMNLP-IJCNLP 2019
Y2 - 3 November 2019
ER -