TY - GEN
T1 - The paradigm discovery problem
AU - Erdmann, Alexander
AU - Elsner, Micha
AU - Wu, Shijie
AU - Cotterell, Ryan
AU - Habash, Nizar
N1 - Publisher Copyright:
© 2020 Association for Computational Linguistics
PY - 2020
Y1 - 2020
N2 - This work treats the paradigm discovery problem (PDP)-the task of learning an inflectional morphological system from unannotated sentences. We formalize the PDP and develop evaluation metrics for judging systems. Using currently available resources, we construct datasets for the task. We also devise a heuristic benchmark for the PDP and report empirical results on five diverse languages. Our benchmark system first makes use of word embeddings and string similarity to cluster forms by cell and by paradigm. Then, we bootstrap a neural transducer on top of the clustered data to predict words to realize the empty paradigm slots. An error analysis of our system suggests clustering by cell across different inflection classes is the most pressing challenge for future work. Our code and data are available at https://github.com/alexerdmann/ParadigmDiscovery.
AB - This work treats the paradigm discovery problem (PDP)-the task of learning an inflectional morphological system from unannotated sentences. We formalize the PDP and develop evaluation metrics for judging systems. Using currently available resources, we construct datasets for the task. We also devise a heuristic benchmark for the PDP and report empirical results on five diverse languages. Our benchmark system first makes use of word embeddings and string similarity to cluster forms by cell and by paradigm. Then, we bootstrap a neural transducer on top of the clustered data to predict words to realize the empty paradigm slots. An error analysis of our system suggests clustering by cell across different inflection classes is the most pressing challenge for future work. Our code and data are available at https://github.com/alexerdmann/ParadigmDiscovery.
UR - http://www.scopus.com/inward/record.url?scp=85095548167&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85095548167&partnerID=8YFLogxK
M3 - Conference contribution
AN - SCOPUS:85095548167
T3 - Proceedings of the Annual Meeting of the Association for Computational Linguistics
SP - 7778
EP - 7790
BT - ACL 2020 - 58th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference
PB - Association for Computational Linguistics (ACL)
T2 - 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020
Y2 - 5 July 2020 through 10 July 2020
ER -