The Paradigm Discovery Problem

Alexander Erdmann, Micha Elsner, Shijie Wu, Ryan Cotterell, Nizar Habash

Research output: Contribution to journalArticle

Abstract

This work treats the paradigm discovery problem (PDP), the task of learning an inflectional morphological system from unannotated sentences. We formalize the PDP and develop evaluation metrics for judging systems. Using currently available resources, we construct datasets for the task. We also devise a heuristic benchmark for the PDP and report empirical results on five diverse languages. Our benchmark system first makes use of word embeddings and string similarity to cluster forms by cell and by paradigm. Then, we bootstrap a neural transducer on top of the clustered data to predict words to realize the empty paradigm slots. An error analysis of our system suggests clustering by cell across different inflection classes is the most pressing challenge for future work. Our code and data are available for public use.
Original languageEnglish (US)
JournalProceedings of ACL 2020
StatePublished - 2020
EventThe 58th Annual Meeting of the Association for Computational Linguistics - Seattle, United States
Duration: Jul 5 2020 → …
Conference number: 2020

Keywords

  • cs.CL

Fingerprint Dive into the research topics of 'The Paradigm Discovery Problem'. Together they form a unique fingerprint.

  • Cite this

    Erdmann, A., Elsner, M., Wu, S., Cotterell, R., & Habash, N. (2020). The Paradigm Discovery Problem. Proceedings of ACL 2020.