Robust Dictionary Lookup in Multiple Noisy Orthographies

Lingliang Zhang; Nizar Habash; Godfried Toussaint

Robust Dictionary Lookup in Multiple Noisy Orthographies

Lingliang Zhang, Nizar Habash, Godfried Toussaint

Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

We present the MultiScript Phonetic Search algorithm to address the problem of language learners looking up unfamiliar words that they heard. We apply it to Arabic dictionary lookup with noisy queries done using both the Arabic and Roman scripts. Our algorithm is based on a computational phonetic distance metric that can be optionally machine learned. To benchmark our performance, we created the ArabScribe dataset, containing 10,000 noisy transcriptions of random Arabic dictionary words. Our algorithm outperforms Google Translate’s “did you mean" feature, as well as the Yamli smart Arabic keyboard.

Original language	English (US)
Title of host publication	WANLP 2017, co-located with EACL 2017 - 3rd Arabic Natural Language Processing Workshop, Proceedings of the Workshop
Publisher	Association for Computational Linguistics (ACL)
Pages	119-129
Number of pages	11
ISBN (Electronic)	9781945626449
State	Published - 2017
Event	3rd Arabic Natural Language Processing Workshop, WANLP 2017 held at EACL 2017 - Valencia, Spain Duration: Apr 3 2017 → …

Publication series

Name	WANLP 2017, co-located with EACL 2017 - 3rd Arabic Natural Language Processing Workshop, Proceedings of the Workshop

Conference

Conference	3rd Arabic Natural Language Processing Workshop, WANLP 2017 held at EACL 2017
Country/Territory	Spain
City	Valencia
Period	4/3/17 → …

ASJC Scopus subject areas

Language and Linguistics
Computational Theory and Mathematics
Computer Science Applications
Software

Cite this

Zhang, L., Habash, N., & Toussaint, G. (2017). Robust Dictionary Lookup in Multiple Noisy Orthographies. In WANLP 2017, co-located with EACL 2017 - 3rd Arabic Natural Language Processing Workshop, Proceedings of the Workshop (pp. 119-129). (WANLP 2017, co-located with EACL 2017 - 3rd Arabic Natural Language Processing Workshop, Proceedings of the Workshop). Association for Computational Linguistics (ACL).

Robust Dictionary Lookup in Multiple Noisy Orthographies. / Zhang, Lingliang; Habash, Nizar; Toussaint, Godfried.
WANLP 2017, co-located with EACL 2017 - 3rd Arabic Natural Language Processing Workshop, Proceedings of the Workshop. Association for Computational Linguistics (ACL), 2017. p. 119-129 (WANLP 2017, co-located with EACL 2017 - 3rd Arabic Natural Language Processing Workshop, Proceedings of the Workshop).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Zhang, L, Habash, N & Toussaint, G 2017, Robust Dictionary Lookup in Multiple Noisy Orthographies. in WANLP 2017, co-located with EACL 2017 - 3rd Arabic Natural Language Processing Workshop, Proceedings of the Workshop. WANLP 2017, co-located with EACL 2017 - 3rd Arabic Natural Language Processing Workshop, Proceedings of the Workshop, Association for Computational Linguistics (ACL), pp. 119-129, 3rd Arabic Natural Language Processing Workshop, WANLP 2017 held at EACL 2017, Valencia, Spain, 4/3/17.

Zhang L, Habash N, Toussaint G. Robust Dictionary Lookup in Multiple Noisy Orthographies. In WANLP 2017, co-located with EACL 2017 - 3rd Arabic Natural Language Processing Workshop, Proceedings of the Workshop. Association for Computational Linguistics (ACL). 2017. p. 119-129. (WANLP 2017, co-located with EACL 2017 - 3rd Arabic Natural Language Processing Workshop, Proceedings of the Workshop).

Zhang, Lingliang ; Habash, Nizar ; Toussaint, Godfried. / Robust Dictionary Lookup in Multiple Noisy Orthographies. WANLP 2017, co-located with EACL 2017 - 3rd Arabic Natural Language Processing Workshop, Proceedings of the Workshop. Association for Computational Linguistics (ACL), 2017. pp. 119-129 (WANLP 2017, co-located with EACL 2017 - 3rd Arabic Natural Language Processing Workshop, Proceedings of the Workshop).

@inproceedings{42520ca939904a0cb6ef222d5411d142,

title = "Robust Dictionary Lookup in Multiple Noisy Orthographies",

abstract = "We present the MultiScript Phonetic Search algorithm to address the problem of language learners looking up unfamiliar words that they heard. We apply it to Arabic dictionary lookup with noisy queries done using both the Arabic and Roman scripts. Our algorithm is based on a computational phonetic distance metric that can be optionally machine learned. To benchmark our performance, we created the ArabScribe dataset, containing 10,000 noisy transcriptions of random Arabic dictionary words. Our algorithm outperforms Google Translate{\textquoteright}s “did you mean{"} feature, as well as the Yamli smart Arabic keyboard.",

author = "Lingliang Zhang and Nizar Habash and Godfried Toussaint",

note = "Publisher Copyright: {\textcopyright}2017 Association for Computational Linguistics; 3rd Arabic Natural Language Processing Workshop, WANLP 2017 held at EACL 2017 ; Conference date: 03-04-2017",

year = "2017",

language = "English (US)",

series = "WANLP 2017, co-located with EACL 2017 - 3rd Arabic Natural Language Processing Workshop, Proceedings of the Workshop",

publisher = "Association for Computational Linguistics (ACL)",

pages = "119--129",

booktitle = "WANLP 2017, co-located with EACL 2017 - 3rd Arabic Natural Language Processing Workshop, Proceedings of the Workshop",

}

TY - GEN

T1 - Robust Dictionary Lookup in Multiple Noisy Orthographies

AU - Zhang, Lingliang

AU - Habash, Nizar

AU - Toussaint, Godfried

PY - 2017

Y1 - 2017

N2 - We present the MultiScript Phonetic Search algorithm to address the problem of language learners looking up unfamiliar words that they heard. We apply it to Arabic dictionary lookup with noisy queries done using both the Arabic and Roman scripts. Our algorithm is based on a computational phonetic distance metric that can be optionally machine learned. To benchmark our performance, we created the ArabScribe dataset, containing 10,000 noisy transcriptions of random Arabic dictionary words. Our algorithm outperforms Google Translate’s “did you mean" feature, as well as the Yamli smart Arabic keyboard.

AB - We present the MultiScript Phonetic Search algorithm to address the problem of language learners looking up unfamiliar words that they heard. We apply it to Arabic dictionary lookup with noisy queries done using both the Arabic and Roman scripts. Our algorithm is based on a computational phonetic distance metric that can be optionally machine learned. To benchmark our performance, we created the ArabScribe dataset, containing 10,000 noisy transcriptions of random Arabic dictionary words. Our algorithm outperforms Google Translate’s “did you mean" feature, as well as the Yamli smart Arabic keyboard.

UR - http://www.scopus.com/inward/record.url?scp=85122950314&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85122950314&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:85122950314

T3 - WANLP 2017, co-located with EACL 2017 - 3rd Arabic Natural Language Processing Workshop, Proceedings of the Workshop

SP - 119

EP - 129

BT - WANLP 2017, co-located with EACL 2017 - 3rd Arabic Natural Language Processing Workshop, Proceedings of the Workshop

PB - Association for Computational Linguistics (ACL)

T2 - 3rd Arabic Natural Language Processing Workshop, WANLP 2017 held at EACL 2017

Y2 - 3 April 2017

ER -

Robust Dictionary Lookup in Multiple Noisy Orthographies

Abstract

Publication series

Conference

ASJC Scopus subject areas

Other files and links

Fingerprint

Cite this