Foreign Words and the Automatic Processing of Arabic Social Media Text Written in Roman Script

Ramy Eskander; Mohamed Al-Badrashiny; Nizar Habash; Owen Rambow

Foreign Words and the Automatic Processing of Arabic Social Media Text Written in Roman Script

Ramy Eskander, Mohamed Al-Badrashiny, Nizar Habash, Owen Rambow

Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Arabic on social media has all the properties of any language on social media that make it tough for natural language processing, plus some specific problems. These include diglossia, the use of an alternative alphabet (Roman), and code switching with foreign languages. In this paper, we present a system which can process Arabic written in Roman alphabet (“Arabizi”). It identifies whether each word is a foreign word or one of another four categories (Arabic, name, punctuation, sound), and transliterates Arabic words and names into the Arabic alphabet. We obtain an overall system performance of 83.8% on an unseen test set.

Original language	English (US)
Title of host publication	1st Workshop on Computational Approaches to Code Switching, Switching 2014 at the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014 - Proceedings
Editors	Mona Diab, Julia Hirschberg, Pascale Fung, Thamar Solorio
Publisher	Association for Computational Linguistics (ACL)
Pages	1-12
Number of pages	12
ISBN (Electronic)	9781937284961
State	Published - 2014
Event	1st Workshop on Computational Approaches to Code Switching, Switching 2014 at the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014 - Doha, Qatar Duration: Oct 25 2014 → …

Publication series

Name	1st Workshop on Computational Approaches to Code Switching, Switching 2014 at the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014 - Proceedings

Conference

Conference	1st Workshop on Computational Approaches to Code Switching, Switching 2014 at the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014
Country/Territory	Qatar
City	Doha
Period	10/25/14 → …

ASJC Scopus subject areas

Computational Theory and Mathematics
Language and Linguistics
Linguistics and Language

Cite this

Eskander, R., Al-Badrashiny, M., Habash, N., & Rambow, O. (2014). Foreign Words and the Automatic Processing of Arabic Social Media Text Written in Roman Script. In M. Diab, J. Hirschberg, P. Fung, & T. Solorio (Eds.), 1st Workshop on Computational Approaches to Code Switching, Switching 2014 at the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014 - Proceedings (pp. 1-12). (1st Workshop on Computational Approaches to Code Switching, Switching 2014 at the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014 - Proceedings). Association for Computational Linguistics (ACL).

Foreign Words and the Automatic Processing of Arabic Social Media Text Written in Roman Script. / Eskander, Ramy; Al-Badrashiny, Mohamed; Habash, Nizar et al.
1st Workshop on Computational Approaches to Code Switching, Switching 2014 at the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014 - Proceedings. ed. / Mona Diab; Julia Hirschberg; Pascale Fung; Thamar Solorio. Association for Computational Linguistics (ACL), 2014. p. 1-12 (1st Workshop on Computational Approaches to Code Switching, Switching 2014 at the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014 - Proceedings).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Eskander, R, Al-Badrashiny, M, Habash, N & Rambow, O 2014, Foreign Words and the Automatic Processing of Arabic Social Media Text Written in Roman Script. in M Diab, J Hirschberg, P Fung & T Solorio (eds), 1st Workshop on Computational Approaches to Code Switching, Switching 2014 at the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014 - Proceedings. 1st Workshop on Computational Approaches to Code Switching, Switching 2014 at the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014 - Proceedings, Association for Computational Linguistics (ACL), pp. 1-12, 1st Workshop on Computational Approaches to Code Switching, Switching 2014 at the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, Doha, Qatar, 10/25/14.

Eskander R, Al-Badrashiny M, Habash N, Rambow O. Foreign Words and the Automatic Processing of Arabic Social Media Text Written in Roman Script. In Diab M, Hirschberg J, Fung P, Solorio T, editors, 1st Workshop on Computational Approaches to Code Switching, Switching 2014 at the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014 - Proceedings. Association for Computational Linguistics (ACL). 2014. p. 1-12. (1st Workshop on Computational Approaches to Code Switching, Switching 2014 at the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014 - Proceedings).

Eskander, Ramy ; Al-Badrashiny, Mohamed ; Habash, Nizar et al. / Foreign Words and the Automatic Processing of Arabic Social Media Text Written in Roman Script. 1st Workshop on Computational Approaches to Code Switching, Switching 2014 at the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014 - Proceedings. editor / Mona Diab ; Julia Hirschberg ; Pascale Fung ; Thamar Solorio. Association for Computational Linguistics (ACL), 2014. pp. 1-12 (1st Workshop on Computational Approaches to Code Switching, Switching 2014 at the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014 - Proceedings).

@inproceedings{2ab529d37d9043ff934e68b5bf6b377c,

title = "Foreign Words and the Automatic Processing of Arabic Social Media Text Written in Roman Script",

abstract = "Arabic on social media has all the properties of any language on social media that make it tough for natural language processing, plus some specific problems. These include diglossia, the use of an alternative alphabet (Roman), and code switching with foreign languages. In this paper, we present a system which can process Arabic written in Roman alphabet (“Arabizi”). It identifies whether each word is a foreign word or one of another four categories (Arabic, name, punctuation, sound), and transliterates Arabic words and names into the Arabic alphabet. We obtain an overall system performance of 83.8% on an unseen test set.",

author = "Ramy Eskander and Mohamed Al-Badrashiny and Nizar Habash and Owen Rambow",

note = "Publisher Copyright: {\textcopyright} 2014 Association for Computational Linguistics; 1st Workshop on Computational Approaches to Code Switching, Switching 2014 at the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014 ; Conference date: 25-10-2014",

year = "2014",

language = "English (US)",

series = "1st Workshop on Computational Approaches to Code Switching, Switching 2014 at the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014 - Proceedings",

publisher = "Association for Computational Linguistics (ACL)",

pages = "1--12",

editor = "Mona Diab and Julia Hirschberg and Pascale Fung and Thamar Solorio",

booktitle = "1st Workshop on Computational Approaches to Code Switching, Switching 2014 at the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014 - Proceedings",

}

TY - GEN

T1 - Foreign Words and the Automatic Processing of Arabic Social Media Text Written in Roman Script

AU - Eskander, Ramy

AU - Al-Badrashiny, Mohamed

AU - Habash, Nizar

AU - Rambow, Owen

PY - 2014

Y1 - 2014

N2 - Arabic on social media has all the properties of any language on social media that make it tough for natural language processing, plus some specific problems. These include diglossia, the use of an alternative alphabet (Roman), and code switching with foreign languages. In this paper, we present a system which can process Arabic written in Roman alphabet (“Arabizi”). It identifies whether each word is a foreign word or one of another four categories (Arabic, name, punctuation, sound), and transliterates Arabic words and names into the Arabic alphabet. We obtain an overall system performance of 83.8% on an unseen test set.

AB - Arabic on social media has all the properties of any language on social media that make it tough for natural language processing, plus some specific problems. These include diglossia, the use of an alternative alphabet (Roman), and code switching with foreign languages. In this paper, we present a system which can process Arabic written in Roman alphabet (“Arabizi”). It identifies whether each word is a foreign word or one of another four categories (Arabic, name, punctuation, sound), and transliterates Arabic words and names into the Arabic alphabet. We obtain an overall system performance of 83.8% on an unseen test set.

UR - http://www.scopus.com/inward/record.url?scp=85082806742&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85082806742&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:85082806742

T3 - 1st Workshop on Computational Approaches to Code Switching, Switching 2014 at the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014 - Proceedings

SP - 1

EP - 12

BT - 1st Workshop on Computational Approaches to Code Switching, Switching 2014 at the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014 - Proceedings

A2 - Diab, Mona

A2 - Hirschberg, Julia

A2 - Fung, Pascale

A2 - Solorio, Thamar

PB - Association for Computational Linguistics (ACL)

T2 - 1st Workshop on Computational Approaches to Code Switching, Switching 2014 at the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014

Y2 - 25 October 2014

ER -

Foreign Words and the Automatic Processing of Arabic Social Media Text Written in Roman Script

Abstract

Publication series

Conference

ASJC Scopus subject areas

Other files and links

Fingerprint

Cite this