Search engine guided neural machine translation

Jiatao Gu, Yong Wang, Kyunghyun Cho, Victor O.K. Li

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In this paper, we extend an attention-based neural machine translation (NMT) model by allowing it to access an entire training set of parallel sentence pairs even after training. The proposed approach consists of two stages. In the first stage-retrieval stage-, an off-the-shelf, black-box search engine is used to retrieve a small subset of sentence pairs from a training set given a source sentence. These pairs are further filtered based on a fuzzy matching score based on edit distance. In the second stage-translation stage-, a novel translation model, called search engine guided NMT (SEG-NMT), seamlessly uses both the source sentence and a set of retrieved sentence pairs to perform the translation. Empirical evaluation on three language pairs (En-Fr, En-De, and En-Es) shows that the proposed approach significantly outperforms the baseline approach and the improvement is more significant when more relevant sentence pairs were retrieved.

Original languageEnglish (US)
Title of host publication32nd AAAI Conference on Artificial Intelligence, AAAI 2018
PublisherAAAI press
Pages5133-5140
Number of pages8
ISBN (Electronic)9781577358008
StatePublished - 2018
Event32nd AAAI Conference on Artificial Intelligence, AAAI 2018 - New Orleans, United States
Duration: Feb 2 2018Feb 7 2018

Publication series

Name32nd AAAI Conference on Artificial Intelligence, AAAI 2018

Other

Other32nd AAAI Conference on Artificial Intelligence, AAAI 2018
CountryUnited States
CityNew Orleans
Period2/2/182/7/18

ASJC Scopus subject areas

  • Artificial Intelligence

Fingerprint Dive into the research topics of 'Search engine guided neural machine translation'. Together they form a unique fingerprint.

  • Cite this

    Gu, J., Wang, Y., Cho, K., & Li, V. O. K. (2018). Search engine guided neural machine translation. In 32nd AAAI Conference on Artificial Intelligence, AAAI 2018 (pp. 5133-5140). (32nd AAAI Conference on Artificial Intelligence, AAAI 2018). AAAI press.