Sequential attention: A context-aware alignment function for machine reading

Sebastian Brarda, Philip Yeres, Samuel R. Bowman

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Abstract

    In this paper we propose a neural network model with a novel Sequential Attention layer that extends soft attention by assigning weights to words in an input sequence in a way that takes into account not just how well that word matches a query, but how well surrounding words match. We evaluate this approach on the task of reading comprehension (on the Who did What and CNN datasets) and show that it dramatically improves a strong baseline-the Stanford Reader-and is competitive with the state of the art.

    Original languageEnglish (US)
    Title of host publicationProceedings of the 2nd Workshop on Representation Learning for NLP, Rep4NLP 2017 at the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017
    EditorsPhil Blunsom, Antoine Bordes, Kyunghyun Cho, Shay Cohen, Chris Dyer, Edward Grefenstette, Karl MoritzHermann, Laura Rimell, Jason Weston, Scott Yih
    PublisherAssociation for Computational Linguistics (ACL)
    Pages75-80
    Number of pages6
    ISBN (Electronic)9781945626623
    StatePublished - 2017
    Event2nd Workshop on Representation Learning for NLP, Rep4NLP 2017 at the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017 - Vancouver, Canada
    Duration: Aug 3 2017 → …

    Publication series

    NameProceedings of the 2nd Workshop on Representation Learning for NLP, Rep4NLP 2017 at the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017

    Conference

    Conference2nd Workshop on Representation Learning for NLP, Rep4NLP 2017 at the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017
    Country/TerritoryCanada
    CityVancouver
    Period8/3/17 → …

    ASJC Scopus subject areas

    • Computational Theory and Mathematics
    • Software

    Fingerprint

    Dive into the research topics of 'Sequential attention: A context-aware alignment function for machine reading'. Together they form a unique fingerprint.

    Cite this