Evolving memory cell structures for sequence learning

Justin Bayer, Daan Wierstra, Julian Togelius, Jürgen Schmidhuber

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Abstract

    Long Short-Term Memory (LSTM) is one of the best recent supervised sequence learning methods. Using gradient descent, it trains memory cells represented as differentiable computational graph structures. Interestingly, LSTM's cell structure seems somewhat arbitrary. In this paper we optimize its computational structure using a multi-objective evolutionary algorithm. The fitness function reflects the structure's usefulness for learning various formal languages. The evolved cells help to understand crucial features that aid sequence learning.

    Original languageEnglish (US)
    Title of host publicationArtificial Neural Networks - ICANN 2009 - 19th International Conference, Proceedings
    Pages755-764
    Number of pages10
    EditionPART 2
    DOIs
    StatePublished - 2009
    Event19th International Conference on Artificial Neural Networks, ICANN 2009 - Limassol, Cyprus
    Duration: Sep 14 2009Sep 17 2009

    Publication series

    NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
    NumberPART 2
    Volume5769 LNCS
    ISSN (Print)0302-9743
    ISSN (Electronic)1611-3349

    Other

    Other19th International Conference on Artificial Neural Networks, ICANN 2009
    CountryCyprus
    CityLimassol
    Period9/14/099/17/09

    ASJC Scopus subject areas

    • Theoretical Computer Science
    • Computer Science(all)

    Fingerprint Dive into the research topics of 'Evolving memory cell structures for sequence learning'. Together they form a unique fingerprint.

    Cite this