Evolving memory cell structures for sequence learning

Justin Bayer, Daan Wierstra, Julian Togelius, Jürgen Schmidhuber

    Research output: Chapter in Book/Report/Conference proceedingConference contribution


    Long Short-Term Memory (LSTM) is one of the best recent supervised sequence learning methods. Using gradient descent, it trains memory cells represented as differentiable computational graph structures. Interestingly, LSTM's cell structure seems somewhat arbitrary. In this paper we optimize its computational structure using a multi-objective evolutionary algorithm. The fitness function reflects the structure's usefulness for learning various formal languages. The evolved cells help to understand crucial features that aid sequence learning.

    Original languageEnglish (US)
    Title of host publicationArtificial Neural Networks - ICANN 2009 - 19th International Conference, Proceedings
    Number of pages10
    EditionPART 2
    StatePublished - 2009
    Event19th International Conference on Artificial Neural Networks, ICANN 2009 - Limassol, Cyprus
    Duration: Sep 14 2009Sep 17 2009

    Publication series

    NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
    NumberPART 2
    Volume5769 LNCS
    ISSN (Print)0302-9743
    ISSN (Electronic)1611-3349


    Other19th International Conference on Artificial Neural Networks, ICANN 2009

    ASJC Scopus subject areas

    • Theoretical Computer Science
    • General Computer Science


    Dive into the research topics of 'Evolving memory cell structures for sequence learning'. Together they form a unique fingerprint.

    Cite this