Dynamic neural turing machine with continuous and discrete addressing schemes

We extend the neural Turing machine (NTM) model into a dynamic neural Turing machine (D-NTM) by introducing trainable address vectors. This addressing scheme maintains for each memory cell twoseparate vectors, content and address vectors. This allows the D-NTM to learn a wide variety of location-based addressing strategies, including both linear and nonlinear ones.We implement theD-NTMwith both continuous and discrete read and write mechanisms.We investigate the mechanisms and effects of learning to read andwrite into a memory through experiments on Facebook bAbI tasks using both a feedforward and GRU controller. We provide extensive analysis of ourmodel and compare different variations of neural Turing machines on this task. We show that our model outperforms long short-term memory and NTM variants. We provide further experimental results on the sequential pMNIST, Stanford Natural Language Inference, associative recall, and copy tasks.