Abstract
Inspired by previous work on emergent communication in referential games, we propose a novel multi-modal, multi-step referential game, where the sender and receiver have access to distinct modalities of an object, and their information exchange is bidirectional and of arbitrary duration. The multi-modal multi-step setting allows agents to develop an internal communication significantly closer to natural language, in that they share a single set of messages, and that the length of the conversation may vary according to the difficulty of the task. We examine these properties empirically using a dataset consisting of images and textual descriptions of mammals, where the agents are tasked with identifying the correct object. Our experiments indicate that a robust and efficient communication protocol emerges, where gradual information exchange informs better predictions and higher communication bandwidth improves generalization.
Original language | English (US) |
---|---|
State | Published - Jan 1 2018 |
Event | 6th International Conference on Learning Representations, ICLR 2018 - Vancouver, Canada Duration: Apr 30 2018 → May 3 2018 |
Conference
Conference | 6th International Conference on Learning Representations, ICLR 2018 |
---|---|
Country/Territory | Canada |
City | Vancouver |
Period | 4/30/18 → 5/3/18 |
ASJC Scopus subject areas
- Language and Linguistics
- Education
- Computer Science Applications
- Linguistics and Language