TY - JOUR
T1 - Mutual exclusivity as a challenge for deep neural networks
AU - Gandhi, Kanishk
AU - Lake, Brenden
N1 - Funding Information:
We are grateful to Marco Baroni, Tal Linzen, Nicholas Tomlin, Emin Orhan, Ethan Perez, and Wai Keen Vong for helpful comments and discussions. Through B. Lake’s position at NYU, this work was partially supported by NSF under the NSF Award 1922658 NRT-HDR: FUTURE Foundations, Translation, and Responsibility for Data Science.
Publisher Copyright:
© 2020 Neural information processing systems foundation. All rights reserved.
PY - 2020
Y1 - 2020
N2 - Strong inductive biases allow children to learn in fast and adaptable ways. Children use the mutual exclusivity (ME) bias to help disambiguate how words map to referents, assuming that if an object has one label then it does not need another. In this paper, we investigate whether or not vanilla neural architectures have an ME bias, demonstrating that they lack this learning assumption. Moreover, we show that their inductive biases are poorly matched to lifelong learning formulations of classification and translation. We demonstrate that there is a compelling case for designing task-general neural networks that learn through mutual exclusivity, which remains an open challenge.
AB - Strong inductive biases allow children to learn in fast and adaptable ways. Children use the mutual exclusivity (ME) bias to help disambiguate how words map to referents, assuming that if an object has one label then it does not need another. In this paper, we investigate whether or not vanilla neural architectures have an ME bias, demonstrating that they lack this learning assumption. Moreover, we show that their inductive biases are poorly matched to lifelong learning formulations of classification and translation. We demonstrate that there is a compelling case for designing task-general neural networks that learn through mutual exclusivity, which remains an open challenge.
UR - http://www.scopus.com/inward/record.url?scp=85108128433&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85108128433&partnerID=8YFLogxK
M3 - Conference article
AN - SCOPUS:85108128433
SN - 1049-5258
VL - 2020-December
JO - Advances in Neural Information Processing Systems
JF - Advances in Neural Information Processing Systems
T2 - 34th Conference on Neural Information Processing Systems, NeurIPS 2020
Y2 - 6 December 2020 through 12 December 2020
ER -