TY - GEN
T1 - Active physical inference via reinforcement learning
AU - Li, Shuaiji
AU - Sun, Yu
AU - Liu, Sijia
AU - Wang, Tianyu
AU - Gureckis, Todd M.
AU - Bramley, Neil R.
N1 - Funding Information:
model-freereinforcementlearning,deepfunctionapproxima-activelyinferringthepropertiesofphysicalobjects.Weused Acknowledgments tion, and simulation based inference to build an agent able to This research was supported by NSF grant BCS-1255538, efficiently reveal the latent physical properties in human-like the John Templeton Foundation “Varieties of Understanding” ways without external input. Part of the insight gleaned from project, a John S. McDonnell Foundation Scholar Award to this project comes from our solutions to the engineering chal-TMG, and the Moore-Sloan Data Science Environment at lenges involved in creating a successful agent. To produce ex-NYU to Neil Bramley. Thanks also to Tim Wu who helped tended actions with richness and qualitative correspondence with video coding. 2131 with humans’, we found success with an action space that combined a discrete set of target locations with a discrete set of smoothing splines. We found that learning to associate action sequences with successful resolution of uncertainty was much more effective with a recurrent network architecture, but that robust strategies could be learned through model-free Q-learning. Following the predicted optimal control policies, not only did the agent uncover the latent parameters of interest, but there were also hints of behavioral correspondence with human subjects in that our mass trained agent would select more jagged and dynamic trajectories aligned with the strategies observed in Bramley et al. (2018).
Funding Information:
This research was supported by NSF grant BCS-1255538, the John Templeton Foundation “Varieties of Understanding” project, a John S. McDonnell Foundation Scholar Award to TMG, and the Moore-Sloan Data Science Environment at NYU to Neil Bramley. Thanks also to Tim Wu who helped with video coding.
Publisher Copyright:
© Cognitive Science Society: Creativity + Cognition + Computation, CogSci 2019.All rights reserved.
PY - 2019
Y1 - 2019
N2 - When encountering unfamiliar physical objects, children and adults often perform structured interrogatory actions such as grasping and prodding, so revealing latent physical properties such as masses and textures. However, the processes driving and supporting these curious behaviors are still largely mysterious. In this paper, we develop and train an agent able to actively uncover latent physical properties such as the mass and force of objects in a simulated physical “micro-world'. Concretely, we used a simulation-based-inference framework to quantify the physical information produced by observation and interaction with the evolving dynamic environment. We used model-free reinforcement learning algorithm to train an agent to implement general strategies for revealing latent physical properties. We compare the behaviors of this agent to the human behaviors observed in a similar task.
AB - When encountering unfamiliar physical objects, children and adults often perform structured interrogatory actions such as grasping and prodding, so revealing latent physical properties such as masses and textures. However, the processes driving and supporting these curious behaviors are still largely mysterious. In this paper, we develop and train an agent able to actively uncover latent physical properties such as the mass and force of objects in a simulated physical “micro-world'. Concretely, we used a simulation-based-inference framework to quantify the physical information produced by observation and interaction with the evolving dynamic environment. We used model-free reinforcement learning algorithm to train an agent to implement general strategies for revealing latent physical properties. We compare the behaviors of this agent to the human behaviors observed in a similar task.
KW - active learning
KW - physical simulation
KW - probabilistic inference
KW - reinforcement learning
UR - http://www.scopus.com/inward/record.url?scp=85139323019&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85139323019&partnerID=8YFLogxK
M3 - Conference contribution
AN - SCOPUS:85139323019
T3 - Proceedings of the 41st Annual Meeting of the Cognitive Science Society: Creativity + Cognition + Computation, CogSci 2019
SP - 2126
EP - 2132
BT - Proceedings of the 41st Annual Meeting of the Cognitive Science Society
PB - The Cognitive Science Society
T2 - 41st Annual Meeting of the Cognitive Science Society: Creativity + Cognition + Computation, CogSci 2019
Y2 - 24 July 2019 through 27 July 2019
ER -