A Causal Model Approach to Dynamic Control

Acting effectively in the world requires learning and controlling dynamic systems, that is, systems involving feedback relations among continuous variables that vary in real time. We introduce a novel class of dynamic control environments using Ornstein-Uhlenbeck processes connected in causal Markov graphs that allow us to systematically test people's ability to learn and control various dynamic systems. We find that performance varied across a range of test environments, roughly matching with complexity defined by a set of models trained on the task (an optimal model, a deep Reinforcement Learning agent, and a PID controller). The testbed of dynamic environments and class of models introduced in this paper lay the groundwork for the systematic study of people's ability to control complex dynamic systems.