TY - CHAP
T1 - Motor learning at intermediate reynolds number
T2 - Experiments with policy gradient on the flapping flight of a rigid wing
AU - Roberts, John W.
AU - Moret, Lionel
AU - Zhang, Jun
AU - Tedrake, Russ
PY - 2010
Y1 - 2010
N2 - This work describes the development of a model-free reinforcement learning-based control methodology for the heaving plate, a laboratory experimental fluid system that serves as a model of flapping flight. Through an optimized policy gradient algorithm, we were able to demonstrate rapid convergence (requiring less than 10 minutes of experiments) to a stroke form which maximized the propulsive efficiency of this very complicated fluid-dynamical system. This success was due in part to an improved sampling distribution and carefully selected policy parameterization, both motivated by a formal analysis of the signal-to-noise ratio of policy gradient algorithms. The resulting optimal policy provides insight into the behavior of the fluid system, and the effectiveness of the learning strategy suggests a number of exciting opportunities for machine learning control of fluid dynamics.
AB - This work describes the development of a model-free reinforcement learning-based control methodology for the heaving plate, a laboratory experimental fluid system that serves as a model of flapping flight. Through an optimized policy gradient algorithm, we were able to demonstrate rapid convergence (requiring less than 10 minutes of experiments) to a stroke form which maximized the propulsive efficiency of this very complicated fluid-dynamical system. This success was due in part to an improved sampling distribution and carefully selected policy parameterization, both motivated by a formal analysis of the signal-to-noise ratio of policy gradient algorithms. The resulting optimal policy provides insight into the behavior of the fluid system, and the effectiveness of the learning strategy suggests a number of exciting opportunities for machine learning control of fluid dynamics.
UR - http://www.scopus.com/inward/record.url?scp=74049113741&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=74049113741&partnerID=8YFLogxK
U2 - 10.1007/978-3-642-05181-4_13
DO - 10.1007/978-3-642-05181-4_13
M3 - Chapter (peer-reviewed)
AN - SCOPUS:74049113741
SN - 9783642051807
T3 - Studies in Computational Intelligence
SP - 293
EP - 309
BT - From motor learning to interaction learning in robots
A2 - Sigaud, Oliver
A2 - Peters, Jan
A2 - Peters, Jan
ER -