Motor learning at intermediate reynolds number: Experiments with policy gradient on the flapping flight of a rigid wing

John W. Roberts, Lionel Moret, Jun Zhang, Russ Tedrake

Research output: Chapter in Book/Report/Conference proceedingChapter (peer-reviewed)

Abstract

This work describes the development of a model-free reinforcement learning-based control methodology for the heaving plate, a laboratory experimental fluid system that serves as a model of flapping flight. Through an optimized policy gradient algorithm, we were able to demonstrate rapid convergence (requiring less than 10 minutes of experiments) to a stroke form which maximized the propulsive efficiency of this very complicated fluid-dynamical system. This success was due in part to an improved sampling distribution and carefully selected policy parameterization, both motivated by a formal analysis of the signal-to-noise ratio of policy gradient algorithms. The resulting optimal policy provides insight into the behavior of the fluid system, and the effectiveness of the learning strategy suggests a number of exciting opportunities for machine learning control of fluid dynamics.

Original languageEnglish (US)
Title of host publicationFrom motor learning to interaction learning in robots
EditorsOliver Sigaud, Jan Peters, Jan Peters
Pages293-309
Number of pages17
DOIs
StatePublished - 2010

Publication series

NameStudies in Computational Intelligence
Volume264
ISSN (Print)1860-949X

ASJC Scopus subject areas

  • Artificial Intelligence

Fingerprint Dive into the research topics of 'Motor learning at intermediate reynolds number: Experiments with policy gradient on the flapping flight of a rigid wing'. Together they form a unique fingerprint.

  • Cite this

    Roberts, J. W., Moret, L., Zhang, J., & Tedrake, R. (2010). Motor learning at intermediate reynolds number: Experiments with policy gradient on the flapping flight of a rigid wing. In O. Sigaud, J. Peters, & J. Peters (Eds.), From motor learning to interaction learning in robots (pp. 293-309). (Studies in Computational Intelligence; Vol. 264). https://doi.org/10.1007/978-3-642-05181-4_13