Controlling a robotic fish via a natural user interface for informal science education

Paul Phamduy, Mauro Debellis, Maurizio Porfiri

Research output: Contribution to journalArticlepeer-review


Informal science education is a key contributing factor to scientific literacy, which determines our capacity as a society to technologically progress and make cognizant decisions on pressing issues of global scale. Both the fields of robotics and natural user interfaces have been separately proposed as effective means to aid informal science education, by increasing users' engagement through multiple interactive features. Here, we demonstrate the integration of these two fields of investigation toward a novel educational platform, revolving around the control of a robotic fish via a natural user interface. Users control the robotic fish through upper limb gestures that are captured by the Kinect. The robotic fish incorporates a temperature sensor, which collects data in a tank instrumented with heating and cooling sources. Participants observe the measurements they are recording in real-time to map the environment. Self-reported post-activity surveys and behavioral coding data on young users were collected to assess their level of engagement in the activity and their perception of the system. Our results indicate that the robotic fish is intuitive to drive with the natural user interface, the activity of collecting water temperature is interesting, and robotics may be a viable and accessible career option.

Original languageEnglish (US)
Article number7272111
Pages (from-to)2328-2337
Number of pages10
JournalIEEE Transactions on Multimedia
Issue number12
StatePublished - Dec 2015


  • Biologically-inspired robots
  • Kinect
  • educational robotics
  • natural user interfaces (NUIs)

ASJC Scopus subject areas

  • Signal Processing
  • Media Technology
  • Computer Science Applications
  • Electrical and Electronic Engineering


Dive into the research topics of 'Controlling a robotic fish via a natural user interface for informal science education'. Together they form a unique fingerprint.

Cite this