Towards teleoperation-based interactive learning of robot kinematics using a mobile augmented reality interface on a tablet

Jared A. Frank, Vikram Kapila

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The integration of augmented reality (AR) techniques in user interface design has enhanced interactive experiences in teleoperation of robots, hands-on learning in classrooms, laboratory, and special education, and user training in an array of fields, e.g., aerospace, automotive, construction, manufacturing, medical, etc. However, AR-based user interfaces that command machines and tools have not been fully explored for their potential to enhance interactive learning of engineering concepts in the laboratory. This paper outlines the development of a mobile application executing on a tablet device, which renders an immersive AR-based graphical user interface to enable users to monitor, interact with, and control a four-link underactuated planar robot. Computer vision routines are used to extract real-time, vision-based measurements of the robot's joint angles and end effector location from the live video captured by the rear-facing camera on the tablet. The obtained measurements are used to render AR content to offer users with additional visual feedback. Touch gesture recognition is implemented to allow users to naturally and intuitively command the robot by tapping and dragging their fingers at desired locations on the tablet screen. Experimental results show the performance and efficacy of the proposed system as it is operated in two different modes: one in which the user has direct control over the angles of the actuated links of the robot and one in which the user has direct control over the end effector location.

Original languageEnglish (US)
Title of host publication2016 Indian Control Conference, ICC 2016 - Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages385-392
Number of pages8
ISBN (Electronic)9781467379939
DOIs
StatePublished - Mar 24 2016
Event2nd Indian Control Conference, ICC 2016 - Hyderabad, India
Duration: Jan 4 2016Jan 6 2016

Publication series

Name2016 Indian Control Conference, ICC 2016 - Proceedings

Other

Other2nd Indian Control Conference, ICC 2016
CountryIndia
CityHyderabad
Period1/4/161/6/16

ASJC Scopus subject areas

  • Control and Systems Engineering

Fingerprint Dive into the research topics of 'Towards teleoperation-based interactive learning of robot kinematics using a mobile augmented reality interface on a tablet'. Together they form a unique fingerprint.

Cite this