Mixed-reality learning environments: Integrating mobile interfaces with laboratory test-beds

Jared A. Frank, Vikram Kapila

Research output: Contribution to journalArticlepeer-review


Even as mobile devices have become increasingly powerful and popular among learners and instructors alike, research involving their comprehensive integration into educational laboratory activities remains largely unexplored. This paper discusses efforts to integrate vision-based measurement and control, augmented reality (AR), and multi-touch interaction on mobile devices in the development of Mixed-Reality Learning Environments (MRLE) that enhance interactions with laboratory test-beds for science and engineering education. A learner points her device at a laboratory test-bed fitted with visual markers while a mobile application supplies a live view of the experiment augmented with interactive media that aid in the visualization of concepts and promote learner engagement. As the learner manipulates the augmented media, her gestures are mapped to commands that alter the behavior of the test-bed on the fly. Running in the background of the mobile application are algorithms performing vision-based estimation and wireless control of the test-bed. In this way, the sensing, storage, computation, and communication (SSCC) capabilities of mobile devices are leveraged to relieve the need for laboratory-grade equipment, improving the cost-effectiveness and portability of platforms to conduct hands-on laboratories. We hypothesize that students using the MRLE platform demonstrate improvement in their knowledge of dynamic systems and control concepts and have generally favorable experiences using the platform. To validate the hypotheses concerning the educational effectiveness and user experience of the MRLEs, an evaluation was conducted with two classes of undergraduate students using an illustrative platform incorporating a tablet computer and motor test-bed to teach concepts of dynamic systems and control. Results of the evaluation validate the hypotheses. The benefits and drawbacks of the MRLEs observed throughout the study are discussed with respect to the traditional hands-on, virtual, and remote laboratory formats.

Original languageEnglish (US)
Pages (from-to)88-104
Number of pages17
JournalComputers and Education
StatePublished - Jul 1 2017


  • Applications in subject areas
  • Architectures for educational technology system
  • Improving classroom teaching
  • Interactive learning environments
  • Virtual reality

ASJC Scopus subject areas

  • General Computer Science
  • Education


Dive into the research topics of 'Mixed-reality learning environments: Integrating mobile interfaces with laboratory test-beds'. Together they form a unique fingerprint.

Cite this