An augmented reality spatial referencing system for mobile robots

The deployment of a mobile service robot in domestic settings is a challenging task due to the dynamic and unstructured nature of such environments. Successful operation of the robot requires continuous human supervision to update its spatial knowledge about the dynamic environment. Thus, it is essential to develop a human-robot interaction (HRI) strategy that is suitable for novice end users to effortlessly provide task-specific spatial information to the robot. Although several approaches have been developed for this purpose, most of them are not feasible or convenient for use in domestic environments. In response, we have developed an augmented reality (AR) spatial referencing system (SRS), which allows a non-expert user to tag any specific locations on a physical surface to allocate tasks to be performed by the robot at those locations. Specifically, in the AR-SRS, the user provides a spatial reference by creating an AR virtual object with a semantic label. The real-world location of the user-created virtual object is estimated and stored as spatial data along with the user-specified semantic label. We present three different approaches to establish the correspondence between the user-created virtual object locations and the real-world coordinates on an a priori static map of the service area available to the robot. The performance of each approach is evaluated and reported. We also present use-case scenarios to demonstrate potential applications of the AR-SRS for mobile service robots.