Tightly coupled semantic RGB-D inertial odometry for accurate long-term localization and mapping

In this paper, we utilize semantically enhanced feature matching and visual inertial bundle adjustment to improve the robustness of odometry especially in feature-sparse environments. A novel semantically enhanced feature matching algorithm is developed for robust: 1) medium and long-term tracking, and 2) loop-closing. Additionally, a semantic visual inertial bundle adjustment algorithm is introduced to robustly estimate pose in presence of ambiguous correspondences or in feature sparse environment. Our tightly coupled semantic RGB-D odometry approach is demonstrated on a real world indoor dataset collected using our unmanned ground vehicle (UGV). Our approach improves traditional visual odometry relying on low-level geometric features like corners, points, and planes for localization and mapping. Additionally, prior approaches are limited due to their sensitivity to scene geometry and changes in light intensity. The semantic inertial odometry is especially important to significantly reduce drifts in longer intervals.