Sensor modality fusion with CNNs for UGV autonomous driving in indoor environments

Naman Patel, Anna Choromanska, Prashanth Krishnamurthy, Farshad Khorrami

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We present a novel end-to-end learning framework to enable ground vehicles to autonomously navigate unknown environments by fusing raw pixels from cameras and depth measurements from a LiDAR. A deep neural network architecture is introduced to effectively perform modality fusion and reliably predict steering commands even in the presence of sensor failures. The proposed network is trained on our own dataset, from LiDAR and a camera mounted on a UGV taken in an indoor corridor environment. Comprehensive experimental evaluation to demonstrate the robustness of our network architecture is performed to show that the proposed deep learning neural network is able to autonomously navigate in the corridor environment. Furthermore, we demonstrate that the fusion of the camera and LiDAR modalities provides further benefits beyond robustness to sensor failures. Specifically, the multimodal fused system shows a potential to navigate around static and dynamic obstacles and to handle changes in environment geometry without being trained for these tasks.

Original languageEnglish (US)
Title of host publicationIROS 2017 - IEEE/RSJ International Conference on Intelligent Robots and Systems
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1531-1536
Number of pages6
ISBN (Electronic)9781538626825
DOIs
StatePublished - Dec 13 2017
Event2017 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2017 - Vancouver, Canada
Duration: Sep 24 2017Sep 28 2017

Publication series

NameIEEE International Conference on Intelligent Robots and Systems
Volume2017-September
ISSN (Print)2153-0858
ISSN (Electronic)2153-0866

Other

Other2017 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2017
CountryCanada
CityVancouver
Period9/24/179/28/17

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Software
  • Computer Vision and Pattern Recognition
  • Computer Science Applications

Fingerprint Dive into the research topics of 'Sensor modality fusion with CNNs for UGV autonomous driving in indoor environments'. Together they form a unique fingerprint.

Cite this