Acquiring Abstract Visual Knowledge of the Real-World Environment for Autonomous Vehicles

Ibrahim F.J. Ghalyan, Vikram Kapila

Research output: Chapter in Book/Report/Conference proceedingConference contribution


This paper considers the problem of modeling the surrounding environment of a driven car by using the images captured by a dash cam during the driving process. Inspired from a human driver's interpretation of the car's surrounding environment, an abstract representation of the environment is developed that can facilitate in decision-making to prevent the car's collisions with surrounding objects. The proposed technique for modeling the car's surrounding environment utilizes the dash cam to capture images as the car is driven facing multiple situations and obstacles. By relying on the human driver's interpretation of various driving scenarios, the images of the car's surrounding environment are manually grouped into classes that reflect the driver's abstract knowledge. Grouping the images allows the formulation of knowledge transfer process from the human driver to an autonomous vehicle as a classification problem, producing a meaningful and efficient representation of models arising from real-world scenarios. The framework of convolutional neural networks (CNN) is employed to model the surrounding environment of the driven car, encapsulating the abstract knowledge of the human driver. The proposed modeling approach is applied to determine its efficacy in two experimental scenarios. In the first experiment, a highway driving scenario is considered with three classes. Alternatively, in the second experiment, a scenario of driving in a residential area is addressed with six classes. Excellent modeling performance is reported for both experiments. Comparisons conducted with alternative image classification techniques reveal the superiority of the CNN for modeling the considered driving scenarios.

Original languageEnglish (US)
Title of host publication2018 IEEE Applied Imagery Pattern Recognition Workshop, AIPR 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781538693063
StatePublished - Jul 2 2018
Event2018 IEEE Applied Imagery Pattern Recognition Workshop, AIPR 2018 - Washington, United States
Duration: Oct 9 2018Oct 11 2018

Publication series

NameProceedings - Applied Imagery Pattern Recognition Workshop
ISSN (Print)2164-2516


Conference2018 IEEE Applied Imagery Pattern Recognition Workshop, AIPR 2018
Country/TerritoryUnited States

ASJC Scopus subject areas

  • General Engineering


Dive into the research topics of 'Acquiring Abstract Visual Knowledge of the Real-World Environment for Autonomous Vehicles'. Together they form a unique fingerprint.

Cite this