Cross-Safe: A Computer Vision-Based Approach to Make All Intersection-Related Pedestrian Signals Accessible for the Visually Impaired

Xiang Li, Hanzhang Cui, John Ross Rizzo, Edward Wong, Yi Fang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Intersections pose great challenges to blind or visually impaired travelers who aim to cross roads safely and efficiently given unpredictable traffic control. Due to decreases in vision and increasingly difficult odds when planning and negotiating dynamic environments, visually impaired travelers require devices and/or assistance (i.e. cane, talking signals) to successfully execute intersection navigation. The proposed research project is to develop a novel computer vision-based approach, named Cross-Safe, that provides accurate and accessible guidance to the visually impaired as one crosses intersections, as part of a larger unified smart wearable device. As a first step, we focused on the red-light-green-light, go-no-go problem, as accessible pedestrian signals are drastically missing from urban infrastructure in New York City. Cross-Safe leverages state-of-the-art deep learning techniques for real-time pedestrian signal detection and recognition. A portable GPU unit, the Nvidia Jetson TX2, provides mobile visual computing and a cognitive assistant provides accurate voice-based guidance. More specifically, a lighter recognition algorithm was developed and equipped for Cross-Safe, enabling robust walking signal sign detection and signal recognition. Recognized signals are conveyed to visually impaired end user by vocal guidance, providing critical information for real-time intersection navigation. Cross-Safe is also able to balance portability, recognition accuracy, computing efficiency and power consumption. A custom image library was built and developed to train, validate, and test our methodology on real traffic intersections, demonstrating the feasibility of Cross-Safe in providing safe guidance to the visually impaired at urban intersections. Subsequently, experimental results show robust preliminary findings of our detection and recognition algorithm.

Original languageEnglish (US)
Title of host publicationAdvances in Computer Vision - Proceedings of the 2019 Computer Vision Conference CVC
EditorsKohei Arai, Supriya Kapoor
PublisherSpringer Verlag
Pages132-146
Number of pages15
ISBN (Print)9783030177973
DOIs
StatePublished - Jan 1 2020
EventComputer Vision Conference, CVC 2019 - Las Vegas, United States
Duration: Apr 25 2019Apr 26 2019

Publication series

NameAdvances in Intelligent Systems and Computing
Volume944
ISSN (Print)2194-5357
ISSN (Electronic)2194-5365

Conference

ConferenceComputer Vision Conference, CVC 2019
CountryUnited States
CityLas Vegas
Period4/25/194/26/19

Keywords

  • Assistive technology
  • Pedestrian safety
  • Portable device
  • Visual impairment

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Computer Science(all)

Fingerprint Dive into the research topics of 'Cross-Safe: A Computer Vision-Based Approach to Make All Intersection-Related Pedestrian Signals Accessible for the Visually Impaired'. Together they form a unique fingerprint.

  • Cite this

    Li, X., Cui, H., Rizzo, J. R., Wong, E., & Fang, Y. (2020). Cross-Safe: A Computer Vision-Based Approach to Make All Intersection-Related Pedestrian Signals Accessible for the Visually Impaired. In K. Arai, & S. Kapoor (Eds.), Advances in Computer Vision - Proceedings of the 2019 Computer Vision Conference CVC (pp. 132-146). (Advances in Intelligent Systems and Computing; Vol. 944). Springer Verlag. https://doi.org/10.1007/978-3-030-17798-0_13