Computing the stereo matching cost with a convolutional neural network

Jure Žbontar; Yann Le Cun

doi:10.1109/CVPR.2015.7298767

Computing the stereo matching cost with a convolutional neural network

Jure Žbontar, Yann Le Cun

Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

We present a method for extracting depth information from a rectified image pair. We train a convolutional neural network to predict how well two image patches match and use it to compute the stereo matching cost. The cost is refined by cross-based cost aggregation and semiglobal matching, followed by a left-right consistency check to eliminate errors in the occluded regions. Our stereo method achieves an error rate of 2.61% on the KITTI stereo dataset and is currently (August 2014) the top performing method on this dataset.

Original language	English (US)
Title of host publication	IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015
Publisher	IEEE Computer Society
Pages	1592-1599
Number of pages	8
ISBN (Electronic)	9781467369640
DOIs	https://doi.org/10.1109/CVPR.2015.7298767
State	Published - Oct 14 2015
Event	IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015 - Boston, United States Duration: Jun 7 2015 → Jun 12 2015

Publication series

Name	Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
Volume	07-12-June-2015
ISSN (Print)	1063-6919

Other

Other	IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015
Country/Territory	United States
City	Boston
Period	6/7/15 → 6/12/15

ASJC Scopus subject areas

Software
Computer Vision and Pattern Recognition

Access to Document

10.1109/CVPR.2015.7298767

Cite this

Žbontar, J., & Le Cun, Y. (2015). Computing the stereo matching cost with a convolutional neural network. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015 (pp. 1592-1599). Article 7298767 (Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition; Vol. 07-12-June-2015). IEEE Computer Society. https://doi.org/10.1109/CVPR.2015.7298767

Computing the stereo matching cost with a convolutional neural network. / Žbontar, Jure; Le Cun, Yann.
IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015. IEEE Computer Society, 2015. p. 1592-1599 7298767 (Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition; Vol. 07-12-June-2015).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Žbontar, J & Le Cun, Y 2015, Computing the stereo matching cost with a convolutional neural network. in IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015., 7298767, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 07-12-June-2015, IEEE Computer Society, pp. 1592-1599, IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015, Boston, United States, 6/7/15. https://doi.org/10.1109/CVPR.2015.7298767

@inproceedings{78194c5903aa418883e384b90161b4c6,

title = "Computing the stereo matching cost with a convolutional neural network",

abstract = "We present a method for extracting depth information from a rectified image pair. We train a convolutional neural network to predict how well two image patches match and use it to compute the stereo matching cost. The cost is refined by cross-based cost aggregation and semiglobal matching, followed by a left-right consistency check to eliminate errors in the occluded regions. Our stereo method achieves an error rate of 2.61% on the KITTI stereo dataset and is currently (August 2014) the top performing method on this dataset.",

author = "Jure {\v Z}bontar and {Le Cun}, Yann",

note = "Publisher Copyright: {\textcopyright} 2015 IEEE.; IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015 ; Conference date: 07-06-2015 Through 12-06-2015",

year = "2015",

month = oct,

day = "14",

doi = "10.1109/CVPR.2015.7298767",

language = "English (US)",

series = "Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition",

publisher = "IEEE Computer Society",

pages = "1592--1599",

booktitle = "IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015",

}

TY - GEN

T1 - Computing the stereo matching cost with a convolutional neural network

AU - Žbontar, Jure

AU - Le Cun, Yann

PY - 2015/10/14

Y1 - 2015/10/14

N2 - We present a method for extracting depth information from a rectified image pair. We train a convolutional neural network to predict how well two image patches match and use it to compute the stereo matching cost. The cost is refined by cross-based cost aggregation and semiglobal matching, followed by a left-right consistency check to eliminate errors in the occluded regions. Our stereo method achieves an error rate of 2.61% on the KITTI stereo dataset and is currently (August 2014) the top performing method on this dataset.

AB - We present a method for extracting depth information from a rectified image pair. We train a convolutional neural network to predict how well two image patches match and use it to compute the stereo matching cost. The cost is refined by cross-based cost aggregation and semiglobal matching, followed by a left-right consistency check to eliminate errors in the occluded regions. Our stereo method achieves an error rate of 2.61% on the KITTI stereo dataset and is currently (August 2014) the top performing method on this dataset.

UR - http://www.scopus.com/inward/record.url?scp=84952674245&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84952674245&partnerID=8YFLogxK

U2 - 10.1109/CVPR.2015.7298767

DO - 10.1109/CVPR.2015.7298767

M3 - Conference contribution

AN - SCOPUS:84952674245

T3 - Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition

SP - 1592

EP - 1599

BT - IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015

PB - IEEE Computer Society

T2 - IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015

Y2 - 7 June 2015 through 12 June 2015

ER -

Computing the stereo matching cost with a convolutional neural network

Abstract

Publication series

Other

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this