TY - JOUR

T1 - Aleatoric uncertainty estimation for dense stereo matching via CNN-based cost volume analysis

AU - Mehltretter, Max

AU - Heipke, Christian

N1 - Funding Information: This work was supported by the German Research Foundation (DFG) as a part of the Research Training Group i.c.sens [ GRK2159 ], the MOBILISE initiative of the Leibniz University Hannover and TU Braunschweig, Germany and by the NVIDIA Corporation with the donation of the Titan V GPU used for this research.

PY - 2021/1

Y1 - 2021/1

N2 - Motivated by the need to identify erroneous disparity estimates, various methods for the estimation of aleatoric uncertainty in the context of dense stereo matching have been presented in recent years. Especially, the introduction of deep learning based methods and the accompanying significant improvement in accuracy have greatly increased the popularity of this field. Despite this remarkable development, most of these methods rely on features learned from disparity maps only, neglecting the corresponding 3-dimensional cost volumes. However, conventional hand-crafted methods have already demonstrated that the additional information contained in such cost volumes are beneficial for the task of uncertainty estimation. In this paper, we combine the advantages of deep learning and cost volume based features and present a new Convolutional Neural Network (CNN) architecture to directly learn features for the task of aleatoric uncertainty estimation from volumetric 3D data. Furthermore, we discuss and apply three different uncertainty models to train our CNN without the need to provide ground truth for uncertainty. In an extensive evaluation on three datasets using three common dense stereo matching methods, we investigate the effects of these uncertainty models and demonstrate the generality and state-of-the-art accuracy of the proposed method.

AB - Motivated by the need to identify erroneous disparity estimates, various methods for the estimation of aleatoric uncertainty in the context of dense stereo matching have been presented in recent years. Especially, the introduction of deep learning based methods and the accompanying significant improvement in accuracy have greatly increased the popularity of this field. Despite this remarkable development, most of these methods rely on features learned from disparity maps only, neglecting the corresponding 3-dimensional cost volumes. However, conventional hand-crafted methods have already demonstrated that the additional information contained in such cost volumes are beneficial for the task of uncertainty estimation. In this paper, we combine the advantages of deep learning and cost volume based features and present a new Convolutional Neural Network (CNN) architecture to directly learn features for the task of aleatoric uncertainty estimation from volumetric 3D data. Furthermore, we discuss and apply three different uncertainty models to train our CNN without the need to provide ground truth for uncertainty. In an extensive evaluation on three datasets using three common dense stereo matching methods, we investigate the effects of these uncertainty models and demonstrate the generality and state-of-the-art accuracy of the proposed method.

KW - Confidence

KW - Deep learning

KW - Depth reconstruction

KW - Uncertainty quantification

UR - http://www.scopus.com/inward/record.url?scp=85096686559&partnerID=8YFLogxK

U2 - 10.1016/j.isprsjprs.2020.11.003

DO - 10.1016/j.isprsjprs.2020.11.003

M3 - Article

AN - SCOPUS:85096686559

VL - 171

SP - 63

EP - 75

JO - ISPRS Journal of Photogrammetry and Remote Sensing

JF - ISPRS Journal of Photogrammetry and Remote Sensing

SN - 0924-2716

ER -