EnhancedNet, an End-to-End Network for Dense Disparity Estimation and its Application to Aerial Images

Publikation: Beitrag in FachzeitschriftArtikelForschungPeer-Review

Autorschaft

  • Junhua Kang
  • Lin Chen
  • Christian Heipke

Externe Organisationen

  • Chang'an University
  • VISCODA GmbH
Forschungs-netzwerk anzeigen

Details

OriginalspracheEnglisch
Seiten (von - bis)531-546
Seitenumfang16
FachzeitschriftPFG - Journal of Photogrammetry, Remote Sensing and Geoinformation Science
Jahrgang92
Ausgabenummer5
Frühes Online-Datum28 Aug. 2024
PublikationsstatusVeröffentlicht - Okt. 2024

Abstract

Recent developments in deep learning technology have boosted the performance of dense stereo reconstruction. However, the state-of-the-art deep learning-based stereo matching methods are mainly trained using close-range synthetic images. Consequently, the application of these methods in aerial photogrammetry and remote sensing is currently far from straightforward. In this paper, we propose a new disparity estimation network for stereo matching and investigate its generalization abilities in regard to aerial images. First, we propose an end-to-end deep learning network for stereo matching, regularized by disparity gradients, which includes a residual cost volume and a reconstruction error volume in a refinement module, and multiple losses. In order to investigate the influence of the multiple losses, a comprehensive analysis is presented. Second, based on this network trained with synthetic close-range data, we propose a new pipeline for matching high-resolution aerial imagery. The experimental results show that the proposed network improves the disparity accuracy by up to 40% in terms of errors larger than 1 px compared to results when not including the refinement network, especially in areas containing detailed small objects. In addition, in qualitative and quantitative experiments, we are able to show that our model, pre-trained on a synthetic stereo dataset, achieves very competitive sub-pixel geometric accuracy on aerial images. These results confirm that the domain gap between synthetic close-range and real aerial images can be satisfactorily bridged using the proposed new deep learning method for dense image matching.

ASJC Scopus Sachgebiete

Zitieren

EnhancedNet, an End-to-End Network for Dense Disparity Estimation and its Application to Aerial Images. / Kang, Junhua; Chen, Lin; Heipke, Christian.
in: PFG - Journal of Photogrammetry, Remote Sensing and Geoinformation Science, Jahrgang 92, Nr. 5, 10.2024, S. 531-546.

Publikation: Beitrag in FachzeitschriftArtikelForschungPeer-Review

Kang, J, Chen, L & Heipke, C 2024, 'EnhancedNet, an End-to-End Network for Dense Disparity Estimation and its Application to Aerial Images', PFG - Journal of Photogrammetry, Remote Sensing and Geoinformation Science, Jg. 92, Nr. 5, S. 531-546. https://doi.org/10.1007/s41064-024-00307-w
Kang, J., Chen, L., & Heipke, C. (2024). EnhancedNet, an End-to-End Network for Dense Disparity Estimation and its Application to Aerial Images. PFG - Journal of Photogrammetry, Remote Sensing and Geoinformation Science, 92(5), 531-546. https://doi.org/10.1007/s41064-024-00307-w
Kang J, Chen L, Heipke C. EnhancedNet, an End-to-End Network for Dense Disparity Estimation and its Application to Aerial Images. PFG - Journal of Photogrammetry, Remote Sensing and Geoinformation Science. 2024 Okt;92(5):531-546. Epub 2024 Aug 28. doi: 10.1007/s41064-024-00307-w
Kang, Junhua ; Chen, Lin ; Heipke, Christian. / EnhancedNet, an End-to-End Network for Dense Disparity Estimation and its Application to Aerial Images. in: PFG - Journal of Photogrammetry, Remote Sensing and Geoinformation Science. 2024 ; Jahrgang 92, Nr. 5. S. 531-546.
Download
@article{7099dc5800f4413d9b9cee28a41ad6fa,
title = "EnhancedNet, an End-to-End Network for Dense Disparity Estimation and its Application to Aerial Images",
abstract = "Recent developments in deep learning technology have boosted the performance of dense stereo reconstruction. However, the state-of-the-art deep learning-based stereo matching methods are mainly trained using close-range synthetic images. Consequently, the application of these methods in aerial photogrammetry and remote sensing is currently far from straightforward. In this paper, we propose a new disparity estimation network for stereo matching and investigate its generalization abilities in regard to aerial images. First, we propose an end-to-end deep learning network for stereo matching, regularized by disparity gradients, which includes a residual cost volume and a reconstruction error volume in a refinement module, and multiple losses. In order to investigate the influence of the multiple losses, a comprehensive analysis is presented. Second, based on this network trained with synthetic close-range data, we propose a new pipeline for matching high-resolution aerial imagery. The experimental results show that the proposed network improves the disparity accuracy by up to 40% in terms of errors larger than 1 px compared to results when not including the refinement network, especially in areas containing detailed small objects. In addition, in qualitative and quantitative experiments, we are able to show that our model, pre-trained on a synthetic stereo dataset, achieves very competitive sub-pixel geometric accuracy on aerial images. These results confirm that the domain gap between synthetic close-range and real aerial images can be satisfactorily bridged using the proposed new deep learning method for dense image matching.",
keywords = "Aerial images, Dense matching, Disparity learning, Disparity refinement, Geometric accuracy",
author = "Junhua Kang and Lin Chen and Christian Heipke",
note = "Publisher Copyright: {\textcopyright} The Author(s) 2024.",
year = "2024",
month = oct,
doi = "10.1007/s41064-024-00307-w",
language = "English",
volume = "92",
pages = "531--546",
number = "5",

}

Download

TY - JOUR

T1 - EnhancedNet, an End-to-End Network for Dense Disparity Estimation and its Application to Aerial Images

AU - Kang, Junhua

AU - Chen, Lin

AU - Heipke, Christian

N1 - Publisher Copyright: © The Author(s) 2024.

PY - 2024/10

Y1 - 2024/10

N2 - Recent developments in deep learning technology have boosted the performance of dense stereo reconstruction. However, the state-of-the-art deep learning-based stereo matching methods are mainly trained using close-range synthetic images. Consequently, the application of these methods in aerial photogrammetry and remote sensing is currently far from straightforward. In this paper, we propose a new disparity estimation network for stereo matching and investigate its generalization abilities in regard to aerial images. First, we propose an end-to-end deep learning network for stereo matching, regularized by disparity gradients, which includes a residual cost volume and a reconstruction error volume in a refinement module, and multiple losses. In order to investigate the influence of the multiple losses, a comprehensive analysis is presented. Second, based on this network trained with synthetic close-range data, we propose a new pipeline for matching high-resolution aerial imagery. The experimental results show that the proposed network improves the disparity accuracy by up to 40% in terms of errors larger than 1 px compared to results when not including the refinement network, especially in areas containing detailed small objects. In addition, in qualitative and quantitative experiments, we are able to show that our model, pre-trained on a synthetic stereo dataset, achieves very competitive sub-pixel geometric accuracy on aerial images. These results confirm that the domain gap between synthetic close-range and real aerial images can be satisfactorily bridged using the proposed new deep learning method for dense image matching.

AB - Recent developments in deep learning technology have boosted the performance of dense stereo reconstruction. However, the state-of-the-art deep learning-based stereo matching methods are mainly trained using close-range synthetic images. Consequently, the application of these methods in aerial photogrammetry and remote sensing is currently far from straightforward. In this paper, we propose a new disparity estimation network for stereo matching and investigate its generalization abilities in regard to aerial images. First, we propose an end-to-end deep learning network for stereo matching, regularized by disparity gradients, which includes a residual cost volume and a reconstruction error volume in a refinement module, and multiple losses. In order to investigate the influence of the multiple losses, a comprehensive analysis is presented. Second, based on this network trained with synthetic close-range data, we propose a new pipeline for matching high-resolution aerial imagery. The experimental results show that the proposed network improves the disparity accuracy by up to 40% in terms of errors larger than 1 px compared to results when not including the refinement network, especially in areas containing detailed small objects. In addition, in qualitative and quantitative experiments, we are able to show that our model, pre-trained on a synthetic stereo dataset, achieves very competitive sub-pixel geometric accuracy on aerial images. These results confirm that the domain gap between synthetic close-range and real aerial images can be satisfactorily bridged using the proposed new deep learning method for dense image matching.

KW - Aerial images

KW - Dense matching

KW - Disparity learning

KW - Disparity refinement

KW - Geometric accuracy

UR - http://www.scopus.com/inward/record.url?scp=85202624058&partnerID=8YFLogxK

U2 - 10.1007/s41064-024-00307-w

DO - 10.1007/s41064-024-00307-w

M3 - Article

AN - SCOPUS:85202624058

VL - 92

SP - 531

EP - 546

JO - PFG - Journal of Photogrammetry, Remote Sensing and Geoinformation Science

JF - PFG - Journal of Photogrammetry, Remote Sensing and Geoinformation Science

SN - 2512-2789

IS - 5

ER -