Details
| Original language | English |
|---|---|
| Title of host publication | XXIV ISPRS Congress “Imaging today, foreseeing tomorrow” |
| Pages | 157-165 |
| Number of pages | 9 |
| Publication status | Published - 30 May 2022 |
| Event | 2022 24th ISPRS Congress on Imaging Today, Foreseeing Tomorrow, Commission I - Nice, France Duration: 6 Jun 2022 → 11 Jun 2022 |
Publication series
| Name | International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences - ISPRS Archives |
|---|---|
| Publisher | International Society for Photogrammetry and Remote Sensing |
| Volume | XLIII-B1-2022 |
| ISSN (Print) | 1682-1750 |
Abstract
Despite the fact that Thermal Infrared (TIR) cameras have been in use for decades, processing of TIR images poses a variety of challenges when compared to optical images, which are captured in the visible part of the electromagnetic spectrum. The estimation of the exterior orientation of TIR cameras by bundle adjustment is a difficult task due to the limited geometric resolution of a TIR camera and the low image quality in terms of contrast and texture compared to optical images. Optical images have a potential to increase TIR external orientation accuracy by incorporating them into a joint bundle adjustment. However, because the modality gap between those two image types is large, classical point matching algorithms typically fail to find matches, making processing both image types in the joint bundle challenging. In order to locate matching points in both modalities, this study suggests using the Edge Histogram Descriptor (EHD) in the frequency domain representation of the images based on phase congruency. To properly allocate edges from the phase congruency, which are then employed in EHD, non-maximum suppression and hysteresis thresholding are used. Considering that both sensors are fixed rigidly to a single platform, the search region for the matching point candidate of the TIR image is determined based on stereo calibration of a thermal/optical stereo setup combined with geometric constraints. The final matching is based on the cosine distance, while RANdom SAmple Consensus (RANSAC) is used in order to eliminate outliers. The findings of this study show that using a joint bundle adjustment with optical images versus a bundle adjustment only with TIR images improves TIR image orientation, which is supported by the increased accuracy of the adjusted Ground Control Point (GCP) coordinates.
Keywords
- bundle adjustment, multimodal feature matching, phase congruency, Thermal infrared and optical image registration
ASJC Scopus subject areas
- Computer Science(all)
- Information Systems
- Social Sciences(all)
- Geography, Planning and Development
Cite this
- Standard
- Harvard
- Apa
- Vancouver
- BibTeX
- RIS
XXIV ISPRS Congress “Imaging today, foreseeing tomorrow”. 2022. p. 157-165 (International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences - ISPRS Archives; Vol. XLIII-B1-2022).
Research output: Chapter in book/report/conference proceeding › Conference contribution › Research › peer review
}
TY - GEN
T1 - JOINT BUNDLE ADJUSTMENT OF THERMAL INFRA-RED AND OPTICAL IMAGES BASED ON MULTIMODAL MATCHING
AU - Sledz, A.
AU - Heipke, C.
N1 - Funding Information: The work is supported by the Arbeitsgemeinschaft industrieller Forschungsvereinigungen (AiF) under IGF-grant no. 19768.
PY - 2022/5/30
Y1 - 2022/5/30
N2 - Despite the fact that Thermal Infrared (TIR) cameras have been in use for decades, processing of TIR images poses a variety of challenges when compared to optical images, which are captured in the visible part of the electromagnetic spectrum. The estimation of the exterior orientation of TIR cameras by bundle adjustment is a difficult task due to the limited geometric resolution of a TIR camera and the low image quality in terms of contrast and texture compared to optical images. Optical images have a potential to increase TIR external orientation accuracy by incorporating them into a joint bundle adjustment. However, because the modality gap between those two image types is large, classical point matching algorithms typically fail to find matches, making processing both image types in the joint bundle challenging. In order to locate matching points in both modalities, this study suggests using the Edge Histogram Descriptor (EHD) in the frequency domain representation of the images based on phase congruency. To properly allocate edges from the phase congruency, which are then employed in EHD, non-maximum suppression and hysteresis thresholding are used. Considering that both sensors are fixed rigidly to a single platform, the search region for the matching point candidate of the TIR image is determined based on stereo calibration of a thermal/optical stereo setup combined with geometric constraints. The final matching is based on the cosine distance, while RANdom SAmple Consensus (RANSAC) is used in order to eliminate outliers. The findings of this study show that using a joint bundle adjustment with optical images versus a bundle adjustment only with TIR images improves TIR image orientation, which is supported by the increased accuracy of the adjusted Ground Control Point (GCP) coordinates.
AB - Despite the fact that Thermal Infrared (TIR) cameras have been in use for decades, processing of TIR images poses a variety of challenges when compared to optical images, which are captured in the visible part of the electromagnetic spectrum. The estimation of the exterior orientation of TIR cameras by bundle adjustment is a difficult task due to the limited geometric resolution of a TIR camera and the low image quality in terms of contrast and texture compared to optical images. Optical images have a potential to increase TIR external orientation accuracy by incorporating them into a joint bundle adjustment. However, because the modality gap between those two image types is large, classical point matching algorithms typically fail to find matches, making processing both image types in the joint bundle challenging. In order to locate matching points in both modalities, this study suggests using the Edge Histogram Descriptor (EHD) in the frequency domain representation of the images based on phase congruency. To properly allocate edges from the phase congruency, which are then employed in EHD, non-maximum suppression and hysteresis thresholding are used. Considering that both sensors are fixed rigidly to a single platform, the search region for the matching point candidate of the TIR image is determined based on stereo calibration of a thermal/optical stereo setup combined with geometric constraints. The final matching is based on the cosine distance, while RANdom SAmple Consensus (RANSAC) is used in order to eliminate outliers. The findings of this study show that using a joint bundle adjustment with optical images versus a bundle adjustment only with TIR images improves TIR image orientation, which is supported by the increased accuracy of the adjusted Ground Control Point (GCP) coordinates.
KW - bundle adjustment
KW - multimodal feature matching
KW - phase congruency
KW - Thermal infrared and optical image registration
UR - http://www.scopus.com/inward/record.url?scp=85131954032&partnerID=8YFLogxK
U2 - 10.5194/isprs-archives-XLIII-B1-2022-157-2022
DO - 10.5194/isprs-archives-XLIII-B1-2022-157-2022
M3 - Conference contribution
AN - SCOPUS:85131954032
T3 - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences - ISPRS Archives
SP - 157
EP - 165
BT - XXIV ISPRS Congress “Imaging today, foreseeing tomorrow”
T2 - 2022 24th ISPRS Congress on Imaging Today, Foreseeing Tomorrow, Commission I
Y2 - 6 June 2022 through 11 June 2022
ER -