Details
| Original language | English |
|---|---|
| Title of host publication | 2022 12th Workshop on Hyperspectral Imaging and Signal Processing |
| Subtitle of host publication | Evolution in Remote Sensing, WHISPERS 2022 |
| Publisher | Institute of Electrical and Electronics Engineers Inc. |
| Pages | 1-5 |
| Number of pages | 5 |
| ISBN (electronic) | 978-1-6654-7069-8 |
| ISBN (print) | 978-1-6654-7070-4 |
| Publication status | Published - 2022 |
Publication series
| Name | Workshop on Hyperspectral Image and Signal Processing, Evolution in Remote Sensing |
|---|---|
| Volume | 2022-September |
| ISSN (Print) | 2158-6276 |
Abstract
We propose a CNN for semantic segmentation with classes which may be non-separable in the spatial domain, but distinguishable by additionally exploiting the spectral domain. In this spatial-spectral two-branches-CNN (SS2B-net), firstly, a spatial-branch-CNN exploiting the spatial domain and a spectral-branch-CNN exploiting the spectral domain are independently trained. Then, their parameters are fixed and their outputs serve as inputs for a new tiny CNN trained for classification into the given classes. We reduce the number of hyperspectral bands from 186 to 96 by converting the VR-bands to RGB. Working on just RGB data in the spatial-branch-CNN, well-tried CNNs for RGB data can be applied. Finally, we use a pretrained VGG16-net to avoid overfitting caused by sparse data, together with a don't care class at the borders of classes to avoid overfitting caused by mixed pixels. In the spectral-branch-CNN, a small 1D CNN is designed and applied. The segmentation results show improvements against the 3D hyper-U-net at class borders, at objects in their completeness, in mixed pixels caused by reflections and in wrongly classified objects. We support that by increasing the macro average recall from 0.90 to 0.97.
Keywords
- CNN, hyperspecral imagery, semantic segmentation
ASJC Scopus subject areas
- Computer Science(all)
- Signal Processing
- Computer Science(all)
- Computer Vision and Pattern Recognition
Cite this
- Standard
- Harvard
- Apa
- Vancouver
- BibTeX
- RIS
2022 12th Workshop on Hyperspectral Imaging and Signal Processing: Evolution in Remote Sensing, WHISPERS 2022. Institute of Electrical and Electronics Engineers Inc., 2022. p. 1-5 (Workshop on Hyperspectral Image and Signal Processing, Evolution in Remote Sensing; Vol. 2022-September).
Research output: Chapter in book/report/conference proceeding › Conference contribution › Research › peer review
}
TY - GEN
T1 - Semantic Segmentation Of Natural And Man-Made Fruits Using A Spatial-Spectral Two-Branches-Cnn For Sparse Data
AU - Pestel-Schiller, Ulrike
AU - Yang, Ye
AU - Ostermann, Jörn
PY - 2022
Y1 - 2022
N2 - We propose a CNN for semantic segmentation with classes which may be non-separable in the spatial domain, but distinguishable by additionally exploiting the spectral domain. In this spatial-spectral two-branches-CNN (SS2B-net), firstly, a spatial-branch-CNN exploiting the spatial domain and a spectral-branch-CNN exploiting the spectral domain are independently trained. Then, their parameters are fixed and their outputs serve as inputs for a new tiny CNN trained for classification into the given classes. We reduce the number of hyperspectral bands from 186 to 96 by converting the VR-bands to RGB. Working on just RGB data in the spatial-branch-CNN, well-tried CNNs for RGB data can be applied. Finally, we use a pretrained VGG16-net to avoid overfitting caused by sparse data, together with a don't care class at the borders of classes to avoid overfitting caused by mixed pixels. In the spectral-branch-CNN, a small 1D CNN is designed and applied. The segmentation results show improvements against the 3D hyper-U-net at class borders, at objects in their completeness, in mixed pixels caused by reflections and in wrongly classified objects. We support that by increasing the macro average recall from 0.90 to 0.97.
AB - We propose a CNN for semantic segmentation with classes which may be non-separable in the spatial domain, but distinguishable by additionally exploiting the spectral domain. In this spatial-spectral two-branches-CNN (SS2B-net), firstly, a spatial-branch-CNN exploiting the spatial domain and a spectral-branch-CNN exploiting the spectral domain are independently trained. Then, their parameters are fixed and their outputs serve as inputs for a new tiny CNN trained for classification into the given classes. We reduce the number of hyperspectral bands from 186 to 96 by converting the VR-bands to RGB. Working on just RGB data in the spatial-branch-CNN, well-tried CNNs for RGB data can be applied. Finally, we use a pretrained VGG16-net to avoid overfitting caused by sparse data, together with a don't care class at the borders of classes to avoid overfitting caused by mixed pixels. In the spectral-branch-CNN, a small 1D CNN is designed and applied. The segmentation results show improvements against the 3D hyper-U-net at class borders, at objects in their completeness, in mixed pixels caused by reflections and in wrongly classified objects. We support that by increasing the macro average recall from 0.90 to 0.97.
KW - CNN
KW - hyperspecral imagery
KW - semantic segmentation
UR - http://www.scopus.com/inward/record.url?scp=85143159823&partnerID=8YFLogxK
U2 - 10.1109/WHISPERS56178.2022.9955066
DO - 10.1109/WHISPERS56178.2022.9955066
M3 - Conference contribution
SN - 978-1-6654-7070-4
T3 - Workshop on Hyperspectral Image and Signal Processing, Evolution in Remote Sensing
SP - 1
EP - 5
BT - 2022 12th Workshop on Hyperspectral Imaging and Signal Processing
PB - Institute of Electrical and Electronics Engineers Inc.
ER -