Details
| Original language | English |
|---|---|
| Pages (from-to) | 777-785 |
| Number of pages | 9 |
| Journal | International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences - ISPRS Archives |
| Volume | 43 |
| Issue number | B2-2022 |
| Publication status | Published - 30 May 2022 |
| Event | 2022 24th ISPRS Congress on Imaging Today, Foreseeing Tomorrow, Commission II - Nice, France Duration: 6 Jun 2022 → 11 Jun 2022 |
Abstract
Learning from imbalanced class distributions generally leads to a classifier that is not able to distinguish classes with few training examples from the other classes. In the context of cultural heritage, addressing this problem becomes important when existing digital online collections consisting of images depicting artifacts and assigned semantic annotations shall be completed automatically; images with known annotations can be used to train a classifier that predicts missing information, where training data is often highly imbalanced. In the present paper, combining a classification loss with an auxiliary clustering loss is proposed to improve the classification performance particularly for underrepresented classes, where additionally different sampling strategies are applied. The proposed auxiliary loss aims to cluster feature vectors with respect to the semantic annotations as well as to visual properties of the images to be classified and thus, is supposed to help the classifier in distinguishing individual classes. We conduct an ablation study on a dataset consisting of images depicting silk fabrics coming along with annotations for different silk-related classification tasks. Experimental results show improvements of up to 10.5% in average F1-score and up to 20.8% in the F1-score averaged over the underrepresented classes in some classification tasks.
Keywords
- auxiliary clustering loss, class imbalances, convolutional neural networks, Deep learning, image classification, silk heritage
ASJC Scopus subject areas
- Computer Science(all)
- Information Systems
- Social Sciences(all)
- Geography, Planning and Development
Cite this
- Standard
- Harvard
- Apa
- Vancouver
- BibTeX
- RIS
In: International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences - ISPRS Archives, Vol. 43, No. B2-2022, 30.05.2022, p. 777-785.
Research output: Contribution to journal › Conference article › Research › peer review
}
TY - JOUR
T1 - ADDRESSING CLASS IMBALANCE IN MULTI-CLASS IMAGE CLASSIFICATION BY MEANS OF AUXILIARY FEATURE SPACE RESTRICTIONS
AU - Dorozynski, M.
AU - Rottensteiner, F.
N1 - Funding Information: The research leading to these results is in the context of the ”SILKNOW. Silk heritage in the Knowledge Society: from punched cards to big data, deep learning and visual/tangible simulations” project, which has received funding from the European Union’s Horizon 2020 research and innovation program under grant agreement No. 769504.
PY - 2022/5/30
Y1 - 2022/5/30
N2 - Learning from imbalanced class distributions generally leads to a classifier that is not able to distinguish classes with few training examples from the other classes. In the context of cultural heritage, addressing this problem becomes important when existing digital online collections consisting of images depicting artifacts and assigned semantic annotations shall be completed automatically; images with known annotations can be used to train a classifier that predicts missing information, where training data is often highly imbalanced. In the present paper, combining a classification loss with an auxiliary clustering loss is proposed to improve the classification performance particularly for underrepresented classes, where additionally different sampling strategies are applied. The proposed auxiliary loss aims to cluster feature vectors with respect to the semantic annotations as well as to visual properties of the images to be classified and thus, is supposed to help the classifier in distinguishing individual classes. We conduct an ablation study on a dataset consisting of images depicting silk fabrics coming along with annotations for different silk-related classification tasks. Experimental results show improvements of up to 10.5% in average F1-score and up to 20.8% in the F1-score averaged over the underrepresented classes in some classification tasks.
AB - Learning from imbalanced class distributions generally leads to a classifier that is not able to distinguish classes with few training examples from the other classes. In the context of cultural heritage, addressing this problem becomes important when existing digital online collections consisting of images depicting artifacts and assigned semantic annotations shall be completed automatically; images with known annotations can be used to train a classifier that predicts missing information, where training data is often highly imbalanced. In the present paper, combining a classification loss with an auxiliary clustering loss is proposed to improve the classification performance particularly for underrepresented classes, where additionally different sampling strategies are applied. The proposed auxiliary loss aims to cluster feature vectors with respect to the semantic annotations as well as to visual properties of the images to be classified and thus, is supposed to help the classifier in distinguishing individual classes. We conduct an ablation study on a dataset consisting of images depicting silk fabrics coming along with annotations for different silk-related classification tasks. Experimental results show improvements of up to 10.5% in average F1-score and up to 20.8% in the F1-score averaged over the underrepresented classes in some classification tasks.
KW - auxiliary clustering loss
KW - class imbalances
KW - convolutional neural networks
KW - Deep learning
KW - image classification
KW - silk heritage
UR - http://www.scopus.com/inward/record.url?scp=85132030391&partnerID=8YFLogxK
U2 - 10.5194/isprs-archives-XLIII-B2-2022-777-2022
DO - 10.5194/isprs-archives-XLIII-B2-2022-777-2022
M3 - Conference article
AN - SCOPUS:85132030391
VL - 43
SP - 777
EP - 785
JO - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences - ISPRS Archives
JF - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences - ISPRS Archives
SN - 1682-1750
IS - B2-2022
T2 - 2022 24th ISPRS Congress on Imaging Today, Foreseeing Tomorrow, Commission II
Y2 - 6 June 2022 through 11 June 2022
ER -