Details
| Original language | English |
|---|---|
| Pages (from-to) | 348-359 |
| Number of pages | 12 |
| Journal | IEEE Transactions on Biometrics, Behavior, and Identity Science |
| Volume | 5 |
| Issue number | 3 |
| Early online date | 19 Jan 2023 |
| Publication status | Published - Jul 2023 |
Abstract
Head pose estimation plays a vital role in biometric systems related to facial and human behavior analysis. Typically, neural networks are trained on head pose datasets. Unfortunately, manual or sensor-based annotation of head pose is impractical. A solution is synthetic training data generated from 3D face models, which can provide an infinite number of perfect labels. However, computer generated images only provide an approximation of real-world images, leading to a performance gap between training and application domain. Therefore, there is a need for strategies that allow simultaneous learning on labeled synthetic data and unlabeled real-world data to overcome the domain gap. In this work we propose relative pose consistency, a semi-supervised learning strategy for head pose estimation based on consistency regularization. Consistency regularization enforces consistent network predictions under random image augmentations, including pose-preserving and pose-altering augmentations. We propose a strategy to exploit the relative pose introduced by pose-altering augmentations between augmented image pairs, to allow the network to benefit from relative pose labels during training on unlabeled data. We evaluate our approach in a domain-adaptation scenario and in a commonly used cross-dataset scenario. Furthermore, we reproduce related works to enforce consistent evaluation protocols and show that for both scenarios we outperform SOTA.
Keywords
- Behavioral sciences, Consistency Regularization, Deep Learning, Domain Adaptation, Feature extraction, Head Pose Estimation, Pose estimation, Task analysis, Three-dimensional displays, Training, Training data, consistency regularization, deep learning, domain adaptation, Head pose estimation
ASJC Scopus subject areas
- Computer Science(all)
- Artificial Intelligence
- Physics and Astronomy(all)
- Instrumentation
- Computer Science(all)
- Computer Vision and Pattern Recognition
- Computer Science(all)
- Computer Science Applications
Cite this
- Standard
- Harvard
- Apa
- Vancouver
- BibTeX
- RIS
In: IEEE Transactions on Biometrics, Behavior, and Identity Science, Vol. 5, No. 3, 07.2023, p. 348-359.
Research output: Contribution to journal › Article › Research › peer review
}
TY - JOUR
T1 - Domain Adaptation for Head Pose Estimation Using Relative Pose Consistency
AU - Kuhnke, Felix
AU - Ostermann, Jorn
PY - 2023/7
Y1 - 2023/7
N2 - Head pose estimation plays a vital role in biometric systems related to facial and human behavior analysis. Typically, neural networks are trained on head pose datasets. Unfortunately, manual or sensor-based annotation of head pose is impractical. A solution is synthetic training data generated from 3D face models, which can provide an infinite number of perfect labels. However, computer generated images only provide an approximation of real-world images, leading to a performance gap between training and application domain. Therefore, there is a need for strategies that allow simultaneous learning on labeled synthetic data and unlabeled real-world data to overcome the domain gap. In this work we propose relative pose consistency, a semi-supervised learning strategy for head pose estimation based on consistency regularization. Consistency regularization enforces consistent network predictions under random image augmentations, including pose-preserving and pose-altering augmentations. We propose a strategy to exploit the relative pose introduced by pose-altering augmentations between augmented image pairs, to allow the network to benefit from relative pose labels during training on unlabeled data. We evaluate our approach in a domain-adaptation scenario and in a commonly used cross-dataset scenario. Furthermore, we reproduce related works to enforce consistent evaluation protocols and show that for both scenarios we outperform SOTA.
AB - Head pose estimation plays a vital role in biometric systems related to facial and human behavior analysis. Typically, neural networks are trained on head pose datasets. Unfortunately, manual or sensor-based annotation of head pose is impractical. A solution is synthetic training data generated from 3D face models, which can provide an infinite number of perfect labels. However, computer generated images only provide an approximation of real-world images, leading to a performance gap between training and application domain. Therefore, there is a need for strategies that allow simultaneous learning on labeled synthetic data and unlabeled real-world data to overcome the domain gap. In this work we propose relative pose consistency, a semi-supervised learning strategy for head pose estimation based on consistency regularization. Consistency regularization enforces consistent network predictions under random image augmentations, including pose-preserving and pose-altering augmentations. We propose a strategy to exploit the relative pose introduced by pose-altering augmentations between augmented image pairs, to allow the network to benefit from relative pose labels during training on unlabeled data. We evaluate our approach in a domain-adaptation scenario and in a commonly used cross-dataset scenario. Furthermore, we reproduce related works to enforce consistent evaluation protocols and show that for both scenarios we outperform SOTA.
KW - Behavioral sciences
KW - Consistency Regularization
KW - Deep Learning
KW - Domain Adaptation
KW - Feature extraction
KW - Head Pose Estimation
KW - Pose estimation
KW - Task analysis
KW - Three-dimensional displays
KW - Training
KW - Training data
KW - consistency regularization
KW - deep learning
KW - domain adaptation
KW - Head pose estimation
UR - http://www.scopus.com/inward/record.url?scp=85147281573&partnerID=8YFLogxK
U2 - 10.1109/TBIOM.2023.3237039
DO - 10.1109/TBIOM.2023.3237039
M3 - Article
AN - SCOPUS:85147281573
VL - 5
SP - 348
EP - 359
JO - IEEE Transactions on Biometrics, Behavior, and Identity Science
JF - IEEE Transactions on Biometrics, Behavior, and Identity Science
IS - 3
ER -