Loading [MathJax]/extensions/tex2jax.js

Utilizing Uncertainty in 2D Pose Detectors for Probabilistic 3D Human Mesh Recovery

Research output: Chapter in book/report/conference proceedingConference contributionResearchpeer review

Authors

Research Organisations

External Research Organisations

  • Linkoping University

Details

Original languageEnglish
Title of host publicationProceedings - 2025 IEEE Winter Conference on Applications of Computer Vision, WACV 2025
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages5852-5862
Number of pages11
ISBN (electronic)9798331510831
ISBN (print)979-8-3315-1084-8
Publication statusPublished - 26 Feb 2025
Event2025 IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2025 - Tucson, United States
Duration: 28 Feb 20254 Mar 2025

Publication series

NameIEEE Winter Conference on Applications of Computer Vision
ISSN (Print)2472-6737
ISSN (electronic)2642-9381

Abstract

Monocular 3D human pose and shape estimation is an inherently ill-posed problem due to depth ambiguities, occlusions, and truncations. Recent probabilistic approaches learn a distribution over plausible 3D human meshes by maximizing the likelihood of the ground-truth pose given an image. We show that this objective function alone is not sufficient to best capture the full distributions. Instead, we propose to additionally supervise the learned distributions by minimizing the distance to distributions encoded in heatmaps of a 2D pose detector. Moreover, we reveal that current methods often generate incorrect hypotheses for invisible joints which is not detected by the evaluation protocols. We demonstrate that person segmentation masks can be utilized during training to significantly decrease the number of invalid samples and introduce two metrics to evaluate it. Our normalizing flow-based approach predicts plausible 3D human mesh hypotheses that are consistent with the image evidence while maintaining high diversity for ambiguous body parts. Experiments on 3DPW and EMDB show that we outperform other state-of-the-art probabilistic methods. Code is available for research purposes at https://github.com/twehrbein/humr.

ASJC Scopus subject areas

Sustainable Development Goals

Cite this

Utilizing Uncertainty in 2D Pose Detectors for Probabilistic 3D Human Mesh Recovery. / Wehrbein, Tom; Rudolph, Marco; Rosenhahn, Bodo et al.
Proceedings - 2025 IEEE Winter Conference on Applications of Computer Vision, WACV 2025. Institute of Electrical and Electronics Engineers Inc., 2025. p. 5852-5862 (IEEE Winter Conference on Applications of Computer Vision).

Research output: Chapter in book/report/conference proceedingConference contributionResearchpeer review

Wehrbein, T, Rudolph, M, Rosenhahn, B & Wandt, B 2025, Utilizing Uncertainty in 2D Pose Detectors for Probabilistic 3D Human Mesh Recovery. in Proceedings - 2025 IEEE Winter Conference on Applications of Computer Vision, WACV 2025. IEEE Winter Conference on Applications of Computer Vision, Institute of Electrical and Electronics Engineers Inc., pp. 5852-5862, 2025 IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2025, Tucson, Arizona, United States, 28 Feb 2025. https://doi.org/10.1109/WACV61041.2025.00571, https://doi.org/10.48550/arXiv.2411.16289
Wehrbein, T., Rudolph, M., Rosenhahn, B., & Wandt, B. (2025). Utilizing Uncertainty in 2D Pose Detectors for Probabilistic 3D Human Mesh Recovery. In Proceedings - 2025 IEEE Winter Conference on Applications of Computer Vision, WACV 2025 (pp. 5852-5862). (IEEE Winter Conference on Applications of Computer Vision). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/WACV61041.2025.00571, https://doi.org/10.48550/arXiv.2411.16289
Wehrbein T, Rudolph M, Rosenhahn B, Wandt B. Utilizing Uncertainty in 2D Pose Detectors for Probabilistic 3D Human Mesh Recovery. In Proceedings - 2025 IEEE Winter Conference on Applications of Computer Vision, WACV 2025. Institute of Electrical and Electronics Engineers Inc. 2025. p. 5852-5862. (IEEE Winter Conference on Applications of Computer Vision). doi: 10.1109/WACV61041.2025.00571, 10.48550/arXiv.2411.16289
Wehrbein, Tom ; Rudolph, Marco ; Rosenhahn, Bodo et al. / Utilizing Uncertainty in 2D Pose Detectors for Probabilistic 3D Human Mesh Recovery. Proceedings - 2025 IEEE Winter Conference on Applications of Computer Vision, WACV 2025. Institute of Electrical and Electronics Engineers Inc., 2025. pp. 5852-5862 (IEEE Winter Conference on Applications of Computer Vision).
Download
@inproceedings{1415cb421d3e48d88d5b4fafa5a7493c,
title = "Utilizing Uncertainty in 2D Pose Detectors for Probabilistic 3D Human Mesh Recovery",
abstract = "Monocular 3D human pose and shape estimation is an inherently ill-posed problem due to depth ambiguities, occlusions, and truncations. Recent probabilistic approaches learn a distribution over plausible 3D human meshes by maximizing the likelihood of the ground-truth pose given an image. We show that this objective function alone is not sufficient to best capture the full distributions. Instead, we propose to additionally supervise the learned distributions by minimizing the distance to distributions encoded in heatmaps of a 2D pose detector. Moreover, we reveal that current methods often generate incorrect hypotheses for invisible joints which is not detected by the evaluation protocols. We demonstrate that person segmentation masks can be utilized during training to significantly decrease the number of invalid samples and introduce two metrics to evaluate it. Our normalizing flow-based approach predicts plausible 3D human mesh hypotheses that are consistent with the image evidence while maintaining high diversity for ambiguous body parts. Experiments on 3DPW and EMDB show that we outperform other state-of-the-art probabilistic methods. Code is available for research purposes at https://github.com/twehrbein/humr.",
author = "Tom Wehrbein and Marco Rudolph and Bodo Rosenhahn and Bastian Wandt",
note = "Publisher Copyright: {\textcopyright} 2025 IEEE.; 2025 IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2025, WACV 2025 ; Conference date: 28-02-2025 Through 04-03-2025",
year = "2025",
month = feb,
day = "26",
doi = "10.1109/WACV61041.2025.00571",
language = "English",
isbn = "979-8-3315-1084-8",
series = "IEEE Winter Conference on Applications of Computer Vision",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "5852--5862",
booktitle = "Proceedings - 2025 IEEE Winter Conference on Applications of Computer Vision, WACV 2025",
address = "United States",

}

Download

TY - GEN

T1 - Utilizing Uncertainty in 2D Pose Detectors for Probabilistic 3D Human Mesh Recovery

AU - Wehrbein, Tom

AU - Rudolph, Marco

AU - Rosenhahn, Bodo

AU - Wandt, Bastian

N1 - Publisher Copyright: © 2025 IEEE.

PY - 2025/2/26

Y1 - 2025/2/26

N2 - Monocular 3D human pose and shape estimation is an inherently ill-posed problem due to depth ambiguities, occlusions, and truncations. Recent probabilistic approaches learn a distribution over plausible 3D human meshes by maximizing the likelihood of the ground-truth pose given an image. We show that this objective function alone is not sufficient to best capture the full distributions. Instead, we propose to additionally supervise the learned distributions by minimizing the distance to distributions encoded in heatmaps of a 2D pose detector. Moreover, we reveal that current methods often generate incorrect hypotheses for invisible joints which is not detected by the evaluation protocols. We demonstrate that person segmentation masks can be utilized during training to significantly decrease the number of invalid samples and introduce two metrics to evaluate it. Our normalizing flow-based approach predicts plausible 3D human mesh hypotheses that are consistent with the image evidence while maintaining high diversity for ambiguous body parts. Experiments on 3DPW and EMDB show that we outperform other state-of-the-art probabilistic methods. Code is available for research purposes at https://github.com/twehrbein/humr.

AB - Monocular 3D human pose and shape estimation is an inherently ill-posed problem due to depth ambiguities, occlusions, and truncations. Recent probabilistic approaches learn a distribution over plausible 3D human meshes by maximizing the likelihood of the ground-truth pose given an image. We show that this objective function alone is not sufficient to best capture the full distributions. Instead, we propose to additionally supervise the learned distributions by minimizing the distance to distributions encoded in heatmaps of a 2D pose detector. Moreover, we reveal that current methods often generate incorrect hypotheses for invisible joints which is not detected by the evaluation protocols. We demonstrate that person segmentation masks can be utilized during training to significantly decrease the number of invalid samples and introduce two metrics to evaluate it. Our normalizing flow-based approach predicts plausible 3D human mesh hypotheses that are consistent with the image evidence while maintaining high diversity for ambiguous body parts. Experiments on 3DPW and EMDB show that we outperform other state-of-the-art probabilistic methods. Code is available for research purposes at https://github.com/twehrbein/humr.

UR - http://www.scopus.com/inward/record.url?scp=105003627374&partnerID=8YFLogxK

U2 - 10.1109/WACV61041.2025.00571

DO - 10.1109/WACV61041.2025.00571

M3 - Conference contribution

AN - SCOPUS:105003627374

SN - 979-8-3315-1084-8

T3 - IEEE Winter Conference on Applications of Computer Vision

SP - 5852

EP - 5862

BT - Proceedings - 2025 IEEE Winter Conference on Applications of Computer Vision, WACV 2025

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2025 IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2025

Y2 - 28 February 2025 through 4 March 2025

ER -

By the same author(s)