Details
| Original language | English |
|---|---|
| Article number | 9067086 |
| Pages (from-to) | 3540-3554 |
| Number of pages | 15 |
| Journal | IEEE Trans. Pattern Anal. Mach. Intell. |
| Volume | 43 |
| Issue number | 10 |
| Early online date | 14 Apr 2020 |
| Publication status | Published - Oct 2021 |
Abstract
In this work, we present a new versatile 3D multilinear statistical face model, based on a tensor factorisation of 3D face scans, that decomposes the shapes into person and expression subspaces. Investigation of the expression subspace reveals an inherent low-dimensional substructure, and further, a star-shaped structure. This is due to two novel findings. (1) Increasing the strength of one emotion approximately forms a linear trajectory in the subspace. (2) All these trajectories intersect at a single point - not at the neutral expression as assumed by almost all prior works - but at an apathetic expression. We utilise these structural findings by reparameterising the expression subspace by the fourth-order moment tensor centred at the point of apathy. We propose a 3D face reconstruction method from single or multiple 2D projections by assuming an uncalibrated projective camera model. The non-linearity caused by the perspective projection can be neatly included into the model. The proposed algorithm separates person and expression subspaces convincingly, and enables flexible, natural modelling of expressions for a wide variety of human faces. Applying the method on independent faces showed that morphing between different persons and expressions can be performed without strong deformations.
Keywords
- 3D-reconstruction, expression transfer, HOSVD, person transfer, Statistical shape model, tensor model
ASJC Scopus subject areas
- Computer Science(all)
- Software
- Computer Science(all)
- Artificial Intelligence
- Mathematics(all)
- Applied Mathematics
- Computer Science(all)
- Computer Vision and Pattern Recognition
- Computer Science(all)
- Computational Theory and Mathematics
Cite this
- Standard
- Harvard
- Apa
- Vancouver
- BibTeX
- RIS
In: IEEE Trans. Pattern Anal. Mach. Intell., Vol. 43, No. 10, 9067086, 10.2021, p. 3540-3554.
Research output: Contribution to journal › Article › Research › peer review
}
TY - JOUR
T1 - Multilinear Modelling of Faces and Expressions
AU - Grasshof, Stella
AU - Ackermann, Hanno
AU - Brandt, Sami Sebastian
AU - Ostermann, Jörn
N1 - Funding Information: This article was supported by German Research Foundation Grant DFG RO 2497/12-2.
PY - 2021/10
Y1 - 2021/10
N2 - In this work, we present a new versatile 3D multilinear statistical face model, based on a tensor factorisation of 3D face scans, that decomposes the shapes into person and expression subspaces. Investigation of the expression subspace reveals an inherent low-dimensional substructure, and further, a star-shaped structure. This is due to two novel findings. (1) Increasing the strength of one emotion approximately forms a linear trajectory in the subspace. (2) All these trajectories intersect at a single point - not at the neutral expression as assumed by almost all prior works - but at an apathetic expression. We utilise these structural findings by reparameterising the expression subspace by the fourth-order moment tensor centred at the point of apathy. We propose a 3D face reconstruction method from single or multiple 2D projections by assuming an uncalibrated projective camera model. The non-linearity caused by the perspective projection can be neatly included into the model. The proposed algorithm separates person and expression subspaces convincingly, and enables flexible, natural modelling of expressions for a wide variety of human faces. Applying the method on independent faces showed that morphing between different persons and expressions can be performed without strong deformations.
AB - In this work, we present a new versatile 3D multilinear statistical face model, based on a tensor factorisation of 3D face scans, that decomposes the shapes into person and expression subspaces. Investigation of the expression subspace reveals an inherent low-dimensional substructure, and further, a star-shaped structure. This is due to two novel findings. (1) Increasing the strength of one emotion approximately forms a linear trajectory in the subspace. (2) All these trajectories intersect at a single point - not at the neutral expression as assumed by almost all prior works - but at an apathetic expression. We utilise these structural findings by reparameterising the expression subspace by the fourth-order moment tensor centred at the point of apathy. We propose a 3D face reconstruction method from single or multiple 2D projections by assuming an uncalibrated projective camera model. The non-linearity caused by the perspective projection can be neatly included into the model. The proposed algorithm separates person and expression subspaces convincingly, and enables flexible, natural modelling of expressions for a wide variety of human faces. Applying the method on independent faces showed that morphing between different persons and expressions can be performed without strong deformations.
KW - 3D-reconstruction
KW - expression transfer
KW - HOSVD
KW - person transfer
KW - Statistical shape model
KW - tensor model
UR - http://www.scopus.com/inward/record.url?scp=85114612751&partnerID=8YFLogxK
U2 - 10.1109/TPAMI.2020.2986496
DO - 10.1109/TPAMI.2020.2986496
M3 - Article
VL - 43
SP - 3540
EP - 3554
JO - IEEE Trans. Pattern Anal. Mach. Intell.
JF - IEEE Trans. Pattern Anal. Mach. Intell.
IS - 10
M1 - 9067086
ER -