Multilinear Modelling of Faces and Expressions

Publikation: Beitrag in FachzeitschriftArtikelForschungPeer-Review

Autoren

Externe Organisationen

  • IT University of Copenhagen
  • University of Copenhagen
Forschungs-netzwerk anzeigen

Details

OriginalspracheEnglisch
Aufsatznummer9067086
Seiten (von - bis)3540-3554
Seitenumfang15
FachzeitschriftIEEE Trans. Pattern Anal. Mach. Intell.
Jahrgang43
Ausgabenummer10
Frühes Online-Datum14 Apr. 2020
PublikationsstatusVeröffentlicht - Okt. 2021

Abstract

In this work, we present a new versatile 3D multilinear statistical face model, based on a tensor factorisation of 3D face scans, that decomposes the shapes into person and expression subspaces. Investigation of the expression subspace reveals an inherent low-dimensional substructure, and further, a star-shaped structure. This is due to two novel findings. (1) Increasing the strength of one emotion approximately forms a linear trajectory in the subspace. (2) All these trajectories intersect at a single point - not at the neutral expression as assumed by almost all prior works - but at an apathetic expression. We utilise these structural findings by reparameterising the expression subspace by the fourth-order moment tensor centred at the point of apathy. We propose a 3D face reconstruction method from single or multiple 2D projections by assuming an uncalibrated projective camera model. The non-linearity caused by the perspective projection can be neatly included into the model. The proposed algorithm separates person and expression subspaces convincingly, and enables flexible, natural modelling of expressions for a wide variety of human faces. Applying the method on independent faces showed that morphing between different persons and expressions can be performed without strong deformations.

ASJC Scopus Sachgebiete

Zitieren

Multilinear Modelling of Faces and Expressions. / Grasshof, Stella; Ackermann, Hanno; Brandt, Sami Sebastian et al.
in: IEEE Trans. Pattern Anal. Mach. Intell., Jahrgang 43, Nr. 10, 9067086, 10.2021, S. 3540-3554.

Publikation: Beitrag in FachzeitschriftArtikelForschungPeer-Review

Grasshof, S, Ackermann, H, Brandt, SS & Ostermann, J 2021, 'Multilinear Modelling of Faces and Expressions', IEEE Trans. Pattern Anal. Mach. Intell., Jg. 43, Nr. 10, 9067086, S. 3540-3554. https://doi.org/10.1109/TPAMI.2020.2986496
Grasshof, S., Ackermann, H., Brandt, S. S., & Ostermann, J. (2021). Multilinear Modelling of Faces and Expressions. IEEE Trans. Pattern Anal. Mach. Intell., 43(10), 3540-3554. Artikel 9067086. https://doi.org/10.1109/TPAMI.2020.2986496
Grasshof S, Ackermann H, Brandt SS, Ostermann J. Multilinear Modelling of Faces and Expressions. IEEE Trans. Pattern Anal. Mach. Intell. 2021 Okt;43(10):3540-3554. 9067086. Epub 2020 Apr 14. doi: 10.1109/TPAMI.2020.2986496
Grasshof, Stella ; Ackermann, Hanno ; Brandt, Sami Sebastian et al. / Multilinear Modelling of Faces and Expressions. in: IEEE Trans. Pattern Anal. Mach. Intell. 2021 ; Jahrgang 43, Nr. 10. S. 3540-3554.
Download
@article{9cce7bf1033e4e7181482e4e153858d8,
title = "Multilinear Modelling of Faces and Expressions",
abstract = "In this work, we present a new versatile 3D multilinear statistical face model, based on a tensor factorisation of 3D face scans, that decomposes the shapes into person and expression subspaces. Investigation of the expression subspace reveals an inherent low-dimensional substructure, and further, a star-shaped structure. This is due to two novel findings. (1) Increasing the strength of one emotion approximately forms a linear trajectory in the subspace. (2) All these trajectories intersect at a single point - not at the neutral expression as assumed by almost all prior works - but at an apathetic expression. We utilise these structural findings by reparameterising the expression subspace by the fourth-order moment tensor centred at the point of apathy. We propose a 3D face reconstruction method from single or multiple 2D projections by assuming an uncalibrated projective camera model. The non-linearity caused by the perspective projection can be neatly included into the model. The proposed algorithm separates person and expression subspaces convincingly, and enables flexible, natural modelling of expressions for a wide variety of human faces. Applying the method on independent faces showed that morphing between different persons and expressions can be performed without strong deformations.",
keywords = "3D-reconstruction, expression transfer, HOSVD, person transfer, Statistical shape model, tensor model",
author = "Stella Grasshof and Hanno Ackermann and Brandt, {Sami Sebastian} and J{\"o}rn Ostermann",
note = "Funding Information: This article was supported by German Research Foundation Grant DFG RO 2497/12-2.",
year = "2021",
month = oct,
doi = "10.1109/TPAMI.2020.2986496",
language = "English",
volume = "43",
pages = "3540--3554",
number = "10",

}

Download

TY - JOUR

T1 - Multilinear Modelling of Faces and Expressions

AU - Grasshof, Stella

AU - Ackermann, Hanno

AU - Brandt, Sami Sebastian

AU - Ostermann, Jörn

N1 - Funding Information: This article was supported by German Research Foundation Grant DFG RO 2497/12-2.

PY - 2021/10

Y1 - 2021/10

N2 - In this work, we present a new versatile 3D multilinear statistical face model, based on a tensor factorisation of 3D face scans, that decomposes the shapes into person and expression subspaces. Investigation of the expression subspace reveals an inherent low-dimensional substructure, and further, a star-shaped structure. This is due to two novel findings. (1) Increasing the strength of one emotion approximately forms a linear trajectory in the subspace. (2) All these trajectories intersect at a single point - not at the neutral expression as assumed by almost all prior works - but at an apathetic expression. We utilise these structural findings by reparameterising the expression subspace by the fourth-order moment tensor centred at the point of apathy. We propose a 3D face reconstruction method from single or multiple 2D projections by assuming an uncalibrated projective camera model. The non-linearity caused by the perspective projection can be neatly included into the model. The proposed algorithm separates person and expression subspaces convincingly, and enables flexible, natural modelling of expressions for a wide variety of human faces. Applying the method on independent faces showed that morphing between different persons and expressions can be performed without strong deformations.

AB - In this work, we present a new versatile 3D multilinear statistical face model, based on a tensor factorisation of 3D face scans, that decomposes the shapes into person and expression subspaces. Investigation of the expression subspace reveals an inherent low-dimensional substructure, and further, a star-shaped structure. This is due to two novel findings. (1) Increasing the strength of one emotion approximately forms a linear trajectory in the subspace. (2) All these trajectories intersect at a single point - not at the neutral expression as assumed by almost all prior works - but at an apathetic expression. We utilise these structural findings by reparameterising the expression subspace by the fourth-order moment tensor centred at the point of apathy. We propose a 3D face reconstruction method from single or multiple 2D projections by assuming an uncalibrated projective camera model. The non-linearity caused by the perspective projection can be neatly included into the model. The proposed algorithm separates person and expression subspaces convincingly, and enables flexible, natural modelling of expressions for a wide variety of human faces. Applying the method on independent faces showed that morphing between different persons and expressions can be performed without strong deformations.

KW - 3D-reconstruction

KW - expression transfer

KW - HOSVD

KW - person transfer

KW - Statistical shape model

KW - tensor model

UR - http://www.scopus.com/inward/record.url?scp=85114612751&partnerID=8YFLogxK

U2 - 10.1109/TPAMI.2020.2986496

DO - 10.1109/TPAMI.2020.2986496

M3 - Article

VL - 43

SP - 3540

EP - 3554

JO - IEEE Trans. Pattern Anal. Mach. Intell.

JF - IEEE Trans. Pattern Anal. Mach. Intell.

IS - 10

M1 - 9067086

ER -

Von denselben Autoren