Details
| Originalsprache | Englisch |
|---|---|
| Titel des Sammelwerks | 2017 IEEE International Conference on Multimedia and Expo |
| Untertitel | ICME 2017 |
| Herausgeber (Verlag) | IEEE Computer Society |
| Seiten | 1075-1080 |
| Seitenumfang | 6 |
| ISBN (elektronisch) | 9781509060672 |
| Publikationsstatus | Veröffentlicht - 28 Aug. 2017 |
| Veranstaltung | 2017 IEEE International Conference on Multimedia and Expo, ICME 2017 - Hong Kong, Hongkong Dauer: 10 Juli 2017 → 14 Juli 2017 |
Publikationsreihe
| Name | Proceedings - IEEE International Conference on Multimedia and Expo |
|---|---|
| ISSN (Print) | 1945-7871 |
| ISSN (elektronisch) | 1945-788X |
Abstract
Given a pre-registered 3D mesh sequence and accompanying phoneme-labeled audio, our system creates an animatable face model and a mapping procedure to produce realistic speech animations for arbitrary speech input. Mapping of speech features to model parameters is done using random forests for regression. We propose a new speech feature based on phonemic labels and acoustic features. The novel feature produces more expressive facial animation and it robustly handles temporal labeling errors. Furthermore, by employing a sliding window approach to feature extraction, the system is easy to train and allows for low-delay synthesis. We show that our novel combination of speech features improves visual speech synthesis. Our findings are confirmed by a subjective user study.
ASJC Scopus Sachgebiete
- Informatik (insg.)
- Computernetzwerke und -kommunikation
- Informatik (insg.)
- Angewandte Informatik
Zitieren
- Standard
- Harvard
- Apa
- Vancouver
- BibTex
- RIS
2017 IEEE International Conference on Multimedia and Expo: ICME 2017. IEEE Computer Society, 2017. S. 1075-1080 8019546 (Proceedings - IEEE International Conference on Multimedia and Expo).
Publikation: Beitrag in Buch/Bericht/Sammelwerk/Konferenzband › Aufsatz in Konferenzband › Forschung › Peer-Review
}
TY - GEN
T1 - Visual speech synthesis from 3D mesh sequences driven by combined speech features
AU - Kuhnke, Felix
AU - Ostermann, Jörn
PY - 2017/8/28
Y1 - 2017/8/28
N2 - Given a pre-registered 3D mesh sequence and accompanying phoneme-labeled audio, our system creates an animatable face model and a mapping procedure to produce realistic speech animations for arbitrary speech input. Mapping of speech features to model parameters is done using random forests for regression. We propose a new speech feature based on phonemic labels and acoustic features. The novel feature produces more expressive facial animation and it robustly handles temporal labeling errors. Furthermore, by employing a sliding window approach to feature extraction, the system is easy to train and allows for low-delay synthesis. We show that our novel combination of speech features improves visual speech synthesis. Our findings are confirmed by a subjective user study.
AB - Given a pre-registered 3D mesh sequence and accompanying phoneme-labeled audio, our system creates an animatable face model and a mapping procedure to produce realistic speech animations for arbitrary speech input. Mapping of speech features to model parameters is done using random forests for regression. We propose a new speech feature based on phonemic labels and acoustic features. The novel feature produces more expressive facial animation and it robustly handles temporal labeling errors. Furthermore, by employing a sliding window approach to feature extraction, the system is easy to train and allows for low-delay synthesis. We show that our novel combination of speech features improves visual speech synthesis. Our findings are confirmed by a subjective user study.
KW - Facial Animation
KW - Lip Synchronization
KW - Speech Features
KW - Visual Speech Synthesis
UR - http://www.scopus.com/inward/record.url?scp=85030238866&partnerID=8YFLogxK
U2 - 10.1109/icme.2017.8019546
DO - 10.1109/icme.2017.8019546
M3 - Conference contribution
AN - SCOPUS:85030238866
T3 - Proceedings - IEEE International Conference on Multimedia and Expo
SP - 1075
EP - 1080
BT - 2017 IEEE International Conference on Multimedia and Expo
PB - IEEE Computer Society
T2 - 2017 IEEE International Conference on Multimedia and Expo, ICME 2017
Y2 - 10 July 2017 through 14 July 2017
ER -