StyleFaceUV: a 3D Face UV Map Generator for View-Consistent Face Image Synthesis
Wei-Chieh Chung (National Taiwan University), Jian-Kai Zhu (National Taiwan University), I-Chao Shen (The University of Tokyo),* Yu-Ting Wu (National Taipei University), Yung-Yu Chuang (National Taiwan University)
The 33rd British Machine Vision Conference
Abstract
Recent deep image generation models, such as StyleGAN2, fail to generate high-quality 2D face images with multi-view consistency. We address this problem by proposing an approach for generating detailed 3D faces using a pre-trained StyleGAN2 model. Our method estimates the 3D Morphable Model (3DMM) coefficients directly from the stylecode of StyleGAN2. To add more details to the image generated by StyleGAN2, we train a generator to produce two UV maps: a diffuse map for a more faithful appearance and a generalized displacement map for adding geometric details. To achieve multi-view consistency, we also add a symmetric view image to recover information regarding the invisible side of a single image. We can modify the viewing angle, expressions, and lighting conditions in a consistent manner using the generated detailed 3D face models. Qualitative and quantitative experiments demonstrate that our method outperforms previous methods.Video
Citation
@inproceedings{Chung_2022_BMVC,
author = {Wei-Chieh Chung and Jian-Kai Zhu and I-Chao Shen and Yu-Ting Wu and Yung-Yu Chuang},
title = {StyleFaceUV: a 3D Face UV Map Generator for View-Consistent Face Image Synthesis},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year = {2022},
url = {https://bmvc2022.mpi-inf.mpg.de/0089.pdf}
}
Copyright © 2022
The British Machine Vision Conference is organised by