Self-Supervised Robustifying Guidance for Monocular 3D Face Reconstruction
Hitika Tiwari (Indian Institute of Technology Kanpur & National Yang Ming Chiao Tung University ),* Min-Hung Chen (Microsoft), Yi-Min Tsai (MediaTek), Hsien-Kai Kuo (MediaTek), Hung-Jen Chen (MediaTek), Kevin Jou (MediaTek Inc.), K. S. Venkatesh (IIT Kanpur), Yong-Sheng Chen (National Yang Ming Chiao Tung University)
The 33rd British Machine Vision Conference
Abstract
Despite the recent developments in 3D Face Reconstruction from occluded and noisy face images, the performance is still unsatisfactory. Moreover, most existing methods rely on additional dependencies, posing numerous constraints over the training procedure. Therefore, we propose a Self-Supervised RObustifying GUidancE (ROGUE) framework to obtain robustness against occlusions and noise in the face images. The proposed network contains 1) the Guidance Pipeline to obtain the 3D face coefficients for the clean faces and 2) the Robustification Pipeline to acquire the consistency between the estimated coefficients for occluded or noisy images and the clean counterpart. The proposed image- and feature-level loss functions aid the ROGUE learning process without posing additional dependencies. To facilitate model evaluation, we propose two challenging occlusion face datasets, ReaChOcc and SynChOcc, containing real-world and synthetic occlusion-based face images for robustness evaluation. Also, a noisy variant of the test dataset of CelebA is produced for evaluation. Our method outperforms the current state-of-the-art method by large margins (e.g., for the perceptual errors, a reduction of 23.8% for real-world occlusions, 26.4% for synthetic occlusions, and 22.7% for noisy images), demonstrating the effectiveness of the proposed approach. The occlusion datasets and the corresponding evaluation code are released publicly at https://github.com/ArcTrinity9/Datasets-ReaChOcc-and-SynChOcc.Video
Citation
@inproceedings{Tiwari_2022_BMVC,
author = {Hitika Tiwari and Min-Hung Chen and Yi-Min Tsai and Hsien-Kai Kuo and Hung-Jen Chen and Kevin Jou and K. S. Venkatesh and Yong-Sheng Chen},
title = {Self-Supervised Robustifying Guidance for Monocular 3D Face Reconstruction},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year = {2022},
url = {https://bmvc2022.mpi-inf.mpg.de/0220.pdf}
}
Copyright © 2022
The British Machine Vision Conference is organised by