Disentangling 3D Attributes from a Single 2D Image: Human Pose, Shape and Garment
Xue Hu (Imperial College London), Xinghui Li (University of Oxford), Benjamin Busam (Technical University of Munich), Yiren Zhou (Huawei Noah's Ark Lab), Ales Leonardis (Huawei Noah's Ark Lab), Shanxin Yuan (Queen Mary University of London)*
The 33rd British Machine Vision Conference
Abstract
For visual manipulation tasks, we aim to represent image content with semantically meaningful features. However, learning implicit representations from images often lacks interpretability, especially when attributes are intertwined. We focus on the challenging task of extracting disentangled 3D attributes only from 2D image data. Specifically, we focus on human appearance and learn implicit pose, shape and garment representations of dressed humans from RGB images. Our method learns an embedding with disentangled latent representations of these three image properties and enables meaningful re-assembling of features and property control through a 2D-to-3D encoder-decoder structure. The 3D model is inferred solely from the feature map in the learned embedding space. To the best of our knowledge, our method is the first to achieve cross-domain disentanglement for this highly under-constrained problem. We qualitatively and quantitatively demonstrate our framework's ability to transfer pose, shape, and garments in 3D reconstruction on virtual data and show how an implicit shape loss can benefit the model's ability to recover fine-grained reconstruction details.Video
Citation
@inproceedings{Hu_2022_BMVC,
author = {Xue Hu and Xinghui Li and Benjamin Busam and Yiren Zhou and Ales Leonardis and Shanxin Yuan},
title = {Disentangling 3D Attributes from a Single 2D Image: Human Pose, Shape and Garment},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year = {2022},
url = {https://bmvc2022.mpi-inf.mpg.de/0031.pdf}
}
Copyright © 2022
The British Machine Vision Conference is organised by