Disentangling 3D Attributes from a Single 2D Image: Human Pose, Shape and Garment

Xue Hu (Imperial College London), Xinghui Li (University of Oxford), Benjamin Busam (Technical University of Munich), Yiren Zhou (Huawei Noah's Ark Lab), Ales Leonardis (Huawei Noah's Ark Lab), Shanxin Yuan (Queen Mary University of London)*
The 33rd British Machine Vision Conference


For visual manipulation tasks, we aim to represent image content with semantically meaningful features. However, learning implicit representations from images often lacks interpretability, especially when attributes are intertwined. We focus on the challenging task of extracting disentangled 3D attributes only from 2D image data. Specifically, we focus on human appearance and learn implicit pose, shape and garment representations of dressed humans from RGB images. Our method learns an embedding with disentangled latent representations of these three image properties and enables meaningful re-assembling of features and property control through a 2D-to-3D encoder-decoder structure. The 3D model is inferred solely from the feature map in the learned embedding space. To the best of our knowledge, our method is the first to achieve cross-domain disentanglement for this highly under-constrained problem. We qualitatively and quantitatively demonstrate our framework's ability to transfer pose, shape, and garments in 3D reconstruction on virtual data and show how an implicit shape loss can benefit the model's ability to recover fine-grained reconstruction details.



author    = {Xue Hu and Xinghui Li and Benjamin  Busam and Yiren Zhou and Ales Leonardis and Shanxin Yuan},
title     = {Disentangling 3D Attributes from a Single 2D Image: Human Pose, Shape and Garment},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year      = {2022},
url       = {https://bmvc2022.mpi-inf.mpg.de/0031.pdf}

Copyright © 2022 The British Machine Vision Association and Society for Pattern Recognition
The British Machine Vision Conference is organised by The British Machine Vision Association and Society for Pattern Recognition. The Association is a Company limited by guarantee, No.2543446, and a non-profit-making body, registered in England and Wales as Charity No.1002307 (Registered Office: Dept. of Computer Science, Durham University, South Road, Durham, DH1 3LE, UK).

Imprint | Data Protection