UV-Based 3D Hand-Object Reconstruction with Grasp Optimization
Ziwei Yu (National University of Singapore),* Linlin Yang ( University of Bonn), You Xie (TUM), Ping Chen (Bean Tech), Angela Yao (National University of Singapore)
The 33rd British Machine Vision Conference
Abstract
We propose a novel framework for 3D hand shape reconstruction and hand-object grasp optimization from a single RGB image. How the hand-object contact regions are represented is critical for accurate reconstructions. Instead of approximating the contact regions with sparse points, as in previous works, we propose a dense representation in the form of a UV coordinate map. Furthermore, we introduce inference-time optimization to fine-tune the grasp and improve interactions between the hand and the object. Our pipeline increases hand shape reconstruction accuracy and produces a vibrant hand texture. Experiments on datasets such as Ho3D, FreiHand, and DexYCB reveal that our proposed method outperforms the state-of-the-art.Video
Citation
@inproceedings{Yu_2022_BMVC,
author = {Ziwei Yu and Linlin Yang and You Xie and Ping Chen and Angela Yao},
title = {UV-Based 3D Hand-Object Reconstruction with Grasp Optimization},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year = {2022},
url = {https://bmvc2022.mpi-inf.mpg.de/0111.pdf}
}
Copyright © 2022
The British Machine Vision Conference is organised by