FoGMesh: 3D Human Mesh Recovery in Videos with Focal Transformer and GRU
Yihao He (Jiangnan University),* Xiaoning Song (Jiangnan University), Tianyang Xu (Jiangnan University), Hua Yang (Jiangnan University), Xiao-Jun Wu (Jiangnan University)
The 33rd British Machine Vision Conference
Abstract
Dense 3D human shape recovery plays an essential role in many computer vision and human-computer interaction tasks. However, accurate and robust 3D human body reconstruction in the wild is very challenging due to non-rigid deformation, occlusion, high-speed motion, etc., restricting the practical applications of the existing 3D human body recovery methods. To alleviate these issues, we propose a novel method using FOcal and Gated Recurrent Unit (GRU) encoders for high-precision 3D human Mesh reconstruction (FoGMesh) in video sequences. Specifically, we first design a new human body prior encoder based on the focal attention mechanism to learn fine-grained local and coarse-grained global interactions. Then, we build a multi-scale feature fusion module to fuse the context information and adaptively adjust the attention weights of small-scale body parts, such as hands. Last, we use the GRU encoder to connect the relevance and implement the proposed FoGMesh method in an end-to-end trainable framework. The proposed method achieves excellent performance on several benchmarking datasets, demonstrating its merits and superiority over the state-of-the-art approaches.Video
Citation
@inproceedings{He_2022_BMVC,
author = {Yihao He and Xiaoning Song and Tianyang Xu and Hua Yang and Xiao-Jun Wu},
title = {FoGMesh: 3D Human Mesh Recovery in Videos with Focal Transformer and GRU},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year = {2022},
url = {https://bmvc2022.mpi-inf.mpg.de/0618.pdf}
}
Copyright © 2022
The British Machine Vision Conference is organised by