FoGMesh: 3D Human Mesh Recovery in Videos with Focal Transformer and GRU


Yihao He (Jiangnan University),* Xiaoning Song (Jiangnan University), Tianyang Xu (Jiangnan University), Hua Yang (Jiangnan University), Xiao-Jun Wu (Jiangnan University)
The 33rd British Machine Vision Conference

Abstract

Dense 3D human shape recovery plays an essential role in many computer vision and human-computer interaction tasks. However, accurate and robust 3D human body reconstruction in the wild is very challenging due to non-rigid deformation, occlusion, high-speed motion, etc., restricting the practical applications of the existing 3D human body recovery methods. To alleviate these issues, we propose a novel method using FOcal and Gated Recurrent Unit (GRU) encoders for high-precision 3D human Mesh reconstruction (FoGMesh) in video sequences. Specifically, we first design a new human body prior encoder based on the focal attention mechanism to learn fine-grained local and coarse-grained global interactions. Then, we build a multi-scale feature fusion module to fuse the context information and adaptively adjust the attention weights of small-scale body parts, such as hands. Last, we use the GRU encoder to connect the relevance and implement the proposed FoGMesh method in an end-to-end trainable framework. The proposed method achieves excellent performance on several benchmarking datasets, demonstrating its merits and superiority over the state-of-the-art approaches.

Video



Citation

@inproceedings{He_2022_BMVC,
author    = {Yihao He and Xiaoning Song and Tianyang Xu and Hua Yang and Xiao-Jun Wu},
title     = {FoGMesh: 3D Human Mesh Recovery in Videos with Focal Transformer and GRU},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year      = {2022},
url       = {https://bmvc2022.mpi-inf.mpg.de/0618.pdf}
}


Copyright © 2022 The British Machine Vision Association and Society for Pattern Recognition
The British Machine Vision Conference is organised by The British Machine Vision Association and Society for Pattern Recognition. The Association is a Company limited by guarantee, No.2543446, and a non-profit-making body, registered in England and Wales as Charity No.1002307 (Registered Office: Dept. of Computer Science, Durham University, South Road, Durham, DH1 3LE, UK).

Imprint | Data Protection