Sparse and Privacy-enhanced Representation for Human Pose Estimation
Ting-Ying Lin (National Tsing Hua University),* Lin-Yung Hsieh (National Tsing Hua University), Fu-En Wang (National Tsing Hua University), Wen-Shen Wuen (Novatek Microelectronics Corp.), Min Sun (NTHU)
The 34th British Machine Vision Conference
Abstract
We propose a sparse and privacy-enhanced representation for Human Pose Estimation (HPE). Given a perspective camera, we use a proprietary motion vector sensor (MVS) to extract an edge image and a two-directional motion vector image at each time frame. Both edge and motion vector images are sparse and contain much less information (i.e., enhancing human privacy). We advocate that edge information is essential for HPE, and motion vectors complement edge information during fast movements. We propose a fusion network leveraging recent advances in sparse convolution used typically for 3D voxels to efficiently process our proposed sparse representation, which achieves about 13x speed-up and 96% reduction in FLOPs. We collect an in-house edge and motion vector dataset with 16 types of actions by 40 users using the proprietary MVS. Our method outperforms individual modalities using only edge or motion vector images. We also demonstrate the generalizability of our approach on MMHPSD and HumanEva datasets. Finally, we validate the privacy-enhanced quality of our sparse representation through face recognition on CelebA (a large face dataset) and a user study on our in-house dataset. The code and dataset are available on the project page: https://lyhsieh.github.io/sphp/.Citation
@inproceedings{Lin_2023_BMVC,
author = {Ting-Ying Lin and Lin-Yung Hsieh and Fu-En Wang and Wen-Shen Wuen and Min Sun},
title = {Sparse and Privacy-enhanced Representation for Human Pose Estimation},
booktitle = {34th British Machine Vision Conference 2023, {BMVC} 2023, Aberdeen, UK, November 20-24, 2023},
publisher = {BMVA},
year = {2023},
url = {https://bmvc2022.mpi-inf.mpg.de/BMVC2023/0143.pdf}
}
Copyright © 2023
The British Machine Vision Conference is organised by