Prior-Aware Synthetic Data to the Rescue: Animal Pose Estimation with Very Limited Real Data
Le Jiang (Northeastern University ), Shuangjun Liu (Northeastern University), Xiangyu Bai (Northeastern University), Sarah Ostadabbas (Northeastern University)*
The 33rd British Machine Vision Conference
Abstract
Accurately annotated image datasets are essential components for studying animal behaviors from their poses. Compared to the number of species we know and may exist, the existing labeled pose datasets cover only a small portion of them, while building comprehensive large-scale datasets is prohibitively expensive. Here, we present a very data efficient strategy targeted for pose estimation in quadrupeds that requires only a small amount of real images from the target animal. It is confirmed that fine-tuning a backbone network with pretrained weights on generic image datasets such as ImageNet can mitigate the high demand for target animal pose data and shorten the training time by learning the the prior knowledge of object segmentation and keypoint estimation in advance. However, when faced with serious data scarcity (i.e., <10^2 real images), the model performance stays unsatisfactory, particularly for limbs with considerable flexibility and several comparable parts. We therefore introduce a prior-aware synthetic animal data generation pipeline called PASyn to augment the animal pose data essential for robust pose estimation. PASyn generates a probabilistically-valid synthetic pose dataset, SynAP, through training a variational generative model on several animated 3D animal models. In addition, a style transfer strategy is utilized to blend the synthetic animal image into the real backgrounds. We evaluate the improvement made by our approach with three popular backbone networks and test their pose estimation accuracy on publicly available animal pose images as well as collected from real animals in a zoo.Video
Citation
@inproceedings{Jiang_2022_BMVC,
author = {Le Jiang and Shuangjun Liu and Xiangyu Bai and Sarah Ostadabbas},
title = {Prior-Aware Synthetic Data to the Rescue: Animal Pose Estimation with Very Limited Real Data},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year = {2022},
url = {https://bmvc2022.mpi-inf.mpg.de/0868.pdf}
}
Copyright © 2022
The British Machine Vision Conference is organised by