Proposal-based Temporal Action Localization with Point-level Supervision
Yuan Yin (Institute of Industrial Science, The University of Tokyo),* Yifei Huang (The University of Tokyo), Ryosuke Furuta (The University of Tokyo), Yoichi Sato (University of Tokyo)
The 34th British Machine Vision Conference
Abstract
Point-level supervised temporal action localization (PTAL) aims at recognizing and localizing actions in untrimmed videos where only a single point (frame) within every action instance is annotated in training data. Without temporal annotations, most previous works adopt the multiple instance learning (MIL) framework, where the input video is segmented into non-overlapped short snippets, and action classification is performed independently on every short snippet. We argue that the MIL framework is suboptimal for PTAL because it operates on separated short snippets that contain limited temporal information. Therefore, the classifier only focuses on several easy-to-distinguish snippets instead of discovering the whole action instance without missing any relevant snippets. To alleviate this problem, we propose a novel method that localizes actions by generating and evaluating action proposals of flexible duration that involve more comprehensive temporal information. Moreover, we introduce an efficient clustering algorithm to efficiently generate dense pseudo labels that provide stronger supervision, and a fine-grained contrastive loss to further refine the quality of pseudo labels. Experiments show that our proposed method achieves competitive or superior performance to the state-of-the-art methods and some fully-supervised methods on four benchmarks: ActivityNet 1.3, THUMOS 14, GTEA, and BEOID datasets.Citation
@inproceedings{Yin_2023_BMVC,
author = {Yuan Yin and Yifei Huang and Ryosuke Furuta and Yoichi Sato},
title = {Proposal-based Temporal Action Localization with Point-level Supervision},
booktitle = {34th British Machine Vision Conference 2023, {BMVC} 2023, Aberdeen, UK, November 20-24, 2023},
publisher = {BMVA},
year = {2023},
url = {https://bmvc2022.mpi-inf.mpg.de/BMVC2023/0820.pdf}
}
Copyright © 2023
The British Machine Vision Conference is organised by