Towards Self-Supervised Gaze Estimation
Arya Farkhondeh (Sapienza University of Rome),* Cristina Palmero (Universitat de Barcelona), Simone Scardapane (Sapienza University), Sergio Escalera (Computer Vision Center (UAB) & University of Barcelona,)
The 33rd British Machine Vision Conference
Abstract
Recent joint embedding-based self-supervised methods have surpassed standard supervised approaches on various image recognition tasks such as image classification. These self-supervised methods aim at maximizing agreement between features extracted from two differently transformed views of the same image, which results in learning an invariant representation with respect to appearance and geometric image transformations. However, the effectiveness of these approaches remains unclear in the context of gaze estimation, a structured regression task that requires equivariance under geometric transformations (e.g., rotations, horizontal flip). In this work, we propose SwAT, an equivariant version of the online clustering-based self-supervised approach SwAV, to learn more informative representations for gaze estimation. We demonstrate that SwAT, with ResNet-50 and supported with uncurated unlabeled face images, outperforms state-of-the-art gaze estimation methods and supervised baselines in various experiments. In particular, we achieve up to 57% and 25% improvements in cross-dataset and within-dataset evaluation tasks on existing benchmarks (ETH-XGaze, Gaze360, and MPIIFaceGaze).Video
Citation
@inproceedings{Farkhondeh_2022_BMVC,
author = {Arya Farkhondeh and Cristina Palmero and Simone Scardapane and Sergio Escalera},
title = {Towards Self-Supervised Gaze Estimation},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year = {2022},
url = {https://bmvc2022.mpi-inf.mpg.de/0549.pdf}
}
Copyright © 2022
The British Machine Vision Conference is organised by