ViewNeRF: Unsupervised Viewpoint Estimation Using Category-Level Neural Radiance Fields
Octave Mariotti (University of Edinburgh),* Oisin Mac Aodha (University of Edinburgh), Hakan Bilen (University of Edinburgh)
The 33rd British Machine Vision Conference
Abstract
We introduce ViewNeRF, a Neural Radiance Field-based viewpoint estimation method that learns to predict category-level viewpoints directly from images during training. While NeRF is usually trained with ground-truth camera poses, multiple extensions have been proposed to reduce the need for this expensive supervision. Nonetheless, most of these methods still struggle in complex settings with large camera movements, and are restricted to single scenes, i.e. they cannot be trained on a collection of scenes depicting the same object category. To address this issue, our method uses an analysis by synthesis approach, combining a conditional NeRF with a viewpoint predictor and a scene encoder in order to produce self-supervised reconstructions for whole object categories. Rather than focusing on high fidelity reconstruction, we target efficient and accurate viewpoint prediction in complex scenarios, e.g. 360° rotation on real data. Our model shows competitive results on synthetic and real datasets, for single scene and multi-object collections.Video
Citation
@inproceedings{Mariotti_2022_BMVC,
author = {Octave Mariotti and Oisin Mac Aodha and Hakan Bilen},
title = {ViewNeRF: Unsupervised Viewpoint Estimation Using Category-Level Neural Radiance Fields},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year = {2022},
url = {https://bmvc2022.mpi-inf.mpg.de/0740.pdf}
}
Copyright © 2022
The British Machine Vision Conference is organised by