SVS: Adversarial refinement for sparse novel view synthesis
Violeta Menéndez González (University of Surrey),* Andrew Gilbert (University of Surrey), Graeme Phillipson (BBC), Stephen Jolly (BBC), Simon Hadfield (University of Surrey)
The 33rd British Machine Vision Conference
Abstract
This paper proposes Sparse View Synthesis. This is a view synthesis problem where the number of reference views is limited, and the baseline between target and reference view is significant. Under these conditions, current radiance field methods fail catastrophically due to inescapable artifacts such 3d floating blobs, blurring and structural duplication, whenever the number of reference views is limited, or the target view diverges significantly from the reference views. Advances in network architecture and loss regularisation are unable to satisfactorily remove these artifacts. The occlusions within the scene ensure that the true contents of these regions is simply not available to the model. In this work, we instead focus on hallucinating plausible scene contents within such regions. To this end we unify radiance field models with adversarial learning and perceptual losses. The resulting system provides up to 60% improvement in perceptual accuracy compared to current state-of-the-art radiance field models on this problem.Video
Citation
@inproceedings{menendez2022svs,
author = {Menéndez González, Violeta and
Gilbert, Andrew and
Phillipson, Graeme and
Jolly, Stephen and
Hadfield, Simon},
title = {SVS: Adversarial refinement for sparse novel view synthesis},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year = {2022},
url = {https://bmvc2022.mpi-inf.mpg.de/0886.pdf}
}
Copyright © 2022
The British Machine Vision Conference is organised by