EpipolarNVS: leveraging on Epipolar geometry for single-image Novel View Synthesis
Gaétan Landreau (Meero-CEA),* Mohamed Tamaazousti (CEA Saclay)
The 33rd British Machine Vision Conference
Abstract
Novel-view synthesis (NVS) can be tackled through different approaches, depending on the general setting: a single source image to a short video sequence, exact or noisy camera pose information, 3D-based information such as point clouds etc. The most challenging scenario, the one where we stand in this work, only considers a unique source image to generate a novel one from another viewpoint. However, in such a tricky situation, the latest learning-based solutions often struggle to integrate the camera viewpoint transformation. Indeed, the extrinsic information is often passed as-is, through a low-dimensional vector. It might even occur that such a camera pose, when parametrized as Euler angles, is quantized through a one-hot representation. This vanilla encoding choice prevents the learnt architecture from inferring novel views on a continuous basis (from a camera pose perspective). We claim it exists an elegant way to better encode relative camera pose, by leveraging 3D-related concepts such as the epipolar constraint. We, therefore, introduce an innovative method that encodes the viewpoint transformation as a 2D feature image. Such a camera encoding strategy gives meaningful insights to the network regarding how the camera has moved in space between the two views. By encoding the camera pose information as a finite number of coloured epilines, we demonstrate through our experiments that our strategy outperforms vanilla encoding.Video
Citation
@inproceedings{Landreau_2022_BMVC,
author = {Gaétan Landreau and Mohamed Tamaazousti},
title = {EpipolarNVS: leveraging on Epipolar geometry for single-image Novel View Synthesis},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year = {2022},
url = {https://bmvc2022.mpi-inf.mpg.de/0030.pdf}
}
Copyright © 2022
The British Machine Vision Conference is organised by