Towards Efficient Neural Scene Graphs by Learning Consistency Fields
Yeji Song (Seoul National University), Chaerin Kong (Seoul National University), Seo Young Lee (Seoul National University), Nojun Kwak (Seoul National University),* Joonseok Lee (Google Research & Seoul National University)
The 33rd British Machine Vision Conference
Abstract
Neural Radiance Fields (NeRF) achieves photo-realistic image rendering from novel views, and the Neural Scene Graphs (NSG) extends it to dynamic scenes (video) with multiple objects. Nevertheless, computationally heavy ray marching for every image frame becomes a huge burden. In this paper, taking advantage of significant redundancy across adjacent frames in videos, we propose a feature-reusing framework. From the first try of naively reusing the NSG features, however, we learn that it is crucial to disentangle object-intrinsic properties consistent across frames from transient ones. Our proposed method, Consistency-Field-based NSG (CF-NSG), reformulates neural radiance fields to additionally consider consistency fields. With disentangled representations, CF-NSG takes full advantage of the feature-reusing scheme and performs an extended degree of scene manipulation in a more controllable manner. We empirically verify that CF-NSG greatly improves the inference efficiency by using 85% less queries than NSG without notable degradation in rendering quality. Code will be available at: https://github.com/ldynx/CF-NSGVideo
Citation
@inproceedings{Song_2022_BMVC,
author = {Yeji Song and Chaerin Kong and Seo Young Lee and Nojun Kwak and Joonseok Lee},
title = {Towards Efficient Neural Scene Graphs by Learning Consistency Fields},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year = {2022},
url = {https://bmvc2022.mpi-inf.mpg.de/0302.pdf}
}
Copyright © 2022
The British Machine Vision Conference is organised by