IronDepth: Iterative Refinement of Single-View Depth using Surface Normal and its Uncertainty
Gwangbin Bae (University of Cambridge),* Ignas Budvytis (Department of Engineering, University of Cambridge), Roberto Cipolla (University of Cambridge)
The 33rd British Machine Vision Conference
Abstract
Single image surface normal estimation and depth estimation are closely related problems as the former can be calculated from the latter. However, the surface normals computed from the output of depth estimation methods are significantly less accurate than the surface normals directly estimated by networks. To reduce such discrepancy, we introduce a novel framework that uses surface normal and its uncertainty to recurrently refine the predicted depth-map. The depth of each pixel can be propagated to a query pixel, using the predicted surface normal as guidance. We thus formulate depth refinement as a classification of choosing the neighboring pixel to propagate from. Then, by propagating to sub-pixel points, we upsample the refined, low-resolution output. The proposed method shows state-of-the-art performance on NYUv2 and iBims-1 - both in terms of depth and normal. Our refinement module can also be attached to the existing depth estimation methods to improve their accuracy. We also show that our framework, only trained for depth estimation, can also be used for depth completion. The code is available at https://github.com/baegwangbin/IronDepth.Video
Citation
@inproceedings{Bae_2022_BMVC,
author = {Gwangbin Bae and Ignas Budvytis and Roberto Cipolla},
title = {IronDepth: Iterative Refinement of Single-View Depth using Surface Normal and its Uncertainty},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year = {2022},
url = {https://bmvc2022.mpi-inf.mpg.de/0476.pdf}
}
Copyright © 2022
The British Machine Vision Conference is organised by