Imagining Hidden Supporting Objects using Volumetric Conditional GANs and Differentiable Stability Scores
Hector Basevi (University of Birmingham),* Ales Leonardis (University of Birmingham)
The 33rd British Machine Vision Conference
Abstract
Objects supporting the physical stability of an unstructured heap of items are often heavily or completely occluded by the objects that they are supporting. Identifying plausible supporting object candidates and their poses from visual information is challenging because there may be many candidates and it is not practical to exhaustively verify each one using physical simulation. We present a generative system which predicts the complete volumetric structure of a heap of objects from visible depth and semantic information. We leverage 3D conditional Wasserstein generative adversarial networks to perform this task and inject differentiable context about physical stability from a second network trained to score the physical stability of object heaps. We demonstrate that our system is capable of generating physically stable heaps from visual information, and that the use of both generative models and context about physical stability are crucial in replicating the true distribution of hidden objects. We train and evaluate our system using a novel simulation-based dataset which we also present in this work.Video
Citation
@inproceedings{Basevi_2022_BMVC,
author = {Hector Basevi and Ales Leonardis},
title = {Imagining Hidden Supporting Objects using Volumetric Conditional GANs and Differentiable Stability Scores},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year = {2022},
url = {https://bmvc2022.mpi-inf.mpg.de/0682.pdf}
}
Copyright © 2022
The British Machine Vision Conference is organised by