Contrastive Learning for Controllable Blind Video Restoration
Givi Meishvili (Computer Vision Group - Computer Science Department - University of Bern), Abdelaziz Djelouah (Disney Research),* Shinobu Hattori (Disney), Christopher Schroers (DisneyResearch|Studios)
The 33rd British Machine Vision Conference
Abstract
A lot of progress has been made since the first neural network models were trained for specific image restoration tasks, such as super-resolution and denoising. Recently multi-degradation models have been proposed, allowing for user control of the restoration process needed for real-world applications. However, this aspect is most powerful if the initial restoration can be done as best as possible in a blind setting. In parallel to this line of work, other methods can target the blind setting where, for example, in the case of super-resolution, the blur kernel is estimated for conditioning the restoration part. In particular, discriminative learning has played a key role in pushing the state of the art. Still, the learned representation cannot be interpreted or manipulated and remains a black box that doesn't offer any possibility for user-guided correction. This work addresses those issues through a representation learning pipeline that helps separate content from degradation by reasoning on pairs of degraded patches. The degradation representation is used as conditioning for a video restoration model that can denoise and upscale to arbitrary resolutions and remove film scratches. Finally, the learned representation can be mutated to fine-tune the restoration results. We demonstrate state-of-the-art results compared to the most recent video super-resolution and denoising methods.Video
Citation
@inproceedings{Meishvili_2022_BMVC,
author = {Givi Meishvili and Abdelaziz Djelouah and Shinobu Hattori and Christopher Schroers},
title = { Contrastive Learning for Controllable Blind Video Restoration},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year = {2022},
url = {https://bmvc2022.mpi-inf.mpg.de/0974.pdf}
}
Copyright © 2022
The British Machine Vision Conference is organised by