Contrastive Learning for Controllable Blind Video Restoration

Givi Meishvili (Computer Vision Group - Computer Science Department - University of Bern), Abdelaziz Djelouah (Disney Research),* Shinobu Hattori (Disney), Christopher Schroers (DisneyResearch|Studios)
The 33rd British Machine Vision Conference


A lot of progress has been made since the first neural network models were trained for specific image restoration tasks, such as super-resolution and denoising. Recently multi-degradation models have been proposed, allowing for user control of the restoration process needed for real-world applications. However, this aspect is most powerful if the initial restoration can be done as best as possible in a blind setting. In parallel to this line of work, other methods can target the blind setting where, for example, in the case of super-resolution, the blur kernel is estimated for conditioning the restoration part. In particular, discriminative learning has played a key role in pushing the state of the art. Still, the learned representation cannot be interpreted or manipulated and remains a black box that doesn't offer any possibility for user-guided correction. This work addresses those issues through a representation learning pipeline that helps separate content from degradation by reasoning on pairs of degraded patches. The degradation representation is used as conditioning for a video restoration model that can denoise and upscale to arbitrary resolutions and remove film scratches. Finally, the learned representation can be mutated to fine-tune the restoration results. We demonstrate state-of-the-art results compared to the most recent video super-resolution and denoising methods.



author    = {Givi Meishvili and Abdelaziz Djelouah and Shinobu Hattori and Christopher Schroers},
title     = { Contrastive Learning for Controllable Blind Video Restoration},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year      = {2022},
url       = {}

Copyright © 2022 The British Machine Vision Association and Society for Pattern Recognition
The British Machine Vision Conference is organised by The British Machine Vision Association and Society for Pattern Recognition. The Association is a Company limited by guarantee, No.2543446, and a non-profit-making body, registered in England and Wales as Charity No.1002307 (Registered Office: Dept. of Computer Science, Durham University, South Road, Durham, DH1 3LE, UK).

Imprint | Data Protection