Region-of-Interest Based Neural Video Compression
Yura M. Perugachi-Diaz (Vrije Universiteit Amsterdam), Guillaume Sautiere (Qualcomm AI Research), Davide Abati (Qualcomm AI Research),* Yang Yang (Google LLC), Amirhossein Habibian (Qualcomm AI Research), Taco S. Cohen (Qualcomm AI Research)
The 33rd British Machine Vision Conference
Abstract
Humans do not perceive all parts of a scene with the same resolution, but rather focus on few regions of interest (ROIs). Traditional Object-Based codecs take advantage of this biological intuition, and are capable of non-uniform allocation of bits in favor of salient regions, at the expense of increased distortion the remaining areas: such a strategy allows a boost in perceptual quality under low rate constraints. Recently, several neural codecs have been introduced for video compression, yet they operate uniformly over all spatial locations, lacking the capability of ROI-based processing. In this paper, we introduce two models for ROI-based neural video coding. First, we propose an implicit model that is fed with a binary ROI mask and it is trained by de-emphasizing the distortion of the background. Secondly, we design an explicit latent scaling method, that allows control over the quantization binwidth for different spatial regions of latent variables, conditioned on the ROI mask. By extensive experiments, we show that our methods outperform all our baselines in terms of Rate-Distortion (R-D) performance in the ROI. Moreover, they can generalize to different datasets and ROI specifications at inference time. Finally, they do not require expensive pixel-level annotations during training, as synthetic ROI masks can be used with little to no degradation in performance. To the best of our knowledge, our proposals are the first solutions that integrate ROI-based capabilities into neural video compression models.Video
Citation
@inproceedings{Perugachi-Diaz_2022_BMVC,
author = {Yura M. Perugachi-Diaz and Guillaume Sautiere and Davide Abati and Yang Yang and Amirhossein Habibian and Taco S. Cohen},
title = {Region-of-Interest Based Neural Video Compression},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year = {2022},
url = {https://bmvc2022.mpi-inf.mpg.de/0288.pdf}
}
Copyright © 2022
The British Machine Vision Conference is organised by