A Simple Plugin for Transforming Images to Arbitrary Scales
Qinye Zhou (Shanghai Jiao Tong University), Ziyi Li (Shanghai Jiao Tong University), Weidi Xie (Shanghai Jiao Tong University),* Xiaoyun Zhang (Shanghai Jiao Tong University), Yan-Feng Wang (Cooperative medianet innovation center of Shanghai Jiao Tong University), Ya Zhang (Cooperative Medianet Innovation Center, Shang hai Jiao Tong University)
The 33rd British Machine Vision Conference
Abstract
Existing models on super-resolution often specialized for one scale, fundamentally limiting their use in practical scenarios. In this paper, we aim to develop a general plugin that can be inserted into existing super-resolution models, conveniently augmenting their ability towards Arbitrary Resolution Image Scaling, thus termed ARIS. We make the following contributions: (i) we propose a transformer-based plugin module, which uses spatial coordinates as query, iteratively attend the low-resolution image feature through cross-attention, and output visual feature for the queried spatial location, resembling an implicit representation for images; (ii) we introduce a novel self-supervised training scheme, that exploits cycle-consistency to effectively augment the model's ability for upsampling images towards unseen scales, i.e. ground-truth high-resolution images are not available; (iii) without loss of generality, we inject the proposed ARIS plugin module into several existing models, namely, IPT, SwinIR, and HAT, showing that the resulting models can not only maintain their original performance on fixed scale factor but also extrapolate to unseen scales, substantially outperforming existing any-scale super-resolution models on standard benchmarks, e.g. Urban100, DIV2K, etc.Video
Citation
@inproceedings{Zhou_2022_BMVC,
author = {Qinye Zhou and Ziyi Li and Weidi Xie and Xiaoyun Zhang and Yan-Feng Wang and Ya Zhang},
title = {A Simple Plugin for Transforming Images to Arbitrary Scales},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year = {2022},
url = {https://bmvc2022.mpi-inf.mpg.de/0107.pdf}
}
Copyright © 2022
The British Machine Vision Conference is organised by