Hierarchical Residual Learning Based Vector Quantized Variational Autoencoder for Image Reconstruction and Generation
Mohammad Adiban (NTNU),* Kalin Stefanov (Monash University), Sabato M Siniscalchi (Norwegian University of Science and Technology), Giampiero Salvi (NTNU)
The 33rd British Machine Vision Conference
Abstract
We propose a multi-layer variational autoencoder method, we call HR-VQVAE, that learns hierarchical discrete representations of the data. By utilizing a novel objective function, each layer in HR-VQVAE learns a discrete representation of the residual from previous layers through a vector quantized encoder. Furthermore, the representations at each layer are hierarchically linked to those at previous layers. We evaluate our method on the tasks of image reconstruction and generation. Experimental results demonstrate that the discrete representations learned by HR-VQVAE enable the decoder to reconstruct high-quality images with less distortion than the baseline methods, namely VQVAE and VQVAE-2. HR-VQVAE can also generate high-quality and diverse images that outperform state-of-the-art generative models, providing further verification of the efficiency of the learned representations. The hierarchical nature of HR-VQVAE i) reduces the decoding search time, making the method particularly suitable for high-load tasks and ii) allows to increase the codebook size without incurring the codebook collapse problem.Video
Citation
@inproceedings{Adiban_2022_BMVC,
author = {Mohammad Adiban and Kalin Stefanov and Sabato M Siniscalchi and Giampiero Salvi},
title = {Hierarchical Residual Learning Based Vector Quantized Variational Autoencoder for Image Reconstruction and Generation},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year = {2022},
url = {https://bmvc2022.mpi-inf.mpg.de/0636.pdf}
}
Copyright © 2022
The British Machine Vision Conference is organised by