GameCodec: Neural Cloud Gaming Video Codec
Hoang Le (Qualcomm AI),* Reza Pourreza (Qualcomm), Amir Said (Qualcomm Technologies, Inc.), Guillaume Sautiere (Qualcomm AI Research), Auke Wiggers (Qualcomm AI Research)
The 33rd British Machine Vision Conference
Abstract
We present GameCodec, the first neural video codec designed for cloud gaming: a type of online gaming where the video game is streamed from a remote server to a user’s device. This application is a challenge for video compression, as players often expect low latency and high visual quality. Additionally, gaming video is often inherently challenging to compress due to extreme camera and object motion, rich textures and visual effects. Although neural video codecs have already shown great progress on natural videos, there is so far little work on compressing gaming video. Furthermore, existing neural codecs are unable to take useful game engine information into account. In this work, we introduce a novel neural network based cloud gaming codec that leverages rendering information in an end-to-end fashion. Specifically, we introduce DMC, a decomposed motion compensation method that splits movement in the video into two separate steps: camera motion and object motion. We demonstrate the effectiveness of our method by showing substantial bit rate savings compared to both neural and classical baseline codecs.Video
Citation
@inproceedings{Le_2022_BMVC,
author = {Hoang Le and Reza Pourreza and Amir Said and Guillaume Sautiere and Auke Wiggers},
title = {GameCodec: Neural Cloud Gaming Video Codec},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year = {2022},
url = {https://bmvc2022.mpi-inf.mpg.de/0204.pdf}
}
Copyright © 2022
The British Machine Vision Conference is organised by