Reading Chinese in Natural Scenes with a Bag-of-Radicals Prior
Yongbin Liu (Beihang University), Qingjie Liu (State Key Laboratory of Virtual Reality Technology and System, Beihang University, Beijing 100191, China),* Jiaxin Chen (Beihang University), Yunhong Wang (State Key Laboratory of Virtual Reality Technology and System, Beihang University, Beijing 100191, China)
The 33rd British Machine Vision Conference
Abstract
Scene text recognition (STR) on Latin datasets has been extensively studied in recent years, and state-of-the-art (SOTA) models often reach high accuracy. However, the performance on non-Latin transcripts, such as Chinese, is not satisfactory. In this paper, we collect six open-source Chinese STR datasets and evaluate a series of classic methods performing well on Latin datasets, finding a significant performance drop. To improve the performance on Chinese datasets, we propose a novel radical-embedding (RE) representation to utilize the ideographic descriptions of Chinese characters. The ideographic descriptions of Chinese characters are firstly converted to bags of radicals and then fused with learnable character embeddings by a character-vector-fusion-module (CVFM). In addition, we utilize a bag of radicals as supervision signals for multi-task training to improve the ideographic structure perception of our model. Experiments show performance of the model with RE + CVFM + multi-task training is superior compared with the baseline on six Chinese STR datasets. In addition, we utilize a bag of radicals as supervision signals for multi-task training to improve the ideographic structure perception of our model. Experiments show performance of the model with RE + CVFM + multi-task training is superior compared with the baseline on six Chinese STR datasets.Video
Citation
@inproceedings{Liu_2022_BMVC,
author = {Yongbin Liu and Qingjie Liu and Jiaxin Chen and Yunhong Wang},
title = {Reading Chinese in Natural Scenes with a Bag-of-Radicals Prior},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year = {2022},
url = {https://bmvc2022.mpi-inf.mpg.de/0969.pdf}
}
Copyright © 2022
The British Machine Vision Conference is organised by