Enhancing Material Features Using Dynamic Backward Attention on Cross-Resolution Patches
Yuwen Heng (University of Southampton),* Yihong Wu (University of Southampton), Srinandan Dasmahapatra (University of Southampton), Hansung Kim (University Of Southampton)
The 33rd British Machine Vision Conference
Abstract
Recent studies in material segmentation crop the image into patches to force the network to learn material features from local visual clues. This design is based on the expectation that the contextually invariant features can generalise the network to unseen images regardless of the object or scene in which the material appears. However, most approaches set a fixed patch resolution for all the images in a dataset, which does not consider the varying areas that materials cover within and across images due to the scene scale. As a consequence, the fixed patch resolution can limit the performance of networks. In consideration of this problem, this paper proposes a Dynamic Backward Attention Transformer (DBAT) to extract features from cross-resolution patches and dynamically aggregate these features based on per-pixel attention masks. Experiments show that DBAT achieves the best performance among state-of-the-art models (86.85% in average pixel accuracy, which is 2.15% higher than the second-best model) that can serve real-time inference. Moreover, we also illustrate the network behaviour through visualisation methods as well as descriptive statistics. The project code is available at https://github.com/heng-yuwen/Dynamic-Backward-Attention-Transformer.Video
Citation
@inproceedings{Heng_2022_BMVC,
author = {Yuwen Heng and Yihong Wu and Srinandan Dasmahapatra and Hansung Kim},
title = {Enhancing Material Features Using Dynamic Backward Attention on Cross-Resolution Patches},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year = {2022},
url = {https://bmvc2022.mpi-inf.mpg.de/0004.pdf}
}
Copyright © 2022
The British Machine Vision Conference is organised by