Efficient Self-Ensemble for Semantic Segmentation

Walid Bousselham (Oregon Health and Science University),* Guillaume Thibault (Oregon Health and Science University), Lucas Pagano (Oregon Health and Science University), Archana Machireddy (Oregon Health and Science University), Joe Gray (Oregon Health and Science University), Young Hwan Chang (Oregon Health and Science University), Xubo Song (Oregon Health and Science University)
The 33rd British Machine Vision Conference


Ensemble of predictions is known to perform better than individual predictions taken separately. However, for tasks that require heavy computational resources, e.g. semantic segmentation, creating an ensemble of learners that needs to be trained separately is hardly tractable. In this work, we propose to leverage the performance boost offered by ensemble methods to enhance semantic segmentation, while avoiding the traditional heavy training cost of the ensemble. Our self-ensemble approach takes advantage of the multi-scale features set produced by feature pyramid network methods to feed independent decoders, thus creating an ensemble within a single model. Similar to the ensemble, the final prediction is the aggregation of the prediction made by each learner. In contrast to previous works, our model can be trained end-to-end, alleviating the traditional cumbersome multi-stage training of ensembles. Our self-ensemble approach outperforms the current state-of-the-art on the benchmark datasets Pascal Context and COCO-Stuff-10K for semantic segmentation and is competitive on ADE20K and Cityscapes. Code is publicly available at https://github.com/WalBouss/SenFormer.



author    = {Walid Bousselham and Guillaume Thibault and Lucas Pagano and Archana Machireddy and Joe  Gray and Young Hwan Chang and Xubo Song },
title     = {Efficient Self-Ensemble for Semantic Segmentation},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year      = {2022},
url       = {https://bmvc2022.mpi-inf.mpg.de/0892.pdf}

Copyright © 2022 The British Machine Vision Association and Society for Pattern Recognition
The British Machine Vision Conference is organised by The British Machine Vision Association and Society for Pattern Recognition. The Association is a Company limited by guarantee, No.2543446, and a non-profit-making body, registered in England and Wales as Charity No.1002307 (Registered Office: Dept. of Computer Science, Durham University, South Road, Durham, DH1 3LE, UK).

Imprint | Data Protection