Semantics-Adding Flaw-Erasing Network for Semantic Human Matting

Jiayu Sun (Dalian University of Technology),* Zhanghan Ke (City University of Hong Kong), Ke Xu (City University of Hong Kong), Fan Shao ( Wonxing Technology), Lihe Zhang (Dalian University of Technology), Huchuan Lu (Dalian University of Technology), Rynson W.H. Lau (City University of Hong Kong)
The 33rd British Machine Vision Conference


Trimap-free semantic human matting is a very challenging task. While using a single model to directly predict an alpha matte can be extremely unreliable, some latest works propose to predict a segmentation mask or a pseudo trimap as a priori for the matting task. However, errors in the priori can significantly affect the subsequent prediction. Motivated by recent flaw-based correction approaches, we propose a different approach to the problem here. We first predict an initial alpha matte with a single model. Guided by a flaw detector to check for potential prediction errors, we then correct the errors in the initial alpha matte with a refinement process to produce the output matte. To this end, we propose a semantics-adding flaw-erasing network (SAFE-Net) with two novel modules, a semantic addition module (SAM) to enhance the human semantics using an attention mechanism and a flaw elimination module (FEM) to refine the details of the predicted defective regions of the initial matte. Experimental results show that SAFE-Net outperforms existing trimap-free human matting methods notably. Finally, we have created a large human matting dataset containing 4,729 unique foregrounds with fine annotations, which will be made available to the public.



author    = {Jiayu Sun and Zhanghan Ke and Ke Xu and Fan Shao and Lihe Zhang and Huchuan Lu and Rynson W.H. Lau},
title     = {Semantics-Adding Flaw-Erasing Network for Semantic Human Matting},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year      = {2022},
url       = {}

Copyright © 2022 The British Machine Vision Association and Society for Pattern Recognition
The British Machine Vision Conference is organised by The British Machine Vision Association and Society for Pattern Recognition. The Association is a Company limited by guarantee, No.2543446, and a non-profit-making body, registered in England and Wales as Charity No.1002307 (Registered Office: Dept. of Computer Science, Durham University, South Road, Durham, DH1 3LE, UK).

Imprint | Data Protection