Blind Removal of Facial Foreign Shadows

Yaojie Liu (Google Research), Andrew Z Hou (Michigan State University),* Xinyu Huang (BOSCH Research North America), Liu Ren (BOSCH Research North America), Xiaoming Liu (Michigan State University)
The 33rd British Machine Vision Conference


In-the-wild face photographs often suffer from undesirable foreign shadows cast by external objects (e.g. hands, phones, and trees). Removing facial foreign shadows not only improves image aesthetics but also mitigates the negative impacts on face-related tasks. This paper tackles the blind removal of facial foreign shadows for both single images and videos by making three contributions. First, we propose a novel two-stage shadow modeling algorithm that consists of gray-scale shadow removal and colorization. This decomposition provides an effective way to handle both color distortion and subsurface scattering effects. Second, we propose a novel Temporal Sharing Module (TSM) to extract hierarchical features across multiple aligned video frames, which represent the shadow-free faces. Third, we collect a real face database with 280 videos captured under highly dynamic environments and annotate pixel-level shadow segmentation maps. Extensive experiments demonstrate that our approach outperforms state-of-the-art methods quantitatively and qualitatively. Code, our database, and pre-trained models are publicly available at



author    = {Yaojie Liu and Andrew Z Hou and Xinyu Huang and Liu Ren and Xiaoming Liu},
title     = {Blind Removal of Facial Foreign Shadows},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year      = {2022},
url       = {}

Copyright © 2022 The British Machine Vision Association and Society for Pattern Recognition
The British Machine Vision Conference is organised by The British Machine Vision Association and Society for Pattern Recognition. The Association is a Company limited by guarantee, No.2543446, and a non-profit-making body, registered in England and Wales as Charity No.1002307 (Registered Office: Dept. of Computer Science, Durham University, South Road, Durham, DH1 3LE, UK).

Imprint | Data Protection