Information Removal at the bottleneck in Deep Neural Networks

Enzo Tartaglione (Télécom Paris - Institut Polytechnique de Paris)*
The 33rd British Machine Vision Conference


Deep learning models are nowadays broadly deployed to solve an incredibly large variety of tasks. Commonly, leveraging over the availability of “big data”, deep neural networks are trained as black boxes, minimizing an objective function at its output. This however does not allow control over the propagation of some specific features through the model, like gender or race, for solving some uncorrelated task. This raises issues either in the privacy domain (considering the propagation of unwanted information) or bias (considering that these features are potentially used to solve the given task). In these contexts, the development of a strategy specifically purposed to remove some part of the information in these models is critical. In this work, we propose IRENE, a method to achieve information removal at the bottleneck of deep neural networks, which explicitly minimizes the estimated mutual information between the features to be kept “private” and the target. Experiments on a synthetic dataset and on CelebA validate the effectiveness of the proposed approach, and open the road toward the development of approaches guaranteeing information removal in deep neural networks.



author    = {Enzo Tartaglione},
title     = {Information Removal at the bottleneck in Deep Neural Networks},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year      = {2022},
url       = {}

Copyright © 2022 The British Machine Vision Association and Society for Pattern Recognition
The British Machine Vision Conference is organised by The British Machine Vision Association and Society for Pattern Recognition. The Association is a Company limited by guarantee, No.2543446, and a non-profit-making body, registered in England and Wales as Charity No.1002307 (Registered Office: Dept. of Computer Science, Durham University, South Road, Durham, DH1 3LE, UK).

Imprint | Data Protection