Quantitative Metrics for Evaluating Explanations of Video DeepFake Detectors

Federico Baldassarre (KTH - Royal Institute of Technology),* Quentin Debard (Huawei Ireland Research Center), Gonzalo Fiz Pontiveros (Huawei), Tri Kurniawan Wijaya (Huawei Ireland Research Centre)
The 33rd British Machine Vision Conference


The proliferation of DeepFake technology is a rising challenge in today's society, owing to more powerful and accessible generation methods. To counter this, the research community has developed detectors of ever-increasing accuracy. However, the ability to explain the decisions of such models to users is lacking behind and is considered an accessory in large-scale benchmarks, despite being a crucial requirement for the correct deployment of automated tools for content moderation. We attribute the issue to the reliance on qualitative comparisons and the lack of established metrics. We describe a simple set of metrics to evaluate the visual quality and informativeness of explanations of video DeepFake classifiers from a human-centric perspective. With these metrics, we compare common approaches to improve explanation quality and discuss their effect on both classification and explanation performance on the recent DFDC and DFD datasets.



author    = {Federico Baldassarre and Quentin Debard and Gonzalo Fiz Pontiveros and Tri Kurniawan Wijaya},
title     = {Quantitative Metrics for Evaluating Explanations of Video DeepFake Detectors},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year      = {2022},
url       = {https://bmvc2022.mpi-inf.mpg.de/0972.pdf}

Copyright © 2022 The British Machine Vision Association and Society for Pattern Recognition
The British Machine Vision Conference is organised by The British Machine Vision Association and Society for Pattern Recognition. The Association is a Company limited by guarantee, No.2543446, and a non-profit-making body, registered in England and Wales as Charity No.1002307 (Registered Office: Dept. of Computer Science, Durham University, South Road, Durham, DH1 3LE, UK).

Imprint | Data Protection