Performance Limiting Factors of Deep Neural Networks for Pedestrian Detection
Yasin Bayzidi (Volkswagen AG),* Alen Smajic (Volkswagen AG), Jan David Schneider (Volkswagen AG), Fabian Hüger (Volkswagen AG), Ruby Moritz (Volkswagen), Alois C. Knoll (Robotics and Embedded Systems)
The 33rd British Machine Vision Conference
Abstract
Deep Neural Networks (DNN) for perception in automated driving have been extensively studied, while achieving strong results in detection performance on pre-annotated test sets. However, there has been a gap in the literature on a systematic analysis of DNN behavior to investigate the factors contributing to their misbehavior. As part of DNN safety, we propose to both analyze DNN behavior in challenging scenarios as well as the respective factors that actually contribute to their misbehavior. Although some of such factors have been studied individually, there is not a thorough study to compare all together in a systematic manner to unveil the impact of each factor leading to DNN failures. In this paper, we propose an approach to evaluate the DNN performance limiting factors~(PLF), and their contribution to the DNN misbehavior. Accordingly, we analyze seventeen factors from the literature, introduce four novel factors and conduct an assessment on all of them to assess their potential as a PLF. Furthermore, we evaluate our results based on six state-of-the-art pedestrian detection DNN including three detection tasks. For our experiments, we study a synthetic as well as a real-world dataset for pedestrian detection. We show that there exist various similarities and dissimilarities when comparing the PLF from a synthetic dataset to a real one, and discuss the causes and effects of such relations. Furthermore, we provide an approach to analyze the common factors from both real-world as well as synthetic datasets which might have similar effects on various DNN performance.Video
Citation
@inproceedings{Bayzidi_2022_BMVC,
author = {Yasin Bayzidi and Alen Smajic and Jan David Schneider and Fabian Hüger and Ruby Moritz and Alois C. Knoll},
title = {Performance Limiting Factors of Deep Neural Networks for Pedestrian Detection},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year = {2022},
url = {https://bmvc2022.mpi-inf.mpg.de/0883.pdf}
}
Copyright © 2022
The British Machine Vision Conference is organised by