Quantifying Attribution-based Explainable AI for Robustness Evaluations

Zusammenfassung

On the one hand, Deep Neural Networks (DNNs) create new opportunities in many digitisation applications such as complex Object Detection (OD) in autonomous driving. On the other hand, the black-box properties of DNNs pose a serious challenge for their application in safety and security-critical domains, where liability and trustworthiness are key requirements. Because formal verification of DNNs is only feasible in very restricted settings, one generally has to rely on empirical robustness tests. Here we investigate the use of explainable artificial intelligence (XAI) to improve such robustness evaluations. Integrated gradients (IG) is an attribution-based XAI method and provides heatmaps that indicate the relevance of inputs for predictions. In contrast to many contributions on improving and comparing XAI techniques, this paper proposes two interpretability metrics to add XAI methods to empirical trustworthiness evaluations of DNNs. In this way XAI becomes usable in real-time applications. In another contribution, we introduce the integration of these metrics into a robustness testing framework where a model trained on the Audi Autonomous Driving Dataset (A2D2) is evaluated with perturbed images that mimic naturally occurring inputs.