Enhancing Performance of Occlusion-Based Explanation Methods by a Hierarchical Search Method on Input Images

Abstract

In this work, we address some drawbacks of back-propagation-based and perturbation-based visualization methods by proposing an explanation method called Fast Multi-resolution Occlusion (FMO). FMO, opposite to the back-propagation-based methods that cannot be applied on all types of Convolutional Neural Networks (CNNs), can highlight the important input features independent of the architecture. Also, FMO introduces a novel fast occlusion strategy called multi-resolution occlusion which not only efficiently addresses the time-consumption issue of the traditional Occlusion Test method but also outperforms the well-known perturbation-based methods. We assess the methods on CNNs DenseNet121, InceptionV3, InceptionResnetV2, MobileNet, and ResNet50 using three datasets ILSVRC2012, PASCAL VOC07, and COCO14.

Appendices

Appendix A: More Details of the Equations

The index j in Eq. (3) indicates the index of elements in the probability matrix $P_{n_i*n_i}^{R_i}$. Also Z is the probability of the original unoccluded image I belonged to the class index Y, and $\hat{Z}$ also shows the probability belonged to the class index Y, but when the occluded image I has been passed through the model. Therefore, to record the changes in the output probability, opposite to the Occlusion Test method that just records the output probability of occluded image, we record the normalized change of probability. As a result, each cell $[h_i^j.w_i^j ]$ in the probability matrix $P_{n_i*n_i}^{R_i}$ indicates the normalized change of probability pertaining to a region of original image. The value in this cell shows the importance of that region in the form of normalized change of probability.

$\gamma ^R_i$ in Eq. (4) indicates the probability matrix weight in the resolution $R_i$. In order to see the heatmap in each resolution $R_i$, the weight of the resolution $R_i$ is set to 1 and the weight of the others is set to 0. In this equation, before performing the weighted sum, all probability matrix $P_{n_i*n_i}^{R_i}$ are resized to the shape of the original image.

Appendix B: Details of Time Consumption

Table 1 shows the average time consumption of the FMO, RISE, LIME, Extremal Perturbation and Meaningful Perturbation methods on the models DenseNet121, InceptionV2, Inception V3, MobileNet, ResNet50. As can be seen, the proposed method, FMO, had the lowest time consumption over all models in comparison to the other methods, whereas Occlusion Test method had the highest time consumption. For example, FMO takes 1.90 s, 4.86 s, 2.71 s, 0.59 s and 2.70 s. On models DenseNet121, InceptionResNetV2, InceptionV3, MobileNet and ResNet50, respectively, which are far less than those of the Occlusion Test, RISE, LIME, Extremal Perturbation and Meaningful Perturbation methods in all of the five models.

Table 1. Average time consumption of the FMO, RISE, LIME, Extremal Perturbation and Meaningful Perturbation methods

Appendix C: Details of Visual Accuracy

Table 2 shows the localization accuracy of the methods on DenseNet121 and ResNet50. As can be seen, FMO outperforms other methods in terms of localization accuracy and time consumption on two datasets VOC07 and COCO14.

Table 2. Localized accuracy of each method on two hard datasets.

Figure 2 and 3 illustrate the visualization results on two datasets VOC07 and COCO14. According to these figures, FMO and Meaningful Perturbation methods can highlight properly the regions of interest in comparison to other methods.

Fig. 2.

The visualization output of the methods with the PASCAL VOC dataset on model DenseNet121.

Fig. 3.

The visualization output of the methods with COCO dataset on the ResNet50 model.