Online continual learning with saliency-guided experience replay using tiny episodic memory

Abstract

Artificial learning systems aspire to mimic human intelligence by continually learning from a stream of tasks without forgetting past knowledge. One way to enable such learning is to store past experiences in the form of input examples in episodic memory and replay them when learning new tasks. However, performance of such method suffers as the size of the memory becomes smaller. In this paper, we propose a new approach for experience replay, where we select the past experiences by looking at the saliency maps, which provide visual explanations for the model’s decision. Guided by these saliency maps, we pack the memory with only the parts or patches of the input images important for the model’s prediction. While learning a new task, we replay these memory patches with appropriate zero-padding to remind the model about its past decisions. We evaluate our algorithm on CIFAR-100, miniImageNet and CUB datasets and report better performance than the state-of-the-art approaches. We perform a detailed study to show the effectiveness of zero-padded patch replay compared to the other candidate approaches. Moreover, with qualitative and quantitative analyses we show that our method captures richer summaries of past experiences without any memory increase and hence performs well with small episodic memory.

Appendices

Appendix A: Algorithm

Pseudocode for the episodic memory (${\mathcal {M}}_E$) update algorithm in EPR is given in Algorithm 2. For each example, I in ${\mathcal {M}}_T$, we generate the corresponding saliency map, $I_{sm}$ using the chosen saliency method, $\texttt {XAI}$ (Line 5). Then, we identify a square region (of size $W_p\times W_p$) in that map that has the maximum average intensity and find the corresponding image patch, $I_p$ (Lines 6–7). We then zero-pad the image patch and record the model prediction on it (Lines 8–11). We also record the true image label and image patch coordinates (Lines 12–14). Given the memory budget ($|{\mathcal {M}}_E|$) and EPF, we then perform the memory selection procedure (Line 16), where memory patches are selected in priority of their correct predictions after zero-padding (see Sect. 4 for details). Each selected image patch is then added to ${\mathcal {M}}_E$ with corresponding task-ID, class label, and localizable coordinates in the original image (Lines 17–19).

Appendix B: Grad-CAM

Gradient-weighted Class Activation Mapping (Grad-CAM) [17] is a saliency method that uses gradients to determine the impact of specific feature map activations on a given prediction. Since later layers in the convolutional neural network capture high-level semantics [65], taking gradients of a model output with respect to the feature map activations from one such layers identifies which high-level semantics are important for the model prediction. In our analysis, we select this layer and refer to as target layer [48].

Let’s consider the target layer has M feature maps where each feature map, $A^m \in {\mathbb {R}}^{u\times v}$ is of width u and height v. Also consider, for a given image ($I \in {\mathbb {R}}^{W\times H\times C}$) belonging to class c, the pre-softmax score of the image classifier is $y_c$. To obtain the class-discriminative saliency map, Grad-CAM first takes derivative of $y_c$ with respect to each feature map $A^m$. These gradients are then global-average-pooled over u and v to obtain importance weight, $\alpha _m^c$ for each feature map:

$$\begin{aligned} \alpha _m^c = \frac{1}{uv} \sum _{i=1}^u\sum _{j=1}^v \frac{\partial y_c}{\partial A_{ij}^m}, \end{aligned}$$

(B1)

where $A^m_{ij}$ denotes location (i, j) in the feature map $A^m$. Next, these weights are used for computing linear combination of the feature map activations, which is then followed by ReLU to obtain the localization map:

$$\begin{aligned} L^c_{{\textit{Grad-CAM}}} = \text {ReLU}\left( \sum _{m=1}^M \alpha _m^cA^m\right) \end{aligned}$$

(B2)

This map is of the same size ($u\times v$) of $A^m$. Finally, saliency map, $I_{sm} \in {\mathbb {R}}^{W\times H}$, is generated by upsampling $L^c_{{\textit{Grad-CAM}}}$ to the input image resolution using bilinear interpolation.

$$\begin{aligned} I_{sm} = \texttt {Upsample}~(L^c_{{\textit{Grad-CAM}}}) \end{aligned}$$

(B3)