A Memory Transformer Network for Incremental Learning
Ahmet Iscen (Google), Thomas Bird (UCL),* Mathilde Caron (Facebook Artificial Intelligence Research), Alireza Fathi (Google), Cordelia Schmid (Google)
The 33rd British Machine Vision Conference
Abstract
We study class-incremental learning, a training setup in which new classes of data are observed over time for the model to learn from. Despite the straightforward problem formulation, the naive application of classification models to class-incremental learning results in the ``catastrophic forgetting" of previously seen classes. Most successful existing methods has been the use of a memory of exemplars, which overcomes the issue of catastrophic forgetting by saving a subset of past data into a memory bank and utilizing it to prevent forgetting when training future tasks. In our paper, we propose to enhance the utilization of this memory bank: we not only use it as a source of additional training data like existing works but also integrate it in the prediction process explicitly. Our method, the Memory Transformer Network (MTN), learns how to combine and aggregate the information from the nearest neighbors in the memory with a transformer to make more accurate predictions. We conduct extensive experiments and ablations to evaluate our approach. We show that MTN achieves state-of-the-art performance on the challenging ImageNet-1k and Google-Landmarks-1k incremental learning benchmarks.Video
Citation
@inproceedings{Iscen_2022_BMVC,
author = {Ahmet Iscen and Thomas Bird and Mathilde Caron and Alireza Fathi and Cordelia Schmid},
title = {A Memory Transformer Network for Incremental Learning},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year = {2022},
url = {https://bmvc2022.mpi-inf.mpg.de/0388.pdf}
}
Copyright © 2022
The British Machine Vision Conference is organised by