zhanwang/Idempotent-Continual-Learning

<div align="center">

IDER: Idempotent Experience Replay for Reliable Continual Learning

   <!--<a href="PROJECTPAGELINK"><img src="https://img.shields.io/badge/Project-Page-green.svg" alt="Project Page"></a>-->

IDER is a novel framework for continual learning based on the idempotent property, which mitigates catastrophic forgetting and improves prediction reliability. It is a simple and robust method that can be easily integrated into other state-of-the-art approaches.

<hr/>

<p> Zhanwang Liu¹, Yuting Li¹^‡, Haoyuan Gao¹, Yexin Li⁴, Linghe Kong¹, Lichao Sun³, Weiran Huang^1,2^† </p>

<p> ¹ School of Computer Science, Shanghai Jiao Tong University&nbsp;&nbsp; ² Shanghai Innovation Institute<br/> ³ Lehigh University&nbsp;&nbsp; ⁴ State Key Laboratory of General Artificial Intelligence, BIGAI </p>

<p> ^* Equal contribution.&nbsp;&nbsp; ^† Corresponding author.&nbsp;&nbsp; ^‡ Project lead. </p>

<br/>

</div>

🎉 News

• —[x] [2026.01.26] Our paper has been accepted by ICLR 2026!

Table of Content

• —1. Quick Start
• —2. Reproduced Results
• —3. Tools
• —4. Citation
• —5. Acknowledgement

1. Quick Start

1.1. Environment

Clone this repository and install the requirements. Our model can be learnt in a single GPU RTX-4090 24G

conda env create -f environment.yaml
conda activate icl

The code was tested on Python 3.10 and PyTorch 1.13.0.

1.2. Training

Train and evaluate ResNet18 on different datasets using ER and ER+ID with different buffers. Run the following command:

<details> <summary><b>CIFAR-10</b></summary>

bash run_para_cifar10.sh

</details> <details> <summary><b>CIFAR-100</b></summary>

bash run_para_cifar100.sh

</details> <details> <summary><b>TinyImageNet</b></summary>

bash run_para_tinyimg.sh

</details>

2. Reproduced Results

The example results are ResNet18 on different datasets using ER and ER+ID as baseline methods with different buffers and 0-4 seeds. All results reported here were obtained by running experiments on an NVIDIA GeForce RTX 4090. | Dataset | Buffer | Method | Forgetting(⬇️) | TIL(⬆️) | CIL(⬆️) | Checkpoint | |---|---:|---|---:|---:|---:|---| | CIFAR-10 | 200 | ER | 59.71&nbsp;±&nbsp;2.62 | 91.48&nbsp;±&nbsp;0.93 | 48.89&nbsp;±&nbsp;2.19 | - | | | | ER+ID | 16.89&nbsp;±&nbsp;2.26 | 95.87&nbsp;±&nbsp;0.36 | 70.68&nbsp;±&nbsp;1.10 | pth | | | 500 | ER | 44.75&nbsp;±&nbsp;2.94 | 93.38&nbsp;±&nbsp;0.36 | 60.62&nbsp;±&nbsp;2.46 | - | | | | ER+ID | 11.59&nbsp;±&nbsp;2.13 | 96.20&nbsp;±&nbsp;0.40 | 75.52&nbsp;±&nbsp;1.35 | pth | | CIFAR‑100 | 500 | ER | 73.81&nbsp;±&nbsp;0.42 | 73.98&nbsp;±&nbsp;1.15 | 21.28&nbsp;±&nbsp;1.08 | - | | | | ER+ID | 32.27&nbsp;±&nbsp;1.96 | 83.30&nbsp;±&nbsp;0.41 | 45.21&nbsp;±&nbsp;1.20 | pth | | | 2000 | ER | 54.52&nbsp;±&nbsp;0.62 | 81.62&nbsp;±&nbsp;0.95 | 37.93&nbsp;±&nbsp;0.76 | - | | | | ER+ID | 18.76&nbsp;±&nbsp;1.52 | 86.54&nbsp;±&nbsp;0.34 | 56.30&nbsp;±&nbsp;0.50 | pth | | Tiny‑ImageNet | 4000 | ER | 56.89&nbsp;±&nbsp;0.74 | 66.68&nbsp;±&nbsp;0.47 | 25.20&nbsp;±&nbsp;0.70 | - | | | | ER+ID | 21.62&nbsp;±&nbsp;1.67 | 74.56&nbsp;±&nbsp;0.55 | 43.25&nbsp;±&nbsp;1.26 | pth |

The results below were obtained using an Ascend 910B. <details> <summary><b>ASCEND</b></summary>

</details>

The checkpoints are saved under experiments folder.

3. Tools

<details> <summary><b>mlflow visulization</b></summary>

1. 1.Setup

pip install mlflow

1. 1.All results are stored in mlflow under the repository mlruns. You can run mlflow ui server locally:

mlflow ui

And then go to http://127.0.0.1:5000/#/ in your brower to see all the results from the experiments we runned and exact hyperparameters used in each run.

</details>

4. Citation

If our project is helpful for your research, please consider citing :

@article{liu2026ider,
  title={IDER: IDempotent Experience Replay for Reliable Continual Learning},
  author={Liu, Zhanwang and Li, Yuting and Gao, Haoyuan and Li, Yexin and Kong, Linghe and Sun, Lichao and Huang, Weiran},
  journal={arXiv preprint arXiv:2603.00624},
  year={2026}
}

5. Acknowledgement

Supported by the Shanghai Municipal Special Program for Basic Research on General AI Foundation Models (Grant No. 2025SHZDZX025G03) and the SJTU Kunpeng \& Ascend Center of Excellence.

This project is heavily based on Mammoth and weight-interpolation-cl. We sincerely appreciate the authors of the mentioned works for sharing such great library as open-source project.

✨ Feel free to contribute and reach out if you have any questions! ✨ 📧 Email: zhanwnagliu@gmail.com