Upload 236 files

.gitattributes

@@ -33,3 +33,13 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+resources/imgs/cl_lora.png filter=lfs diff=lfs merge=lfs -text
+resources/imgs/codaprompt.png filter=lfs diff=lfs merge=lfs -text
+resources/imgs/dap.png filter=lfs diff=lfs merge=lfs -text
+resources/imgs/flowchart.png filter=lfs diff=lfs merge=lfs -text
+resources/imgs/InfLoRA.png filter=lfs diff=lfs merge=lfs -text
+resources/imgs/LUCIR.png filter=lfs diff=lfs merge=lfs -text
+resources/imgs/moe_adapter4cl.png filter=lfs diff=lfs merge=lfs -text
+resources/imgs/praka.png filter=lfs diff=lfs merge=lfs -text
+resources/imgs/rapf.png filter=lfs diff=lfs merge=lfs -text
+resources/imgs/wa.png filter=lfs diff=lfs merge=lfs -text

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024 VIG@R&L
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -1,3 +1,178 @@
----
-license: mit
----
+<!--# LibContinual-->
+<div align="center">
+  <h1>LibContinual: Make Continual Learning Easy</h1>
+<!--   <h3>Make Continual Learning Easy</h3> -->
+    <img src="./resources/imgs/flowchart.png" width="75%"/>
+</div>
+
+## Introduction
+<div style="text-align: justify">
+LibContinual is an open-source continual learning toolbox based on PyTorch. The framework currently supports PyTorch 1.13+ (compatibility with earlier versions not fully guaranteed) and provides comprehensive implementations of state-of-the-art continual learning algorithms.
+</div>
+
+<!--![flowchart](./resources/imgs/flowchart.png)-->
+
+
+## Supported Methods
+### Conventional methods
++ [LwF (ECCV 2016)](https://github.com/RL-VIG/LibContinual/blob/master/reproduce/lwf/README.md): Learning without Forgetting.
++ [EWC (PNAS 2017)](https://github.com/RL-VIG/LibContinual/blob/master/reproduce/ewc/README.md): Overcoming catastrophic forgetting in neural networks.
++ [iCaRL (CVPR 2017)](https://github.com/RL-VIG/LibContinual/blob/master/reproduce/icarl/README.md): Incremental Classifier and Representation Learning.
++ [BiC (CVPR 2019)](https://github.com/RL-VIG/LibContinual/blob/master/reproduce/bic/README.md): Large Scale Incremental Learning.
++ [LUCIR (CVPR 2019)](https://github.com/RL-VIG/LibContinual/blob/master/reproduce/lucir/README.md): Learning a Unified Classifier Incrementally via Rebalancing.
++ [WA (CVPR 2020)](https://github.com/RL-VIG/LibContinual/blob/master/reproduce/wa/README.md): Maintaining Discrimination and Fairness in Class Incremental Learning.
++ [OCM (ICML 2022)](https://github.com/RL-VIG/LibContinual/blob/master/reproduce/ocm/README.md): Online Continual Learning through Mutual Information Maximization.
++ [ERACE, ERAML (ICLR 2022)](https://github.com/RL-VIG/LibContinual/blob/master/reproduce/erace,eraml/README.md): New Insights on reducing abrupt representation change in online continual learning.
++ [GPM (ICLR 2021)](https://github.com/RL-VIG/LibContinual/blob/master/reproduce/gpm/README.md): Gradient Projection Memory for Continual Learning.
++ [TRGP (ICLR 2022)](https://github.com/RL-VIG/LibContinual/blob/master/reproduce/trgp/README.md): Trust Region Gradient Projection for Continual Learning.
++ [API (CVPR 2023)](https://github.com/RL-VIG/LibContinual/blob/master/reproduce/api/README.md): Adaptive Plasticity Improvement for Continual Learning.
++ [RanPAC (NeurIPS 2023)](https://github.com/RL-VIG/LibContinual/blob/master/reproduce/ranpac/README.md): Random Projections and Pre-trained Models for Continual Learning.
+<!-- + [PRAKA (ICCV 2023)](https://github.com/RL-VIG/LibContinual/blob/master/reproduce/praka/README.md): Prototype Reminiscence and Augmented Asymmetric Knowledge Aggregation for Non-Exemplar Class-Incremental Learning. -->
+<!-- + [DER (CVPR 2021)](https://github.com/RL-VIG/LibContinual/blob/master/reproduce/der/README.md): Dynamically Expandable Representation for Class Incremental Learning. -->
+
+### Foundation model based methods
++ [L2P (CVPR 2022)](https://github.com/RL-VIG/LibContinual/blob/master/reproduce/l2p/README.md): Learning to Prompt for Continual Learning.
++ [DualPrompt (ECCV 2022)](https://github.com/RL-VIG/LibContinual/blob/master/reproduce/dualprompt/README.md): Complementary Prompting for Rehearsal-free Continual Learning.
++ [CodaPrompt (CVPR 2023)](https://github.com/RL-VIG/LibContinual/blob/master/reproduce/codaprompt/README.md): COntinual Decomposed Attention-based Prompting for Rehearsal-Free Continual Learning.
++ [InfLoRA (CVPR 2024)](https://github.com/RL-VIG/LibContinual/blob/master/reproduce/inflora/README.md):  Interference-Free Low-Rank Adaptation for Continual Learning.
++ [MoE_Adapter4CL (CVPR 2024)](https://github.com/RL-VIG/LibContinual/blob/master/reproduce/moe_adapter4cl/README.md): Boosting Continual Learning of Vision-Language Models via Mixture-of-Experts Adapters.
++ [RAPF (ECCV 2024)](https://github.com/RL-VIG/LibContinual/tree/master/reproduce/rapf): Class-Incremental Learning with CLIP: Adaptive Representation Adjustment and Parameter Fusion.
++ [SD_LoRA (ICLR 2025)](https://github.com/RL-VIG/LibContinual/blob/master/reproduce/sd_lora/README.md): Scalable Decoupled Low-Rank Adaptation for Class Incremental Learning
++ [LoRA_Sub_DRS (CVPR 2025)](https://github.com/RL-VIG/LibContinual/blob/master/reproduce/lora_sub_drs/README.md): LoRA Subtraction for Drift-Resistant Space in Exemplar-Free Continual Learning
++ [CL-LoRA (CVPR 2025)](https://github.com/RL-VIG/LibContinual/tree/master/reproduce/cl_lora/README.md): Continual Low-Rank Adaptation for Rehearsal-Free Class-Incremental Learning
+
+<!-- ## Quick Installation
+
+Please refer to [`install.md`](https://libcontinual.readthedocs.io/en/latest/docs/install.html) <br>
+Complete tutorials can be found at [`./docs`](https://libcontinual.readthedocs.io/en/latest/)
+-->
+
+## Installation
+Please refer to [`install.md`](https://libcontinual.readthedocs.io/en/latest/docs/install.html) <br>
+Complete tutorials can be found at [`./docs`](https://libcontinual.readthedocs.io/en/latest/)
+
+
+## Datasets
+- CIFAR-10 is avaliable at [Google Drive](https://drive.google.com/drive/folders/1sl2aW1sRpEfQJuJZwajXO2QhR06gQYZx?usp=drive_link) <br>
+- CIFAR-100 is available at [Google Drive](https://drive.google.com/drive/folders/1EL46LQ3ww-F1NVTwFDPIg-nO198cUqWm?usp=sharing)  <br>
+- CUB200, ImageNet-R, Tiny-Imagenet, 5-Dataset are available at [Google Drive](https://drive.google.com/drive/folders/16afRW2952coWJSbiH7cZT1b8pRibA8nH?usp=sharing)  <br>
+
+After the dataset is downloaded, please extract the compressed file to the specified path.
+```
+unzip cifar100.zip -d /path/to/your/dataset
+```
+Set the `data_root` in `.yaml`：
+```
+data_root: /path/to/your/dataset
+```
+To add a custom dataset, please refer to [`dataset.md`](https://libcontinual.readthedocs.io/en/latest/docs/data_module_en.html).
+
+## Getting Started
+
+Once you have completed the "Installation" and "Datasets" sections, you can now proceed to demonstrate how to use the "LibContinual" framework with the [`LUCIR`](https://github.com/RL-VIG/LibContinual/blob/master/reproduce/lucir/README.md) method. 
+
+- **Step1:** Configure the parameters in the `./config/lucir.yaml` file. Please refer to [`config.md`](https://libcontinual.readthedocs.io/en/latest/docs/config_file_en.html) for the meanings of each parameter.
+- **Step2:** Run code `python run_trainer.py --config lucir.yaml`
+- **Step3:** After the training is completed, the log files will be saved in the path specified by the `save_path` parameter.
+
+## Benchmarks
+
+We adopt standardized evaluation metrics from continual learning literature. Given T tasks where $R_{t,i}$ represents the accuracy of model after training on task $t$ when tested on task $i$:
+
+<!--
+* $T \in [1, \text{task num}]$
+* $R_{t,i}$ be accuracy of **model after training on task $t$ being test on test data of task $i$**
+* $R_{t,{i\sim j}}$ be accuracy of **model after training on task $t$ being test on test data of task $i\sim j$**
+-->
+
+### Evaluation Metrics
+
+#### **1.** Last Average Accuracy 
+$$ 
+ Acc_T=R_{T, {0\sim T}} \quad (1)
+$$
+
+
+#### **2.** Backward Transfer (BWT)
+$$ 
+ BWT_T = \frac{\sum_{i=3}^T\sum_{j=1}^{i-2}R_{i,j}-R{j,j}}{T(T-1)/2} \quad (2)
+$$
+
+#### **3.** Forgetting 
+$$ 
+ Frgt_T = \frac{\sum_{j=1}^{T-2}R_{T-1,j}-R_{j,j}}{T-1} \quad (3)
+$$
+> Equivalent to Positive BwT in ["new metrics for Continual Learning"](https://arxiv.org/pdf/1810.13166)
+
+<!--
+#### **4.** Overall Average Accuracy (Batch Version) 
+$$ 
+ \overline{Acc_T}=\frac{1}{T}\sum_{t=1}^T(R_{t,{0\sim T}}) \quad (4)
+$$
+-->
+#### **4.** Overall Average Accuracy
+$$ 
+ \overline{Acc_T}=\frac{1}{T}\sum_{t=1}^T(\frac{1}{t}\sum_{i=1}^t R_{t,i}) \quad (4)
+$$
+<!--
+> **Note**:  
+> - “Overall” and “Batch“ versions are equivalent when initial class count equals incremental class count
+
+> - Implementation follows: **Lucir** (Task), **Praka** (Batch)
+-->
+
+<!--
+### \[Batch\] Last Average Accuracy
+
+> $Acc_T=R_{T, {0\sim T}}$
+
+### Backward Transfer 
+
+> Equivalent to Positive BwT of Continuum
+> 
+> https://arxiv.org/pdf/1810.13166
+> 
+> $BWT_T = \frac{\sum_{i=3}^T\sum_{j=1}^{i-2}R_{i,j}-R{j,j}}{T(T-1)/2}$
+
+### Forgetting
+
+> Equivalent to Forgetting of Continuum
+> 
+> $Frgt_T = \frac{\sum_{j=1}^{T-2}R_{T-1,j}-R_{j,j}}{T-1}$
+
+### \[Batch\] Overall Average Accuracy
+
+> $\overline{Acc_T}=\frac{1}{T}\sum_{t=1}^T(R_{t,{0\sim T}})$
+
+### \[Task\] Overall Average Accuracy
+
+> $\overline{Acc_T}=\frac{1}{T}\sum_{t=1}^T(\frac{1}{t}\sum_{i=1}^t R_{t,i})$
+
+Note: 
+* \[Batch\] Overall Average Accuracy and \[Task\] Overall Average Accuracy are equal if initial num of classes equal to increment num of classes
+* We keep both implementation of Overall Average Accuracy as **Lucir** use \[Task\] in its original experiment, while **Praka** use \[Batch\]. 
+
+### Overall Forgetting
+
+> $\overline{Frgt_T} = \frac{1}{T-2}\sum_{t=3}^TFrgt_{t}$
+
+### Overall Backward Transfer
+
+> $\overline{BWT_T} = \frac{1}{T-2}\sum_{t=3}^TBWT_{t}$
+-->
+
+## Acknowledgement
+LibContinual is an open source project designed to help continual learning researchers quickly understand the classic methods and code structures. We welcome other contributors to use this framework to implement their own or other impressive methods and add them to LibContinual. This library can only be used for academic research. We welcome any feedback during using LibContinual and will try our best to continually improve the library.
+Special thanks to the authors of [FACIL](https://github.com/mmasana/FACIL) and [PyCIL](https://github.com/G-U-N/PyCIL) for their inspiration on framework design.
+
+<!--This project has been developed with references to the following repositories:
+
+- [FACIL](https://github.com/mmasana/FACIL)
+- [PyCIL](https://github.com/G-U-N/PyCIL)
+-->
+
+
+We have referenced useful modules from these repositories in our work. We deeply appreciate the authors of these repositories.
+
+## License
+This project is licensed under the MIT License. See LICENSE for more details.

README_ZH.md

@@ -0,0 +1,62 @@
+# LibContinual
+Make continual learning easy.
+
+## Introduction
+LibContinual is an open source continual learning toolbox based on PyTorch. The master branch works with PyTorch 1.13. The compatibility to earlier versions of PyTorch is not fully tested.
+
+![flowchart](./resources/imgs/flowchart.png)
+
+## Supported Methods
++ [BiC (CVPR 2019)](https://github.com/RL-VIG/LibContinual/blob/master/reproduce/bic/README.md)
++ [EWC (PNAS 2017)](https://github.com/RL-VIG/LibContinual/blob/master/reproduce/ewc/README.md)
++ [iCaRL (CVPR2017)](https://github.com/RL-VIG/LibContinual/blob/master/reproduce/icarl/README.md)
++ [LUCIR (CVPR 2019)](https://github.com/RL-VIG/LibContinual/blob/master/reproduce/lucir/README.md)
++ [LwF (ECCV 2016)](https://github.com/RL-VIG/LibContinual/blob/master/reproduce/lwf/README.md)
++ [WA (CVPR 2020)](https://github.com/RL-VIG/LibContinual/blob/master/reproduce/wa/README.md)
++ [OCM (PMLR 2022)](https://github.com/RL-VIG/LibContinual/blob/master/reproduce/ocm/README.md)
++ [DER (CVPR 2021)](https://github.com/RL-VIG/LibContinual/blob/master/reproduce/der/README.md)
++ [ERACE,ERAML (ICLR 2022)](https://github.com/RL-VIG/LibContinual/blob/master/reproduce/erace,eraml/README.md)
++ [L2P (CVPR 2022)](https://github.com/RL-VIG/LibContinual/blob/master/reproduce/l2p/README.md)
++ [DualPrompt (ECCV 2022)](https://github.com/RL-VIG/LibContinual/blob/master/reproduce/dualprompt/README.md)
++ [CodaPrompt (CVPR 2023)](https://github.com/RL-VIG/LibContinual/blob/master/reproduce/codaprompt/README.md)
+
+## Quick Installation
+(待文档部分完成)  <br>
+请参考文档中[`安装`](https://github.com/RL-VIG/LibContinual/blob/master/docs/tutorials/install.md)部分。 <br>
+完整文档：[`./docs`](https://github.com/RL-VIG/LibContinual/tree/master/docs)
+
+## Datasets
+[`CIFAR-100`](https://drive.google.com/drive/folders/1EL46LQ3ww-F1NVTwFDPIg-nO198cUqWm?usp=sharing), `miniImageNet(todo)`  <br>
+
+将对应数据集的压缩包解压至指定路径:
+```
+unzip cifar100.zip -d /path/to/your/dataset
+```
+修改.yaml文件的data_root参数：
+```
+data_root: /path/to/your/dataset
+```
+如何添加自定义数据集请参考文档:[`添加自定义数据集`](https://github.com/RL-VIG/LibContinual/blob/master/docs/tutorials/zh/data_module_zh.md)
+
+## Get Start
+
+当您已经完成`Quick Installation`和`Datasets`后，我们以`LUCIR`方法为例展示如何使用`LibContinual`。
+- **Step1**: 修改`run_trainer.py`中`Config`参数为`./config/lucir.yaml`
+- **Step2**：配置`./config/lucir.yaml`文件中的参数，各参数含义请参考[配置文件](https://github.com/RL-VIG/LibContinual/blob/master/docs/tutorials/config_file.md)
+- **Step3**: 运行代码`python run_trainer.py`
+- **Step4**：日志保存在配置文件中`save_path`路径下
+
+
+## Acknowledgement
+LibContinual is an open source project designed to help continual learning researchers quickly understand the classic methods and code structures. We welcome other contributors to use this framework to implement their own or other impressive methods and add them to LibContinual. This library can only be used for academic research. We welcome any feedback during using LibContinual and will try our best to continually improve the library.
+
+
+在本项目开发过程中参考了下列仓库：
+
+- [FACIL](https://github.com/mmasana/FACIL)
+- [PyCIL](https://github.com/G-U-N/PyCIL)
+
+在我们的工作中参考了这些仓库中有用的模块。我们深深感谢这些仓库的作者们。
+
+## License
+This project is licensed under the MIT License. See LICENSE for more details.

config/InfLoRA.yaml

@@ -0,0 +1,64 @@
+save_path : ""
+
+image_size: 224
+save_path: ./
+init_cls_num: 10
+inc_cls_num: 10
+task_num: 10
+val_per_epoch: 20
+
+
+epoch: 20
+n_gpu: 1
+seed: 2
+
+shuffle: false
+
+batch_size: 128
+
+
+optimizer:
+  name: Adam
+  kwargs:
+    lr: 0.0005
+    weight_decay: 0
+    betas: [0.9, 0.999]
+
+
+
+
+lr_scheduler:
+  name: CosineSchedule
+  kwargs:
+    K: 20
+
+backbone:
+  name: SiNet_vit
+  kwargs:
+    total_sessions: 10
+    rank: 10
+    init_cls: 10
+    embd_dim: 768
+
+
+buffer:
+  name: LinearBuffer
+  kwargs:
+    buffer_size: 0
+    batch_size: 128
+    strategy: herding     # random, equal_random, reservoir, herding
+
+classifier:
+  name: InfLoRA
+  kwargs:
+    feat_dim: 64
+    num_class: 100
+    inc_cls_num: 10
+    # device: 0
+    lame: 1.0
+    lamb: 0.95
+    total_sessions: 10
+
+
+
+

config/InfLoRA_opt-vit-cifar100-b10-10-10.yaml

@@ -0,0 +1,72 @@
+dataset: &dataset cifar100
+init_cls_num: &init_cls_num 10
+inc_cls_num: &inc_cls_num 10
+total_cls_num: &total_cls_num 100
+task_num: &task_num 10
+image_size: &image_size 224
+
+dataset: *dataset
+init_cls_num: *init_cls_num
+inc_cls_num: *inc_cls_num
+total_cls_num: *total_cls_num
+task_num: *task_num
+
+epoch: 20
+val_per_epoch: 20
+
+batch_size: 128
+
+setting: task-agnostic 
+seed: 1993
+
+testing_times: 1
+
+train_trfms:
+  - RandomResizedCrop:
+      size: *image_size
+  - RandomHorizontalFlip: {}
+  - ToTensor: {}
+  - Normalize:
+      mean: [0., 0., 0.]
+      std: [1., 1., 1.]
+
+test_trfms: 
+  - Resize:
+      size: *image_size
+  - ToTensor: {}
+  - Normalize:
+      mean: [0., 0., 0.]
+      std: [1., 1., 1.]
+
+optimizer:
+  name: Adam
+  kwargs:
+    lr: 0.0005
+    weight_decay: 0
+    betas: [0.9, 0.999]
+
+lr_scheduler:
+  name: CosineSchedule
+  kwargs:
+    K: 20
+
+backbone:
+  name: vit_pt_imnet
+  kwargs:
+    pretrained: True
+    model_name : vit_base_patch16_224_in21k
+    attn_layer: MultiHeadAttention_LoRA
+    lora_rank: 10
+
+classifier:
+  name: InfLoRA_OPT
+  kwargs:
+    use_ca: False
+    dataset: *dataset
+    init_cls_num: *init_cls_num
+    inc_cls_num: *inc_cls_num
+    task_num: *task_num
+    lame: 1.0
+    lamb: 0.95
+    embd_dim: 768
+    

config/InfLoRA_opt-vit-imagenetr-b20-20-10.yaml

@@ -0,0 +1,73 @@
+
+dataset: &dataset "imagenet-r"
+data_root: "/home/lvqiexuan/temp_data/imagenet-r/"
+
+init_cls_num: &init_cls_num 20
+inc_cls_num: &inc_cls_num 20
+task_num: &task_num 10
+image_size: &image_size 224
+epoch: &epoch 20
+ 
+dataset: *dataset
+task_num: *task_num
+init_cls_num: *init_cls_num
+inc_cls_num: *inc_cls_num
+epoch: *epoch
+val_per_epoch: *epoch
+
+workers: 24
+
+seed: 42
+
+batch_size: 128
+
+setting: task-agnostic 
+
+testing_times: 1
+
+train_trfms:
+  - RandomResizedCrop:
+      size: *image_size
+      scale: [0.05, 1.0]
+      ratio: [0.75, 1.333]
+  - RandomHorizontalFlip:
+      p: 0.5
+  - ToTensor: {}
+
+test_trfms: 
+  - Resize:
+      size: 256
+      interpolation: BICUBIC
+  - CenterCrop:
+      size: *image_size
+  - ToTensor: {}
+
+optimizer:
+  name: SGD
+  kwargs:
+    lr: 8e-3
+    momentum: 0.9
+
+lr_scheduler:
+  name: Constant
+
+backbone:
+  name: vit_pt_imnet
+  kwargs:
+    pretrained: True
+    model_name : vit_base_patch16_224_in21k # vit_base_patch16_224
+    attn_layer: MultiHeadAttention_LoRA
+    lora_rank: 10
+
+classifier:
+  name: InfLoRA_OPT
+  kwargs:
+    use_ca: False
+    dataset: *dataset
+    init_cls_num: *init_cls_num
+    inc_cls_num: *inc_cls_num
+    task_num: *task_num
+    lame: 1.0
+    lamb: 0.95
+    embd_dim: 768
+    

config/InfLoRA_opt.yaml

@@ -0,0 +1,71 @@
+dataset: &dataset cifar100
+init_cls_num: &init_cls_num 10
+inc_cls_num: &inc_cls_num 10
+total_cls_num: &total_cls_num 100
+task_num: &task_num 10
+image_size: &image_size 224
+
+dataset: *dataset
+init_cls_num: *init_cls_num
+inc_cls_num: *inc_cls_num
+total_cls_num: *total_cls_num
+task_num: *task_num
+
+epoch: 20
+val_per_epoch: 20
+
+batch_size: 128 # 128
+
+setting: task-agnostic 
+
+testing_times: 5
+
+train_trfms:
+  - RandomResizedCrop:
+      size: *image_size
+  - RandomHorizontalFlip: {}
+  - ToTensor: {}
+  - Normalize:
+      mean: [0., 0., 0.]
+      std: [1., 1., 1.]
+
+test_trfms: 
+  - Resize:
+      size: *image_size
+  - ToTensor: {}
+  - Normalize:
+      mean: [0., 0., 0.]
+      std: [1., 1., 1.]
+
+optimizer:
+  name: Adam
+  kwargs:
+    lr: 0.0005
+    weight_decay: 0
+    betas: [0.9, 0.999]
+
+lr_scheduler:
+  name: CosineSchedule
+  kwargs:
+    K: 20
+
+backbone:
+  name: vit_pt_imnet
+  kwargs:
+    pretrained: True
+    model_name : vit_base_patch16_224_in21k
+    attn_layer: MultiHeadAttention_LoRA
+    lora_rank: 10
+
+classifier:
+  name: InfLoRA_OPT
+  kwargs:
+    use_ca: False
+    dataset: *dataset
+    init_cls_num: *init_cls_num
+    inc_cls_num: *inc_cls_num
+    task_num: *task_num
+    lame: 1.0
+    lamb: 0.95
+    embd_dim: 768
+    

config/InfLoRA_opt_clip.yaml

@@ -0,0 +1,75 @@
+dataset: &dataset cifar100
+init_cls_num: &init_cls_num 20
+inc_cls_num: &inc_cls_num 20
+total_cls_num: &total_cls_num 100
+task_num: &task_num 5
+image_size: &image_size 224
+
+dataset: *dataset
+init_cls_num: *init_cls_num
+inc_cls_num: *inc_cls_num
+total_cls_num: *total_cls_num
+task_num: *task_num
+
+epoch: 20 # 20
+val_per_epoch: 20
+
+batch_size: 128 # 128
+
+setting: task-agnostic 
+
+testing_times: 5
+
+train_trfms:
+  - RandomResizedCrop:
+      size: *image_size
+  - RandomHorizontalFlip: {}
+  - ToTensor: {}
+  - Normalize:
+      mean: [0., 0., 0.]
+      std: [1., 1., 1.]
+
+test_trfms: 
+  - Resize:
+      size: *image_size
+  - ToTensor: {}
+  - Normalize:
+      mean: [0., 0., 0.]
+      std: [1., 1., 1.]
+
+optimizer:
+  name: Adam
+  kwargs:
+    lr: 0.0005
+    weight_decay: 0
+    betas: [0.9, 0.999]
+
+lr_scheduler:
+  name: CosineSchedule
+  kwargs:
+    K: 20
+
+
+backbone:
+  name: clip
+  kwargs:
+    pretrained : True
+    model_name : ViT-B/16
+    experts_num: 0
+    act_layer: QuickGELU
+    norm_layer: LayerNorm
+    attn_layer: MultiHeadAttention_LoRA
+
+classifier:
+  name: InfLoRA_OPT
+  kwargs:
+    use_ca: False
+    dataset: *dataset
+    init_cls_num: *init_cls_num
+    inc_cls_num: *inc_cls_num
+    task_num: *task_num
+    lame: 1.0
+    lamb: 0.95
+    embd_dim: 768
+    prompt_template : "a bad photo of a {}." # For CLIP
+    visual_only: True # For CLIP, apply lora to only visual encoder or visual and text encoder

config/PRAKA.yaml

@@ -0,0 +1,78 @@
+
+dataset: &dataset cifar100
+init_cls_num: &init_cls_num 50 # 50  50
+inc_cls_num: &inc_cls_num 10 # 5   10
+total_cls_num: &total_cls_num 100 
+task_num: &task_num 6 # 11  6
+image_size: &image_size 32
+
+image_size: *image_size
+# data
+init_cls_num: *init_cls_num  
+inc_cls_num: *inc_cls_num   
+task_num: *task_num  
+batch_size: 128 # 128
+epoch: 100 # 100
+
+val_per_epoch: 100
+seed: 2
+
+testing_times: 10 # 10
+
+train_trfms:
+  - RandomCrop: 
+      size: [*image_size, *image_size]
+      padding: 4
+  - RandomHorizontalFlip: 
+      p: 0.5
+  - ColorJitter: 
+      brightness: 0.24705882352941178
+  - ToTensor: {}
+  - Normalize:
+      mean: [0.5071, 0.4866, 0.4409] # don't change
+      std: [0.2675, 0.2565, 0.2761] # don't change
+      #mean: [0.5071, 0.4867, 0.4408]
+      #std: [0.2675, 0.2565, 0.2761]
+
+
+test_trfms: 
+  - ToTensor: {}
+  - Normalize:
+      mean: [0.5071, 0.4866, 0.4409] # don't change
+      std: [0.2675, 0.2565, 0.2761] # don't change
+      #mean: [0.5071, 0.4867, 0.4408]
+      #std: [0.2675, 0.2565, 0.2761]
+
+optimizer:
+  name: Adam
+  kwargs:
+    lr: 0.001
+    #betas: [0.9, 0.999]
+    weight_decay: 2e-4
+    #eps: 1e-8
+
+lr_scheduler:
+  name: CosineAnnealingLR
+  kwargs:
+    T_max: 32
+
+backbone:
+  name: resnet18_cbam
+  kwargs:
+    num_classes: *total_cls_num
+    args:
+      dataset: *dataset
+
+classifier:
+  name: PRAKA
+  kwargs:
+    num_class: *total_cls_num
+    init_cls_num: *init_cls_num
+    inc_cls_num: *inc_cls_num
+    feat_dim: 512
+
+    log_root: log
+    total_nc: *total_cls_num
+    protoAug_weight: 15.0
+    kd_weight: 15.0
+    temp: 0.1

config/api_til-alexnet-cifar100-b5-5-20.yaml

@@ -0,0 +1,39 @@
+init_cls_num: &init_cls_num 5
+inc_cls_num: &inc_cls_num 5
+total_cls_num: &total_cls_num 100
+task_num: &task_num 20
+
+task_num: *task_num
+init_cls_num: *init_cls_num
+inc_cls_num: *inc_cls_num
+total_cls_num: *total_cls_num
+epoch: 200
+batch_size: 64 
+val_per_epoch: 200
+
+setting: task-aware # task-aware
+seed: 2
+testing_times: 1 # Don't set too high, it will take eternity
+
+optimizer:
+  name: SGD
+  kwargs:
+    lr: 0.01
+
+lr_scheduler:
+  name: PatienceSchedule
+  kwargs:
+    patience: 6
+    factor: 2
+    stopping_lr: 1e-5
+
+backbone:
+  name: AlexNet_API
+  kwargs:
+
+classifier:
+  name: API
+  kwargs:
+    init_cls_num: *init_cls_num
+    inc_cls_num: *inc_cls_num
+    task_num: *task_num

config/backbones/CifarResnet.yaml

@@ -0,0 +1,7 @@
+backbone:
+  name: CifarResnet
+  kwargs:
+    keep_prob: 0.0
+    avg_pool: True
+    is_flatten: True
+    maxpool_last2: True

config/backbones/resnet12.yaml

@@ -0,0 +1,7 @@
+backbone:
+  name: resnet12
+  kwargs:
+    keep_prob: 0.0
+    avg_pool: True
+    is_flatten: True
+    maxpool_last2: True

config/codaprompt.yaml

@@ -0,0 +1,72 @@
+
+dataset: &dataset cifar100
+init_cls_num: &init_cls_num 10
+inc_cls_num: &inc_cls_num 10
+total_cls_num: &total_cls_num 100
+task_num: &task_num 10
+image_size: &image_size 224
+
+image_size: *image_size
+init_cls_num: *init_cls_num
+inc_cls_num: *inc_cls_num
+task_num: *task_num
+
+epoch: 20
+val_per_epoch: 20
+
+batch_size: 128
+
+train_trfms:
+  - RandomResizedCrop: 
+      size: *image_size
+  - RandomHorizontalFlip: {}
+  - ToTensor: {}
+  - Normalize:
+      mean: [0., 0., 0.]
+      std: [1., 1., 1.]
+
+test_trfms: 
+  - Resize: 
+      size: *image_size
+  - ToTensor: {}
+  - Normalize:
+      mean: [0., 0., 0.]
+      std: [1., 1., 1.]
+
+optimizer:
+  name: Adam
+  kwargs:
+    lr: 0.001
+    betas: [0.9, 0.999]
+    weight_decay: 0
+
+#lr_scheduler:
+#  name: MultiStepLR
+#  kwargs:
+#    gamma: 0.1
+#    milestones: [80, 120]
+
+lr_scheduler:
+  name: CosineSchedule
+  kwargs:
+    K: 20
+
+backbone:
+  name: vit_pt_imnet
+  kwargs:
+    num_classes: *total_cls_num
+    pretrained: true
+    model_name : vit_base_patch16_224
+
+classifier:
+  name: CodaPrompt
+  kwargs:
+    num_class: *total_cls_num
+    task_num: *task_num
+    init_cls_num: *init_cls_num
+    inc_cls_num: *inc_cls_num
+    feat_dim: 768
+    prompt_length: 8
+    pool_size: 100
+    mu: 0.0
+

config/dap.yaml

@@ -0,0 +1,74 @@
+includes:
+  - headers/data.yaml
+  - headers/device.yaml
+  - headers/model.yaml
+
+data_root: data/cifar100/
+image_size: 224
+save_path: ./
+init_cls_num: 10
+inc_cls_num: 10
+task_num: 10
+
+epoch: 5
+n_gpu: 1
+seed: 42
+val_per_epoch: 5
+
+imb_type: exp_re
+imb_factor: 0.002
+shuffle: false
+
+batch_size: 64
+
+optimizer:
+  name: Adam
+  kwargs:
+    lr: 0.01
+    eps: 1e-8
+    weight_decay: 0.0
+    betas: [0.9, 0.9]
+
+lr_scheduler:
+  name: MultiStepLR
+  kwargs:
+    gamma: 0.1
+    milestones: [80, 120]
+
+backbone:
+  name: vit_pt_imnet_dap
+  kwargs:
+    pretrained: true
+    model_name: vit_base_patch16_224_dap
+    num_classes: 100
+    drop: 0.0
+    drop_path: 0.0
+    length: 5
+    embedding_key: cls
+    prompt_key_init: uniform
+    prompt_pool: true
+    prompt_key: true
+    size: 10
+    top_k: 5
+    batchwise_prompt: true
+    head_type: prompt
+    use_prompt_mask: false
+
+classifier:
+  name: DAP
+  kwargs:
+    num_class: 100
+    feat_dim: 768
+    task_num: 10
+    init_cls_num: 10
+    inc_cls_num: 10
+    train_mask: true
+    task_inc: false
+    freeze:
+      - blocks
+      - patch_embed
+      - cls_token
+      - norm
+      - pos_embed
+    pull_constraint: true
+    pull_constraint_coeff: 0.1

config/der.yaml

@@ -0,0 +1,59 @@
+includes:
+  - headers/data.yaml
+  - headers/device.yaml
+  - headers/model.yaml
+  - headers/optimizer.yaml
+  - backbones/resnet12.yaml
+
+image_size: 32
+  
+save_path: ./
+# data
+init_cls_num: 10
+inc_cls_num: 10
+task_num: 10
+
+# init_epoch can be none
+init_epoch: 170 #200
+epoch: 170 #170
+device_ids: 2
+n_gpu: 1
+val_per_epoch: 170
+
+batch_size: 128
+
+optimizer:
+  name: SGD
+  kwargs:
+    momentum: 0.9
+    lr: 0.1
+    weight_decay: 2e-4
+
+lr_scheduler:
+  name: MultiStepLR
+  kwargs:
+    gamma: 0.1
+    milestones: [70, 100, 150]
+
+backbone:
+  name: resnet18
+  kwargs:
+    num_classes: 10
+    args: 
+      dataset: cifar100
+
+
+buffer:
+  name: LinearBuffer
+  kwargs:
+    buffer_size: 0 # 2000
+    batch_size: 32
+    strategy: random     # random, equal_random, reservoir, herding
+
+classifier:
+  name: DER
+  kwargs:
+    num_class: 100
+    feat_dim: 512
+    init_cls_num: 10
+    inc_cls_num: 10

config/dmnsp.yaml

@@ -0,0 +1,71 @@
+
+init_cls_num: &init_cls_num 5
+inc_cls_num: &inc_cls_num 5
+total_cls_num: &total_cls_num 100
+task_num: &task_num 20
+
+image_size: &image_size 224
+
+task_num: *task_num
+init_cls_num: *init_cls_num
+inc_cls_num: *inc_cls_num
+total_cls_num: *total_cls_num
+epoch: 4
+val_per_epoch: 4
+
+train_batch_size: 128
+test_batch_size: 64
+
+testing_times: 10
+
+# setting: task-agnostic # class-incremental settings
+setting: task-aware # task-incremental settings
+
+train_trfms:
+  - RandomResizedCrop :
+      size: *image_size
+      scale: [0.9, 1.0]
+      interpolation: BICUBIC
+  - ToTensor: {}
+  - Normalize:
+      mean: [0.48145466, 0.4578275, 0.40821073]
+      std: [0.26862954, 0.26130258, 0.27577711]
+
+test_trfms: 
+  - Resize :
+      size: *image_size
+      interpolation: BICUBIC
+  - ToTensor: {}
+  - Normalize:
+      mean: [0.48145466, 0.4578275, 0.40821073]
+      std: [0.26862954, 0.26130258, 0.27577711]
+
+optimizer:
+  name: AdamW
+  kwargs:
+    lr: 1e-3
+    weight_decay: 0.
+
+lr_scheduler:
+  name: CosineAnnealingWarmUp
+  kwargs:
+    T_max: 0 # Will be replaced in trainter.py with epoch * len(dataloader)
+    warmup_length: 30
+
+backbone:
+  name: clip
+  kwargs:
+    model_name : ViT-B/16
+    pretrained : True
+    block_layer: ResidualAttentionBlock_MLP
+    act_layer: QuickGELU
+    norm_layer: LayerNorm
+
+classifier:
+  name: DMNSP
+  kwargs:
+    init_cls_num: *init_cls_num
+    inc_cls_num: *inc_cls_num
+    prompt_template : "a bad photo of a {}."
+    label_smoothing: 0.
+    lamda_scale: 30

config/dmnsp_imgnr.yaml

@@ -0,0 +1,55 @@
+
+dataset: &dataset "imagenet-r"
+data_root: "/home/lvqiexuan/temp_data/imagenet-r/"
+
+init_cls_num: &init_cls_num 10
+inc_cls_num: &inc_cls_num 10
+total_cls_num: &total_cls_num 200
+task_num: &task_num 20
+image_size: &image_size 224
+
+dataset: *dataset
+task_num: *task_num
+init_cls_num: *init_cls_num
+inc_cls_num: *inc_cls_num
+total_cls_num: *total_cls_num
+epoch: 4
+val_per_epoch: 4
+
+train_batch_size: 128
+test_batch_size: 64
+
+testing_times: 10
+
+#setting: task-agnostic 
+setting: task-aware
+
+optimizer:
+  name: AdamW
+  kwargs:
+    lr: 1e-3
+    weight_decay: 0.
+
+lr_scheduler:
+  name: CosineAnnealingWarmUp
+  kwargs:
+    T_max: 0 # Will be replaced in trainter.py with epoch * len(dataloader)
+    warmup_length: 30
+
+backbone:
+  name: clip
+  kwargs:
+    model_name : ViT-B/16
+    pretrained : True
+    block_layer: ResidualAttentionBlock_MLP
+    act_layer: QuickGELU
+    norm_layer: LayerNorm
+
+classifier:
+  name: DMNSP
+  kwargs:
+    init_cls_num: *init_cls_num
+    inc_cls_num: *inc_cls_num
+    prompt_template : "a bad photo of a {}."
+    label_smoothing: 0.
+    lamda_scale: 30

config/dmnsp_vit.yaml

@@ -0,0 +1,69 @@
+init_cls_num: &init_cls_num 20
+inc_cls_num: &inc_cls_num 20
+total_cls_num: &total_cls_num 100
+task_num: &task_num 5
+image_size: &image_size 224
+
+task_num: *task_num
+init_cls_num: *init_cls_num
+inc_cls_num: *inc_cls_num
+total_cls_num: *total_cls_num
+epoch: 4 # 4
+val_per_epoch: 4 # 4
+
+train_batch_size: 128
+test_batch_size: 64
+
+testing_times: 10
+
+setting: task-agnostic 
+
+train_trfms:
+  - RandomResizedCrop :
+      size: *image_size
+      scale: [0.9, 1.0]
+      interpolation: BICUBIC
+  - ToTensor: {}
+  - Normalize:
+      mean: [0.48145466, 0.4578275, 0.40821073]
+      std: [0.26862954, 0.26130258, 0.27577711]
+
+test_trfms: 
+  - Resize :
+      size: *image_size
+      interpolation: BICUBIC
+  - ToTensor: {}
+  - Normalize:
+      mean: [0.48145466, 0.4578275, 0.40821073]
+      std: [0.26862954, 0.26130258, 0.27577711]
+
+optimizer:
+  name: AdamW
+  kwargs:
+    lr: 1e-3
+    weight_decay: 0.
+
+lr_scheduler:
+  name: CosineAnnealingWarmUp
+  kwargs:
+    T_max: 0 # Will be replaced in trainter.py with epoch * len(dataloader)
+    warmup_length: 30
+
+backbone:
+  name: vit_pt_imnet
+  kwargs:
+    pretrained: True
+    model_name : vit_base_patch16_224_in21k
+    experts_num: 1
+    act_layer: QuickGELU
+    norm_layer: LayerNorm
+
+classifier:
+  name: DMNSP
+  kwargs:
+    init_cls_num: *init_cls_num
+    inc_cls_num: *inc_cls_num
+    task_num: *task_num
+    embd_dim: 768
+    prompt_template : "a bad photo of a {}."
+    label_smoothing: 0.

config/dualprompt.yaml

@@ -0,0 +1,54 @@
+includes:
+  - headers/data.yaml
+  - headers/device.yaml
+  - headers/model.yaml
+  # - headers/optimizer.yaml
+  - backbones/resnet12.yaml
+
+image_size: 32
+
+# data
+init_cls_num: 10
+inc_cls_num: 10
+task_num: 10
+
+epoch: 10 #160
+device_ids: 0
+n_gpu: 1
+val_per_epoch: 5
+
+
+batch_size: 128
+
+
+optimizer:
+  name: Adam
+  kwargs:
+    lr: 0.001
+    betas: [0.9, 0.999]
+    weight_decay: 0
+
+lr_scheduler:
+  name: MultiStepLR
+  kwargs:
+    gamma: 0.1
+    milestones: [80, 120]
+
+backbone:
+  name: vit_pt_imnet
+  kwargs:
+    num_classes: 100
+    pretrained: true
+    model_name : vit_base_patch16_224
+
+classifier:
+  name: DualPrompt
+  kwargs:
+    num_class: 100
+    feat_dim: 768
+    task_num: 10
+    init_cls_num: 10
+    inc_cls_num: 10
+    g_prompt_length: 6
+    e_prompt_length: 20
+

config/ewc.yaml

@@ -0,0 +1,41 @@
+image_size: 32
+
+# data
+init_cls_num: 10 # 50
+inc_cls_num: 10 # 25
+task_num: 10 # 3
+
+epoch: 100 # 100
+n_gpu: 1
+val_per_epoch: 50
+
+batch_size: 128
+
+optimizer:
+  name: SGD
+  kwargs:
+    lr: 0.1
+    momentum: 0.9
+    weight_decay: 0.0005
+
+lr_scheduler:
+  name: MultiStepLR
+  kwargs:
+    gamma: 0.1
+    milestones: [60, 120, 170]
+
+backbone:
+  name: resnet34 # cifar_resnet32
+  kwargs:
+    num_classes: 100
+    args: 
+      dataset: cifar100
+
+classifier:
+  name: EWC
+  kwargs:
+    num_class: 100
+    feat_dim: 512 # 64 for backbone cifar_resnet32
+    init_cls_num: 10
+    inc_cls_num: 10
+    lamda: 1000

config/finetune.yaml

@@ -0,0 +1,53 @@
+includes:
+  - headers/data.yaml
+  - headers/device.yaml
+  - headers/model.yaml
+  - headers/optimizer.yaml
+  - backbones/resnet12.yaml
+
+data_root: /home/xiongyakun/cifar10
+image_size: 32
+  
+save_path: ./
+# data
+init_cls_num: 2
+inc_cls_num: 2
+task_num: 5
+
+
+epoch: 3
+device_ids: 5
+n_gpu: 1
+val_per_epoch: 1
+
+optimizer:
+  name: SGD
+  kwargs:
+    lr: 0.1
+
+lr_scheduler:
+  name: StepLR
+  kwargs:
+    gamma: 0.5
+    step_size: 10
+
+backbone:
+  name: resnet18
+  kwargs:
+    num_classes: 10
+    args: 
+      dataset: cifar10
+
+
+buffer:
+  name: LinearBuffer
+  kwargs:
+    buffer_size: 500
+    batch_size: 32
+    strategy: random     # random, equal_random, reservoir, herding
+
+classifier:
+  name: Finetune
+  kwargs:
+    num_class: 10
+    feat_dim: 512

config/headers/data.yaml

@@ -0,0 +1,7 @@
+save_path: ''
+dataset: cifar100 # [cifar100, binary_cifar100, ...]
+data_root: /data/lqx/cifar100
+image_size: 32
+pin_memory: False
+augment: True
+num_workers: 24

config/headers/device.yaml

@@ -0,0 +1,4 @@
+device_ids: 'auto' # determine GPU ids in bus order, auto for automatic select gpu, or number to specify device
+n_gpu: 1 # select the number of gpus to use
+seed: 1993 # random seed for numpy, torch and cuda
+deterministic: True # option for torch.backends.cudnn.benchmark and torch.backends.cudnn.deterministic

config/headers/model.yaml

@@ -0,0 +1,11 @@
+epoch: 50
+
+batch_size: 64
+val_per_epoch: 1
+
+buffer: # By default Buffer is not used, set buffer_size to 0
+  name: LinearBuffer
+  kwargs:
+    buffer_size: 0
+    batch_size: 128
+    strategy: herding # random, equal_random, reservoir, herding

config/headers/optimizer.yaml

@@ -0,0 +1,11 @@
+# optimizer info
+optimizer:
+  name: SGD
+  kwargs:
+    lr: 0.1
+  
+# lr_scheduler info
+lr_scheduler: # By Default, No LR Scheduler is used
+  name: Constant
+
+warmup: 3

config/headers/test.yaml

@@ -0,0 +1,11 @@
+# Testing Settings
+testing_times: 10 # Take average of 10 testings 
+setting: task-agnostic # Task ID is not provided during inference
+# or task-aware, provide Task ID during inference
+
+testing_per_task: True # Test data comes in per task (Each batch of test data will belongs to same task)
+# False, each batch of test data is not assure to be in same task
+# Not yet Implemented
+
+eval_with_test: True # Use Test data for in-epoch validation, If False
+# Another eval dateset will seperated from train dataset for in-epoch validation

config/icarl.yaml

@@ -0,0 +1,68 @@
+includes:
+  - headers/data.yaml
+  - headers/device.yaml
+  # - headers/model.yaml
+  # - headers/optimizer.yaml
+  # - backbones/resnet12.yaml
+
+# data_root: /data/fanzhichen/continual/cifar100
+data_root: ~/datasets/cifar100
+image_size: 32
+
+
+warmup: 3
+
+
+save_path: ./
+# data
+init_cls_num: 10
+inc_cls_num: 10
+task_num: 10
+
+
+batch_size: 128
+
+init_epoch: 200  #100
+epoch: 170  #100
+ 
+device_ids: 2
+n_gpu: 1
+val_per_epoch: 1
+
+
+optimizer:
+  name: SGD
+  kwargs:
+    lr: 0.1
+    momentum: 0.9
+    weight_decay: 0.0005
+
+lr_scheduler:
+  name: CosineAnnealingLR
+  kwargs:
+    T_max: 100
+
+
+backbone:
+  name: cifar_resnet32
+  kwargs:
+    num_classes: 100
+    args: 
+      dataset: cifar100
+
+
+buffer:
+  name: LinearHerdingBuffer
+  kwargs:
+    buffer_size: 2000
+    batch_size: 64
+    # strategy: herding     # random, equal_random, reservoir, herding
+
+classifier:
+  name: ICarl
+  kwargs:
+    num_class: 100
+    feat_dim: 64
+    init_cls_num: 10
+    inc_cls_num: 10
+    task_num: 10

config/icarl_5dataset.yaml

@@ -0,0 +1,72 @@
+includes:
+  - headers/data.yaml
+  - headers/device.yaml
+  # - headers/model.yaml
+  # - headers/optimizer.yaml
+  # - backbones/resnet12.yaml
+
+warmup: 0
+
+
+dataset: &dataset 5-datasets
+data_root: /data/Dataset/5-dataset
+class_order: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9,
+              10, 11, 12, 13, 14, 15, 16, 17, 18, 19,
+              20, 21, 22, 23, 24, 25, 26, 27, 28, 29,
+              30, 31, 32, 33, 34, 35, 36, 37, 38, 39,
+              40, 41, 42, 43, 44, 45, 46, 47, 48, 49]
+testing_times: 1
+
+save_path: ./
+# data
+init_cls_num: 10
+inc_cls_num: 10
+task_num: 5
+
+
+batch_size: 128
+
+init_epoch: 1  #100
+epoch: 1  #100
+ 
+device_ids: 2
+n_gpu: 1
+val_per_epoch: 1
+
+
+optimizer:
+  name: SGD
+  kwargs:
+    lr: 0.1
+    momentum: 0.9
+    weight_decay: 0.0005
+
+lr_scheduler:
+  name: CosineAnnealingLR
+  kwargs:
+    T_max: 100
+
+
+backbone:
+  name: cifar_resnet32
+  kwargs:
+    num_classes: 100
+    args: 
+      dataset: cifar100
+
+
+buffer:
+  name: LinearHerdingBuffer
+  kwargs:
+    buffer_size: 2000
+    batch_size: 64
+    # strategy: herding     # random, equal_random, reservoir, herding
+
+classifier:
+  name: ICarl
+  kwargs:
+    num_class: 50
+    feat_dim: 64
+    init_cls_num: 10
+    inc_cls_num: 10
+    task_num: 5

config/l2p-vit-cifar100-b10-10-10.yaml

@@ -0,0 +1,67 @@
+
+dataset: &dataset cifar100
+init_cls_num: &init_cls_num 10
+inc_cls_num: &inc_cls_num 10
+total_cls_num: &total_cls_num 100
+task_num: &task_num 10
+image_size: &image_size 224
+
+init_cls_num: *init_cls_num
+inc_cls_num: *inc_cls_num
+task_num: *task_num
+epoch: 1 # 5
+val_per_epoch: 5
+
+batch_size: 16 # Source code is 16 per device * 8 devices, since we don't use distribution device, set batch_size to 16
+testing_times: 1
+
+seed: 2
+
+train_trfms:
+  - RandomResizedCrop: 
+      size: *image_size
+      scale: [0.05, 1.0]
+      ratio: [0.75, 1.3333] # [0.75, 1.3333333333]
+      interpolation: BILINEAR
+  - RandomHorizontalFlip: 
+      p: 0.5
+  - ToTensor: {}
+
+test_trfms: 
+  - Resize:
+      size: 256 # Stated in source code of L2P
+      interpolation: BICUBIC # 3 # LANCZOS
+  - CenterCrop: 
+      size: *image_size
+  - ToTensor: {}
+
+optimizer:
+  name: Adam
+  kwargs:
+    lr: 0.001875 # 0.03
+    betas: [0.9, 0.999]
+    weight_decay: 0
+
+lr_scheduler:
+  name: Constant
+
+backbone:
+  name: vit_pt_imnet
+  kwargs:
+    num_classes: 100
+    pretrained: true
+    model_name : vit_base_patch16_224
+
+classifier:
+  name: L2P
+  kwargs:
+    init_cls_num: *init_cls_num
+    inc_cls_num: *inc_cls_num
+    num_class: *total_cls_num
+    task_num: *task_num
+    feat_dim: 768
+    prompt_length: 5 # L_p in paper
+    pool_size: 10 # M in paper
+    top_k: 5 # N in paper
+    pull_constraint_coeff: 1.0 # -0.5 in paper, 1.0 in source code
+

config/lwf.yaml

@@ -0,0 +1,42 @@
+image_size: 32
+  
+# data
+init_cls_num: 20 # 10
+inc_cls_num: 20 # 10
+task_num: 5 # 10
+
+epoch: 100 # 100
+n_gpu: 1
+val_per_epoch: 10
+
+batch_size: 128
+
+optimizer:
+  name: SGD
+  kwargs:
+    lr: 0.1
+
+lr_scheduler:
+  name: StepLR
+  kwargs:
+    gamma: 0.5
+    step_size: 30
+
+backbone:
+  name: resnet34 # resnet18
+  kwargs:
+    num_classes: 100
+    args: 
+      dataset: cifar100
+
+classifier:
+  name: LWF
+  kwargs:
+    num_class: 100
+    feat_dim: 512
+    init_cls_num: 20 #10
+    inc_cls_num: 20 #10
+    dist: 0.5
+    lamda: 10
+    K: 2
+    lw_mr: 1

config/ranpac.yaml

@@ -0,0 +1,63 @@
+init_cls_num: &init_cls_num 10
+inc_cls_num: &inc_cls_num 10
+total_cls_num: &total_cls_num 100
+task_num: &task_num 10
+image_size: &image_size 224
+
+task_num: *task_num
+init_cls_num: *init_cls_num
+inc_cls_num: *inc_cls_num
+total_cls_num: *total_cls_num
+init_epoch: 20 # 20
+epoch: 1 # 1
+batch_size: 48 # 128
+val_per_epoch: 20
+
+seed: 2
+
+train_trfms:
+  - RandomResizedCrop:
+      size: *image_size
+      scale: [0.05, 1.0]
+      ratio: [0.75, 1.33333333] # [3./4., 4./3.]
+  - RandomHorizontalFlip: 
+      p: 0.5 
+  - ToTensor: {}
+
+test_trfms: 
+  - Resize:
+      size: *image_size
+      interpolation: BICUBIC
+  - CenterCrop:
+      size: *image_size
+  - ToTensor: {}
+
+optimizer:
+  name: SGD
+  kwargs:
+    momentum: 0.9
+    lr: 0.01
+    weight_decay: 0.0005
+
+lr_scheduler:
+  name: CosineAnnealingLR
+  kwargs:
+    T_max: 20
+    eta_min: 0.0
+
+backbone:
+  name: vit_pt_imnet_in21k_adapter
+  kwargs:
+    pretrained: true
+    model_name : vit_base_patch16_224_in21k
+
+classifier:
+  name: RanPAC
+  kwargs:
+    use_RP: True
+    M: 10000
+    first_session_training: True
+    init_cls_num: *init_cls_num
+    inc_cls_num: *inc_cls_num
+    task_num: *task_num
+    total_cls_num: *total_cls_num

config/ranpac_clip.yaml

@@ -0,0 +1,66 @@
+init_cls_num: &init_cls_num 20
+inc_cls_num: &inc_cls_num 20
+total_cls_num: &total_cls_num 100
+task_num: &task_num 5
+image_size: &image_size 224
+
+task_num: *task_num
+init_cls_num: *init_cls_num
+inc_cls_num: *inc_cls_num
+total_cls_num: *total_cls_num
+init_epoch: 20 # 20
+epoch: 1 # 1
+batch_size: 48 # 128
+val_per_epoch: 20
+
+train_trfms:
+  - RandomResizedCrop:
+      size: *image_size
+      scale: [0.05, 1.0]
+      ratio: [0.75, 1.33333333] # [3./4., 4./3.]
+  - RandomHorizontalFlip: 
+      p: 0.5 
+  - ToTensor: {}
+
+test_trfms: 
+  - Resize:
+      size: *image_size
+      interpolation: BICUBIC
+  - CenterCrop:
+      size: *image_size
+  - ToTensor: {}
+
+optimizer:
+  name: AdamW
+  kwargs:
+    lr: 1e-3
+    weight_decay: 0.
+
+lr_scheduler:
+  name: CosineAnnealingWarmUp
+  kwargs:
+    T_max: 0 # Will be replaced in trainter.py with epoch * len(dataloader)
+    warmup_length: 30
+
+backbone:
+  name: clip
+  kwargs:
+    model_name : ViT-B/16
+    pretrained : True
+    block_layer: ResidualAttentionBlock_MoE_MLP
+    experts_num: 1
+    step: -1 # think again
+    act_layer: QuickGELU
+    norm_layer: LayerNorm
+
+classifier:
+  name: RanPAC
+  kwargs:
+    use_RP: True
+    M: 10000
+    first_session_training: True
+    init_cls_num: *init_cls_num
+    inc_cls_num: *inc_cls_num
+    task_num: *task_num
+    total_cls_num: *total_cls_num
+    prompt_template : "a bad photo of a {}."

config/rapf10-10.yaml

@@ -0,0 +1,74 @@
+includesL:
+  - headers/data.yaml
+  - headers/device.yaml
+  - headers/model.yaml
+
+data_root: /home/xtx/datasets/cifar100
+image_size: &image_size 224
+num_workers: &num_workers 16
+
+save_path: ./
+
+seed: &seed 1993
+
+is_rapf: True
+
+
+# Control B and 
+init_cls_num: &init_cls_num 10
+inc_cls_num: &inc_cls_num 10
+task_num: &task_num 10
+
+epoch: &epoch 15
+batch_size: &batch_size 128
+train_batch_size: &train_batch_size 100
+n_gpu: 1
+beta: &beta 2
+shrinkage: &shrinkage False
+threshold: &threshold 0.55
+val_per_epoch: &val_per_epoch 10
+
+optimizer:
+  name: Adam
+  kwargs:
+    lr: 0.001
+    weight_decay: 0.0000
+
+lr_scheduler:
+  name: MultiStepLR
+  kwargs:
+    gamma: 0.1
+    milestones: [4, 10]
+    last_epoch: -1
+
+backbone:
+  name: clip
+  kwargs:
+    model_name: ViT-B/16
+    device: cuda
+    experts_num: 1
+    block_layer: ResidualAttentionBlock_MoE_MLP
+    top_k : 1
+    step: 1
+    act_layer: QuickGELU
+    norm_layer: LayerNorm
+
+classifier:
+  name: RAPF
+  kwargs:
+    init_cls_num: *init_cls_num
+    inc_cls_num: *inc_cls_num
+    threshold: *threshold
+    beta: *beta
+    shrinkage: *shrinkage
+    train_batch_size: *train_batch_size
+    batch_size: *batch_size
+    num_workers: *num_workers
+    prompt_template: "a good photo of a {}"
+    seed: *seed
+    fp16: False
+    # class_order: *class_order
+    mix_bias: 0.6
+    feat_dim: 64
+    num_class: 100
+

config/rapf50-10.yaml

@@ -0,0 +1,72 @@
+includesL:
+  - headers/data.yaml
+  - headers/device.yaml
+  - headers/model.yaml
+
+data_root: /home/xtx/datasets/cifar100
+image_size: &image_size 224
+num_workers: &num_workers 16
+
+save_path: ./
+
+class_order: &class_order [87, 0, 52, 58, 44, 91, 68, 97, 51, 15, 94, 92, 10, 72, 49, 78, 61, 14, 8, 86, 84, 96, 18, 24, 32, 45, 88, 11, 4, 67, 69, 66, 77, 47, 79, 93, 29, 50, 57, 83, 17, 81, 41, 12, 37, 59, 25, 20, 80, 73, 1, 28, 6, 46, 62, 82, 53, 9, 31, 75, 38, 63, 33, 74, 27, 22, 36, 3, 16, 21, 60, 19, 70, 90, 89, 43, 5, 42, 65, 76, 40, 30, 23, 85, 2, 95, 56, 48, 71, 64, 98, 13, 99, 7, 34, 55, 54, 26, 35, 39]
+seed: &seed 1919810
+
+is_rapf: True
+
+
+# Control B and 
+init_cls_num: &init_cls_num 50
+inc_cls_num: &inc_cls_num 10
+task_num: &task_num 6
+
+epoch: &epoch 15
+batch_size: &batch_size 128
+train_batch_size: &train_batch_size 100
+n_gpu: 1
+beta: &beta 2
+shrinkage: &shrinkage False
+threshold: &threshold 0.55
+val_per_epoch: &val_per_epoch 10
+
+optimizer:
+  name: Adam
+  kwargs:
+    lr: 0.001
+    weight_decay: 0.0000
+
+lr_scheduler:
+  name: MultiStepLR
+  kwargs:
+    gamma: 0.1
+    milestones: [4, 10]
+    last_epoch: -1
+
+backbone:
+  name: clip
+  kwargs:
+    model_name: ViT-B/16
+    device: cuda
+    experts_num: 1
+    block_layer: ResidualAttentionBlock_MoE_MLP
+    top_k : 1
+    step: 1
+    act_layer: QuickGELU
+    norm_layer: LayerNorm
+ 
+classifier:
+  name: RAPF
+  kwargs:
+    init_cls_num: *init_cls_num
+    inc_cls_num: *inc_cls_num
+    threshold: *threshold
+    beta: *beta
+    shrinkage: *shrinkage
+    train_batch_size: *train_batch_size
+    batch_size: *batch_size
+    num_workers: *num_workers
+    prompt_template: "a good photo of a {}"
+    seed: *seed
+    fp16: False
+    class_order: *class_order
+    mix_bias: 0.6

config/rapf50-5.yaml

@@ -0,0 +1,71 @@
+includesL:
+  - headers/data.yaml
+  - headers/device.yaml
+  - headers/model.yaml
+
+data_root: /home/xtx/datasets/cifar100
+image_size: &image_size 224
+num_workers: &num_workers 16
+
+save_path: ./
+
+seed: &seed 1919810
+
+is_rapf: True
+
+
+# Control B and 
+init_cls_num: &init_cls_num 50
+inc_cls_num: &inc_cls_num 5
+task_num: &task_num 11
+
+epoch: &epoch 15
+batch_size: &batch_size 128
+train_batch_size: &train_batch_size 100
+n_gpu: 1
+beta: &beta 2
+shrinkage: &shrinkage False
+threshold: &threshold 0.65
+val_per_epoch: &val_per_epoch 10
+
+optimizer:
+  name: Adam
+  kwargs:
+    lr: 0.001
+    weight_decay: 0.0000
+
+lr_scheduler:
+  name: MultiStepLR
+  kwargs:
+    gamma: 0.1
+    milestones: [4, 10]
+    last_epoch: -1
+
+backbone:
+  name: clip
+  kwargs:
+    model_name: ViT-B/16
+    device: cuda
+    experts_num: 1
+    block_layer: ResidualAttentionBlock_MoE_MLP
+    top_k : 1
+    step: 1
+    act_layer: QuickGELU
+    norm_layer: LayerNorm
+
+classifier:
+  name: RAPF
+  kwargs:
+    init_cls_num: *init_cls_num
+    inc_cls_num: *inc_cls_num
+    threshold: *threshold
+    beta: *beta
+    shrinkage: *shrinkage
+    train_batch_size: *train_batch_size
+    batch_size: *batch_size
+    num_workers: *num_workers
+    prompt_template: "a good photo of a {}"
+    seed: *seed
+    fp16: False
+    class_order: *class_order
+    mix_bias: 0.6

config/tam.yaml

@@ -0,0 +1,70 @@
+includes:
+  - headers/data.yaml
+  - headers/device.yaml
+  - headers/model.yaml
+  # - headers/optimizer.yaml
+  - backbones/resnet12.yaml
+
+data_root: /data/fanzhichen/continual/cifar100
+image_size: 32
+
+
+save_path: ./
+# data
+init_cls_num: 20
+inc_cls_num: 20
+task_num: 5
+
+
+epoch: 0 # 160
+device_ids: 4
+n_gpu: 1
+val_per_epoch: 1
+
+
+batch_size: 128
+
+
+optimizer:
+  name: SGD
+  kwargs:
+    lr: 0.03
+    momentum: 0.9
+    weight_decay: 0.0005
+
+lr_scheduler:
+  name: MultiStepLR
+  kwargs:
+    gamma: 0.1
+    milestones: [60, 120, 170]
+
+backbone:
+  name: resnet18
+  kwargs:
+    num_classes: 100
+    args: 
+      dataset: cifar100
+
+
+buffer:
+  name: LinearBuffer
+  kwargs:
+    buffer_size: 1000
+    batch_size: 128
+    strategy: herding     # random, equal_random, reservoir, herding
+
+classifier:
+  name: TAM
+  kwargs:
+    num_class: 100
+    feat_dim: 512
+    init_cls_num: 20
+    inc_cls_num: 20
+    lamda: 1000
+    reg_weight: 0.1
+    ema_update_freq: 0.05
+    ema_alpha: 0.999
+    pairwise_weight: 0.1
+    alpha: 0.2
+    beta: 0.5
+    code_dims: 64

config/zz_BIC/bic-resnet32-5dataset-b10-10-5.yaml

@@ -0,0 +1,65 @@
+dataset: &dataset 5-datasets
+data_root: /data/Dataset/5-dataset
+class_order: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9,
+              10, 11, 12, 13, 14, 15, 16, 17, 18, 19,
+              20, 21, 22, 23, 24, 25, 26, 27, 28, 29,
+              30, 31, 32, 33, 34, 35, 36, 37, 38, 39,
+              40, 41, 42, 43, 44, 45, 46, 47, 48, 49]
+
+total_cls_num: &total_cls_num 50
+init_cls_num: &init_cls_num 10
+inc_cls_num: &inc_cls_num 10
+task_num: &task_num 5
+image_size: &image_size 32
+
+init_cls_num: *init_cls_num
+inc_cls_num: *inc_cls_num
+task_num: *task_num
+
+epoch: 250
+stage2_epoch: 250
+val_per_epoch: 50
+batch_size: 128
+
+testing_times: 1
+
+seed: 0
+
+num_workers: 0
+
+optimizer:
+  name: SGD
+  kwargs:
+    lr: 0.1
+    momentum: 0.9
+    weight_decay: 2e-4
+
+lr_scheduler:
+  name: MultiStepLR
+  kwargs:
+    gamma: 0.1
+    milestones: [100, 150, 200]
+
+# done
+backbone:
+  name: cifar_resnet32_V2 # cifar_resnet32_V2 ， resnet32 for dataset cifar100, see original paper
+  kwargs:
+    num_classes: *total_cls_num
+    args: 
+      dataset: *dataset
+
+buffer:
+  name: LinearSpiltBuffer
+  kwargs:
+    buffer_size: 1000
+    batch_size: 128
+    strategy: balance_random      # random, equal_random, reservoir, herding 
+    val_ratio: 0.1
+
+classifier:
+  name: bic
+  kwargs:
+    num_class: *total_cls_num
+    init_cls_num: *init_cls_num
+    inc_cls_num: *inc_cls_num
+    task_num: *task_num

config/zz_BIC/bic-resnet32-cifar100-b10-10-10.yaml

@@ -0,0 +1,77 @@
+
+dataset: &dataset binary_cifar100
+data_root: /home/lvqiexuan/temp_data/binary_cifar100
+
+total_cls_num: &total_cls_num 100
+init_cls_num: &init_cls_num 10
+inc_cls_num: &inc_cls_num 10
+task_num: &task_num 10
+image_size: &image_size 32
+
+init_cls_num: *init_cls_num
+inc_cls_num: *inc_cls_num
+task_num: *task_num
+
+epoch: 1 # normally 250, 1 for online setting
+stage2_epoch: 1 # normally 250, 1 for online setting
+val_per_epoch: 50
+batch_size: 10 # normally 128, 10 for online setting
+
+testing_times: 1
+
+seed: 1993
+
+num_workers: 0
+
+train_trfms:
+  - RandomHorizontalFlip: {}
+  - RandomCrop: 
+      size : *image_size
+      padding : 4
+  - ToTensor: {}
+  - Normalize:
+      mean: [0.5071, 0.4866, 0.4409]
+      std: [0.2673, 0.2564, 0.2762]
+
+test_trfms: 
+  - ToTensor: {}
+  - Normalize:
+      mean: [0.5071, 0.4866, 0.4409]
+      std: [0.2673, 0.2564, 0.2762]
+
+optimizer:
+  name: SGD
+  kwargs:
+    lr: 0.1
+    momentum: 0.9
+    weight_decay: 2e-4
+
+lr_scheduler:
+  name: MultiStepLR
+  kwargs:
+    gamma: 0.1
+    milestones: [100, 150, 200]
+
+# done
+backbone:
+  name: cifar_resnet32_V2 # cifar_resnet32_V2 ， resnet32 for dataset cifar100, see original paper
+  kwargs:
+    num_classes: *total_cls_num
+    args: 
+      dataset: *dataset
+
+buffer:
+  name: LinearSpiltBuffer
+  kwargs:
+    buffer_size: 2000
+    batch_size: 128
+    strategy: balance_random      # random, equal_random, reservoir, herding 
+    val_ratio: 0.1
+
+classifier:
+  name: bic
+  kwargs:
+    num_class: *total_cls_num
+    init_cls_num: *init_cls_num
+    inc_cls_num: *inc_cls_num
+    task_num: *task_num

config/zz_BIC/bic-resnet32-cifar100-b20-20-5.yaml

@@ -0,0 +1,77 @@
+
+dataset: &dataset binary_cifar100
+data_root: /home/lvqiexuan/temp_data/binary_cifar100
+
+total_cls_num: &total_cls_num 100
+init_cls_num: &init_cls_num 20
+inc_cls_num: &inc_cls_num 20
+task_num: &task_num 5
+image_size: &image_size 32
+
+init_cls_num: *init_cls_num
+inc_cls_num: *inc_cls_num
+task_num: *task_num
+
+epoch: 250
+stage2_epoch: 250
+val_per_epoch: 50
+batch_size: 128
+
+testing_times: 5
+
+seed: 2
+
+num_workers: 0
+
+train_trfms:
+  - RandomHorizontalFlip: {}
+  - RandomCrop: 
+      size : *image_size
+      padding : 4
+  - ToTensor: {}
+  - Normalize:
+      mean: [0.5071, 0.4866, 0.4409]
+      std: [0.2673, 0.2564, 0.2762]
+
+test_trfms: 
+  - ToTensor: {}
+  - Normalize:
+      mean: [0.5071, 0.4866, 0.4409]
+      std: [0.2673, 0.2564, 0.2762]
+
+optimizer:
+  name: SGD
+  kwargs:
+    lr: 0.1
+    momentum: 0.9
+    weight_decay: 2e-4
+
+lr_scheduler:
+  name: MultiStepLR
+  kwargs:
+    gamma: 0.1
+    milestones: [100, 150, 200]
+
+# done
+backbone:
+  name: cifar_resnet32_V2 # cifar_resnet32_V2 ， resnet32 for dataset cifar100, see original paper
+  kwargs:
+    num_classes: *total_cls_num
+    args: 
+      dataset: *dataset
+
+buffer:
+  name: LinearSpiltBuffer
+  kwargs:
+    buffer_size: 2000
+    batch_size: 128
+    strategy: balance_random      # random, equal_random, reservoir, herding 
+    val_ratio: 0.1
+
+classifier:
+  name: bic
+  kwargs:
+    num_class: *total_cls_num
+    init_cls_num: *init_cls_num
+    inc_cls_num: *inc_cls_num
+    task_num: *task_num

config/zz_BIC/bic-resnet32-imagenetr-b20-20-10.yaml

@@ -0,0 +1,59 @@
+dataset: &dataset imagenet-r
+data_root: /data/Dataset/imagenet-r
+
+total_cls_num: &total_cls_num 200
+init_cls_num: &init_cls_num 20
+inc_cls_num: &inc_cls_num 20
+task_num: &task_num 10
+
+init_cls_num: *init_cls_num
+inc_cls_num: *inc_cls_num
+task_num: *task_num
+
+epoch: 1 #250
+stage2_epoch: 1 #250
+val_per_epoch: 50
+batch_size: 10 #128
+
+testing_times: 1
+
+seed: 0
+
+num_workers: 0
+
+optimizer:
+  name: SGD
+  kwargs:
+    lr: 0.1
+    momentum: 0.9
+    weight_decay: 2e-4
+
+lr_scheduler:
+  name: MultiStepLR
+  kwargs:
+    gamma: 0.1
+    milestones: [100, 150, 200]
+
+# done
+backbone:
+  name: cifar_resnet32_V2 # cifar_resnet32_V2 ， resnet32 for dataset cifar100, see original paper
+  kwargs:
+    num_classes: *total_cls_num
+    args: 
+      dataset: *dataset
+
+buffer:
+  name: LinearSpiltBuffer
+  kwargs:
+    buffer_size: 2000
+    batch_size: 128
+    strategy: balance_random      # random, equal_random, reservoir, herding 
+    val_ratio: 0.1
+
+classifier:
+  name: bic
+  kwargs:
+    num_class: *total_cls_num
+    init_cls_num: *init_cls_num
+    inc_cls_num: *inc_cls_num
+    task_num: *task_num

config/zz_BIC/bic-resnet32-tiny-b20-20-10.yaml

@@ -0,0 +1,59 @@
+dataset: &dataset tiny-imagenet
+data_root: /data/Dataset/
+
+total_cls_num: &total_cls_num 200
+init_cls_num: &init_cls_num 20
+inc_cls_num: &inc_cls_num 20
+task_num: &task_num 10
+
+init_cls_num: *init_cls_num
+inc_cls_num: *inc_cls_num
+task_num: *task_num
+
+epoch: 1 #250
+stage2_epoch: 1 #250
+val_per_epoch: 50
+batch_size: 10 #128
+
+testing_times: 1
+
+seed: 0
+
+num_workers: 0
+
+optimizer:
+  name: SGD
+  kwargs:
+    lr: 0.1
+    momentum: 0.9
+    weight_decay: 2e-4
+
+lr_scheduler:
+  name: MultiStepLR
+  kwargs:
+    gamma: 0.1
+    milestones: [100, 150, 200]
+
+# done
+backbone:
+  name: cifar_resnet32_V2 # cifar_resnet32_V2 ， resnet32 for dataset cifar100, see original paper
+  kwargs:
+    num_classes: *total_cls_num
+    args: 
+      dataset: *dataset
+
+buffer:
+  name: LinearSpiltBuffer
+  kwargs:
+    buffer_size: 2000
+    batch_size: 128
+    strategy: balance_random      # random, equal_random, reservoir, herding 
+    val_ratio: 0.1
+
+classifier:
+  name: bic
+  kwargs:
+    num_class: *total_cls_num
+    init_cls_num: *init_cls_num
+    inc_cls_num: *inc_cls_num
+    task_num: *task_num

config/zz_CL-LoRA/cl_lora-cifar100-b5-5-20.yaml

@@ -0,0 +1,75 @@
+# cl_lora-cifar100-b5-5-20
+
+dataset: &dataset cifar100
+data_root: /data/lqx/cifar100
+init_cls_num: &init_cls_num 5
+inc_cls_num: &inc_cls_num 5
+total_cls_num: &total_cls_num 100
+task_num: &task_num 20
+image_size: &image_size 224
+epoch: &epoch 30
+
+dataset: *dataset
+init_cls_num: *init_cls_num
+inc_cls_num: *inc_cls_num
+total_cls_num: *total_cls_num
+task_num: *task_num
+
+epoch: *epoch
+val_per_epoch: *epoch
+
+batch_size: 64
+
+setting: task-agnostic 
+
+testing_times: 1
+testing_per_task: False
+
+seed: 1993
+
+train_trfms:
+  - RandomResizedCrop:
+      size: *image_size
+      scale: [0.05, 1.0]
+      ratio: [0.75, 1.3333]
+  - RandomHorizontalFlip: {}
+  - ToTensor: {}
+
+test_trfms: 
+  - Resize:
+      size: 256
+      interpolation: 3 # BICUBIC
+  - CenterCrop:
+      size: [*image_size, *image_size]
+  - ToTensor: {}
+
+optimizer:
+  name: SGD
+  kwargs:
+    lr: 0.03
+    momentum: 0.9
+    weight_decay: 0.0001
+
+lr_scheduler:
+  name: CosineAnnealingLR
+  kwargs:
+    T_max: *epoch
+    eta_min: 0
+
+backbone:
+  name: vit_cl_lora
+  kwargs:
+    pretrained: True
+    model_name : vit_base_patch16_224_in21k
+    attn_layer: MultiHeadAttention_CL_LoRA
+    transformer_layer: Transformer_CL_LoRA
+    lora_rank: 8
+    norm_layer_eps: 1e-6
+
+classifier:
+  name: CL_LoRA
+  kwargs:
+    init_cls_num: *init_cls_num
+    inc_cls_num: *inc_cls_num
+    task_num: *task_num
+    embd_dim: 768

config/zz_CL-LoRA/cl_lora-imagenetr-b5-5-40.yaml

@@ -0,0 +1,76 @@
+
+
+dataset: &dataset imagenet-r
+data_root: /data/lqx/imagenet-r
+
+init_cls_num: &init_cls_num 5
+inc_cls_num: &inc_cls_num 5
+total_cls_num: &total_cls_num 200
+task_num: &task_num 40
+image_size: &image_size 224
+epoch: &epoch 20
+
+dataset: *dataset
+init_cls_num: *init_cls_num
+inc_cls_num: *inc_cls_num
+total_cls_num: *total_cls_num
+task_num: *task_num
+
+epoch: *epoch
+val_per_epoch: *epoch
+
+batch_size: 32
+seed: 1993
+
+setting: task-agnostic 
+
+testing_times: 1
+testing_per_task: False
+
+train_trfms:
+  - RandomResizedCrop:
+      size: *image_size
+      scale: [0.05, 1.0]
+      ratio: [0.75, 1.3333]
+  - RandomHorizontalFlip: {}
+  - ToTensor: {}
+
+test_trfms: 
+  - Resize:
+      size: 256
+      interpolation: 3 # BICUBIC
+  - CenterCrop:
+      size: [*image_size, *image_size]
+  - ToTensor: {}
+
+optimizer:
+  name: SGD
+  kwargs:
+    lr: 0.05
+    momentum: 0.9
+    weight_decay: 0.0005
+
+lr_scheduler:
+  name: CosineAnnealingLR
+  kwargs:
+    T_max: *epoch
+    eta_min: 0
+
+backbone:
+  name: vit_cl_lora
+  kwargs:
+    pretrained: True
+    model_name : vit_base_patch16_224
+    attn_layer: MultiHeadAttention_CL_LoRA
+    transformer_layer: Transformer_CL_LoRA
+    lora_rank: 10
+    norm_layer_eps: 1e-6
+
+classifier:
+  name: CL_LoRA
+  kwargs:
+    init_cls_num: *init_cls_num
+    inc_cls_num: *inc_cls_num
+    task_num: *task_num
+    embd_dim: 768
+    

config/zz_ERACE/erace-resnet18-5dataset-b10-10-5.yaml

@@ -0,0 +1,46 @@
+
+
+dataset: &dataset 5-datasets
+data_root: /data/Dataset/5-dataset
+class_order: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9,
+              10, 11, 12, 13, 14, 15, 16, 17, 18, 19,
+              20, 21, 22, 23, 24, 25, 26, 27, 28, 29,
+              30, 31, 32, 33, 34, 35, 36, 37, 38, 39,
+              40, 41, 42, 43, 44, 45, 46, 47, 48, 49]
+
+total_cls_num: &total_cls_num 50
+init_cls_num: &init_cls_num 10
+inc_cls_num: &inc_cls_num 10
+task_num: &task_num 5
+image_size: &image_size 32
+
+epoch: 1
+seed: 2
+
+batch_size: 10
+testing_times: 1
+
+optimizer:
+  name: SGD
+  kwargs:
+    lr: 0.1
+
+buffer:
+  name: ERBuffer
+  kwargs:
+    capacity: 10000 # num_classes * M
+
+backbone: 
+  name: resnet18_AML
+  kwargs:
+    dataset: *dataset
+    num_classes: *total_cls_num
+
+classifier:
+  name: ERACE
+  kwargs:
+    num_classes: *total_cls_num
+    init_cls_num: *init_cls_num
+    inc_cls_num: *inc_cls_num
+    task_free: True
+    use_augs: False

config/zz_ERACE/erace-resnet18-imagenetr-b20-20-10.yaml

@@ -0,0 +1,41 @@
+
+dataset: &dataset imagenet-r
+data_root: /data/Dataset/imagenet-r
+
+total_cls_num: &total_cls_num 200
+init_cls_num: &init_cls_num 20
+inc_cls_num: &inc_cls_num 20
+task_num: &task_num 10
+image_size: &image_size 224
+
+epoch: 1
+seed: 2
+
+batch_size: 10
+testing_times: 1
+
+optimizer:
+  name: SGD
+  kwargs:
+    lr: 0.1
+
+buffer:
+  name: ERBuffer
+  kwargs:
+    capacity: 20000 # num_classes * M
+
+backbone: 
+  name: resnet18_AML
+  kwargs:
+    dataset: *dataset
+    num_classes: *total_cls_num
+    input_size: [3, *image_size, *image_size]
+
+classifier:
+  name: ERACE
+  kwargs:
+    num_classes: *total_cls_num
+    init_cls_num: *init_cls_num
+    inc_cls_num: *inc_cls_num
+    task_free: True
+    use_augs: False

config/zz_ERACE/erace.yaml

@@ -0,0 +1,58 @@
+
+dataset: &dataset binary_cifar100
+data_root: /home/lvqiexuan/temp_data/binary_cifar100
+
+total_cls_num: &total_cls_num 100
+init_cls_num: &init_cls_num 5
+inc_cls_num: &inc_cls_num 5
+task_num: &task_num 20
+image_size: &image_size 32
+
+epoch: 1
+seed: 2
+
+batch_size: 10
+testing_times: 1
+
+train_trfms:
+  - ToTensor: {}
+  - Normalize:
+      mean: [0.5, 0.5, 0.5]
+      std: [0.5, 0.5, 0.5]
+  - RandomCrop:
+      size: *image_size
+      padding: 4
+      padding_mode: constant
+      fill: -1
+  - RandomHorizontalFlip: {}
+
+test_trfms:
+  - ToTensor: {}
+  - Normalize:
+      mean: [0.5, 0.5, 0.5]
+      std: [0.5, 0.5, 0.5]
+
+optimizer:
+  name: SGD
+  kwargs:
+    lr: 0.1
+
+buffer:
+  name: ERBuffer
+  kwargs:
+    capacity: 10000 # num_classes * M
+
+backbone: 
+  name: resnet18_AML
+  kwargs:
+    dataset: *dataset
+    num_classes: *total_cls_num
+
+classifier:
+  name: ERACE
+  kwargs:
+    num_classes: *total_cls_num
+    init_cls_num: *init_cls_num
+    inc_cls_num: *inc_cls_num
+    task_free: True
+    use_augs: False

config/zz_ERAML/eraml-resnet18-5dataset-b10-10-5.yaml

@@ -0,0 +1,46 @@
+dataset: &dataset 5-datasets
+data_root: /data/Dataset/5-dataset
+class_order: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9,
+              10, 11, 12, 13, 14, 15, 16, 17, 18, 19,
+              20, 21, 22, 23, 24, 25, 26, 27, 28, 29,
+              30, 31, 32, 33, 34, 35, 36, 37, 38, 39,
+              40, 41, 42, 43, 44, 45, 46, 47, 48, 49]
+
+
+total_cls_num: &total_cls_num 50
+init_cls_num: &init_cls_num 10
+inc_cls_num: &inc_cls_num 10
+task_num: &task_num 5
+image_size: &image_size 32
+
+epoch: 1
+
+batch_size: 10
+testing_times: 1
+
+optimizer:
+  name: SGD
+  kwargs:
+    lr: 0.1
+
+buffer:
+  name: ERBuffer
+  kwargs:
+    capacity: 10000 # num_classes * M
+
+backbone: 
+  name: resnet18_AML
+  kwargs:
+    dataset: *dataset
+    num_classes: *total_cls_num
+
+classifier:
+  name: ERAML
+  kwargs:
+    num_classes: *total_cls_num
+    init_cls_num: *init_cls_num
+    inc_cls_num: *inc_cls_num
+    task_free: True
+    use_augs: False
+    supcon_temperature: 0.2
+    use_minimal_selection: False

config/zz_ERAML/eraml-resnet18-imagenetr-b20-20-10.yaml

@@ -0,0 +1,41 @@
+dataset: &dataset imagenet-r
+data_root: /data/Dataset/imagenet-r
+
+total_cls_num: &total_cls_num 200
+init_cls_num: &init_cls_num 20
+inc_cls_num: &inc_cls_num 20
+task_num: &task_num 10
+image_size: &image_size 224
+
+epoch: 1
+
+batch_size: 10
+testing_times: 1
+
+optimizer:
+  name: SGD
+  kwargs:
+    lr: 0.1
+
+buffer:
+  name: ERBuffer
+  kwargs:
+    capacity: 20000 # num_classes * M
+
+backbone: 
+  name: resnet18_AML
+  kwargs:
+    dataset: *dataset
+    num_classes: *total_cls_num
+    input_size: [3, *image_size, *image_size]
+
+classifier:
+  name: ERAML
+  kwargs:
+    num_classes: *total_cls_num
+    init_cls_num: *init_cls_num
+    inc_cls_num: *inc_cls_num
+    task_free: True
+    use_augs: False
+    supcon_temperature: 0.2
+    use_minimal_selection: False

config/zz_ERAML/eraml.yaml

@@ -0,0 +1,58 @@
+dataset: &dataset binary_cifar100
+total_cls_num: &total_cls_num 100
+init_cls_num: &init_cls_num 5
+inc_cls_num: &inc_cls_num 5
+task_num: &task_num 20
+image_size: &image_size 32
+
+data_root: /home/lvqiexuan/temp_data/binary_cifar100
+
+epoch: 1
+
+batch_size: 10
+testing_times: 1
+
+train_trfms:
+  - ToTensor: {}
+  - Normalize:
+      mean: [0.5, 0.5, 0.5]
+      std: [0.5, 0.5, 0.5]
+  - RandomCrop:
+      size: *image_size
+      padding: 4
+      padding_mode: constant
+      fill: -1
+  - RandomHorizontalFlip: {}
+
+test_trfms:
+  - ToTensor: {}
+  - Normalize:
+      mean: [0.5, 0.5, 0.5]
+      std: [0.5, 0.5, 0.5]
+
+optimizer:
+  name: SGD
+  kwargs:
+    lr: 0.1
+
+buffer:
+  name: ERBuffer
+  kwargs:
+    capacity: 10000 # num_classes * M
+
+backbone: 
+  name: resnet18_AML
+  kwargs:
+    dataset: *dataset
+    num_classes: *total_cls_num
+
+classifier:
+  name: ERAML
+  kwargs:
+    num_classes: *total_cls_num
+    init_cls_num: *init_cls_num
+    inc_cls_num: *inc_cls_num
+    task_free: True
+    use_augs: False
+    supcon_temperature: 0.2
+    use_minimal_selection: False