> [!IMPORTANT]
> 🌟 Stay up to date at [opendrivelab.com](https://opendrivelab.com/#news)!
R2SE: Reinforced Refinement with Self-Aware Expansion for End-to-End Autonomous Driving
[](https://arxiv.org/abs/2506.09800)
> - [Haochen Liu](https://scholar.google.com/citations?user=iizqKUsAAAAJ&hl), [Tianyu Li](https://scholar.google.com/citations?user=X6vTmEMAAAAJ&hl=zh-CN), Haohan Yang, Li Chen, Caojun Wang, Ke Guo, Haochen Tian, Hongchen Li, Hongyang Li and Chen Lv
> - [Paper](https://arxiv.org/abs/2506.09800)
> - If you have any questions, please feel free to contact: *Haochen Liu* ( haochen002@e.ntu.edu.sg )
---
**[2025-11]** Initial project released.
## Overview
**R2SE** is a RL finetuning framework targeting hard cases and forgetting in End-to-end Autonmous Driving;
## Get Started
### Data Preparation
We follow the PARA-Drive pipeline in formating train/test data.
```
python scripts/data_converter/nuplan/e2e_nuplan_openscenes_navsim_multiprocessing.py
```
### R2SE-GP Training
Following the pipeline in UniAD using specific configs in ```config/paradrive```
1. Pretrain the perception / backbone:
```
configs/paradrive/exp_100pct/e2e_r2se_gp_perception.py
```
2. Pretrain the full R2SE-GP:
```
configs/paradrive/exp_100pct/e2e_r2se_gp.py
```
### R2SE RL Finetuning
1. Hard case sampling: User may rerun the navsim inference and sort for hard case tokens, we provide some preprocessed ones in ```/data_yaml``` for your convenience.
2. RL Finetuning:
```
configs/paradrive/exp_100pct/e2e_r2se.py
```
After finetuning, run the testing script for both ```R2SE-GP``` and ```R2SE```, you are expected to get two ```.pkl``` for saved token and planning traj.
3. OOD Inference:
Load ```.pkl``` from ```R2SE``. Run this script to get the switched planning results
```
python ood_inference.py
```
### Testing
run the script in ```navsim```, ensure you load all the cached data and inference result of ```R2SE-GP``` in ```.csv``` format for PDMS and Forgetting metric calculations.
### Weights
All weights are provided in ```unknownuser6666/R2SE_weights```
## TODO List
- [x] Initial release
- [] Code Reorganization
## Citation
If you find the project helpful for your research, please consider citing our paper:
```bibtex
@article{liu2025reinforced,
title={Reinforced Refinement with Self-Aware Expansion for End-to-End Autonomous Driving},
author={Liu, Haochen and Li, Tianyu and Yang, Haohan and Chen, Li and Wang, Caojun and Guo, Ke and Tian, Haochen and Li, Hongchen and Li, Hongyang and Lv, Chen},
journal={arXiv preprint arXiv:2506.09800},
year={2025}
}
```
