RLVMR: Reinforcement Learning with Verifiable Meta-Reasoning Rewards for Robust Long-Horizon Agents
Paper • 2507.22844 • Published
qwen3-4b-rlvmr-alfworld-v6
LoRA adapter trained with RLVMR (GRPO-MR) from Kaito-F/qwen3-4b-sft-v13-mixed-rlvmr.
Method: RLVMR (Reinforcement Learning with Verifiable Meta-Reasoning Rewards)
Based on RLVMR paper (arXiv:2507.22844):
- Dense meta-reasoning rewards for planning, exploration, reflection, monitoring
- GRPO-MR advantage: alpha=0.5 (outcome + tag-level grouping)
- Format reward penalty for missing meta-reasoning tags
Rollout Statistics
- Total trajectories: 300
- Overall success: 43.7%
- Tasks: all 6 ALFWorld types
- K=6 per environment, max_steps=30
| Task | Success |
|---|---|
| put | 31/48 (64.6%) |
| clean | 25/60 (41.7%) |
| heat | 6/30 (20.0%) |
| cool | 7/36 (19.4%) |
| examine | 28/54 (51.9%) |
| puttwo | 34/72 (47.2%) |
Training Configuration
- Base model: Kaito-F/qwen3-4b-sft-v13-mixed-rlvmr
- LoRA: r=16, alpha=32
- LR: 5e-06, Epochs: 1
- KL coefficient: 0.01
- Downloads last month
- -
Model tree for Kaito-F/qwen3-4b-rlvmr-alfworld-v6.1
Base model
Qwen/Qwen3-4B-Instruct-2507