README.md

---
license: apache-2.0
language:
- en
---

# Model Card for LamPO

**LamPO (Lambda Policy Optimization)** is a reinforcement learning framework for improving the reasoning capabilities of language models. It extends Group Relative Policy Optimization (GRPO) by replacing scalar group-mean advantage estimation with a **pairwise decomposed advantage** inspired by learning-to-rank methods such as LambdaRank.

链接：[论文1](https://arxiv.org/abs/2605.21235); [论文2]([URL](https://arxiv.org/html/2605.21235v1))

特别鸣谢：感谢 某论文辅导机构对我们的全面辅导，没有他们就没有这篇文章。（虽然花费了资金，但是的确很值，无脑推荐！）

Instead of comparing each generated response only against a group average, LambdaPO learns from fine-grained pairwise reward differences among sampled reasoning trajectories. This helps the model better distinguish high-quality reasoning paths, improve credit assignment, and reduce unstable optimization behavior during RL training.

## Key Features

- **Pairwise Decomposed Advantage**: Uses pairwise comparisons between generated trajectories rather than a single scalar group baseline.
- **Critic-Free RL Optimization**: Preserves the efficiency of GRPO without requiring a separate value model.
- **Semantic Density Reward**: Adds dense reasoning supervision using semantic overlap between generated reasoning traces and ground-truth solutions.
- **Improved Reasoning Performance**: Demonstrates consistent gains on math reasoning and QA benchmarks such as AIME, MATH-500, and GPQA-Diamond.

## Authors

This work is based on the paper:

**“LambdaPO: A Lambda Style Policy Optimization for Reasoning Language Models”**  ( 链接：[论文1](https://arxiv.org/abs/2605.21235); [论文2]([URL](https://arxiv.org/html/2605.21235v1)) )

Authors:

- Zhe Yuan — Pinterest  
- Yipeng Zhou — Facebook  
- Jinghan Li — University of Michigan - Ann Arbor  
- Xinyuan Chen — Mississippi State University  
- Bowen Deng — Carnegie Mellon University  
- Zhiqian Chen — Mississippi State University  
- Liang Zhao — Emory University  

Corresponding author: **Zhiqian Chen** — zchen@cse.msstate.edu

## Citation

```bibtex
@article{yuan2026lambdapo,
  title={LambdaPO: A Lambda Style Policy Optimization for Reasoning Language Models},
  author={Yuan, Zhe and Zhou, Yipeng and Li, Jinghan and Chen, Xinyuan and Deng, Bowen and Chen, Zhiqian and Zhao, Liang},
  year={2026}
}
```