Improve model card for RLVER: Add metadata, links, abstract, and usage

README.md

@@ -1,8 +1,80 @@
 ---
+base_model:
+- Qwen/Qwen2.5-7B-Instruct
 license: other
 license_name: license
 license_link: LICENSE
-base_model:
-- Qwen/Qwen2.5-7B-Instruct
+pipeline_tag: text-generation
+library_name: transformers
 ---
-https://www.arxiv.org/abs/2507.03112
+
+# RLVER: Reinforcement Learning with Verifiable Emotion Rewards for Empathetic Agents
+
+This repository contains the model checkpoint for **RLVER: Reinforcement Learning with Verifiable Emotion Rewards for Empathetic Agents**.
+
+## Abstract
+
+Large language models (LLMs) excel at logical and algorithmic reasoning, yet their emotional intelligence (EQ) still lags far behind their cognitive prowess. While reinforcement learning from verifiable rewards (RLVR) has advanced in other domains, its application to dialogue-especially for emotional intelligence-remains underexplored. In this work, we introduce RLVER, the first end-to-end reinforcement learning framework that leverages verifiable emotion rewards from simulated users to cultivate higher-order empathetic abilities in LLMs. Within this framework, self-consistent affective simulated users engage in dialogue rollouts and produce deterministic emotion scores during conversations, serving as reward signals to guide the LLM's learning. Fine-tuning publicly available Qwen2.5-7B-Instruct model with PPO boosts its Sentient-Benchmark score from 13.3 to 79.2 while largely preserving mathematical and coding competence. Extensive experiments reveal that: (i) RLVER consistently improves multiple dialogue capabilities; (ii) Thinking and non-thinking models show distinct trends--thinking models excel in empathy and insight, while non-thinking models favor action; (iii) GRPO often yields stable gains, while PPO can push certain capabilities to a higher ceiling; (iv) More challenging environments are not always better-moderate ones can yield stronger outcomes. Our results show that RLVER is a practical route toward emotionally intelligent and broadly capable language agents.
+
+## Links
+
+*   **Paper (Hugging Face)**: [https://huggingface.co/papers/2507.03112](https://huggingface.co/papers/2507.03112)
+*   **Paper (arXiv)**: [https://arxiv.org/abs/2507.03112](https://arxiv.org/abs/2507.03112)
+*   **GitHub Repository**: [https://github.com/Tencent/digitalhuman/tree/main/RLVER](https://github.com/Tencent/digitalhuman/tree/main/RLVER)
+
+## Usage
+
+This model can be loaded and used with the Hugging Face `transformers` library for text generation and empathetic dialogue tasks.
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+import torch
+
+# Replace "RLVER/RLVER-Qwen2.5-7B-Instruct" with the actual model ID if different for this repository.
+model_id = "RLVER/RLVER-Qwen2.5-7B-Instruct" 
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+model = AutoModelForCausalLM.from_pretrained(
+    model_id,
+    torch_dtype=torch.bfloat16,
+    device_map="auto"
+)
+
+# Example for empathetic dialogue
+messages = [
+    {"role": "system", "content": "You are a helpful and empathetic assistant."},
+    {"role": "user", "content": "I'm feeling really down today. My cat just ran away."},
+]
+
+text = tokenizer.apply_chat_template(
+    messages,
+    tokenize=False,
+    add_generation_prompt=True
+)
+
+model_inputs = tokenizer(text, return_tensors="pt").to(model.device)
+
+generated_ids = model.generate(
+    model_inputs.input_ids,
+    max_new_tokens=200,
+    do_sample=True,
+    temperature=0.7,
+    top_k=50,
+    top_p=0.95
+)
+
+response = tokenizer.decode(generated_ids[0], skip_special_tokens=True)
+print(response)
+```
+
+## Citation
+
+If you find our work helpful or inspiring, please feel free to cite it using the following BibTeX entry:
+
+```bibtex
+@article{zhou2025rlver,
+  title={RLVER: Reinforcement Learning with Verifiable Emotion Rewards for Empathetic Agents},
+  author={Zhou, Yifei and Zhang, Haoran and Fang, Jieming and Tian, Wei and Zheng, Yichen and Xu, Haoyu and Hu, Bo and Zhang, Yang and Liu, Wenhai and Zhao, Wei and Lin, Li and Zhao, Xingyu and Yan, Yan},
+  journal={arXiv preprint arXiv:2507.03112},
+  year={2025}
+}
+```