Model Card for r1-qwen-7b-gold-60k-alpha50

This model is a fine-tuned version of deepseek-ai/DeepSeek-R1-Distill-Qwen-7B. It has been trained using TRL.

Quick start

from transformers import pipeline

question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
generator = pipeline("text-generation", model="None", device="cuda")
output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
print(output["generated_text"])

Training procedure

This model was trained with GOLD.

Framework versions

• TRL: 0.28.0.dev0
• Transformers: 4.57.6
• Pytorch: 2.9.0
• Datasets: 4.5.0
• Tokenizers: 0.22.2

Citations

Cite GOLD as:

@misc{patino2025unlocking,
    title        = {{Unlocking On-Policy Distillation for Any Model Family}},
    author       = {Carlos Miguel Patiño and Kashif Rasul and Quentin Gallouédec and Ben Burtenshaw and Sergio Paniego and Vaibhav Srivastav and Thibaud Frere and Ed Beeching and Lewis Tunstall and Leandro von Werra and Thomas Wolf},
    year         = 2025,
    url          = {https://huggingface.co/spaces/HuggingFaceH4/general-on-policy-logit-distillation},
}

Cite TRL as:

@software{vonwerra2020trl,
  title   = {{TRL: Transformers Reinforcement Learning}},
  author  = {von Werra, Leandro and Belkada, Younes and Tunstall, Lewis and Beeching, Edward and Thrush, Tristan and Lambert, Nathan and Huang, Shengyi and Rasul, Kashif and Gallouédec, Quentin},
  license = {Apache-2.0},
  url     = {https://github.com/huggingface/trl},
  year    = {2020}
}