File size: 3,009 Bytes
55f1404 4424cfd 55f1404 4424cfd 55f1404 e239c1f 4424cfd 55f1404 4424cfd 55f1404 4424cfd 617cfd3 55f1404 4424cfd 55f1404 4424cfd 55f1404 4424cfd 55f1404 4424cfd | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 | ---
base_model:
- Qwen/Qwen3-8B-Base
datasets:
- HuggingFaceFW/fineweb-edu
license: apache-2.0
model_name: Qwen3_8B_LoopUS
tags:
- LoopUS
- LoopedTransformer
pipeline_tag: text-generation
---
<div align="center">
<h1>LoopUS: <br> Recasting Pretrained LLMs into Looped Latent Refinement Models</h1>
</div>
<p align="center">
<a href="https://pnubaelab.github.io/"><b>BAELAB</b></a>, Pusan National University, Busan, Korea <br>
<a href="https://aidoheekim.github.io/"><b>DOLAB</b></a>, Changwon National University, Changwon, Korea
</p>
<p align="center">
<a href="https://thrillcrazyer.github.io/" target="_blank"><strong>Taekhyun Park</strong></a><sup>1</sup>,
<a href="https://yongzzai.com/" target="_blank"><strong>Yongjae Lee</strong></a><sup>1</sup>,
<a href="https://aidoheekim.github.io/" target="_blank"><strong>Dohee Kim</strong></a><sup>2</sup>,
<a href="https://pnubaelab.github.io/" target="_blank"><strong>Hyerim Bae</string></a><sup>1,†</sup>
</p>
<p align="center">
<a href="https://github.com/Thrillcrazyer/LoopUS"><b>🌟 Github</b></a> |
<a href="https://thrillcrazyer.github.io/LoopUS"><b>🔗 Project Page</b></a> |
<a href="https://arxiv.org/abs/2605.11011"><b>📄 Paper</b></a>
</p>
# Abstract
Looped computation shows promise in improving the reasoning-oriented performance of LLMs by scaling test-time compute. **Looped Depth Up-Scaling** (LoopUS) is a post-training framework that converts a standard pretrained LLM into a looped architecture. LoopUS recasts the pretrained LLM into an encoder, a looped reasoning block, and a decoder. This mechanism transforms a standard non-looped model into a looped form while stabilizing it against both computational bottlenecks and representation collapse. Through stable latent looping, LoopUS improves reasoning-oriented performance without extending the generated traces or requiring recurrent training from scratch.
# QuickStart
To use this model, please follow the installation and usage instructions from the [official repository](https://github.com/Thrillcrazyer/LoopUS).
### Installation
```bash
git clone https://github.com/Thrillcrazyer/LoopUS.git
cd LoopUS
uv sync
```
### Chat Mode
You can run the model in chatting mode using the following command:
```bash
uv run chat.py --model-name Thrillcrazyer/Qwen3_8B_LoopUS
```
### Qualitative Generation
```bash
uv run LoopUS-generate \
--model-name Qwen/Qwen3-8B-Base \
--decomposed-model Thrillcrazyer/Qwen3_8B_LoopUS \
--prompt "The meaning of life is" \
--n-recursion 8
```
# Illustration of LoopUS
<div align="center">
<img src="https://raw.githubusercontent.com/Thrillcrazyer/LoopUS/main/assets/Framework.png" width="800"/>
</div>
# Citation
```bibtex
@misc{park2024loopus,
title={LoopUS: Recasting Pretrained LLMs into Looped Latent Refinement Models},
author={Taekhyun Park and Yongjae Lee and Dohee Kim and Hyerim Bae},
year={2024},
eprint={2605.11011},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
``` |