| --- |
| license: mit |
| pipeline_tag: text-generation |
| --- |
| |
| # Solve the Loop: Attractor Models for Language and Reasoning |
|
|
| This repository contains the Attractor-140M model presented in [Solve the Loop: Attractor Models for Language and Reasoning](https://huggingface.co/papers/2605.12466). |
|
|
| [**Project Page**](https://attractor-models.github.io/) | [**GitHub**](https://github.com/jacobfa/Attractor) | [**Paper**](https://arxiv.org/abs/2605.12466) |
|
|
| ## Introduction |
|
|
| Attractor Models are a family of models that use a backbone module to propose output embeddings and an attractor module to refine them by solving for a fixed point through implicit differentiation. This architecture allows training memory to remain constant relative to effective depth and enables iterations to be chosen adaptively. In language modeling, Attractor Models deliver a Pareto improvement over standard Transformers and stable looped models across sizes. |
|
|
| ## Sample Usage |
|
|
| To use this model, you first need to install the package from the [official repository](https://github.com/jacobfa/Attractor): |
|
|
| ```bash |
| pip install -e . |
| ``` |
|
|
| Then you can initialize the model as follows: |
|
|
| ```python |
| from attractor.models.attractor import Attractor, AttractorConfig |
| |
| config = AttractorConfig.from_name("attractor-small-140m") |
| model = config.construct_model() |
| ``` |
|
|
| ## Citation |
|
|
| ```bibtex |
| @article{feinashley2026attractor, |
| title={Solve the Loop: Attractor Models for Language and Reasoning}, |
| author={Fein-Ashley, Jacob and Rashidinejad, Paria}, |
| year={2026}, |
| url={https://arxiv.org/abs/2605.12466} |
| } |
| ``` |
