| --- |
| license: mit |
| pipeline_tag: text-generation |
| --- |
| |
| # Solve the Loop: Attractor Models for Language and Reasoning |
|
|
| This repository contains the 770M parameter Attractor Model introduced in [Solve the Loop: Attractor Models for Language and Reasoning](https://huggingface.co/papers/2605.12466). |
|
|
| Attractor Models offer a promising alternative to purely feed-forward computation by iteratively refining latent representations. In this architecture, a backbone module first proposes output embeddings, then an attractor module refines them by solving for the fixed point using implicit differentiation. This model delivers a Pareto improvement over standard Transformers in language modeling and demonstrates strong performance on reasoning tasks. |
|
|
| - **Paper:** [Solve the Loop: Attractor Models for Language and Reasoning](https://huggingface.co/papers/2605.12466) |
| - **Project Page:** [attractor-models.github.io](https://attractor-models.github.io/) |
| - **Repository:** [jacobfa/Attractor](https://github.com/jacobfa/Attractor) |
|
|
| ## Usage |
|
|
| To use this model, you need to install the official package from the [GitHub repository](https://github.com/jacobfa/Attractor). |
|
|
| ```python |
| from attractor.models.attractor import Attractor, AttractorConfig |
| |
| # Construct the configuration for the 770m model |
| config = AttractorConfig.from_name("attractor-large-770m") |
| model = config.construct_model() |
| ``` |
|
|
| ## Citation |
|
|
| ```bibtex |
| @article{feinashley2026attractor, |
| title={Solve the Loop: Attractor Models for Language and Reasoning}, |
| author={Fein-Ashley, Jacob and Rashidinejad, Paria}, |
| year={2026}, |
| url={https://arxiv.org/abs/2605.12466} |
| } |
| ``` |
