| license: mit | |
| pipeline_tag: text-generation | |
| # Solve the Loop: Attractor Models for Language and Reasoning | |
| This repository contains the 770M parameter Attractor Model introduced in [Solve the Loop: Attractor Models for Language and Reasoning](https://huggingface.co/papers/2605.12466). | |
| Attractor Models offer a promising alternative to purely feed-forward computation by iteratively refining latent representations. In this architecture, a backbone module first proposes output embeddings, then an attractor module refines them by solving for the fixed point using implicit differentiation. This model delivers a Pareto improvement over standard Transformers in language modeling and demonstrates strong performance on reasoning tasks. | |
| - **Paper:** [Solve the Loop: Attractor Models for Language and Reasoning](https://huggingface.co/papers/2605.12466) | |
| - **Project Page:** [attractor-models.github.io](https://attractor-models.github.io/) | |
| - **Repository:** [jacobfa/Attractor](https://github.com/jacobfa/Attractor) | |
| ## Usage | |
| To use this model, you need to install the official package from the [GitHub repository](https://github.com/jacobfa/Attractor). | |
| ```python | |
| from attractor.models.attractor import Attractor, AttractorConfig | |
| # Construct the configuration for the 770m model | |
| config = AttractorConfig.from_name("attractor-large-770m") | |
| model = config.construct_model() | |
| ``` | |
| ## Citation | |
| ```bibtex | |
| @article{feinashley2026attractor, | |
| title={Solve the Loop: Attractor Models for Language and Reasoning}, | |
| author={Fein-Ashley, Jacob and Rashidinejad, Paria}, | |
| year={2026}, | |
| url={https://arxiv.org/abs/2605.12466} | |
| } | |
| ``` |