---
license: mit
pipeline_tag: text-generation
---

# Solve the Loop: Attractor Models for Language and Reasoning

This repository contains the 770M parameter Attractor Model introduced in [Solve the Loop: Attractor Models for Language and Reasoning](https://huggingface.co/papers/2605.12466).

Attractor Models offer a promising alternative to purely feed-forward computation by iteratively refining latent representations. In this architecture, a backbone module first proposes output embeddings, then an attractor module refines them by solving for the fixed point using implicit differentiation. This model delivers a Pareto improvement over standard Transformers in language modeling and demonstrates strong performance on reasoning tasks.

- **Paper:** [Solve the Loop: Attractor Models for Language and Reasoning](https://huggingface.co/papers/2605.12466)
- **Project Page:** [attractor-models.github.io](https://attractor-models.github.io/)
- **Repository:** [jacobfa/Attractor](https://github.com/jacobfa/Attractor)

## Usage

To use this model, you need to install the official package from the [GitHub repository](https://github.com/jacobfa/Attractor).

```python
from attractor.models.attractor import Attractor, AttractorConfig

# Construct the configuration for the 770m model
config = AttractorConfig.from_name("attractor-large-770m")
model = config.construct_model()
```

## Citation

```bibtex
@article{feinashley2026attractor,
  title={Solve the Loop: Attractor Models for Language and Reasoning},
  author={Fein-Ashley, Jacob and Rashidinejad, Paria},
  year={2026},
  url={https://arxiv.org/abs/2605.12466}
}
```