---
license: mit
pipeline_tag: text-generation
---

# Solve the Loop: Attractor Models for Language and Reasoning

This repository contains the Attractor-140M model presented in [Solve the Loop: Attractor Models for Language and Reasoning](https://huggingface.co/papers/2605.12466).

[**Project Page**](https://attractor-models.github.io/) | [**GitHub**](https://github.com/jacobfa/Attractor) | [**Paper**](https://arxiv.org/abs/2605.12466)

## Introduction

Attractor Models are a family of models that use a backbone module to propose output embeddings and an attractor module to refine them by solving for a fixed point through implicit differentiation. This architecture allows training memory to remain constant relative to effective depth and enables iterations to be chosen adaptively. In language modeling, Attractor Models deliver a Pareto improvement over standard Transformers and stable looped models across sizes.

## Sample Usage

To use this model, you first need to install the package from the [official repository](https://github.com/jacobfa/Attractor):

```bash
pip install -e .
```

Then you can initialize the model as follows:

```python
from attractor.models.attractor import Attractor, AttractorConfig

config = AttractorConfig.from_name("attractor-small-140m")
model = config.construct_model()
```

## Citation

```bibtex
@article{feinashley2026attractor,
  title={Solve the Loop: Attractor Models for Language and Reasoning},
  author={Fein-Ashley, Jacob and Rashidinejad, Paria},
  year={2026},
  url={https://arxiv.org/abs/2605.12466}
}
```