Add model card
Browse filesThis PR adds a model card for the Attractor-140M model presented in the paper [Solve the Loop: Attractor Models for Language and Reasoning](https://huggingface.co/papers/2605.12466).
The model card includes:
- Relevant metadata (`pipeline_tag` and `license`).
- Links to the paper, project page, and official GitHub repository.
- A sample usage snippet derived from the official implementation.
README.md
CHANGED
|
@@ -1,3 +1,42 @@
|
|
| 1 |
-
---
|
| 2 |
-
license: mit
|
| 3 |
-
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: mit
|
| 3 |
+
pipeline_tag: text-generation
|
| 4 |
+
---
|
| 5 |
+
|
| 6 |
+
# Solve the Loop: Attractor Models for Language and Reasoning
|
| 7 |
+
|
| 8 |
+
This repository contains the Attractor-140M model presented in [Solve the Loop: Attractor Models for Language and Reasoning](https://huggingface.co/papers/2605.12466).
|
| 9 |
+
|
| 10 |
+
[**Project Page**](https://attractor-models.github.io/) | [**GitHub**](https://github.com/jacobfa/Attractor) | [**Paper**](https://arxiv.org/abs/2605.12466)
|
| 11 |
+
|
| 12 |
+
## Introduction
|
| 13 |
+
|
| 14 |
+
Attractor Models are a family of models that use a backbone module to propose output embeddings and an attractor module to refine them by solving for a fixed point through implicit differentiation. This architecture allows training memory to remain constant relative to effective depth and enables iterations to be chosen adaptively. In language modeling, Attractor Models deliver a Pareto improvement over standard Transformers and stable looped models across sizes.
|
| 15 |
+
|
| 16 |
+
## Sample Usage
|
| 17 |
+
|
| 18 |
+
To use this model, you first need to install the package from the [official repository](https://github.com/jacobfa/Attractor):
|
| 19 |
+
|
| 20 |
+
```bash
|
| 21 |
+
pip install -e .
|
| 22 |
+
```
|
| 23 |
+
|
| 24 |
+
Then you can initialize the model as follows:
|
| 25 |
+
|
| 26 |
+
```python
|
| 27 |
+
from attractor.models.attractor import Attractor, AttractorConfig
|
| 28 |
+
|
| 29 |
+
config = AttractorConfig.from_name("attractor-small-140m")
|
| 30 |
+
model = config.construct_model()
|
| 31 |
+
```
|
| 32 |
+
|
| 33 |
+
## Citation
|
| 34 |
+
|
| 35 |
+
```bibtex
|
| 36 |
+
@article{feinashley2026attractor,
|
| 37 |
+
title={Solve the Loop: Attractor Models for Language and Reasoning},
|
| 38 |
+
author={Fein-Ashley, Jacob and Rashidinejad, Paria},
|
| 39 |
+
year={2026},
|
| 40 |
+
url={https://arxiv.org/abs/2605.12466}
|
| 41 |
+
}
|
| 42 |
+
```
|