nielsr HF Staff commited on
Commit
c92ae9b
·
verified ·
1 Parent(s): 4e3a75b

Add model card

Browse files

This PR adds a model card for the Attractor-140M model presented in the paper [Solve the Loop: Attractor Models for Language and Reasoning](https://huggingface.co/papers/2605.12466).

The model card includes:
- Relevant metadata (`pipeline_tag` and `license`).
- Links to the paper, project page, and official GitHub repository.
- A sample usage snippet derived from the official implementation.

Files changed (1) hide show
  1. README.md +42 -3
README.md CHANGED
@@ -1,3 +1,42 @@
1
- ---
2
- license: mit
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ pipeline_tag: text-generation
4
+ ---
5
+
6
+ # Solve the Loop: Attractor Models for Language and Reasoning
7
+
8
+ This repository contains the Attractor-140M model presented in [Solve the Loop: Attractor Models for Language and Reasoning](https://huggingface.co/papers/2605.12466).
9
+
10
+ [**Project Page**](https://attractor-models.github.io/) | [**GitHub**](https://github.com/jacobfa/Attractor) | [**Paper**](https://arxiv.org/abs/2605.12466)
11
+
12
+ ## Introduction
13
+
14
+ Attractor Models are a family of models that use a backbone module to propose output embeddings and an attractor module to refine them by solving for a fixed point through implicit differentiation. This architecture allows training memory to remain constant relative to effective depth and enables iterations to be chosen adaptively. In language modeling, Attractor Models deliver a Pareto improvement over standard Transformers and stable looped models across sizes.
15
+
16
+ ## Sample Usage
17
+
18
+ To use this model, you first need to install the package from the [official repository](https://github.com/jacobfa/Attractor):
19
+
20
+ ```bash
21
+ pip install -e .
22
+ ```
23
+
24
+ Then you can initialize the model as follows:
25
+
26
+ ```python
27
+ from attractor.models.attractor import Attractor, AttractorConfig
28
+
29
+ config = AttractorConfig.from_name("attractor-small-140m")
30
+ model = config.construct_model()
31
+ ```
32
+
33
+ ## Citation
34
+
35
+ ```bibtex
36
+ @article{feinashley2026attractor,
37
+ title={Solve the Loop: Attractor Models for Language and Reasoning},
38
+ author={Fein-Ashley, Jacob and Rashidinejad, Paria},
39
+ year={2026},
40
+ url={https://arxiv.org/abs/2605.12466}
41
+ }
42
+ ```