File size: 1,447 Bytes
a62d2b0 ad7ea67 a62d2b0 11b9e5b a62d2b0 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 | ---
language: en
license: apache-2.0
tags: [neuroscience, brain-connectivity, gpt2, axonlm]
---
# AxonLM-Base
Part of **AxonLM** — GPT-2 architecture trained for neuroanatomical knowledge probing.
## Description
Pretrained on FineWeb-Edu (295M tokens, 9000 steps). HellaSwag: 0.327.
## Key Results
- Linear probe **AUC = 0.963** (p=0.002) on Allen Mouse Brain Connectivity Atlas (N=90)
- Full Atlas **AUC = 0.847** (31σ above null, N=159,872 pairs, 428 structures)
- FFN L9 activations encode neuroanatomical connectivity (*sleeping knowledge*)
- AxonLM-Expert retrieval system: **100% accuracy** on 12 anatomical queries
## Model Family
| Model | Training | Params | Probe AUC |
|-------|----------|--------|-----------|
| AxonLM-Base | FineWeb-Edu (295M tokens) | 124M | 0.844 |
| AxonLM-Neuro | + PubMed FT (98M tokens) | 124M | 0.847 |
## Citation
```bibtex
@article{[AxonLM](https://zenodo.org/records/20027966),
title={Neuroanatomical Connectivity is Linearly Decodable from AxonLM Feed-Forward Network Activations},
author={Efekan Salman},
year={2026}
}
```
## Usage
```python
from transformers import GPT2LMHeadModel, GPT2Tokenizer
model = GPT2LMHeadModel.from_pretrained("YOUR_HF_USERNAME/AxonLM-Base")
tok = GPT2Tokenizer.from_pretrained("YOUR_HF_USERNAME/AxonLM-Base")
inputs = tok("CA3 sends projections to", return_tensors="pt")
output = model.generate(**inputs, max_new_tokens=10)
print(tok.decode(output[0]))
```
|