---
language: en
license: apache-2.0
tags: [neuroscience, brain-connectivity, gpt2, axonlm]
---
# AxonLM-Base

Part of **AxonLM** — GPT-2 architecture trained for neuroanatomical knowledge probing.

## Description
Pretrained on FineWeb-Edu (295M tokens, 9000 steps). HellaSwag: 0.327.

## Key Results
- Linear probe **AUC = 0.963** (p=0.002) on Allen Mouse Brain Connectivity Atlas (N=90)
- Full Atlas **AUC = 0.847** (31σ above null, N=159,872 pairs, 428 structures)
- FFN L9 activations encode neuroanatomical connectivity (*sleeping knowledge*)
- AxonLM-Expert retrieval system: **100% accuracy** on 12 anatomical queries

## Model Family
| Model | Training | Params | Probe AUC |
|-------|----------|--------|-----------|
| AxonLM-Base  | FineWeb-Edu (295M tokens) | 124M | 0.844 |
| AxonLM-Neuro | + PubMed FT (98M tokens)  | 124M | 0.847 |

## Citation
```bibtex
@article{[AxonLM](https://zenodo.org/records/20027966),
  title={Neuroanatomical Connectivity is Linearly Decodable from AxonLM Feed-Forward Network Activations},
  author={Efekan Salman},
  year={2026}
}
```

## Usage
```python
from transformers import GPT2LMHeadModel, GPT2Tokenizer
model = GPT2LMHeadModel.from_pretrained("YOUR_HF_USERNAME/AxonLM-Base")
tok   = GPT2Tokenizer.from_pretrained("YOUR_HF_USERNAME/AxonLM-Base")
inputs = tok("CA3 sends projections to", return_tensors="pt")
output = model.generate(**inputs, max_new_tokens=10)
print(tok.decode(output[0]))
```