| --- |
| language: en |
| license: apache-2.0 |
| tags: [neuroscience, brain-connectivity, gpt2, axonlm] |
| --- |
| # AxonLM-Base |
|
|
| Part of **AxonLM** — GPT-2 architecture trained for neuroanatomical knowledge probing. |
|
|
| ## Description |
| Pretrained on FineWeb-Edu (295M tokens, 9000 steps). HellaSwag: 0.327. |
|
|
| ## Key Results |
| - Linear probe **AUC = 0.963** (p=0.002) on Allen Mouse Brain Connectivity Atlas (N=90) |
| - Full Atlas **AUC = 0.847** (31σ above null, N=159,872 pairs, 428 structures) |
| - FFN L9 activations encode neuroanatomical connectivity (*sleeping knowledge*) |
| - AxonLM-Expert retrieval system: **100% accuracy** on 12 anatomical queries |
|
|
| ## Model Family |
| | Model | Training | Params | Probe AUC | |
| |-------|----------|--------|-----------| |
| | AxonLM-Base | FineWeb-Edu (295M tokens) | 124M | 0.844 | |
| | AxonLM-Neuro | + PubMed FT (98M tokens) | 124M | 0.847 | |
|
|
| ## Citation |
| ```bibtex |
| @article{[AxonLM](https://zenodo.org/records/20027966), |
| title={Neuroanatomical Connectivity is Linearly Decodable from AxonLM Feed-Forward Network Activations}, |
| author={Efekan Salman}, |
| year={2026} |
| } |
| ``` |
|
|
| ## Usage |
| ```python |
| from transformers import GPT2LMHeadModel, GPT2Tokenizer |
| model = GPT2LMHeadModel.from_pretrained("YOUR_HF_USERNAME/AxonLM-Base") |
| tok = GPT2Tokenizer.from_pretrained("YOUR_HF_USERNAME/AxonLM-Base") |
| inputs = tok("CA3 sends projections to", return_tensors="pt") |
| output = model.generate(**inputs, max_new_tokens=10) |
| print(tok.decode(output[0])) |
| ``` |
|
|
