File size: 1,452 Bytes
92028c7 c1914e7 92028c7 83ab551 92028c7 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 | ---
language: en
license: apache-2.0
tags: [neuroscience, brain-connectivity, gpt2, axonlm]
---
# AxonLM-Neuro
Part of **AxonLM** — GPT-2 architecture trained for neuroanatomical knowledge probing.
## Description
AxonLM-Base fine-tuned on PubMed abstracts (98M tokens). Val loss: 2.90.
## Key Results
- Linear probe **AUC = 0.963** (p=0.002) on Allen Mouse Brain Connectivity Atlas (N=90)
- Full Atlas **AUC = 0.847** (31σ above null, N=159,872 pairs, 428 structures)
- FFN L9 activations encode neuroanatomical connectivity (*sleeping knowledge*)
- AxonLM-Expert retrieval system: **100% accuracy** on 12 anatomical queries
## Model Family
| Model | Training | Params | Probe AUC |
|-------|----------|--------|-----------|
| AxonLM-Base | FineWeb-Edu (295M tokens) | 124M | 0.844 |
| AxonLM-Neuro | + PubMed FT (98M tokens) | 124M | 0.847 |
## Citation
```bibtex
@article{[AxonLM](https://zenodo.org/records/20027966),
title={Neuroanatomical Connectivity is Linearly Decodable from AxonLM Feed-Forward Network Activations},
author={Efekan Salman},
year={2026}
}
```
## Usage
```python
from transformers import GPT2LMHeadModel, GPT2Tokenizer
model = GPT2LMHeadModel.from_pretrained("YOUR_HF_USERNAME/AxonLM-Neuro")
tok = GPT2Tokenizer.from_pretrained("YOUR_HF_USERNAME/AxonLM-Neuro")
inputs = tok("CA3 sends projections to", return_tensors="pt")
output = model.generate(**inputs, max_new_tokens=10)
print(tok.decode(output[0]))
```
|