metadata
language: en
license: apache-2.0
tags:
- neuroscience
- brain-connectivity
- gpt2
- axonlm
AxonLM-Base
Part of AxonLM — GPT-2 architecture trained for neuroanatomical knowledge probing.
Description
Pretrained on FineWeb-Edu (295M tokens, 9000 steps). HellaSwag: 0.327.
Key Results
- Linear probe AUC = 0.963 (p=0.002) on Allen Mouse Brain Connectivity Atlas (N=90)
- Full Atlas AUC = 0.847 (31σ above null, N=159,872 pairs, 428 structures)
- FFN L9 activations encode neuroanatomical connectivity (sleeping knowledge)
- AxonLM-Expert retrieval system: 100% accuracy on 12 anatomical queries
Model Family
| Model | Training | Params | Probe AUC |
|---|---|---|---|
| AxonLM-Base | FineWeb-Edu (295M tokens) | 124M | 0.844 |
| AxonLM-Neuro | + PubMed FT (98M tokens) | 124M | 0.847 |
Citation
@article{[AxonLM](https://zenodo.org/records/20027966),
title={Neuroanatomical Connectivity is Linearly Decodable from AxonLM Feed-Forward Network Activations},
author={Efekan Salman},
year={2026}
}
Usage
from transformers import GPT2LMHeadModel, GPT2Tokenizer
model = GPT2LMHeadModel.from_pretrained("YOUR_HF_USERNAME/AxonLM-Base")
tok = GPT2Tokenizer.from_pretrained("YOUR_HF_USERNAME/AxonLM-Base")
inputs = tok("CA3 sends projections to", return_tensors="pt")
output = model.generate(**inputs, max_new_tokens=10)
print(tok.decode(output[0]))