Upload folder using huggingface_hub

README.md

@@ -0,0 +1,43 @@
+---
+language: en
+license: apache-2.0
+tags: [neuroscience, brain-connectivity, gpt2, axonlm]
+---
+# AxonLM-Neuro
+
+Part of **AxonLM** — GPT-2 architecture trained for neuroanatomical knowledge probing.
+
+## Description
+AxonLM-Base fine-tuned on PubMed abstracts (98M tokens). Val loss: 2.90.
+
+## Key Results
+- Linear probe **AUC = 0.963** (p=0.002) on Allen Mouse Brain Connectivity Atlas (N=90)
+- Full Atlas **AUC = 0.847** (31σ above null, N=159,872 pairs, 428 structures)
+- FFN L9 activations encode neuroanatomical connectivity (*sleeping knowledge*)
+- AxonLM-Expert retrieval system: **100% accuracy** on 12 anatomical queries
+
+## Model Family
+| Model | Training | Params | Probe AUC |
+|-------|----------|--------|-----------|
+| AxonLM-Base  | FineWeb-Edu (295M tokens) | 124M | 0.844 |
+| AxonLM-Neuro | + PubMed FT (98M tokens)  | 124M | 0.847 |
+
+## Citation
+```bibtex
+@article{axonlm2025,
+  title={Neuroanatomical Connectivity is Linearly Decodable from AxonLM Feed-Forward Network Activations},
+  author={Anonymous},
+  journal={NeurIPS NeuroAI Workshop},
+  year={2025}
+}
+```
+
+## Usage
+```python
+from transformers import GPT2LMHeadModel, GPT2Tokenizer
+model = GPT2LMHeadModel.from_pretrained("YOUR_HF_USERNAME/AxonLM-Neuro")
+tok   = GPT2Tokenizer.from_pretrained("YOUR_HF_USERNAME/AxonLM-Neuro")
+inputs = tok("CA3 sends projections to", return_tensors="pt")
+output = model.generate(**inputs, max_new_tokens=10)
+print(tok.decode(output[0]))
+```

config.json

@@ -0,0 +1,34 @@
+{
+  "activation_function": "gelu_new",
+  "add_cross_attention": false,
+  "architectures": [
+    "GPT2LMHeadModel"
+  ],
+  "attn_pdrop": 0.0,
+  "bos_token_id": 50256,
+  "dtype": "float32",
+  "embd_pdrop": 0.0,
+  "eos_token_id": 50256,
+  "initializer_range": 0.02,
+  "layer_norm_epsilon": 1e-05,
+  "model_type": "gpt2",
+  "n_embd": 768,
+  "n_head": 12,
+  "n_inner": 3072,
+  "n_layer": 12,
+  "n_positions": 1024,
+  "pad_token_id": null,
+  "reorder_and_upcast_attn": false,
+  "resid_pdrop": 0.0,
+  "scale_attn_by_inverse_layer_idx": false,
+  "scale_attn_weights": true,
+  "summary_activation": null,
+  "summary_first_dropout": 0.1,
+  "summary_proj_to_labels": true,
+  "summary_type": "cls_index",
+  "summary_use_proj": true,
+  "tie_word_embeddings": true,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 50304
+}

generation_config.json

@@ -0,0 +1,9 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 50256,
+  "eos_token_id": 50256,
+  "output_attentions": false,
+  "output_hidden_states": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:75f331c88ef64daac664ca3a8099d5ffe79a2501ebc9cace125b6b34397fd288
+size 497918592

tokenizer_config.json

@@ -0,0 +1,12 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": "<|endoftext|>",
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "is_local": false,
+  "model_max_length": 1024,
+  "pad_token": "<|endoftext|>",
+  "tokenizer_class": "GPT2Tokenizer",
+  "unk_token": "<|endoftext|>"
+}