GPT-1900 Drafts
Collection
Experimental and intermediate GPT-1900 checkpoints. Working artifacts, not for general use. • 49 items • Updated
GPT-1900 D34 Discovery RL (Pre-1900 Only Base)
3.29B parameter GPT-1900 with contradiction/discovery RL trained from a pre-1900-only base model. Chain: pre-1900-only base → physics CLM (pre-1900) → v3 SFT (safe) → discovery RL.
Best checkpoint: step 700 (physics eval avg 1.125, tied with steps 525/665).
Model Details
- Architecture: Custom GPT with RoPE, QK-norm, ReLU², value embeddings (ResFormer), per-layer residual/skip scalars
- Parameters: 3.29B
- Layers: 34
- Hidden dim: 2176
- Attention heads: 17 (query) / 17 (kv)
- Head dim: 128
- Context length: 2048 tokens
- Vocab size: 32,768 (BPE, GPT-4 style split pattern)
- Training: Discovery RL on pre-1900-only pipeline, step 700 (best performer)
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support