ceselder/cot-oracle-paper-ablation-adam-recipe-1layer

CoT Oracle Paper Ablation: Adam Recipe, 1 Layer

This repo contains the paper ablation that keeps the Adam-style training recipe inside the cot-oracle codebase and trains a single activation readout layer.

What This Checkpoint Is

• Base model: Qwen/Qwen3-8B
• Adapter format: PEFT LoRA
• Activation readout layers: [18]
• Task order: shuffled
• Seed: 42
• Planned budget: 50M input tokens
• Paper label: 17M logged training tokens

Exact Training Mixture

• latentqa: enabled, n: -1 (all available examples in the Adam-style LatentQA export used by this repo)
• classification: enabled, n: 20000, datasets = sst2, ag_news, snli
• fineweb: enabled, n: 60000, variants = futurelens_fineweb,pastlens_fineweb
• On-policy futurelens: disabled
• On-policy pastlens: disabled
• chunked_convqa: disabled
• All other tasks in configs/train.yaml: disabled

Notes

• This is the paper's "Adam recipe in this repo" ablation, not a byte-for-byte rerun of Adam Karvonen's original training script.
• The main approximations are the use of FineWeb future/past-lens readouts and a narrower 3-dataset classification mix.
• The token label above follows the paper bookkeeping from the run logs, while the config itself was set up with a 50M input-token budget.