A reproducible spectral signal that predicts reasoning ability — verified on r1, ask help to verify on model
#10
by wehe1pwe - opened
We ran SVD on Q/K weight matrices of several open-source LLMs and found two cross-model regularities:
Pearson r(Q,K singular values) → 1 (median 0.94–0.99 across LLaMA-3, Qwen-2.5, Gemma-4)
Deep-layer SSR (Spectral Shape Residue) systematically lower in reasoning-tuned models — when comparing this R1-distill against its base Qwen2.5-14B, SSR improvement reaches 5.95% at Layer 43
No benchmarks needed. Just SVD on static weights.
GitHub: https://github.com/emis-framework/math-under-llm
White Paper: WHITEPAPER.md in the repo.