Spaces:
Running on CPU Upgrade
Apply for a GPU community grant: Personal project
I’m building a Hugging Face Space to share a set of novel results on grokking in small transformers.
https://huggingface.co/spaces/zboralski/grokking-introspection
The current experiments focus on modular addition and multiplication. We introduce a structural constraint inside the MLP: a small probe learns the input–hidden mapping, and the model is regularized toward that learned structure during training. The probe is discarded afterward; it only shapes the trajectory.
On both add and mul, we observe a consistent gain. Generalization emerges significantly earlier, the grokking transition sharpens, and the internal spectrum organizes sooner. Representation drift is reduced, and hidden states stabilize faster under temporal stencil constraints.
The Space will include the full stencil suite (seven temporal variants), checkpoints, and spectral visualizations. Everything will be open-source and clonable, with additional reproductions and extensions added over time.
Support through a free or discounted tier would help us keep the experiments and artifacts persistently available to the community.
Merci!