How to Scale Mixture-of-Experts: From muP to the Maximally Scale-Stable Parameterization
Paper • 2605.14200 • Published
Leena Chennuru Vankadara
leenachennuru
AI & ML interests
None yet
Recent Activity
authored a paper about 10 hours ago
How to Scale Mixture-of-Experts: From muP to the Maximally Scale-Stable Parameterization authored a paper about 10 hours ago
On the Surprising Effectiveness of Large Learning Rates under Standard Width Scaling authored a paper about 10 hours ago
μP$^2$: Effective Sharpness Aware Minimization Requires Layerwise Perturbation ScalingOrganizations
None yet
authored 4
papers about 10 hours ago
On the Surprising Effectiveness of Large Learning Rates under Standard Width Scaling
Paper • 2505.22491 • Published
μP$^2$: Effective Sharpness Aware Minimization Requires Layerwise Perturbation Scaling
Paper • 2411.00075 • Published
Training Neural Networks at Any Scale
Paper • 2511.11163 • Published