fix: MoE intermediate_size not scaled for tiny — 158M→4M MoE params

Bug: setdefault('moe_intermediate_size', ...) doesn't override the
1728 value from the full-scale config.json. Result: each expert is
SwiGLUMLP(256, 1728) = 1.3M params × 17 experts × 7 layers = 158M
params in MoE alone (70% of model), when it should be ~4M.

Fix: when a scale preset is active, force-override moe_intermediate_size
to intermediate_size // 4 (= 128 for tiny). Also force n_routed_experts
to 8 (not 16 from full config).

Param reduction: 227M → ~73M total. Step time ~3× faster."

chimera/training/common.py

@@ -64,12 +64,24 @@ def apply_standard_config_tweaks(config: dict[str, Any], *, scale: str, seq_len:
         "persistent_memory_slots": 16,
         "local_window_size": min(seq_len, 256),
     })
+
+    # ── MoE config ──
+    # IMPORTANT: force-override MoE sizes when a scale preset is active.
+    # The full-scale config.json has moe_intermediate_size=1728 (for hidden=2560).
+    # Using setdefault() would silently keep 1728 even at tiny scale (hidden=256),
+    # resulting in 158M params in MoE alone (70% of model).
     moe_cfg = config.setdefault("backbone", {}).setdefault("moe", {})
     moe_cfg.setdefault("layers", [3, 7, 11, 15, 19, 23, 27])
-    moe_cfg.setdefault("moe_intermediate_size", config["intermediate_size"] // 4)
-    moe_cfg.setdefault("n_routed_experts", 8)
+    if scale in DEFAULT_SCALE_PRESETS:
+        # Force scale-appropriate MoE sizes
+        moe_cfg["moe_intermediate_size"] = config["intermediate_size"] // 4
+        moe_cfg["n_routed_experts"] = 8
+    else:
+        moe_cfg.setdefault("moe_intermediate_size", config["intermediate_size"] // 4)
+        moe_cfg.setdefault("n_routed_experts", 8)
     moe_cfg.setdefault("n_shared_experts", 1)
     moe_cfg.setdefault("num_experts_per_tok", 2)
+
     config.setdefault("looping", {}).update({
         "enabled": True,
         "prelude": [0, 3],