Regarding RoPE frequency

#29
by juneyongyang - opened

Currently, Qwen3-30B-A3B-Instruct-2507 's config.json states that
"rope_theta": 10000000 -> 1e7

while Qwen3-30B-A3B 's config.json states that
"rope_theta": 1000000.0, -> 1e6

Is the correct value for Qwen3-30B-A3B-Instruct-2507 1e7?

Sign up or log in to comment