Select number of experts in request?
with 2 experts activated by default, however you can activate up to all 12 experts if you need the extra "brainpower".
This allows you to run the model at 4, 8, 12, 16, 20, 24 and up to 48B "power levels" as needed.
If this is true, and the part about activating experts by name is also true, then it should be possible to modify the template to accept a "number_of_experts: n" parameter, and insert n keywords (expert names) at the start of the output, to allow clients to dynamically request quick or complex reasoning, right? You would want to insert the "best" experts in order of preference, but I think it should work? And if that parameter is not provided by the client, it would continue to work as normal.
That would work.
When the gating was written (and experts organized) it was done with the intention of the best/strongest distills first and on down so to speak.
All the models (13: 12 + 1 base) were bench marked prior to building this.
Order of models in a MOE matters a lot ; even if they are gated.
Order of models in a MOE matters a lot ; even if they are gated.
Oh, right, non-commutative operations. Hadn't thought of that.