Mistral 12B - 34B ?? Please 🥰
Sadly the Mistral 3 models didn't do it for a lot of us, specifically for creative writing... It was very dull and empty. Mistral's are some of the best open weight models, esp for creative writing, usually.
Wish we got an update, hopefully in time? ❤️❤️We love you Mistral 🥰
i think we'll be getting a Ministral 4 later this year.
Would love it Mistral could release >100B MoE model. Maybe an MoE that seeks to be close in performance to the 3.5-128B dense (Mixtral-3.5-128B?).
100B MoEs perform similarly to 32B-ish dense models. an MoE model from them would perform similarly to M3.5 if it released later and with higher-quality data.
this is why Mistral is comparing Medium 3.5 with 700B-1T models, and not other 100B MoEs. the current 100B MoE we have from them is Small 4, which is a tier lower, as expected