Sampling parameters?
Dear @Azure99 please have a look, if my current sampling parameters are set correctly:
Up to this point I used different ones, but it seems that these "unlock" additional features (like "fancy" paragraph formatting, which I didn't have before)?
Much appreciated! 🌸
@McG-221 sorry for the late reply — I only just saw your message.
For llama.cpp, I’d recommend using the following sampling parameters:
--temp 0.5 --top-p 0.85 --top-k 50 --repeat-penalty 1.05 --repeat-last-n -1 --min-p 0 --samplers "penalties;temperature;top_k;top_p"
In my experience, this setup tends to give a good balance between stability and flexibility.
Thank you very much! It's often not so easy to determine, if a model has a "sweet spot". I guess that's the nature of stochastic sampling... but sometimes, I just feel the urge to "optimize" towards a model's trained state, in the sense of "works as designed". Well, will try that out, thanks mate! ✌️