Thank you!
#1
by tasticleeze - opened
Your Magistral quant is great, can’t wait to try this
Took some doing but I have it up on vllm nightly. It barely fits and I can’t quantize the cache, but it works and I’m liking so far!
tasticleeze changed discussion status to closed