Problems with tool calling
#1
by Jon-Nielsen - opened
Hello,
Thank you for this quant.
Unfortunately it has problems with tool calling failing where the cyankiwi AWQ quant and the mistralai FP8 version does not. I had the issue with vllm (didn't note down versions, later than the required), but also tested with sglang nightly 20260404 just now.
I hope you'll be able to fix it.