Question about tool definition format in chat template: should tools be passed as JSON or XML?
Hi, I have a question about the expected tool format during inference.
The chat template renders tool-related instructions in an XML-style format, so I am not sure what users are supposed to pass in when calling the model:
• Should tools be provided in JSON format (for example, OpenAI-style function definitions), and then be converted by the chat template automatically?
• Or should tools be provided directly in XML format?
If the expected input is XML, is there a standard template or example for the tool definition format?
In short, my main question is:
At inference time, should the input tools be JSON or XML?
Thanks.
Well, tools are converted automatically by the chat template, but I have had many problems with native tool calling with tools defined as JSON, at least with a local SGLang server.
So I disabled the tool call parser completely and enforced gave a system prompt which guides the model to use XML tools.
I hope we get some kind of solution for this.