Dose this model support tool calling?

#27
by danielyf2023 - opened

I want to use some local model,but don't know if this can support too calling?

yes. in fact it is very good at it from how i see it. i was surprised when this 9b variant made a tool call for web search, which returned [null] due to searxng container still warming up, and the model took it upon itself to make another tool call using several different ways of phrasing, and on the second attempt it got the responses as intented. not shabby.

Why when I talk to him just response me with junk text? like

Are you unable to respond properly to my querieS?
as the heart of the group.
*The Hero's Rivals' Parents' Parents' Parents' Parents' Parents' Love Interest - A mysterious figure who is often seen as the main love interest of the parents of the parents of the parents of the parents of the parents of
the main rivals of the group. He has a talent for magic and is often the one to come up with the group's plans.
*The Hero's Rivals' Parents' Parents' Parents' Parents' Parents' Mentor - A mysterious figure who is often seen as the main mentor of the parents of the parents of the parents of the parents of the parents of the main
rivals of the group. He has a talent for magic and is often the one to come up with the group's plans.
*The Hero's Rivals' Parents' Parents' Parents' Parents' Parents' Arch-Nemesis - A mysterious figure who is often seen as the main arch-nemesis of the parents of the parents of the parents of the parents of the parents of
the main rivals of the group. He has a talent for magic and is often the one to come up with the group's plans.
*The Hero's Rivals' Parents' Parents' Parents' Parents' Parents' Sidekicks - A group of sidekicks who are often seen as the main sidekicks of the parents of the^C

This is unusable for me, do you know guys what I am doing wrong?

yes. in fact it is very good at it from how i see it. i was surprised when this 9b variant made a tool call for web search, which returned [null] due to searxng container still warming up, and the model took it upon itself to make another tool call using several different ways of phrasing, and on the second attempt it got the responses as intented. not shabby.

Could you please explain how it works? When I try to run Ollama, I get the following error: 400 registry.ollama.ai/library/Qwen3.5-9B-Uncensored:latest does not support tools.

  • Well, to give some needed context on my setup, i am using open-webui for the gui, but the backend is ollama.
  • In open-webui i created a custom model derived from qwen3.5-9b. I configured the settings exactly how the qwen authors advised, i.e 128k context, temp, min_p, etc.
  • It is worth noting that context size may play a bigger role than we are used to, I'm not entirely sure. There must be some reason they advise a huge 128k. Surprisingly an rtx 3060 12GB still fits it all onto vram.
  • Digress.
  • The one notable thing I changed in the open-webui advanced params is 'function calling', which i changed from 'default' to 'native'. This is owui's description of the param:
    "Defauit mode works with a wider range of models by calling tools once: before execution. Native mode leverages the models builtin tool-calling capabiltes, but requires the model to inherenty support this feature."
  • Furthermore, assuming you are not using owui and only ollama, claude did catch a few discrepencies with the model descriptions/related docs: some sources say the default mode of operation is everything is set to ON, however other sources differ. Granted your problem, i believe it is probably the case that the sources alluring to the fact the small models >9B come with thinking etc off by default is likely the correct one. Now, i didnt try to get it working in ollama, but claude was spitting code at me earlier for exactly how to get it working, so it is definitely possible, albeit the conflicting info makes it less straight-forward than it should be.
  • Just a disclaimer that i didnt check to see where claude was reading this stuff from, so if i say description/docs and everything im saying is literally in the first 2 lines of text, thats why. Anyhow, claude said the reason for this was the small models >9B were not balancing reasoning effort with the quality of outputs, and it was deemed that they were taking up too much context for little in return more times than qwen team wanted, so they just decided to disable it by default for the small >9B models. I havent tested it much more than a couple of messages, so i cant say whether they jumped the gun by disabling it on the 9b model by default too, however, ive not had tool calling simply work right off the bat like that before so that is something worth mentioning.
  • I know ive basically been no help at all but it was my pleasure.
  • tool calling = function calling | reasoning = thinking. Thinking is not functionally related to tool calling, but significantly impacts the quality of responses, so ive used probably all of the aforementioned interchangeably above.
  • Well, to give some needed context on my setup, i am using open-webui for the gui, but the backend is ollama.
  • In open-webui i created a custom model derived from qwen3.5-9b. I configured the settings exactly how the qwen authors advised, i.e 128k context, temp, min_p, etc.
  • It is worth noting that context size may play a bigger role than we are used to, I'm not entirely sure. There must be some reason they advise a huge 128k. Surprisingly an rtx 3060 12GB still fits it all onto vram.
  • Digress.
  • The one notable thing I changed in the open-webui advanced params is 'function calling', which i changed from 'default' to 'native'. This is owui's description of the param:
    "Defauit mode works with a wider range of models by calling tools once: before execution. Native mode leverages the models builtin tool-calling capabiltes, but requires the model to inherenty support this feature."
  • Furthermore, assuming you are not using owui and only ollama, claude did catch a few discrepencies with the model descriptions/related docs: some sources say the default mode of operation is everything is set to ON, however other sources differ. Granted your problem, i believe it is probably the case that the sources alluring to the fact the small models >9B come with thinking etc off by default is likely the correct one. Now, i didnt try to get it working in ollama, but claude was spitting code at me earlier for exactly how to get it working, so it is definitely possible, albeit the conflicting info makes it less straight-forward than it should be.
  • Just a disclaimer that i didnt check to see where claude was reading this stuff from, so if i say description/docs and everything im saying is literally in the first 2 lines of text, thats why. Anyhow, claude said the reason for this was the small models >9B were not balancing reasoning effort with the quality of outputs, and it was deemed that they were taking up too much context for little in return more times than qwen team wanted, so they just decided to disable it by default for the small >9B models. I havent tested it much more than a couple of messages, so i cant say whether they jumped the gun by disabling it on the 9b model by default too, however, ive not had tool calling simply work right off the bat like that before so that is something worth mentioning.
  • I know ive basically been no help at all but it was my pleasure.
  • tool calling = function calling | reasoning = thinking. Thinking is not functionally related to tool calling, but significantly impacts the quality of responses, so ive used probably all of the aforementioned interchangeably above.

Thanks for the explanation, really appreciate it!

In my case, I'm using OpenClaw with Ollama, and the error (does not support tools) suggests that the model itself doesn’t support tool calling, might be different from the Open WebUI setup.

Thanks again for your help!

When I am trying to use this model (q4 variant) for ANYTHING, it collapses to an infinite cycle (despite the beginning being very reasonable)

'''

How to exit Vim?

To exit Vim, press :q! and hit Enter. This command quits Vim and discards any changes made to the file.

If you have made changes and want to exit, you can use :q or :wq (write and quit) to save the file and exit.

Here are the common commands to exit Vim:

  1. :q! (Quit and discard):

    • Press : to open the command line.
    • Type q! (q for quit, ! for force/ignore changes).
    • Press Enter.
    • Use when: You want to exit without saving.
  2. :q (Quit):

    • Press :.
    • Type q.
    • Press Enter.
    • Use when: You have no changes or have already saved.
  3. :wq (Write and Quit):

    • Press :.
    • Type wq.
    • Press Enter.
    • Use when: You want to save your changes and exit.
  4. :x (Write and Quit):

    • Press :.
    • Type x.
    • Press Enter.
    • Use when: Similar to :wq, it saves and exits.
  5. :wq! (Force Write and Quit):

    • Press :.
    • Type wq!.
    • Press Enter.
    • Use when: You have changes but want to force save and exit.
  6. ZZ (Write and Quit):

    • Press Z twice (without the colon).
    • Use when: A quick way to save and exit if the file is not modified.
  7. ZQ (Quit):

    • Press Z twice.
    • Use when: A quick way to quit without saving.
  8. :quit! (Force Quit):

    • Press :.
    • Type quit!.
    • Press Enter.
    • Use when: You want to force quit even if there are unsaved changes.
  9. :quitall! (Force Quit All Files):

    • Press :.
    • Type quitall!.
    • Press Enter.
    • Use when: You have multiple files open and want to force quit all of them.
  10. :quitall (Quit All Files):

    • Press :.
    • Type quitall.
    • Press Enter.
    • Use when: You have multiple files open and want to quit all of them if they are not modified.
  11. :quitall! (Force Quit All Files):

    • Press :.
    • Type quitall!.
    • Press Enter.
    • Use when: You have multiple files open and want to force quit all of them even if they are modified.
      ...
      ''

Sign up or log in to comment