tiiuae

/

Falcon-H1R-7B

like 217

Follow

Technology Innovation Institute 1.99k

Did you set

in the model params or the chat params?

and

for example, in the model params?

Should work. What does it answer if you ask it what tools it has access to?

Thanks, I updated the model settings, the chat settings, and I obtain 14 tools (see image) missing some of the one you mentioned. Is it due to me invoking Falcon via llama-server?

llama.cpp/build/bin/llama-server \
-m ../models/Falcon-H1R-7B-Q8_0.gguf
--temp 0.6
--top-p 0.95
-n 65536
--host 0.0.0.0
--port 10000
--jinja
-t 2
-b 64
-np 1

Hmm that's intriguing because it seems the only two you are missing is the web search and fetch URL, but you have the Web Search one checked, so this should work.

My config:

Did you config the web search engine in the Web Search tab? And also, are your open-webui up to date (or at least >= 0.7.0)? Cause I know they support the tool calls chaining since then (https://github.com/open-webui/open-webui/releases/tag/v0.7.0) and maybe the search and fetch url tools have been added to the native fn calling only then?
Is your search tool working as expected if keeping the Function Calling setting to Default?

And concerning your llama-server cmd, it seems all good, but here is mine for reference:

  "Falcon-H1R-7B-Q8_0":
    cmd: |
      /home/user/llama.cpp/build/bin/llama-server
      --model /mnt/storage/GGUFs/Falcon-H1R-7B-Q8_0.gguf
      --no-warmup
      --ctx-size 262144
      --no-context-shift
      --n-gpu-layers 100
      --temp 0.6
      --top-p 0.95
      --top-k 0
      --min-p 0
      --repeat-penalty 1.05
      --jinja
      --host 0.0.0.0
      --port ${PORT}
      --flash-attn on

So nothing different regarding the issue youre facing

Thanks a lot, the culprit (besides myself of course) was the missing initialization of the Web search engine (it seemed to me that every option require an API key but external, in spite of requiring it, does not really use it) so now everything is working! However I guess that to build a simple agent I will have to resort to a python script: any suggestion, template, examples?

Again, thank you so much for your fast and effective help!

I don't feel I did much! But thanks for the kind words :)

I just discovered the external provider option in the web search tab! I'm actually curious what engine it could be! The most "external" I've been using so far was a locally hosted Searxng instance!
Hmm, I only tried the open-webui tool call chaining feature a few times to test out. I don't know what would be the use case for your agent, but I believe open-webui isn't really suited for that. But there are plenty of solutions for agent creation and running!
Maybe if you already have an idea of the task(s) your agent should tackle, we could narrow it down. The context window you can handle could also give a direction, because on the side of "interactive agents" (the ones you talk to and acts on the go) for example, while Opencode or Roocode/Cline etc, I think Goose too, burn ~10k tokens right from the start because of extensive system prompt, some others are more minimalist, for example shai from OVH and Zed (the native Zed IDE agent), only take something like ~2k at start. But there are so many existing scaffolding that it really depend on your use case and the model you use (it's always tricky to find models with strong instruction following while still being smart enough).
Other than "interactive agents", there are also many tools that are far more minimalist yet really powerful, oriented to build autonomous agents/crews of agents and orchestrate them, such as CrewAI, maybe that's more what you are looking for (you can also build really small, simple agents with it, it is really flexible). The advantage with those is that you don't have to rewrite the logic/template from scratch, it's already done, so you mainly set the goals, knowledge, triggers, etc, but it could be almost entirely codeless. OpenCode actually offers this no-code approach, but again it's token heavy and not really suitable for small models. I think minimalist scaffoldings with clear and well crafted instructions are better suited for SLM.
But to be honest, I never dove deep into any agentic framework for myself. I played with some, some times ago, but on a daily basis I mostly use interactive ones for small punctual tasks. But what I can say is that there are many, many, many (too much?) solutions!
You could still write a python script from scratch but for other than learning purpose and having fun, using an already existing scaffolding is the way to go not to reinvent the wheel :)

I'm not sure I could help more, but feel free to add more context! If I can, I'd be happy to help!