Failed to run on llama.cpp b7873 due to chat template parsing issue
#1
by darkgeekyang - opened
Hi,
Yesterday this model failed to run after my updating llama.cpp to b7873:
...
srv load_model: use `--cache-ram 0` to disable the prompt cache
srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391
srv init: init: chat template parsing error:
------------
While executing FilterExpression at line 6, column 86 in source:
... none -%}{{- '<functions>' -}}{{- tools | tojson -}}{{- '</functions>' -}}{%- el...
^
Error: Unknown (built-in) filter 'tojson' for type Undefined (hint: 'tools')
srv init: init: please consider disabling jinja via --no-jinja, or use a custom chat template via --chat-template
srv init: init: for example: --no-jinja --chat-template chatml
srv operator(): operator(): cleaning up before exit...
main: exiting due to model loading error
The command I use to load model:
llama-server --timeout 1200 --jinja --port 8081 --mmap -hf unsloth/Olmo-3-7B-Instruct-GGUF:IQ4_NL -c 10240 --temp 0.6 --top-p 0.95
I tweak the template a bit as a workaround and it works now:
--- /tmp/olmo.jinja 2026-01-31 13:15:50.180171219 +0800
+++ /home/justin/Shares/olmo3-7b-instruct.jinja 2026-01-31 01:45:51.137615895 +0800
@@ -2,7 +2,7 @@
{%- set has_system = messages | selectattr("role", "equalto", "system") | list | length > 0 -%}
{%- if not has_system -%}
{{- "<|im_start|>system\nYou are a helpful function-calling AI assistant. " -}}
- {%- if tools is none -%}
+ {%- if tools is none or not tools is defined -%}
{{- "You do not currently have access to any functions. <functions></functions><|im_end|>\n" -}}
{%- else -%}
{{- "You are provided with function signatures within <functions></functions> XML tags. You may call one or more functions to assist with the user query. Output any function calls within <function_calls></function_calls> XML tags. Do not make assumptions about what values to plug into functions." -}}
@@ -14,7 +14,7 @@
{%- for message in messages -%}
{%- if message["role"] == "system" -%}
{{- "<|im_start|>system\n" + message["content"] -}}
- {%- if tools is not none -%}
+ {%- if tools is not none and tools is defined -%}
{{- "<functions>" -}}
{{- tools | tojson -}}
{{- "</functions>" -}}
Will unsloth team update it in gguf?