Upload folder using huggingface_hub
Browse files
README.md
CHANGED
|
@@ -15,7 +15,7 @@ tags:
|
|
| 15 |
|
| 16 |
# Fixed Chat Templates for Qwen 3.5 & 3.6
|
| 17 |
|
| 18 |
-
Drop-in Jinja templates that fix
|
| 19 |
|
| 20 |
## Why you need this
|
| 21 |
|
|
@@ -23,8 +23,8 @@ The official Qwen templates have bugs that break real usage:
|
|
| 23 |
|
| 24 |
| Problem | Impact |
|
| 25 |
|---------|--------|
|
| 26 |
-
| Tool calls
|
| 27 |
-
| `developer` role
|
| 28 |
| Empty thinking blocks spam context | Every past turn gets wrapped in tags, even with nothing inside |
|
| 29 |
| No way to toggle thinking | You're stuck with whatever the model defaults to |
|
| 30 |
| Qwen 3.6: `</thinking>` hallucination | Model sometimes generates the wrong closing tag; parser fails |
|
|
@@ -105,7 +105,7 @@ The official template iterates tool call arguments with `|items`:
|
|
| 105 |
{%- for key, value in tool_call.arguments|items %}
|
| 106 |
```
|
| 107 |
|
| 108 |
-
Python's Jinja supports `|items`. C++ runtimes (LM Studio, llama.cpp, MLX) do not. This template uses direct dictionary key lookups instead:
|
| 109 |
|
| 110 |
```jinja
|
| 111 |
{%- for args_name in tool_call.arguments %}
|
|
@@ -147,7 +147,7 @@ It also handles interrupted generation (max tokens hit mid-thought) by rescuing
|
|
| 147 |
|
| 148 |
## Arguments serialization
|
| 149 |
|
| 150 |
-
The official template serializes argument values with `|tojson` unconditionally, which turns Python `True` into JSON `true` correctly but
|
| 151 |
|
| 152 |
</details>
|
| 153 |
|
|
@@ -156,8 +156,8 @@ The official template serializes argument values with `|tojson` unconditionally,
|
|
| 156 |
|
| 157 |
| Feature | Official | LuffyTheFox | mod-ellary | Pneuny | **This** |
|
| 158 |
|---------|----------|-------------|------------|--------|----------|
|
| 159 |
-
| Tool arguments |
|
| 160 |
-
| `\|safe` removed |
|
| 161 |
| `developer` role | Missing | Missing | Missing | Missing | **Added** |
|
| 162 |
| Thinking toggle | None | None | `/think` (system only) | None | **`<\|think_off\|>` anywhere** |
|
| 163 |
| Empty think in history | Broken | Broken | Tags omitted | Broken | **Fixed** |
|
|
@@ -171,8 +171,8 @@ The official template serializes argument values with `|tojson` unconditionally,
|
|
| 171 |
|
| 172 |
| Feature | Official | **This** |
|
| 173 |
|---------|----------|----------|
|
| 174 |
-
| Tool arguments |
|
| 175 |
-
| `\|safe` removed |
|
| 176 |
| `developer` role | Missing | **Added** |
|
| 177 |
| Thinking toggle | None | **`<\|think_off\|>` anywhere** |
|
| 178 |
| `preserve_thinking` | Spams empty blocks | **Dynamic length checks** |
|
|
|
|
| 15 |
|
| 16 |
# Fixed Chat Templates for Qwen 3.5 & 3.6
|
| 17 |
|
| 18 |
+
Drop-in Jinja templates that fix rendering errors, token waste, and missing features in the official Qwen chat templates. Works in LM Studio, llama.cpp, vLLM, MLX, oMLX, and any engine that supports HuggingFace Jinja templates.
|
| 19 |
|
| 20 |
## Why you need this
|
| 21 |
|
|
|
|
| 23 |
|
| 24 |
| Problem | Impact |
|
| 25 |
|---------|--------|
|
| 26 |
+
| Tool calls fail on C++ engines | `|items` filter doesn't exist in LM Studio, llama.cpp, MLX, oMLX — tool calls produce a template error |
|
| 27 |
+
| `developer` role rejected | Modern APIs send it; the official template raises an error |
|
| 28 |
| Empty thinking blocks spam context | Every past turn gets wrapped in tags, even with nothing inside |
|
| 29 |
| No way to toggle thinking | You're stuck with whatever the model defaults to |
|
| 30 |
| Qwen 3.6: `</thinking>` hallucination | Model sometimes generates the wrong closing tag; parser fails |
|
|
|
|
| 105 |
{%- for key, value in tool_call.arguments|items %}
|
| 106 |
```
|
| 107 |
|
| 108 |
+
Python's Jinja supports `|items`. C++ runtimes (LM Studio, llama.cpp, MLX) do not — the template produces a rendering error instead of output. This template uses direct dictionary key lookups instead:
|
| 109 |
|
| 110 |
```jinja
|
| 111 |
{%- for args_name in tool_call.arguments %}
|
|
|
|
| 147 |
|
| 148 |
## Arguments serialization
|
| 149 |
|
| 150 |
+
The official template serializes argument values with `|tojson` unconditionally, which turns Python `True` into JSON `true` correctly but fails when the value is already a string. The fixed templates check the type first — strings pass through as-is, everything else goes through `|tojson`.
|
| 151 |
|
| 152 |
</details>
|
| 153 |
|
|
|
|
| 156 |
|
| 157 |
| Feature | Official | LuffyTheFox | mod-ellary | Pneuny | **This** |
|
| 158 |
|---------|----------|-------------|------------|--------|----------|
|
| 159 |
+
| Tool arguments | Fails | Fixed | Missing | Fixed | **Fixed** |
|
| 160 |
+
| `\|safe` removed | Fails | Fixed | Missing | Fixed | **Fixed** |
|
| 161 |
| `developer` role | Missing | Missing | Missing | Missing | **Added** |
|
| 162 |
| Thinking toggle | None | None | `/think` (system only) | None | **`<\|think_off\|>` anywhere** |
|
| 163 |
| Empty think in history | Broken | Broken | Tags omitted | Broken | **Fixed** |
|
|
|
|
| 171 |
|
| 172 |
| Feature | Official | **This** |
|
| 173 |
|---------|----------|----------|
|
| 174 |
+
| Tool arguments | Fails (`\|items`) | **Fixed** |
|
| 175 |
+
| `\|safe` removed | Fails | **Fixed** |
|
| 176 |
| `developer` role | Missing | **Added** |
|
| 177 |
| Thinking toggle | None | **`<\|think_off\|>` anywhere** |
|
| 178 |
| `preserve_thinking` | Spams empty blocks | **Dynamic length checks** |
|