dluthyc commited on
Commit
0774bb8
·
verified ·
1 Parent(s): 6a9e13b

Update chat_template.jinja

Browse files

Title: Fix partial assistant prefills when `continue_final_message=True`

Summary:
The current `chat_template.jinja` always appends `<|im_end|>` after assistant turns.
When callers disable `add_generation_prompt` and use a final assistant message as a prefill, this closes the assistant message before generation starts, so the model tends to emit an empty continuation or start a new turn incorrectly.

This patch makes the final assistant turn honor Hugging Face's `continue_final_message` contract by skipping the closing `<|im_end|>` only when:

- the current message is the last message
- `continue_final_message` is defined
- `continue_final_message` is `true`

All other rendering paths are unchanged.

Minimal repro:
```python
messages = [
{"role": "user", "content": "Reply in JSON."},
{"role": "assistant", "content": '{"name": "'},
]

tokenizer.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=False,
continue_final_message=True,
)
```

Before:
```text
<|im_start|>user
Reply in JSON.<|im_end|>
<|im_start|>assistant
{"name": "<|im_end|>
```

After:
```text
<|im_start|>user
Reply in JSON.<|im_end|>
<|im_start|>assistant
{"name": "
```

Files changed (1) hide show
  1. chat_template.jinja +3 -1
chat_template.jinja CHANGED
@@ -127,7 +127,9 @@
127
  {{- '</function>\n</tool_call>' }}
128
  {%- endfor %}
129
  {%- endif %}
130
- {{- '<|im_end|>\n' }}
 
 
131
  {%- elif message.role == "tool" %}
132
  {%- if loop.previtem and loop.previtem.role != "tool" %}
133
  {{- '<|im_start|>user' }}
 
127
  {{- '</function>\n</tool_call>' }}
128
  {%- endfor %}
129
  {%- endif %}
130
+ {%- if not (loop.last and continue_final_message is defined and continue_final_message) %}
131
+ {{- '<|im_end|>\n' }}
132
+ {%- endif %}
133
  {%- elif message.role == "tool" %}
134
  {%- if loop.previtem and loop.previtem.role != "tool" %}
135
  {{- '<|im_start|>user' }}