vLLM: 'Replace the chat_template string in your tokenizer_config.json with the file contents.' ... where?

#5
by pathosethoslogos - opened

In the model card, it says the following for vLLM:

Replace the chat_template string in your tokenizer_config.json with the file contents.

I searched in tokenizer_config.json but there is no chat_template in there. Do you mean the Jinja file?

See the example below, this is what a tokenizer_config.json file should look like:

{
  "add_prefix_space": false,
  "audio_bos_token": "<|audio_start|>",
  "audio_eos_token": "<|audio_end|>",
  "audio_token": "<|audio_pad|>",
  "backend": "tokenizers",
  "bos_token": null,
  "clean_up_tokenization_spaces": false,
  "eos_token": "<|im_end|>",
  "errors": "replace",
  "image_token": "<|image_pad|>",
  "is_local": true,
  "max_length": null,
  "model_max_length": 262144,
  "model_specific_special_tokens": {
    "audio_bos_token": "<|audio_start|>",
    "audio_eos_token": "<|audio_end|>",
    "audio_token": "<|audio_pad|>",
    "image_token": "<|image_pad|>",
    "video_token": "<|video_pad|>",
    "vision_bos_token": "<|vision_start|>",
    "vision_eos_token": "<|vision_end|>"
  },
  "pad_to_multiple_of": null,
  "pad_token": "<|endoftext|>",
  "pad_token_type_id": 0,
  "padding_side": "left",
  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
  "processor_class": "Qwen3VLProcessor",
  "split_special_tokens": false,
  "tokenizer_class": "TokenizersBackend",
  "unk_token": null,
  "video_token": "<|video_pad|>",
  "vision_bos_token": "<|vision_start|>",
  "vision_eos_token": "<|vision_end|>",
  "chat_template": "..."

Interesting, it seems that chat_template is in the original Qwen's files, but not Intel or Lorbus files. There is at least one other part that's different too, besides chat_template.

Anyway, I can confirm v9 made improvements for vLLM. Still not 100% fixed with tool calls, but mostly fixed. Thanks!

--chat-template chat_template-v9.jinja works

BTW, I have now included the one line version of the template for tokenizer_config.json and I have also made some big improvements for tool calls in v10. Please test.

Sign up or log in to comment