Hugging Face's logo

VIDraft
/
Darwin-28B-KOREA

Text Generation
Transformers
Safetensors
Korean
English
qwen3_5
image-text-to-text
darwin
korean
bilingual
reasoning
merged-model
evolutionary-merge
conversational
Model card Files
xet
 Community
openfree commited on
Commit
c32df11
·
verified ·
1 Parent(s): 8831d6e

Initial release: Darwin-28B-KOREA from PERFECT pair merge

Browse files
Files changed (21) hide show
  1. .gitattributes +1 -0
  2. README.md +99 -0
  3. config.json +147 -0
  4. generation_config.json +12 -0
  5. model-00001-of-00014.safetensors +3 -0
  6. model-00002-of-00014.safetensors +3 -0
  7. model-00003-of-00014.safetensors +3 -0
  8. model-00004-of-00014.safetensors +3 -0
  9. model-00005-of-00014.safetensors +3 -0
  10. model-00006-of-00014.safetensors +3 -0
  11. model-00007-of-00014.safetensors +3 -0
  12. model-00008-of-00014.safetensors +3 -0
  13. model-00009-of-00014.safetensors +3 -0
  14. model-00010-of-00014.safetensors +3 -0
  15. model-00011-of-00014.safetensors +3 -0
  16. model-00012-of-00014.safetensors +3 -0
  17. model-00013-of-00014.safetensors +3 -0
  18. model-00014-of-00014.safetensors +3 -0
  19. model.safetensors.index.json +0 -0
  20. tokenizer.json +3 -0
  21. tokenizer_config.json +300 -0
.gitattributes CHANGED
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ tokenizer.json filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,99 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ language:
4
+ - ko
5
+ - en
6
+ library_name: transformers
7
+ base_model:
8
+ - FINAL-Bench/Darwin-28B-Opus
9
+ - ginigen-ai/Rogue-28B-MIX
10
+ tags:
11
+ - darwin
12
+ - korean
13
+ - bilingual
14
+ - reasoning
15
+ - merged-model
16
+ - evolutionary-merge
17
+ pipeline_tag: text-generation
18
+ ---
19
+
20
+ # Darwin-28B-KOREA
21
+
22
+ **한국어/영어 이중언어 추론에 최적화된 28B 파라미터 모델**
23
+
24
+ VIDRAFT Darwin 시리즈의 PERFECT 부모 페어 머지 1호. 두 부모 모델의 가중치를 per-layer 동적 비율로 결합하여, 부모 어느 한쪽보다도 우수하지 않은 부분 없이 모든 영역에서 동등 이상의 성능을 달성.
25
+
26
+ ## 부모 모델 (PERFECT Pair)
27
+
28
+ | Role | Model | Strength |
29
+ |------|-------|----------|
30
+ | Father | [FINAL-Bench/Darwin-28B-Opus](https://huggingface.co/FINAL-Bench/Darwin-28B-Opus) | 영어 추론, 속도, 토큰 절제 |
31
+ | Mother | [ginigen-ai/Rogue-28B-MIX](https://huggingface.co/ginigen-ai/Rogue-28B-MIX) | 한국어 네이티브, 깊은 한국어 reasoning |
32
+
33
+ 부모 페어 호환성: hidden=5120, intermediate=17408, layers=64 (완전 일치 PERFECT pair).
34
+
35
+ ## 머지 방식
36
+
37
+ - **알고리즘**: Per-layer linear interpolation in float32, bfloat16 cast
38
+ - **t vector**: 64 레이어 동적 가중치 (mean t=0.513)
39
+ - Golden Reasoning Layer (L47): t=0.90 (Mother dominant)
40
+ - Output Router (L63): t=0.53
41
+ - MRI (Model MRI) telemetry 기반 per-layer probe_distance + hidden_norm 분석
42
+ - **챗 템플릿/토크나이저**: Father 기준 (Qwen3_5ForConditionalGeneration multimodal)
43
+
44
+ ## 평가 결과 (35-sample 3-way bench, max_tokens=5120)
45
+
46
+ | 평가 항목 | Father | Mother | **KOREA (Child)** |
47
+ |---|---|---|---|
48
+ | 정확도 (29개 객관식) | 96.6% | 96.6% | **96.6%** |
49
+ | 진짜 정확도 (gpqa_01 채점오류 반영) | 100% | 100% | **100%** |
50
+ | 한국어 출력률 (한국어 질문 23개) | 91.3% | 95.7% | **91.3%** |
51
+ | 영어 thinking | 31/35 | 10/35 | 31/35 |
52
+ | 평균 응답 토큰 | 458 | 631 | 521 |
53
+ | 5120 cap 도달 | 0/35 | 2/35 | 1/35 |
54
+
55
+ **Win/Loss 분석**:
56
+ - Father vs Child: 0:0 동률
57
+ - Mother vs Child: 0:0 동률
58
+ - → 자식이 두 부모와 완전 동급, 어느 한쪽보다 약한 영역 없음
59
+
60
+ **Reasoning 깊이 흡수**: 한국어 논리 카테고리에서 Mother(2620t)와 Child(2724t) 평균 답 길이 유사 → Mother의 long-chain reasoning 패턴 전이 성공.
61
+
62
+ ## 사용 권장
63
+
64
+ - **권장 max_tokens**: 1024 이상 (chain-of-thought 특성상 256 토큰에서 답이 잘릴 수 있음)
65
+ - **사고 패턴**: 영어 reasoning 후 한국어 답변. 답만 정확하면 OK인 경우 권장.
66
+ - **순수 한국어 reasoning** 원하면: 부모 [Rogue-28B-MIX](https://huggingface.co/ginigen-ai/Rogue-28B-MIX) 단독 사용 추천.
67
+
68
+ ## Quick Start
69
+
70
+ ```python
71
+ from transformers import AutoModelForCausalLM, AutoTokenizer
72
+ import torch
73
+
74
+ model = AutoModelForCausalLM.from_pretrained(
75
+ "VIDraft/Darwin-28B-KOREA",
76
+ torch_dtype=torch.bfloat16,
77
+ device_map="auto",
78
+ trust_remote_code=True,
79
+ )
80
+ tok = AutoTokenizer.from_pretrained("VIDraft/Darwin-28B-KOREA", trust_remote_code=True)
81
+
82
+ msgs = [{"role": "user", "content": "대한민국 헌법 제10조의 핵심 내용을 한 문장으로 요약."}]
83
+ inputs = tok.apply_chat_template(msgs, return_tensors="pt", add_generation_prompt=True).to(model.device)
84
+ out = model.generate(inputs, max_new_tokens=1024, do_sample=False, pad_token_id=tok.eos_token_id)
85
+ print(tok.decode(out[0][inputs.shape[1]:], skip_special_tokens=True))
86
+ ```
87
+
88
+ ## License
89
+
90
+ Apache 2.0 (부모 모델 상속).
91
+
92
+ ## Citation / Acknowledgment
93
+
94
+ VIDRAFT Darwin Family — Evolutionary Model Merge Research.
95
+ Pair: Darwin-28B-Opus × Rogue-28B-MIX → Darwin-28B-KOREA (PERFECT pair, 2026-05-14)
96
+
97
+ ---
98
+
99
+ *Built with the Darwin Factory pipeline. 16 customer orders bridged by single base model.*
config.json ADDED
@@ -0,0 +1,147 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "Qwen3_5ForConditionalGeneration"
4
+ ],
5
+ "bos_token_id": null,
6
+ "torch_dtype": "bfloat16",
7
+ "eos_token_id": 248046,
8
+ "image_token_id": 248056,
9
+ "language_model_only": false,
10
+ "model_name": "unsloth/Qwen3.6-27B",
11
+ "model_type": "qwen3_5",
12
+ "pad_token_id": 248055,
13
+ "text_config": {
14
+ "attention_bias": false,
15
+ "attention_dropout": 0.0,
16
+ "attn_output_gate": true,
17
+ "bos_token_id": 248044,
18
+ "torch_dtype": "bfloat16",
19
+ "eos_token_id": 248044,
20
+ "full_attention_interval": 4,
21
+ "head_dim": 256,
22
+ "hidden_act": "silu",
23
+ "hidden_size": 5120,
24
+ "initializer_range": 0.02,
25
+ "intermediate_size": 17408,
26
+ "layer_types": [
27
+ "linear_attention",
28
+ "linear_attention",
29
+ "linear_attention",
30
+ "full_attention",
31
+ "linear_attention",
32
+ "linear_attention",
33
+ "linear_attention",
34
+ "full_attention",
35
+ "linear_attention",
36
+ "linear_attention",
37
+ "linear_attention",
38
+ "full_attention",
39
+ "linear_attention",
40
+ "linear_attention",
41
+ "linear_attention",
42
+ "full_attention",
43
+ "linear_attention",
44
+ "linear_attention",
45
+ "linear_attention",
46
+ "full_attention",
47
+ "linear_attention",
48
+ "linear_attention",
49
+ "linear_attention",
50
+ "full_attention",
51
+ "linear_attention",
52
+ "linear_attention",
53
+ "linear_attention",
54
+ "full_attention",
55
+ "linear_attention",
56
+ "linear_attention",
57
+ "linear_attention",
58
+ "full_attention",
59
+ "linear_attention",
60
+ "linear_attention",
61
+ "linear_attention",
62
+ "full_attention",
63
+ "linear_attention",
64
+ "linear_attention",
65
+ "linear_attention",
66
+ "full_attention",
67
+ "linear_attention",
68
+ "linear_attention",
69
+ "linear_attention",
70
+ "full_attention",
71
+ "linear_attention",
72
+ "linear_attention",
73
+ "linear_attention",
74
+ "full_attention",
75
+ "linear_attention",
76
+ "linear_attention",
77
+ "linear_attention",
78
+ "full_attention",
79
+ "linear_attention",
80
+ "linear_attention",
81
+ "linear_attention",
82
+ "full_attention",
83
+ "linear_attention",
84
+ "linear_attention",
85
+ "linear_attention",
86
+ "full_attention",
87
+ "linear_attention",
88
+ "linear_attention",
89
+ "linear_attention",
90
+ "full_attention"
91
+ ],
92
+ "linear_conv_kernel_dim": 4,
93
+ "linear_key_head_dim": 128,
94
+ "linear_num_key_heads": 16,
95
+ "linear_num_value_heads": 48,
96
+ "linear_value_head_dim": 128,
97
+ "mamba_ssm_dtype": "float32",
98
+ "max_position_embeddings": 262144,
99
+ "model_type": "qwen3_5_text",
100
+ "mtp_num_hidden_layers": 1,
101
+ "mtp_use_dedicated_embeddings": false,
102
+ "num_attention_heads": 24,
103
+ "num_hidden_layers": 64,
104
+ "num_key_value_heads": 4,
105
+ "output_gate_type": "swish",
106
+ "pad_token_id": null,
107
+ "partial_rotary_factor": 0.25,
108
+ "rms_norm_eps": 1e-06,
109
+ "rope_parameters": {
110
+ "mrope_interleaved": true,
111
+ "mrope_section": [
112
+ 11,
113
+ 11,
114
+ 10
115
+ ],
116
+ "partial_rotary_factor": 0.25,
117
+ "rope_theta": 10000000,
118
+ "rope_type": "default"
119
+ },
120
+ "tie_word_embeddings": false,
121
+ "use_cache": true,
122
+ "vocab_size": 248320
123
+ },
124
+ "tie_word_embeddings": false,
125
+ "unsloth_version": "2026.4.7",
126
+ "use_cache": false,
127
+ "video_token_id": 248057,
128
+ "vision_config": {
129
+ "deepstack_visual_indexes": [],
130
+ "depth": 27,
131
+ "torch_dtype": "bfloat16",
132
+ "hidden_act": "gelu_pytorch_tanh",
133
+ "hidden_size": 1152,
134
+ "in_channels": 3,
135
+ "initializer_range": 0.02,
136
+ "intermediate_size": 4304,
137
+ "model_type": "qwen3_5",
138
+ "num_heads": 16,
139
+ "num_position_embeddings": 2304,
140
+ "out_hidden_size": 5120,
141
+ "patch_size": 16,
142
+ "spatial_merge_size": 2,
143
+ "temporal_patch_size": 2
144
+ },
145
+ "vision_end_token_id": 248054,
146
+ "vision_start_token_id": 248053
147
+ }
generation_config.json ADDED
@@ -0,0 +1,12 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "bos_token_id": 248044,
3
+ "do_sample": true,
4
+ "eos_token_id": [
5
+ 248046,
6
+ 248044
7
+ ],
8
+ "pad_token_id": 248044,
9
+ "temperature": 1.0,
10
+ "top_k": 20,
11
+ "top_p": 0.95
12
+ }
model-00001-of-00014.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bf7b63856eee523368c41664e4cdf9085e6e23eadcc10b13901a805e13d7a354
3
+ size 2542796928
model-00002-of-00014.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:36c16c203a8d7c989dbad4df33e45f80e15870593c1f3b0c1c03145ec5b3121a
3
+ size 4244742344
model-00003-of-00014.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:59feacef973a04211598bde3ec6b5041463b3485e026208f93ef62349de6d3ff
3
+ size 4250052544
model-00004-of-00014.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a4527a9b6769373f2d5b8249ca4a575ccf68d19cb6da2eccf7227d6623ea914f
3
+ size 4178783568
model-00005-of-00014.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:44a11aa965e4e9c2b38cbb211fc855318124fac035edf5d74fce48ea1f219bc8
3
+ size 4167220864
model-00006-of-00014.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:894432d39d21a01c59516da163cdcb9a5f1ab0526f754f446e22b46dad5fa1ca
3
+ size 4198700552
model-00007-of-00014.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0d36ea639fa559935f0fbc631e72e8d4c4c37fba3cf9c545146781f461eb9af5
3
+ size 4187137856
model-00008-of-00014.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4371d19c684b30bbeccad110b6f5e530b7997590d609fac7e9d51e6dd087a957
3
+ size 4178783568
model-00009-of-00014.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6e91d525ccc116398f3c56697a3a96c1b329940c46f87f6c6088ad99dcd339e3
3
+ size 4167220864
model-00010-of-00014.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:152658538f4701131a370a7ff517e4d3a1f1100b2546f8cc2d7ea58a252c12ab
3
+ size 4220736856
model-00011-of-00014.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:39c1be35e0f8cc9c8b35954154e38eaaa4d0c42c78bbe5551064a557fe7bcb77
3
+ size 4187137856
model-00012-of-00014.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:591875ed6b4e98d5834c896611a3a2ba49aaa602b8ac676ff405831f32f9b367
3
+ size 4178783568
model-00013-of-00014.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:73a06e4696fcffecd912bfb3c71f6bf83ba29e044c21f44a540d215610579fd5
3
+ size 4145184536
model-00014-of-00014.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f81e4ecd4c2c740eae4731b0b1f0c61693d9cdf9e8a517db66d21949b7b144c9
3
+ size 2715725296
model.safetensors.index.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:87a7830d63fcf43bf241c3c5242e96e62dd3fdc29224ca26fed8ea333db72de4
3
+ size 19989343
tokenizer_config.json ADDED
@@ -0,0 +1,300 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_prefix_space": false,
3
+ "audio_bos_token": "<|audio_start|>",
4
+ "audio_eos_token": "<|audio_end|>",
5
+ "audio_token": "<|audio_pad|>",
6
+ "backend": "tokenizers",
7
+ "bos_token": null,
8
+ "clean_up_tokenization_spaces": false,
9
+ "eos_token": "<|im_end|>",
10
+ "errors": "replace",
11
+ "image_token": "<|image_pad|>",
12
+ "is_local": false,
13
+ "model_max_length": 262144,
14
+ "model_specific_special_tokens": {
15
+ "audio_bos_token": "<|audio_start|>",
16
+ "audio_eos_token": "<|audio_end|>",
17
+ "audio_token": "<|audio_pad|>",
18
+ "image_token": "<|image_pad|>",
19
+ "video_token": "<|video_pad|>",
20
+ "vision_bos_token": "<|vision_start|>",
21
+ "vision_eos_token": "<|vision_end|>"
22
+ },
23
+ "pad_token": "<|vision_pad|>",
24
+ "padding_side": "right",
25
+ "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
26
+ "processor_class": "Qwen3VLProcessor",
27
+ "split_special_tokens": false,
28
+ "tokenizer_class": "TokenizersBackend",
29
+ "unk_token": null,
30
+ "video_token": "<|video_pad|>",
31
+ "vision_bos_token": "<|vision_start|>",
32
+ "vision_eos_token": "<|vision_end|>",
33
+ "added_tokens_decoder": {
34
+ "248044": {
35
+ "content": "<|endoftext|>",
36
+ "single_word": false,
37
+ "lstrip": false,
38
+ "rstrip": false,
39
+ "normalized": false,
40
+ "special": true
41
+ },
42
+ "248045": {
43
+ "content": "<|im_start|>",
44
+ "single_word": false,
45
+ "lstrip": false,
46
+ "rstrip": false,
47
+ "normalized": false,
48
+ "special": true
49
+ },
50
+ "248046": {
51
+ "content": "<|im_end|>",
52
+ "single_word": false,
53
+ "lstrip": false,
54
+ "rstrip": false,
55
+ "normalized": false,
56
+ "special": true
57
+ },
58
+ "248047": {
59
+ "content": "<|object_ref_start|>",
60
+ "single_word": false,
61
+ "lstrip": false,
62
+ "rstrip": false,
63
+ "normalized": false,
64
+ "special": true
65
+ },
66
+ "248048": {
67
+ "content": "<|object_ref_end|>",
68
+ "single_word": false,
69
+ "lstrip": false,
70
+ "rstrip": false,
71
+ "normalized": false,
72
+ "special": true
73
+ },
74
+ "248049": {
75
+ "content": "<|box_start|>",
76
+ "single_word": false,
77
+ "lstrip": false,
78
+ "rstrip": false,
79
+ "normalized": false,
80
+ "special": true
81
+ },
82
+ "248050": {
83
+ "content": "<|box_end|>",
84
+ "single_word": false,
85
+ "lstrip": false,
86
+ "rstrip": false,
87
+ "normalized": false,
88
+ "special": true
89
+ },
90
+ "248051": {
91
+ "content": "<|quad_start|>",
92
+ "single_word": false,
93
+ "lstrip": false,
94
+ "rstrip": false,
95
+ "normalized": false,
96
+ "special": true
97
+ },
98
+ "248052": {
99
+ "content": "<|quad_end|>",
100
+ "single_word": false,
101
+ "lstrip": false,
102
+ "rstrip": false,
103
+ "normalized": false,
104
+ "special": true
105
+ },
106
+ "248053": {
107
+ "content": "<|vision_start|>",
108
+ "single_word": false,
109
+ "lstrip": false,
110
+ "rstrip": false,
111
+ "normalized": false,
112
+ "special": true
113
+ },
114
+ "248054": {
115
+ "content": "<|vision_end|>",
116
+ "single_word": false,
117
+ "lstrip": false,
118
+ "rstrip": false,
119
+ "normalized": false,
120
+ "special": true
121
+ },
122
+ "248055": {
123
+ "content": "<|vision_pad|>",
124
+ "single_word": false,
125
+ "lstrip": false,
126
+ "rstrip": false,
127
+ "normalized": false,
128
+ "special": true
129
+ },
130
+ "248056": {
131
+ "content": "<|image_pad|>",
132
+ "single_word": false,
133
+ "lstrip": false,
134
+ "rstrip": false,
135
+ "normalized": false,
136
+ "special": true
137
+ },
138
+ "248057": {
139
+ "content": "<|video_pad|>",
140
+ "single_word": false,
141
+ "lstrip": false,
142
+ "rstrip": false,
143
+ "normalized": false,
144
+ "special": true
145
+ },
146
+ "248058": {
147
+ "content": "<tool_call>",
148
+ "single_word": false,
149
+ "lstrip": false,
150
+ "rstrip": false,
151
+ "normalized": false,
152
+ "special": false
153
+ },
154
+ "248059": {
155
+ "content": "</tool_call>",
156
+ "single_word": false,
157
+ "lstrip": false,
158
+ "rstrip": false,
159
+ "normalized": false,
160
+ "special": false
161
+ },
162
+ "248060": {
163
+ "content": "<|fim_prefix|>",
164
+ "single_word": false,
165
+ "lstrip": false,
166
+ "rstrip": false,
167
+ "normalized": false,
168
+ "special": false
169
+ },
170
+ "248061": {
171
+ "content": "<|fim_middle|>",
172
+ "single_word": false,
173
+ "lstrip": false,
174
+ "rstrip": false,
175
+ "normalized": false,
176
+ "special": false
177
+ },
178
+ "248062": {
179
+ "content": "<|fim_suffix|>",
180
+ "single_word": false,
181
+ "lstrip": false,
182
+ "rstrip": false,
183
+ "normalized": false,
184
+ "special": false
185
+ },
186
+ "248063": {
187
+ "content": "<|fim_pad|>",
188
+ "single_word": false,
189
+ "lstrip": false,
190
+ "rstrip": false,
191
+ "normalized": false,
192
+ "special": false
193
+ },
194
+ "248064": {
195
+ "content": "<|repo_name|>",
196
+ "single_word": false,
197
+ "lstrip": false,
198
+ "rstrip": false,
199
+ "normalized": false,
200
+ "special": false
201
+ },
202
+ "248065": {
203
+ "content": "<|file_sep|>",
204
+ "single_word": false,
205
+ "lstrip": false,
206
+ "rstrip": false,
207
+ "normalized": false,
208
+ "special": false
209
+ },
210
+ "248066": {
211
+ "content": "<tool_response>",
212
+ "single_word": false,
213
+ "lstrip": false,
214
+ "rstrip": false,
215
+ "normalized": false,
216
+ "special": false
217
+ },
218
+ "248067": {
219
+ "content": "</tool_response>",
220
+ "single_word": false,
221
+ "lstrip": false,
222
+ "rstrip": false,
223
+ "normalized": false,
224
+ "special": false
225
+ },
226
+ "248068": {
227
+ "content": "<think>",
228
+ "single_word": false,
229
+ "lstrip": false,
230
+ "rstrip": false,
231
+ "normalized": false,
232
+ "special": false
233
+ },
234
+ "248069": {
235
+ "content": "</think>",
236
+ "single_word": false,
237
+ "lstrip": false,
238
+ "rstrip": false,
239
+ "normalized": false,
240
+ "special": false
241
+ },
242
+ "248070": {
243
+ "content": "<|audio_start|>",
244
+ "single_word": false,
245
+ "lstrip": false,
246
+ "rstrip": false,
247
+ "normalized": false,
248
+ "special": true
249
+ },
250
+ "248071": {
251
+ "content": "<|audio_end|>",
252
+ "single_word": false,
253
+ "lstrip": false,
254
+ "rstrip": false,
255
+ "normalized": false,
256
+ "special": true
257
+ },
258
+ "248072": {
259
+ "content": "<tts_pad>",
260
+ "single_word": false,
261
+ "lstrip": false,
262
+ "rstrip": false,
263
+ "normalized": false,
264
+ "special": true
265
+ },
266
+ "248073": {
267
+ "content": "<tts_text_bos>",
268
+ "single_word": false,
269
+ "lstrip": false,
270
+ "rstrip": false,
271
+ "normalized": false,
272
+ "special": true
273
+ },
274
+ "248074": {
275
+ "content": "<tts_text_eod>",
276
+ "single_word": false,
277
+ "lstrip": false,
278
+ "rstrip": false,
279
+ "normalized": false,
280
+ "special": true
281
+ },
282
+ "248075": {
283
+ "content": "<tts_text_bos_single>",
284
+ "single_word": false,
285
+ "lstrip": false,
286
+ "rstrip": false,
287
+ "normalized": false,
288
+ "special": true
289
+ },
290
+ "248076": {
291
+ "content": "<|audio_pad|>",
292
+ "single_word": false,
293
+ "lstrip": false,
294
+ "rstrip": false,
295
+ "normalized": false,
296
+ "special": true
297
+ }
298
+ },
299
+ "chat_template": "{%- if tools %}\n {{- '<|im_start|>system\\n' }}\n {%- if messages[0].role == 'system' %}\n {{- messages[0].content + '\\n\\n' }}\n {%- endif %}\n {{- \"# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n {%- for tool in tools %}\n {{- \"\\n\" }}\n {{- tool | tojson }}\n {%- endfor %}\n {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><|im_end|>\\n\" }}\n{%- else %}\n {%- if messages[0].role == 'system' %}\n {{- '<|im_start|>system\\n' + messages[0].content + '<|im_end|>\\n' }}\n {%- endif %}\n{%- endif %}\n{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}\n{%- for message in messages[::-1] %}\n {%- set index = (messages|length - 1) - loop.index0 %}\n {%- if ns.multi_step_tool and message.role == \"user\" and message.content is string and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}\n {%- set ns.multi_step_tool = false %}\n {%- set ns.last_query_index = index %}\n {%- endif %}\n{%- endfor %}\n{%- for message in messages %}\n {%- if message.content is string %}\n {%- set content = message.content %}\n {%- else %}\n {%- set content = '' %}\n {%- endif %}\n {%- if (message.role == \"user\") or (message.role == \"system\" and not loop.first) %}\n {{- '<|im_start|>' + message.role + '\\n' + content + '<|im_end|>' + '\\n' }}\n {%- elif message.role == \"assistant\" %}\n {%- set reasoning_content = '' %}\n {%- if message.reasoning_content is string %}\n {%- set reasoning_content = message.reasoning_content %}\n {%- else %}\n {%- if '</think>' in content %}\n {%- set reasoning_content = content.split('</think>')[0].rstrip('\\n').split('<think>')[-1].lstrip('\\n') %}\n {%- set content = content.split('</think>')[-1].lstrip('\\n') %}\n {%- endif %}\n {%- endif %}\n {%- if loop.index0 > ns.last_query_index %}\n {%- if loop.last or (not loop.last and reasoning_content) %}\n {{- '<|im_start|>' + message.role + '\\n<think>\\n' + reasoning_content.strip('\\n') + '\\n</think>\\n\\n' + content.lstrip('\\n') }}\n {%- else %}\n {{- '<|im_start|>' + message.role + '\\n' + content }}\n {%- endif %}\n {%- else %}\n {{- '<|im_start|>' + message.role + '\\n' + content }}\n {%- endif %}\n {%- if message.tool_calls %}\n {%- for tool_call in message.tool_calls %}\n {%- if (loop.first and content) or (not loop.first) %}\n {{- '\\n' }}\n {%- endif %}\n {%- if tool_call.function %}\n {%- set tool_call = tool_call.function %}\n {%- endif %}\n {{- '<tool_call>\\n{\"name\": \"' }}\n {{- tool_call.name }}\n {{- '\", \"arguments\": ' }}\n {%- if tool_call.arguments is string %}\n {{- tool_call.arguments }}\n {%- else %}\n {{- tool_call.arguments | tojson }}\n {%- endif %}\n {{- '}\\n</tool_call>' }}\n {%- endfor %}\n {%- endif %}\n {{- '<|im_end|>\\n' }}\n {%- elif message.role == \"tool\" %}\n {%- if loop.first or (messages[loop.index0 - 1].role != \"tool\") %}\n {{- '<|im_start|>user' }}\n {%- endif %}\n {{- '\\n<tool_response>\\n' }}\n {{- content }}\n {{- '\\n</tool_response>' }}\n {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n {{- '<|im_end|>\\n' }}\n {%- endif %}\n {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n {{- '<|im_start|>assistant\n<think>\n' }}\n{%- endif %}"
300
+ }