File size: 3,766 Bytes
c32df11
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
---
license: apache-2.0
language:
- ko
- en
library_name: transformers
base_model:
- FINAL-Bench/Darwin-28B-Opus
- ginigen-ai/Rogue-28B-MIX
tags:
- darwin
- korean
- bilingual
- reasoning
- merged-model
- evolutionary-merge
pipeline_tag: text-generation
---

# Darwin-28B-KOREA

**ํ•œ๊ตญ์–ด/์˜์–ด ์ด์ค‘์–ธ์–ด ์ถ”๋ก ์— ์ตœ์ ํ™”๋œ 28B ํŒŒ๋ผ๋ฏธํ„ฐ ๋ชจ๋ธ**

VIDRAFT Darwin ์‹œ๋ฆฌ์ฆˆ์˜ PERFECT ๋ถ€๋ชจ ํŽ˜์–ด ๋จธ์ง€ 1ํ˜ธ. ๋‘ ๋ถ€๋ชจ ๋ชจ๋ธ์˜ ๊ฐ€์ค‘์น˜๋ฅผ per-layer ๋™์  ๋น„์œจ๋กœ ๊ฒฐํ•ฉํ•˜์—ฌ, ๋ถ€๋ชจ ์–ด๋А ํ•œ์ชฝ๋ณด๋‹ค๋„ ์šฐ์ˆ˜ํ•˜์ง€ ์•Š์€ ๋ถ€๋ถ„ ์—†์ด ๋ชจ๋“  ์˜์—ญ์—์„œ ๋™๋“ฑ ์ด์ƒ์˜ ์„ฑ๋Šฅ์„ ๋‹ฌ์„ฑ.

## ๋ถ€๋ชจ ๋ชจ๋ธ (PERFECT Pair)

| Role | Model | Strength |
|------|-------|----------|
| Father | [FINAL-Bench/Darwin-28B-Opus](https://huggingface.co/FINAL-Bench/Darwin-28B-Opus) | ์˜์–ด ์ถ”๋ก , ์†๋„, ํ† ํฐ ์ ˆ์ œ |
| Mother | [ginigen-ai/Rogue-28B-MIX](https://huggingface.co/ginigen-ai/Rogue-28B-MIX) | ํ•œ๊ตญ์–ด ๋„ค์ดํ‹ฐ๋ธŒ, ๊นŠ์€ ํ•œ๊ตญ์–ด reasoning |

๋ถ€๋ชจ ํŽ˜์–ด ํ˜ธํ™˜์„ฑ: hidden=5120, intermediate=17408, layers=64 (์™„์ „ ์ผ์น˜ PERFECT pair).

## ๋จธ์ง€ ๋ฐฉ์‹

- **์•Œ๊ณ ๋ฆฌ์ฆ˜**: Per-layer linear interpolation in float32, bfloat16 cast
- **t vector**: 64 ๋ ˆ์ด์–ด ๋™์  ๊ฐ€์ค‘์น˜ (mean t=0.513)
  - Golden Reasoning Layer (L47): t=0.90 (Mother dominant)
  - Output Router (L63): t=0.53
  - MRI (Model MRI) telemetry ๊ธฐ๋ฐ˜ per-layer probe_distance + hidden_norm ๋ถ„์„
- **์ฑ— ํ…œํ”Œ๋ฆฟ/ํ† ํฌ๋‚˜์ด์ €**: Father ๊ธฐ์ค€ (Qwen3_5ForConditionalGeneration multimodal)

## ํ‰๊ฐ€ ๊ฒฐ๊ณผ (35-sample 3-way bench, max_tokens=5120)

| ํ‰๊ฐ€ ํ•ญ๋ชฉ | Father | Mother | **KOREA (Child)** |
|---|---|---|---|
| ์ •ํ™•๋„ (29๊ฐœ ๊ฐ๊ด€์‹) | 96.6% | 96.6% | **96.6%** |
| ์ง„์งœ ์ •ํ™•๋„ (gpqa_01 ์ฑ„์ ์˜ค๋ฅ˜ ๋ฐ˜์˜) | 100% | 100% | **100%** |
| ํ•œ๊ตญ์–ด ์ถœ๋ ฅ๋ฅ  (ํ•œ๊ตญ์–ด ์งˆ๋ฌธ 23๊ฐœ) | 91.3% | 95.7% | **91.3%** |
| ์˜์–ด thinking | 31/35 | 10/35 | 31/35 |
| ํ‰๊ท  ์‘๋‹ต ํ† ํฐ | 458 | 631 | 521 |
| 5120 cap ๋„๋‹ฌ | 0/35 | 2/35 | 1/35 |

**Win/Loss ๋ถ„์„**:
- Father vs Child: 0:0 ๋™๋ฅ 
- Mother vs Child: 0:0 ๋™๋ฅ 
- โ†’ ์ž์‹์ด ๋‘ ๋ถ€๋ชจ์™€ ์™„์ „ ๋™๊ธ‰, ์–ด๋А ํ•œ์ชฝ๋ณด๋‹ค ์•ฝํ•œ ์˜์—ญ ์—†์Œ

**Reasoning ๊นŠ์ด ํก์ˆ˜**: ํ•œ๊ตญ์–ด ๋…ผ๋ฆฌ ์นดํ…Œ๊ณ ๋ฆฌ์—์„œ Mother(2620t)์™€ Child(2724t) ํ‰๊ท  ๋‹ต ๊ธธ์ด ์œ ์‚ฌ โ†’ Mother์˜ long-chain reasoning ํŒจํ„ด ์ „์ด ์„ฑ๊ณต.

## ์‚ฌ์šฉ ๊ถŒ์žฅ

- **๊ถŒ์žฅ max_tokens**: 1024 ์ด์ƒ (chain-of-thought ํŠน์„ฑ์ƒ 256 ํ† ํฐ์—์„œ ๋‹ต์ด ์ž˜๋ฆด ์ˆ˜ ์žˆ์Œ)
- **์‚ฌ๊ณ  ํŒจํ„ด**: ์˜์–ด reasoning ํ›„ ํ•œ๊ตญ์–ด ๋‹ต๋ณ€. ๋‹ต๋งŒ ์ •ํ™•ํ•˜๋ฉด OK์ธ ๊ฒฝ์šฐ ๊ถŒ์žฅ.
- **์ˆœ์ˆ˜ ํ•œ๊ตญ์–ด reasoning** ์›ํ•˜๋ฉด: ๋ถ€๋ชจ [Rogue-28B-MIX](https://huggingface.co/ginigen-ai/Rogue-28B-MIX) ๋‹จ๋… ์‚ฌ์šฉ ์ถ”์ฒœ.

## Quick Start

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model = AutoModelForCausalLM.from_pretrained(
    "VIDraft/Darwin-28B-KOREA",
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)
tok = AutoTokenizer.from_pretrained("VIDraft/Darwin-28B-KOREA", trust_remote_code=True)

msgs = [{"role": "user", "content": "๋Œ€ํ•œ๋ฏผ๊ตญ ํ—Œ๋ฒ• ์ œ10์กฐ์˜ ํ•ต์‹ฌ ๋‚ด์šฉ์„ ํ•œ ๋ฌธ์žฅ์œผ๋กœ ์š”์•ฝ."}]
inputs = tok.apply_chat_template(msgs, return_tensors="pt", add_generation_prompt=True).to(model.device)
out = model.generate(inputs, max_new_tokens=1024, do_sample=False, pad_token_id=tok.eos_token_id)
print(tok.decode(out[0][inputs.shape[1]:], skip_special_tokens=True))
```

## License

Apache 2.0 (๋ถ€๋ชจ ๋ชจ๋ธ ์ƒ์†).

## Citation / Acknowledgment

VIDRAFT Darwin Family โ€” Evolutionary Model Merge Research.
Pair: Darwin-28B-Opus ร— Rogue-28B-MIX โ†’ Darwin-28B-KOREA (PERFECT pair, 2026-05-14)

---

*Built with the Darwin Factory pipeline. 16 customer orders bridged by single base model.*