Darwin-28B-KOREA / README.md
openfree's picture
Initial release: Darwin-28B-KOREA from PERFECT pair merge
c32df11 verified
---
license: apache-2.0
language:
- ko
- en
library_name: transformers
base_model:
- FINAL-Bench/Darwin-28B-Opus
- ginigen-ai/Rogue-28B-MIX
tags:
- darwin
- korean
- bilingual
- reasoning
- merged-model
- evolutionary-merge
pipeline_tag: text-generation
---
# Darwin-28B-KOREA
**ํ•œ๊ตญ์–ด/์˜์–ด ์ด์ค‘์–ธ์–ด ์ถ”๋ก ์— ์ตœ์ ํ™”๋œ 28B ํŒŒ๋ผ๋ฏธํ„ฐ ๋ชจ๋ธ**
VIDRAFT Darwin ์‹œ๋ฆฌ์ฆˆ์˜ PERFECT ๋ถ€๋ชจ ํŽ˜์–ด ๋จธ์ง€ 1ํ˜ธ. ๋‘ ๋ถ€๋ชจ ๋ชจ๋ธ์˜ ๊ฐ€์ค‘์น˜๋ฅผ per-layer ๋™์  ๋น„์œจ๋กœ ๊ฒฐํ•ฉํ•˜์—ฌ, ๋ถ€๋ชจ ์–ด๋А ํ•œ์ชฝ๋ณด๋‹ค๋„ ์šฐ์ˆ˜ํ•˜์ง€ ์•Š์€ ๋ถ€๋ถ„ ์—†์ด ๋ชจ๋“  ์˜์—ญ์—์„œ ๋™๋“ฑ ์ด์ƒ์˜ ์„ฑ๋Šฅ์„ ๋‹ฌ์„ฑ.
## ๋ถ€๋ชจ ๋ชจ๋ธ (PERFECT Pair)
| Role | Model | Strength |
|------|-------|----------|
| Father | [FINAL-Bench/Darwin-28B-Opus](https://huggingface.co/FINAL-Bench/Darwin-28B-Opus) | ์˜์–ด ์ถ”๋ก , ์†๋„, ํ† ํฐ ์ ˆ์ œ |
| Mother | [ginigen-ai/Rogue-28B-MIX](https://huggingface.co/ginigen-ai/Rogue-28B-MIX) | ํ•œ๊ตญ์–ด ๋„ค์ดํ‹ฐ๋ธŒ, ๊นŠ์€ ํ•œ๊ตญ์–ด reasoning |
๋ถ€๋ชจ ํŽ˜์–ด ํ˜ธํ™˜์„ฑ: hidden=5120, intermediate=17408, layers=64 (์™„์ „ ์ผ์น˜ PERFECT pair).
## ๋จธ์ง€ ๋ฐฉ์‹
- **์•Œ๊ณ ๋ฆฌ์ฆ˜**: Per-layer linear interpolation in float32, bfloat16 cast
- **t vector**: 64 ๋ ˆ์ด์–ด ๋™์  ๊ฐ€์ค‘์น˜ (mean t=0.513)
- Golden Reasoning Layer (L47): t=0.90 (Mother dominant)
- Output Router (L63): t=0.53
- MRI (Model MRI) telemetry ๊ธฐ๋ฐ˜ per-layer probe_distance + hidden_norm ๋ถ„์„
- **์ฑ— ํ…œํ”Œ๋ฆฟ/ํ† ํฌ๋‚˜์ด์ €**: Father ๊ธฐ์ค€ (Qwen3_5ForConditionalGeneration multimodal)
## ํ‰๊ฐ€ ๊ฒฐ๊ณผ (35-sample 3-way bench, max_tokens=5120)
| ํ‰๊ฐ€ ํ•ญ๋ชฉ | Father | Mother | **KOREA (Child)** |
|---|---|---|---|
| ์ •ํ™•๋„ (29๊ฐœ ๊ฐ๊ด€์‹) | 96.6% | 96.6% | **96.6%** |
| ์ง„์งœ ์ •ํ™•๋„ (gpqa_01 ์ฑ„์ ์˜ค๋ฅ˜ ๋ฐ˜์˜) | 100% | 100% | **100%** |
| ํ•œ๊ตญ์–ด ์ถœ๋ ฅ๋ฅ  (ํ•œ๊ตญ์–ด ์งˆ๋ฌธ 23๊ฐœ) | 91.3% | 95.7% | **91.3%** |
| ์˜์–ด thinking | 31/35 | 10/35 | 31/35 |
| ํ‰๊ท  ์‘๋‹ต ํ† ํฐ | 458 | 631 | 521 |
| 5120 cap ๋„๋‹ฌ | 0/35 | 2/35 | 1/35 |
**Win/Loss ๋ถ„์„**:
- Father vs Child: 0:0 ๋™๋ฅ 
- Mother vs Child: 0:0 ๋™๋ฅ 
- โ†’ ์ž์‹์ด ๋‘ ๋ถ€๋ชจ์™€ ์™„์ „ ๋™๊ธ‰, ์–ด๋А ํ•œ์ชฝ๋ณด๋‹ค ์•ฝํ•œ ์˜์—ญ ์—†์Œ
**Reasoning ๊นŠ์ด ํก์ˆ˜**: ํ•œ๊ตญ์–ด ๋…ผ๋ฆฌ ์นดํ…Œ๊ณ ๋ฆฌ์—์„œ Mother(2620t)์™€ Child(2724t) ํ‰๊ท  ๋‹ต ๊ธธ์ด ์œ ์‚ฌ โ†’ Mother์˜ long-chain reasoning ํŒจํ„ด ์ „์ด ์„ฑ๊ณต.
## ์‚ฌ์šฉ ๊ถŒ์žฅ
- **๊ถŒ์žฅ max_tokens**: 1024 ์ด์ƒ (chain-of-thought ํŠน์„ฑ์ƒ 256 ํ† ํฐ์—์„œ ๋‹ต์ด ์ž˜๋ฆด ์ˆ˜ ์žˆ์Œ)
- **์‚ฌ๊ณ  ํŒจํ„ด**: ์˜์–ด reasoning ํ›„ ํ•œ๊ตญ์–ด ๋‹ต๋ณ€. ๋‹ต๋งŒ ์ •ํ™•ํ•˜๋ฉด OK์ธ ๊ฒฝ์šฐ ๊ถŒ์žฅ.
- **์ˆœ์ˆ˜ ํ•œ๊ตญ์–ด reasoning** ์›ํ•˜๋ฉด: ๋ถ€๋ชจ [Rogue-28B-MIX](https://huggingface.co/ginigen-ai/Rogue-28B-MIX) ๋‹จ๋… ์‚ฌ์šฉ ์ถ”์ฒœ.
## Quick Start
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
model = AutoModelForCausalLM.from_pretrained(
"VIDraft/Darwin-28B-KOREA",
torch_dtype=torch.bfloat16,
device_map="auto",
trust_remote_code=True,
)
tok = AutoTokenizer.from_pretrained("VIDraft/Darwin-28B-KOREA", trust_remote_code=True)
msgs = [{"role": "user", "content": "๋Œ€ํ•œ๋ฏผ๊ตญ ํ—Œ๋ฒ• ์ œ10์กฐ์˜ ํ•ต์‹ฌ ๋‚ด์šฉ์„ ํ•œ ๋ฌธ์žฅ์œผ๋กœ ์š”์•ฝ."}]
inputs = tok.apply_chat_template(msgs, return_tensors="pt", add_generation_prompt=True).to(model.device)
out = model.generate(inputs, max_new_tokens=1024, do_sample=False, pad_token_id=tok.eos_token_id)
print(tok.decode(out[0][inputs.shape[1]:], skip_special_tokens=True))
```
## License
Apache 2.0 (๋ถ€๋ชจ ๋ชจ๋ธ ์ƒ์†).
## Citation / Acknowledgment
VIDRAFT Darwin Family โ€” Evolutionary Model Merge Research.
Pair: Darwin-28B-Opus ร— Rogue-28B-MIX โ†’ Darwin-28B-KOREA (PERFECT pair, 2026-05-14)
---
*Built with the Darwin Factory pipeline. 16 customer orders bridged by single base model.*