Warecube commited on
Commit
aaf9908
Β·
verified Β·
1 Parent(s): 0248f0c

upload README.md

Browse files
Files changed (1) hide show
  1. README.md +160 -0
README.md ADDED
@@ -0,0 +1,160 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ language:
4
+ - ko
5
+ - en
6
+ library_name: transformers
7
+ tags:
8
+ - korean
9
+ - reasoning
10
+ - darwin
11
+ - evolutionary-merge
12
+ base_model:
13
+ - FINAL-Bench/Darwin-27B-Opus
14
+ ---
15
+
16
+ # Warecube-KO-27B
17
+
18
+ ν•œκ΅­μ–΄ reasoning λͺ¨λΈ β€” Darwin 진화적 λ¨Έμ§€ 기반.
19
+
20
+ ---
21
+
22
+ ## 🧬 Darwin μ§„ν™” 컨셉
23
+
24
+ λ³Έ λͺ¨λΈμ€ **Darwin V7 진화적 λͺ¨λΈ λ¨Έμ§€(Evolutionary Model Merge)**
25
+ νŒ¨λŸ¬λ‹€μž„μœΌλ‘œ μ œμž‘λ˜μ—ˆμŠ΅λ‹ˆλ‹€.
26
+
27
+ ```
28
+ μžμ—° μ§„ν™” Darwin λ¨Έμ§€
29
+ ───────── ───────────
30
+ μœ μ „μž ꡐ차 (crossover) β†’ κ°€μ€‘μΉ˜ λͺ¨λ“ˆλ³„ λΉ„μœ¨ κ²°ν•©
31
+ μžμ—° 선택 (selection) β†’ 적합도 평가 ν›„ 졜적 후손 선별
32
+ μ„ΈλŒ€ μ§„ν™” (generations) β†’ λ‹€μ„ΈλŒ€ λ¨Έμ§€Β·μ •μ œ 반볡
33
+ 적자 생쑴 β†’ K-AI 도메인 우수 μžμ†λ§Œ 보쑴
34
+ ```
35
+
36
+ λΆ€λͺ¨μ˜ λŠ₯λ ₯이 μžμ‹ λͺ¨λΈλ‘œ **μœ μ „μ μœΌλ‘œ κ³„μŠΉ**되며,
37
+ μ„ΈλŒ€λ₯Ό 거쳐 ν•œκ΅­μ–΄Β·μΆ”λ‘ Β·λ¬Έν™” μ§€λŠ₯이 μ§„ν™”ν•©λ‹ˆλ‹€.
38
+
39
+ ---
40
+
41
+ ## πŸ›οΈ κ°€λ¬Έ 계보
42
+
43
+ ```
44
+ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
45
+ β”‚ 증쑰뢀 (Great-Grandfather) β”‚
46
+ β”‚ Qwen-3.6-27B β”‚
47
+ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
48
+ β”‚
49
+ β–Ό Darwin V7 μ§„ν™” λ¨Έμ§€
50
+ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
51
+ β”‚ μ‘°λΆ€ (Grandfather) β”‚
52
+ β”‚ Darwin-3.6-28B β”‚
53
+ β”‚ - Qwen 3.6 μ§„ν™” λ¨Έμ§€μ˜ 정점 β”‚
54
+ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
55
+ β”‚
56
+ β–Ό ν•œκ΅­μ–΄Β·μΆ”λ‘  νŠΉν™” μ§„ν™”
57
+ ╔══════════════════════════════════════════╗
58
+ β•‘ μ•„λΉ  (Father) β•‘
59
+ β•‘ FINAL-Bench/Darwin-27B-Opus β•‘
60
+ β•‘ β•‘
61
+ β•‘ - Darwin family reasoning 정점 β•‘
62
+ β•‘ - GPQA 88.4% (μ˜μ–΄ μΆ”λ‘ ) β•‘
63
+ β•‘ - <think> 트레이슀 νŒ¨ν„΄ 보유 β•‘
64
+ β•‘ - Apache 2.0 β•‘
65
+ β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•
66
+ β”‚
67
+ Γ—Γ— λ‹€μœˆ ꡐ배 Γ—Γ—
68
+ β”‚
69
+ β–Ό Darwin 진화적 λ¨Έμ§€ + ν•œκ΅­μ–΄ μ •μ œ
70
+ ╔══════════════════════════════════════════╗
71
+ β•‘ μžμ‹ (Child) β€” λ³Έ λͺ¨λΈ β•‘
72
+ β•‘ Warecube/Warecube-KO-27B β•‘
73
+ β•‘ β•‘
74
+ β•‘ ✦ μ•„λΉ μ˜ reasoning DNA 직접 κ³„μŠΉ β•‘
75
+ β•‘ ✦ ν•œκ΅­μ–΄ ν‘œν˜„Β·μ§€μ‹ μ§„ν™” κ°•ν™” β•‘
76
+ β•‘ ✦ <think> μΆ”λ‘  트레이슀 보쑴 β•‘
77
+ β•‘ ✦ K-AI 도메인 적합도 μ§„ν™” β•‘
78
+ β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•
79
+ ```
80
+
81
+ ---
82
+
83
+ ## πŸŽ“ μ§„ν™” 단계
84
+
85
+ | Stage | 개랡 |
86
+ |:---|:---|
87
+ | **1. ꡐ배 (Crossover)** | μΉœκ°€ κ°€μ€‘μΉ˜λ₯Ό λͺ¨λ“ˆλ³„ λΉ„μœ¨λ‘œ μ§„ν™” λ¨Έμ§€ |
88
+ | **2. 선택 (Selection)** | ν•œκ΅­μ–΄ 도메인 적합도 ν‰κ°€λ‘œ 우수 후손 선별 |
89
+ | **3. μ •μ œ (Refinement)** | ν•œκ΅­μ–΄ instruction λ°μ΄ν„°λ‘œ μΆ”κ°€ μ§„ν™” |
90
+ | **4. 적응 (Adaptation)** | K-AI Leaderboard Docker ν˜Έν™˜ ν˜•μ‹μœΌλ‘œ μ •λΉ„ |
91
+
92
+ ---
93
+
94
+ ## 🎯 μ‚¬μš©λ²•
95
+
96
+ ```python
97
+ from transformers import AutoTokenizer, AutoModelForCausalLM
98
+ import torch
99
+
100
+ model_id = "Warecube/Warecube-KO-27B"
101
+ tokenizer = AutoTokenizer.from_pretrained(
102
+ model_id, trust_remote_code=True
103
+ )
104
+ model = AutoModelForCausalLM.from_pretrained(
105
+ model_id,
106
+ torch_dtype=torch.bfloat16,
107
+ device_map="auto",
108
+ trust_remote_code=True,
109
+ )
110
+
111
+ prompt = "ν•œκ΅­μ˜ 좔석에 λŒ€ν•΄ μ„€λͺ…ν•΄μ£Όμ„Έμš”."
112
+ messages = [{"role": "user", "content": prompt}]
113
+ inputs = tokenizer.apply_chat_template(
114
+ messages, return_tensors="pt", add_generation_prompt=True
115
+ )
116
+ out = model.generate(
117
+ inputs.to(model.device),
118
+ max_new_tokens=512,
119
+ do_sample=False,
120
+ )
121
+ print(tokenizer.decode(out[0], skip_special_tokens=False))
122
+ ```
123
+
124
+ ---
125
+
126
+ ## πŸ› οΈ 사양
127
+
128
+ - νŒŒλΌλ―Έν„°: 27B (text)
129
+ - μ–‘μžν™”: bf16
130
+ - μ»¨ν…μŠ€νŠΈ: 8K (ν™•μž₯ κ°€λŠ₯)
131
+ - μ–Έμ–΄: ν•œκ΅­μ–΄ + μ˜μ–΄
132
+ - μΆ”λ‘ : `<think>` reasoning trace
133
+ - License: Apache 2.0
134
+
135
+ ---
136
+
137
+ ## πŸ“Š 평가
138
+
139
+ ν•œκ΅­μ–΄ 곡개 10 데이터셋, 100문제 Γ— 1 seed.
140
+
141
+ | Dataset | Score |
142
+ |:---|---:|
143
+ | CLIcK | **87%** |
144
+ | KMMLU History | **50%** |
145
+ | KMMLU Law | **29%** |
146
+ | KMMLU Health | 78% |
147
+ | HAERAE General | 58% |
148
+ | HAERAE History | 86% |
149
+ | HAERAE Linguistics | 89% |
150
+ | KoBEST Hellaswag | 89% |
151
+ | KoBEST COPA | **100%** |
152
+ | KoBEST BoolQ | 97% |
153
+ | **Macro Avg** | **76.3%** |
154
+
155
+ ---
156
+
157
+ ## 🀝 좜처
158
+
159
+ - μ•„λΉ : [FINAL-Bench/Darwin-27B-Opus](https://huggingface.co/FINAL-Bench/Darwin-27B-Opus)
160
+ - κ°€λ¬Έ: Darwin family (Darwin V7 진화적 λ¨Έμ§€ μ‹œλ¦¬μ¦ˆ)