Warecube-KO-27B / README.md
Warecube's picture
upload README.md
aaf9908 verified
|
raw
history blame
5.36 kB
metadata
license: apache-2.0
language:
  - ko
  - en
library_name: transformers
tags:
  - korean
  - reasoning
  - darwin
  - evolutionary-merge
base_model:
  - FINAL-Bench/Darwin-27B-Opus

Warecube-KO-27B

ν•œκ΅­μ–΄ reasoning λͺ¨λΈ β€” Darwin 진화적 λ¨Έμ§€ 기반.


🧬 Darwin μ§„ν™” 컨셉

λ³Έ λͺ¨λΈμ€ Darwin V7 진화적 λͺ¨λΈ λ¨Έμ§€(Evolutionary Model Merge) νŒ¨λŸ¬λ‹€μž„μœΌλ‘œ μ œμž‘λ˜μ—ˆμŠ΅λ‹ˆλ‹€.

   μžμ—° μ§„ν™”                    Darwin λ¨Έμ§€
   ─────────                    ───────────
   μœ μ „μž ꡐ차 (crossover)  β†’   κ°€μ€‘μΉ˜ λͺ¨λ“ˆλ³„ λΉ„μœ¨ κ²°ν•©
   μžμ—° 선택 (selection)    β†’   적합도 평가 ν›„ 졜적 후손 선별
   μ„ΈλŒ€ μ§„ν™” (generations)  β†’   λ‹€μ„ΈλŒ€ λ¨Έμ§€Β·μ •μ œ 반볡
   적자 생쑴                β†’   K-AI 도메인 우수 μžμ†λ§Œ 보쑴

λΆ€λͺ¨μ˜ λŠ₯λ ₯이 μžμ‹ λͺ¨λΈλ‘œ μœ μ „μ μœΌλ‘œ κ³„μŠΉλ˜λ©°, μ„ΈλŒ€λ₯Ό 거쳐 ν•œκ΅­μ–΄Β·μΆ”λ‘ Β·λ¬Έν™” μ§€λŠ₯이 μ§„ν™”ν•©λ‹ˆλ‹€.


πŸ›οΈ κ°€λ¬Έ 계보

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚  증쑰뢀 (Great-Grandfather)               β”‚
β”‚  Qwen-3.6-27B                             β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                  β”‚
                  β–Ό Darwin V7 μ§„ν™” λ¨Έμ§€
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚  μ‘°λΆ€ (Grandfather)                       β”‚
β”‚  Darwin-3.6-28B                           β”‚
β”‚  - Qwen 3.6 μ§„ν™” λ¨Έμ§€μ˜ 정점               β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                  β”‚
                  β–Ό ν•œκ΅­μ–΄Β·μΆ”λ‘  νŠΉν™” μ§„ν™”
╔══════════════════════════════════════════╗
β•‘  μ•„λΉ  (Father)                            β•‘
β•‘  FINAL-Bench/Darwin-27B-Opus              β•‘
β•‘                                            β•‘
β•‘  - Darwin family reasoning 정점             β•‘
β•‘  - GPQA 88.4% (μ˜μ–΄ μΆ”λ‘ )                   β•‘
β•‘  - <think> 트레이슀 νŒ¨ν„΄ 보유                β•‘
β•‘  - Apache 2.0                              β•‘
β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•
                  β”‚
                  Γ—Γ— λ‹€μœˆ ꡐ배 Γ—Γ—
                  β”‚
                  β–Ό Darwin 진화적 λ¨Έμ§€ + ν•œκ΅­μ–΄ μ •μ œ
╔══════════════════════════════════════════╗
β•‘  μžμ‹ (Child) β€” λ³Έ λͺ¨λΈ                    β•‘
β•‘  Warecube/Warecube-KO-27B                 β•‘
β•‘                                            β•‘
β•‘  ✦ μ•„λΉ μ˜ reasoning DNA 직접 κ³„μŠΉ          β•‘
β•‘  ✦ ν•œκ΅­μ–΄ ν‘œν˜„Β·μ§€μ‹ μ§„ν™” κ°•ν™”               β•‘
β•‘  ✦ <think> μΆ”λ‘  트레이슀 보쑴              β•‘
β•‘  ✦ K-AI 도메인 적합도 μ§„ν™”                  β•‘
β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•

πŸŽ“ μ§„ν™” 단계

Stage 개랡
1. ꡐ배 (Crossover) μΉœκ°€ κ°€μ€‘μΉ˜λ₯Ό λͺ¨λ“ˆλ³„ λΉ„μœ¨λ‘œ μ§„ν™” λ¨Έμ§€
2. 선택 (Selection) ν•œκ΅­μ–΄ 도메인 적합도 ν‰κ°€λ‘œ 우수 후손 선별
3. μ •μ œ (Refinement) ν•œκ΅­μ–΄ instruction λ°μ΄ν„°λ‘œ μΆ”κ°€ μ§„ν™”
4. 적응 (Adaptation) K-AI Leaderboard Docker ν˜Έν™˜ ν˜•μ‹μœΌλ‘œ μ •λΉ„

🎯 μ‚¬μš©λ²•

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_id = "Warecube/Warecube-KO-27B"
tokenizer = AutoTokenizer.from_pretrained(
    model_id, trust_remote_code=True
)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)

prompt = "ν•œκ΅­μ˜ 좔석에 λŒ€ν•΄ μ„€λͺ…ν•΄μ£Όμ„Έμš”."
messages = [{"role": "user", "content": prompt}]
inputs = tokenizer.apply_chat_template(
    messages, return_tensors="pt", add_generation_prompt=True
)
out = model.generate(
    inputs.to(model.device),
    max_new_tokens=512,
    do_sample=False,
)
print(tokenizer.decode(out[0], skip_special_tokens=False))

πŸ› οΈ 사양

  • νŒŒλΌλ―Έν„°: 27B (text)
  • μ–‘μžν™”: bf16
  • μ»¨ν…μŠ€νŠΈ: 8K (ν™•μž₯ κ°€λŠ₯)
  • μ–Έμ–΄: ν•œκ΅­μ–΄ + μ˜μ–΄
  • μΆ”λ‘ : <think> reasoning trace
  • License: Apache 2.0

πŸ“Š 평가

ν•œκ΅­μ–΄ 곡개 10 데이터셋, 100문제 Γ— 1 seed.

Dataset Score
CLIcK 87%
KMMLU History 50%
KMMLU Law 29%
KMMLU Health 78%
HAERAE General 58%
HAERAE History 86%
HAERAE Linguistics 89%
KoBEST Hellaswag 89%
KoBEST COPA 100%
KoBEST BoolQ 97%
Macro Avg 76.3%

🀝 좜처