Direct Preference Optimization: Your Language Model is Secretly a Reward Model
Paper • 2305.18290 • Published • 64
ENERGY-DRINK-LOVE/DataVortexS_dpov3
Our Team
- Youjin Chung
- Jingyeom Kim
Model
Base Model
Hardware and Software
- Hardware: A100 * 8 for training our model
- Deepspeed library & Huggingface TRL Trainer
Dataset
- DPO_dataset
- 자체 제작 dpo dataset(AI-hub dataset 활용)
- OpenOrca DPO 등 영어 데이터셋 번역(ENERGY-DRINK-LOVE/translate_share_gpt_dedup_llama_SFT_1024, 자체모델 활용)
Training Method
Benchmark
- (240316기준 7등)
| Average | Ko-ARC | Ko-HellaSwag | Ko-MMLU | Ko-TruthfulQA | Ko-CommonGen V2 |
|---|---|---|---|---|---|
| 60.18 | 56.23 | 69.15 | 52.76 | 67.87 | 54.9 |
- Downloads last month
- 5