# Model: LLaMA (IFD Top 30% + High Instruction Entropy)

## 🔍 Purpose

Fine-tune `meta-llama/Llama-3.2-1B` on instructions with **high entropy** among the high-IFD group.  
These instructions are **more diverse and complex** in word usage.

## 📂 Dataset

- `alpaca2000_entropy_high.csv`
  - From `alpaca2000.csv` (IFD 상위 30%)
  - instruction entropy 상위 30% 추출 (약 180개)

## ⚙️ Training Config

- Model: `meta-llama/Llama-3.2-1B`
- Precision: `bf16` or `float32`
- Epochs: 3
- Max length: 2048
- Output: `output/llama_entropy_high`

## 🧪 Goal

Evaluate whether **complex and diverse** instructions improve the fine-tuning performance on instruction-following tasks.