amkyawdev/myanmar-llm-train

🧠 Myanmar LLM Training

Fine-tune Qwen2.5-0.5B-Instruct with Myanmar language dataset.

⚡ No License Required!

This model is fully open. No Llama license needed!

📋 Requirements

• Python 3.8+
• GPU with 6GB+ VRAM
• HuggingFace Account

🚀 Quick Start

1. Install dependencies

pip install -r requirements.txt

2. Login to HuggingFace

huggingface-cli login

3. Run training

python train.py

⚙️ Configuration

Parameter	Default	Description
MODEL_NAME	Qwen/Qwen2.5-0.5B-Instruct	Base model (fully open!)
num_train_epochs	3	Training iterations
per_device_train_batch_size	4	Batch size
gradient_accumulation_steps	4	Effective batch = 16
learning_rate	2e-5	Learning rate

📊 Features

• ✅ Fully open model - လိုင်စင်မလိုပါသည်။
• ✅ FP16 precision - ပိုမိုမြန်ပါသည်။
• ✅ Gradient checkpointing - Memory ချွေတာပါသည်။
• ✅ Test/Validation evaluation - နှစ်ခုလုံးအတွက် စမ်းသပ်ပါသည်။

📊 Training Data

Dataset: amkyawdev/AmkyawDev-Dataset

Split	Samples
Train	~29,100
Validation	~29,100
Test	~29,100

Note: Each file (train.jsonl, test.jsonl, validation.jsonl) has ~29,100 conversations!

💾 Output

Trained model saved to ./myanmar-qwen-output/

📤 Upload to HuggingFace

cd myanmar-qwen-output
huggingface-cli upload amkyawdev/my-myanmar-qwen . --repo-type model

🖥️ Google Colab

# Install
!pip install transformers datasets torch accelerate

# Login
from huggingface_hub import login
login("YOUR_TOKEN")

# Run
%run train.py

---

Built by amkyawdev