annasus10/xlmr-burmese-pragmatics-stage3

XLM-RoBERTa Burmese Pragmatics — Stage 3 (Full Pragmatic Input)

Fine-tuned version of xlm-roberta-base for Burmese politeness classification.
This is Stage 3 of a 3-stage ablation study on pragmatic classification in Burmese.

Task

Given a Burmese utterance with full pragmatic metadata, classify its politeness level into one of 6 classes:
neutral, polite, informal, professional, blunt, rude

Dataset

freococo/burmese-contextual-pragmatics
2,200 Burmese utterances covering 22 root meanings with pragmatic annotations.
Split: 70% train / 15% val / 15% test (seed=42)

Stage Description

Stage 3 — Full pragmatic input.
Input format: [register: X] [power: Y] [tone: Z] [utterance] </s> [context] </s> [instruction]
Adds explicit pragmatic metadata (register, power relation, tone) as text prefix.

Results (Test Set)

Metric	Score
Accuracy	0.8394
Macro F1	0.8252
Weighted F1	0.8421
Loss	0.5390

Per-class F1

Class	Precision	Recall	F1
blunt	0.83	0.71	0.77
informal	0.84	0.90	0.87
neutral	0.93	0.82	0.87
polite	0.66	0.86	0.74
professional	0.79	1.00	0.88
rude	0.90	0.75	0.82

Training

• Base model: xlm-roberta-base
• Epochs: 8
• Learning rate: 2e-5
• Batch size: 8
• Weighted cross-entropy loss (handles class imbalance)
• Best checkpoint selected by macro F1

Ablation Study

Stage	Input	Macro F1
Stage 1	Utterance only	see annasus10/xlmr-burmese-pragmatics-stage1
Stage 2	+ context + instruction	see annasus10/xlmr-burmese-pragmatics-stage2
Stage 3 (this model)	+ register + power + tone	0.825

Citation

Group: AttentionIsAllUNeed — NLP Final Project