Hodfa71/gpt-sw3-356m-icelandic-delta-dpo

GPT-SW3 356M — Icelandic Grammar-Aligned (SAGA)

This is a fine-tuned version of GPT-SW3 356M trained to generate grammatically correct Icelandic text.

The model was trained using SAGA (Syntax-Aware Generation Alignment), a two-stage pipeline that first performs supervised fine-tuning on Icelandic Wikipedia, then applies Delta-DPO to align generation quality toward sentences that pass Greynir grammatical parsing.

This checkpoint is a fully merged model (SFT weights + Delta-DPO LoRA merged in). You can load and run it directly without any PEFT setup.

Results

Evaluated on 200 Icelandic Wikipedia sentences:

Metric	Base	This model
Greynir parse success	85.5%	97.5%
Parse score	0.662	0.734
PPL-Wiki	24.4	31.9

Parse score = parse success rate times mean parse quality.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("Hodfa71/gpt-sw3-356m-icelandic-delta-dpo")
tokenizer = AutoTokenizer.from_pretrained("Hodfa71/gpt-sw3-356m-icelandic-delta-dpo")

prompt = "Íslenska er"
inputs = tokenizer(prompt, return_tensors="pt")
output = model.generate(**inputs, max_new_tokens=50, temperature=0.8, do_sample=True)
print(tokenizer.decode(output[0], skip_special_tokens=True))

Training details

The training data is Icelandic Wikipedia (10,000 sentences, filtered for quality).

Stage 1 is supervised fine-tuning (5 epochs, LoRA r=16). Stage 2 is Delta-DPO: the model generates 8 candidate continuations per prompt, pairs are filtered by a quality gap threshold of 0.25, and preference optimization runs for 2 epochs with beta=0.1.

The grammar reward uses Greynir, an Icelandic NLP library that checks whether a sentence has a valid constituency parse with a verbal root and a nominal subject.

Citation

Paper is under review. Will update with citation when available.

License

The base model (GPT-SW3) is released by AI Sweden under their LLM license. This fine-tuned version inherits the same license. Attribution: AI Sweden, RISE, and WASP.