mohamedrayyan/dhivehi-news-lora-qwen3-30b

Dhivehi News LoRA - Qwen3-30B-A3B

A LoRA fine-tuned adapter that teaches Qwen3-30B-A3B to generate Dhivehi (Maldivian) news articles.

Model Details

Property	Value
Base Model	Qwen/Qwen3-30B-A3B
Training Method	LoRA (Low-Rank Adaptation)
LoRA Rank	32
Task Type	Causal Language Modeling

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

# Load base model
base_model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen3-30B-A3B",
    torch_dtype="auto",
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-30B-A3B")

# Load LoRA adapter
model = PeftModel.from_pretrained(base_model, "mohamedrayyan/dhivehi-news-lora-qwen3-30b")

# Generate
prompt = "Write a Dhivehi news article with the title: މާލޭގައި ބޮޑު އަލިފާނުގެ ހާދިސާއެއް ހިނގައިފި"
messages = [{"role": "user", "content": prompt}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training Details

• Dataset: alakxender/dhivehi-news-corpus (64.9k news articles)
• Training Platform: Tinker API
• Epochs: 3
• Learning Rate: 2e-4 (with linear decay)
• Max Sequence Length: 2048 tokens
• Batch Size: 128

Training Format

The model was trained on instruction-following format:

User: Write a Dhivehi news article with the title: {title}
Assistant: {article content}

Results

The fine-tuned model shows significant improvement over the base model for Dhivehi text generation:

Fine-Tuned Model

• Generates coherent Dhivehi news article content
• Follows the news writing style from the training corpus
• Produces grammatically correct Dhivehi text

Base Model (without fine-tuning)

• Struggles with the Dhivehi language task
• Often produces repetitive or garbled text
• Falls back to English "thinking" mode
• Misinterprets Dhivehi words

Example

Prompt:

Write a Dhivehi news article with the title: މާލޭގައި ބޮޑު އަލިފާނުގެ ހާދިސާއެއް ހިނގައިފި

Fine-Tuned Output:

މާލޭގައި ބޮޑު އަލިފާނުގެ ހާދިސާއެއް ހިނގައިފި އެވެ.

އިމާމް ނައިބު ރޭ މައުލޫމާތު ދެއްވަމުން މާލެ ކައުންސިލުން ވިދާޅުވީ މި ހާދިސާގައި މީހަކަށް އަނިޔާ ނުވާކަމަށާއި އެ މީހާ ހައްޔަރުކޮށްފައިވާ ކަމަށެވެ...

Requirements

• transformers>=4.40.0
• peft>=0.10.0
• torch>=2.0.0

Citation

If you use this model, please cite:

@misc{dhivehi-news-lora-2026,
  author = {mohamedrayyan},
  title = {Dhivehi News LoRA for Qwen3-30B-A3B},
  year = {2026},
  publisher = {HuggingFace},
  howpublished = {\url{https://huggingface.co/mohamedrayyan/dhivehi-news-lora-qwen3-30b}}
}

License

This model adapter follows the license of the base model (Qwen3-30B-A3B).

Acknowledgments

• Dataset: alakxender/dhivehi-news-corpus
• Training Infrastructure: Tinker API
• Base Model: Qwen Team