Hindi Prose → Sanskrit Anuṣṭubh Poetry (Phi-4 LoRA)

Model Overview

This model is fine-tuned to generate Sanskrit Anuṣṭubh poetry from Hindi prose input. It is based on Phi-4 (unsloth) and trained using LoRA fine-tuning.

About Anuṣṭubh Meter

Anuṣṭubh is the most widely used Sanskrit poetic meter, forming the basis of the classical śloka. It consists of:

32 syllables (akṣaras) in total
Divided into 4 pādas, each with 8 syllables

Beyond syllable count, the meter enforces positional constraints:

The 5th syllable of each pāda is typically laghu (short)
The 6th syllable is guru (long)
The 7th syllable follows stricter rules depending on the pāda position

A valid Anuṣṭubh verse must satisfy both:

Exact 32-syllable structure
Metrical pattern constraints

This makes generation significantly more constrained than standard text generation tasks.

Model Details

Base Model: unsloth/Phi-4
Fine-tuning: LoRA
Precision: 4-bit (QLoRA)
Max Sequence Length: 1024
Best Checkpoint: checkpoint-3400

Training Configuration

Batch size: 8
Gradient accumulation: 2
Epochs: 10
Learning rate: 2e-4
Weight decay: 0.01
Warmup ratio: 0.03
LoRA r: 16
LoRA alpha: 32
LoRA dropout: 0.05
Early Stop Patience: 5
Early Stop Threshold: 0.001
Early stop triggered at ~2.5 epochs (25% of allowed 10 epochs training stopped)

Requirements

torch==2.10.0
transformers==5.3.0
peft==0.18.1
unsloth==2026.3.5
unsloth_zoo==2026.3.4
xformers==0.0.35
bitsandbytes==0.49.2
accelerate==1.13.0

Dataset

Train: 28,163
Validation: 3,130
Test: 1,648

Description

Input: Hindi prose
Output: Sanskrit Anuṣṭubh verse

Example:

PROSE : राक्षस कुल के आनंद, तुम्हारे कारण लंका की अवस्था और हम सब अब निराश्रित हो गए हैं। अपने कर्मों से तुमने अपने शरीर को गिद्धों द्वारा खाए जाने योग्य और अपनी आत्मा को नरक जाने योग्य बना लिया है।

POETRY: अस्माकं च निराशानां लङ्का च तव कारणात् गिद्धभक्ष्यशरीरस्य नरकाय च कर्मभिः ॥

Evaluation

Setup (Training → Input/Output)	Full (%)	Partial (%)	Invalid 32 (%)	Semantic (%)
Ground Truth (DEV Sanskrit + Hindi Prose)	99.51	99.51	0.00	74.04
Phi-4 (SLP1 → SLP1+0-shot/SLP1) `[*]` greedy	24.76	62.44	37.68	73.23
Phi-4 (DEV → DEV+0-shot/DEV) greedy	43.08	66.99	23.91	73.76
Phi-4 (DEV → DEV+3-shot/DEV) greedy	50.97	72.81	21.84	73.21
Phi-4 (DEV → DEV+6-shot/DEV) greedy	46.72	69.05	22.33	72.93
Phi-4 (DEV → DEV+0-shot/DEV) Sampling	42.90	64.68	21.78	72.71
Phi-4 (DEV → DEV+3-shot/DEV) Sampling	51.70	72.51	20.81	72.05

The ground truth row shows the metrics computed directly on the dataset (no model involved). Bold indicates the highest value.
Italics indicate the second highest value.

[*] Evaluation performed in Devanagari (SLP1 outputs converted to DEV)

Notes

Full (%):: Full Anuṣṭubh percentage: Percentage of outputs with 32 syllables and valid Anuṣṭubh pattern.
Partial (%): Partial Anuṣṭubh percentage: Percentage of outputs with 32 syllables no matter its Anuṣṭubh pattern or not.
Invalid 32 (%): 32-syllable outputs that violate metrical constraints (Partial − Full)
Semantic (%): Semantic similarity calculated using 'sanganaka/bge-m3-sanskritFT'
★ Evaluation performed in Devanagari (SLP1 outputs converted to DEV)
All verses are obtained via greedy decoding unless stated otherwise
The metrics are computed following Chandomitra.

Usage

import torch
import pandas as pd
from transformers import AutoTokenizer, AutoModelForCausalLM
from unsloth.chat_templates import get_chat_template

MAX_NEW_TOKENS = 110
device = "cuda" if torch.cuda.is_available() else "cpu"

model_path = "sanganaka/phi4-hindi2sanskrit-anustubh-lora-merged-step3400"

tokenizer = AutoTokenizer.from_pretrained(model_path)
tokenizer = get_chat_template(tokenizer, chat_template="phi-4")
if tokenizer.pad_token is None:
    tokenizer.pad_token = tokenizer.eos_token

model = AutoModelForCausalLM.from_pretrained(
    model_path,
    torch_dtype=torch.bfloat16,
)
model.to(device)
model.eval()

print("MODEL LOADED")

ANUSHTUP_INSTRUCTION = """The goal is to generate Sanskrit verse that follows the anushtup meter rules for the given input text.
RULES:
Verse Rules:
The verse contains 32 syllables/akshara and 4 padas in total.
The verse is divided into 2 lines, each containing 16 syllables.
Each line is divided into 2 padas (quartets), each containing exactly 8 syllables.
The fifth syllable of every pada must be LAGHU or short.
The sixth syllable of every pada must be GURU or long.
The seventh syllable of the second and fourth pada must be HRASVA.
The seventh syllable of the first and third pada must be DEERGHA.

Syllable Rules:
LAGHU vowels: अ, इ, उ, ऋ, ऌ
GURU vowels: आ, ई, ऊ, ॠ, ॡ, ए, ऐ, ओ, औ
HRASVA vowels: अ, इ, उ, ऋ, ऌ
DEERGHA vowels: आ, ई, ऊ, ॠ, ॡ, ए, ऐ, ओ, औ

Syllable classification rules:
- A syllable is marked Laghu/Guru and Hrasva/Deergha based on the vowel it contains.
- Any syllable containing anusvāra (ं) or visarga (ः) is always Guru.
- Any syllable followed by a conjunct consonant (saṁyuktākṣara) is always Guru.

Now convert the given Hindi text into a Sanskrit Anushtup verse in Devanagari:"""

def generate(hi_text):
    messages = [
        {"role": "system", "content": ANUSHTUP_INSTRUCTION},
        {"role": "user", "content": "यह समय वाणीकी पहुँचके परे था उसका वर्णन करना कठिन था उस समय कोई भूपाल वहाँ इस विषयमें कुछ भी न बोल सके मौन रह गये वे बारचार केवल श्रीकृष्णके मुखकी ओर देखते रहे ॥"},
        {"role": "assistant", "content": "ततः केचिन्महीपाला नानुवंस्तत्र किंचन अतीतवाक्पथे काले प्रेक्षमाणा जनार्दनम् ॥"},
        {"role": "user", "content": "फिर तो उसने एक दूसरे भयंकर शत्रुको वहाँ आया हुआ देखा, जो सरकण्डेके फूलके समान भूरे रंगका था वह धरतीमें विवर बनाकर उसके भीतर सोया करता था"},
        {"role": "assistant", "content": "अपश्यदपरं घोरमात्मनः शत्रुमागतम् शरप्रसूनसङ्काशं महीविवरशायिनम्॥"},
        {"role": "user", "content": "जो मनुष्य पाण्डुनन्दन अर्जुनके इस चरित्रको प्रतिदिन सुनता है, उसके मनमै पापपूर्ण विषयभोगोंकी इच्छा नहीं होती ॥"},
        {"role": "assistant", "content": "इदं यः शृणुयाद् वृत्तं नित्यं पाण्डुसुतस्य थे न तस्य कामः कामेषु पापकेषु प्रवर्तते ॥"},
        {"role": "user", "content": hi_text},
    ]

    inputs = tokenizer.apply_chat_template(
        messages,
        tokenize=True,
        add_generation_prompt=True,
        return_dict=True,
        return_tensors="pt",
    ).to(device)

    with torch.inference_mode():
        outputs = model.generate(
            **inputs,
            max_new_tokens=MAX_NEW_TOKENS,
            do_sample=False,
            pad_token_id=tokenizer.eos_token_id,
            eos_token_id=tokenizer.eos_token_id,
        )

    generated_tokens = outputs[0][inputs["input_ids"].shape[-1]:]
    return tokenizer.decode(generated_tokens, skip_special_tokens=True).strip()


text = "राक्षस कुल के आनंद, तुम्हारे कारण लंका की अवस्था और हम सब अब निराश्रित हो गए हैं। अपने कर्मों से तुमने अपने शरीर को गिद्धों द्वारा खाए जाने योग्य और अपनी आत्मा को नरक जाने योग्य बना लिया है।"
out = generate(text)

print(f"PROSE : {text}\nPOETRY: {out}")

NOTE In this setting 3 shot is enabled. If you want 0-shot setting then just replace messages = [] block with

    messages = [
        {"role": "system", "content": ANUSHTUP_INSTRUCTION},
        {"role": "user", "content": hi_text},
    ]

NOTE In this setting greedy decoding is enabled. To set non-deterministic decoding you can add these in settings.

do_sample=True,
temperature=0.6, 
top_p=0.9, 
top_k=50,

Limitations

It was observed that batched inference is degrading the outputs so it is advised to not use it.
Both greedy and non-greedy decoding scores are somewhat similar.
Meter compliance is not guaranteed
Sanskrit grammar may vary
Sensitive to input phrasing

Citation

@misc{jagadeeshan2026chandomitrageneratingstructuredsanskrit,
      title={Chandomitra: Towards Generating Structured Sanskrit Poetry from Natural Language Inputs}, 
      author={Manoj Balaji Jagadeeshan and Samarth Bhatia and Pretam Ray and Harshul Raj Surana and Akhil Rajeev P and Priya Mishra and Annarao Kulkarni and Ganesh Ramakrishnan and Prathosh AP and Pawan Goyal},
      year={2026},
      eprint={2506.00815},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2506.00815}, 
}