Model Card: byt5-akkadian-translator
This model is a fine-tuned ByT5-Base architecture specialized in translating Old Assyrian (OA) transliterations into English. It was developed specifically for the Deep Past competition.
Model Details
Model Description
The model utilizes a Parameter-Efficient Fine-Tuning (PEFT) approach with LoRA to adapt a byte-level Transformer (ByT5) to the specific phonology and semantics of Akkadian cuneiform transliterations.
- Developed by: Vinil Patel
- Model type: Seq2Seq (Byte-level Transformer)
- Language(s) (NLP): Akkadian (transliterated), English
- Finetuned from model: google/byt5-base
Model Sources
Uses
Direct Use
This model is intended for the automated translation of standard Old Assyrian administrative and legal formulas. It excels at translating standardized letter introductions, weights, measures, and seal descriptions. (translate Akkadian to English: {akkadin}) is required.
Out-of-Scope Use
The model is not suitable for literary or poetic Akkadian (e.g., Epic of Gilgamesh) as it was trained specifically on Old Assyrian dialect datasets. It should not be used for critical historical research without human-in-the-loop verification.
Bias, Risks, and Limitations
The model exhibits a complexity bottleneck. While highly accurate for formulaic text (0–12 tokens), its syntactic coherence degrades as sentence length increases.
Recommendations
- Formulaic Texts: Highly reliable.
- Complex Clauses: May hallucinate high-frequency names (e.g., Pūšu-kēn) when encountering rare vocabulary.
- Preprocessing: Best results are achieved when splitting multi-clause sentences into smaller segments.
How to Get Started with the Model
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
model = AutoModelForSeq2SeqLM.from_pretrained("Yoshaaa7/byt5-akkadian-translator")
tokenizer = AutoTokenizer.from_pretrained("Yoshaaa7/byt5-akkadian-translator")
text = "translate Akkadian to English: a-na ma-ar-ti-ia qi2-bi-ma"
inputs = tokenizer(text, return_tensors="pt").input_ids
outputs = model.generate(inputs, max_length=50, num_beams=5, repetition_penalty=2.5)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
- Downloads last month
- 11