Venkateswarlu15/mT5-finetuned-english-Azerbaijani

Instruction-Following Multilingual LLM (English + Azerbaijani)

Overview

This model is a fine-tuned version of mT5 (multilingual T5) designed for instruction-following tasks in English and Azerbaijani. It effectively understands and generates responses in the same language as the input, making it highly suitable for multilingual NLP applications.

---

Model Details

• Base Model: mT5
• Supported Languages: English, Azerbaijani
• Primary Task: Instruction following
• Fine-tuned Dataset: Custom dataset with diverse instructions and outputs in both languages
• Dataset Size: Approximately 118,000 examples
• Applications: Multilingual assistants, text generation, and language-specific QA systems

---

Training Information

Dataset

The fine-tuning dataset is in JSON format, with fields structured as follows:

• instruction: The task to be performed.
• input: Additional context or input (optional).
• output: The expected response.

Example:

{
  "instruction": "Translate the following to Azerbaijani: How are you?",
  "input": "",
  "output": "Necəsən?"
}

Training Parameters

• Epochs: 3
• Batch Size: 16
• Learning Rate: 5e-5
• Optimizer: AdamW
• Max Input Length: 512 tokens
• Max Output Length: 512 tokens

---

Evaluation

Evaluation Strategy

The model was assessed using both quantitative and qualitative methods:

Quantitative Metrics

• Average Loss: 2.15
• BLEU Score: ~30 (for translation tasks)
• Exact Match (EM): ~75%

Qualitative Analysis

• Reviewed sample predictions for accuracy in both languages.
• Verified translations using reliable references like Google Translate.

---

Example Predictions

English Example:

Instruction: "Summarize the following article: ..."
Prediction: "The article discusses ..."
Reference: "The article is about ..."

Azerbaijani Example:

Instruction: "Translate the following to Azerbaijani: How are you?"
Prediction: "Necəsən?"
Reference: "Necəsən?"

---

Model Performance

• Training Loss Curve: [Insert visualization if available]
• Evaluation Loss Curve: [Insert visualization if available]

---

Usage

Loading the Model

from transformers import T5ForConditionalGeneration, T5Tokenizer

# Load the model and tokenizer
model = T5ForConditionalGeneration.from_pretrained("your-username/instruction-following-mT5-finetuned-english-azerbaijani")
tokenizer = T5Tokenizer.from_pretrained("your-username/instruction-following-mT5-finetuned-english-azerbaijani")

# Prepare input
instruction = "Translate the following to Azerbaijani: How are you?"
input_text = tokenizer(instruction, return_tensors="pt", truncation=True, padding="max_length", max_length=512)

# Generate output
outputs = model.generate(input_text["input_ids"])
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

---

Limitations

• Performance may degrade for very complex or out-of-scope instructions.
• Dataset biases could affect accuracy in certain domains.
• Currently limited to English and Azerbaijani languages.

---

Citation

If you use this model, please cite:

@article{yourname2024instructionfollowing,
  title={Fine-tuned Instruction Following Model for English and Azerbaijani},
  author={Your Name},
  year={2024}
}

---

Contact

For questions or support, please reach out to: venkateshnaidu588@gmail.com

---