nllb-lua-fra-mt-v1

This model is a bidirectional French (fra) ↔ Tshiluba (lua) translation model. It is a fine-tuned version of facebook/nllb-200-distilled-600M, specifically optimized for translation in the Tshiluba language context.

Model Description

• Developed by: Salomon Metre
• Model Type: NLLB (No Language Left Behind) Encoder-Decoder
• Language(s): French (fra_Latn), Tshiluba (lua_Latn)
• License: CC-BY-NC-4.0
• Fine-tuned from: facebook/nllb-200-distilled-600M

Training and Evaluation Data

The model was fine-tuned on a parallel corpus of scraped Bible-based sentences. This data provides high-quality alignment for the Tshiluba language, which is otherwise a high-sparsity, low-resource language in the digital domain.

• Dataset Link: https://huggingface.co/datasets/SalomonMetre13/lua_fra
• Domain: Religious/Scriptural text (Bible)

Intended Uses & Limitations

Intended Use

This model is intended for research purposes, specifically exploring translation capabilities for Congolese languages. It can be used for:

• Drafting translations between French and Tshiluba.
• Studying transfer learning from NLLB to low-resource Bantu languages.

Limitations

• Domain Shift: Since the training data is predominantly Bible-based, the model may perform poorly on contemporary slang, technical documentation, or legal texts.
• Hallucinations: Like many NLLB-based models, it may occasionally generate incorrect translations for names or rare entities not present in the fine-tuning set.

Training Procedure

Training Hyperparameters

The following hyperparameters were used during training:

• Learning Rate: 3e-05
• Train Batch Size: 4
• Eval Batch Size: 4
• Optimizer: AdamW (Fused)
• LR Scheduler: Linear with 200 warmup steps
• Mixed Precision: Native AMP (FP16)

Evaluation Results (at step 8000)

• Eval Loss: 0.2361
• Epoch: 0.64 (partial epoch)

Framework Versions

• Transformers: 4.51.3
• Pytorch: 2.6.0+cu124
• Datasets: 3.6.0
• Tokenizers: 0.21.1