nllb-akkadian-cuneiform

This model is a fine-tuned version of facebook/nllb-200-distilled-600M on the None dataset. It achieves the following results on the evaluation set:

• Loss: 3.6888

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 32
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 3
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
48.3526	0.2097	250	5.2291
37.1553	0.4193	500	4.3959
34.5367	0.6290	750	4.1047
32.9178	0.8387	1000	3.9736
31.9991	1.0478	1250	3.8967
31.8572	1.2575	1500	3.8430
31.2891	1.4671	1750	3.7997
31.1313	1.6768	2000	3.7668
30.9491	1.8865	2250	3.7429
30.7158	2.0956	2500	3.7238
30.4444	2.3053	2750	3.7081
30.5281	2.5149	3000	3.6992
30.5450	2.7246	3250	3.6922
30.3365	2.9343	3500	3.6888

Framework versions

• PEFT 0.18.1
• Transformers 5.0.0
• Pytorch 2.10.0+cu128
• Datasets 4.0.0
• Tokenizers 0.22.2