ssc-mmc-mms-model-mix-adapt-max-lowlr

This model is a fine-tuned version of facebook/mms-1b-all on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.2748
• Cer: 0.2792
• Wer: 0.6328

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 8
• eval_batch_size: 6
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 16
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 10
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Cer	Wer
2.4444	0.3630	200	1.6953	0.3718	0.8006
1.6754	0.7260	400	1.4696	0.3340	0.7420
1.5773	1.0889	600	1.3822	0.3183	0.6998
1.4642	1.4519	800	1.3502	0.3136	0.7004
1.4418	1.8149	1000	1.3125	0.3030	0.6762
1.3594	2.1779	1200	1.3078	0.3016	0.6710
1.4188	2.5408	1400	1.3147	0.2995	0.6719
1.4157	2.9038	1600	1.2933	0.2984	0.6632
1.3717	3.2668	1800	1.2817	0.2905	0.6629
1.3127	3.6298	2000	1.2784	0.2905	0.6532
1.2914	3.9927	2200	1.2843	0.2907	0.6532
1.3149	4.3557	2400	1.2781	0.2884	0.6563
1.2927	4.7187	2600	1.2779	0.2867	0.6486
1.289	5.0817	2800	1.2766	0.2853	0.6441
1.2659	5.4446	3000	1.2808	0.2841	0.6454
1.2769	5.8076	3200	1.2634	0.2817	0.6385
1.2407	6.1706	3400	1.2764	0.2811	0.6447
1.2399	6.5336	3600	1.2775	0.2809	0.6392
1.2309	6.8966	3800	1.2719	0.2812	0.6385
1.2341	7.2595	4000	1.2717	0.2805	0.6375
1.2258	7.6225	4200	1.2705	0.2783	0.6346
1.2167	7.9855	4400	1.2715	0.2797	0.6338
1.2458	8.3485	4600	1.2731	0.2782	0.6344
1.1659	8.7114	4800	1.2721	0.2776	0.6317
1.1922	9.0744	5000	1.2733	0.2791	0.6336
1.1726	9.4374	5200	1.2761	0.2792	0.6359
1.2057	9.8004	5400	1.2748	0.2792	0.6328

Framework versions

• Transformers 4.57.2
• Pytorch 2.9.1+cu128
• Datasets 3.6.0
• Tokenizers 0.22.0