vlsp-vit5-30epoch

This model is a fine-tuned version of VietAI/vit5-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.1955
• Exact Match: 0.926

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Exact Match
0.8774	1.0	188	0.6815	0.008
0.608	2.0	376	0.5527	0.056
0.4671	3.0	564	0.4087	0.156
0.367	4.0	752	0.3285	0.302
0.2967	5.0	940	0.3034	0.33
0.2483	6.0	1128	0.2598	0.498
0.2112	7.0	1316	0.2461	0.494
0.1762	8.0	1504	0.2234	0.644
0.1635	9.0	1692	0.2124	0.642
0.1349	10.0	1880	0.2066	0.706
0.12	11.0	2068	0.1929	0.744
0.108	12.0	2256	0.1941	0.724
0.0939	13.0	2444	0.1843	0.744
0.0793	14.0	2632	0.2232	0.644
0.0681	15.0	2820	0.1769	0.826
0.0611	16.0	3008	0.2049	0.756
0.0551	17.0	3196	0.1836	0.814
0.049	18.0	3384	0.2048	0.838
0.0456	19.0	3572	0.1837	0.82
0.0372	20.0	3760	0.1884	0.87
0.0361	21.0	3948	0.2012	0.85
0.0312	22.0	4136	0.1866	0.886
0.0269	23.0	4324	0.1839	0.888
0.0233	24.0	4512	0.1931	0.886
0.0213	25.0	4700	0.1883	0.9
0.02	26.0	4888	0.1934	0.892
0.0147	27.0	5076	0.1914	0.92
0.0122	28.0	5264	0.1945	0.926
0.0114	29.0	5452	0.1939	0.932
0.0108	30.0	5640	0.1955	0.926

Framework versions

• Transformers 4.52.4
• Pytorch 2.6.0+cu124
• Datasets 3.6.0
• Tokenizers 0.21.2