embc25_finetuned_30000_en_fr_es_it

This model is a fine-tuned version of Kyungjin-Kim/mmc_roberta_500000_en_fr_es_it on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.4861
• Accuracy: 0.8065
• Precision: 0.8351
• Recall: 0.7638
• F1: 0.7979

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 64
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 10
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Precision	Recall	F1
0.5461	0.2963	500	0.5511	0.7183	0.8170	0.5625	0.6663
0.4715	0.5926	1000	0.4862	0.7642	0.8260	0.6695	0.7396
0.4256	0.8889	1500	0.4442	0.7887	0.7856	0.7943	0.7899
0.3859	1.1849	2000	0.4457	0.793	0.8233	0.7462	0.7828
0.3653	1.4812	2500	0.4300	0.7976	0.7927	0.806	0.7993
0.3705	1.7775	3000	0.4205	0.802	0.8161	0.7797	0.7975
0.2896	2.0735	3500	0.4656	0.8034	0.8199	0.7777	0.7982
0.2802	2.3698	4000	0.4979	0.7963	0.7618	0.8623	0.8089
0.2656	2.6661	4500	0.4620	0.7974	0.7736	0.841	0.8059
0.2676	2.9624	5000	0.4861	0.8065	0.8351	0.7638	0.7979
0.1897	3.2584	5500	0.5667	0.8052	0.8353	0.7602	0.7960
0.1973	3.5547	6000	0.5649	0.8047	0.8068	0.8013	0.8041
0.2014	3.8510	6500	0.5497	0.801	0.7994	0.8037	0.8015
0.1296	4.1470	7000	0.6694	0.8043	0.8189	0.7813	0.7997
0.1326	4.4433	7500	0.7186	0.799	0.8416	0.7367	0.7856
0.1419	4.7396	8000	0.7058	0.7992	0.7997	0.7983	0.7990
0.1029	5.0356	8500	0.8050	0.7958	0.7642	0.8557	0.8074
0.097	5.3319	9000	0.8109	0.7971	0.8117	0.7737	0.7922
0.0939	5.6281	9500	0.8684	0.8007	0.7890	0.821	0.8047
0.0936	5.9244	10000	0.8230	0.7983	0.8129	0.7748	0.7934
0.0687	6.2204	10500	1.0075	0.7985	0.7882	0.8163	0.8020
0.0741	6.5167	11000	0.9793	0.7983	0.7888	0.8148	0.8016
0.0769	6.8130	11500	0.9970	0.7971	0.7958	0.7993	0.7975
0.0467	7.1090	12000	1.0775	0.8007	0.8123	0.782	0.7969
0.0479	7.4053	12500	1.1501	0.7999	0.8128	0.7793	0.7957
0.0519	7.7016	13000	1.2080	0.8003	0.7910	0.8162	0.8034
0.0593	7.9979	13500	1.1970	0.7994	0.7896	0.8163	0.8028
0.043	8.2939	14000	1.2732	0.8007	0.8074	0.79	0.7986
0.0339	8.5902	14500	1.3546	0.7989	0.8195	0.7667	0.7922
0.0404	8.8865	15000	1.3777	0.801	0.8140	0.7803	0.7968
0.0351	9.1825	15500	1.3782	0.8023	0.8028	0.8013	0.8021
0.0348	9.4788	16000	1.3818	0.8016	0.8056	0.795	0.8003
0.0375	9.7751	16500	1.4040	0.8012	0.8054	0.7942	0.7998

Framework versions

• Transformers 4.48.1
• Pytorch 2.3.1
• Datasets 3.2.0
• Tokenizers 0.21.0