ModernBERT-base_en-comp-job-desc-clf

This model is a fine-tuned version of answerdotai/ModernBERT-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.0504
• F1: 0.9895
• Roc: 0.9959

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 64
• eval_batch_size: 512
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	F1	Roc
0.3391	0.0558	100	0.1183	0.9720	0.9776
0.0969	0.1115	200	0.0878	0.9810	0.9895
0.0918	0.1673	300	0.0741	0.9827	0.9923
0.0724	0.2231	400	0.0685	0.9845	0.9926
0.0704	0.2789	500	0.0814	0.9826	0.9915
0.0647	0.3346	600	0.0572	0.9874	0.9943
0.0626	0.3904	700	0.0685	0.9843	0.9940
0.0624	0.4462	800	0.0610	0.9863	0.9942
0.0541	0.5020	900	0.0578	0.9873	0.9948
0.0634	0.5577	1000	0.0693	0.9846	0.9945
0.0558	0.6135	1100	0.0535	0.9885	0.9945
0.0522	0.6693	1200	0.0571	0.9866	0.9948
0.0496	0.7250	1300	0.0546	0.9881	0.9948
0.0505	0.7808	1400	0.0548	0.9883	0.9943
0.0482	0.8366	1500	0.0479	0.9886	0.9961
0.0542	0.8924	1600	0.0517	0.9900	0.9950
0.0526	0.9481	1700	0.0554	0.9889	0.9935
0.0527	1.0039	1800	0.0470	0.9901	0.9955
0.0307	1.0597	1900	0.0509	0.9899	0.9953
0.0321	1.1154	2000	0.0504	0.9895	0.9959

Framework versions

• Transformers 4.48.3
• Pytorch 2.6.0+cu124
• Datasets 3.3.0
• Tokenizers 0.21.0