xlm-roberta-bambara-01

This model is a fine-tuned version of oza75/xlm-roberta-bambara-base-01 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.1160
• Accuracy: 0.7599

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 64
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 0.06
• num_epochs: 100.0

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
2.9609	1.8458	1000	1.4658	0.6942
2.8570	3.6907	2000	1.4204	0.7014
2.7516	5.5355	3000	1.3654	0.7116
2.6739	7.3804	4000	1.3460	0.7167
2.6104	9.2253	5000	1.3267	0.7195
2.5437	11.0702	6000	1.3044	0.7239
2.5264	12.9160	7000	1.2751	0.7296
2.4831	14.7608	8000	1.2648	0.7314
2.4471	16.6057	9000	1.2492	0.7345
2.4112	18.4506	10000	1.2466	0.7352
2.3945	20.2955	11000	1.2400	0.7364
2.3746	22.1404	12000	1.2290	0.7380
2.3774	23.9861	13000	1.2189	0.7400
2.3462	25.8310	14000	1.2187	0.7391
2.3205	27.6759	15000	1.2137	0.7405
2.3026	29.5208	16000	1.2014	0.7425
2.2770	31.3657	17000	1.1973	0.7446
2.2654	33.2105	18000	1.1902	0.7461
2.2580	35.0554	19000	1.1864	0.7461
2.2299	36.9012	20000	1.1864	0.7455
2.2272	38.7461	21000	1.1784	0.7480
2.2221	40.5910	22000	1.1819	0.7470
2.2076	42.4358	23000	1.1661	0.7496
2.1860	44.2807	24000	1.1729	0.7494
2.1880	46.1256	25000	1.1617	0.7510
2.1815	47.9714	26000	1.1598	0.7516
2.1620	49.8163	27000	1.1568	0.7512
2.1703	51.6611	28000	1.1516	0.7525
2.1563	53.5060	29000	1.1529	0.7518
2.1396	55.3509	30000	1.1537	0.7524
2.1182	57.1958	31000	1.1410	0.7544
2.1404	59.0406	32000	1.1506	0.7538
2.1401	60.8864	33000	1.1422	0.7540
2.1146	62.7313	34000	1.1428	0.7538
2.1252	64.5762	35000	1.1358	0.7544
2.1011	66.4211	36000	1.1356	0.7565
2.1142	68.2659	37000	1.1321	0.7557
2.0950	70.1108	38000	1.1362	0.7562
2.0930	71.9566	39000	1.1387	0.7556
2.0881	73.8015	40000	1.1272	0.7573
2.0858	75.6464	41000	1.1327	0.7566
2.0640	77.4912	42000	1.1252	0.7575
2.0456	79.3361	43000	1.1283	0.7579
2.0573	81.1810	44000	1.1283	0.7565
2.0496	83.0259	45000	1.1317	0.7568
2.0701	84.8717	46000	1.1320	0.7567
2.0779	86.7165	47000	1.1219	0.7579
2.0462	88.5614	48000	1.1201	0.7580
2.0726	90.4063	49000	1.1282	0.7580
2.0605	92.2512	50000	1.1221	0.7578
2.0428	94.0960	51000	1.1202	0.7580
2.0565	95.9418	52000	1.1143	0.7592
2.0513	97.7867	53000	1.1258	0.7579
2.0514	99.6316	54000	1.1242	0.7577

Framework versions

• Transformers 5.0.1.dev0
• Pytorch 2.4.1+cu124
• Datasets 4.5.0
• Tokenizers 0.22.2