Llama-3.3-70B-Instruct-3d-1M-100K-0.1-reverse-padzero-plus-mul-sub-99-128D-2L-8H-512I

This model is a fine-tuned version of meta-llama/Llama-3.3-70B-Instruct on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.0986

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.05
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss
No log	0	0	3.0906
1.7769	0.0640	500	1.7594
1.5163	0.1280	1000	1.4832
1.415	0.1920	1500	1.4071
1.3782	0.2560	2000	1.3831
1.3615	0.3200	2500	1.3645
1.3511	0.3840	3000	1.3488
1.2438	0.4480	3500	1.2387
1.2119	0.5120	4000	1.2105
1.2009	0.5760	4500	1.2000
1.187	0.6400	5000	1.1869
1.1754	0.7040	5500	1.1760
1.1679	0.7680	6000	1.1664
1.1622	0.8319	6500	1.1620
1.1633	0.8959	7000	1.1619
1.1593	0.9599	7500	1.1587
1.1572	1.0239	8000	1.1593
1.1554	1.0879	8500	1.1543
1.1526	1.1519	9000	1.1524
1.1506	1.2159	9500	1.1506
1.1506	1.2799	10000	1.1497
1.1501	1.3439	10500	1.1487
1.1462	1.4079	11000	1.1462
1.1473	1.4719	11500	1.1452
1.1464	1.5359	12000	1.1444
1.1453	1.5999	12500	1.1450
1.1435	1.6639	13000	1.1454
1.1421	1.7279	13500	1.1415
1.1418	1.7919	14000	1.1413
1.1387	1.8559	14500	1.1398
1.141	1.9199	15000	1.1417
1.1377	1.9839	15500	1.1367
1.137	2.0479	16000	1.1362
1.135	2.1119	16500	1.1351
1.135	2.1759	17000	1.1347
1.1323	2.2399	17500	1.1332
1.1325	2.3039	18000	1.1335
1.1317	2.3678	18500	1.1302
1.1293	2.4318	19000	1.1292
1.1279	2.4958	19500	1.1271
1.1279	2.5598	20000	1.1272
1.1264	2.6238	20500	1.1262
1.1224	2.6878	21000	1.1229
1.1235	2.7518	21500	1.1221
1.1199	2.8158	22000	1.1192
1.1182	2.8798	22500	1.1183
1.1148	2.9438	23000	1.1153
1.114	3.0078	23500	1.1129
1.1131	3.0718	24000	1.1128
1.1099	3.1358	24500	1.1105
1.1089	3.1998	25000	1.1090
1.1079	3.2638	25500	1.1077
1.1063	3.3278	26000	1.1063
1.1053	3.3918	26500	1.1053
1.1035	3.4558	27000	1.1044
1.1034	3.5198	27500	1.1037
1.1024	3.5838	28000	1.1023
1.1017	3.6478	28500	1.1015
1.1019	3.7118	29000	1.1010
1.1001	3.7758	29500	1.1006
1.0999	3.8398	30000	1.1001
1.0998	3.9038	30500	1.0997
1.0993	3.9677	31000	1.0996
1.1005	4.0317	31500	1.0992
1.0997	4.0957	32000	1.0991
1.0994	4.1597	32500	1.0989
1.0996	4.2237	33000	1.0988
1.0981	4.2877	33500	1.0988
1.0982	4.3517	34000	1.0987
1.0979	4.4157	34500	1.0987
1.0979	4.4797	35000	1.0986
1.0995	4.5437	35500	1.0986
1.0974	4.6077	36000	1.0986
1.0977	4.6717	36500	1.0986
1.0982	4.7357	37000	1.0986
1.0987	4.7997	37500	1.0986
1.0982	4.8637	38000	1.0986
1.0982	4.9277	38500	1.0986
1.098	4.9917	39000	1.0986

Framework versions

• Transformers 4.57.1
• Pytorch 2.9.0+cu128
• Datasets 4.5.0
• Tokenizers 0.22.1