Llama-3.3-70B-Instruct-3d-500K-50K-0.2-reverse-padzero-plus-mul-sub-99-256D-2L-4H-1024I

This model is a fine-tuned version of meta-llama/Llama-3.3-70B-Instruct on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.1100

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.05
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss
No log	0	0	3.1022
1.6256	0.1280	500	1.5539
1.2658	0.2560	1000	1.2555
1.2301	0.3839	1500	1.2159
1.1816	0.5119	2000	1.1828
1.1691	0.6399	2500	1.1690
1.1643	0.7679	3000	1.1633
1.1569	0.8958	3500	1.1581
1.1566	1.0238	4000	1.1553
1.1538	1.1518	4500	1.1563
1.15	1.2798	5000	1.1502
1.1489	1.4077	5500	1.1490
1.1464	1.5357	6000	1.1465
1.1444	1.6637	6500	1.1411
1.14	1.7917	7000	1.1431
1.1361	1.9196	7500	1.1378
1.1352	2.0476	8000	1.1353
1.1293	2.1756	8500	1.1305
1.1268	2.3036	9000	1.1314
1.1261	2.4315	9500	1.1237
1.1214	2.5595	10000	1.1220
1.1218	2.6875	10500	1.1215
1.1171	2.8155	11000	1.1176
1.1178	2.9434	11500	1.1161
1.1137	3.0714	12000	1.1143
1.1126	3.1994	12500	1.1133
1.1121	3.3274	13000	1.1120
1.1106	3.4553	13500	1.1112
1.111	3.5833	14000	1.1108
1.1101	3.7113	14500	1.1105
1.1096	3.8393	15000	1.1103
1.11	3.9672	15500	1.1102
1.1089	4.0952	16000	1.1101
1.1098	4.2232	16500	1.1101
1.1094	4.3512	17000	1.1100
1.1086	4.4791	17500	1.1100
1.1094	4.6071	18000	1.1100
1.1089	4.7351	18500	1.1100
1.1093	4.8631	19000	1.1100
1.1084	4.9910	19500	1.1100

Framework versions

• Transformers 4.57.1
• Pytorch 2.9.0+cu128
• Datasets 4.5.0
• Tokenizers 0.22.1