ARC-Challenge_Llama-3.2-1B-yex1s81j

This model is a fine-tuned version of meta-llama/Llama-3.2-1B on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.2918
• Model Preparation Time: 0.006
• Mdl: 988.5890
• Accumulated Loss: 685.2377
• Correct Preds: 158.0
• Total Preds: 299.0
• Accuracy: 0.5284
• Correct Gen Preds: 34.0
• Gen Accuracy: 0.1137
• Correct Gen Preds 32: 6.0
• Correct Preds 32: 30.0
• Total Labels 32: 64.0
• Accuracy 32: 0.4688
• Gen Accuracy 32: 0.0938
• Correct Gen Preds 33: 9.0
• Correct Preds 33: 39.0
• Total Labels 33: 73.0
• Accuracy 33: 0.5342
• Gen Accuracy 33: 0.1233
• Correct Gen Preds 34: 14.0
• Correct Preds 34: 47.0
• Total Labels 34: 78.0
• Accuracy 34: 0.6026
• Gen Accuracy 34: 0.1795
• Correct Gen Preds 35: 5.0
• Correct Preds 35: 42.0
• Total Labels 35: 83.0
• Accuracy 35: 0.5060
• Gen Accuracy 35: 0.0602
• Correct Gen Preds 36: 0.0
• Correct Preds 36: 0.0
• Total Labels 36: 1.0
• Accuracy 36: 0.0
• Gen Accuracy 36: 0.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 112
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.01
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Model Preparation Time	Mdl	Accumulated Loss	Correct Preds	Total Preds	Accuracy	Correct Gen Preds	Gen Accuracy	Correct Gen Preds 32	Correct Preds 32	Total Labels 32	Accuracy 32	Gen Accuracy 32	Correct Gen Preds 33	Correct Preds 33	Total Labels 33	Accuracy 33	Gen Accuracy 33	Correct Gen Preds 34	Correct Preds 34	Total Labels 34	Accuracy 34	Gen Accuracy 34	Correct Gen Preds 35	Correct Preds 35	Total Labels 35	Accuracy 35	Gen Accuracy 35	Correct Gen Preds 36	Correct Preds 36	Total Labels 36	Accuracy 36	Gen Accuracy 36
No log	0	0	1.6389	0.006	706.9523	490.0220	66.0	299.0	0.2207	66.0	0.2207	62.0	62.0	64.0	0.9688	0.9688	0.0	0.0	73.0	0.0	0.0	4.0	4.0	78.0	0.0513	0.0513	0.0	0.0	83.0	0.0	0.0	0.0	0.0	1.0	0.0	0.0
1.4185	1.0	18	1.2899	0.006	556.4093	385.6735	134.0	299.0	0.4482	134.0	0.4482	35.0	35.0	64.0	0.5469	0.5469	44.0	44.0	73.0	0.6027	0.6027	19.0	19.0	78.0	0.2436	0.2436	36.0	36.0	83.0	0.4337	0.4337	0.0	0.0	1.0	0.0	0.0
0.9931	2.0	36	1.2057	0.006	520.0862	360.4963	153.0	299.0	0.5117	153.0	0.5117	26.0	26.0	64.0	0.4062	0.4062	53.0	53.0	73.0	0.7260	0.7260	41.0	41.0	78.0	0.5256	0.5256	33.0	33.0	83.0	0.3976	0.3976	0.0	0.0	1.0	0.0	0.0
0.2541	3.0	54	1.5262	0.006	658.3298	456.3195	141.0	299.0	0.4716	141.0	0.4716	24.0	24.0	64.0	0.375	0.375	37.0	37.0	73.0	0.5068	0.5068	46.0	46.0	78.0	0.5897	0.5897	34.0	34.0	83.0	0.4096	0.4096	0.0	0.0	1.0	0.0	0.0
0.1817	4.0	72	3.1028	0.006	1338.4550	927.7463	152.0	299.0	0.5084	141.0	0.4716	22.0	25.0	64.0	0.3906	0.3438	44.0	48.0	73.0	0.6575	0.6027	42.0	43.0	78.0	0.5513	0.5385	33.0	36.0	83.0	0.4337	0.3976	0.0	0.0	1.0	0.0	0.0
0.2215	5.0	90	2.2918	0.006	988.5890	685.2377	158.0	299.0	0.5284	34.0	0.1137	6.0	30.0	64.0	0.4688	0.0938	9.0	39.0	73.0	0.5342	0.1233	14.0	47.0	78.0	0.6026	0.1795	5.0	42.0	83.0	0.5060	0.0602	0.0	0.0	1.0	0.0	0.0
0.1399	6.0	108	4.0080	0.006	1728.9256	1198.3999	151.0	299.0	0.5050	151.0	0.5050	24.0	24.0	64.0	0.375	0.375	38.0	38.0	73.0	0.5205	0.5205	45.0	45.0	78.0	0.5769	0.5769	43.0	43.0	83.0	0.5181	0.5181	1.0	1.0	1.0	1.0	1.0
0.0003	7.0	126	4.4427	0.006	1916.4119	1328.3555	153.0	299.0	0.5117	152.0	0.5084	33.0	33.0	64.0	0.5156	0.5156	37.0	37.0	73.0	0.5068	0.5068	42.0	42.0	78.0	0.5385	0.5385	39.0	40.0	83.0	0.4819	0.4699	1.0	1.0	1.0	1.0	1.0
0.0177	8.0	144	4.6523	0.006	2006.8334	1391.0309	150.0	299.0	0.5017	150.0	0.5017	25.0	25.0	64.0	0.3906	0.3906	36.0	36.0	73.0	0.4932	0.4932	48.0	48.0	78.0	0.6154	0.6154	40.0	40.0	83.0	0.4819	0.4819	1.0	1.0	1.0	1.0	1.0
0.0	9.0	162	5.3731	0.006	2317.7876	1606.5680	146.0	299.0	0.4883	146.0	0.4883	29.0	29.0	64.0	0.4531	0.4531	36.0	36.0	73.0	0.4932	0.4932	43.0	43.0	78.0	0.5513	0.5513	38.0	38.0	83.0	0.4578	0.4578	0.0	0.0	1.0	0.0	0.0
0.0	10.0	180	5.4677	0.006	2358.5715	1634.8372	146.0	299.0	0.4883	146.0	0.4883	27.0	27.0	64.0	0.4219	0.4219	36.0	36.0	73.0	0.4932	0.4932	44.0	44.0	78.0	0.5641	0.5641	38.0	38.0	83.0	0.4578	0.4578	1.0	1.0	1.0	1.0	1.0
0.0	11.0	198	5.4692	0.006	2359.2405	1635.3009	146.0	299.0	0.4883	146.0	0.4883	27.0	27.0	64.0	0.4219	0.4219	36.0	36.0	73.0	0.4932	0.4932	43.0	43.0	78.0	0.5513	0.5513	39.0	39.0	83.0	0.4699	0.4699	1.0	1.0	1.0	1.0	1.0
0.0	12.0	216	5.4800	0.006	2363.8988	1638.5298	147.0	299.0	0.4916	146.0	0.4883	27.0	27.0	64.0	0.4219	0.4219	37.0	37.0	73.0	0.5068	0.5068	43.0	43.0	78.0	0.5513	0.5513	38.0	39.0	83.0	0.4699	0.4578	1.0	1.0	1.0	1.0	1.0
0.0	13.0	234	5.5029	0.006	2373.7580	1645.3637	146.0	299.0	0.4883	145.0	0.4849	28.0	28.0	64.0	0.4375	0.4375	35.0	35.0	73.0	0.4795	0.4795	44.0	44.0	78.0	0.5641	0.5641	37.0	38.0	83.0	0.4578	0.4458	1.0	1.0	1.0	1.0	1.0
0.0	14.0	252	5.4979	0.006	2371.6052	1643.8715	148.0	299.0	0.4950	148.0	0.4950	29.0	29.0	64.0	0.4531	0.4531	35.0	35.0	73.0	0.4795	0.4795	44.0	44.0	78.0	0.5641	0.5641	39.0	39.0	83.0	0.4699	0.4699	1.0	1.0	1.0	1.0	1.0
0.0	15.0	270	5.5845	0.006	2408.9735	1669.7732	144.0	299.0	0.4816	144.0	0.4816	28.0	28.0	64.0	0.4375	0.4375	35.0	35.0	73.0	0.4795	0.4795	42.0	42.0	78.0	0.5385	0.5385	39.0	39.0	83.0	0.4699	0.4699	0.0	0.0	1.0	0.0	0.0

Framework versions

• Transformers 4.51.3
• Pytorch 2.6.0+cu124
• Datasets 3.5.0
• Tokenizers 0.21.1