ARC-Challenge_Llama-3.2-1B-reewqb30

This model is a fine-tuned version of meta-llama/Llama-3.2-1B on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 5.1655
• Model Preparation Time: 0.0059
• Mdl: 2228.2316
• Accumulated Loss: 1544.4924
• Correct Preds: 148.0
• Total Preds: 299.0
• Accuracy: 0.4950
• Correct Gen Preds: 141.0
• Gen Accuracy: 0.4716
• Correct Gen Preds 32: 24.0
• Correct Preds 32: 25.0
• Total Labels 32: 64.0
• Accuracy 32: 0.3906
• Gen Accuracy 32: 0.375
• Correct Gen Preds 33: 32.0
• Correct Preds 33: 33.0
• Total Labels 33: 73.0
• Accuracy 33: 0.4521
• Gen Accuracy 33: 0.4384
• Correct Gen Preds 34: 44.0
• Correct Preds 34: 46.0
• Total Labels 34: 78.0
• Accuracy 34: 0.5897
• Gen Accuracy 34: 0.5641
• Correct Gen Preds 35: 40.0
• Correct Preds 35: 43.0
• Total Labels 35: 83.0
• Accuracy 35: 0.5181
• Gen Accuracy 35: 0.4819
• Correct Gen Preds 36: 1.0
• Correct Preds 36: 1.0
• Total Labels 36: 1.0
• Accuracy 36: 1.0
• Gen Accuracy 36: 1.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 112
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.01
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Model Preparation Time	Mdl	Accumulated Loss	Correct Preds	Total Preds	Accuracy	Correct Gen Preds	Gen Accuracy	Correct Gen Preds 32	Correct Preds 32	Total Labels 32	Accuracy 32	Gen Accuracy 32	Correct Gen Preds 33	Correct Preds 33	Total Labels 33	Accuracy 33	Gen Accuracy 33	Correct Gen Preds 34	Correct Preds 34	Total Labels 34	Accuracy 34	Gen Accuracy 34	Correct Gen Preds 35	Correct Preds 35	Total Labels 35	Accuracy 35	Gen Accuracy 35	Correct Gen Preds 36	Correct Preds 36	Total Labels 36	Accuracy 36	Gen Accuracy 36
No log	0	0	1.6389	0.0059	706.9523	490.0220	66.0	299.0	0.2207	66.0	0.2207	62.0	62.0	64.0	0.9688	0.9688	0.0	0.0	73.0	0.0	0.0	4.0	4.0	78.0	0.0513	0.0513	0.0	0.0	83.0	0.0	0.0	0.0	0.0	1.0	0.0	0.0
1.2648	1.0	9	1.3329	0.0059	574.9807	398.5463	109.0	299.0	0.3645	109.0	0.3645	30.0	30.0	64.0	0.4688	0.4688	20.0	20.0	73.0	0.2740	0.2740	35.0	35.0	78.0	0.4487	0.4487	24.0	24.0	83.0	0.2892	0.2892	0.0	0.0	1.0	0.0	0.0
1.1141	2.0	18	1.3847	0.0059	597.3254	414.0344	120.0	299.0	0.4013	120.0	0.4013	39.0	39.0	64.0	0.6094	0.6094	41.0	41.0	73.0	0.5616	0.5616	18.0	18.0	78.0	0.2308	0.2308	22.0	22.0	83.0	0.2651	0.2651	0.0	0.0	1.0	0.0	0.0
0.3427	3.0	27	2.3301	0.0059	1005.1232	696.6983	143.0	299.0	0.4783	139.0	0.4649	34.0	36.0	64.0	0.5625	0.5312	26.0	26.0	73.0	0.3562	0.3562	35.0	36.0	78.0	0.4615	0.4487	43.0	44.0	83.0	0.5301	0.5181	1.0	1.0	1.0	1.0	1.0
0.0939	4.0	36	2.4715	0.0059	1066.1117	738.9723	138.0	299.0	0.4615	136.0	0.4548	25.0	26.0	64.0	0.4062	0.3906	46.0	47.0	73.0	0.6438	0.6301	30.0	30.0	78.0	0.3846	0.3846	35.0	35.0	83.0	0.4217	0.4217	0.0	0.0	1.0	0.0	0.0
0.0074	5.0	45	2.7997	0.0059	1207.6916	837.1080	143.0	299.0	0.4783	143.0	0.4783	14.0	14.0	64.0	0.2188	0.2188	47.0	47.0	73.0	0.6438	0.6438	37.0	37.0	78.0	0.4744	0.4744	44.0	44.0	83.0	0.5301	0.5301	1.0	1.0	1.0	1.0	1.0
0.1394	6.0	54	4.9617	0.0059	2140.3199	1483.5567	138.0	299.0	0.4615	137.0	0.4582	23.0	24.0	64.0	0.375	0.3594	36.0	36.0	73.0	0.4932	0.4932	43.0	43.0	78.0	0.5513	0.5513	34.0	34.0	83.0	0.4096	0.4096	1.0	1.0	1.0	1.0	1.0
0.0001	7.0	63	5.2712	0.0059	2273.8297	1576.0986	133.0	299.0	0.4448	128.0	0.4281	23.0	24.0	64.0	0.375	0.3594	39.0	40.0	73.0	0.5479	0.5342	35.0	37.0	78.0	0.4744	0.4487	31.0	31.0	83.0	0.3735	0.3735	0.0	1.0	1.0	1.0	0.0
0.0004	8.0	72	4.7643	0.0059	2055.1726	1424.5371	140.0	299.0	0.4682	128.0	0.4281	27.0	28.0	64.0	0.4375	0.4219	30.0	31.0	73.0	0.4247	0.4110	40.0	43.0	78.0	0.5513	0.5128	31.0	37.0	83.0	0.4458	0.3735	0.0	1.0	1.0	1.0	0.0
0.0016	9.0	81	5.1655	0.0059	2228.2316	1544.4924	148.0	299.0	0.4950	141.0	0.4716	24.0	25.0	64.0	0.3906	0.375	32.0	33.0	73.0	0.4521	0.4384	44.0	46.0	78.0	0.5897	0.5641	40.0	43.0	83.0	0.5181	0.4819	1.0	1.0	1.0	1.0	1.0
0.0	10.0	90	6.1908	0.0059	2670.4851	1851.0392	139.0	299.0	0.4649	137.0	0.4582	20.0	20.0	64.0	0.3125	0.3125	38.0	38.0	73.0	0.5205	0.5205	44.0	44.0	78.0	0.5641	0.5641	34.0	36.0	83.0	0.4337	0.4096	1.0	1.0	1.0	1.0	1.0
0.0	11.0	99	6.5711	0.0059	2834.5634	1964.7697	132.0	299.0	0.4415	130.0	0.4348	17.0	17.0	64.0	0.2656	0.2656	41.0	41.0	73.0	0.5616	0.5616	42.0	42.0	78.0	0.5385	0.5385	30.0	32.0	83.0	0.3855	0.3614	0.0	0.0	1.0	0.0	0.0
0.0	12.0	108	6.5883	0.0059	2841.9866	1969.9150	131.0	299.0	0.4381	129.0	0.4314	17.0	17.0	64.0	0.2656	0.2656	41.0	41.0	73.0	0.5616	0.5616	41.0	41.0	78.0	0.5256	0.5256	30.0	32.0	83.0	0.3855	0.3614	0.0	0.0	1.0	0.0	0.0
0.0	13.0	117	6.6199	0.0059	2855.5882	1979.3429	131.0	299.0	0.4381	128.0	0.4281	17.0	17.0	64.0	0.2656	0.2656	41.0	41.0	73.0	0.5616	0.5616	41.0	41.0	78.0	0.5256	0.5256	29.0	32.0	83.0	0.3855	0.3494	0.0	0.0	1.0	0.0	0.0
0.0	14.0	126	6.6230	0.0059	2856.9272	1980.2711	131.0	299.0	0.4381	129.0	0.4314	17.0	17.0	64.0	0.2656	0.2656	41.0	41.0	73.0	0.5616	0.5616	41.0	41.0	78.0	0.5256	0.5256	30.0	32.0	83.0	0.3855	0.3614	0.0	0.0	1.0	0.0	0.0
0.0	15.0	135	6.6217	0.0059	2856.3750	1979.8883	131.0	299.0	0.4381	129.0	0.4314	17.0	17.0	64.0	0.2656	0.2656	41.0	41.0	73.0	0.5616	0.5616	41.0	41.0	78.0	0.5256	0.5256	30.0	32.0	83.0	0.3855	0.3614	0.0	0.0	1.0	0.0	0.0
0.0	16.0	144	6.6212	0.0059	2856.1519	1979.7336	130.0	299.0	0.4348	128.0	0.4281	17.0	17.0	64.0	0.2656	0.2656	41.0	41.0	73.0	0.5616	0.5616	40.0	40.0	78.0	0.5128	0.5128	30.0	32.0	83.0	0.3855	0.3614	0.0	0.0	1.0	0.0	0.0
0.0	17.0	153	6.6191	0.0059	2855.2502	1979.1086	131.0	299.0	0.4381	129.0	0.4314	18.0	18.0	64.0	0.2812	0.2812	41.0	41.0	73.0	0.5616	0.5616	41.0	41.0	78.0	0.5256	0.5256	29.0	31.0	83.0	0.3735	0.3494	0.0	0.0	1.0	0.0	0.0
0.0	18.0	162	6.6336	0.0059	2861.5271	1983.4594	130.0	299.0	0.4348	128.0	0.4281	17.0	17.0	64.0	0.2656	0.2656	41.0	41.0	73.0	0.5616	0.5616	40.0	40.0	78.0	0.5128	0.5128	30.0	32.0	83.0	0.3855	0.3614	0.0	0.0	1.0	0.0	0.0
0.0	19.0	171	6.6500	0.0059	2868.5902	1988.3552	132.0	299.0	0.4415	130.0	0.4348	17.0	17.0	64.0	0.2656	0.2656	42.0	42.0	73.0	0.5753	0.5753	41.0	41.0	78.0	0.5256	0.5256	30.0	32.0	83.0	0.3855	0.3614	0.0	0.0	1.0	0.0	0.0
0.0	20.0	180	6.6735	0.0059	2878.7257	1995.3806	130.0	299.0	0.4348	128.0	0.4281	17.0	17.0	64.0	0.2656	0.2656	41.0	41.0	73.0	0.5616	0.5616	41.0	41.0	78.0	0.5256	0.5256	29.0	31.0	83.0	0.3735	0.3494	0.0	0.0	1.0	0.0	0.0
0.0	21.0	189	6.6351	0.0059	2862.1416	1983.8854	132.0	299.0	0.4415	130.0	0.4348	17.0	17.0	64.0	0.2656	0.2656	41.0	41.0	73.0	0.5616	0.5616	42.0	42.0	78.0	0.5385	0.5385	30.0	32.0	83.0	0.3855	0.3614	0.0	0.0	1.0	0.0	0.0
0.0	22.0	198	6.6549	0.0059	2870.7075	1989.8228	131.0	299.0	0.4381	129.0	0.4314	17.0	17.0	64.0	0.2656	0.2656	41.0	41.0	73.0	0.5616	0.5616	41.0	41.0	78.0	0.5256	0.5256	30.0	32.0	83.0	0.3855	0.3614	0.0	0.0	1.0	0.0	0.0
0.0	23.0	207	6.6381	0.0059	2863.4371	1984.7834	131.0	299.0	0.4381	129.0	0.4314	17.0	17.0	64.0	0.2656	0.2656	41.0	41.0	73.0	0.5616	0.5616	41.0	41.0	78.0	0.5256	0.5256	30.0	32.0	83.0	0.3855	0.3614	0.0	0.0	1.0	0.0	0.0
0.0	24.0	216	6.6672	0.0059	2876.0102	1993.4984	131.0	299.0	0.4381	129.0	0.4314	17.0	17.0	64.0	0.2656	0.2656	41.0	41.0	73.0	0.5616	0.5616	41.0	41.0	78.0	0.5256	0.5256	30.0	32.0	83.0	0.3855	0.3614	0.0	0.0	1.0	0.0	0.0
0.0	25.0	225	6.6590	0.0059	2872.4652	1991.0411	131.0	299.0	0.4381	129.0	0.4314	17.0	17.0	64.0	0.2656	0.2656	41.0	41.0	73.0	0.5616	0.5616	41.0	41.0	78.0	0.5256	0.5256	30.0	32.0	83.0	0.3855	0.3614	0.0	0.0	1.0	0.0	0.0
0.0	26.0	234	6.7108	0.0059	2894.7914	2006.5165	130.0	299.0	0.4348	128.0	0.4281	17.0	17.0	64.0	0.2656	0.2656	41.0	41.0	73.0	0.5616	0.5616	40.0	40.0	78.0	0.5128	0.5128	30.0	32.0	83.0	0.3855	0.3614	0.0	0.0	1.0	0.0	0.0
0.0	27.0	243	6.6298	0.0059	2859.8692	1982.3102	132.0	299.0	0.4415	130.0	0.4348	18.0	18.0	64.0	0.2812	0.2812	41.0	41.0	73.0	0.5616	0.5616	41.0	41.0	78.0	0.5256	0.5256	30.0	32.0	83.0	0.3855	0.3614	0.0	0.0	1.0	0.0	0.0
0.0	28.0	252	6.6753	0.0059	2879.4858	1995.9075	132.0	299.0	0.4415	130.0	0.4348	17.0	17.0	64.0	0.2656	0.2656	41.0	41.0	73.0	0.5616	0.5616	41.0	41.0	78.0	0.5256	0.5256	31.0	33.0	83.0	0.3976	0.3735	0.0	0.0	1.0	0.0	0.0
0.0	29.0	261	6.6725	0.0059	2878.2884	1995.0775	131.0	299.0	0.4381	129.0	0.4314	17.0	17.0	64.0	0.2656	0.2656	41.0	41.0	73.0	0.5616	0.5616	41.0	41.0	78.0	0.5256	0.5256	30.0	32.0	83.0	0.3855	0.3614	0.0	0.0	1.0	0.0	0.0

Framework versions

• Transformers 4.51.3
• Pytorch 2.6.0+cu124
• Datasets 3.5.0
• Tokenizers 0.21.1