ARC-Challenge_Llama-3.2-1B-loluosf6

This model is a fine-tuned version of meta-llama/Llama-3.2-1B on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 5.8432
• Model Preparation Time: 0.0059
• Mdl: 2520.5494
• Accumulated Loss: 1747.1117
• Correct Preds: 122.0
• Total Preds: 299.0
• Accuracy: 0.4080
• Correct Gen Preds: 97.0
• Gen Accuracy: 0.3244
• Correct Gen Preds 32: 10.0
• Correct Preds 32: 14.0
• Total Labels 32: 64.0
• Accuracy 32: 0.2188
• Gen Accuracy 32: 0.1562
• Correct Gen Preds 33: 26.0
• Correct Preds 33: 29.0
• Total Labels 33: 73.0
• Accuracy 33: 0.3973
• Gen Accuracy 33: 0.3562
• Correct Gen Preds 34: 23.0
• Correct Preds 34: 31.0
• Total Labels 34: 78.0
• Accuracy 34: 0.3974
• Gen Accuracy 34: 0.2949
• Correct Gen Preds 35: 38.0
• Correct Preds 35: 48.0
• Total Labels 35: 83.0
• Accuracy 35: 0.5783
• Gen Accuracy 35: 0.4578
• Correct Gen Preds 36: 0.0
• Correct Preds 36: 0.0
• Total Labels 36: 1.0
• Accuracy 36: 0.0
• Gen Accuracy 36: 0.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 112
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.01
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Model Preparation Time	Mdl	Accumulated Loss	Correct Preds	Total Preds	Accuracy	Correct Gen Preds	Gen Accuracy	Correct Gen Preds 32	Correct Preds 32	Total Labels 32	Accuracy 32	Gen Accuracy 32	Correct Gen Preds 33	Correct Preds 33	Total Labels 33	Accuracy 33	Gen Accuracy 33	Correct Gen Preds 34	Correct Preds 34	Total Labels 34	Accuracy 34	Gen Accuracy 34	Correct Gen Preds 35	Correct Preds 35	Total Labels 35	Accuracy 35	Gen Accuracy 35	Correct Gen Preds 36	Correct Preds 36	Total Labels 36	Accuracy 36	Gen Accuracy 36
No log	0	0	1.6389	0.0059	706.9523	490.0220	66.0	299.0	0.2207	66.0	0.2207	62.0	62.0	64.0	0.9688	0.9688	0.0	0.0	73.0	0.0	0.0	4.0	4.0	78.0	0.0513	0.0513	0.0	0.0	83.0	0.0	0.0	0.0	0.0	1.0	0.0	0.0
1.4979	1.0	4	1.5457	0.0059	666.7473	462.1540	81.0	299.0	0.2709	67.0	0.2241	5.0	10.0	64.0	0.1562	0.0781	0.0	0.0	73.0	0.0	0.0	0.0	0.0	78.0	0.0	0.0	62.0	71.0	83.0	0.8554	0.7470	0.0	0.0	1.0	0.0	0.0
1.4682	2.0	8	1.4092	0.0059	607.9005	421.3645	79.0	299.0	0.2642	79.0	0.2642	1.0	1.0	64.0	0.0156	0.0156	73.0	73.0	73.0	1.0	1.0	1.0	1.0	78.0	0.0128	0.0128	4.0	4.0	83.0	0.0482	0.0482	0.0	0.0	1.0	0.0	0.0
1.2176	3.0	12	1.3841	0.0059	597.0342	413.8326	92.0	299.0	0.3077	91.0	0.3043	10.0	11.0	64.0	0.1719	0.1562	16.0	16.0	73.0	0.2192	0.2192	18.0	18.0	78.0	0.2308	0.2308	47.0	47.0	83.0	0.5663	0.5663	0.0	0.0	1.0	0.0	0.0
0.5182	4.0	16	1.7969	0.0059	775.1112	537.2661	94.0	299.0	0.3144	73.0	0.2441	27.0	33.0	64.0	0.5156	0.4219	32.0	40.0	73.0	0.5479	0.4384	9.0	14.0	78.0	0.1795	0.1154	5.0	7.0	83.0	0.0843	0.0602	0.0	0.0	1.0	0.0	0.0
0.3996	5.0	20	1.9680	0.0059	848.9255	588.4303	109.0	299.0	0.3645	83.0	0.2776	18.0	23.0	64.0	0.3594	0.2812	23.0	28.0	73.0	0.3836	0.3151	23.0	30.0	78.0	0.3846	0.2949	19.0	28.0	83.0	0.3373	0.2289	0.0	0.0	1.0	0.0	0.0
0.0084	6.0	24	2.7735	0.0059	1196.3982	829.2800	110.0	299.0	0.3679	76.0	0.2542	3.0	10.0	64.0	0.1562	0.0469	30.0	38.0	73.0	0.5205	0.4110	24.0	37.0	78.0	0.4744	0.3077	19.0	25.0	83.0	0.3012	0.2289	0.0	0.0	1.0	0.0	0.0
0.0049	7.0	28	5.6064	0.0059	2418.4263	1676.3254	105.0	299.0	0.3512	97.0	0.3244	23.0	25.0	64.0	0.3906	0.3594	39.0	41.0	73.0	0.5616	0.5342	21.0	23.0	78.0	0.2949	0.2692	14.0	16.0	83.0	0.1928	0.1687	0.0	0.0	1.0	0.0	0.0
0.0	8.0	32	5.8432	0.0059	2520.5494	1747.1117	122.0	299.0	0.4080	97.0	0.3244	10.0	14.0	64.0	0.2188	0.1562	26.0	29.0	73.0	0.3973	0.3562	23.0	31.0	78.0	0.3974	0.2949	38.0	48.0	83.0	0.5783	0.4578	0.0	0.0	1.0	0.0	0.0
0.0	9.0	36	6.4728	0.0059	2792.1242	1935.3530	115.0	299.0	0.3846	92.0	0.3077	5.0	7.0	64.0	0.1094	0.0781	20.0	25.0	73.0	0.3425	0.2740	25.0	31.0	78.0	0.3974	0.3205	42.0	52.0	83.0	0.6265	0.5060	0.0	0.0	1.0	0.0	0.0
0.0	10.0	40	6.6898	0.0059	2885.7536	2000.2520	115.0	299.0	0.3846	91.0	0.3043	5.0	7.0	64.0	0.1094	0.0781	19.0	22.0	73.0	0.3014	0.2603	25.0	32.0	78.0	0.4103	0.3205	42.0	54.0	83.0	0.6506	0.5060	0.0	0.0	1.0	0.0	0.0
0.0	11.0	44	6.7335	0.0059	2904.6211	2013.3300	113.0	299.0	0.3779	89.0	0.2977	5.0	7.0	64.0	0.1094	0.0781	17.0	21.0	73.0	0.2877	0.2329	26.0	32.0	78.0	0.4103	0.3333	41.0	53.0	83.0	0.6386	0.4940	0.0	0.0	1.0	0.0	0.0
0.0	12.0	48	6.7738	0.0059	2921.9869	2025.3670	115.0	299.0	0.3846	91.0	0.3043	5.0	6.0	64.0	0.0938	0.0781	18.0	22.0	73.0	0.3014	0.2466	26.0	34.0	78.0	0.4359	0.3333	42.0	53.0	83.0	0.6386	0.5060	0.0	0.0	1.0	0.0	0.0
0.0	13.0	52	6.7875	0.0059	2927.9036	2029.4681	113.0	299.0	0.3779	88.0	0.2943	6.0	7.0	64.0	0.1094	0.0938	18.0	23.0	73.0	0.3151	0.2466	24.0	32.0	78.0	0.4103	0.3077	40.0	51.0	83.0	0.6145	0.4819	0.0	0.0	1.0	0.0	0.0
0.0	14.0	56	6.7621	0.0059	2916.9297	2021.8616	114.0	299.0	0.3813	90.0	0.3010	5.0	6.0	64.0	0.0938	0.0781	19.0	24.0	73.0	0.3288	0.2603	25.0	33.0	78.0	0.4231	0.3205	41.0	51.0	83.0	0.6145	0.4940	0.0	0.0	1.0	0.0	0.0
0.0	15.0	60	6.7680	0.0059	2919.4702	2023.6225	114.0	299.0	0.3813	90.0	0.3010	5.0	7.0	64.0	0.1094	0.0781	18.0	23.0	73.0	0.3151	0.2466	27.0	34.0	78.0	0.4359	0.3462	40.0	50.0	83.0	0.6024	0.4819	0.0	0.0	1.0	0.0	0.0
0.0	16.0	64	6.7693	0.0059	2920.0377	2024.0159	113.0	299.0	0.3779	90.0	0.3010	5.0	5.0	64.0	0.0781	0.0781	19.0	24.0	73.0	0.3288	0.2603	27.0	34.0	78.0	0.4359	0.3462	39.0	50.0	83.0	0.6024	0.4699	0.0	0.0	1.0	0.0	0.0
0.0	17.0	68	6.7961	0.0059	2931.5977	2032.0287	114.0	299.0	0.3813	88.0	0.2943	5.0	7.0	64.0	0.1094	0.0781	18.0	22.0	73.0	0.3014	0.2466	26.0	34.0	78.0	0.4359	0.3333	39.0	51.0	83.0	0.6145	0.4699	0.0	0.0	1.0	0.0	0.0
0.0	18.0	72	6.7655	0.0059	2918.4058	2022.8847	115.0	299.0	0.3846	91.0	0.3043	6.0	7.0	64.0	0.1094	0.0938	19.0	24.0	73.0	0.3288	0.2603	26.0	33.0	78.0	0.4231	0.3333	40.0	51.0	83.0	0.6145	0.4819	0.0	0.0	1.0	0.0	0.0
0.0	19.0	76	6.7818	0.0059	2925.4243	2027.7496	116.0	299.0	0.3880	89.0	0.2977	5.0	8.0	64.0	0.125	0.0781	19.0	24.0	73.0	0.3288	0.2603	25.0	33.0	78.0	0.4231	0.3205	40.0	51.0	83.0	0.6145	0.4819	0.0	0.0	1.0	0.0	0.0
0.0	20.0	80	6.7830	0.0059	2925.9735	2028.1302	118.0	299.0	0.3946	93.0	0.3110	6.0	7.0	64.0	0.1094	0.0938	19.0	24.0	73.0	0.3288	0.2603	28.0	36.0	78.0	0.4615	0.3590	40.0	51.0	83.0	0.6145	0.4819	0.0	0.0	1.0	0.0	0.0
0.0	21.0	84	6.7955	0.0059	2931.3584	2031.8628	116.0	299.0	0.3880	90.0	0.3010	6.0	7.0	64.0	0.1094	0.0938	19.0	24.0	73.0	0.3288	0.2603	25.0	33.0	78.0	0.4231	0.3205	40.0	52.0	83.0	0.6265	0.4819	0.0	0.0	1.0	0.0	0.0
0.0	22.0	88	6.7723	0.0059	2921.3602	2024.9326	118.0	299.0	0.3946	92.0	0.3077	6.0	8.0	64.0	0.125	0.0938	19.0	24.0	73.0	0.3288	0.2603	26.0	34.0	78.0	0.4359	0.3333	41.0	52.0	83.0	0.6265	0.4940	0.0	0.0	1.0	0.0	0.0
0.0	23.0	92	6.7650	0.0059	2918.1885	2022.7341	116.0	299.0	0.3880	92.0	0.3077	5.0	7.0	64.0	0.1094	0.0781	19.0	24.0	73.0	0.3288	0.2603	27.0	34.0	78.0	0.4359	0.3462	41.0	51.0	83.0	0.6145	0.4940	0.0	0.0	1.0	0.0	0.0
0.0	24.0	96	6.7826	0.0059	2925.7924	2028.0048	117.0	299.0	0.3913	91.0	0.3043	5.0	7.0	64.0	0.1094	0.0781	19.0	24.0	73.0	0.3288	0.2603	26.0	34.0	78.0	0.4359	0.3333	41.0	52.0	83.0	0.6265	0.4940	0.0	0.0	1.0	0.0	0.0
0.0	25.0	100	6.7962	0.0059	2931.6604	2032.0721	118.0	299.0	0.3946	92.0	0.3077	6.0	8.0	64.0	0.125	0.0938	19.0	24.0	73.0	0.3288	0.2603	26.0	34.0	78.0	0.4359	0.3333	41.0	52.0	83.0	0.6265	0.4940	0.0	0.0	1.0	0.0	0.0
0.0	26.0	104	6.8102	0.0059	2937.6706	2036.2381	117.0	299.0	0.3913	91.0	0.3043	6.0	8.0	64.0	0.125	0.0938	18.0	23.0	73.0	0.3151	0.2466	26.0	34.0	78.0	0.4359	0.3333	41.0	52.0	83.0	0.6265	0.4940	0.0	0.0	1.0	0.0	0.0
0.0	27.0	108	6.7930	0.0059	2930.2745	2031.1115	115.0	299.0	0.3846	90.0	0.3010	5.0	6.0	64.0	0.0938	0.0781	18.0	23.0	73.0	0.3151	0.2466	27.0	35.0	78.0	0.4487	0.3462	40.0	51.0	83.0	0.6145	0.4819	0.0	0.0	1.0	0.0	0.0
0.0	28.0	112	6.8260	0.0059	2944.5097	2040.9786	114.0	299.0	0.3813	90.0	0.3010	6.0	7.0	64.0	0.1094	0.0938	19.0	24.0	73.0	0.3288	0.2603	26.0	34.0	78.0	0.4359	0.3333	39.0	49.0	83.0	0.5904	0.4699	0.0	0.0	1.0	0.0	0.0
0.0	29.0	116	6.7808	0.0059	2925.0005	2027.4559	116.0	299.0	0.3880	90.0	0.3010	5.0	7.0	64.0	0.1094	0.0781	19.0	24.0	73.0	0.3288	0.2603	26.0	34.0	78.0	0.4359	0.3333	40.0	51.0	83.0	0.6145	0.4819	0.0	0.0	1.0	0.0	0.0
0.0	30.0	120	6.7914	0.0059	2929.5611	2030.6170	117.0	299.0	0.3913	91.0	0.3043	5.0	7.0	64.0	0.1094	0.0781	18.0	23.0	73.0	0.3151	0.2466	27.0	35.0	78.0	0.4487	0.3462	41.0	52.0	83.0	0.6265	0.4940	0.0	0.0	1.0	0.0	0.0
0.0	31.0	124	6.7832	0.0059	2926.0476	2028.1816	119.0	299.0	0.3980	94.0	0.3144	6.0	7.0	64.0	0.1094	0.0938	19.0	24.0	73.0	0.3288	0.2603	28.0	36.0	78.0	0.4615	0.3590	41.0	52.0	83.0	0.6265	0.4940	0.0	0.0	1.0	0.0	0.0
0.0	32.0	128	6.7726	0.0059	2921.4872	2025.0206	118.0	299.0	0.3946	92.0	0.3077	5.0	6.0	64.0	0.0938	0.0781	19.0	24.0	73.0	0.3288	0.2603	27.0	35.0	78.0	0.4487	0.3462	41.0	53.0	83.0	0.6386	0.4940	0.0	0.0	1.0	0.0	0.0
0.0	33.0	132	6.8025	0.0059	2934.3651	2033.9469	116.0	299.0	0.3880	92.0	0.3077	6.0	7.0	64.0	0.1094	0.0938	19.0	24.0	73.0	0.3288	0.2603	25.0	32.0	78.0	0.4103	0.3205	42.0	53.0	83.0	0.6386	0.5060	0.0	0.0	1.0	0.0	0.0
0.0	34.0	136	6.8048	0.0059	2935.3420	2034.6240	115.0	299.0	0.3846	92.0	0.3077	5.0	6.0	64.0	0.0938	0.0781	19.0	23.0	73.0	0.3151	0.2603	27.0	35.0	78.0	0.4487	0.3462	41.0	51.0	83.0	0.6145	0.4940	0.0	0.0	1.0	0.0	0.0
0.0	35.0	140	6.7888	0.0059	2928.4631	2029.8560	114.0	299.0	0.3813	91.0	0.3043	5.0	6.0	64.0	0.0938	0.0781	19.0	24.0	73.0	0.3288	0.2603	27.0	34.0	78.0	0.4359	0.3462	40.0	50.0	83.0	0.6024	0.4819	0.0	0.0	1.0	0.0	0.0
0.0	36.0	144	6.8412	0.0059	2951.0670	2045.5238	117.0	299.0	0.3913	89.0	0.2977	5.0	8.0	64.0	0.125	0.0781	19.0	24.0	73.0	0.3288	0.2603	26.0	34.0	78.0	0.4359	0.3333	39.0	51.0	83.0	0.6145	0.4699	0.0	0.0	1.0	0.0	0.0
0.0	37.0	148	6.8074	0.0059	2936.4973	2035.4248	119.0	299.0	0.3980	93.0	0.3110	6.0	7.0	64.0	0.1094	0.0938	19.0	24.0	73.0	0.3288	0.2603	26.0	34.0	78.0	0.4359	0.3333	42.0	54.0	83.0	0.6506	0.5060	0.0	0.0	1.0	0.0	0.0
0.0	38.0	152	6.7758	0.0059	2922.8346	2025.9546	114.0	299.0	0.3813	90.0	0.3010	5.0	6.0	64.0	0.0938	0.0781	19.0	24.0	73.0	0.3288	0.2603	25.0	33.0	78.0	0.4231	0.3205	41.0	51.0	83.0	0.6145	0.4940	0.0	0.0	1.0	0.0	0.0

Framework versions

• Transformers 4.51.3
• Pytorch 2.6.0+cu124
• Datasets 3.5.0
• Tokenizers 0.21.1