ARC-Challenge_Llama-3.2-1B-c10ra53s

This model is a fine-tuned version of meta-llama/Llama-3.2-1B on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 6.0995
• Model Preparation Time: 0.0058
• Mdl: 2631.1071
• Accumulated Loss: 1823.7444
• Correct Preds: 97.0
• Total Preds: 299.0
• Accuracy: 0.3244
• Correct Gen Preds: 82.0
• Gen Accuracy: 0.2742
• Correct Gen Preds 32: 16.0
• Correct Preds 32: 19.0
• Total Labels 32: 64.0
• Accuracy 32: 0.2969
• Gen Accuracy 32: 0.25
• Correct Gen Preds 33: 16.0
• Correct Preds 33: 21.0
• Total Labels 33: 73.0
• Accuracy 33: 0.2877
• Gen Accuracy 33: 0.2192
• Correct Gen Preds 34: 27.0
• Correct Preds 34: 29.0
• Total Labels 34: 78.0
• Accuracy 34: 0.3718
• Gen Accuracy 34: 0.3462
• Correct Gen Preds 35: 23.0
• Correct Preds 35: 28.0
• Total Labels 35: 83.0
• Accuracy 35: 0.3373
• Gen Accuracy 35: 0.2771
• Correct Gen Preds 36: 0.0
• Correct Preds 36: 0.0
• Total Labels 36: 1.0
• Accuracy 36: 0.0
• Gen Accuracy 36: 0.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 112
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: constant
• lr_scheduler_warmup_ratio: 0.001
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Model Preparation Time	Mdl	Accumulated Loss	Correct Preds	Total Preds	Accuracy	Correct Gen Preds	Gen Accuracy	Correct Gen Preds 32	Correct Preds 32	Total Labels 32	Accuracy 32	Gen Accuracy 32	Correct Gen Preds 33	Correct Preds 33	Total Labels 33	Accuracy 33	Gen Accuracy 33	Correct Gen Preds 34	Correct Preds 34	Total Labels 34	Accuracy 34	Gen Accuracy 34	Correct Gen Preds 35	Correct Preds 35	Total Labels 35	Accuracy 35	Gen Accuracy 35	Correct Gen Preds 36	Correct Preds 36	Total Labels 36	Accuracy 36	Gen Accuracy 36
No log	0	0	1.6389	0.0058	706.9523	490.0220	66.0	299.0	0.2207	66.0	0.2207	62.0	62.0	64.0	0.9688	0.9688	0.0	0.0	73.0	0.0	0.0	4.0	4.0	78.0	0.0513	0.0513	0.0	0.0	83.0	0.0	0.0	0.0	0.0	1.0	0.0	0.0
1.3758	1.0	18	1.4013	0.0058	604.4840	418.9964	71.0	299.0	0.2375	71.0	0.2375	0.0	0.0	64.0	0.0	0.0	69.0	69.0	73.0	0.9452	0.9452	0.0	0.0	78.0	0.0	0.0	2.0	2.0	83.0	0.0241	0.0241	0.0	0.0	1.0	0.0	0.0
1.3755	2.0	36	1.4586	0.0058	629.1936	436.1238	76.0	299.0	0.2542	76.0	0.2542	23.0	23.0	64.0	0.3594	0.3594	53.0	53.0	73.0	0.7260	0.7260	0.0	0.0	78.0	0.0	0.0	0.0	0.0	83.0	0.0	0.0	0.0	0.0	1.0	0.0	0.0
1.3377	3.0	54	1.4563	0.0058	628.1836	435.4237	81.0	299.0	0.2709	68.0	0.2274	3.0	6.0	64.0	0.0938	0.0469	21.0	28.0	73.0	0.3836	0.2877	27.0	30.0	78.0	0.3846	0.3462	17.0	17.0	83.0	0.2048	0.2048	0.0	0.0	1.0	0.0	0.0
1.0744	4.0	72	1.9822	0.0058	855.0704	592.6897	95.0	299.0	0.3177	25.0	0.0836	10.0	29.0	64.0	0.4531	0.1562	8.0	31.0	73.0	0.4247	0.1096	1.0	6.0	78.0	0.0769	0.0128	6.0	29.0	83.0	0.3494	0.0723	0.0	0.0	1.0	0.0	0.0
0.0686	5.0	90	3.5919	0.0058	1549.4205	1073.9764	79.0	299.0	0.2642	66.0	0.2207	11.0	13.0	64.0	0.2031	0.1719	12.0	16.0	73.0	0.2192	0.1644	17.0	20.0	78.0	0.2564	0.2179	26.0	30.0	83.0	0.3614	0.3133	0.0	0.0	1.0	0.0	0.0
0.2085	6.0	108	3.0425	0.0058	1312.4147	909.6965	92.0	299.0	0.3077	54.0	0.1806	20.0	31.0	64.0	0.4844	0.3125	18.0	29.0	73.0	0.3973	0.2466	7.0	13.0	78.0	0.1667	0.0897	9.0	19.0	83.0	0.2289	0.1084	0.0	0.0	1.0	0.0	0.0
0.0068	7.0	126	3.7973	0.0058	1638.0238	1135.3916	95.0	299.0	0.3177	70.0	0.2341	14.0	19.0	64.0	0.2969	0.2188	18.0	23.0	73.0	0.3151	0.2466	6.0	13.0	78.0	0.1667	0.0769	32.0	40.0	83.0	0.4819	0.3855	0.0	0.0	1.0	0.0	0.0
0.0011	8.0	144	4.9191	0.0058	2121.9171	1470.8009	89.0	299.0	0.2977	65.0	0.2174	6.0	9.0	64.0	0.1406	0.0938	20.0	24.0	73.0	0.3288	0.2740	5.0	10.0	78.0	0.1282	0.0641	34.0	46.0	83.0	0.5542	0.4096	0.0	0.0	1.0	0.0	0.0
0.0016	9.0	162	4.6631	0.0058	2011.4961	1394.2628	88.0	299.0	0.2943	79.0	0.2642	5.0	5.0	64.0	0.0781	0.0781	42.0	45.0	73.0	0.6164	0.5753	16.0	18.0	78.0	0.2308	0.2051	16.0	20.0	83.0	0.2410	0.1928	0.0	0.0	1.0	0.0	0.0
0.0007	10.0	180	4.5146	0.0058	1947.4411	1349.8633	95.0	299.0	0.3177	67.0	0.2241	8.0	13.0	64.0	0.2031	0.125	14.0	22.0	73.0	0.3014	0.1918	31.0	34.0	78.0	0.4359	0.3974	14.0	26.0	83.0	0.3133	0.1687	0.0	0.0	1.0	0.0	0.0
0.0	11.0	198	5.8208	0.0058	2510.8767	1740.4071	84.0	299.0	0.2809	71.0	0.2375	10.0	12.0	64.0	0.1875	0.1562	23.0	26.0	73.0	0.3562	0.3151	27.0	29.0	78.0	0.3718	0.3462	11.0	17.0	83.0	0.2048	0.1325	0.0	0.0	1.0	0.0	0.0
0.0	12.0	216	6.1160	0.0058	2638.2529	1828.6975	91.0	299.0	0.3043	73.0	0.2441	16.0	20.0	64.0	0.3125	0.25	15.0	22.0	73.0	0.3014	0.2055	27.0	29.0	78.0	0.3718	0.3462	15.0	20.0	83.0	0.2410	0.1807	0.0	0.0	1.0	0.0	0.0
0.0	13.0	234	6.0995	0.0058	2631.1071	1823.7444	97.0	299.0	0.3244	82.0	0.2742	16.0	19.0	64.0	0.2969	0.25	16.0	21.0	73.0	0.2877	0.2192	27.0	29.0	78.0	0.3718	0.3462	23.0	28.0	83.0	0.3373	0.2771	0.0	0.0	1.0	0.0	0.0
0.0	14.0	252	6.1211	0.0058	2640.4380	1830.2121	97.0	299.0	0.3244	83.0	0.2776	16.0	19.0	64.0	0.2969	0.25	16.0	20.0	73.0	0.2740	0.2192	27.0	29.0	78.0	0.3718	0.3462	24.0	29.0	83.0	0.3494	0.2892	0.0	0.0	1.0	0.0	0.0
0.0	15.0	270	6.1095	0.0058	2635.4327	1826.7427	96.0	299.0	0.3211	80.0	0.2676	15.0	19.0	64.0	0.2969	0.2344	16.0	21.0	73.0	0.2877	0.2192	26.0	28.0	78.0	0.3590	0.3333	23.0	28.0	83.0	0.3373	0.2771	0.0	0.0	1.0	0.0	0.0
0.0	16.0	288	6.1098	0.0058	2635.5722	1826.8394	97.0	299.0	0.3244	82.0	0.2742	15.0	19.0	64.0	0.2969	0.2344	16.0	21.0	73.0	0.2877	0.2192	28.0	29.0	78.0	0.3718	0.3590	23.0	28.0	83.0	0.3373	0.2771	0.0	0.0	1.0	0.0	0.0
0.0	17.0	306	6.1077	0.0058	2634.6538	1826.2029	93.0	299.0	0.3110	78.0	0.2609	14.0	17.0	64.0	0.2656	0.2188	16.0	22.0	73.0	0.3014	0.2192	25.0	27.0	78.0	0.3462	0.3205	23.0	27.0	83.0	0.3253	0.2771	0.0	0.0	1.0	0.0	0.0
0.0	18.0	324	6.1211	0.0058	2640.4341	1830.2095	96.0	299.0	0.3211	82.0	0.2742	15.0	18.0	64.0	0.2812	0.2344	15.0	20.0	73.0	0.2740	0.2055	27.0	29.0	78.0	0.3718	0.3462	25.0	29.0	83.0	0.3494	0.3012	0.0	0.0	1.0	0.0	0.0
0.0	19.0	342	6.1326	0.0058	2645.3767	1833.6354	92.0	299.0	0.3077	77.0	0.2575	14.0	17.0	64.0	0.2656	0.2188	15.0	19.0	73.0	0.2603	0.2055	26.0	29.0	78.0	0.3718	0.3333	22.0	27.0	83.0	0.3253	0.2651	0.0	0.0	1.0	0.0	0.0
0.0	20.0	360	6.1207	0.0058	2640.2736	1830.0982	94.0	299.0	0.3144	78.0	0.2609	14.0	18.0	64.0	0.2812	0.2188	15.0	20.0	73.0	0.2740	0.2055	27.0	29.0	78.0	0.3718	0.3462	22.0	27.0	83.0	0.3253	0.2651	0.0	0.0	1.0	0.0	0.0
0.0	21.0	378	6.1287	0.0058	2643.7165	1832.4846	96.0	299.0	0.3211	79.0	0.2642	15.0	19.0	64.0	0.2969	0.2344	15.0	21.0	73.0	0.2877	0.2055	27.0	29.0	78.0	0.3718	0.3462	22.0	27.0	83.0	0.3253	0.2651	0.0	0.0	1.0	0.0	0.0
0.0	22.0	396	6.1133	0.0058	2637.0631	1827.8728	97.0	299.0	0.3244	82.0	0.2742	16.0	19.0	64.0	0.2969	0.25	16.0	22.0	73.0	0.3014	0.2192	28.0	29.0	78.0	0.3718	0.3590	22.0	27.0	83.0	0.3253	0.2651	0.0	0.0	1.0	0.0	0.0
0.0	23.0	414	6.1366	0.0058	2647.1070	1834.8347	97.0	299.0	0.3244	80.0	0.2676	15.0	19.0	64.0	0.2969	0.2344	15.0	21.0	73.0	0.2877	0.2055	27.0	29.0	78.0	0.3718	0.3462	23.0	28.0	83.0	0.3373	0.2771	0.0	0.0	1.0	0.0	0.0

Framework versions

• Transformers 4.51.3
• Pytorch 2.6.0+cu124
• Datasets 3.5.0
• Tokenizers 0.21.1