ARC-Challenge_Llama-3.2-1B-69bpzmft

This model is a fine-tuned version of meta-llama/Llama-3.2-1B on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 3.4975
• Model Preparation Time: 0.006
• Mdl: 1508.7190
• Accumulated Loss: 1045.7643
• Correct Preds: 106.0
• Total Preds: 299.0
• Accuracy: 0.3545
• Correct Gen Preds: 60.0
• Gen Accuracy: 0.2007
• Correct Gen Preds 32: 8.0
• Correct Preds 32: 24.0
• Total Labels 32: 64.0
• Accuracy 32: 0.375
• Gen Accuracy 32: 0.125
• Correct Gen Preds 33: 15.0
• Correct Preds 33: 29.0
• Total Labels 33: 73.0
• Accuracy 33: 0.3973
• Gen Accuracy 33: 0.2055
• Correct Gen Preds 34: 19.0
• Correct Preds 34: 25.0
• Total Labels 34: 78.0
• Accuracy 34: 0.3205
• Gen Accuracy 34: 0.2436
• Correct Gen Preds 35: 18.0
• Correct Preds 35: 28.0
• Total Labels 35: 83.0
• Accuracy 35: 0.3373
• Gen Accuracy 35: 0.2169
• Correct Gen Preds 36: 0.0
• Correct Preds 36: 0.0
• Total Labels 36: 1.0
• Accuracy 36: 0.0
• Gen Accuracy 36: 0.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 112
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.01
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Model Preparation Time	Mdl	Accumulated Loss	Correct Preds	Total Preds	Accuracy	Correct Gen Preds	Gen Accuracy	Correct Gen Preds 32	Correct Preds 32	Total Labels 32	Accuracy 32	Gen Accuracy 32	Correct Gen Preds 33	Correct Preds 33	Total Labels 33	Accuracy 33	Gen Accuracy 33	Correct Gen Preds 34	Correct Preds 34	Total Labels 34	Accuracy 34	Gen Accuracy 34	Correct Gen Preds 35	Correct Preds 35	Total Labels 35	Accuracy 35	Gen Accuracy 35	Correct Gen Preds 36	Correct Preds 36	Total Labels 36	Accuracy 36	Gen Accuracy 36
No log	0	0	1.6389	0.006	706.9523	490.0220	66.0	299.0	0.2207	66.0	0.2207	62.0	62.0	64.0	0.9688	0.9688	0.0	0.0	73.0	0.0	0.0	4.0	4.0	78.0	0.0513	0.0513	0.0	0.0	83.0	0.0	0.0	0.0	0.0	1.0	0.0	0.0
1.5464	1.0	1	1.6389	0.006	706.9523	490.0220	66.0	299.0	0.2207	66.0	0.2207	62.0	62.0	64.0	0.9688	0.9688	0.0	0.0	73.0	0.0	0.0	4.0	4.0	78.0	0.0513	0.0513	0.0	0.0	83.0	0.0	0.0	0.0	0.0	1.0	0.0	0.0
1.5451	2.0	2	1.9494	0.006	840.8880	582.8591	73.0	299.0	0.2441	73.0	0.2441	0.0	0.0	64.0	0.0	0.0	73.0	73.0	73.0	1.0	1.0	0.0	0.0	78.0	0.0	0.0	0.0	0.0	83.0	0.0	0.0	0.0	0.0	1.0	0.0	0.0
2.1729	3.0	3	1.3883	0.006	598.8449	415.0876	89.0	299.0	0.2977	89.0	0.2977	8.0	8.0	64.0	0.125	0.125	53.0	53.0	73.0	0.7260	0.7260	0.0	0.0	78.0	0.0	0.0	28.0	28.0	83.0	0.3373	0.3373	0.0	0.0	1.0	0.0	0.0
1.2974	4.0	4	2.3810	0.006	1027.0673	711.9088	64.0	299.0	0.2140	64.0	0.2140	64.0	64.0	64.0	1.0	1.0	0.0	0.0	73.0	0.0	0.0	0.0	0.0	78.0	0.0	0.0	0.0	0.0	83.0	0.0	0.0	0.0	0.0	1.0	0.0	0.0
1.3902	5.0	5	1.5833	0.006	682.9726	473.4005	68.0	299.0	0.2274	68.0	0.2274	64.0	64.0	64.0	1.0	1.0	0.0	0.0	73.0	0.0	0.0	2.0	2.0	78.0	0.0256	0.0256	2.0	2.0	83.0	0.0241	0.0241	0.0	0.0	1.0	0.0	0.0
0.9586	6.0	6	1.6103	0.006	694.6407	481.4882	81.0	299.0	0.2709	79.0	0.2642	30.0	32.0	64.0	0.5	0.4688	2.0	2.0	73.0	0.0274	0.0274	17.0	17.0	78.0	0.2179	0.2179	30.0	30.0	83.0	0.3614	0.3614	0.0	0.0	1.0	0.0	0.0
0.5845	7.0	7	2.2464	0.006	969.0044	671.6627	89.0	299.0	0.2977	79.0	0.2642	30.0	36.0	64.0	0.5625	0.4688	12.0	14.0	73.0	0.1918	0.1644	17.0	18.0	78.0	0.2308	0.2179	20.0	21.0	83.0	0.2530	0.2410	0.0	0.0	1.0	0.0	0.0
0.2352	8.0	8	2.7941	0.006	1205.2816	835.4375	97.0	299.0	0.3244	74.0	0.2475	5.0	11.0	64.0	0.1719	0.0781	30.0	40.0	73.0	0.5479	0.4110	19.0	21.0	78.0	0.2692	0.2436	20.0	25.0	83.0	0.3012	0.2410	0.0	0.0	1.0	0.0	0.0
0.0743	9.0	9	3.4975	0.006	1508.7190	1045.7643	106.0	299.0	0.3545	60.0	0.2007	8.0	24.0	64.0	0.375	0.125	15.0	29.0	73.0	0.3973	0.2055	19.0	25.0	78.0	0.3205	0.2436	18.0	28.0	83.0	0.3373	0.2169	0.0	0.0	1.0	0.0	0.0
0.0085	10.0	10	3.8403	0.006	1656.5664	1148.2444	102.0	299.0	0.3411	51.0	0.1706	5.0	20.0	64.0	0.3125	0.0781	10.0	24.0	73.0	0.3288	0.1370	20.0	30.0	78.0	0.3846	0.2564	16.0	28.0	83.0	0.3373	0.1928	0.0	0.0	1.0	0.0	0.0
0.0014	11.0	11	4.6550	0.006	2008.0190	1391.8527	96.0	299.0	0.3211	41.0	0.1371	5.0	23.0	64.0	0.3594	0.0781	11.0	29.0	73.0	0.3973	0.1507	17.0	29.0	78.0	0.3718	0.2179	8.0	15.0	83.0	0.1807	0.0964	0.0	0.0	1.0	0.0	0.0
0.0001	12.0	12	5.6982	0.006	2458.0069	1703.7606	88.0	299.0	0.2943	42.0	0.1405	7.0	24.0	64.0	0.375	0.1094	14.0	33.0	73.0	0.4521	0.1918	17.0	24.0	78.0	0.3077	0.2179	4.0	7.0	83.0	0.0843	0.0482	0.0	0.0	1.0	0.0	0.0
0.0001	13.0	13	6.9024	0.006	2977.4599	2063.8179	91.0	299.0	0.3043	49.0	0.1639	11.0	26.0	64.0	0.4062	0.1719	16.0	33.0	73.0	0.4521	0.2192	19.0	27.0	78.0	0.3462	0.2436	3.0	5.0	83.0	0.0602	0.0361	0.0	0.0	1.0	0.0	0.0
0.0	14.0	14	7.7997	0.006	3364.5048	2332.0970	88.0	299.0	0.2943	61.0	0.2040	15.0	25.0	64.0	0.3906	0.2344	21.0	31.0	73.0	0.4247	0.2877	22.0	27.0	78.0	0.3462	0.2821	3.0	5.0	83.0	0.0602	0.0361	0.0	0.0	1.0	0.0	0.0
0.0	15.0	15	8.4535	0.006	3646.5475	2527.5941	86.0	299.0	0.2876	66.0	0.2207	18.0	25.0	64.0	0.3906	0.2812	21.0	30.0	73.0	0.4110	0.2877	24.0	26.0	78.0	0.3333	0.3077	3.0	5.0	83.0	0.0602	0.0361	0.0	0.0	1.0	0.0	0.0
0.0	16.0	16	8.9836	0.006	3875.2248	2686.1011	84.0	299.0	0.2809	67.0	0.2241	21.0	26.0	64.0	0.4062	0.3281	19.0	27.0	73.0	0.3699	0.2603	24.0	26.0	78.0	0.3333	0.3077	3.0	5.0	83.0	0.0602	0.0361	0.0	0.0	1.0	0.0	0.0
0.0	17.0	17	9.3455	0.006	4031.3285	2794.3040	83.0	299.0	0.2776	70.0	0.2341	22.0	28.0	64.0	0.4375	0.3438	21.0	26.0	73.0	0.3562	0.2877	24.0	24.0	78.0	0.3077	0.3077	3.0	5.0	83.0	0.0602	0.0361	0.0	0.0	1.0	0.0	0.0
0.0	18.0	18	9.6760	0.006	4173.8792	2893.1126	82.0	299.0	0.2742	73.0	0.2441	25.0	28.0	64.0	0.4375	0.3906	21.0	24.0	73.0	0.3288	0.2877	24.0	25.0	78.0	0.3205	0.3077	3.0	5.0	83.0	0.0602	0.0361	0.0	0.0	1.0	0.0	0.0
0.0	19.0	19	9.9141	0.006	4276.5853	2964.3030	81.0	299.0	0.2709	74.0	0.2475	27.0	29.0	64.0	0.4531	0.4219	20.0	23.0	73.0	0.3151	0.2740	24.0	24.0	78.0	0.3077	0.3077	3.0	5.0	83.0	0.0602	0.0361	0.0	0.0	1.0	0.0	0.0
0.0	20.0	20	10.0790	0.006	4347.7385	3013.6227	82.0	299.0	0.2742	75.0	0.2508	27.0	29.0	64.0	0.4531	0.4219	21.0	22.0	73.0	0.3014	0.2877	24.0	25.0	78.0	0.3205	0.3077	3.0	6.0	83.0	0.0723	0.0361	0.0	0.0	1.0	0.0	0.0
0.0	21.0	21	10.1972	0.006	4398.7081	3048.9521	80.0	299.0	0.2676	75.0	0.2508	28.0	30.0	64.0	0.4688	0.4375	20.0	21.0	73.0	0.2877	0.2740	24.0	24.0	78.0	0.3077	0.3077	3.0	5.0	83.0	0.0602	0.0361	0.0	0.0	1.0	0.0	0.0
0.0	22.0	22	10.3103	0.006	4447.5317	3082.7941	81.0	299.0	0.2709	74.0	0.2475	27.0	29.0	64.0	0.4531	0.4219	20.0	21.0	73.0	0.2877	0.2740	24.0	25.0	78.0	0.3205	0.3077	3.0	6.0	83.0	0.0723	0.0361	0.0	0.0	1.0	0.0	0.0
0.0	23.0	23	10.4009	0.006	4486.6061	3109.8784	81.0	299.0	0.2709	76.0	0.2542	29.0	31.0	64.0	0.4844	0.4531	20.0	21.0	73.0	0.2877	0.2740	24.0	24.0	78.0	0.3077	0.3077	3.0	5.0	83.0	0.0602	0.0361	0.0	0.0	1.0	0.0	0.0
0.0	24.0	24	10.4894	0.006	4524.7495	3136.3174	80.0	299.0	0.2676	76.0	0.2542	29.0	31.0	64.0	0.4844	0.4531	20.0	21.0	73.0	0.2877	0.2740	24.0	24.0	78.0	0.3077	0.3077	3.0	4.0	83.0	0.0482	0.0361	0.0	0.0	1.0	0.0	0.0
0.0	25.0	25	10.5657	0.006	4557.6851	3159.1466	80.0	299.0	0.2676	75.0	0.2508	28.0	30.0	64.0	0.4688	0.4375	20.0	21.0	73.0	0.2877	0.2740	24.0	24.0	78.0	0.3077	0.3077	3.0	5.0	83.0	0.0602	0.0361	0.0	0.0	1.0	0.0	0.0
0.0	26.0	26	10.5629	0.006	4556.4933	3158.3205	81.0	299.0	0.2709	76.0	0.2542	29.0	31.0	64.0	0.4844	0.4531	20.0	21.0	73.0	0.2877	0.2740	24.0	24.0	78.0	0.3077	0.3077	3.0	5.0	83.0	0.0602	0.0361	0.0	0.0	1.0	0.0	0.0
0.0	27.0	27	10.6133	0.006	4578.2155	3173.3771	79.0	299.0	0.2642	74.0	0.2475	28.0	30.0	64.0	0.4688	0.4375	19.0	20.0	73.0	0.2740	0.2603	24.0	24.0	78.0	0.3077	0.3077	3.0	5.0	83.0	0.0602	0.0361	0.0	0.0	1.0	0.0	0.0
0.0	28.0	28	10.6343	0.006	4587.2687	3179.6523	80.0	299.0	0.2676	75.0	0.2508	28.0	30.0	64.0	0.4688	0.4375	20.0	21.0	73.0	0.2877	0.2740	24.0	24.0	78.0	0.3077	0.3077	3.0	5.0	83.0	0.0602	0.0361	0.0	0.0	1.0	0.0	0.0
0.0	29.0	29	10.6752	0.006	4604.9267	3191.8920	78.0	299.0	0.2609	73.0	0.2441	27.0	29.0	64.0	0.4531	0.4219	19.0	20.0	73.0	0.2740	0.2603	24.0	24.0	78.0	0.3077	0.3077	3.0	5.0	83.0	0.0602	0.0361	0.0	0.0	1.0	0.0	0.0
0.0	30.0	30	10.7068	0.006	4618.5561	3201.3392	80.0	299.0	0.2676	75.0	0.2508	29.0	31.0	64.0	0.4844	0.4531	19.0	20.0	73.0	0.2740	0.2603	24.0	24.0	78.0	0.3077	0.3077	3.0	5.0	83.0	0.0602	0.0361	0.0	0.0	1.0	0.0	0.0
0.0	31.0	31	10.7169	0.006	4622.9235	3204.3664	81.0	299.0	0.2709	76.0	0.2542	30.0	32.0	64.0	0.5	0.4688	19.0	20.0	73.0	0.2740	0.2603	24.0	24.0	78.0	0.3077	0.3077	3.0	5.0	83.0	0.0602	0.0361	0.0	0.0	1.0	0.0	0.0
0.0	32.0	32	10.7301	0.006	4628.5873	3208.2922	79.0	299.0	0.2642	74.0	0.2475	28.0	30.0	64.0	0.4688	0.4375	19.0	20.0	73.0	0.2740	0.2603	24.0	24.0	78.0	0.3077	0.3077	3.0	5.0	83.0	0.0602	0.0361	0.0	0.0	1.0	0.0	0.0
0.0	33.0	33	10.7590	0.006	4641.0636	3216.9401	80.0	299.0	0.2676	75.0	0.2508	29.0	31.0	64.0	0.4844	0.4531	19.0	20.0	73.0	0.2740	0.2603	24.0	24.0	78.0	0.3077	0.3077	3.0	5.0	83.0	0.0602	0.0361	0.0	0.0	1.0	0.0	0.0
0.0	34.0	34	10.7597	0.006	4641.3614	3217.1465	79.0	299.0	0.2642	74.0	0.2475	29.0	31.0	64.0	0.4844	0.4531	18.0	19.0	73.0	0.2603	0.2466	24.0	24.0	78.0	0.3077	0.3077	3.0	5.0	83.0	0.0602	0.0361	0.0	0.0	1.0	0.0	0.0
0.0	35.0	35	10.8047	0.006	4660.7928	3230.6154	79.0	299.0	0.2642	74.0	0.2475	27.0	29.0	64.0	0.4531	0.4219	20.0	21.0	73.0	0.2877	0.2740	24.0	24.0	78.0	0.3077	0.3077	3.0	5.0	83.0	0.0602	0.0361	0.0	0.0	1.0	0.0	0.0
0.0	36.0	36	10.7758	0.006	4648.3271	3221.9749	78.0	299.0	0.2609	73.0	0.2441	28.0	30.0	64.0	0.4688	0.4375	18.0	19.0	73.0	0.2603	0.2466	24.0	24.0	78.0	0.3077	0.3077	3.0	5.0	83.0	0.0602	0.0361	0.0	0.0	1.0	0.0	0.0
0.0	37.0	37	10.7398	0.006	4632.7890	3211.2047	80.0	299.0	0.2676	75.0	0.2508	28.0	30.0	64.0	0.4688	0.4375	20.0	21.0	73.0	0.2877	0.2740	24.0	24.0	78.0	0.3077	0.3077	3.0	5.0	83.0	0.0602	0.0361	0.0	0.0	1.0	0.0	0.0
0.0	38.0	38	10.7692	0.006	4645.4815	3220.0024	79.0	299.0	0.2642	74.0	0.2475	28.0	30.0	64.0	0.4688	0.4375	19.0	20.0	73.0	0.2740	0.2603	24.0	24.0	78.0	0.3077	0.3077	3.0	5.0	83.0	0.0602	0.0361	0.0	0.0	1.0	0.0	0.0
0.0	39.0	39	10.7422	0.006	4633.8010	3211.9061	79.0	299.0	0.2642	74.0	0.2475	29.0	31.0	64.0	0.4844	0.4531	18.0	19.0	73.0	0.2603	0.2466	24.0	24.0	78.0	0.3077	0.3077	3.0	5.0	83.0	0.0602	0.0361	0.0	0.0	1.0	0.0	0.0

Framework versions

• Transformers 4.51.3
• Pytorch 2.6.0+cu124
• Datasets 3.5.0
• Tokenizers 0.21.1