ARC-Challenge_Llama-3.2-1B-tnxr6u44

This model is a fine-tuned version of meta-llama/Llama-3.2-1B on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 6.2749
• Model Preparation Time: 0.0058
• Mdl: 2706.7820
• Accumulated Loss: 1876.1983
• Correct Preds: 108.0
• Total Preds: 299.0
• Accuracy: 0.3612
• Correct Gen Preds: 90.0
• Gen Accuracy: 0.3010
• Correct Gen Preds 32: 4.0
• Correct Preds 32: 5.0
• Total Labels 32: 64.0
• Accuracy 32: 0.0781
• Gen Accuracy 32: 0.0625
• Correct Gen Preds 33: 29.0
• Correct Preds 33: 36.0
• Total Labels 33: 73.0
• Accuracy 33: 0.4932
• Gen Accuracy 33: 0.3973
• Correct Gen Preds 34: 36.0
• Correct Preds 34: 40.0
• Total Labels 34: 78.0
• Accuracy 34: 0.5128
• Gen Accuracy 34: 0.4615
• Correct Gen Preds 35: 21.0
• Correct Preds 35: 27.0
• Total Labels 35: 83.0
• Accuracy 35: 0.3253
• Gen Accuracy 35: 0.2530
• Correct Gen Preds 36: 0.0
• Correct Preds 36: 0.0
• Total Labels 36: 1.0
• Accuracy 36: 0.0
• Gen Accuracy 36: 0.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 112
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.01
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Model Preparation Time	Mdl	Accumulated Loss	Correct Preds	Total Preds	Accuracy	Correct Gen Preds	Gen Accuracy	Correct Gen Preds 32	Correct Preds 32	Total Labels 32	Accuracy 32	Gen Accuracy 32	Correct Gen Preds 33	Correct Preds 33	Total Labels 33	Accuracy 33	Gen Accuracy 33	Correct Gen Preds 34	Correct Preds 34	Total Labels 34	Accuracy 34	Gen Accuracy 34	Correct Gen Preds 35	Correct Preds 35	Total Labels 35	Accuracy 35	Gen Accuracy 35	Correct Gen Preds 36	Correct Preds 36	Total Labels 36	Accuracy 36	Gen Accuracy 36
No log	0	0	1.6389	0.0058	706.9523	490.0220	66.0	299.0	0.2207	66.0	0.2207	62.0	62.0	64.0	0.9688	0.9688	0.0	0.0	73.0	0.0	0.0	4.0	4.0	78.0	0.0513	0.0513	0.0	0.0	83.0	0.0	0.0	0.0	0.0	1.0	0.0	0.0
1.774	1.0	1	1.6389	0.0058	706.9523	490.0220	66.0	299.0	0.2207	66.0	0.2207	62.0	62.0	64.0	0.9688	0.9688	0.0	0.0	73.0	0.0	0.0	4.0	4.0	78.0	0.0513	0.0513	0.0	0.0	83.0	0.0	0.0	0.0	0.0	1.0	0.0	0.0
1.774	2.0	2	2.2203	0.0058	957.7460	663.8590	73.0	299.0	0.2441	73.0	0.2441	0.0	0.0	64.0	0.0	0.0	72.0	72.0	73.0	0.9863	0.9863	1.0	1.0	78.0	0.0128	0.0128	0.0	0.0	83.0	0.0	0.0	0.0	0.0	1.0	0.0	0.0
1.4645	3.0	3	1.4500	0.0058	625.4967	433.5612	94.0	299.0	0.3144	94.0	0.3144	0.0	0.0	64.0	0.0	0.0	17.0	17.0	73.0	0.2329	0.2329	2.0	2.0	78.0	0.0256	0.0256	75.0	75.0	83.0	0.9036	0.9036	0.0	0.0	1.0	0.0	0.0
1.0605	4.0	4	1.6302	0.0058	703.2070	487.4260	74.0	299.0	0.2475	73.0	0.2441	0.0	0.0	64.0	0.0	0.0	71.0	72.0	73.0	0.9863	0.9726	1.0	1.0	78.0	0.0128	0.0128	1.0	1.0	83.0	0.0120	0.0120	0.0	0.0	1.0	0.0	0.0
0.4866	5.0	5	2.1799	0.0058	940.3344	651.7902	85.0	299.0	0.2843	73.0	0.2441	1.0	2.0	64.0	0.0312	0.0156	52.0	63.0	73.0	0.8630	0.7123	10.0	10.0	78.0	0.1282	0.1282	10.0	10.0	83.0	0.1205	0.1205	0.0	0.0	1.0	0.0	0.0
0.0795	6.0	6	2.8177	0.0058	1215.4778	842.5050	96.0	299.0	0.3211	56.0	0.1873	4.0	14.0	64.0	0.2188	0.0625	27.0	48.0	73.0	0.6575	0.3699	9.0	14.0	78.0	0.1795	0.1154	16.0	20.0	83.0	0.2410	0.1928	0.0	0.0	1.0	0.0	0.0
0.0017	7.0	7	4.5963	0.0058	1982.6942	1374.2989	107.0	299.0	0.3579	79.0	0.2642	5.0	8.0	64.0	0.125	0.0781	27.0	42.0	73.0	0.5753	0.3699	23.0	26.0	78.0	0.3333	0.2949	24.0	31.0	83.0	0.3735	0.2892	0.0	0.0	1.0	0.0	0.0
0.0	8.0	8	6.2749	0.0058	2706.7820	1876.1983	108.0	299.0	0.3612	90.0	0.3010	4.0	5.0	64.0	0.0781	0.0625	29.0	36.0	73.0	0.4932	0.3973	36.0	40.0	78.0	0.5128	0.4615	21.0	27.0	83.0	0.3253	0.2530	0.0	0.0	1.0	0.0	0.0
0.0	9.0	9	7.6088	0.0058	3282.1736	2275.0294	102.0	299.0	0.3411	88.0	0.2943	3.0	4.0	64.0	0.0625	0.0469	25.0	28.0	73.0	0.3836	0.3425	42.0	47.0	78.0	0.6026	0.5385	18.0	23.0	83.0	0.2771	0.2169	0.0	0.0	1.0	0.0	0.0
0.0	10.0	10	8.4823	0.0058	3658.9878	2536.2171	99.0	299.0	0.3311	90.0	0.3010	3.0	3.0	64.0	0.0469	0.0469	26.0	28.0	73.0	0.3836	0.3562	41.0	44.0	78.0	0.5641	0.5256	20.0	24.0	83.0	0.2892	0.2410	0.0	0.0	1.0	0.0	0.0
0.0	11.0	11	9.0925	0.0058	3922.2033	2718.6642	100.0	299.0	0.3344	93.0	0.3110	3.0	3.0	64.0	0.0469	0.0469	26.0	28.0	73.0	0.3836	0.3562	41.0	44.0	78.0	0.5641	0.5256	23.0	25.0	83.0	0.3012	0.2771	0.0	0.0	1.0	0.0	0.0
0.0	12.0	12	9.3124	0.0058	4017.0339	2784.3958	97.0	299.0	0.3244	92.0	0.3077	1.0	1.0	64.0	0.0156	0.0156	24.0	26.0	73.0	0.3562	0.3288	42.0	43.0	78.0	0.5513	0.5385	25.0	27.0	83.0	0.3253	0.3012	0.0	0.0	1.0	0.0	0.0
0.0	13.0	13	9.4349	0.0058	4069.8925	2821.0345	100.0	299.0	0.3344	95.0	0.3177	1.0	1.0	64.0	0.0156	0.0156	25.0	28.0	73.0	0.3836	0.3425	40.0	41.0	78.0	0.5256	0.5128	29.0	30.0	83.0	0.3614	0.3494	0.0	0.0	1.0	0.0	0.0
0.0	14.0	14	9.5769	0.0058	4131.1632	2863.5042	102.0	299.0	0.3411	96.0	0.3211	1.0	1.0	64.0	0.0156	0.0156	23.0	26.0	73.0	0.3562	0.3151	40.0	41.0	78.0	0.5256	0.5128	32.0	34.0	83.0	0.4096	0.3855	0.0	0.0	1.0	0.0	0.0
0.0	15.0	15	9.7479	0.0058	4204.9260	2914.6326	101.0	299.0	0.3378	96.0	0.3211	1.0	1.0	64.0	0.0156	0.0156	22.0	25.0	73.0	0.3425	0.3014	40.0	41.0	78.0	0.5256	0.5128	33.0	34.0	83.0	0.4096	0.3976	0.0	0.0	1.0	0.0	0.0
0.0	16.0	16	9.8237	0.0058	4237.6167	2937.2921	101.0	299.0	0.3378	96.0	0.3211	1.0	1.0	64.0	0.0156	0.0156	23.0	26.0	73.0	0.3562	0.3151	39.0	40.0	78.0	0.5128	0.5	33.0	34.0	83.0	0.4096	0.3976	0.0	0.0	1.0	0.0	0.0
0.0	17.0	17	9.8771	0.0058	4260.6302	2953.2438	102.0	299.0	0.3411	97.0	0.3244	1.0	1.0	64.0	0.0156	0.0156	22.0	25.0	73.0	0.3425	0.3014	39.0	40.0	78.0	0.5128	0.5	35.0	36.0	83.0	0.4337	0.4217	0.0	0.0	1.0	0.0	0.0
0.0	18.0	18	10.0256	0.0058	4324.7020	2997.6550	99.0	299.0	0.3311	94.0	0.3144	1.0	1.0	64.0	0.0156	0.0156	21.0	24.0	73.0	0.3288	0.2877	38.0	39.0	78.0	0.5	0.4872	34.0	35.0	83.0	0.4217	0.4096	0.0	0.0	1.0	0.0	0.0
0.0	19.0	19	9.9950	0.0058	4311.4824	2988.4919	99.0	299.0	0.3311	94.0	0.3144	1.0	1.0	64.0	0.0156	0.0156	22.0	25.0	73.0	0.3425	0.3014	37.0	38.0	78.0	0.4872	0.4744	34.0	35.0	83.0	0.4217	0.4096	0.0	0.0	1.0	0.0	0.0
0.0	20.0	20	10.0191	0.0058	4321.9140	2995.7225	102.0	299.0	0.3411	97.0	0.3244	1.0	1.0	64.0	0.0156	0.0156	22.0	25.0	73.0	0.3425	0.3014	39.0	40.0	78.0	0.5128	0.5	35.0	36.0	83.0	0.4337	0.4217	0.0	0.0	1.0	0.0	0.0
0.0	21.0	21	10.0464	0.0058	4333.6653	3003.8679	101.0	299.0	0.3378	96.0	0.3211	1.0	1.0	64.0	0.0156	0.0156	22.0	24.0	73.0	0.3288	0.3014	39.0	40.0	78.0	0.5128	0.5	34.0	36.0	83.0	0.4337	0.4096	0.0	0.0	1.0	0.0	0.0
0.0	22.0	22	10.0371	0.0058	4329.6583	3001.0905	102.0	299.0	0.3411	97.0	0.3244	1.0	1.0	64.0	0.0156	0.0156	23.0	25.0	73.0	0.3425	0.3151	39.0	40.0	78.0	0.5128	0.5	34.0	36.0	83.0	0.4337	0.4096	0.0	0.0	1.0	0.0	0.0
0.0	23.0	23	10.0929	0.0058	4353.7427	3017.7845	100.0	299.0	0.3344	95.0	0.3177	1.0	1.0	64.0	0.0156	0.0156	21.0	24.0	73.0	0.3288	0.2877	38.0	39.0	78.0	0.5	0.4872	35.0	36.0	83.0	0.4337	0.4217	0.0	0.0	1.0	0.0	0.0
0.0	24.0	24	10.0993	0.0058	4356.5032	3019.6979	101.0	299.0	0.3378	97.0	0.3244	1.0	1.0	64.0	0.0156	0.0156	22.0	24.0	73.0	0.3288	0.3014	39.0	40.0	78.0	0.5128	0.5	35.0	36.0	83.0	0.4337	0.4217	0.0	0.0	1.0	0.0	0.0
0.0	25.0	25	10.0677	0.0058	4342.8494	3010.2338	99.0	299.0	0.3311	94.0	0.3144	1.0	1.0	64.0	0.0156	0.0156	21.0	24.0	73.0	0.3288	0.2877	38.0	39.0	78.0	0.5	0.4872	34.0	35.0	83.0	0.4217	0.4096	0.0	0.0	1.0	0.0	0.0
0.0	26.0	26	10.0313	0.0058	4327.1634	2999.3611	100.0	299.0	0.3344	95.0	0.3177	1.0	1.0	64.0	0.0156	0.0156	22.0	25.0	73.0	0.3425	0.3014	38.0	39.0	78.0	0.5	0.4872	34.0	35.0	83.0	0.4217	0.4096	0.0	0.0	1.0	0.0	0.0
0.0	27.0	27	10.0884	0.0058	4351.8004	3016.4382	97.0	299.0	0.3244	93.0	0.3110	1.0	1.0	64.0	0.0156	0.0156	21.0	24.0	73.0	0.3288	0.2877	38.0	39.0	78.0	0.5	0.4872	33.0	33.0	83.0	0.3976	0.3976	0.0	0.0	1.0	0.0	0.0
0.0	28.0	28	10.0954	0.0058	4354.7990	3018.5167	97.0	299.0	0.3244	92.0	0.3077	1.0	1.0	64.0	0.0156	0.0156	21.0	24.0	73.0	0.3288	0.2877	37.0	38.0	78.0	0.4872	0.4744	33.0	34.0	83.0	0.4096	0.3976	0.0	0.0	1.0	0.0	0.0
0.0	29.0	29	10.0557	0.0058	4337.6871	3006.6556	98.0	299.0	0.3278	94.0	0.3144	1.0	1.0	64.0	0.0156	0.0156	21.0	24.0	73.0	0.3288	0.2877	38.0	39.0	78.0	0.5	0.4872	34.0	34.0	83.0	0.4096	0.4096	0.0	0.0	1.0	0.0	0.0
0.0	30.0	30	10.0989	0.0058	4356.3412	3019.5856	94.0	299.0	0.3144	90.0	0.3010	1.0	1.0	64.0	0.0156	0.0156	22.0	24.0	73.0	0.3288	0.3014	34.0	35.0	78.0	0.4487	0.4359	33.0	34.0	83.0	0.4096	0.3976	0.0	0.0	1.0	0.0	0.0
0.0	31.0	31	10.1199	0.0058	4365.3782	3025.8496	98.0	299.0	0.3278	93.0	0.3110	1.0	1.0	64.0	0.0156	0.0156	21.0	24.0	73.0	0.3288	0.2877	37.0	38.0	78.0	0.4872	0.4744	34.0	35.0	83.0	0.4217	0.4096	0.0	0.0	1.0	0.0	0.0
0.0	32.0	32	10.0788	0.0058	4347.6442	3013.5573	99.0	299.0	0.3311	95.0	0.3177	1.0	1.0	64.0	0.0156	0.0156	21.0	23.0	73.0	0.3151	0.2877	37.0	38.0	78.0	0.4872	0.4744	36.0	37.0	83.0	0.4458	0.4337	0.0	0.0	1.0	0.0	0.0
0.0	33.0	33	10.0715	0.0058	4344.5197	3011.3916	100.0	299.0	0.3344	95.0	0.3177	1.0	1.0	64.0	0.0156	0.0156	22.0	25.0	73.0	0.3425	0.3014	38.0	39.0	78.0	0.5	0.4872	34.0	35.0	83.0	0.4217	0.4096	0.0	0.0	1.0	0.0	0.0
0.0	34.0	34	10.1057	0.0058	4359.2507	3021.6024	95.0	299.0	0.3177	91.0	0.3043	1.0	1.0	64.0	0.0156	0.0156	19.0	21.0	73.0	0.2877	0.2603	37.0	38.0	78.0	0.4872	0.4744	34.0	35.0	83.0	0.4217	0.4096	0.0	0.0	1.0	0.0	0.0
0.0	35.0	35	10.0934	0.0058	4353.9602	3017.9352	96.0	299.0	0.3211	92.0	0.3077	1.0	1.0	64.0	0.0156	0.0156	20.0	22.0	73.0	0.3014	0.2740	38.0	39.0	78.0	0.5	0.4872	33.0	34.0	83.0	0.4096	0.3976	0.0	0.0	1.0	0.0	0.0
0.0	36.0	36	10.0532	0.0058	4336.6062	3005.9064	99.0	299.0	0.3311	94.0	0.3144	1.0	1.0	64.0	0.0156	0.0156	22.0	25.0	73.0	0.3425	0.3014	38.0	39.0	78.0	0.5	0.4872	33.0	34.0	83.0	0.4096	0.3976	0.0	0.0	1.0	0.0	0.0
0.0	37.0	37	10.0802	0.0058	4348.2578	3013.9826	95.0	299.0	0.3177	92.0	0.3077	1.0	1.0	64.0	0.0156	0.0156	21.0	23.0	73.0	0.3151	0.2877	36.0	37.0	78.0	0.4744	0.4615	34.0	34.0	83.0	0.4096	0.4096	0.0	0.0	1.0	0.0	0.0
0.0	38.0	38	10.0748	0.0058	4345.9386	3012.3751	99.0	299.0	0.3311	94.0	0.3144	1.0	1.0	64.0	0.0156	0.0156	22.0	25.0	73.0	0.3425	0.3014	36.0	37.0	78.0	0.4744	0.4615	35.0	36.0	83.0	0.4337	0.4217	0.0	0.0	1.0	0.0	0.0

Framework versions

• Transformers 4.51.3
• Pytorch 2.6.0+cu124
• Datasets 3.5.0
• Tokenizers 0.21.1