ARC-Challenge_Llama-3.2-1B-rx87l0zg

This model is a fine-tuned version of meta-llama/Llama-3.2-1B on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 4.7104
• Model Preparation Time: 0.0073
• Mdl: 2031.9081
• Accumulated Loss: 1408.4113
• Correct Preds: 102.0
• Total Preds: 299.0
• Accuracy: 0.3411
• Correct Gen Preds: 62.0
• Gen Accuracy: 0.2074
• Correct Gen Preds 32: 7.0
• Correct Preds 32: 18.0
• Total Labels 32: 64.0
• Accuracy 32: 0.2812
• Gen Accuracy 32: 0.1094
• Correct Gen Preds 33: 27.0
• Correct Preds 33: 46.0
• Total Labels 33: 73.0
• Accuracy 33: 0.6301
• Gen Accuracy 33: 0.3699
• Correct Gen Preds 34: 19.0
• Correct Preds 34: 27.0
• Total Labels 34: 78.0
• Accuracy 34: 0.3462
• Gen Accuracy 34: 0.2436
• Correct Gen Preds 35: 9.0
• Correct Preds 35: 11.0
• Total Labels 35: 83.0
• Accuracy 35: 0.1325
• Gen Accuracy 35: 0.1084
• Correct Gen Preds 36: 0.0
• Correct Preds 36: 0.0
• Total Labels 36: 1.0
• Accuracy 36: 0.0
• Gen Accuracy 36: 0.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 112
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.01
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Model Preparation Time	Mdl	Accumulated Loss	Correct Preds	Total Preds	Accuracy	Correct Gen Preds	Gen Accuracy	Correct Gen Preds 32	Correct Preds 32	Total Labels 32	Accuracy 32	Gen Accuracy 32	Correct Gen Preds 33	Correct Preds 33	Total Labels 33	Accuracy 33	Gen Accuracy 33	Correct Gen Preds 34	Correct Preds 34	Total Labels 34	Accuracy 34	Gen Accuracy 34	Correct Gen Preds 35	Correct Preds 35	Total Labels 35	Accuracy 35	Gen Accuracy 35	Correct Gen Preds 36	Correct Preds 36	Total Labels 36	Accuracy 36	Gen Accuracy 36
No log	0	0	1.6389	0.0073	706.9523	490.0220	66.0	299.0	0.2207	66.0	0.2207	62.0	62.0	64.0	0.9688	0.9688	0.0	0.0	73.0	0.0	0.0	4.0	4.0	78.0	0.0513	0.0513	0.0	0.0	83.0	0.0	0.0	0.0	0.0	1.0	0.0	0.0
1.6964	1.0	1	1.6389	0.0073	706.9523	490.0220	66.0	299.0	0.2207	66.0	0.2207	62.0	62.0	64.0	0.9688	0.9688	0.0	0.0	73.0	0.0	0.0	4.0	4.0	78.0	0.0513	0.0513	0.0	0.0	83.0	0.0	0.0	0.0	0.0	1.0	0.0	0.0
1.6964	2.0	2	2.1206	0.0073	914.7418	634.0507	73.0	299.0	0.2441	73.0	0.2441	0.0	0.0	64.0	0.0	0.0	72.0	72.0	73.0	0.9863	0.9863	1.0	1.0	78.0	0.0128	0.0128	0.0	0.0	83.0	0.0	0.0	0.0	0.0	1.0	0.0	0.0
1.8796	3.0	3	1.3938	0.0073	601.2525	416.7565	73.0	299.0	0.2441	73.0	0.2441	0.0	0.0	64.0	0.0	0.0	71.0	71.0	73.0	0.9726	0.9726	0.0	0.0	78.0	0.0	0.0	2.0	2.0	83.0	0.0241	0.0241	0.0	0.0	1.0	0.0	0.0
1.267	4.0	4	1.7835	0.0073	769.3428	533.2678	74.0	299.0	0.2475	74.0	0.2475	7.0	7.0	64.0	0.1094	0.1094	67.0	67.0	73.0	0.9178	0.9178	0.0	0.0	78.0	0.0	0.0	0.0	0.0	83.0	0.0	0.0	0.0	0.0	1.0	0.0	0.0
1.0678	5.0	5	1.7931	0.0073	773.4821	536.1369	80.0	299.0	0.2676	80.0	0.2676	15.0	15.0	64.0	0.2344	0.2344	56.0	56.0	73.0	0.7671	0.7671	8.0	8.0	78.0	0.1026	0.1026	1.0	1.0	83.0	0.0120	0.0120	0.0	0.0	1.0	0.0	0.0
0.5476	6.0	6	2.2998	0.0073	992.0668	687.6483	80.0	299.0	0.2676	64.0	0.2140	18.0	31.0	64.0	0.4844	0.2812	22.0	25.0	73.0	0.3425	0.3014	8.0	8.0	78.0	0.1026	0.1026	16.0	16.0	83.0	0.1928	0.1928	0.0	0.0	1.0	0.0	0.0
0.1661	7.0	7	2.6623	0.0073	1148.4055	796.0140	87.0	299.0	0.2910	39.0	0.1304	3.0	15.0	64.0	0.2344	0.0469	19.0	53.0	73.0	0.7260	0.2603	8.0	10.0	78.0	0.1282	0.1026	9.0	9.0	83.0	0.1084	0.1084	0.0	0.0	1.0	0.0	0.0
0.0151	8.0	8	3.6879	0.0073	1590.8183	1102.6712	95.0	299.0	0.3177	51.0	0.1706	5.0	19.0	64.0	0.2969	0.0781	25.0	50.0	73.0	0.6849	0.3425	11.0	14.0	78.0	0.1795	0.1410	10.0	12.0	83.0	0.1446	0.1205	0.0	0.0	1.0	0.0	0.0
0.0005	9.0	9	4.7104	0.0073	2031.9081	1408.4113	102.0	299.0	0.3411	62.0	0.2074	7.0	18.0	64.0	0.2812	0.1094	27.0	46.0	73.0	0.6301	0.3699	19.0	27.0	78.0	0.3462	0.2436	9.0	11.0	83.0	0.1325	0.1084	0.0	0.0	1.0	0.0	0.0
0.0	10.0	10	5.5714	0.0073	2403.3181	1665.8531	98.0	299.0	0.3278	64.0	0.2140	5.0	15.0	64.0	0.2344	0.0781	28.0	42.0	73.0	0.5753	0.3836	24.0	33.0	78.0	0.4231	0.3077	7.0	8.0	83.0	0.0964	0.0843	0.0	0.0	1.0	0.0	0.0
0.0	11.0	11	6.2048	0.0073	2676.5357	1855.2332	99.0	299.0	0.3311	71.0	0.2375	5.0	15.0	64.0	0.2344	0.0781	29.0	40.0	73.0	0.5479	0.3973	29.0	35.0	78.0	0.4487	0.3718	8.0	9.0	83.0	0.1084	0.0964	0.0	0.0	1.0	0.0	0.0
0.0	12.0	12	6.6923	0.0073	2886.8300	2000.9981	98.0	299.0	0.3278	74.0	0.2475	5.0	14.0	64.0	0.2188	0.0781	30.0	40.0	73.0	0.5479	0.4110	33.0	37.0	78.0	0.4744	0.4231	6.0	7.0	83.0	0.0843	0.0723	0.0	0.0	1.0	0.0	0.0
0.0	13.0	13	7.1236	0.0073	3072.8734	2129.9536	100.0	299.0	0.3344	77.0	0.2575	5.0	14.0	64.0	0.2188	0.0781	29.0	36.0	73.0	0.4932	0.3973	35.0	42.0	78.0	0.5385	0.4487	8.0	8.0	83.0	0.0964	0.0964	0.0	0.0	1.0	0.0	0.0
0.0	14.0	14	7.4788	0.0073	3226.1112	2236.1699	98.0	299.0	0.3278	78.0	0.2609	5.0	13.0	64.0	0.2031	0.0781	31.0	36.0	73.0	0.4932	0.4247	36.0	43.0	78.0	0.5513	0.4615	6.0	6.0	83.0	0.0723	0.0723	0.0	0.0	1.0	0.0	0.0
0.0	15.0	15	7.7339	0.0073	3336.1252	2312.4258	98.0	299.0	0.3278	78.0	0.2609	5.0	12.0	64.0	0.1875	0.0781	31.0	36.0	73.0	0.4932	0.4247	37.0	45.0	78.0	0.5769	0.4744	5.0	5.0	83.0	0.0602	0.0602	0.0	0.0	1.0	0.0	0.0
0.0	16.0	16	7.9662	0.0073	3436.3575	2381.9015	100.0	299.0	0.3344	82.0	0.2742	5.0	12.0	64.0	0.1875	0.0781	32.0	37.0	73.0	0.5068	0.4384	39.0	45.0	78.0	0.5769	0.5	6.0	6.0	83.0	0.0723	0.0723	0.0	0.0	1.0	0.0	0.0
0.0	17.0	17	8.1410	0.0073	3511.7307	2434.1462	98.0	299.0	0.3278	81.0	0.2709	5.0	12.0	64.0	0.1875	0.0781	31.0	36.0	73.0	0.4932	0.4247	40.0	45.0	78.0	0.5769	0.5128	5.0	5.0	83.0	0.0602	0.0602	0.0	0.0	1.0	0.0	0.0
0.0	18.0	18	8.2545	0.0073	3560.7299	2468.1099	98.0	299.0	0.3278	79.0	0.2642	5.0	13.0	64.0	0.2031	0.0781	29.0	34.0	73.0	0.4658	0.3973	40.0	46.0	78.0	0.5897	0.5128	5.0	5.0	83.0	0.0602	0.0602	0.0	0.0	1.0	0.0	0.0
0.0	19.0	19	8.3711	0.0073	3610.9981	2502.9531	98.0	299.0	0.3278	81.0	0.2709	5.0	12.0	64.0	0.1875	0.0781	31.0	36.0	73.0	0.4932	0.4247	40.0	45.0	78.0	0.5769	0.5128	5.0	5.0	83.0	0.0602	0.0602	0.0	0.0	1.0	0.0	0.0
0.0	20.0	20	8.4942	0.0073	3664.0992	2539.7600	98.0	299.0	0.3278	79.0	0.2642	5.0	13.0	64.0	0.2031	0.0781	30.0	35.0	73.0	0.4795	0.4110	40.0	46.0	78.0	0.5897	0.5128	4.0	4.0	83.0	0.0482	0.0482	0.0	0.0	1.0	0.0	0.0
0.0	21.0	21	8.5955	0.0073	3707.7867	2570.0419	97.0	299.0	0.3244	79.0	0.2642	5.0	13.0	64.0	0.2031	0.0781	29.0	34.0	73.0	0.4658	0.3973	40.0	45.0	78.0	0.5769	0.5128	5.0	5.0	83.0	0.0602	0.0602	0.0	0.0	1.0	0.0	0.0
0.0	22.0	22	8.6160	0.0073	3716.6263	2576.1691	99.0	299.0	0.3311	80.0	0.2676	5.0	13.0	64.0	0.2031	0.0781	30.0	35.0	73.0	0.4795	0.4110	40.0	46.0	78.0	0.5897	0.5128	5.0	5.0	83.0	0.0602	0.0602	0.0	0.0	1.0	0.0	0.0
0.0	23.0	23	8.6760	0.0073	3742.5240	2594.1199	97.0	299.0	0.3244	80.0	0.2676	5.0	13.0	64.0	0.2031	0.0781	29.0	33.0	73.0	0.4521	0.3973	41.0	46.0	78.0	0.5897	0.5256	5.0	5.0	83.0	0.0602	0.0602	0.0	0.0	1.0	0.0	0.0
0.0	24.0	24	8.6943	0.0073	3750.4229	2599.5951	98.0	299.0	0.3278	79.0	0.2642	5.0	13.0	64.0	0.2031	0.0781	29.0	33.0	73.0	0.4521	0.3973	40.0	47.0	78.0	0.6026	0.5128	5.0	5.0	83.0	0.0602	0.0602	0.0	0.0	1.0	0.0	0.0
0.0	25.0	25	8.7113	0.0073	3757.7507	2604.6743	99.0	299.0	0.3311	78.0	0.2609	5.0	13.0	64.0	0.2031	0.0781	29.0	34.0	73.0	0.4658	0.3973	40.0	47.0	78.0	0.6026	0.5128	4.0	5.0	83.0	0.0602	0.0482	0.0	0.0	1.0	0.0	0.0
0.0	26.0	26	8.7311	0.0073	3766.2962	2610.5976	97.0	299.0	0.3244	78.0	0.2609	4.0	12.0	64.0	0.1875	0.0625	29.0	34.0	73.0	0.4658	0.3973	41.0	47.0	78.0	0.6026	0.5256	4.0	4.0	83.0	0.0482	0.0482	0.0	0.0	1.0	0.0	0.0
0.0	27.0	27	8.7594	0.0073	3778.4903	2619.0499	98.0	299.0	0.3278	80.0	0.2676	5.0	13.0	64.0	0.2031	0.0781	29.0	34.0	73.0	0.4658	0.3973	41.0	46.0	78.0	0.5897	0.5256	5.0	5.0	83.0	0.0602	0.0602	0.0	0.0	1.0	0.0	0.0
0.0	28.0	28	8.7681	0.0073	3782.2393	2621.6485	97.0	299.0	0.3244	80.0	0.2676	5.0	12.0	64.0	0.1875	0.0781	29.0	34.0	73.0	0.4658	0.3973	41.0	46.0	78.0	0.5897	0.5256	5.0	5.0	83.0	0.0602	0.0602	0.0	0.0	1.0	0.0	0.0
0.0	29.0	29	8.8233	0.0073	3806.0822	2638.1752	97.0	299.0	0.3244	82.0	0.2742	4.0	12.0	64.0	0.1875	0.0625	29.0	34.0	73.0	0.4658	0.3973	44.0	46.0	78.0	0.5897	0.5641	5.0	5.0	83.0	0.0602	0.0602	0.0	0.0	1.0	0.0	0.0
0.0	30.0	30	8.8120	0.0073	3801.1761	2634.7745	96.0	299.0	0.3211	78.0	0.2609	3.0	11.0	64.0	0.1719	0.0469	29.0	34.0	73.0	0.4658	0.3973	41.0	46.0	78.0	0.5897	0.5256	5.0	5.0	83.0	0.0602	0.0602	0.0	0.0	1.0	0.0	0.0
0.0	31.0	31	8.8427	0.0073	3814.4253	2643.9581	96.0	299.0	0.3211	80.0	0.2676	5.0	12.0	64.0	0.1875	0.0781	29.0	34.0	73.0	0.4658	0.3973	42.0	46.0	78.0	0.5897	0.5385	4.0	4.0	83.0	0.0482	0.0482	0.0	0.0	1.0	0.0	0.0
0.0	32.0	32	8.7954	0.0073	3794.0408	2629.8287	98.0	299.0	0.3278	82.0	0.2742	6.0	13.0	64.0	0.2031	0.0938	29.0	33.0	73.0	0.4521	0.3973	42.0	47.0	78.0	0.6026	0.5385	5.0	5.0	83.0	0.0602	0.0602	0.0	0.0	1.0	0.0	0.0
0.0	33.0	33	8.8254	0.0073	3806.9690	2638.7898	100.0	299.0	0.3344	81.0	0.2709	5.0	13.0	64.0	0.2031	0.0781	29.0	34.0	73.0	0.4658	0.3973	42.0	48.0	78.0	0.6154	0.5385	5.0	5.0	83.0	0.0602	0.0602	0.0	0.0	1.0	0.0	0.0
0.0	34.0	34	8.8195	0.0073	3804.4106	2637.0165	96.0	299.0	0.3211	78.0	0.2609	3.0	11.0	64.0	0.1719	0.0469	29.0	34.0	73.0	0.4658	0.3973	41.0	46.0	78.0	0.5897	0.5256	5.0	5.0	83.0	0.0602	0.0602	0.0	0.0	1.0	0.0	0.0
0.0	35.0	35	8.8524	0.0073	3818.6222	2646.8672	96.0	299.0	0.3211	80.0	0.2676	5.0	12.0	64.0	0.1875	0.0781	29.0	34.0	73.0	0.4658	0.3973	41.0	45.0	78.0	0.5769	0.5256	5.0	5.0	83.0	0.0602	0.0602	0.0	0.0	1.0	0.0	0.0
0.0	36.0	36	8.8625	0.0073	3822.9959	2649.8988	99.0	299.0	0.3311	81.0	0.2709	6.0	14.0	64.0	0.2188	0.0938	29.0	34.0	73.0	0.4658	0.3973	41.0	46.0	78.0	0.5897	0.5256	5.0	5.0	83.0	0.0602	0.0602	0.0	0.0	1.0	0.0	0.0
0.0	37.0	37	8.8324	0.0073	3809.9963	2640.8882	97.0	299.0	0.3244	80.0	0.2676	4.0	12.0	64.0	0.1875	0.0625	29.0	34.0	73.0	0.4658	0.3973	42.0	46.0	78.0	0.5897	0.5385	5.0	5.0	83.0	0.0602	0.0602	0.0	0.0	1.0	0.0	0.0
0.0	38.0	38	8.8095	0.0073	3800.1094	2634.0351	98.0	299.0	0.3278	83.0	0.2776	5.0	12.0	64.0	0.1875	0.0781	29.0	34.0	73.0	0.4658	0.3973	44.0	47.0	78.0	0.6026	0.5641	5.0	5.0	83.0	0.0602	0.0602	0.0	0.0	1.0	0.0	0.0
0.0	39.0	39	8.8868	0.0073	3833.4428	2657.1401	96.0	299.0	0.3211	79.0	0.2642	4.0	12.0	64.0	0.1875	0.0625	29.0	34.0	73.0	0.4658	0.3973	41.0	45.0	78.0	0.5769	0.5256	5.0	5.0	83.0	0.0602	0.0602	0.0	0.0	1.0	0.0	0.0

Framework versions

• Transformers 4.51.3
• Pytorch 2.6.0+cu124
• Datasets 3.5.0
• Tokenizers 0.21.1