ARC-Challenge_Llama-3.2-1B-wgzurb4i

This model is a fine-tuned version of meta-llama/Llama-3.2-1B on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.9193
• Model Preparation Time: 0.0058
• Mdl: 827.9179
• Accumulated Loss: 573.8690
• Correct Preds: 85.0
• Total Preds: 299.0
• Accuracy: 0.2843
• Correct Gen Preds: 1.0
• Gen Accuracy: 0.0033
• Correct Gen Preds 32: 0.0
• Correct Preds 32: 23.0
• Total Labels 32: 64.0
• Accuracy 32: 0.3594
• Gen Accuracy 32: 0.0
• Correct Gen Preds 33: 0.0
• Correct Preds 33: 48.0
• Total Labels 33: 73.0
• Accuracy 33: 0.6575
• Gen Accuracy 33: 0.0
• Correct Gen Preds 34: 0.0
• Correct Preds 34: 1.0
• Total Labels 34: 78.0
• Accuracy 34: 0.0128
• Gen Accuracy 34: 0.0
• Correct Gen Preds 35: 1.0
• Correct Preds 35: 13.0
• Total Labels 35: 83.0
• Accuracy 35: 0.1566
• Gen Accuracy 35: 0.0120
• Correct Gen Preds 36: 0.0
• Correct Preds 36: 0.0
• Total Labels 36: 1.0
• Accuracy 36: 0.0
• Gen Accuracy 36: 0.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 112
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.01
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Model Preparation Time	Mdl	Accumulated Loss	Correct Preds	Total Preds	Accuracy	Correct Gen Preds	Gen Accuracy	Correct Gen Preds 32	Correct Preds 32	Total Labels 32	Accuracy 32	Gen Accuracy 32	Correct Gen Preds 33	Correct Preds 33	Total Labels 33	Accuracy 33	Gen Accuracy 33	Correct Gen Preds 34	Correct Preds 34	Total Labels 34	Accuracy 34	Gen Accuracy 34	Correct Gen Preds 35	Correct Preds 35	Total Labels 35	Accuracy 35	Gen Accuracy 35	Correct Gen Preds 36	Correct Preds 36	Total Labels 36	Accuracy 36	Gen Accuracy 36
No log	0	0	1.6389	0.0058	706.9523	490.0220	66.0	299.0	0.2207	66.0	0.2207	62.0	62.0	64.0	0.9688	0.9688	0.0	0.0	73.0	0.0	0.0	4.0	4.0	78.0	0.0513	0.0513	0.0	0.0	83.0	0.0	0.0	0.0	0.0	1.0	0.0	0.0
1.7999	1.0	1	1.6389	0.0058	706.9523	490.0220	66.0	299.0	0.2207	66.0	0.2207	62.0	62.0	64.0	0.9688	0.9688	0.0	0.0	73.0	0.0	0.0	4.0	4.0	78.0	0.0513	0.0513	0.0	0.0	83.0	0.0	0.0	0.0	0.0	1.0	0.0	0.0
1.8225	2.0	2	2.6831	0.0058	1157.4179	802.2610	73.0	299.0	0.2441	73.0	0.2441	0.0	0.0	64.0	0.0	0.0	73.0	73.0	73.0	1.0	1.0	0.0	0.0	78.0	0.0	0.0	0.0	0.0	83.0	0.0	0.0	0.0	0.0	1.0	0.0	0.0
1.3461	3.0	3	1.9193	0.0058	827.9179	573.8690	85.0	299.0	0.2843	1.0	0.0033	0.0	23.0	64.0	0.3594	0.0	0.0	48.0	73.0	0.6575	0.0	0.0	1.0	78.0	0.0128	0.0	1.0	13.0	83.0	0.1566	0.0120	0.0	0.0	1.0	0.0	0.0
0.9697	4.0	4	1.8682	0.0058	805.8738	558.5892	78.0	299.0	0.2609	68.0	0.2274	0.0	0.0	64.0	0.0	0.0	53.0	61.0	73.0	0.8356	0.7260	0.0	0.0	78.0	0.0	0.0	15.0	17.0	83.0	0.2048	0.1807	0.0	0.0	1.0	0.0	0.0
0.6039	5.0	5	2.2833	0.0058	984.9399	682.7083	74.0	299.0	0.2475	54.0	0.1806	0.0	0.0	64.0	0.0	0.0	53.0	73.0	73.0	1.0	0.7260	0.0	0.0	78.0	0.0	0.0	1.0	1.0	83.0	0.0120	0.0120	0.0	0.0	1.0	0.0	0.0
0.1602	6.0	6	2.6340	0.0058	1136.2166	787.5654	75.0	299.0	0.2508	27.0	0.0903	0.0	0.0	64.0	0.0	0.0	25.0	71.0	73.0	0.9726	0.3425	1.0	3.0	78.0	0.0385	0.0128	1.0	1.0	83.0	0.0120	0.0120	0.0	0.0	1.0	0.0	0.0
0.0187	7.0	7	2.9628	0.0058	1278.0297	885.8627	72.0	299.0	0.2408	17.0	0.0569	0.0	1.0	64.0	0.0156	0.0	13.0	64.0	73.0	0.8767	0.1781	3.0	4.0	78.0	0.0513	0.0385	1.0	3.0	83.0	0.0361	0.0120	0.0	0.0	1.0	0.0	0.0
0.0013	8.0	8	3.2575	0.0058	1405.1697	973.9894	72.0	299.0	0.2408	13.0	0.0435	0.0	1.0	64.0	0.0156	0.0	9.0	62.0	73.0	0.8493	0.1233	3.0	4.0	78.0	0.0513	0.0385	1.0	5.0	83.0	0.0602	0.0120	0.0	0.0	1.0	0.0	0.0
0.0002	9.0	9	3.4603	0.0058	1492.6757	1034.6439	72.0	299.0	0.2408	12.0	0.0401	0.0	1.0	64.0	0.0156	0.0	7.0	61.0	73.0	0.8356	0.0959	3.0	4.0	78.0	0.0513	0.0385	2.0	6.0	83.0	0.0723	0.0241	0.0	0.0	1.0	0.0	0.0
0.0001	10.0	10	3.6246	0.0058	1563.5373	1083.7615	73.0	299.0	0.2441	13.0	0.0435	0.0	1.0	64.0	0.0156	0.0	8.0	60.0	73.0	0.8219	0.1096	3.0	5.0	78.0	0.0641	0.0385	2.0	7.0	83.0	0.0843	0.0241	0.0	0.0	1.0	0.0	0.0
0.0	11.0	11	3.7564	0.0058	1620.3818	1123.1631	74.0	299.0	0.2475	12.0	0.0401	0.0	1.0	64.0	0.0156	0.0	8.0	58.0	73.0	0.7945	0.1096	3.0	7.0	78.0	0.0897	0.0385	1.0	8.0	83.0	0.0964	0.0120	0.0	0.0	1.0	0.0	0.0
0.0	12.0	12	3.8610	0.0058	1665.5082	1154.4423	72.0	299.0	0.2408	11.0	0.0368	0.0	1.0	64.0	0.0156	0.0	7.0	56.0	73.0	0.7671	0.0959	3.0	7.0	78.0	0.0897	0.0385	1.0	8.0	83.0	0.0964	0.0120	0.0	0.0	1.0	0.0	0.0
0.0	13.0	13	3.9492	0.0058	1703.5624	1180.8195	72.0	299.0	0.2408	11.0	0.0368	0.0	1.0	64.0	0.0156	0.0	7.0	56.0	73.0	0.7671	0.0959	3.0	7.0	78.0	0.0897	0.0385	1.0	8.0	83.0	0.0964	0.0120	0.0	0.0	1.0	0.0	0.0
0.0	14.0	14	4.0205	0.0058	1734.2980	1202.1238	72.0	299.0	0.2408	11.0	0.0368	0.0	1.0	64.0	0.0156	0.0	7.0	56.0	73.0	0.7671	0.0959	3.0	7.0	78.0	0.0897	0.0385	1.0	8.0	83.0	0.0964	0.0120	0.0	0.0	1.0	0.0	0.0
0.0	15.0	15	4.0650	0.0058	1753.4997	1215.4334	72.0	299.0	0.2408	11.0	0.0368	0.0	1.0	64.0	0.0156	0.0	7.0	56.0	73.0	0.7671	0.0959	3.0	7.0	78.0	0.0897	0.0385	1.0	8.0	83.0	0.0964	0.0120	0.0	0.0	1.0	0.0	0.0
0.0	16.0	16	4.1048	0.0058	1770.6704	1227.3352	74.0	299.0	0.2475	11.0	0.0368	0.0	1.0	64.0	0.0156	0.0	7.0	56.0	73.0	0.7671	0.0959	3.0	9.0	78.0	0.1154	0.0385	1.0	8.0	83.0	0.0964	0.0120	0.0	0.0	1.0	0.0	0.0
0.0	17.0	17	4.1270	0.0058	1780.2283	1233.9602	71.0	299.0	0.2375	11.0	0.0368	0.0	1.0	64.0	0.0156	0.0	7.0	54.0	73.0	0.7397	0.0959	3.0	8.0	78.0	0.1026	0.0385	1.0	8.0	83.0	0.0964	0.0120	0.0	0.0	1.0	0.0	0.0
0.0	18.0	18	4.1560	0.0058	1792.7773	1242.6585	72.0	299.0	0.2408	11.0	0.0368	0.0	1.0	64.0	0.0156	0.0	7.0	54.0	73.0	0.7397	0.0959	3.0	9.0	78.0	0.1154	0.0385	1.0	8.0	83.0	0.0964	0.0120	0.0	0.0	1.0	0.0	0.0
0.0	19.0	19	4.1837	0.0058	1804.7127	1250.9315	70.0	299.0	0.2341	11.0	0.0368	0.0	1.0	64.0	0.0156	0.0	7.0	53.0	73.0	0.7260	0.0959	3.0	8.0	78.0	0.1026	0.0385	1.0	8.0	83.0	0.0964	0.0120	0.0	0.0	1.0	0.0	0.0
0.0	20.0	20	4.2006	0.0058	1811.9983	1255.9815	72.0	299.0	0.2408	11.0	0.0368	0.0	1.0	64.0	0.0156	0.0	7.0	54.0	73.0	0.7397	0.0959	3.0	9.0	78.0	0.1154	0.0385	1.0	8.0	83.0	0.0964	0.0120	0.0	0.0	1.0	0.0	0.0
0.0	21.0	21	4.2145	0.0058	1818.0010	1260.1423	71.0	299.0	0.2375	11.0	0.0368	0.0	1.0	64.0	0.0156	0.0	7.0	53.0	73.0	0.7260	0.0959	3.0	9.0	78.0	0.1154	0.0385	1.0	8.0	83.0	0.0964	0.0120	0.0	0.0	1.0	0.0	0.0
0.0	22.0	22	4.2290	0.0058	1824.2639	1264.4834	71.0	299.0	0.2375	11.0	0.0368	0.0	1.0	64.0	0.0156	0.0	7.0	53.0	73.0	0.7260	0.0959	3.0	9.0	78.0	0.1154	0.0385	1.0	8.0	83.0	0.0964	0.0120	0.0	0.0	1.0	0.0	0.0
0.0	23.0	23	4.2366	0.0058	1827.5084	1266.7323	71.0	299.0	0.2375	11.0	0.0368	0.0	1.0	64.0	0.0156	0.0	7.0	53.0	73.0	0.7260	0.0959	3.0	9.0	78.0	0.1154	0.0385	1.0	8.0	83.0	0.0964	0.0120	0.0	0.0	1.0	0.0	0.0
0.0	24.0	24	4.2348	0.0058	1826.7551	1266.2101	70.0	299.0	0.2341	11.0	0.0368	0.0	1.0	64.0	0.0156	0.0	7.0	52.0	73.0	0.7123	0.0959	3.0	9.0	78.0	0.1154	0.0385	1.0	8.0	83.0	0.0964	0.0120	0.0	0.0	1.0	0.0	0.0
0.0	25.0	25	4.2429	0.0058	1830.2455	1268.6295	70.0	299.0	0.2341	11.0	0.0368	0.0	1.0	64.0	0.0156	0.0	7.0	52.0	73.0	0.7123	0.0959	3.0	9.0	78.0	0.1154	0.0385	1.0	8.0	83.0	0.0964	0.0120	0.0	0.0	1.0	0.0	0.0
0.0	26.0	26	4.2432	0.0058	1830.3748	1268.7191	71.0	299.0	0.2375	11.0	0.0368	0.0	1.0	64.0	0.0156	0.0	7.0	53.0	73.0	0.7260	0.0959	3.0	9.0	78.0	0.1154	0.0385	1.0	8.0	83.0	0.0964	0.0120	0.0	0.0	1.0	0.0	0.0
0.0	27.0	27	4.2533	0.0058	1834.7450	1271.7483	71.0	299.0	0.2375	11.0	0.0368	0.0	1.0	64.0	0.0156	0.0	7.0	53.0	73.0	0.7260	0.0959	3.0	9.0	78.0	0.1154	0.0385	1.0	8.0	83.0	0.0964	0.0120	0.0	0.0	1.0	0.0	0.0
0.0	28.0	28	4.2639	0.0058	1839.2829	1274.8938	71.0	299.0	0.2375	11.0	0.0368	0.0	1.0	64.0	0.0156	0.0	7.0	53.0	73.0	0.7260	0.0959	3.0	9.0	78.0	0.1154	0.0385	1.0	8.0	83.0	0.0964	0.0120	0.0	0.0	1.0	0.0	0.0
0.0	29.0	29	4.2638	0.0058	1839.2620	1274.8792	70.0	299.0	0.2341	11.0	0.0368	0.0	1.0	64.0	0.0156	0.0	7.0	52.0	73.0	0.7123	0.0959	3.0	9.0	78.0	0.1154	0.0385	1.0	8.0	83.0	0.0964	0.0120	0.0	0.0	1.0	0.0	0.0
0.0	30.0	30	4.2640	0.0058	1839.3466	1274.9379	71.0	299.0	0.2375	11.0	0.0368	0.0	2.0	64.0	0.0312	0.0	7.0	52.0	73.0	0.7123	0.0959	3.0	9.0	78.0	0.1154	0.0385	1.0	8.0	83.0	0.0964	0.0120	0.0	0.0	1.0	0.0	0.0
0.0	31.0	31	4.2660	0.0058	1840.1879	1275.5211	71.0	299.0	0.2375	11.0	0.0368	0.0	2.0	64.0	0.0312	0.0	7.0	52.0	73.0	0.7123	0.0959	3.0	9.0	78.0	0.1154	0.0385	1.0	8.0	83.0	0.0964	0.0120	0.0	0.0	1.0	0.0	0.0
0.0	32.0	32	4.2655	0.0058	1839.9913	1275.3848	70.0	299.0	0.2341	11.0	0.0368	0.0	1.0	64.0	0.0156	0.0	7.0	52.0	73.0	0.7123	0.0959	3.0	9.0	78.0	0.1154	0.0385	1.0	8.0	83.0	0.0964	0.0120	0.0	0.0	1.0	0.0	0.0
0.0	33.0	33	4.2671	0.0058	1840.6805	1275.8625	71.0	299.0	0.2375	11.0	0.0368	0.0	2.0	64.0	0.0312	0.0	7.0	52.0	73.0	0.7123	0.0959	3.0	9.0	78.0	0.1154	0.0385	1.0	8.0	83.0	0.0964	0.0120	0.0	0.0	1.0	0.0	0.0

Framework versions

• Transformers 4.51.3
• Pytorch 2.6.0+cu124
• Datasets 3.5.0
• Tokenizers 0.21.1