ARC-Challenge_Llama-3.2-1B-840xiqfg

This model is a fine-tuned version of meta-llama/Llama-3.2-1B on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 7.1244
• Model Preparation Time: 0.0058
• Mdl: 3073.2087
• Accumulated Loss: 2130.1860
• Correct Preds: 104.0
• Total Preds: 299.0
• Accuracy: 0.3478
• Correct Gen Preds: 86.0
• Gen Accuracy: 0.2876
• Correct Gen Preds 32: 2.0
• Correct Preds 32: 8.0
• Total Labels 32: 64.0
• Accuracy 32: 0.125
• Gen Accuracy 32: 0.0312
• Correct Gen Preds 33: 33.0
• Correct Preds 33: 34.0
• Total Labels 33: 73.0
• Accuracy 33: 0.4658
• Gen Accuracy 33: 0.4521
• Correct Gen Preds 34: 39.0
• Correct Preds 34: 44.0
• Total Labels 34: 78.0
• Accuracy 34: 0.5641
• Gen Accuracy 34: 0.5
• Correct Gen Preds 35: 12.0
• Correct Preds 35: 17.0
• Total Labels 35: 83.0
• Accuracy 35: 0.2048
• Gen Accuracy 35: 0.1446
• Correct Gen Preds 36: 0.0
• Correct Preds 36: 1.0
• Total Labels 36: 1.0
• Accuracy 36: 1.0
• Gen Accuracy 36: 0.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 112
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.01
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Model Preparation Time	Mdl	Accumulated Loss	Correct Preds	Total Preds	Accuracy	Correct Gen Preds	Gen Accuracy	Correct Gen Preds 32	Correct Preds 32	Total Labels 32	Accuracy 32	Gen Accuracy 32	Correct Gen Preds 33	Correct Preds 33	Total Labels 33	Accuracy 33	Gen Accuracy 33	Correct Gen Preds 34	Correct Preds 34	Total Labels 34	Accuracy 34	Gen Accuracy 34	Correct Gen Preds 35	Correct Preds 35	Total Labels 35	Accuracy 35	Gen Accuracy 35	Correct Gen Preds 36	Correct Preds 36	Total Labels 36	Accuracy 36	Gen Accuracy 36
No log	0	0	1.6389	0.0058	706.9523	490.0220	66.0	299.0	0.2207	66.0	0.2207	62.0	62.0	64.0	0.9688	0.9688	0.0	0.0	73.0	0.0	0.0	4.0	4.0	78.0	0.0513	0.0513	0.0	0.0	83.0	0.0	0.0	0.0	0.0	1.0	0.0	0.0
1.6898	1.0	1	1.6389	0.0058	706.9523	490.0220	66.0	299.0	0.2207	66.0	0.2207	62.0	62.0	64.0	0.9688	0.9688	0.0	0.0	73.0	0.0	0.0	4.0	4.0	78.0	0.0513	0.0513	0.0	0.0	83.0	0.0	0.0	0.0	0.0	1.0	0.0	0.0
1.6898	2.0	2	2.1720	0.0058	936.9069	649.4144	74.0	299.0	0.2475	74.0	0.2475	0.0	0.0	64.0	0.0	0.0	72.0	72.0	73.0	0.9863	0.9863	2.0	2.0	78.0	0.0256	0.0256	0.0	0.0	83.0	0.0	0.0	0.0	0.0	1.0	0.0	0.0
2.0035	3.0	3	1.3938	0.0058	601.2288	416.7400	87.0	299.0	0.2910	87.0	0.2910	0.0	0.0	64.0	0.0	0.0	67.0	67.0	73.0	0.9178	0.9178	0.0	0.0	78.0	0.0	0.0	20.0	20.0	83.0	0.2410	0.2410	0.0	0.0	1.0	0.0	0.0
1.3004	4.0	4	1.7271	0.0058	745.0113	516.4025	71.0	299.0	0.2375	71.0	0.2375	23.0	23.0	64.0	0.3594	0.3594	48.0	48.0	73.0	0.6575	0.6575	0.0	0.0	78.0	0.0	0.0	0.0	0.0	83.0	0.0	0.0	0.0	0.0	1.0	0.0	0.0
1.1945	5.0	5	1.5970	0.0058	688.8756	477.4922	83.0	299.0	0.2776	83.0	0.2776	14.0	14.0	64.0	0.2188	0.2188	50.0	50.0	73.0	0.6849	0.6849	18.0	18.0	78.0	0.2308	0.2308	1.0	1.0	83.0	0.0120	0.0120	0.0	0.0	1.0	0.0	0.0
0.756	6.0	6	1.8474	0.0058	796.9090	552.3752	79.0	299.0	0.2642	71.0	0.2375	12.0	19.0	64.0	0.2969	0.1875	31.0	31.0	73.0	0.4247	0.4247	11.0	12.0	78.0	0.1538	0.1410	17.0	17.0	83.0	0.2048	0.2048	0.0	0.0	1.0	0.0	0.0
0.2928	7.0	7	2.5100	0.0058	1082.7278	750.4897	97.0	299.0	0.3244	55.0	0.1839	2.0	20.0	64.0	0.3125	0.0312	27.0	46.0	73.0	0.6301	0.3699	13.0	17.0	78.0	0.2179	0.1667	13.0	14.0	83.0	0.1687	0.1566	0.0	0.0	1.0	0.0	0.0
0.0388	8.0	8	3.4579	0.0058	1491.6158	1033.9093	98.0	299.0	0.3278	60.0	0.2007	1.0	7.0	64.0	0.1094	0.0156	29.0	48.0	73.0	0.6575	0.3973	17.0	26.0	78.0	0.3333	0.2179	13.0	17.0	83.0	0.2048	0.1566	0.0	0.0	1.0	0.0	0.0
0.0013	9.0	9	4.8580	0.0058	2095.5588	1452.5307	95.0	299.0	0.3177	74.0	0.2475	1.0	7.0	64.0	0.1094	0.0156	32.0	36.0	73.0	0.4932	0.4384	27.0	33.0	78.0	0.4231	0.3462	14.0	19.0	83.0	0.2289	0.1687	0.0	0.0	1.0	0.0	0.0
0.0	10.0	10	6.0745	0.0058	2620.3151	1816.2640	98.0	299.0	0.3278	83.0	0.2776	2.0	7.0	64.0	0.1094	0.0312	32.0	33.0	73.0	0.4521	0.4384	35.0	41.0	78.0	0.5256	0.4487	14.0	17.0	83.0	0.2048	0.1687	0.0	0.0	1.0	0.0	0.0
0.0	11.0	11	7.1244	0.0058	3073.2087	2130.1860	104.0	299.0	0.3478	86.0	0.2876	2.0	8.0	64.0	0.125	0.0312	33.0	34.0	73.0	0.4658	0.4521	39.0	44.0	78.0	0.5641	0.5	12.0	17.0	83.0	0.2048	0.1446	0.0	1.0	1.0	1.0	0.0
0.0	12.0	12	7.9021	0.0058	3408.6810	2362.7176	100.0	299.0	0.3344	88.0	0.2943	3.0	7.0	64.0	0.1094	0.0469	31.0	31.0	73.0	0.4247	0.4247	40.0	45.0	78.0	0.5769	0.5128	13.0	16.0	83.0	0.1928	0.1566	1.0	1.0	1.0	1.0	1.0
0.0	13.0	13	8.4872	0.0058	3661.0915	2537.6752	100.0	299.0	0.3344	89.0	0.2977	3.0	6.0	64.0	0.0938	0.0469	29.0	30.0	73.0	0.4110	0.3973	44.0	46.0	78.0	0.5897	0.5641	12.0	17.0	83.0	0.2048	0.1446	1.0	1.0	1.0	1.0	1.0
0.0	14.0	14	8.8610	0.0058	3822.3422	2649.4457	98.0	299.0	0.3278	88.0	0.2943	3.0	6.0	64.0	0.0938	0.0469	29.0	30.0	73.0	0.4110	0.3973	45.0	46.0	78.0	0.5897	0.5769	10.0	15.0	83.0	0.1807	0.1205	1.0	1.0	1.0	1.0	1.0
0.0	15.0	15	9.1039	0.0058	3927.1210	2722.0729	103.0	299.0	0.3445	92.0	0.3077	3.0	6.0	64.0	0.0938	0.0469	31.0	32.0	73.0	0.4384	0.4247	47.0	49.0	78.0	0.6282	0.6026	10.0	15.0	83.0	0.1807	0.1205	1.0	1.0	1.0	1.0	1.0
0.0	16.0	16	9.3212	0.0058	4020.8412	2787.0347	101.0	299.0	0.3378	91.0	0.3043	3.0	6.0	64.0	0.0938	0.0469	31.0	32.0	73.0	0.4384	0.4247	47.0	48.0	78.0	0.6154	0.6026	9.0	14.0	83.0	0.1687	0.1084	1.0	1.0	1.0	1.0	1.0
0.0	17.0	17	9.4870	0.0058	4092.3579	2836.6063	99.0	299.0	0.3311	90.0	0.3010	3.0	5.0	64.0	0.0781	0.0469	28.0	29.0	73.0	0.3973	0.3836	48.0	49.0	78.0	0.6282	0.6154	10.0	15.0	83.0	0.1807	0.1205	1.0	1.0	1.0	1.0	1.0
0.0	18.0	18	9.6468	0.0058	4161.3095	2884.4000	97.0	299.0	0.3244	87.0	0.2910	2.0	5.0	64.0	0.0781	0.0312	28.0	29.0	73.0	0.3973	0.3836	47.0	48.0	78.0	0.6154	0.6026	9.0	14.0	83.0	0.1687	0.1084	1.0	1.0	1.0	1.0	1.0
0.0	19.0	19	9.7337	0.0058	4198.7998	2910.3862	99.0	299.0	0.3311	89.0	0.2977	3.0	5.0	64.0	0.0781	0.0469	27.0	28.0	73.0	0.3836	0.3699	48.0	49.0	78.0	0.6282	0.6154	10.0	16.0	83.0	0.1928	0.1205	1.0	1.0	1.0	1.0	1.0
0.0	20.0	20	9.8529	0.0058	4250.1953	2946.0109	99.0	299.0	0.3311	91.0	0.3043	4.0	5.0	64.0	0.0781	0.0625	28.0	29.0	73.0	0.3973	0.3836	49.0	50.0	78.0	0.6410	0.6282	9.0	14.0	83.0	0.1687	0.1084	1.0	1.0	1.0	1.0	1.0
0.0	21.0	21	9.9188	0.0058	4278.6280	2965.7189	100.0	299.0	0.3344	91.0	0.3043	4.0	5.0	64.0	0.0781	0.0625	28.0	29.0	73.0	0.3973	0.3836	49.0	50.0	78.0	0.6410	0.6282	9.0	15.0	83.0	0.1807	0.1084	1.0	1.0	1.0	1.0	1.0
0.0	22.0	22	9.8962	0.0058	4268.8615	2958.9493	99.0	299.0	0.3311	90.0	0.3010	3.0	4.0	64.0	0.0625	0.0469	29.0	30.0	73.0	0.4110	0.3973	49.0	50.0	78.0	0.6410	0.6282	8.0	14.0	83.0	0.1687	0.0964	1.0	1.0	1.0	1.0	1.0
0.0	23.0	23	9.9553	0.0058	4294.3882	2976.6431	98.0	299.0	0.3278	88.0	0.2943	2.0	4.0	64.0	0.0625	0.0312	29.0	30.0	73.0	0.4110	0.3973	49.0	50.0	78.0	0.6410	0.6282	7.0	13.0	83.0	0.1566	0.0843	1.0	1.0	1.0	1.0	1.0
0.0	24.0	24	10.0124	0.0058	4318.9876	2993.6941	99.0	299.0	0.3311	89.0	0.2977	3.0	4.0	64.0	0.0625	0.0469	28.0	30.0	73.0	0.4110	0.3836	49.0	50.0	78.0	0.6410	0.6282	8.0	14.0	83.0	0.1687	0.0964	1.0	1.0	1.0	1.0	1.0
0.0	25.0	25	10.0608	0.0058	4339.9027	3008.1913	102.0	299.0	0.3411	92.0	0.3077	4.0	5.0	64.0	0.0781	0.0625	29.0	31.0	73.0	0.4247	0.3973	50.0	51.0	78.0	0.6538	0.6410	8.0	14.0	83.0	0.1687	0.0964	1.0	1.0	1.0	1.0	1.0
0.0	26.0	26	10.1292	0.0058	4369.4067	3028.6420	99.0	299.0	0.3311	89.0	0.2977	3.0	4.0	64.0	0.0625	0.0469	28.0	30.0	73.0	0.4110	0.3836	49.0	50.0	78.0	0.6410	0.6282	8.0	14.0	83.0	0.1687	0.0964	1.0	1.0	1.0	1.0	1.0
0.0	27.0	27	10.1221	0.0058	4366.3251	3026.5059	100.0	299.0	0.3344	90.0	0.3010	3.0	4.0	64.0	0.0625	0.0469	29.0	31.0	73.0	0.4247	0.3973	49.0	50.0	78.0	0.6410	0.6282	8.0	14.0	83.0	0.1687	0.0964	1.0	1.0	1.0	1.0	1.0
0.0	28.0	28	10.1719	0.0058	4387.8296	3041.4117	102.0	299.0	0.3411	92.0	0.3077	3.0	4.0	64.0	0.0625	0.0469	29.0	31.0	73.0	0.4247	0.3973	50.0	51.0	78.0	0.6538	0.6410	9.0	15.0	83.0	0.1807	0.1084	1.0	1.0	1.0	1.0	1.0
0.0	29.0	29	10.1813	0.0058	4391.8675	3044.2106	98.0	299.0	0.3278	89.0	0.2977	3.0	4.0	64.0	0.0625	0.0469	28.0	30.0	73.0	0.4110	0.3836	49.0	50.0	78.0	0.6410	0.6282	8.0	13.0	83.0	0.1566	0.0964	1.0	1.0	1.0	1.0	1.0
0.0	30.0	30	10.2484	0.0058	4420.8028	3064.2670	98.0	299.0	0.3278	89.0	0.2977	3.0	4.0	64.0	0.0625	0.0469	28.0	30.0	73.0	0.4110	0.3836	49.0	50.0	78.0	0.6410	0.6282	8.0	13.0	83.0	0.1566	0.0964	1.0	1.0	1.0	1.0	1.0
0.0	31.0	31	10.2337	0.0058	4414.4781	3059.8831	100.0	299.0	0.3344	90.0	0.3010	3.0	4.0	64.0	0.0625	0.0469	28.0	30.0	73.0	0.4110	0.3836	50.0	51.0	78.0	0.6538	0.6410	8.0	14.0	83.0	0.1687	0.0964	1.0	1.0	1.0	1.0	1.0
0.0	32.0	32	10.2096	0.0058	4404.0563	3052.6592	99.0	299.0	0.3311	90.0	0.3010	3.0	4.0	64.0	0.0625	0.0469	28.0	30.0	73.0	0.4110	0.3836	50.0	51.0	78.0	0.6538	0.6410	8.0	13.0	83.0	0.1566	0.0964	1.0	1.0	1.0	1.0	1.0
0.0	33.0	33	10.1974	0.0058	4398.8077	3049.0212	98.0	299.0	0.3278	89.0	0.2977	3.0	4.0	64.0	0.0625	0.0469	28.0	30.0	73.0	0.4110	0.3836	49.0	50.0	78.0	0.6410	0.6282	8.0	13.0	83.0	0.1566	0.0964	1.0	1.0	1.0	1.0	1.0
0.0	34.0	34	10.1697	0.0058	4386.8795	3040.7532	101.0	299.0	0.3378	90.0	0.3010	3.0	4.0	64.0	0.0625	0.0469	28.0	30.0	73.0	0.4110	0.3836	50.0	51.0	78.0	0.6538	0.6410	8.0	15.0	83.0	0.1807	0.0964	1.0	1.0	1.0	1.0	1.0
0.0	35.0	35	10.2728	0.0058	4431.3466	3071.5754	98.0	299.0	0.3278	90.0	0.3010	3.0	4.0	64.0	0.0625	0.0469	28.0	30.0	73.0	0.4110	0.3836	50.0	50.0	78.0	0.6410	0.6410	8.0	13.0	83.0	0.1566	0.0964	1.0	1.0	1.0	1.0	1.0
0.0	36.0	36	10.1882	0.0058	4394.8573	3046.2830	102.0	299.0	0.3411	91.0	0.3043	3.0	4.0	64.0	0.0625	0.0469	29.0	31.0	73.0	0.4247	0.3973	50.0	52.0	78.0	0.6667	0.6410	8.0	14.0	83.0	0.1687	0.0964	1.0	1.0	1.0	1.0	1.0
0.0	37.0	37	10.1529	0.0058	4379.6087	3035.7134	101.0	299.0	0.3378	90.0	0.3010	3.0	4.0	64.0	0.0625	0.0469	28.0	30.0	73.0	0.4110	0.3836	50.0	52.0	78.0	0.6667	0.6410	8.0	14.0	83.0	0.1687	0.0964	1.0	1.0	1.0	1.0	1.0
0.0	38.0	38	10.2562	0.0058	4424.1953	3066.6185	99.0	299.0	0.3311	89.0	0.2977	3.0	4.0	64.0	0.0625	0.0469	28.0	30.0	73.0	0.4110	0.3836	49.0	50.0	78.0	0.6410	0.6282	8.0	14.0	83.0	0.1687	0.0964	1.0	1.0	1.0	1.0	1.0
0.0	39.0	39	10.2689	0.0058	4429.6724	3070.4149	100.0	299.0	0.3344	89.0	0.2977	3.0	4.0	64.0	0.0625	0.0469	28.0	30.0	73.0	0.4110	0.3836	49.0	51.0	78.0	0.6538	0.6282	8.0	14.0	83.0	0.1687	0.0964	1.0	1.0	1.0	1.0	1.0
0.0	40.0	40	10.2724	0.0058	4431.1454	3071.4359	99.0	299.0	0.3311	90.0	0.3010	3.0	4.0	64.0	0.0625	0.0469	28.0	30.0	73.0	0.4110	0.3836	49.0	50.0	78.0	0.6410	0.6282	9.0	14.0	83.0	0.1687	0.1084	1.0	1.0	1.0	1.0	1.0
0.0	41.0	41	10.2245	0.0058	4410.5007	3057.1261	101.0	299.0	0.3378	91.0	0.3043	3.0	4.0	64.0	0.0625	0.0469	29.0	31.0	73.0	0.4247	0.3973	50.0	51.0	78.0	0.6538	0.6410	8.0	14.0	83.0	0.1687	0.0964	1.0	1.0	1.0	1.0	1.0

Framework versions

• Transformers 4.51.3
• Pytorch 2.6.0+cu124
• Datasets 3.5.0
• Tokenizers 0.21.1