ARC-Challenge_Llama-3.2-1B-kh5qrc44

This model is a fine-tuned version of meta-llama/Llama-3.2-1B on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 7.1787
• Model Preparation Time: 0.006
• Mdl: 3096.6477
• Accumulated Loss: 2146.4326
• Correct Preds: 121.0
• Total Preds: 299.0
• Accuracy: 0.4047
• Correct Gen Preds: 121.0
• Gen Accuracy: 0.4047
• Correct Gen Preds 32: 19.0
• Correct Preds 32: 19.0
• Total Labels 32: 64.0
• Accuracy 32: 0.2969
• Gen Accuracy 32: 0.2969
• Correct Gen Preds 33: 20.0
• Correct Preds 33: 20.0
• Total Labels 33: 73.0
• Accuracy 33: 0.2740
• Gen Accuracy 33: 0.2740
• Correct Gen Preds 34: 46.0
• Correct Preds 34: 46.0
• Total Labels 34: 78.0
• Accuracy 34: 0.5897
• Gen Accuracy 34: 0.5897
• Correct Gen Preds 35: 36.0
• Correct Preds 35: 36.0
• Total Labels 35: 83.0
• Accuracy 35: 0.4337
• Gen Accuracy 35: 0.4337
• Correct Gen Preds 36: 0.0
• Correct Preds 36: 0.0
• Total Labels 36: 1.0
• Accuracy 36: 0.0
• Gen Accuracy 36: 0.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 112
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.01
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Model Preparation Time	Mdl	Accumulated Loss	Correct Preds	Total Preds	Accuracy	Correct Gen Preds	Gen Accuracy	Correct Gen Preds 32	Correct Preds 32	Total Labels 32	Accuracy 32	Gen Accuracy 32	Correct Gen Preds 33	Correct Preds 33	Total Labels 33	Accuracy 33	Gen Accuracy 33	Correct Gen Preds 34	Correct Preds 34	Total Labels 34	Accuracy 34	Gen Accuracy 34	Correct Gen Preds 35	Correct Preds 35	Total Labels 35	Accuracy 35	Gen Accuracy 35	Correct Gen Preds 36	Correct Preds 36	Total Labels 36	Accuracy 36	Gen Accuracy 36
No log	0	0	1.6389	0.006	706.9523	490.0220	66.0	299.0	0.2207	66.0	0.2207	62.0	62.0	64.0	0.9688	0.9688	0.0	0.0	73.0	0.0	0.0	4.0	4.0	78.0	0.0513	0.0513	0.0	0.0	83.0	0.0	0.0	0.0	0.0	1.0	0.0	0.0
1.6453	1.0	2	1.5379	0.006	663.3944	459.8300	83.0	299.0	0.2776	83.0	0.2776	0.0	0.0	64.0	0.0	0.0	48.0	48.0	73.0	0.6575	0.6575	28.0	28.0	78.0	0.3590	0.3590	7.0	7.0	83.0	0.0843	0.0843	0.0	0.0	1.0	0.0	0.0
1.3573	2.0	4	1.4484	0.006	624.7902	433.0716	64.0	299.0	0.2140	64.0	0.2140	64.0	64.0	64.0	1.0	1.0	0.0	0.0	73.0	0.0	0.0	0.0	0.0	78.0	0.0	0.0	0.0	0.0	83.0	0.0	0.0	0.0	0.0	1.0	0.0	0.0
0.842	3.0	6	2.1104	0.006	910.3498	631.0064	113.0	299.0	0.3779	105.0	0.3512	20.0	27.0	64.0	0.4219	0.3125	28.0	28.0	73.0	0.3836	0.3836	27.0	27.0	78.0	0.3462	0.3462	30.0	31.0	83.0	0.3735	0.3614	0.0	0.0	1.0	0.0	0.0
0.688	4.0	8	3.3900	0.006	1462.3426	1013.6187	106.0	299.0	0.3545	101.0	0.3378	25.0	30.0	64.0	0.4688	0.3906	42.0	42.0	73.0	0.5753	0.5753	25.0	25.0	78.0	0.3205	0.3205	9.0	9.0	83.0	0.1084	0.1084	0.0	0.0	1.0	0.0	0.0
0.0895	5.0	10	4.1995	0.006	1811.4999	1255.6360	109.0	299.0	0.3645	105.0	0.3512	9.0	12.0	64.0	0.1875	0.1406	40.0	41.0	73.0	0.5616	0.5479	41.0	41.0	78.0	0.5256	0.5256	15.0	15.0	83.0	0.1807	0.1807	0.0	0.0	1.0	0.0	0.0
0.0023	6.0	12	5.2346	0.006	2258.0383	1565.1529	116.0	299.0	0.3880	115.0	0.3846	17.0	18.0	64.0	0.2812	0.2656	28.0	28.0	73.0	0.3836	0.3836	45.0	45.0	78.0	0.5769	0.5769	25.0	25.0	83.0	0.3012	0.3012	0.0	0.0	1.0	0.0	0.0
0.1889	7.0	14	6.2403	0.006	2691.8421	1865.8427	114.0	299.0	0.3813	114.0	0.3813	19.0	19.0	64.0	0.2969	0.2969	24.0	24.0	73.0	0.3288	0.3288	45.0	45.0	78.0	0.5769	0.5769	26.0	26.0	83.0	0.3133	0.3133	0.0	0.0	1.0	0.0	0.0
0.0	8.0	16	6.8915	0.006	2972.7767	2060.5718	114.0	299.0	0.3813	113.0	0.3779	11.0	12.0	64.0	0.1875	0.1719	24.0	24.0	73.0	0.3288	0.3288	51.0	51.0	78.0	0.6538	0.6538	27.0	27.0	83.0	0.3253	0.3253	0.0	0.0	1.0	0.0	0.0
0.0	9.0	18	7.2701	0.006	3136.0735	2173.7605	117.0	299.0	0.3913	116.0	0.3880	10.0	11.0	64.0	0.1719	0.1562	24.0	24.0	73.0	0.3288	0.3288	53.0	53.0	78.0	0.6795	0.6795	29.0	29.0	83.0	0.3494	0.3494	0.0	0.0	1.0	0.0	0.0
0.0855	10.0	20	7.2748	0.006	3138.1158	2175.1761	116.0	299.0	0.3880	115.0	0.3846	11.0	12.0	64.0	0.1875	0.1719	22.0	22.0	73.0	0.3014	0.3014	51.0	51.0	78.0	0.6538	0.6538	31.0	31.0	83.0	0.3735	0.3735	0.0	0.0	1.0	0.0	0.0
0.0	11.0	22	7.1787	0.006	3096.6477	2146.4326	121.0	299.0	0.4047	121.0	0.4047	19.0	19.0	64.0	0.2969	0.2969	20.0	20.0	73.0	0.2740	0.2740	46.0	46.0	78.0	0.5897	0.5897	36.0	36.0	83.0	0.4337	0.4337	0.0	0.0	1.0	0.0	0.0
0.0	12.0	24	7.2046	0.006	3107.8332	2154.1858	116.0	299.0	0.3880	116.0	0.3880	21.0	21.0	64.0	0.3281	0.3281	18.0	18.0	73.0	0.2466	0.2466	43.0	43.0	78.0	0.5513	0.5513	34.0	34.0	83.0	0.4096	0.4096	0.0	0.0	1.0	0.0	0.0
0.0001	13.0	26	7.2833	0.006	3141.7852	2177.7195	119.0	299.0	0.3980	119.0	0.3980	24.0	24.0	64.0	0.375	0.375	18.0	18.0	73.0	0.2466	0.2466	42.0	42.0	78.0	0.5385	0.5385	35.0	35.0	83.0	0.4217	0.4217	0.0	0.0	1.0	0.0	0.0
0.0	14.0	28	7.3402	0.006	3166.2986	2194.7109	120.0	299.0	0.4013	119.0	0.3980	24.0	24.0	64.0	0.375	0.375	18.0	18.0	73.0	0.2466	0.2466	42.0	42.0	78.0	0.5385	0.5385	35.0	36.0	83.0	0.4337	0.4217	0.0	0.0	1.0	0.0	0.0
0.0	15.0	30	7.3767	0.006	3182.0471	2205.6270	118.0	299.0	0.3946	117.0	0.3913	25.0	25.0	64.0	0.3906	0.3906	18.0	18.0	73.0	0.2466	0.2466	43.0	43.0	78.0	0.5513	0.5513	31.0	32.0	83.0	0.3855	0.3735	0.0	0.0	1.0	0.0	0.0
0.0	16.0	32	7.3591	0.006	3174.4564	2200.3655	120.0	299.0	0.4013	119.0	0.3980	27.0	27.0	64.0	0.4219	0.4219	17.0	17.0	73.0	0.2329	0.2329	43.0	43.0	78.0	0.5513	0.5513	32.0	33.0	83.0	0.3976	0.3855	0.0	0.0	1.0	0.0	0.0
0.0	17.0	34	7.4185	0.006	3200.0987	2218.1394	116.0	299.0	0.3880	115.0	0.3846	27.0	27.0	64.0	0.4219	0.4219	16.0	16.0	73.0	0.2192	0.2192	41.0	41.0	78.0	0.5256	0.5256	31.0	32.0	83.0	0.3855	0.3735	0.0	0.0	1.0	0.0	0.0
0.0	18.0	36	7.4267	0.006	3203.6335	2220.5895	117.0	299.0	0.3913	116.0	0.3880	27.0	27.0	64.0	0.4219	0.4219	16.0	16.0	73.0	0.2192	0.2192	43.0	43.0	78.0	0.5513	0.5513	30.0	31.0	83.0	0.3735	0.3614	0.0	0.0	1.0	0.0	0.0
0.0	19.0	38	7.4611	0.006	3218.4831	2230.8825	115.0	299.0	0.3846	114.0	0.3813	27.0	27.0	64.0	0.4219	0.4219	16.0	16.0	73.0	0.2192	0.2192	42.0	42.0	78.0	0.5385	0.5385	29.0	30.0	83.0	0.3614	0.3494	0.0	0.0	1.0	0.0	0.0
0.0	20.0	40	7.4334	0.006	3206.5009	2222.5771	117.0	299.0	0.3913	116.0	0.3880	27.0	27.0	64.0	0.4219	0.4219	16.0	16.0	73.0	0.2192	0.2192	43.0	43.0	78.0	0.5513	0.5513	30.0	31.0	83.0	0.3735	0.3614	0.0	0.0	1.0	0.0	0.0
0.0	21.0	42	7.4702	0.006	3222.3990	2233.5968	117.0	299.0	0.3913	116.0	0.3880	27.0	27.0	64.0	0.4219	0.4219	16.0	16.0	73.0	0.2192	0.2192	42.0	42.0	78.0	0.5385	0.5385	31.0	32.0	83.0	0.3855	0.3735	0.0	0.0	1.0	0.0	0.0
0.0	22.0	44	7.4213	0.006	3201.2974	2218.9703	117.0	299.0	0.3913	116.0	0.3880	27.0	27.0	64.0	0.4219	0.4219	16.0	16.0	73.0	0.2192	0.2192	42.0	42.0	78.0	0.5385	0.5385	31.0	32.0	83.0	0.3855	0.3735	0.0	0.0	1.0	0.0	0.0
0.0	23.0	46	7.4656	0.006	3220.4146	2232.2213	115.0	299.0	0.3846	114.0	0.3813	27.0	27.0	64.0	0.4219	0.4219	16.0	16.0	73.0	0.2192	0.2192	42.0	42.0	78.0	0.5385	0.5385	29.0	30.0	83.0	0.3614	0.3494	0.0	0.0	1.0	0.0	0.0
0.0	24.0	48	7.4580	0.006	3217.1432	2229.9537	119.0	299.0	0.3980	118.0	0.3946	27.0	27.0	64.0	0.4219	0.4219	16.0	16.0	73.0	0.2192	0.2192	43.0	43.0	78.0	0.5513	0.5513	32.0	33.0	83.0	0.3976	0.3855	0.0	0.0	1.0	0.0	0.0
0.0	25.0	50	7.4591	0.006	3217.5908	2230.2640	114.0	299.0	0.3813	113.0	0.3779	27.0	27.0	64.0	0.4219	0.4219	16.0	16.0	73.0	0.2192	0.2192	41.0	41.0	78.0	0.5256	0.5256	29.0	30.0	83.0	0.3614	0.3494	0.0	0.0	1.0	0.0	0.0
0.0	26.0	52	7.4486	0.006	3213.0496	2227.1163	116.0	299.0	0.3880	115.0	0.3846	27.0	27.0	64.0	0.4219	0.4219	16.0	16.0	73.0	0.2192	0.2192	42.0	42.0	78.0	0.5385	0.5385	30.0	31.0	83.0	0.3735	0.3614	0.0	0.0	1.0	0.0	0.0
0.0	27.0	54	7.4796	0.006	3226.4340	2236.3936	116.0	299.0	0.3880	115.0	0.3846	27.0	27.0	64.0	0.4219	0.4219	16.0	16.0	73.0	0.2192	0.2192	42.0	42.0	78.0	0.5385	0.5385	30.0	31.0	83.0	0.3735	0.3614	0.0	0.0	1.0	0.0	0.0
0.0	28.0	56	7.4605	0.006	3218.2068	2230.6910	116.0	299.0	0.3880	115.0	0.3846	27.0	27.0	64.0	0.4219	0.4219	16.0	16.0	73.0	0.2192	0.2192	42.0	42.0	78.0	0.5385	0.5385	30.0	31.0	83.0	0.3735	0.3614	0.0	0.0	1.0	0.0	0.0
0.0	29.0	58	7.4934	0.006	3232.4115	2240.5369	116.0	299.0	0.3880	115.0	0.3846	27.0	27.0	64.0	0.4219	0.4219	16.0	16.0	73.0	0.2192	0.2192	41.0	41.0	78.0	0.5256	0.5256	31.0	32.0	83.0	0.3855	0.3735	0.0	0.0	1.0	0.0	0.0
0.0	30.0	60	7.5146	0.006	3241.5567	2246.8759	114.0	299.0	0.3813	113.0	0.3779	26.0	26.0	64.0	0.4062	0.4062	16.0	16.0	73.0	0.2192	0.2192	42.0	42.0	78.0	0.5385	0.5385	29.0	30.0	83.0	0.3614	0.3494	0.0	0.0	1.0	0.0	0.0
0.0	31.0	62	7.4440	0.006	3211.0716	2225.7452	116.0	299.0	0.3880	115.0	0.3846	27.0	27.0	64.0	0.4219	0.4219	16.0	16.0	73.0	0.2192	0.2192	42.0	42.0	78.0	0.5385	0.5385	30.0	31.0	83.0	0.3735	0.3614	0.0	0.0	1.0	0.0	0.0
0.0	32.0	64	7.5110	0.006	3239.9892	2245.7894	117.0	299.0	0.3913	116.0	0.3880	28.0	28.0	64.0	0.4375	0.4375	16.0	16.0	73.0	0.2192	0.2192	41.0	41.0	78.0	0.5256	0.5256	31.0	32.0	83.0	0.3855	0.3735	0.0	0.0	1.0	0.0	0.0
0.0	33.0	66	7.4848	0.006	3228.6771	2237.9484	115.0	299.0	0.3846	114.0	0.3813	27.0	27.0	64.0	0.4219	0.4219	16.0	16.0	73.0	0.2192	0.2192	42.0	42.0	78.0	0.5385	0.5385	29.0	30.0	83.0	0.3614	0.3494	0.0	0.0	1.0	0.0	0.0
0.0	34.0	68	7.5193	0.006	3243.5477	2248.2559	118.0	299.0	0.3946	117.0	0.3913	28.0	28.0	64.0	0.4375	0.4375	16.0	16.0	73.0	0.2192	0.2192	42.0	42.0	78.0	0.5385	0.5385	31.0	32.0	83.0	0.3855	0.3735	0.0	0.0	1.0	0.0	0.0
0.0	35.0	70	7.4827	0.006	3227.7753	2237.3234	114.0	299.0	0.3813	113.0	0.3779	27.0	27.0	64.0	0.4219	0.4219	16.0	16.0	73.0	0.2192	0.2192	41.0	41.0	78.0	0.5256	0.5256	29.0	30.0	83.0	0.3614	0.3494	0.0	0.0	1.0	0.0	0.0
0.0	36.0	72	7.4764	0.006	3225.0592	2235.4407	115.0	299.0	0.3846	114.0	0.3813	27.0	27.0	64.0	0.4219	0.4219	16.0	16.0	73.0	0.2192	0.2192	42.0	42.0	78.0	0.5385	0.5385	29.0	30.0	83.0	0.3614	0.3494	0.0	0.0	1.0	0.0	0.0
0.0	37.0	74	7.4991	0.006	3234.8411	2242.2210	118.0	299.0	0.3946	117.0	0.3913	27.0	27.0	64.0	0.4219	0.4219	16.0	16.0	73.0	0.2192	0.2192	42.0	42.0	78.0	0.5385	0.5385	32.0	33.0	83.0	0.3976	0.3855	0.0	0.0	1.0	0.0	0.0
0.0	38.0	76	7.4801	0.006	3226.6618	2236.5515	118.0	299.0	0.3946	117.0	0.3913	28.0	28.0	64.0	0.4375	0.4375	16.0	16.0	73.0	0.2192	0.2192	42.0	42.0	78.0	0.5385	0.5385	31.0	32.0	83.0	0.3855	0.3735	0.0	0.0	1.0	0.0	0.0
0.0	39.0	78	7.4917	0.006	3231.6519	2240.0104	117.0	299.0	0.3913	116.0	0.3880	27.0	27.0	64.0	0.4219	0.4219	16.0	16.0	73.0	0.2192	0.2192	42.0	42.0	78.0	0.5385	0.5385	31.0	32.0	83.0	0.3855	0.3735	0.0	0.0	1.0	0.0	0.0
0.0	40.0	80	7.5154	0.006	3241.8667	2247.0908	116.0	299.0	0.3880	115.0	0.3846	27.0	27.0	64.0	0.4219	0.4219	16.0	16.0	73.0	0.2192	0.2192	42.0	42.0	78.0	0.5385	0.5385	30.0	31.0	83.0	0.3735	0.3614	0.0	0.0	1.0	0.0	0.0
0.0	41.0	82	7.5021	0.006	3236.1541	2243.1311	115.0	299.0	0.3846	114.0	0.3813	27.0	27.0	64.0	0.4219	0.4219	16.0	16.0	73.0	0.2192	0.2192	42.0	42.0	78.0	0.5385	0.5385	29.0	30.0	83.0	0.3614	0.3494	0.0	0.0	1.0	0.0	0.0

Framework versions

• Transformers 4.51.3
• Pytorch 2.6.0+cu124
• Datasets 3.5.0
• Tokenizers 0.21.1