ARC-Challenge_Llama-3.2-1B-s28f2lpx

This model is a fine-tuned version of meta-llama/Llama-3.2-1B on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 3.0060
• Model Preparation Time: 0.0057
• Mdl: 1296.6762
• Accumulated Loss: 898.7874
• Correct Preds: 126.0
• Total Preds: 299.0
• Accuracy: 0.4214
• Correct Gen Preds: 108.0
• Gen Accuracy: 0.3612
• Correct Gen Preds 32: 15.0
• Correct Preds 32: 18.0
• Total Labels 32: 64.0
• Accuracy 32: 0.2812
• Gen Accuracy 32: 0.2344
• Correct Gen Preds 33: 37.0
• Correct Preds 33: 41.0
• Total Labels 33: 73.0
• Accuracy 33: 0.5616
• Gen Accuracy 33: 0.5068
• Correct Gen Preds 34: 32.0
• Correct Preds 34: 38.0
• Total Labels 34: 78.0
• Accuracy 34: 0.4872
• Gen Accuracy 34: 0.4103
• Correct Gen Preds 35: 24.0
• Correct Preds 35: 29.0
• Total Labels 35: 83.0
• Accuracy 35: 0.3494
• Gen Accuracy 35: 0.2892
• Correct Gen Preds 36: 0.0
• Correct Preds 36: 0.0
• Total Labels 36: 1.0
• Accuracy 36: 0.0
• Gen Accuracy 36: 0.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 112
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.01
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Model Preparation Time	Mdl	Accumulated Loss	Correct Preds	Total Preds	Accuracy	Correct Gen Preds	Gen Accuracy	Correct Gen Preds 32	Correct Preds 32	Total Labels 32	Accuracy 32	Gen Accuracy 32	Correct Gen Preds 33	Correct Preds 33	Total Labels 33	Accuracy 33	Gen Accuracy 33	Correct Gen Preds 34	Correct Preds 34	Total Labels 34	Accuracy 34	Gen Accuracy 34	Correct Gen Preds 35	Correct Preds 35	Total Labels 35	Accuracy 35	Gen Accuracy 35	Correct Gen Preds 36	Correct Preds 36	Total Labels 36	Accuracy 36	Gen Accuracy 36
No log	0	0	1.6389	0.0057	706.9523	490.0220	66.0	299.0	0.2207	66.0	0.2207	62.0	62.0	64.0	0.9688	0.9688	0.0	0.0	73.0	0.0	0.0	4.0	4.0	78.0	0.0513	0.0513	0.0	0.0	83.0	0.0	0.0	0.0	0.0	1.0	0.0	0.0
1.158	1.0	5	1.8034	0.0057	777.9201	539.2131	78.0	299.0	0.2609	68.0	0.2274	28.0	34.0	64.0	0.5312	0.4375	0.0	0.0	73.0	0.0	0.0	0.0	0.0	78.0	0.0	0.0	40.0	44.0	83.0	0.5301	0.4819	0.0	0.0	1.0	0.0	0.0
1.2621	2.0	10	1.4166	0.0057	611.0825	423.5701	95.0	299.0	0.3177	95.0	0.3177	12.0	12.0	64.0	0.1875	0.1875	59.0	59.0	73.0	0.8082	0.8082	22.0	22.0	78.0	0.2821	0.2821	2.0	2.0	83.0	0.0241	0.0241	0.0	0.0	1.0	0.0	0.0
0.8047	3.0	15	1.8805	0.0057	811.2041	562.2839	119.0	299.0	0.3980	97.0	0.3244	15.0	22.0	64.0	0.3438	0.2344	13.0	15.0	73.0	0.2055	0.1781	27.0	35.0	78.0	0.4487	0.3462	42.0	47.0	83.0	0.5663	0.5060	0.0	0.0	1.0	0.0	0.0
0.713	4.0	20	1.7794	0.0057	767.5938	532.0555	118.0	299.0	0.3946	113.0	0.3779	20.0	21.0	64.0	0.3281	0.3125	26.0	27.0	73.0	0.3699	0.3562	38.0	38.0	78.0	0.4872	0.4872	29.0	32.0	83.0	0.3855	0.3494	0.0	0.0	1.0	0.0	0.0
0.4094	5.0	25	3.0060	0.0057	1296.6762	898.7874	126.0	299.0	0.4214	108.0	0.3612	15.0	18.0	64.0	0.2812	0.2344	37.0	41.0	73.0	0.5616	0.5068	32.0	38.0	78.0	0.4872	0.4103	24.0	29.0	83.0	0.3494	0.2892	0.0	0.0	1.0	0.0	0.0
0.0005	6.0	30	3.0044	0.0057	1295.9968	898.3165	112.0	299.0	0.3746	78.0	0.2609	10.0	15.0	64.0	0.2344	0.1562	30.0	38.0	73.0	0.5205	0.4110	12.0	24.0	78.0	0.3077	0.1538	26.0	35.0	83.0	0.4217	0.3133	0.0	0.0	1.0	0.0	0.0
0.0004	7.0	35	5.6948	0.0057	2456.5227	1702.7318	120.0	299.0	0.4013	118.0	0.3946	18.0	18.0	64.0	0.2812	0.2812	31.0	32.0	73.0	0.4384	0.4247	27.0	27.0	78.0	0.3462	0.3462	42.0	43.0	83.0	0.5181	0.5060	0.0	0.0	1.0	0.0	0.0
0.0	8.0	40	6.9817	0.0057	3011.6472	2087.5148	120.0	299.0	0.4013	119.0	0.3980	20.0	20.0	64.0	0.3125	0.3125	32.0	32.0	73.0	0.4384	0.4384	29.0	29.0	78.0	0.3718	0.3718	38.0	39.0	83.0	0.4699	0.4578	0.0	0.0	1.0	0.0	0.0
0.0	9.0	45	7.1908	0.0057	3101.8574	2150.0437	117.0	299.0	0.3913	116.0	0.3880	18.0	18.0	64.0	0.2812	0.2812	32.0	32.0	73.0	0.4384	0.4384	31.0	31.0	78.0	0.3974	0.3974	35.0	36.0	83.0	0.4337	0.4217	0.0	0.0	1.0	0.0	0.0
0.0	10.0	50	7.2800	0.0057	3140.3348	2176.7142	119.0	299.0	0.3980	118.0	0.3946	16.0	16.0	64.0	0.25	0.25	33.0	33.0	73.0	0.4521	0.4521	31.0	31.0	78.0	0.3974	0.3974	38.0	39.0	83.0	0.4699	0.4578	0.0	0.0	1.0	0.0	0.0
0.0	11.0	55	7.4971	0.0057	3234.0139	2241.6476	118.0	299.0	0.3946	117.0	0.3913	14.0	14.0	64.0	0.2188	0.2188	35.0	35.0	73.0	0.4795	0.4795	29.0	29.0	78.0	0.3718	0.3718	39.0	40.0	83.0	0.4819	0.4699	0.0	0.0	1.0	0.0	0.0
0.0	12.0	60	7.5472	0.0057	3255.5900	2256.6030	118.0	299.0	0.3946	117.0	0.3913	13.0	13.0	64.0	0.2031	0.2031	35.0	35.0	73.0	0.4795	0.4795	30.0	30.0	78.0	0.3846	0.3846	39.0	40.0	83.0	0.4819	0.4699	0.0	0.0	1.0	0.0	0.0
0.0	13.0	65	7.5822	0.0057	3270.6972	2267.0745	118.0	299.0	0.3946	117.0	0.3913	14.0	14.0	64.0	0.2188	0.2188	35.0	35.0	73.0	0.4795	0.4795	29.0	29.0	78.0	0.3718	0.3718	39.0	40.0	83.0	0.4819	0.4699	0.0	0.0	1.0	0.0	0.0
0.0	14.0	70	7.5847	0.0057	3271.7766	2267.8228	115.0	299.0	0.3846	114.0	0.3813	13.0	13.0	64.0	0.2031	0.2031	35.0	35.0	73.0	0.4795	0.4795	28.0	28.0	78.0	0.3590	0.3590	38.0	39.0	83.0	0.4699	0.4578	0.0	0.0	1.0	0.0	0.0
0.0	15.0	75	7.5878	0.0057	3273.1139	2268.7497	119.0	299.0	0.3980	118.0	0.3946	14.0	14.0	64.0	0.2188	0.2188	36.0	36.0	73.0	0.4932	0.4932	30.0	30.0	78.0	0.3846	0.3846	38.0	39.0	83.0	0.4699	0.4578	0.0	0.0	1.0	0.0	0.0
0.0	16.0	80	7.6154	0.0057	3285.0362	2277.0136	115.0	299.0	0.3846	114.0	0.3813	13.0	13.0	64.0	0.2031	0.2031	35.0	35.0	73.0	0.4795	0.4795	28.0	28.0	78.0	0.3590	0.3590	38.0	39.0	83.0	0.4699	0.4578	0.0	0.0	1.0	0.0	0.0
0.0	17.0	85	7.6350	0.0057	3293.4646	2282.8557	114.0	299.0	0.3813	113.0	0.3779	13.0	13.0	64.0	0.2031	0.2031	34.0	34.0	73.0	0.4658	0.4658	28.0	28.0	78.0	0.3590	0.3590	38.0	39.0	83.0	0.4699	0.4578	0.0	0.0	1.0	0.0	0.0
0.0	18.0	90	7.6204	0.0057	3287.1680	2278.4912	118.0	299.0	0.3946	117.0	0.3913	13.0	13.0	64.0	0.2031	0.2031	36.0	36.0	73.0	0.4932	0.4932	29.0	29.0	78.0	0.3718	0.3718	39.0	40.0	83.0	0.4819	0.4699	0.0	0.0	1.0	0.0	0.0
0.0	19.0	95	7.6193	0.0057	3286.6964	2278.1644	118.0	299.0	0.3946	117.0	0.3913	14.0	14.0	64.0	0.2188	0.2188	35.0	35.0	73.0	0.4795	0.4795	29.0	29.0	78.0	0.3718	0.3718	39.0	40.0	83.0	0.4819	0.4699	0.0	0.0	1.0	0.0	0.0
0.0	20.0	100	7.5929	0.0057	3275.3102	2270.2720	117.0	299.0	0.3913	116.0	0.3880	13.0	13.0	64.0	0.2031	0.2031	35.0	35.0	73.0	0.4795	0.4795	30.0	30.0	78.0	0.3846	0.3846	38.0	39.0	83.0	0.4699	0.4578	0.0	0.0	1.0	0.0	0.0
0.0	21.0	105	7.6193	0.0057	3286.7184	2278.1796	117.0	299.0	0.3913	116.0	0.3880	14.0	14.0	64.0	0.2188	0.2188	34.0	34.0	73.0	0.4658	0.4658	30.0	30.0	78.0	0.3846	0.3846	38.0	39.0	83.0	0.4699	0.4578	0.0	0.0	1.0	0.0	0.0
0.0	22.0	110	7.6262	0.0057	3289.6618	2280.2198	118.0	299.0	0.3946	117.0	0.3913	13.0	13.0	64.0	0.2031	0.2031	35.0	35.0	73.0	0.4795	0.4795	30.0	30.0	78.0	0.3846	0.3846	39.0	40.0	83.0	0.4819	0.4699	0.0	0.0	1.0	0.0	0.0
0.0	23.0	115	7.6148	0.0057	3284.7647	2276.8254	117.0	299.0	0.3913	116.0	0.3880	13.0	13.0	64.0	0.2031	0.2031	34.0	34.0	73.0	0.4658	0.4658	30.0	30.0	78.0	0.3846	0.3846	39.0	40.0	83.0	0.4819	0.4699	0.0	0.0	1.0	0.0	0.0
0.0	24.0	120	7.6192	0.0057	3286.6654	2278.1428	116.0	299.0	0.3880	115.0	0.3846	13.0	13.0	64.0	0.2031	0.2031	34.0	34.0	73.0	0.4658	0.4658	30.0	30.0	78.0	0.3846	0.3846	38.0	39.0	83.0	0.4699	0.4578	0.0	0.0	1.0	0.0	0.0
0.0	25.0	125	7.5948	0.0057	3276.1543	2270.8571	118.0	299.0	0.3946	117.0	0.3913	14.0	14.0	64.0	0.2188	0.2188	35.0	35.0	73.0	0.4795	0.4795	29.0	29.0	78.0	0.3718	0.3718	39.0	40.0	83.0	0.4819	0.4699	0.0	0.0	1.0	0.0	0.0
0.0	26.0	130	7.5984	0.0057	3277.6940	2271.9244	118.0	299.0	0.3946	117.0	0.3913	14.0	14.0	64.0	0.2188	0.2188	34.0	34.0	73.0	0.4658	0.4658	31.0	31.0	78.0	0.3974	0.3974	38.0	39.0	83.0	0.4699	0.4578	0.0	0.0	1.0	0.0	0.0
0.0	27.0	135	7.6010	0.0057	3278.8114	2272.6989	117.0	299.0	0.3913	116.0	0.3880	15.0	15.0	64.0	0.2344	0.2344	34.0	34.0	73.0	0.4658	0.4658	28.0	28.0	78.0	0.3590	0.3590	39.0	40.0	83.0	0.4819	0.4699	0.0	0.0	1.0	0.0	0.0
0.0	28.0	140	7.6274	0.0057	3290.2179	2280.6053	117.0	299.0	0.3913	116.0	0.3880	13.0	13.0	64.0	0.2031	0.2031	35.0	35.0	73.0	0.4795	0.4795	29.0	29.0	78.0	0.3718	0.3718	39.0	40.0	83.0	0.4819	0.4699	0.0	0.0	1.0	0.0	0.0
0.0	29.0	145	7.6225	0.0057	3288.0859	2279.1274	119.0	299.0	0.3980	118.0	0.3946	13.0	13.0	64.0	0.2031	0.2031	36.0	36.0	73.0	0.4932	0.4932	30.0	30.0	78.0	0.3846	0.3846	39.0	40.0	83.0	0.4819	0.4699	0.0	0.0	1.0	0.0	0.0
0.0	30.0	150	7.6112	0.0057	3283.2011	2275.7416	117.0	299.0	0.3913	116.0	0.3880	13.0	13.0	64.0	0.2031	0.2031	34.0	34.0	73.0	0.4658	0.4658	30.0	30.0	78.0	0.3846	0.3846	39.0	40.0	83.0	0.4819	0.4699	0.0	0.0	1.0	0.0	0.0
0.0	31.0	155	7.6424	0.0057	3296.6696	2285.0772	120.0	299.0	0.4013	119.0	0.3980	13.0	13.0	64.0	0.2031	0.2031	36.0	36.0	73.0	0.4932	0.4932	31.0	31.0	78.0	0.3974	0.3974	39.0	40.0	83.0	0.4819	0.4699	0.0	0.0	1.0	0.0	0.0
0.0	32.0	160	7.6202	0.0057	3287.1060	2278.4482	115.0	299.0	0.3846	114.0	0.3813	13.0	13.0	64.0	0.2031	0.2031	34.0	34.0	73.0	0.4658	0.4658	30.0	30.0	78.0	0.3846	0.3846	37.0	38.0	83.0	0.4578	0.4458	0.0	0.0	1.0	0.0	0.0
0.0	33.0	165	7.6123	0.0057	3283.6669	2276.0645	121.0	299.0	0.4047	120.0	0.4013	14.0	14.0	64.0	0.2188	0.2188	36.0	36.0	73.0	0.4932	0.4932	31.0	31.0	78.0	0.3974	0.3974	39.0	40.0	83.0	0.4819	0.4699	0.0	0.0	1.0	0.0	0.0
0.0	34.0	170	7.6315	0.0057	3291.9582	2281.8115	117.0	299.0	0.3913	116.0	0.3880	13.0	13.0	64.0	0.2031	0.2031	35.0	35.0	73.0	0.4795	0.4795	30.0	30.0	78.0	0.3846	0.3846	38.0	39.0	83.0	0.4699	0.4578	0.0	0.0	1.0	0.0	0.0
0.0	35.0	175	7.6261	0.0057	3289.6442	2280.2076	116.0	299.0	0.3880	115.0	0.3846	14.0	14.0	64.0	0.2188	0.2188	34.0	34.0	73.0	0.4658	0.4658	28.0	28.0	78.0	0.3590	0.3590	39.0	40.0	83.0	0.4819	0.4699	0.0	0.0	1.0	0.0	0.0

Framework versions

• Transformers 4.51.3
• Pytorch 2.6.0+cu124
• Datasets 3.5.0
• Tokenizers 0.21.1