ARC-Challenge_Llama-3.2-1B-7krs1d7e

This model is a fine-tuned version of meta-llama/Llama-3.2-1B on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.4637
• Model Preparation Time: 0.0065
• Mdl: 631.4114
• Accumulated Loss: 437.6610
• Correct Preds: 118.0
• Total Preds: 299.0
• Accuracy: 0.3946
• Correct Gen Preds: 118.0
• Gen Accuracy: 0.3946
• Correct Gen Preds 32: 26.0
• Correct Preds 32: 26.0
• Total Labels 32: 64.0
• Accuracy 32: 0.4062
• Gen Accuracy 32: 0.4062
• Correct Gen Preds 33: 28.0
• Correct Preds 33: 28.0
• Total Labels 33: 73.0
• Accuracy 33: 0.3836
• Gen Accuracy 33: 0.3836
• Correct Gen Preds 34: 30.0
• Correct Preds 34: 30.0
• Total Labels 34: 78.0
• Accuracy 34: 0.3846
• Gen Accuracy 34: 0.3846
• Correct Gen Preds 35: 34.0
• Correct Preds 35: 34.0
• Total Labels 35: 83.0
• Accuracy 35: 0.4096
• Gen Accuracy 35: 0.4096
• Correct Gen Preds 36: 0.0
• Correct Preds 36: 0.0
• Total Labels 36: 1.0
• Accuracy 36: 0.0
• Gen Accuracy 36: 0.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-06
• train_batch_size: 64
• eval_batch_size: 112
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.01
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Model Preparation Time	Mdl	Accumulated Loss	Correct Preds	Total Preds	Accuracy	Correct Gen Preds	Gen Accuracy	Correct Gen Preds 32	Correct Preds 32	Total Labels 32	Accuracy 32	Gen Accuracy 32	Correct Gen Preds 33	Correct Preds 33	Total Labels 33	Accuracy 33	Gen Accuracy 33	Correct Gen Preds 34	Correct Preds 34	Total Labels 34	Accuracy 34	Gen Accuracy 34	Correct Gen Preds 35	Correct Preds 35	Total Labels 35	Accuracy 35	Gen Accuracy 35	Correct Gen Preds 36	Correct Preds 36	Total Labels 36	Accuracy 36	Gen Accuracy 36
No log	0	0	1.6389	0.0065	706.9523	490.0220	66.0	299.0	0.2207	66.0	0.2207	62.0	62.0	64.0	0.9688	0.9688	0.0	0.0	73.0	0.0	0.0	4.0	4.0	78.0	0.0513	0.0513	0.0	0.0	83.0	0.0	0.0	0.0	0.0	1.0	0.0	0.0
1.4473	1.0	17	1.4371	0.0065	619.9126	429.6907	92.0	299.0	0.3077	92.0	0.3077	25.0	25.0	64.0	0.3906	0.3906	4.0	4.0	73.0	0.0548	0.0548	60.0	60.0	78.0	0.7692	0.7692	3.0	3.0	83.0	0.0361	0.0361	0.0	0.0	1.0	0.0	0.0
1.3505	2.0	34	1.3657	0.0065	589.1182	408.3456	88.0	299.0	0.2943	88.0	0.2943	11.0	11.0	64.0	0.1719	0.1719	43.0	43.0	73.0	0.5890	0.5890	17.0	17.0	78.0	0.2179	0.2179	17.0	17.0	83.0	0.2048	0.2048	0.0	0.0	1.0	0.0	0.0
1.4121	3.0	51	1.3409	0.0065	578.4062	400.9206	103.0	299.0	0.3445	103.0	0.3445	16.0	16.0	64.0	0.25	0.25	32.0	32.0	73.0	0.4384	0.4384	32.0	32.0	78.0	0.4103	0.4103	23.0	23.0	83.0	0.2771	0.2771	0.0	0.0	1.0	0.0	0.0
1.2332	4.0	68	1.3372	0.0065	576.8067	399.8119	104.0	299.0	0.3478	104.0	0.3478	22.0	22.0	64.0	0.3438	0.3438	29.0	29.0	73.0	0.3973	0.3973	32.0	32.0	78.0	0.4103	0.4103	21.0	21.0	83.0	0.2530	0.2530	0.0	0.0	1.0	0.0	0.0
1.3464	5.0	85	1.3408	0.0065	578.3790	400.9017	103.0	299.0	0.3445	103.0	0.3445	23.0	23.0	64.0	0.3594	0.3594	29.0	29.0	73.0	0.3973	0.3973	30.0	30.0	78.0	0.3846	0.3846	21.0	21.0	83.0	0.2530	0.2530	0.0	0.0	1.0	0.0	0.0
1.2229	6.0	102	1.3467	0.0065	580.9145	402.6592	109.0	299.0	0.3645	109.0	0.3645	23.0	23.0	64.0	0.3594	0.3594	28.0	28.0	73.0	0.3836	0.3836	31.0	31.0	78.0	0.3974	0.3974	27.0	27.0	83.0	0.3253	0.3253	0.0	0.0	1.0	0.0	0.0
1.2148	7.0	119	1.3595	0.0065	586.4457	406.4932	114.0	299.0	0.3813	114.0	0.3813	23.0	23.0	64.0	0.3594	0.3594	30.0	30.0	73.0	0.4110	0.4110	33.0	33.0	78.0	0.4231	0.4231	28.0	28.0	83.0	0.3373	0.3373	0.0	0.0	1.0	0.0	0.0
1.0739	8.0	136	1.3646	0.0065	588.6428	408.0161	110.0	299.0	0.3679	110.0	0.3679	22.0	22.0	64.0	0.3438	0.3438	31.0	31.0	73.0	0.4247	0.4247	32.0	32.0	78.0	0.4103	0.4103	25.0	25.0	83.0	0.3012	0.3012	0.0	0.0	1.0	0.0	0.0
1.1568	9.0	153	1.3719	0.0065	591.7861	410.1949	113.0	299.0	0.3779	113.0	0.3779	24.0	24.0	64.0	0.375	0.375	29.0	29.0	73.0	0.3973	0.3973	32.0	32.0	78.0	0.4103	0.4103	28.0	28.0	83.0	0.3373	0.3373	0.0	0.0	1.0	0.0	0.0
1.1744	10.0	170	1.3822	0.0065	596.2377	413.2804	112.0	299.0	0.3746	112.0	0.3746	22.0	22.0	64.0	0.3438	0.3438	29.0	29.0	73.0	0.3973	0.3973	32.0	32.0	78.0	0.4103	0.4103	29.0	29.0	83.0	0.3494	0.3494	0.0	0.0	1.0	0.0	0.0
1.0784	11.0	187	1.3933	0.0065	601.0431	416.6114	112.0	299.0	0.3746	112.0	0.3746	22.0	22.0	64.0	0.3438	0.3438	32.0	32.0	73.0	0.4384	0.4384	29.0	29.0	78.0	0.3718	0.3718	29.0	29.0	83.0	0.3494	0.3494	0.0	0.0	1.0	0.0	0.0
0.8901	12.0	204	1.4029	0.0065	605.1659	419.4691	114.0	299.0	0.3813	114.0	0.3813	26.0	26.0	64.0	0.4062	0.4062	27.0	27.0	73.0	0.3699	0.3699	30.0	30.0	78.0	0.3846	0.3846	31.0	31.0	83.0	0.3735	0.3735	0.0	0.0	1.0	0.0	0.0
0.8294	13.0	221	1.4229	0.0065	613.7773	425.4380	113.0	299.0	0.3779	113.0	0.3779	23.0	23.0	64.0	0.3594	0.3594	30.0	30.0	73.0	0.4110	0.4110	31.0	31.0	78.0	0.3974	0.3974	29.0	29.0	83.0	0.3494	0.3494	0.0	0.0	1.0	0.0	0.0
0.7338	14.0	238	1.4235	0.0065	614.0695	425.6405	115.0	299.0	0.3846	115.0	0.3846	22.0	22.0	64.0	0.3438	0.3438	30.0	30.0	73.0	0.4110	0.4110	30.0	30.0	78.0	0.3846	0.3846	33.0	33.0	83.0	0.3976	0.3976	0.0	0.0	1.0	0.0	0.0
0.9321	15.0	255	1.4442	0.0065	622.9754	431.8136	117.0	299.0	0.3913	117.0	0.3913	27.0	27.0	64.0	0.4219	0.4219	29.0	29.0	73.0	0.3973	0.3973	30.0	30.0	78.0	0.3846	0.3846	31.0	31.0	83.0	0.3735	0.3735	0.0	0.0	1.0	0.0	0.0
1.1668	16.0	272	1.4483	0.0065	624.7264	433.0274	114.0	299.0	0.3813	114.0	0.3813	23.0	23.0	64.0	0.3594	0.3594	30.0	30.0	73.0	0.4110	0.4110	30.0	30.0	78.0	0.3846	0.3846	31.0	31.0	83.0	0.3735	0.3735	0.0	0.0	1.0	0.0	0.0
1.0039	17.0	289	1.4637	0.0065	631.4114	437.6610	118.0	299.0	0.3946	118.0	0.3946	26.0	26.0	64.0	0.4062	0.4062	28.0	28.0	73.0	0.3836	0.3836	30.0	30.0	78.0	0.3846	0.3846	34.0	34.0	83.0	0.4096	0.4096	0.0	0.0	1.0	0.0	0.0
0.7995	18.0	306	1.4676	0.0065	633.0521	438.7982	114.0	299.0	0.3813	114.0	0.3813	26.0	26.0	64.0	0.4062	0.4062	28.0	28.0	73.0	0.3836	0.3836	28.0	28.0	78.0	0.3590	0.3590	32.0	32.0	83.0	0.3855	0.3855	0.0	0.0	1.0	0.0	0.0
0.7896	19.0	323	1.4810	0.0065	638.8445	442.8133	114.0	299.0	0.3813	114.0	0.3813	25.0	25.0	64.0	0.3906	0.3906	27.0	27.0	73.0	0.3699	0.3699	29.0	29.0	78.0	0.3718	0.3718	33.0	33.0	83.0	0.3976	0.3976	0.0	0.0	1.0	0.0	0.0
1.052	20.0	340	1.5130	0.0065	652.6464	452.3800	113.0	299.0	0.3779	113.0	0.3779	25.0	25.0	64.0	0.3906	0.3906	28.0	28.0	73.0	0.3836	0.3836	27.0	27.0	78.0	0.3462	0.3462	33.0	33.0	83.0	0.3976	0.3976	0.0	0.0	1.0	0.0	0.0
1.0827	21.0	357	1.5157	0.0065	653.8151	453.1901	114.0	299.0	0.3813	114.0	0.3813	25.0	25.0	64.0	0.3906	0.3906	27.0	27.0	73.0	0.3699	0.3699	28.0	28.0	78.0	0.3590	0.3590	34.0	34.0	83.0	0.4096	0.4096	0.0	0.0	1.0	0.0	0.0
0.7417	22.0	374	1.5402	0.0065	664.4085	460.5329	115.0	299.0	0.3846	115.0	0.3846	26.0	26.0	64.0	0.4062	0.4062	28.0	28.0	73.0	0.3836	0.3836	28.0	28.0	78.0	0.3590	0.3590	33.0	33.0	83.0	0.3976	0.3976	0.0	0.0	1.0	0.0	0.0
1.182	23.0	391	1.5382	0.0065	663.5372	459.9290	113.0	299.0	0.3779	113.0	0.3779	23.0	23.0	64.0	0.3594	0.3594	27.0	27.0	73.0	0.3699	0.3699	28.0	28.0	78.0	0.3590	0.3590	35.0	35.0	83.0	0.4217	0.4217	0.0	0.0	1.0	0.0	0.0
0.7924	24.0	408	1.5484	0.0065	667.9058	462.9570	115.0	299.0	0.3846	115.0	0.3846	26.0	26.0	64.0	0.4062	0.4062	28.0	28.0	73.0	0.3836	0.3836	27.0	27.0	78.0	0.3462	0.3462	34.0	34.0	83.0	0.4096	0.4096	0.0	0.0	1.0	0.0	0.0
0.5855	25.0	425	1.5609	0.0065	673.3183	466.7087	116.0	299.0	0.3880	116.0	0.3880	25.0	25.0	64.0	0.3906	0.3906	27.0	27.0	73.0	0.3699	0.3699	30.0	30.0	78.0	0.3846	0.3846	34.0	34.0	83.0	0.4096	0.4096	0.0	0.0	1.0	0.0	0.0
0.7355	26.0	442	1.5872	0.0065	684.6536	474.5657	113.0	299.0	0.3779	113.0	0.3779	23.0	23.0	64.0	0.3594	0.3594	28.0	28.0	73.0	0.3836	0.3836	28.0	28.0	78.0	0.3590	0.3590	34.0	34.0	83.0	0.4096	0.4096	0.0	0.0	1.0	0.0	0.0
0.7662	27.0	459	1.6051	0.0065	692.3811	479.9220	115.0	299.0	0.3846	115.0	0.3846	26.0	26.0	64.0	0.4062	0.4062	29.0	29.0	73.0	0.3973	0.3973	28.0	28.0	78.0	0.3590	0.3590	32.0	32.0	83.0	0.3855	0.3855	0.0	0.0	1.0	0.0	0.0

Framework versions

• Transformers 4.51.3
• Pytorch 2.6.0+cu124
• Datasets 3.5.0
• Tokenizers 0.21.1