ARC-Challenge_Llama-3.2-1B-lwkebjbv

This model is a fine-tuned version of meta-llama/Llama-3.2-1B on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 4.6827
• Model Preparation Time: 0.006
• Mdl: 2019.9566
• Accumulated Loss: 1400.1272
• Correct Preds: 154.0
• Total Preds: 299.0
• Accuracy: 0.5151
• Correct Gen Preds: 153.0
• Gen Accuracy: 0.5117
• Correct Gen Preds 32: 35.0
• Correct Preds 32: 35.0
• Total Labels 32: 64.0
• Accuracy 32: 0.5469
• Gen Accuracy 32: 0.5469
• Correct Gen Preds 33: 42.0
• Correct Preds 33: 42.0
• Total Labels 33: 73.0
• Accuracy 33: 0.5753
• Gen Accuracy 33: 0.5753
• Correct Gen Preds 34: 42.0
• Correct Preds 34: 42.0
• Total Labels 34: 78.0
• Accuracy 34: 0.5385
• Gen Accuracy 34: 0.5385
• Correct Gen Preds 35: 33.0
• Correct Preds 35: 34.0
• Total Labels 35: 83.0
• Accuracy 35: 0.4096
• Gen Accuracy 35: 0.3976
• Correct Gen Preds 36: 1.0
• Correct Preds 36: 1.0
• Total Labels 36: 1.0
• Accuracy 36: 1.0
• Gen Accuracy 36: 1.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 112
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.01
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Model Preparation Time	Mdl	Accumulated Loss	Correct Preds	Total Preds	Accuracy	Correct Gen Preds	Gen Accuracy	Correct Gen Preds 32	Correct Preds 32	Total Labels 32	Accuracy 32	Gen Accuracy 32	Correct Gen Preds 33	Correct Preds 33	Total Labels 33	Accuracy 33	Gen Accuracy 33	Correct Gen Preds 34	Correct Preds 34	Total Labels 34	Accuracy 34	Gen Accuracy 34	Correct Gen Preds 35	Correct Preds 35	Total Labels 35	Accuracy 35	Gen Accuracy 35	Correct Gen Preds 36	Correct Preds 36	Total Labels 36	Accuracy 36	Gen Accuracy 36
No log	0	0	1.6389	0.006	706.9523	490.0220	66.0	299.0	0.2207	66.0	0.2207	62.0	62.0	64.0	0.9688	0.9688	0.0	0.0	73.0	0.0	0.0	4.0	4.0	78.0	0.0513	0.0513	0.0	0.0	83.0	0.0	0.0	0.0	0.0	1.0	0.0	0.0
1.3456	1.0	14	1.2743	0.006	549.6850	381.0126	119.0	299.0	0.3980	119.0	0.3980	12.0	12.0	64.0	0.1875	0.1875	36.0	36.0	73.0	0.4932	0.4932	52.0	52.0	78.0	0.6667	0.6667	19.0	19.0	83.0	0.2289	0.2289	0.0	0.0	1.0	0.0	0.0
0.9613	2.0	28	1.3169	0.006	568.0817	393.7642	135.0	299.0	0.4515	135.0	0.4515	30.0	30.0	64.0	0.4688	0.4688	44.0	44.0	73.0	0.6027	0.6027	29.0	29.0	78.0	0.3718	0.3718	32.0	32.0	83.0	0.3855	0.3855	0.0	0.0	1.0	0.0	0.0
0.4074	3.0	42	1.5609	0.006	673.3394	466.7233	146.0	299.0	0.4883	146.0	0.4883	31.0	31.0	64.0	0.4844	0.4844	46.0	46.0	73.0	0.6301	0.6301	38.0	38.0	78.0	0.4872	0.4872	31.0	31.0	83.0	0.3735	0.3735	0.0	0.0	1.0	0.0	0.0
0.1012	4.0	56	1.9102	0.006	824.0133	571.1625	151.0	299.0	0.5050	149.0	0.4983	39.0	40.0	64.0	0.625	0.6094	29.0	29.0	73.0	0.3973	0.3973	40.0	40.0	78.0	0.5128	0.5128	41.0	42.0	83.0	0.5060	0.4940	0.0	0.0	1.0	0.0	0.0
0.2334	5.0	70	3.9162	0.006	1689.3207	1170.9479	149.0	299.0	0.4983	136.0	0.4548	30.0	33.0	64.0	0.5156	0.4688	34.0	40.0	73.0	0.5479	0.4658	40.0	41.0	78.0	0.5256	0.5128	31.0	34.0	83.0	0.4096	0.3735	1.0	1.0	1.0	1.0	1.0
0.0	6.0	84	4.6827	0.006	2019.9566	1400.1272	154.0	299.0	0.5151	153.0	0.5117	35.0	35.0	64.0	0.5469	0.5469	42.0	42.0	73.0	0.5753	0.5753	42.0	42.0	78.0	0.5385	0.5385	33.0	34.0	83.0	0.4096	0.3976	1.0	1.0	1.0	1.0	1.0
0.0445	7.0	98	3.9592	0.006	1707.8440	1183.7873	149.0	299.0	0.4983	148.0	0.4950	30.0	30.0	64.0	0.4688	0.4688	39.0	39.0	73.0	0.5342	0.5342	38.0	38.0	78.0	0.4872	0.4872	41.0	42.0	83.0	0.5060	0.4940	0.0	0.0	1.0	0.0	0.0
0.0006	8.0	112	4.3498	0.006	1876.3451	1300.5833	149.0	299.0	0.4983	148.0	0.4950	31.0	31.0	64.0	0.4844	0.4844	48.0	48.0	73.0	0.6575	0.6575	33.0	33.0	78.0	0.4231	0.4231	35.0	36.0	83.0	0.4337	0.4217	1.0	1.0	1.0	1.0	1.0
0.0021	9.0	126	4.8229	0.006	2080.4425	1442.0529	143.0	299.0	0.4783	143.0	0.4783	34.0	34.0	64.0	0.5312	0.5312	39.0	39.0	73.0	0.5342	0.5342	33.0	33.0	78.0	0.4231	0.4231	36.0	36.0	83.0	0.4337	0.4337	1.0	1.0	1.0	1.0	1.0
0.0	10.0	140	6.4836	0.006	2796.8220	1938.6093	132.0	299.0	0.4415	131.0	0.4381	28.0	28.0	64.0	0.4375	0.4375	51.0	51.0	73.0	0.6986	0.6986	28.0	29.0	78.0	0.3718	0.3590	24.0	24.0	83.0	0.2892	0.2892	0.0	0.0	1.0	0.0	0.0
0.0	11.0	154	6.4580	0.006	2785.7456	1930.9317	139.0	299.0	0.4649	136.0	0.4548	25.0	26.0	64.0	0.4062	0.3906	50.0	50.0	73.0	0.6849	0.6849	35.0	36.0	78.0	0.4615	0.4487	25.0	26.0	83.0	0.3133	0.3012	1.0	1.0	1.0	1.0	1.0
0.0	12.0	168	6.2785	0.006	2708.3215	1877.2654	144.0	299.0	0.4816	139.0	0.4649	24.0	26.0	64.0	0.4062	0.375	49.0	49.0	73.0	0.6712	0.6712	37.0	38.0	78.0	0.4872	0.4744	28.0	30.0	83.0	0.3614	0.3373	1.0	1.0	1.0	1.0	1.0
0.0	13.0	182	6.2504	0.006	2696.2038	1868.8661	144.0	299.0	0.4816	139.0	0.4649	24.0	26.0	64.0	0.4062	0.375	49.0	49.0	73.0	0.6712	0.6712	38.0	39.0	78.0	0.5	0.4872	27.0	29.0	83.0	0.3494	0.3253	1.0	1.0	1.0	1.0	1.0
0.0	14.0	196	6.2645	0.006	2702.2946	1873.0879	145.0	299.0	0.4849	140.0	0.4682	24.0	26.0	64.0	0.4062	0.375	48.0	48.0	73.0	0.6575	0.6575	38.0	39.0	78.0	0.5	0.4872	29.0	31.0	83.0	0.3735	0.3494	1.0	1.0	1.0	1.0	1.0
0.0	15.0	210	6.2772	0.006	2707.7720	1876.8845	146.0	299.0	0.4883	141.0	0.4716	24.0	26.0	64.0	0.4062	0.375	48.0	48.0	73.0	0.6575	0.6575	38.0	39.0	78.0	0.5	0.4872	30.0	32.0	83.0	0.3855	0.3614	1.0	1.0	1.0	1.0	1.0
0.0	16.0	224	6.2825	0.006	2710.0621	1878.4719	145.0	299.0	0.4849	140.0	0.4682	25.0	27.0	64.0	0.4219	0.3906	49.0	49.0	73.0	0.6712	0.6712	37.0	38.0	78.0	0.4872	0.4744	28.0	30.0	83.0	0.3614	0.3373	1.0	1.0	1.0	1.0	1.0
0.0	17.0	238	6.2927	0.006	2714.4760	1881.5314	144.0	299.0	0.4816	139.0	0.4649	25.0	27.0	64.0	0.4219	0.3906	48.0	48.0	73.0	0.6575	0.6575	38.0	39.0	78.0	0.5	0.4872	27.0	29.0	83.0	0.3494	0.3253	1.0	1.0	1.0	1.0	1.0
0.0	18.0	252	6.2524	0.006	2697.0800	1869.4734	147.0	299.0	0.4916	142.0	0.4749	25.0	27.0	64.0	0.4219	0.3906	49.0	49.0	73.0	0.6712	0.6712	38.0	39.0	78.0	0.5	0.4872	29.0	31.0	83.0	0.3735	0.3494	1.0	1.0	1.0	1.0	1.0

Framework versions

• Transformers 4.51.3
• Pytorch 2.6.0+cu124
• Datasets 3.5.0
• Tokenizers 0.21.1