ARC-Challenge_Llama-3.2-1B-zbw42gu2

This model is a fine-tuned version of meta-llama/Llama-3.2-1B on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 4.6897
• Model Preparation Time: 0.0057
• Mdl: 2022.9575
• Accumulated Loss: 1402.2073
• Correct Preds: 159.0
• Total Preds: 299.0
• Accuracy: 0.5318
• Correct Gen Preds: 156.0
• Gen Accuracy: 0.5217
• Correct Gen Preds 32: 30.0
• Correct Preds 32: 30.0
• Total Labels 32: 64.0
• Accuracy 32: 0.4688
• Gen Accuracy 32: 0.4688
• Correct Gen Preds 33: 40.0
• Correct Preds 33: 40.0
• Total Labels 33: 73.0
• Accuracy 33: 0.5479
• Gen Accuracy 33: 0.5479
• Correct Gen Preds 34: 39.0
• Correct Preds 34: 40.0
• Total Labels 34: 78.0
• Accuracy 34: 0.5128
• Gen Accuracy 34: 0.5
• Correct Gen Preds 35: 46.0
• Correct Preds 35: 48.0
• Total Labels 35: 83.0
• Accuracy 35: 0.5783
• Gen Accuracy 35: 0.5542
• Correct Gen Preds 36: 1.0
• Correct Preds 36: 1.0
• Total Labels 36: 1.0
• Accuracy 36: 1.0
• Gen Accuracy 36: 1.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 112
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.01
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Model Preparation Time	Mdl	Accumulated Loss	Correct Preds	Total Preds	Accuracy	Correct Gen Preds	Gen Accuracy	Correct Gen Preds 32	Correct Preds 32	Total Labels 32	Accuracy 32	Gen Accuracy 32	Correct Gen Preds 33	Correct Preds 33	Total Labels 33	Accuracy 33	Gen Accuracy 33	Correct Gen Preds 34	Correct Preds 34	Total Labels 34	Accuracy 34	Gen Accuracy 34	Correct Gen Preds 35	Correct Preds 35	Total Labels 35	Accuracy 35	Gen Accuracy 35	Correct Gen Preds 36	Correct Preds 36	Total Labels 36	Accuracy 36	Gen Accuracy 36
No log	0	0	1.6389	0.0057	706.9523	490.0220	66.0	299.0	0.2207	66.0	0.2207	62.0	62.0	64.0	0.9688	0.9688	0.0	0.0	73.0	0.0	0.0	4.0	4.0	78.0	0.0513	0.0513	0.0	0.0	83.0	0.0	0.0	0.0	0.0	1.0	0.0	0.0
1.341	1.0	17	1.3116	0.0057	565.7899	392.1757	116.0	299.0	0.3880	116.0	0.3880	39.0	39.0	64.0	0.6094	0.6094	42.0	42.0	73.0	0.5753	0.5753	19.0	19.0	78.0	0.2436	0.2436	16.0	16.0	83.0	0.1928	0.1928	0.0	0.0	1.0	0.0	0.0
0.9802	2.0	34	1.3351	0.0057	575.9009	399.1841	144.0	299.0	0.4816	144.0	0.4816	23.0	23.0	64.0	0.3594	0.3594	43.0	43.0	73.0	0.5890	0.5890	37.0	37.0	78.0	0.4744	0.4744	41.0	41.0	83.0	0.4940	0.4940	0.0	0.0	1.0	0.0	0.0
0.5104	3.0	51	1.6004	0.0057	690.3438	478.5098	144.0	299.0	0.4816	144.0	0.4816	27.0	27.0	64.0	0.4219	0.4219	42.0	42.0	73.0	0.5753	0.5753	32.0	32.0	78.0	0.4103	0.4103	43.0	43.0	83.0	0.5181	0.5181	0.0	0.0	1.0	0.0	0.0
0.3768	4.0	68	2.7793	0.0057	1198.8963	831.0116	143.0	299.0	0.4783	138.0	0.4615	21.0	22.0	64.0	0.3438	0.3281	34.0	34.0	73.0	0.4658	0.4658	47.0	48.0	78.0	0.6154	0.6026	36.0	39.0	83.0	0.4699	0.4337	0.0	0.0	1.0	0.0	0.0
0.0128	5.0	85	2.9737	0.0057	1282.7691	889.1478	143.0	299.0	0.4783	143.0	0.4783	22.0	22.0	64.0	0.3438	0.3438	31.0	31.0	73.0	0.4247	0.4247	47.0	47.0	78.0	0.6026	0.6026	43.0	43.0	83.0	0.5181	0.5181	0.0	0.0	1.0	0.0	0.0
0.1291	6.0	102	4.0784	0.0057	1759.2814	1219.4409	142.0	299.0	0.4749	137.0	0.4582	28.0	29.0	64.0	0.4531	0.4375	40.0	40.0	73.0	0.5479	0.5479	38.0	41.0	78.0	0.5256	0.4872	31.0	32.0	83.0	0.3855	0.3735	0.0	0.0	1.0	0.0	0.0
0.0026	7.0	119	4.6897	0.0057	2022.9575	1402.2073	159.0	299.0	0.5318	156.0	0.5217	30.0	30.0	64.0	0.4688	0.4688	40.0	40.0	73.0	0.5479	0.5479	39.0	40.0	78.0	0.5128	0.5	46.0	48.0	83.0	0.5783	0.5542	1.0	1.0	1.0	1.0	1.0
0.0717	8.0	136	4.7403	0.0057	2044.7947	1417.3437	149.0	299.0	0.4983	149.0	0.4983	25.0	25.0	64.0	0.3906	0.3906	41.0	41.0	73.0	0.5616	0.5616	46.0	46.0	78.0	0.5897	0.5897	37.0	37.0	83.0	0.4458	0.4458	0.0	0.0	1.0	0.0	0.0
0.0002	9.0	153	4.4499	0.0057	1919.5185	1330.5088	149.0	299.0	0.4983	149.0	0.4983	28.0	28.0	64.0	0.4375	0.4375	40.0	40.0	73.0	0.5479	0.5479	44.0	44.0	78.0	0.5641	0.5641	37.0	37.0	83.0	0.4458	0.4458	0.0	0.0	1.0	0.0	0.0
0.2568	10.0	170	5.0550	0.0057	2180.5548	1511.4454	149.0	299.0	0.4983	149.0	0.4983	30.0	30.0	64.0	0.4688	0.4688	40.0	40.0	73.0	0.5479	0.5479	44.0	44.0	78.0	0.5641	0.5641	35.0	35.0	83.0	0.4217	0.4217	0.0	0.0	1.0	0.0	0.0
0.0004	11.0	187	4.7930	0.0057	2067.5455	1433.1133	148.0	299.0	0.4950	148.0	0.4950	34.0	34.0	64.0	0.5312	0.5312	44.0	44.0	73.0	0.6027	0.6027	39.0	39.0	78.0	0.5	0.5	31.0	31.0	83.0	0.3735	0.3735	0.0	0.0	1.0	0.0	0.0
0.0749	12.0	204	5.1575	0.0057	2224.7853	1542.1036	142.0	299.0	0.4749	142.0	0.4749	19.0	19.0	64.0	0.2969	0.2969	38.0	38.0	73.0	0.5205	0.5205	43.0	43.0	78.0	0.5513	0.5513	41.0	41.0	83.0	0.4940	0.4940	1.0	1.0	1.0	1.0	1.0
0.0	13.0	221	5.4388	0.0057	2346.1061	1626.1968	145.0	299.0	0.4849	145.0	0.4849	30.0	30.0	64.0	0.4688	0.4688	36.0	36.0	73.0	0.4932	0.4932	43.0	43.0	78.0	0.5513	0.5513	35.0	35.0	83.0	0.4217	0.4217	1.0	1.0	1.0	1.0	1.0
0.0	14.0	238	5.5872	0.0057	2410.1369	1670.5796	145.0	299.0	0.4849	145.0	0.4849	31.0	31.0	64.0	0.4844	0.4844	36.0	36.0	73.0	0.4932	0.4932	42.0	42.0	78.0	0.5385	0.5385	35.0	35.0	83.0	0.4217	0.4217	1.0	1.0	1.0	1.0	1.0
0.0	15.0	255	5.6061	0.0057	2418.2871	1676.2289	145.0	299.0	0.4849	145.0	0.4849	31.0	31.0	64.0	0.4844	0.4844	35.0	35.0	73.0	0.4795	0.4795	43.0	43.0	78.0	0.5513	0.5513	35.0	35.0	83.0	0.4217	0.4217	1.0	1.0	1.0	1.0	1.0
0.0	16.0	272	5.6490	0.0057	2436.7772	1689.0453	145.0	299.0	0.4849	145.0	0.4849	31.0	31.0	64.0	0.4844	0.4844	35.0	35.0	73.0	0.4795	0.4795	43.0	43.0	78.0	0.5513	0.5513	35.0	35.0	83.0	0.4217	0.4217	1.0	1.0	1.0	1.0	1.0
0.0	17.0	289	5.6868	0.0057	2453.0896	1700.3521	145.0	299.0	0.4849	145.0	0.4849	31.0	31.0	64.0	0.4844	0.4844	35.0	35.0	73.0	0.4795	0.4795	43.0	43.0	78.0	0.5513	0.5513	35.0	35.0	83.0	0.4217	0.4217	1.0	1.0	1.0	1.0	1.0

Framework versions

• Transformers 4.51.3
• Pytorch 2.6.0+cu124
• Datasets 3.5.0
• Tokenizers 0.21.1