ARC-Challenge_Llama-3.2-1B-1gb7xcyc

This model is a fine-tuned version of meta-llama/Llama-3.2-1B on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 6.4571
• Model Preparation Time: 0.0059
• Mdl: 2785.3751
• Accumulated Loss: 1930.6749
• Correct Preds: 108.0
• Total Preds: 299.0
• Accuracy: 0.3612
• Correct Gen Preds: 104.0
• Gen Accuracy: 0.3478
• Correct Gen Preds 32: 20.0
• Correct Preds 32: 20.0
• Total Labels 32: 64.0
• Accuracy 32: 0.3125
• Gen Accuracy 32: 0.3125
• Correct Gen Preds 33: 32.0
• Correct Preds 33: 34.0
• Total Labels 33: 73.0
• Accuracy 33: 0.4658
• Gen Accuracy 33: 0.4384
• Correct Gen Preds 34: 24.0
• Correct Preds 34: 25.0
• Total Labels 34: 78.0
• Accuracy 34: 0.3205
• Gen Accuracy 34: 0.3077
• Correct Gen Preds 35: 28.0
• Correct Preds 35: 29.0
• Total Labels 35: 83.0
• Accuracy 35: 0.3494
• Gen Accuracy 35: 0.3373
• Correct Gen Preds 36: 0.0
• Correct Preds 36: 0.0
• Total Labels 36: 1.0
• Accuracy 36: 0.0
• Gen Accuracy 36: 0.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 112
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.01
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Model Preparation Time	Mdl	Accumulated Loss	Correct Preds	Total Preds	Accuracy	Correct Gen Preds	Gen Accuracy	Correct Gen Preds 32	Correct Preds 32	Total Labels 32	Accuracy 32	Gen Accuracy 32	Correct Gen Preds 33	Correct Preds 33	Total Labels 33	Accuracy 33	Gen Accuracy 33	Correct Gen Preds 34	Correct Preds 34	Total Labels 34	Accuracy 34	Gen Accuracy 34	Correct Gen Preds 35	Correct Preds 35	Total Labels 35	Accuracy 35	Gen Accuracy 35	Correct Gen Preds 36	Correct Preds 36	Total Labels 36	Accuracy 36	Gen Accuracy 36
No log	0	0	1.6389	0.0059	706.9523	490.0220	66.0	299.0	0.2207	66.0	0.2207	62.0	62.0	64.0	0.9688	0.9688	0.0	0.0	73.0	0.0	0.0	4.0	4.0	78.0	0.0513	0.0513	0.0	0.0	83.0	0.0	0.0	0.0	0.0	1.0	0.0	0.0
1.3831	1.0	2	1.3880	0.0059	598.7203	415.0013	90.0	299.0	0.3010	90.0	0.3010	8.0	8.0	64.0	0.125	0.125	57.0	57.0	73.0	0.7808	0.7808	14.0	14.0	78.0	0.1795	0.1795	11.0	11.0	83.0	0.1325	0.1325	0.0	0.0	1.0	0.0	0.0
3.582	2.0	4	1.8424	0.0059	794.7361	550.8691	69.0	299.0	0.2308	28.0	0.0936	26.0	62.0	64.0	0.9688	0.4062	0.0	0.0	73.0	0.0	0.0	1.0	1.0	78.0	0.0128	0.0128	1.0	6.0	83.0	0.0723	0.0120	0.0	0.0	1.0	0.0	0.0
1.1863	3.0	6	1.4459	0.0059	623.7050	432.3193	78.0	299.0	0.2609	78.0	0.2609	37.0	37.0	64.0	0.5781	0.5781	0.0	0.0	73.0	0.0	0.0	0.0	0.0	78.0	0.0	0.0	41.0	41.0	83.0	0.4940	0.4940	0.0	0.0	1.0	0.0	0.0
1.0655	4.0	8	1.3996	0.0059	603.7273	418.4719	81.0	299.0	0.2709	79.0	0.2642	23.0	24.0	64.0	0.375	0.3594	14.0	14.0	73.0	0.1918	0.1918	29.0	30.0	78.0	0.3846	0.3718	13.0	13.0	83.0	0.1566	0.1566	0.0	0.0	1.0	0.0	0.0
0.5478	5.0	10	1.8142	0.0059	782.5957	542.4540	77.0	299.0	0.2575	77.0	0.2575	42.0	42.0	64.0	0.6562	0.6562	4.0	4.0	73.0	0.0548	0.0548	18.0	18.0	78.0	0.2308	0.2308	13.0	13.0	83.0	0.1566	0.1566	0.0	0.0	1.0	0.0	0.0
0.422	6.0	12	2.0348	0.0059	877.7277	608.3945	97.0	299.0	0.3244	95.0	0.3177	21.0	21.0	64.0	0.3281	0.3281	30.0	32.0	73.0	0.4384	0.4110	30.0	30.0	78.0	0.3846	0.3846	14.0	14.0	83.0	0.1687	0.1687	0.0	0.0	1.0	0.0	0.0
0.1946	7.0	14	2.6277	0.0059	1133.4914	785.6764	99.0	299.0	0.3311	89.0	0.2977	12.0	13.0	64.0	0.2031	0.1875	37.0	43.0	73.0	0.5890	0.5068	30.0	33.0	78.0	0.4231	0.3846	10.0	10.0	83.0	0.1205	0.1205	0.0	0.0	1.0	0.0	0.0
0.0092	8.0	16	4.3676	0.0059	1884.0491	1305.9233	95.0	299.0	0.3177	94.0	0.3144	27.0	27.0	64.0	0.4219	0.4219	33.0	34.0	73.0	0.4658	0.4521	31.0	31.0	78.0	0.3974	0.3974	3.0	3.0	83.0	0.0361	0.0361	0.0	0.0	1.0	0.0	0.0
0.0001	9.0	18	5.7053	0.0059	2461.0922	1705.8991	95.0	299.0	0.3177	95.0	0.3177	18.0	18.0	64.0	0.2812	0.2812	38.0	38.0	73.0	0.5205	0.5205	37.0	37.0	78.0	0.4744	0.4744	2.0	2.0	83.0	0.0241	0.0241	0.0	0.0	1.0	0.0	0.0
0.0014	10.0	20	6.1083	0.0059	2634.9195	1826.3871	99.0	299.0	0.3311	99.0	0.3311	19.0	19.0	64.0	0.2969	0.2969	40.0	40.0	73.0	0.5479	0.5479	33.0	33.0	78.0	0.4231	0.4231	7.0	7.0	83.0	0.0843	0.0843	0.0	0.0	1.0	0.0	0.0
0.0	11.0	22	6.4705	0.0059	2791.1610	1934.6854	84.0	299.0	0.2809	84.0	0.2809	21.0	21.0	64.0	0.3281	0.3281	42.0	42.0	73.0	0.5753	0.5753	11.0	11.0	78.0	0.1410	0.1410	10.0	10.0	83.0	0.1205	0.1205	0.0	0.0	1.0	0.0	0.0
0.0	12.0	24	6.5793	0.0059	2838.0983	1967.2198	89.0	299.0	0.2977	89.0	0.2977	20.0	20.0	64.0	0.3125	0.3125	44.0	44.0	73.0	0.6027	0.6027	10.0	10.0	78.0	0.1282	0.1282	15.0	15.0	83.0	0.1807	0.1807	0.0	0.0	1.0	0.0	0.0
0.0	13.0	26	6.3791	0.0059	2751.7205	1907.3473	97.0	299.0	0.3244	96.0	0.3211	17.0	17.0	64.0	0.2656	0.2656	38.0	38.0	73.0	0.5205	0.5205	20.0	21.0	78.0	0.2692	0.2564	21.0	21.0	83.0	0.2530	0.2530	0.0	0.0	1.0	0.0	0.0
0.0001	14.0	28	6.4571	0.0059	2785.3751	1930.6749	108.0	299.0	0.3612	104.0	0.3478	20.0	20.0	64.0	0.3125	0.3125	32.0	34.0	73.0	0.4658	0.4384	24.0	25.0	78.0	0.3205	0.3077	28.0	29.0	83.0	0.3494	0.3373	0.0	0.0	1.0	0.0	0.0
0.0	15.0	30	6.5573	0.0059	2828.6005	1960.6364	106.0	299.0	0.3545	104.0	0.3478	20.0	20.0	64.0	0.3125	0.3125	29.0	30.0	73.0	0.4110	0.3973	25.0	26.0	78.0	0.3333	0.3205	30.0	30.0	83.0	0.3614	0.3614	0.0	0.0	1.0	0.0	0.0
0.0	16.0	32	6.6279	0.0059	2859.0436	1981.7380	104.0	299.0	0.3478	103.0	0.3445	20.0	20.0	64.0	0.3125	0.3125	27.0	27.0	73.0	0.3699	0.3699	25.0	26.0	78.0	0.3333	0.3205	31.0	31.0	83.0	0.3735	0.3735	0.0	0.0	1.0	0.0	0.0
0.0	17.0	34	6.6508	0.0059	2868.9203	1988.5840	104.0	299.0	0.3478	103.0	0.3445	20.0	20.0	64.0	0.3125	0.3125	26.0	26.0	73.0	0.3562	0.3562	26.0	27.0	78.0	0.3462	0.3333	31.0	31.0	83.0	0.3735	0.3735	0.0	0.0	1.0	0.0	0.0
0.0	18.0	36	6.6708	0.0059	2877.5668	1994.5773	107.0	299.0	0.3579	106.0	0.3545	21.0	21.0	64.0	0.3281	0.3281	26.0	26.0	73.0	0.3562	0.3562	26.0	27.0	78.0	0.3462	0.3333	33.0	33.0	83.0	0.3976	0.3976	0.0	0.0	1.0	0.0	0.0
0.0	19.0	38	6.7103	0.0059	2894.6142	2006.3937	104.0	299.0	0.3478	103.0	0.3445	21.0	21.0	64.0	0.3281	0.3281	26.0	26.0	73.0	0.3562	0.3562	25.0	26.0	78.0	0.3333	0.3205	31.0	31.0	83.0	0.3735	0.3735	0.0	0.0	1.0	0.0	0.0
0.0	20.0	40	6.7416	0.0059	2908.0775	2015.7257	103.0	299.0	0.3445	102.0	0.3411	21.0	21.0	64.0	0.3281	0.3281	23.0	23.0	73.0	0.3151	0.3151	26.0	27.0	78.0	0.3462	0.3333	32.0	32.0	83.0	0.3855	0.3855	0.0	0.0	1.0	0.0	0.0
0.0	21.0	42	6.7503	0.0059	2911.8640	2018.3503	101.0	299.0	0.3378	100.0	0.3344	21.0	21.0	64.0	0.3281	0.3281	24.0	24.0	73.0	0.3288	0.3288	24.0	25.0	78.0	0.3205	0.3077	31.0	31.0	83.0	0.3735	0.3735	0.0	0.0	1.0	0.0	0.0
0.0	22.0	44	6.7454	0.0059	2909.7279	2016.8697	102.0	299.0	0.3411	100.0	0.3344	21.0	21.0	64.0	0.3281	0.3281	24.0	24.0	73.0	0.3288	0.3288	25.0	26.0	78.0	0.3333	0.3205	30.0	31.0	83.0	0.3735	0.3614	0.0	0.0	1.0	0.0	0.0
0.0	23.0	46	6.7583	0.0059	2915.2938	2020.7277	100.0	299.0	0.3344	99.0	0.3311	21.0	21.0	64.0	0.3281	0.3281	23.0	23.0	73.0	0.3151	0.3151	24.0	25.0	78.0	0.3205	0.3077	31.0	31.0	83.0	0.3735	0.3735	0.0	0.0	1.0	0.0	0.0
0.0	24.0	48	6.7481	0.0059	2910.8792	2017.6677	100.0	299.0	0.3344	99.0	0.3311	21.0	21.0	64.0	0.3281	0.3281	23.0	23.0	73.0	0.3151	0.3151	25.0	26.0	78.0	0.3333	0.3205	30.0	30.0	83.0	0.3614	0.3614	0.0	0.0	1.0	0.0	0.0
0.0	25.0	50	6.7423	0.0059	2908.3829	2015.9374	101.0	299.0	0.3378	98.0	0.3278	21.0	21.0	64.0	0.3281	0.3281	23.0	23.0	73.0	0.3151	0.3151	24.0	26.0	78.0	0.3333	0.3077	30.0	31.0	83.0	0.3735	0.3614	0.0	0.0	1.0	0.0	0.0
0.0	26.0	52	6.7330	0.0059	2904.3800	2013.1628	102.0	299.0	0.3411	100.0	0.3344	21.0	21.0	64.0	0.3281	0.3281	23.0	23.0	73.0	0.3151	0.3151	25.0	26.0	78.0	0.3333	0.3205	31.0	32.0	83.0	0.3855	0.3735	0.0	0.0	1.0	0.0	0.0
0.0	27.0	54	6.7496	0.0059	2911.5393	2018.1252	103.0	299.0	0.3445	101.0	0.3378	21.0	21.0	64.0	0.3281	0.3281	23.0	23.0	73.0	0.3151	0.3151	26.0	27.0	78.0	0.3462	0.3333	31.0	32.0	83.0	0.3855	0.3735	0.0	0.0	1.0	0.0	0.0
0.0	28.0	56	6.7499	0.0059	2911.6943	2018.2327	102.0	299.0	0.3411	100.0	0.3344	21.0	21.0	64.0	0.3281	0.3281	23.0	23.0	73.0	0.3151	0.3151	26.0	27.0	78.0	0.3462	0.3333	30.0	31.0	83.0	0.3735	0.3614	0.0	0.0	1.0	0.0	0.0
0.0	29.0	58	6.7589	0.0059	2915.5583	2020.9110	102.0	299.0	0.3411	100.0	0.3344	21.0	21.0	64.0	0.3281	0.3281	23.0	23.0	73.0	0.3151	0.3151	26.0	27.0	78.0	0.3462	0.3333	30.0	31.0	83.0	0.3735	0.3614	0.0	0.0	1.0	0.0	0.0
0.0	30.0	60	6.7573	0.0059	2914.8715	2020.4349	101.0	299.0	0.3378	100.0	0.3344	21.0	21.0	64.0	0.3281	0.3281	23.0	23.0	73.0	0.3151	0.3151	25.0	26.0	78.0	0.3333	0.3205	31.0	31.0	83.0	0.3735	0.3735	0.0	0.0	1.0	0.0	0.0
0.0	31.0	62	6.7557	0.0059	2914.1780	2019.9543	102.0	299.0	0.3411	101.0	0.3378	21.0	21.0	64.0	0.3281	0.3281	24.0	24.0	73.0	0.3288	0.3288	25.0	26.0	78.0	0.3333	0.3205	31.0	31.0	83.0	0.3735	0.3735	0.0	0.0	1.0	0.0	0.0
0.0	32.0	64	6.7648	0.0059	2918.1175	2022.6849	101.0	299.0	0.3378	100.0	0.3344	21.0	21.0	64.0	0.3281	0.3281	23.0	23.0	73.0	0.3151	0.3151	26.0	27.0	78.0	0.3462	0.3333	30.0	30.0	83.0	0.3614	0.3614	0.0	0.0	1.0	0.0	0.0
0.0	33.0	66	6.7437	0.0059	2909.0133	2016.3744	103.0	299.0	0.3445	102.0	0.3411	21.0	21.0	64.0	0.3281	0.3281	24.0	24.0	73.0	0.3288	0.3288	26.0	27.0	78.0	0.3462	0.3333	31.0	31.0	83.0	0.3735	0.3735	0.0	0.0	1.0	0.0	0.0
0.0	34.0	68	6.7974	0.0059	2932.1864	2032.4368	101.0	299.0	0.3378	99.0	0.3311	22.0	22.0	64.0	0.3438	0.3438	24.0	24.0	73.0	0.3288	0.3288	24.0	25.0	78.0	0.3205	0.3077	29.0	30.0	83.0	0.3614	0.3494	0.0	0.0	1.0	0.0	0.0
0.0	35.0	70	6.7538	0.0059	2913.3714	2019.3951	102.0	299.0	0.3411	101.0	0.3378	21.0	21.0	64.0	0.3281	0.3281	23.0	23.0	73.0	0.3151	0.3151	25.0	26.0	78.0	0.3333	0.3205	32.0	32.0	83.0	0.3855	0.3855	0.0	0.0	1.0	0.0	0.0
0.0	36.0	72	6.7732	0.0059	2921.7128	2025.1770	102.0	299.0	0.3411	100.0	0.3344	21.0	21.0	64.0	0.3281	0.3281	23.0	23.0	73.0	0.3151	0.3151	25.0	26.0	78.0	0.3333	0.3205	31.0	32.0	83.0	0.3855	0.3735	0.0	0.0	1.0	0.0	0.0
0.0	37.0	74	6.7754	0.0059	2922.6892	2025.8538	101.0	299.0	0.3378	100.0	0.3344	21.0	21.0	64.0	0.3281	0.3281	23.0	23.0	73.0	0.3151	0.3151	24.0	25.0	78.0	0.3205	0.3077	32.0	32.0	83.0	0.3855	0.3855	0.0	0.0	1.0	0.0	0.0
0.0	38.0	76	6.7524	0.0059	2912.7424	2018.9592	103.0	299.0	0.3445	102.0	0.3411	22.0	22.0	64.0	0.3438	0.3438	23.0	23.0	73.0	0.3151	0.3151	26.0	27.0	78.0	0.3462	0.3333	31.0	31.0	83.0	0.3735	0.3735	0.0	0.0	1.0	0.0	0.0
0.0	39.0	78	6.7791	0.0059	2924.2525	2026.9374	103.0	299.0	0.3445	102.0	0.3411	22.0	22.0	64.0	0.3438	0.3438	23.0	23.0	73.0	0.3151	0.3151	25.0	26.0	78.0	0.3333	0.3205	32.0	32.0	83.0	0.3855	0.3855	0.0	0.0	1.0	0.0	0.0
0.0	40.0	80	6.7544	0.0059	2913.5955	2019.5505	103.0	299.0	0.3445	102.0	0.3411	22.0	22.0	64.0	0.3438	0.3438	23.0	23.0	73.0	0.3151	0.3151	26.0	27.0	78.0	0.3462	0.3333	31.0	31.0	83.0	0.3735	0.3735	0.0	0.0	1.0	0.0	0.0
0.0	41.0	82	6.7391	0.0059	2907.0003	2014.9791	103.0	299.0	0.3445	101.0	0.3378	21.0	21.0	64.0	0.3281	0.3281	24.0	24.0	73.0	0.3288	0.3288	25.0	26.0	78.0	0.3333	0.3205	31.0	32.0	83.0	0.3855	0.3735	0.0	0.0	1.0	0.0	0.0
0.0	42.0	84	6.7620	0.0059	2916.8757	2021.8242	101.0	299.0	0.3378	100.0	0.3344	21.0	21.0	64.0	0.3281	0.3281	23.0	23.0	73.0	0.3151	0.3151	25.0	26.0	78.0	0.3333	0.3205	31.0	31.0	83.0	0.3735	0.3735	0.0	0.0	1.0	0.0	0.0
0.0	43.0	86	6.7368	0.0059	2906.0266	2014.3041	102.0	299.0	0.3411	100.0	0.3344	21.0	21.0	64.0	0.3281	0.3281	24.0	24.0	73.0	0.3288	0.3288	25.0	26.0	78.0	0.3333	0.3205	30.0	31.0	83.0	0.3735	0.3614	0.0	0.0	1.0	0.0	0.0
0.0	44.0	88	6.7531	0.0059	2913.0412	2019.1663	101.0	299.0	0.3378	100.0	0.3344	21.0	21.0	64.0	0.3281	0.3281	23.0	23.0	73.0	0.3151	0.3151	26.0	27.0	78.0	0.3462	0.3333	30.0	30.0	83.0	0.3614	0.3614	0.0	0.0	1.0	0.0	0.0

Framework versions

• Transformers 4.51.3
• Pytorch 2.6.0+cu124
• Datasets 3.5.0
• Tokenizers 0.21.1