GSM8K-Binary_Llama-3.2-1B-bfe9d8o1

This model is a fine-tuned version of meta-llama/Llama-3.2-1B on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.3034
• Model Preparation Time: 0.0058
• Mdl: 4653.9298
• Accumulated Loss: 3225.8583
• Correct Preds: 1917.0
• Total Preds: 2475.0
• Accuracy: 0.7745
• Correct Gen Preds: 1919.0
• Gen Accuracy: 0.7754
• Correct Gen Preds 34192: 1046.0
• Correct Preds 34192: 1049.0
• Total Labels 34192: 1196.0
• Accuracy 34192: 0.8771
• Gen Accuracy 34192: 0.8746
• Correct Gen Preds 41568: 865.0
• Correct Preds 41568: 868.0
• Total Labels 41568: 1267.0
• Accuracy 41568: 0.6851
• Gen Accuracy 41568: 0.6827

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 32
• eval_batch_size: 64
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.01
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Model Preparation Time	Mdl	Accumulated Loss	Correct Preds	Total Preds	Accuracy	Correct Gen Preds	Gen Accuracy	Correct Gen Preds 34192	Correct Preds 34192	Total Labels 34192	Accuracy 34192	Gen Accuracy 34192	Correct Gen Preds 41568	Correct Preds 41568	Total Labels 41568	Accuracy 41568	Gen Accuracy 41568
No log	0	0	1.4656	0.0058	5233.1723	3627.3586	1196.0	2475.0	0.4832	1204.0	0.4865	1196.0	1196.0	1196.0	1.0	1.0	0.0	0.0	1267.0	0.0	0.0
0.3909	1.0	13	0.9147	0.0058	3265.9349	2263.7736	1196.0	2475.0	0.4832	8.0	0.0032	0.0	1196.0	1196.0	1.0	0.0	0.0	0.0	1267.0	0.0	0.0
2.5838	2.0	26	0.8758	0.0058	3127.0958	2167.5377	1517.0	2475.0	0.6129	139.0	0.0562	0.0	1180.0	1196.0	0.9866	0.0	131.0	337.0	1267.0	0.2660	0.1034
0.1806	3.0	39	0.6158	0.0058	2198.9720	1524.2113	1760.0	2475.0	0.7111	215.0	0.0869	0.0	642.0	1196.0	0.5368	0.0	207.0	1118.0	1267.0	0.8824	0.1634
0.0087	4.0	52	1.3144	0.0058	4693.3429	3253.1774	1519.0	2475.0	0.6137	1024.0	0.4137	16.0	301.0	1196.0	0.2517	0.0134	1001.0	1218.0	1267.0	0.9613	0.7901
0.0061	5.0	65	1.0468	0.0058	3737.9158	2590.9258	1678.0	2475.0	0.6780	603.0	0.2436	402.0	1158.0	1196.0	0.9682	0.3361	194.0	520.0	1267.0	0.4104	0.1531
0.0896	6.0	78	0.7674	0.0058	2740.0578	1899.2633	1834.0	2475.0	0.7410	1177.0	0.4756	471.0	828.0	1196.0	0.6923	0.3938	698.0	1006.0	1267.0	0.7940	0.5509
0.0001	7.0	91	0.7845	0.0058	2801.2835	1941.7018	1901.0	2475.0	0.7681	1802.0	0.7281	869.0	930.0	1196.0	0.7776	0.7266	926.0	971.0	1267.0	0.7664	0.7309
0.0	8.0	104	1.0404	0.0058	3714.9602	2575.0142	1882.0	2475.0	0.7604	1488.0	0.6012	846.0	1035.0	1196.0	0.8654	0.7074	634.0	847.0	1267.0	0.6685	0.5004
0.0001	9.0	117	1.1473	0.0058	4096.4963	2839.4749	1905.0	2475.0	0.7697	1908.0	0.7709	999.0	1003.0	1196.0	0.8386	0.8353	901.0	902.0	1267.0	0.7119	0.7111
0.0	10.0	130	1.2243	0.0058	4371.6047	3030.1655	1895.0	2475.0	0.7657	1896.0	0.7661	1033.0	1037.0	1196.0	0.8671	0.8637	855.0	858.0	1267.0	0.6772	0.6748
0.0001	11.0	143	1.2098	0.0058	4319.8084	2994.2630	1899.0	2475.0	0.7673	1899.0	0.7673	1028.0	1032.0	1196.0	0.8629	0.8595	863.0	867.0	1267.0	0.6843	0.6811
0.0002	12.0	156	1.2321	0.0058	4399.4227	3049.4475	1900.0	2475.0	0.7677	1901.0	0.7681	1038.0	1042.0	1196.0	0.8712	0.8679	855.0	858.0	1267.0	0.6772	0.6748
0.0	13.0	169	1.2505	0.0058	4465.1374	3094.9974	1895.0	2475.0	0.7657	1896.0	0.7661	1044.0	1048.0	1196.0	0.8763	0.8729	844.0	847.0	1267.0	0.6685	0.6661
0.0	14.0	182	1.2541	0.0058	4477.9552	3103.8821	1900.0	2475.0	0.7677	1900.0	0.7677	1045.0	1050.0	1196.0	0.8779	0.8737	847.0	850.0	1267.0	0.6709	0.6685
0.0	15.0	195	1.2553	0.0058	4482.1598	3106.7965	1900.0	2475.0	0.7677	1901.0	0.7681	1043.0	1047.0	1196.0	0.8754	0.8721	850.0	853.0	1267.0	0.6732	0.6709
0.0001	16.0	208	1.2586	0.0058	4493.9093	3114.9405	1903.0	2475.0	0.7689	1902.0	0.7685	1045.0	1050.0	1196.0	0.8779	0.8737	849.0	853.0	1267.0	0.6732	0.6701
0.0	17.0	221	1.2582	0.0058	4492.4502	3113.9292	1903.0	2475.0	0.7689	1904.0	0.7693	1043.0	1047.0	1196.0	0.8754	0.8721	853.0	856.0	1267.0	0.6756	0.6732
0.0	18.0	234	1.2603	0.0058	4500.1384	3119.2583	1902.0	2475.0	0.7685	1902.0	0.7685	1042.0	1046.0	1196.0	0.8746	0.8712	852.0	856.0	1267.0	0.6756	0.6725
0.0001	19.0	247	1.2631	0.0058	4510.1478	3126.1962	1905.0	2475.0	0.7697	1905.0	0.7697	1043.0	1048.0	1196.0	0.8763	0.8721	854.0	857.0	1267.0	0.6764	0.6740
0.0	20.0	260	1.2732	0.0058	4546.3417	3151.2839	1903.0	2475.0	0.7689	1902.0	0.7685	1046.0	1051.0	1196.0	0.8788	0.8746	848.0	852.0	1267.0	0.6725	0.6693
0.0	21.0	273	1.2775	0.0058	4561.5521	3161.8270	1903.0	2475.0	0.7689	1903.0	0.7689	1045.0	1049.0	1196.0	0.8771	0.8737	850.0	854.0	1267.0	0.6740	0.6709
0.0001	22.0	286	1.2805	0.0058	4572.4133	3169.3554	1902.0	2475.0	0.7685	1903.0	0.7689	1047.0	1051.0	1196.0	0.8788	0.8754	848.0	851.0	1267.0	0.6717	0.6693
0.0	23.0	299	1.2884	0.0058	4600.5452	3188.8550	1902.0	2475.0	0.7685	1902.0	0.7685	1047.0	1051.0	1196.0	0.8788	0.8754	847.0	851.0	1267.0	0.6717	0.6685
0.0001	24.0	312	1.2899	0.0058	4605.7894	3192.4899	1904.0	2475.0	0.7693	1905.0	0.7697	1049.0	1052.0	1196.0	0.8796	0.8771	848.0	852.0	1267.0	0.6725	0.6693
0.0	25.0	325	1.2924	0.0058	4614.6624	3198.6403	1903.0	2475.0	0.7689	1902.0	0.7685	1046.0	1051.0	1196.0	0.8788	0.8746	848.0	852.0	1267.0	0.6725	0.6693
0.0	26.0	338	1.2919	0.0058	4612.9212	3197.4333	1907.0	2475.0	0.7705	1906.0	0.7701	1047.0	1052.0	1196.0	0.8796	0.8754	851.0	855.0	1267.0	0.6748	0.6717
0.0001	27.0	351	1.2923	0.0058	4614.5171	3198.5395	1906.0	2475.0	0.7701	1908.0	0.7709	1046.0	1049.0	1196.0	0.8771	0.8746	854.0	857.0	1267.0	0.6764	0.6740
0.0	28.0	364	1.2936	0.0058	4619.1850	3201.7751	1906.0	2475.0	0.7701	1906.0	0.7701	1046.0	1050.0	1196.0	0.8779	0.8746	852.0	856.0	1267.0	0.6756	0.6725
0.0	29.0	377	1.2941	0.0058	4620.8184	3202.9072	1910.0	2475.0	0.7717	1910.0	0.7717	1046.0	1050.0	1196.0	0.8779	0.8746	856.0	860.0	1267.0	0.6788	0.6756
0.0	30.0	390	1.2948	0.0058	4623.4765	3204.7497	1910.0	2475.0	0.7717	1911.0	0.7721	1047.0	1050.0	1196.0	0.8779	0.8754	856.0	860.0	1267.0	0.6788	0.6756
0.0	31.0	403	1.2954	0.0058	4625.4138	3206.0926	1908.0	2475.0	0.7709	1908.0	0.7709	1047.0	1051.0	1196.0	0.8788	0.8754	853.0	857.0	1267.0	0.6764	0.6732
0.0	32.0	416	1.2973	0.0058	4632.3642	3210.9102	1907.0	2475.0	0.7705	1906.0	0.7701	1043.0	1048.0	1196.0	0.8763	0.8721	855.0	859.0	1267.0	0.6780	0.6748
0.0001	33.0	429	1.2967	0.0058	4630.0987	3209.3398	1910.0	2475.0	0.7717	1911.0	0.7721	1045.0	1049.0	1196.0	0.8771	0.8737	858.0	861.0	1267.0	0.6796	0.6772
0.0	34.0	442	1.2934	0.0058	4618.3014	3201.1626	1911.0	2475.0	0.7721	1912.0	0.7725	1043.0	1047.0	1196.0	0.8754	0.8721	861.0	864.0	1267.0	0.6819	0.6796
0.0	35.0	455	1.2942	0.0058	4621.1757	3203.1549	1912.0	2475.0	0.7725	1913.0	0.7729	1043.0	1047.0	1196.0	0.8754	0.8721	862.0	865.0	1267.0	0.6827	0.6803
0.0	36.0	468	1.2965	0.0058	4629.3912	3208.8495	1911.0	2475.0	0.7721	1912.0	0.7725	1042.0	1045.0	1196.0	0.8737	0.8712	862.0	866.0	1267.0	0.6835	0.6803
11.7618	37.0	481	1.2975	0.0058	4632.7811	3211.1991	1907.0	2475.0	0.7705	1908.0	0.7709	1041.0	1045.0	1196.0	0.8737	0.8704	859.0	862.0	1267.0	0.6803	0.6780
0.0	38.0	494	1.2986	0.0058	4636.7347	3213.9396	1914.0	2475.0	0.7733	1916.0	0.7741	1045.0	1048.0	1196.0	0.8763	0.8737	863.0	866.0	1267.0	0.6835	0.6811
0.0002	39.0	507	1.2973	0.0058	4632.3065	3210.8702	1912.0	2475.0	0.7725	1912.0	0.7725	1041.0	1045.0	1196.0	0.8737	0.8704	863.0	867.0	1267.0	0.6843	0.6811
0.0	40.0	520	1.2929	0.0058	4616.3620	3199.8183	1913.0	2475.0	0.7729	1913.0	0.7729	1040.0	1044.0	1196.0	0.8729	0.8696	865.0	869.0	1267.0	0.6859	0.6827
0.0001	41.0	533	1.2947	0.0058	4622.9787	3204.4047	1912.0	2475.0	0.7725	1913.0	0.7729	1040.0	1044.0	1196.0	0.8729	0.8696	865.0	868.0	1267.0	0.6851	0.6827
0.0	42.0	546	1.2924	0.0058	4614.8297	3198.7562	1911.0	2475.0	0.7721	1911.0	0.7721	1039.0	1043.0	1196.0	0.8721	0.8687	864.0	868.0	1267.0	0.6851	0.6819
0.0	43.0	559	1.2938	0.0058	4619.6900	3202.1251	1912.0	2475.0	0.7725	1914.0	0.7733	1040.0	1043.0	1196.0	0.8721	0.8696	866.0	869.0	1267.0	0.6859	0.6835
0.0	44.0	572	1.2952	0.0058	4624.5569	3205.4986	1913.0	2475.0	0.7729	1914.0	0.7733	1039.0	1043.0	1196.0	0.8721	0.8687	867.0	870.0	1267.0	0.6867	0.6843
0.0	45.0	585	1.2954	0.0058	4625.2850	3206.0033	1914.0	2475.0	0.7733	1916.0	0.7741	1040.0	1043.0	1196.0	0.8721	0.8696	868.0	871.0	1267.0	0.6875	0.6851
0.0	46.0	598	1.2966	0.0058	4629.6851	3209.0532	1913.0	2475.0	0.7729	1915.0	0.7737	1040.0	1043.0	1196.0	0.8721	0.8696	867.0	870.0	1267.0	0.6867	0.6843
0.0	47.0	611	1.2978	0.0058	4633.9231	3211.9907	1910.0	2475.0	0.7717	1910.0	0.7717	1040.0	1044.0	1196.0	0.8729	0.8696	862.0	866.0	1267.0	0.6835	0.6803
0.0	48.0	624	1.2984	0.0058	4636.1114	3213.5075	1913.0	2475.0	0.7729	1914.0	0.7733	1041.0	1044.0	1196.0	0.8729	0.8704	865.0	869.0	1267.0	0.6859	0.6827
0.0	49.0	637	1.2997	0.0058	4640.9520	3216.8628	1912.0	2475.0	0.7725	1912.0	0.7725	1039.0	1043.0	1196.0	0.8721	0.8687	865.0	869.0	1267.0	0.6859	0.6827
0.0	50.0	650	1.3008	0.0058	4644.5525	3219.3585	1911.0	2475.0	0.7721	1913.0	0.7729	1042.0	1045.0	1196.0	0.8737	0.8712	863.0	866.0	1267.0	0.6835	0.6811
0.0	51.0	663	1.3020	0.0058	4648.9058	3222.3759	1910.0	2475.0	0.7717	1910.0	0.7717	1038.0	1042.0	1196.0	0.8712	0.8679	864.0	868.0	1267.0	0.6851	0.6819
0.0001	52.0	676	1.3034	0.0058	4653.9298	3225.8583	1917.0	2475.0	0.7745	1919.0	0.7754	1046.0	1049.0	1196.0	0.8771	0.8746	865.0	868.0	1267.0	0.6851	0.6827
0.0	53.0	689	1.3087	0.0058	4672.7635	3238.9128	1910.0	2475.0	0.7717	1910.0	0.7717	1043.0	1046.0	1196.0	0.8746	0.8721	859.0	864.0	1267.0	0.6819	0.6780
0.0	54.0	702	1.3095	0.0058	4675.9439	3241.1173	1908.0	2475.0	0.7709	1908.0	0.7709	1044.0	1048.0	1196.0	0.8763	0.8729	856.0	860.0	1267.0	0.6788	0.6756
0.0	55.0	715	1.3086	0.0058	4672.5673	3238.7769	1910.0	2475.0	0.7717	1911.0	0.7721	1046.0	1049.0	1196.0	0.8771	0.8746	857.0	861.0	1267.0	0.6796	0.6764
0.0001	56.0	728	1.3105	0.0058	4679.2462	3243.4063	1913.0	2475.0	0.7729	1912.0	0.7725	1044.0	1048.0	1196.0	0.8763	0.8729	860.0	865.0	1267.0	0.6827	0.6788
0.0	57.0	741	1.3130	0.0058	4688.2581	3249.6529	1911.0	2475.0	0.7721	1910.0	0.7717	1044.0	1048.0	1196.0	0.8763	0.8729	858.0	863.0	1267.0	0.6811	0.6772
0.0	58.0	754	1.3128	0.0058	4687.7221	3249.2814	1912.0	2475.0	0.7725	1913.0	0.7729	1045.0	1048.0	1196.0	0.8763	0.8737	860.0	864.0	1267.0	0.6819	0.6788
0.0	59.0	767	1.3124	0.0058	4686.0279	3248.1070	1910.0	2475.0	0.7717	1911.0	0.7721	1045.0	1048.0	1196.0	0.8763	0.8737	858.0	862.0	1267.0	0.6803	0.6772
0.0	60.0	780	1.3120	0.0058	4684.6308	3247.1387	1914.0	2475.0	0.7733	1915.0	0.7737	1046.0	1049.0	1196.0	0.8771	0.8746	861.0	865.0	1267.0	0.6827	0.6796
0.0	61.0	793	1.3135	0.0058	4689.9652	3250.8362	1909.0	2475.0	0.7713	1910.0	0.7717	1045.0	1048.0	1196.0	0.8763	0.8737	857.0	861.0	1267.0	0.6796	0.6764
0.0	62.0	806	1.3124	0.0058	4686.0363	3248.1129	1908.0	2475.0	0.7709	1909.0	0.7713	1045.0	1048.0	1196.0	0.8763	0.8737	856.0	860.0	1267.0	0.6788	0.6756
0.0	63.0	819	1.3124	0.0058	4686.0005	3248.0880	1910.0	2475.0	0.7717	1911.0	0.7721	1044.0	1047.0	1196.0	0.8754	0.8729	859.0	863.0	1267.0	0.6811	0.6780
0.0	64.0	832	1.3121	0.0058	4685.2140	3247.5429	1912.0	2475.0	0.7725	1913.0	0.7729	1046.0	1049.0	1196.0	0.8771	0.8746	859.0	863.0	1267.0	0.6811	0.6780
0.0	65.0	845	1.3139	0.0058	4691.3697	3251.8097	1915.0	2475.0	0.7737	1915.0	0.7737	1046.0	1050.0	1196.0	0.8779	0.8746	861.0	865.0	1267.0	0.6827	0.6796
0.0	66.0	858	1.3140	0.0058	4691.6976	3252.0369	1910.0	2475.0	0.7717	1911.0	0.7721	1044.0	1047.0	1196.0	0.8754	0.8729	859.0	863.0	1267.0	0.6811	0.6780
11.7619	67.0	871	1.3121	0.0058	4684.9991	3247.3939	1914.0	2475.0	0.7733	1914.0	0.7733	1046.0	1050.0	1196.0	0.8779	0.8746	860.0	864.0	1267.0	0.6819	0.6788
0.0	68.0	884	1.3133	0.0058	4689.5215	3250.5286	1915.0	2475.0	0.7737	1915.0	0.7737	1047.0	1050.0	1196.0	0.8779	0.8754	860.0	865.0	1267.0	0.6827	0.6788
0.0	69.0	897	1.3134	0.0058	4689.6052	3250.5867	1913.0	2475.0	0.7729	1915.0	0.7737	1046.0	1049.0	1196.0	0.8771	0.8746	861.0	864.0	1267.0	0.6819	0.6796
0.0	70.0	910	1.3139	0.0058	4691.5900	3251.9624	1912.0	2475.0	0.7725	1910.0	0.7717	1046.0	1051.0	1196.0	0.8788	0.8746	856.0	861.0	1267.0	0.6796	0.6756
0.0	71.0	923	1.3146	0.0058	4693.9451	3253.5948	1912.0	2475.0	0.7725	1913.0	0.7729	1047.0	1050.0	1196.0	0.8779	0.8754	858.0	862.0	1267.0	0.6803	0.6772
0.0001	72.0	936	1.3148	0.0058	4694.7558	3254.1568	1912.0	2475.0	0.7725	1913.0	0.7729	1046.0	1049.0	1196.0	0.8771	0.8746	859.0	863.0	1267.0	0.6811	0.6780
0.0	73.0	949	1.3150	0.0058	4695.4219	3254.6185	1912.0	2475.0	0.7725	1911.0	0.7721	1044.0	1048.0	1196.0	0.8763	0.8729	859.0	864.0	1267.0	0.6819	0.6780
0.0001	74.0	962	1.3142	0.0058	4692.7482	3252.7652	1912.0	2475.0	0.7725	1912.0	0.7725	1047.0	1050.0	1196.0	0.8779	0.8754	857.0	862.0	1267.0	0.6803	0.6764
0.0	75.0	975	1.3150	0.0058	4695.4690	3254.6511	1910.0	2475.0	0.7717	1910.0	0.7717	1043.0	1047.0	1196.0	0.8754	0.8721	859.0	863.0	1267.0	0.6811	0.6780
0.0	76.0	988	1.3138	0.0058	4691.0539	3251.5908	1914.0	2475.0	0.7733	1915.0	0.7737	1046.0	1049.0	1196.0	0.8771	0.8746	861.0	865.0	1267.0	0.6827	0.6796
0.0	77.0	1001	1.3148	0.0058	4694.6546	3254.0866	1913.0	2475.0	0.7729	1913.0	0.7729	1046.0	1049.0	1196.0	0.8771	0.8746	859.0	864.0	1267.0	0.6819	0.6780
0.0	78.0	1014	1.3145	0.0058	4693.5080	3253.2919	1913.0	2475.0	0.7729	1914.0	0.7733	1046.0	1049.0	1196.0	0.8771	0.8746	860.0	864.0	1267.0	0.6819	0.6788
0.0	79.0	1027	1.3141	0.0058	4692.2144	3252.3952	1913.0	2475.0	0.7729	1911.0	0.7721	1045.0	1050.0	1196.0	0.8779	0.8737	858.0	863.0	1267.0	0.6811	0.6772
0.0	80.0	1040	1.3147	0.0058	4694.4856	3253.9695	1913.0	2475.0	0.7729	1914.0	0.7733	1047.0	1050.0	1196.0	0.8779	0.8754	859.0	863.0	1267.0	0.6811	0.6780
0.0	81.0	1053	1.3145	0.0058	4693.7574	3253.4647	1913.0	2475.0	0.7729	1912.0	0.7725	1047.0	1051.0	1196.0	0.8788	0.8754	857.0	862.0	1267.0	0.6803	0.6764
0.0	82.0	1066	1.3146	0.0058	4693.8938	3253.5592	1911.0	2475.0	0.7721	1912.0	0.7725	1046.0	1049.0	1196.0	0.8771	0.8746	858.0	862.0	1267.0	0.6803	0.6772

Framework versions

• Transformers 4.51.3
• Pytorch 2.6.0+cu124
• Datasets 3.5.0
• Tokenizers 0.21.1